Merge master into PAP-242 timeout diagnostics branch

Resolve the setup-supertest conflict by taking master's removeTestHome
helper (#15931), which fixes the same read-only runtime-context bundle
teardown bug that fea2612 addressed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
PaperclipandClaude Opus 5.5 committed 2026-10-12 18:40:51 +00:00
commit e9c0a16e8d
251 files changed
+25723 -26026

No files matched your search

@@ -0,0 +1,46 @@
# create-verification-skill attribution
This skill comes from Paperclip's portable pstack fork in
`paperclipai/agent-plugins`, commit
`410257663e0b344df656e604f1601aa1db95f7e4`, under `vendor/pstack/`.
The fork imported pstack 0.15.15 by Lauren Tan (poteto) from
[cursor/plugins](https://github.com/cursor/plugins/tree/ccb5507cec1546dc88135c1139c811e6c59115ba/pstack),
upstream revision `ccb5507cec1546dc88135c1139c811e6c59115ba`.
Paperclip adapts the workflow to the existing repo-root `feature-map/`.
It preserves the invocation policy in `agents/openai.yaml`, removes the
Cursor-specific `disable-model-invocation` frontmatter field, and omits the
separate runtime skill dependency because Paperclip supplies execution.
No upstream Cursor built-in skills are redistributed.
The imported resources include `SKILL.md`, `agents/openai.yaml`, and all three
files in `references/feature-map-example/`. Keep those examples with the skill
when copying or refreshing it.
## License
The original license is reproduced verbatim below.
```text
MIT License
Copyright (c) 2026 Lauren Tan
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```
@@ -0,0 +1,45 @@
---
name: create-verification-skill
description: "Generate a project-local verification skill that drives your app the way a user does — any language, framework, or platform. Use for /create-verification-skill, \"make a control skill for this repo\", or when a project has no scripted way to prove UI/CLI/service behavior."
---
# Create a verification skill
Every serious project needs a scripted way to drive the real app and prove behavior: launch it, exercise a feature the way a user would, and capture evidence. This skill generates that as a project-local skill (`.agents/skills/verify-<app>/`) tailored to the repo. You write the generator's output for the next agent, not for a human: it will be read cold, mid-task, by an agent that has never seen the app.
## 1. Interview the repo, not the user
Answer these from the codebase and only ask the user what you cannot observe:
- **Surface:** what does a user actually touch? A web UI, a CLI/TUI, a desktop app, an API, a mobile app, a library? A repo can have several; pick the primary one and note the rest.
- **Run:** how does the app start locally? Prefer the repo's own documented dev command (package scripts, Makefile, README quickstart). Note ports, env vars, seed data, auth.
- **Drive:** how can an agent interact with it programmatically? Existing harnesses first — Playwright/Cypress specs, expect scripts, PTY helpers, curl-able endpoints, a debug port. Only then pick a generic recipe: browser/CDP for web and Electron, a tmux/PTY harness for CLI/TUI, plain HTTP for services.
- **Observe:** what evidence can be captured? Screenshots, terminal transcripts, response bodies, logs, exit codes, DB state.
- **Isolate:** can two instances run side by side (ports, data dirs, profiles)? If not, say so in the generated skill: refusing to double-drive a shared instance beats corrupting the user's session.
If the checkout doesn't build or start as-is, fix that first (or report it precisely) before generating; a skill written against a broken base teaches wrong steps. When an irrelevant missing asset blocks startup (a static dir the API never serves, a sample config), the generated skill may create it, clearly marked as verification scaffolding, and remove it in cleanup.
## 2. Generate the skill
Write `.agents/skills/verify-<app>/SKILL.md` with YAML frontmatter (`name: verify-<app>` and a `description` that names the app, the surface, and when to reach for it — without frontmatter the skill never registers) and these sections, each grounded in what the interview actually found (no placeholders left):
- **Launch:** the exact command that starts the app for verification, and how to tell it's ready (a log line, a port answering, a prompt). Include teardown. For a short-lived CLI or TUI there is no server to keep alive: launch means build the binary (or install deps) once, then start each drive in its own isolated PTY or tmux session.
- **Doctor:** one read-only check that answers "is this instance worth driving?" — process up, right version/build, port owned by us, auth valid. An agent runs this first whenever anything looks off.
- **Drive:** the harness recipe with real selectors/commands from this repo, not examples. Prefer stable handles (ARIA labels, data attributes, prompt strings, route paths) over coordinates and tab order.
- **Evidence:** what to capture for a proof and where it goes. State the proof standards: exercise the real user path, not internal setters or test-only endpoints; capture the action and the resulting state, not just the final screen; verify side effects (files written, rows inserted, messages sent) alongside what's visible; mocks only where a production boundary already isolates the external system. When the safe path is a dry-run or test mode, verify what it actually skips by observing (files, network, git refs) rather than trusting its name: some dry-runs still touch the network or open a browser.
- **Cleanup:** how to tear down instances the run created. Never kill by process name; kill what you started. Cleanup removes instances and scratch state, never the evidence: proof artifacts survive the teardown, in a location the skill names.
- **Helpers:** any script the skill ships is executable and its invocation is shown in the skill body. A helper the reader has to reverse-engineer is not a helper.
## 3. Seed the feature map
For Paperclip, use the existing repo-root `feature-map/` directory and its `README.md`; have the generated verification skill reference that map instead of creating a second map inside the skill directory. Reuse existing feature files and use Poteto's format below for new or updated entries. For other projects, create `.agents/skills/verify-<app>/features/README.md` plus one file per user-facing feature you can identify (aim for the top 3-5 to start, from routes, commands, menus, or docs). Follow the shape in [`references/feature-map-example/`](references/feature-map-example/), with a README index and one file per feature. Each file answers, from the user's point of view: what the feature is, how to reach it, how to drive it with the harness, and what observable end state proves it works. The four H2s are `Sub-features`, `How to get to it (user POV)`, `Driving it with <harness>`, and `Gotchas`. The map is the repo's maintained verification source; a proof that drives one convenient entry point is incomplete when the map lists others.
## 4. Prove the generated skill before handing it over
Run its own instructions end to end once: launch, doctor, drive ONE mapped feature (one is enough; the map exists so later runs can cover the rest), capture evidence, clean up. After cleanup, confirm the evidence still exists at the named location — a cleanup that eats the proof fails this step. Fix what fails, and run the generated cleanup after every failed iteration too, so broken attempts don't strand processes and ports. A generated skill that was never executed is a draft, not a deliverable.
## 5. Offer the maintenance loop
Point the user at `/maintain-verification-skill` for keeping the map honest as the app changes. Suggest a cadence only if they ask.
Origin and license: [NOTICE.md](NOTICE.md).
@@ -0,0 +1,2 @@
policy:
allow_implicit_invocation: false
@@ -0,0 +1,47 @@
# Notes verification map
This directory is the maintained source for verifying the user-facing behavior of Notes. Read the index before driving the app, then use the matching feature file as the recipe.
## Baseline preconditions
- Launch Notes at `http://127.0.0.1:4173` with a disposable data directory.
- Set `NOTES_DATA_DIR=/tmp/notes-verify-$RUN_ID` so concurrent runs do not share state.
- Seed notes titled `Quarterly plan` and `Grocery list`.
- Put `control-notes` and the `notes` CLI on `PATH`.
- Run `control-notes doctor` and require the expected URL, data directory, and build revision.
- Never drive an instance that was not started by this verification run.
## Driving conventions
- Start every recipe from the baseline state unless its preconditions say otherwise.
- Prefer ARIA roles and accessible names over CSS selectors or DOM position.
- Treat every command as literal. Keep quoted names and flags unchanged.
- Run browser actions through `control-notes browser`.
- Run terminal actions through `control-notes cli -- <command>`.
- Restore seeded data after a mutation. Do not remove proof artifacts during cleanup.
## Proof and skip reporting
- Capture the user action and the resulting state, not only the final screen.
- UI proof includes an ARIA snapshot and a screenshot with the app identity visible.
- CLI proof includes the command, stdout, stderr, and exit code.
- Mutation proof includes a read-only second view of the stored value.
- Record the feature ID and entry point used with every artifact.
- Report an unreachable path with the attempted command and the unmet precondition.
- Do not report a skipped entry point as verified through a different path.
## Feature entry contract
Each feature file starts with an H1 title and one paragraph describing the user-visible behavior. It then uses exactly four H2 sections in this order.
1. `Sub-features` lists short IDs with one line for each behavior.
2. `How to get to it (user POV)` lists every user entry point.
3. `Driving it with <harness>` starts with `Preconditions:` and uses labeled bullets that pair each user action with an exact command and observable result.
4. `Gotchas` lists traps that can waste or invalidate a verification run.
Keep implementation details out of the map. Name only user paths, stable handles, required state, commands, and observable proof.
## Features
- [Create a note](./create-note.md) covers browser and CLI creation, cancellation, persistence, and cleanup.
- [Search notes](./search.md) covers toolbar, keyboard, and CLI search with matching, empty, and clear states.
@@ -0,0 +1,39 @@
# Create a note
Create note lets a user save a titled note from the browser or CLI, cancel an unfinished draft, and confirm the saved note from a second user-facing view.
## Sub-features
- `create-open` opens a blank editor from each browser entry point.
- `create-save` persists a title and body.
- `create-cancel` discards an unfinished browser draft.
- `create-cli` creates the same note shape from the terminal.
## How to get to it (user POV)
- Choose the `New note` button in the browser toolbar.
- Press `n` in the browser while focus is outside an editable field.
- Run `notes create --title <title> --body <body>` in a terminal.
## Driving it with control-notes
Preconditions:
- Notes is healthy at `http://127.0.0.1:4173`.
- No note is titled `Release checklist`.
- `control-notes doctor` reports the expected URL and disposable data directory.
- **Open editor.** Choose `New note`. Run `control-notes browser click --role button --name "New note"`. A form named `Note editor` appears with focus in the `Title` textbox.
- **Enter content.** Type the title and body. Run `control-notes browser fill --role textbox --name "Title" --value "Release checklist"` and `control-notes browser fill --role textbox --name "Body" --value "Tag and publish"`. The `Save note` button becomes enabled.
- **Save note.** Choose `Save note`. Run `control-notes browser click --role button --name "Save note"`. A status named `Note saved` appears and the heading reads `Release checklist`.
- **Confirm persistence.** Return to the note list and reopen the note. Run `control-notes browser click --role link --name "All notes"` and `control-notes browser click --role link --name "Release checklist"`. The editor shows both saved values.
- **Cancel draft.** Open a new note, enter `Discard me`, and choose `Cancel`. Run `control-notes browser click --role button --name "New note"`, `control-notes browser fill --role textbox --name "Title" --value "Discard me"`, and `control-notes browser click --role button --name "Cancel"`. The note list returns and has no `Discard me` link.
- **CLI entry.** Create a second note. Run `control-notes cli -- notes create --title "CLI note" --body "Created from terminal" --format json`. Exit code `0` and stdout contain the new note ID and title.
- **Proof.** Reopen both saved notes from `All notes`. Run `control-notes browser snapshot --aria --path artifacts/create-note/list.aria.txt` and `control-notes browser screenshot --path artifacts/create-note/list.png`. The artifacts show `Release checklist` and `CLI note`.
## Gotchas
- Pressing `n` while a textbox has focus types the character instead of opening a new editor.
- Titles are trimmed on save. Assert the rendered title, not the draft input value.
- A save status alone is insufficient proof. Reopen the note from the list.
- Remove `Release checklist` and `CLI note` during fixture cleanup, but retain their proof artifacts.
@@ -0,0 +1,45 @@
# Search notes
Search lets a user find notes by title or body text, inspect a matching note, and distinguish no matches from an unavailable search.
## Sub-features
- `search-open` opens search from each supported browser entry point.
- `search-match` returns title and body matches without changing note data.
- `search-open-result` opens a result in the note editor.
- `search-empty` shows a complete empty state for a query with no matches.
- `search-clear` removes the query and restores the recent-notes view.
- `search-cli` returns the same matching notes from the terminal.
## How to get to it (user POV)
- Choose the `Search` button in the browser toolbar.
- Press `/` in the browser while focus is outside an editable field.
- Run `notes search <query>` in a terminal.
## Driving it with control-notes
Preconditions:
- Notes is healthy at `http://127.0.0.1:4173`.
- The disposable data directory contains `Quarterly plan` with body text `Draft budget`.
- `control-notes doctor` reports the expected URL and data directory.
- **Toolbar entry.** Choose the `Search` button. Run `control-notes browser click --role button --name "Search"`. A dialog named `Search notes` appears with focus in its searchbox.
- **Keyboard entry.** Close the dialog, focus the page, and press `/`. Run `control-notes browser press --key "/"`. The same dialog appears and the page does not insert a slash.
- **Title match.** Type `quarterly`. Run `control-notes browser fill --role searchbox --name "Search notes" --value "quarterly"`. The `Search results` list contains `Quarterly plan` and does not contain `Grocery list`.
- **Body match.** Replace the query with `budget`. Run `control-notes browser fill --role searchbox --name "Search notes" --value "budget"`. The result `Quarterly plan` remains visible with a body-match excerpt.
- **Open result.** Choose `Quarterly plan`. Run `control-notes browser click --role link --name "Quarterly plan"`. The dialog closes and the editor heading reads `Quarterly plan`.
- **Empty state.** Reopen search and enter `volcano`. Run `control-notes browser fill --role searchbox --name "Search notes" --value "volcano"`. A status named `No matching notes` appears after search completes.
- **Clear query.** Choose `Clear search`. Run `control-notes browser click --role button --name "Clear search"`. The searchbox is empty and the `Recent notes` region replaces the result list.
- **CLI match.** Search from the terminal. Run `control-notes cli -- notes search "quarterly" --format json`. Exit code `0` and stdout contain one object whose title is `Quarterly plan`.
- **CLI miss.** Search for an absent value. Run `control-notes cli -- notes search "volcano" --format json`. Exit code `0` and stdout are `[]`.
- **Proof.** Capture the populated result state. Run `control-notes browser snapshot --aria --path artifacts/search/results.aria.txt` and `control-notes browser screenshot --path artifacts/search/results.png`. Both artifacts identify Notes, the query, and `Quarterly plan`.
## Gotchas
- Pressing `/` while the editor or searchbox has focus inserts text instead of opening search.
- Results update after a short debounce. Wait for the results list or empty status, not a fixed sleep.
- Archived notes are excluded unless the user enables `Include archived`.
- The CLI defaults to human-readable output. Use `--format json` for stable assertions.
- Opening a result changes browser state. Reopen search before proving another query.
@@ -0,0 +1,46 @@
# maintain-verification-skill attribution
This skill comes from Paperclip's portable pstack fork in
`paperclipai/agent-plugins`, commit
`410257663e0b344df656e604f1601aa1db95f7e4`, under `vendor/pstack/`.
The fork imported pstack 0.15.15 by Lauren Tan (poteto) from
[cursor/plugins](https://github.com/cursor/plugins/tree/ccb5507cec1546dc88135c1139c811e6c59115ba/pstack),
upstream revision `ccb5507cec1546dc88135c1139c811e6c59115ba`.
Paperclip adapts the workflow to the existing repo-root `feature-map/`.
It preserves the invocation policy in `agents/openai.yaml`, removes the
Cursor-specific `disable-model-invocation` frontmatter field, and omits the
separate runtime skill dependency because Paperclip supplies execution.
No upstream Cursor built-in skills are redistributed.
The imported resources include `SKILL.md` and `agents/openai.yaml`.
The workflow uses the examples shipped with
[`create-verification-skill`](../create-verification-skill/references/feature-map-example/).
## License
The original license is reproduced verbatim below.
```text
MIT License
Copyright (c) 2026 Lauren Tan
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```
@@ -0,0 +1,40 @@
---
name: maintain-verification-skill
description: "Periodic pass that keeps a project's verification skill and feature map honest: parallel source readers per feature, one live session driving every feature, at most one PR of proven corrections. Use for /maintain-verification-skill or \"audit the verify skill\"."
---
# Maintain a verification skill
A feature map rots the moment the app changes. This skill is the upkeep loop for a skill generated by `/create-verification-skill` (or any project-local verification skill with a feature map). The unit of rigor is the feature, not every sentence: cover every feature file from source and exercise every feature live, without terminalising every bullet.
## Outcomes
Pick one, and say which:
- **clean** — every feature got source and live coverage; nothing worth shipping. No branch, no PR.
- **changed** — one PR ships proven doc, harness, or map corrections.
- **blocked** — coverage could not finish or a proven fix could not ship safely. Say exactly what blocked it.
## Edit scope
Only edit the verification skill's own directory (its SKILL.md and any harness scripts it owns) and its feature-map directory. For Paperclip, the map is the existing repo-root `feature-map/`; maintain it in place instead of creating or moving it into a skill directory. Use [Poteto's feature-map format](../create-verification-skill/references/feature-map-example/) for new or updated entries. Never edit product code during a run: a behavior the map describes that the app no longer does is either doc drift (fix the map) or a product regression (report it, don't paper over it in docs).
## Pass
0. **Locate the target.** Find the verification skill to maintain: the project-local skill whose body has launch/drive sections and a feature map (usually `.agents/skills/verify-*/`). Several candidates → ask which one; none → stop and point at `/create-verification-skill` instead of inventing a target.
1. **Index hygiene.** Read the feature map README and glob its sibling files. Fix missing, extra, duplicate, or dead entries. Lightweight; no generated inventory.
2. **Source wave.** One read-only subagent per feature file, launched concurrently. Each explains "how does this user-facing feature work?" from source, flags likely doc drift with citations, and returns one concise live-verification recipe. Children never drive the app and never edit files. Return shape: feature summary / source entry points / likely drift or none / one recipe.
3. **Reconcile.** Every feature file has a returned summary. Merge overlapping recipes into as few app states as practical. Spot-check cited drift; don't re-prove clean claims. Sweep recent churn for user-facing surfaces missing from the map — require a concrete source path before calling one missing.
4. **Live pass.** Required even when source looks clean. The coordinator owns all driving; follow the verification skill's own launch model — one long-lived instance driven serially for servers and UIs, or a fresh isolated session per drive for short-lived CLIs (the skill's Launch section decides, not this one). Exercise every feature at least once, and hold three invariants the whole pass, whatever the failure: (1) never drive an instance you haven't health-checked since it last did something surprising — doctor before first drive, doctor on each fresh session where sessions are the unit, doctor again after any failed drive, and where doctor can't see the failure (a wedged UI state on a healthy process), reset to a known state or relaunch rather than hoping; (2) evidence captured so far survives every cleanup, checked at its named location, not assumed; (3) nothing a drive started outlives that drive's usefulness — failed-iteration residue is cleaned whether the session is stuck, exited, or shared (for a shared instance, clean the residue, not the instance). A doctor failure caused by skill drift is drift: fix it under edit scope and retry once — restart whatever the fix invalidated, nothing more — before calling the pass `blocked`. A feature that can't be reached is `verified-unreachable` only with the concrete prerequisite (auth, entitlement, OS, external state) and the route attempted; if the map omits that prerequisite, that's drift. Any harness fix from triage gets re-driven live before it ships. Final teardown happens after the last drive of the run — including those re-proofs — so nothing outlives the run (evidence stays, per the skill).
5. **Triage.** Wrong or missing user-POV description → doc drift, fix it. Working behavior the harness can't drive → harness gap, fix it; a harness fix follows the same helpers rule as generation (scripts executable, invocation documented in the skill body). App behavior that's actually broken → product gap; record it for the user, keep it out of this PR.
6. **Ship or stop.** For changed: one PR of proven corrections, re-read every changed file first. For clean or blocked: no PR, report the outcome and the coverage honestly.
Keep concise run notes (features covered, unreachable prerequisites, confirmed drift, outcome) in a scratch location; don't commit them.
Origin and license: [NOTICE.md](NOTICE.md).
@@ -0,0 +1,2 @@
policy:
allow_implicit_invocation: false
+166
View File
@@ -0,0 +1,166 @@
---
name: verify-paperclip
description: Launch an isolated Paperclip instance, drive its real web UI or CLI using the existing feature-map recipes, capture evidence, and clean up owned processes. Use to verify user-facing Paperclip behavior.
---
# Verify Paperclip
Use the repo-root [feature map](../../../feature-map/README.md) to select
verification recipes. Read the relevant feature file before driving it. Report
any recipe drift with evidence; map updates belong to
[maintain-verification-skill](../maintain-verification-skill/SKILL.md).
Read [development instructions](../../../doc/DEVELOPING.md) for current commands.
## Launch
Run from the repository root with Node 24.11+, Python 3, `lsof`, and dependencies
installed with `pnpm install --frozen-lockfile`. Browser driving uses the existing
`@playwright/test` dependency. If Chromium is unavailable, run
`node node_modules/@playwright/test/cli.js install chromium`, or select an installed
browser with `PAPERCLIP_PLAYWRIGHT_CHANNEL=chrome`.
Create separate, unique data and evidence directories. Keep the evidence outside
the data directory:
```sh
VERIFY_DATA_DIR=$(mktemp -d "${TMPDIR:-/tmp}/paperclip-verify-XXXXXX")
VERIFY_EVIDENCE_DIR=$(mktemp -d "${TMPDIR:-/tmp}/paperclip-proof-XXXXXX")
python3 .agents/skills/verify-paperclip/scripts/launch.py \
--data-dir "$VERIFY_DATA_DIR" --evidence-dir "$VERIFY_EVIDENCE_DIR"
```
Keep this command in a foreground terminal/session. The helper runs the supported
`onboard --yes --bind loopback --no-install-service --run` command, with isolated
embedded PostgreSQL, a free loopback port, no inherited instance routing or cwd
`.env`, and no automatic browser opening. It records `launch.json` in the evidence
directory. Source startup builds its required shared/plugin packages and migrates
the empty database; readiness can take longer on the first run.
This path starts an empty instance without model credentials. Create company and
agent state through the journey being checked. A UI check that stops before model
execution does not prove an agent run. For a preconfigured company/CEO and live
model work, use the documented [test-drive](../../../doc/DEVELOPING.md#one-command-isolated-manual-test-drive)
command with an authorized environment-backed credential and `--no-browser`.
Record its printed data directory, URL, runtime PID and commit before driving;
never use a fake credential to claim provider verification.
For Paperclip-managed reusable services, follow
[dev-workspace run/verify/fix](../paperclip-dev-workspace-run-verify-fix/SKILL.md)
instead of launching a second server. Its cloned-data/login requirements differ
from this empty local instance.
## Doctor
After startup, and again after any surprising behavior, run:
```sh
python3 .agents/skills/verify-paperclip/scripts/doctor.py \
--evidence-dir "$VERIFY_EVIDENCE_DIR"
```
This read-only check compares the recorded launcher, runtime PID, port owner,
process cwd, configuration, served Git commit, health/bootstrap readiness, root
HTML response, and company-list access. It writes `doctor.json` or exits nonzero.
Do not trust a root 200 or CLI doctor output alone. The CLI doctor checks whether
the port is available before startup; it is not proof of a running server's
ownership. Browser readiness still requires the expected visible page. Reset or
relaunch a wedged browser even when the process health passes.
## Drive
Use accessible roles, labels, and placeholders from the latest page state.
Existing Playwright specs in `tests/e2e/` supply recipes and selectors; their
configured servers and fixtures are separate from this live instance. Run a named
suite through its own configuration only when that is the evidence needed.
The executable onboarding smoke drives `onboarding.md` / `web-wizard`: open
`/onboarding` in the disposable self-hosted instance,
enter a unique organization name, choose Continue, reach the agent-name field,
reopen `/<company-prefix>/onboarding`, reload, and corroborate the saved company
through read-only API checks:
```sh
node .agents/skills/verify-paperclip/scripts/onboarding-smoke.mjs \
--evidence-dir "$VERIFY_EVIDENCE_DIR"
```
This covers company creation and setup resumption, with no mocked responses or
provider calls. Hiring, first task execution, and the alternate CLI entry remain
separate journeys. The script requires an empty instance and refuses a non-owned
or non-loopback target. For other features, reuse the owned URL and follow the
map's entry points. Company routes normally require the selected company's
`issuePrefix`, for example `/<PREFIX>/goals`; use current navigation rather than
guessing resource URLs. The unprefixed `/onboarding` route starts naming; use the
company-prefixed onboarding route to resume its agent setup. Check relevant
experimental gates and actor permissions.
For CLI checks, discover exact flags with
`node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts <command> --help`, then
select the disposable data directory/API/company explicitly. API reads can
corroborate UI writes; API-created state does not prove a creation UI.
## Evidence
Record feature file, sub-feature and entry-point ID, commit/dirty state, owned URL,
company/user, flags, runtime/harness, live versus simulated dependencies, command,
exit code, expected and observed behavior, and evidence links. Report each entry
as passed, failed, blocked with its prerequisite, or not run.
The smoke saves before/after/reload screenshots, visible-page text, a Playwright
trace, and `result.json` with company ID and API corroboration. On failure it
retains the trace, visible state, and failure result; rerun doctor before retrying.
Preserve launch output and relevant application logs without secrets. Reopen or
reload changed state and verify persistence/continuation, not just a success toast.
One passing sub-feature does not certify the entire feature family or other
surfaces. A full maintenance pass must exercise every mapped feature or report
incomplete coverage.
For Paperclip-assigned work, attach important proof through
[the artifact workflow](../../../doc/AGENT-ARTIFACTS.md). Keep runtime results in
the evidence directory or issue, not in the map; `coverage.json` describes
documentation coverage rather than live test health.
## Cleanup
Close the browser context you created (the smoke does this in `finally`). Stop the owned runtime with:
```sh
python3 .agents/skills/verify-paperclip/scripts/stop.py \
--evidence-dir "$VERIFY_EVIDENCE_DIR"
```
The helper verifies the runtime's port, cwd, and launch ancestry before sending
SIGTERM to that PID, then waits for runtime exit and checks that its HTTP
listener and embedded database have stopped. Signaling the runtime PID lets the server drain before stopping
PostgreSQL; a terminal group interrupt can reach the database first. Do this after
failed drives as well as successful runs. If startup fails before a runtime record
exists, stop only the foreground session you created and inspect its recorded data
directory for residue. Never kill by process name or stop another checkout's server.
Verify the recorded HTTP port has no listener and the owned runtime PID has
exited. Embedded PostgreSQL should stop with the server; inspect only the recorded
data directory if shutdown is incomplete. Retain the data directory for diagnosis
until no owned process uses it; remove only that exact temporary directory when
it is no longer useful. Never delete the evidence directory during teardown.
Confirm screenshots, trace, `result.json`, `launch.json`, and `doctor.json` still
exist after shutdown. A failed launch leaves its evidence intact.
## Helpers
All helper invocations are shown above. `launch.py`, `doctor.py`, and `stop.py` use Python's
standard library and the repo CLI; `onboarding-smoke.mjs` uses the repo's existing
Playwright dependency. They introduce no new package dependency or test framework.
## Helper safety check
Run the standard-library ownership test before changing launch/doctor/stop helpers:
```sh
python3 -m unittest discover -s .agents/skills/verify-paperclip/tests -v
```
It starts an owned fixture in another temporary checkout and confirms that both
doctor and stop reject its claimed port/PID without stopping the server. A second
fixture closes its listener before exiting and proves that cleanup waits for
runtime exit. These checks do
not qualify Paperclip startup, database readiness, or onboarding.
+60
View File
@@ -0,0 +1,60 @@
#!/usr/bin/env python3
"""Read-only identity and readiness check for an owned verification launch."""
import argparse
import json
import os
from pathlib import Path
import subprocess
import sys
import urllib.request
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--evidence-dir', required=True, type=Path)
args = parser.parse_args()
evidence = args.evidence_dir.resolve()
try:
launch = json.loads((evidence / 'launch.json').read_text())
repo = Path(launch['repo']).resolve()
assert repo == Path(__file__).resolve().parents[4], 'Launch belongs to another checkout.'
runtime = json.loads((Path(launch['dataDir']) / 'instances/default/runtime-info.json').read_text())
assert runtime['dashboardUrl'] == launch['url'], 'Runtime URL changed.'
assert runtime['port'] == launch['port'], 'Runtime port changed.'
assert runtime['host'] == '127.0.0.1', 'Expected trusted loopback.'
pid = runtime['pid']
os.kill(pid, 0)
listeners = subprocess.check_output(['lsof', '-nP', f'-iTCP:{launch["port"]}',
'-sTCP:LISTEN', '-t'], text=True).split()
assert set(listeners) == {str(pid)}, 'Port owner does not match runtime PID.'
if sys.platform == 'darwin':
rows = subprocess.check_output(['lsof', '-a', '-p', str(pid), '-d', 'cwd', '-Fn'], text=True).splitlines()
cwd = Path(next(row[1:] for row in rows if row.startswith('n'))).resolve()
else:
cwd = Path(f'/proc/{pid}/cwd').resolve(strict=True)
assert cwd == repo or repo in cwd.parents, 'Port owner cwd belongs to another checkout.'
ancestor = pid
seen = set()
while ancestor != launch['launcherPid']:
assert ancestor > 1 and ancestor not in seen, 'Runtime is not descended from this launch.'
seen.add(ancestor)
ancestor = int(subprocess.check_output(['ps', '-p', str(ancestor), '-o', 'ppid='], text=True).strip())
config = json.loads((Path(launch['dataDir']) / 'instances/default/config.json').read_text())
assert config['database']['mode'] == 'embedded-postgres', 'Expected isolated embedded database.'
assert Path(config['database']['embeddedPostgresDataDir']).resolve().is_relative_to(Path(launch['dataDir'])), 'Database path escaped isolation.'
with urllib.request.urlopen(launch['url'] + '/api/health', timeout=15) as response:
health = json.load(response)
assert health['status'] == 'ok' and health['bootstrapStatus'] == 'ready', 'Instance is not ready.'
assert health['deploymentMode'] == 'local_trusted' and health['authReady'], 'Unexpected auth mode.'
commit = subprocess.check_output(['git', '-C', str(repo), 'rev-parse', 'HEAD'], text=True).strip()
assert health['commit'] == launch['commit'] == commit, 'Served commit differs from checkout/launch.'
with urllib.request.urlopen(launch['url'], timeout=15) as response:
assert response.status == 200 and 'text/html' in response.headers.get('Content-Type', ''), 'Root UI is unavailable.'
with urllib.request.urlopen(launch['url'] + '/api/companies', timeout=15) as response:
companies = json.load(response)
assert isinstance(companies, list), 'Board company access failed.'
result = {'status': 'passed', 'pid': pid, 'cwd': str(cwd), 'url': launch['url'],
'health': health, 'companyIds': [company['id'] for company in companies]}
(evidence / 'doctor.json').write_text(json.dumps(result, indent=2) + '\n')
print(f'PASS: owned PID {pid}, {launch["url"]}, commit {commit}, ready board UI/API.')
except Exception as error:
(evidence / 'doctor-failure.json').write_text(json.dumps({'status': 'failed', 'error': str(error)}, indent=2) + '\n')
raise SystemExit(f'Doctor failed: {error}')
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env python3
"""Run the repository's credential-free isolated foreground onboarding path."""
import argparse
import json
import os
from pathlib import Path
import socket
import subprocess
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--data-dir', required=True, type=Path)
parser.add_argument('--evidence-dir', required=True, type=Path)
args = parser.parse_args()
repo = Path(__file__).resolve().parents[4]
data = args.data_dir.resolve()
evidence = args.evidence_dir.resolve()
if data == evidence or data in evidence.parents or evidence in data.parents:
parser.error('Data and evidence directories must be separate.')
data.mkdir(parents=True, exist_ok=True)
evidence.mkdir(parents=True, exist_ok=True)
if any(data.iterdir()) or (evidence / 'launch.json').exists():
parser.error('Use a fresh empty data directory and a new evidence directory.')
cli = repo / 'cli/node_modules/tsx/dist/cli.mjs'
if not cli.is_file():
parser.error('Install checkout dependencies first: pnpm install --frozen-lockfile')
for port in range(3100, 4100):
try:
with socket.socket() as probe:
probe.bind(('127.0.0.1', port))
break
except OSError:
continue
else:
raise RuntimeError('No available verification port between 3100 and 4099.')
excluded = {'DATABASE_URL', 'DATABASE_MIGRATION_URL', 'HOST', 'PORT', 'SERVE_UI',
'BETTER_AUTH_URL', 'BETTER_AUTH_BASE_URL', 'NODE_OPTIONS'}
env = {key: value for key, value in os.environ.items()
if not key.startswith('PAPERCLIP_') and key not in excluded}
env.update({'PAPERCLIP_HOME': str(data), 'PAPERCLIP_INSTANCE_ID': 'default',
'PAPERCLIP_DISABLE_CWD_ENV_FILE': 'true', 'PAPERCLIP_NO_BROWSER': 'true',
'PAPERCLIP_OPEN_ON_LISTEN': 'false', 'PAPERCLIP_UPDATE_CHECK': '0',
'PAPERCLIP_TELEMETRY_DISABLED': '1', 'PAPERCLIP_DEPLOYMENT_MODE': 'local_trusted',
'PAPERCLIP_DEPLOYMENT_EXPOSURE': 'private', 'PAPERCLIP_BIND': 'loopback',
'HOST': '127.0.0.1', 'PORT': str(port)})
record = {'dataDir': str(data), 'evidenceDir': str(evidence), 'repo': str(repo),
'launcherPid': os.getpid(), 'port': port, 'url': f'http://127.0.0.1:{port}',
'commit': subprocess.check_output(['git', '-C', str(repo), 'rev-parse', 'HEAD'], text=True).strip()}
(evidence / 'launch.json').write_text(json.dumps(record, indent=2) + '\n')
print(json.dumps(record), flush=True)
os.chdir(repo)
os.execvpe('node', ['node', str(cli), 'cli/src/index.ts', 'onboard', '--data-dir',
str(data), '--yes', '--bind', 'loopback', '--no-install-service', '--run'], env)
@@ -0,0 +1,79 @@
#!/usr/bin/env node
import assert from 'node:assert/strict';
import fs from 'node:fs/promises';
import path from 'node:path';
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { execFileSync } from 'node:child_process';
const repo = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../..');
const require = createRequire(path.join(repo, 'package.json'));
const { chromium, expect } = require('@playwright/test');
assert.equal(process.argv[2], '--evidence-dir', 'Usage: onboarding-smoke.mjs --evidence-dir <path>');
assert.ok(process.argv[3]);
const evidence = path.resolve(process.argv[3]);
execFileSync('python3', [path.join(repo, '.agents/skills/verify-paperclip/scripts/doctor.py'), '--evidence-dir', evidence], { stdio: 'inherit' });
const launch = JSON.parse(await fs.readFile(path.join(evidence, 'launch.json'), 'utf8'));
assert.equal(new URL(launch.url).hostname, '127.0.0.1');
const browser = await chromium.launch({ headless: true, ...(process.env.PAPERCLIP_PLAYWRIGHT_CHANNEL ? { channel: process.env.PAPERCLIP_PLAYWRIGHT_CHANNEL } : {}) });
const context = await browser.newContext({ viewport: { width: 1440, height: 1000 }, serviceWorkers: 'block' });
const page = await context.newPage();
const errors = [];
page.on('pageerror', error => errors.push(error.message));
const companyName = `Verification ${Date.now()}`;
const result = { feature: 'onboarding.md', subFeatures: ['company', 'resume'], entryPoint: 'web-wizard', commit: launch.commit, url: launch.url, actor: 'local_trusted board', providerCalls: false, mockedResponses: false, companyName, status: 'failed' };
await context.tracing.start({ screenshots: true, snapshots: true, sources: true });
async function capture(name) {
await page.screenshot({ path: path.join(evidence, `${name}.png`), fullPage: true });
await fs.writeFile(path.join(evidence, `${name}.txt`), await page.locator('body').innerText());
}
try {
const before = await page.request.get(`${launch.url}/api/companies`);
assert.equal(before.ok(), true);
assert.deepEqual(await before.json(), [], 'Smoke requires a fresh instance.');
await page.goto(`${launch.url}/onboarding`);
const start = page.getByRole('button', { name: /Start Onboarding|New Organization|Add Agent/ });
if (await start.count()) await start.first().click();
const create = page.getByRole('button', { name: /Build a new organization/ });
if (await create.count()) await create.first().click();
await expect(page.getByRole('heading', { name: 'What is the name of your organization?' })).toBeVisible({ timeout: 60_000 });
await page.getByPlaceholder('e.g. Northwind Labs').fill(companyName);
await capture('before-create');
await page.getByRole('button', { name: /^Continue/ }).click();
await expect(page.locator('#onboarding-agent-name')).toBeVisible({ timeout: 30_000 });
await capture('after-create');
const companiesResponse = await page.request.get(`${launch.url}/api/companies`);
assert.equal(companiesResponse.ok(), true);
const companies = await companiesResponse.json();
const company = companies.find(item => item.name === companyName);
assert.ok(company, 'UI-created company must be persisted.');
result.companyId = company.id;
const goalsResponse = await page.request.get(`${launch.url}/api/companies/${company.id}/goals`);
assert.equal(goalsResponse.ok(), true);
assert.deepEqual(await goalsResponse.json(), [], 'Naming must not invent a mission.');
await page.goto(`${launch.url}/${company.issuePrefix}/onboarding`);
await expect(page.locator('#onboarding-agent-name')).toBeVisible({ timeout: 30_000 });
await capture('after-reopen');
await page.reload();
await expect(page.locator('#onboarding-agent-name')).toBeVisible({ timeout: 30_000 });
await capture('after-reload');
const persistedResponse = await page.request.get(`${launch.url}/api/companies`);
assert.equal(persistedResponse.ok(), true);
const persisted = await persistedResponse.json();
assert.equal(persisted.length, 1, 'Resume must not create another company.');
assert.equal(persisted[0].id, company.id);
assert.deepEqual(errors, [], 'No browser exceptions.');
result.status = 'passed';
result.observed = 'Company persisted; onboarding resumed at the agent step after reload; no mission invented.';
result.notRun = ['agent hiring', 'first task/model execution', 'cli-setup entry point'];
} catch (error) {
result.error = error.message;
await capture('failure').catch(() => {});
process.exitCode = 1;
} finally {
result.browserErrors = errors;
await context.tracing.stop({ path: path.join(evidence, 'trace.zip') });
await browser.close();
await fs.writeFile(path.join(evidence, 'result.json'), `${JSON.stringify(result, null, 2)}\n`);
console.log(JSON.stringify(result));
}
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env python3
"""Stop only an owned verification runtime and check that its database exits."""
import argparse
import json
import os
from pathlib import Path
import signal
import subprocess
import sys
import time
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--evidence-dir', required=True, type=Path)
args = parser.parse_args()
evidence = args.evidence_dir.resolve()
launch = json.loads((evidence / 'launch.json').read_text())
repo = Path(launch['repo']).resolve()
assert repo == Path(__file__).resolve().parents[4], 'Launch belongs to another checkout.'
instance = Path(launch['dataDir']) / 'instances/default'
runtime_file = instance / 'runtime-info.json'
runtime = json.loads(runtime_file.read_text()) if runtime_file.exists() else None
pid = None
if runtime:
assert runtime['port'] == launch['port'] and runtime['dashboardUrl'] == launch['url']
pid = runtime['pid']
owners = subprocess.run(['lsof', '-nP', f'-iTCP:{launch["port"]}', '-sTCP:LISTEN', '-t'], capture_output=True, text=True).stdout.split()
assert set(owners) == {str(pid)}, 'Refusing to signal a different or unverified port owner.'
if sys.platform == 'darwin':
rows = subprocess.check_output(['lsof', '-a', '-p', str(pid), '-d', 'cwd', '-Fn'], text=True).splitlines()
cwd = Path(next(row[1:] for row in rows if row.startswith('n'))).resolve()
else:
cwd = Path(f'/proc/{pid}/cwd').resolve(strict=True)
assert cwd == repo or repo in cwd.parents, 'Refusing to signal another checkout.'
ancestor = pid
seen = set()
while ancestor != launch['launcherPid']:
assert ancestor > 1 and ancestor not in seen, 'Runtime is not descended from this launch.'
seen.add(ancestor)
ancestor = int(subprocess.check_output(['ps', '-p', str(ancestor), '-o', 'ppid='], text=True).strip())
os.kill(pid, signal.SIGTERM)
def runtime_exited():
if pid is None:
return True
try:
os.kill(pid, 0)
except ProcessLookupError:
return True
# An exited child can remain a zombie until its launcher reaps it.
state = subprocess.run(['ps', '-p', str(pid), '-o', 'stat='],
capture_output=True, text=True)
return state.returncode == 0 and state.stdout.strip().startswith('Z')
deadline = time.monotonic() + 45
while time.monotonic() < deadline:
owners = subprocess.run(['lsof', '-nP', f'-iTCP:{launch["port"]}', '-sTCP:LISTEN', '-t'], capture_output=True, text=True).stdout.split()
if not owners and not (instance / 'db/postmaster.pid').exists() and runtime_exited():
result = {'status': 'passed', 'port': launch['port'], 'serverStopped': True,
'databaseStopped': True, 'runtimeExited': True, 'evidenceRetained': sorted(p.name for p in evidence.iterdir())}
(evidence / 'cleanup.json').write_text(json.dumps(result, indent=2) + '\n')
print(f'PASS: owned runtime/database stopped; evidence retained at {evidence}.')
break
time.sleep(0.5)
else:
raise SystemExit('Shutdown incomplete; retain data and evidence and inspect only this owned instance.')
@@ -0,0 +1,124 @@
"""Check that verification helpers cannot take ownership of another checkout."""
import json
import os
import time
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
import urllib.request
class OwnershipSafety(unittest.TestCase):
def test_foreign_checkout_server_remains_alive(self):
repo = Path(__file__).resolve().parents[4]
helpers = repo / '.agents/skills/verify-paperclip/scripts'
with tempfile.TemporaryDirectory(prefix='paperclip-ownership-test-') as temp:
root = Path(temp)
foreign = root / 'another-checkout'
foreign.mkdir()
server = subprocess.Popen(
[sys.executable, '-u', '-c',
"from http.server import HTTPServer, BaseHTTPRequestHandler\n"
"class Handler(BaseHTTPRequestHandler):\n"
" def do_GET(self):\n"
" self.send_response(200); self.end_headers(); self.wfile.write(b'foreign fixture alive')\n"
" def log_message(self, *args): pass\n"
"server = HTTPServer(('127.0.0.1', 0), Handler)\n"
"print(server.server_port, flush=True)\n"
"server.serve_forever()\n"],
cwd=foreign, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
text=True,
)
try:
port = int(server.stdout.readline().strip())
url = f'http://127.0.0.1:{port}'
data = root / 'verification-data'
instance = data / 'instances/default'
instance.mkdir(parents=True)
evidence = root / 'evidence'
evidence.mkdir()
# The claimed PID really owns this port, but its cwd is foreign.
(evidence / 'launch.json').write_text(json.dumps({
'repo': str(repo), 'dataDir': str(data), 'port': port,
'url': url, 'launcherPid': server.pid,
}))
(instance / 'runtime-info.json').write_text(json.dumps({
'pid': server.pid, 'port': port, 'dashboardUrl': url,
'host': '127.0.0.1',
}))
for helper in ['doctor.py', 'stop.py']:
with self.subTest(helper=helper):
result = subprocess.run(
[sys.executable, str(helpers / helper),
'--evidence-dir', str(evidence)],
capture_output=True, text=True, timeout=10,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn('checkout', result.stdout + result.stderr)
self.assertIsNone(server.poll(), 'Helper killed foreign server')
with urllib.request.urlopen(url, timeout=5) as response:
self.assertEqual(response.read(), b'foreign fixture alive')
self.assertFalse((evidence / 'cleanup.json').exists())
finally:
server.terminate()
try:
server.communicate(timeout=5)
except subprocess.TimeoutExpired:
server.kill()
server.communicate()
def test_cleanup_waits_for_exit_after_listener_closes(self):
repo = Path(__file__).resolve().parents[4]
with tempfile.TemporaryDirectory(prefix='paperclip-delayed-exit-test-') as temp:
root = Path(temp)
instance = root / 'data/instances/default'
instance.mkdir(parents=True)
evidence = root / 'evidence'
evidence.mkdir()
marker = root / 'listener-closed'
server = subprocess.Popen(
[sys.executable, '-u', '-c',
"import signal, socket, sys, time\n"
"from pathlib import Path\n"
"listener = socket.socket(); listener.bind(('127.0.0.1', 0)); listener.listen()\n"
"def stop(signum, frame):\n"
" listener.close(); Path(sys.argv[1]).write_text('closed')\n"
" time.sleep(2); sys.exit(0)\n"
"signal.signal(signal.SIGTERM, stop)\n"
"print(listener.getsockname()[1], flush=True)\n"
"signal.pause()\n", str(marker)],
cwd=repo, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
)
try:
port = int(server.stdout.readline().strip())
url = f'http://127.0.0.1:{port}'
(evidence / 'launch.json').write_text(json.dumps({
'repo': str(repo), 'dataDir': str(root / 'data'),
'port': port, 'url': url, 'launcherPid': os.getpid(),
}))
(instance / 'runtime-info.json').write_text(json.dumps({
'pid': server.pid, 'port': port, 'dashboardUrl': url,
}))
started = time.monotonic()
result = subprocess.run(
[sys.executable, str(repo / '.agents/skills/verify-paperclip/scripts/stop.py'),
'--evidence-dir', str(evidence)],
capture_output=True, text=True, timeout=12,
)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
self.assertTrue(marker.exists(), 'Fixture did not close its listener')
self.assertGreaterEqual(time.monotonic() - started, 2)
self.assertEqual(server.poll(), 0, 'Cleanup returned before runtime exit')
cleanup = json.loads((evidence / 'cleanup.json').read_text())
self.assertTrue(cleanup['runtimeExited'])
finally:
if server.poll() is None:
server.kill()
server.communicate(timeout=5)
if __name__ == '__main__':
unittest.main()
+153
View File
@@ -475,6 +475,42 @@ jobs:
# Restore only. Never let a pull request evict master's entry.
save-if: false
# The Runner's own crate is the one thing the entry above leaves out:
# rust-cache drops workspace crates when it saves, and a fresh checkout
# would invalidate cargo's mtime fingerprints for them anyway. So this
# job compiled paperclip-runner-core in release mode on every run,
# 28-49s (runs 38169436532 to 38183149920, 2026-10-11), while 221 of
# the 239 master commits of the preceding week left its inputs alone.
# The master Rust lane in release-verify.yml saves that crate's release
# artifacts (about 70 MB) under a key made from the compiler, the
# checkout root and a hash of every input the build reads. A pull
# request whose merge tree carries byte-identical inputs restores them
# and marks them newer than the checkout, so cargo keeps them; any
# other pull request misses and compiles exactly as before. Exact key
# only: a prefix match would hand cargo another revision's output. The
# path list and the key live in scripts/runner-release-crate-cache.mjs
# and must match the writer's; a guard test holds them together.
- name: Compute the Runner release crate cache key
id: runner_release_crate_key
run: echo "scope=$(node scripts/runner-release-crate-cache.mjs scope)" >> "$GITHUB_OUTPUT"
- name: Restore the Runner release crate (read only)
id: runner_release_crate
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
packages/paperclip-runner/runner/target/release/.fingerprint/paperclip-runner-core-*
packages/paperclip-runner/runner/target/release/deps/libpaperclip_runner_core-*
packages/paperclip-runner/runner/target/release/deps/paperclip_runner_core-*.d
packages/paperclip-runner/runner/target/release/deps/paperclip_runnerd-*
packages/paperclip-runner/runner/target/release/paperclip-runnerd
packages/paperclip-runner/runner/target/release/paperclip-runnerd.d
key: runner-release-crate-v1-${{ steps.runner_release_crate_key.outputs.scope }}-${{ hashFiles('packages/paperclip-runner/rust-toolchain.toml', 'packages/paperclip-runner/runner/Cargo.toml', 'packages/paperclip-runner/runner/Cargo.lock', 'packages/paperclip-runner/runner/crates/**', 'packages/paperclip-runner/protocol/**', 'packages/paperclip-runner/test-fixtures/**', 'packages/paperclip-runner/test/fixtures/**', 'tests/runner-e2e/fixtures/**') }}
- name: Mark the restored Runner release crate current
if: ${{ steps.runner_release_crate.outputs.cache-hit == 'true' }}
run: node scripts/runner-release-crate-cache.mjs mark-current
- name: Typecheck workspaces whose build scripts skip TypeScript
run: pnpm run typecheck:build-gaps
@@ -749,10 +785,12 @@ jobs:
command: check:static
- lane_label: rust
command: check:runner
release_crate: true
- lane_label: vitest 1/2
command: test:typescript:vitest --shard=1/2
- lane_label: vitest 2/2
command: test:typescript:vitest --shard=2/2
release_crate: true
steps:
- name: Checkout repository
@@ -878,6 +916,49 @@ jobs:
# Restore only. Never let a pull request evict master's entry.
save-if: false
# The Runner's own crate is the one thing the entry above leaves out:
# rust-cache drops workspace crates when it saves, and a fresh checkout
# would invalidate cargo's mtime fingerprints for them anyway. So this
# job compiled paperclip-runner-core in release mode on every run,
# 28-49s (runs 38169436532 to 38183149920, 2026-10-11), while 221 of
# the 239 master commits of the preceding week left its inputs alone.
# The master Rust lane in release-verify.yml saves that crate's release
# artifacts (about 70 MB) under a key made from the compiler, the
# checkout root and a hash of every input the build reads. A pull
# request whose merge tree carries byte-identical inputs restores them
# and marks them newer than the checkout, so cargo keeps them; any
# other pull request misses and compiles exactly as before. Exact key
# only: a prefix match would hand cargo another revision's output. The
# path list and the key live in scripts/runner-release-crate-cache.mjs
# and must match the writer's; a guard test holds them together.
#
# Only the lanes that build the release crate restore it: the Rust
# lane's release tests link the lib, and the final vitest shard hosts
# the server's native-runner suite, which builds the binary in its
# beforeAll. The static and first vitest lanes run no cargo at all.
- name: Compute the Runner release crate cache key
id: runner_release_crate_key
if: ${{ matrix.release_crate == true }}
run: echo "scope=$(node scripts/runner-release-crate-cache.mjs scope)" >> "$GITHUB_OUTPUT"
- name: Restore the Runner release crate (read only)
id: runner_release_crate
if: ${{ matrix.release_crate == true }}
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
packages/paperclip-runner/runner/target/release/.fingerprint/paperclip-runner-core-*
packages/paperclip-runner/runner/target/release/deps/libpaperclip_runner_core-*
packages/paperclip-runner/runner/target/release/deps/paperclip_runner_core-*.d
packages/paperclip-runner/runner/target/release/deps/paperclip_runnerd-*
packages/paperclip-runner/runner/target/release/paperclip-runnerd
packages/paperclip-runner/runner/target/release/paperclip-runnerd.d
key: runner-release-crate-v1-${{ steps.runner_release_crate_key.outputs.scope }}-${{ hashFiles('packages/paperclip-runner/rust-toolchain.toml', 'packages/paperclip-runner/runner/Cargo.toml', 'packages/paperclip-runner/runner/Cargo.lock', 'packages/paperclip-runner/runner/crates/**', 'packages/paperclip-runner/protocol/**', 'packages/paperclip-runner/test-fixtures/**', 'packages/paperclip-runner/test/fixtures/**', 'tests/runner-e2e/fixtures/**') }}
- name: Mark the restored Runner release crate current
if: ${{ steps.runner_release_crate.outputs.cache-hit == 'true' }}
run: node scripts/runner-release-crate-cache.mjs mark-current
- name: Verify Paperclip Runner
# pnpm appends trailing args to the end of the script's shell chain,
# so the vitest lanes' --shard lands on `vitest run`. Do not add a
@@ -1013,6 +1094,42 @@ jobs:
# Restore only. Never let a pull request evict master's entry.
save-if: false
# The Runner's own crate is the one thing the entry above leaves out:
# rust-cache drops workspace crates when it saves, and a fresh checkout
# would invalidate cargo's mtime fingerprints for them anyway. So this
# job compiled paperclip-runner-core in release mode on every run,
# 28-49s (runs 38169436532 to 38183149920, 2026-10-11), while 221 of
# the 239 master commits of the preceding week left its inputs alone.
# The master Rust lane in release-verify.yml saves that crate's release
# artifacts (about 70 MB) under a key made from the compiler, the
# checkout root and a hash of every input the build reads. A pull
# request whose merge tree carries byte-identical inputs restores them
# and marks them newer than the checkout, so cargo keeps them; any
# other pull request misses and compiles exactly as before. Exact key
# only: a prefix match would hand cargo another revision's output. The
# path list and the key live in scripts/runner-release-crate-cache.mjs
# and must match the writer's; a guard test holds them together.
- name: Compute the Runner release crate cache key
id: runner_release_crate_key
run: echo "scope=$(node scripts/runner-release-crate-cache.mjs scope)" >> "$GITHUB_OUTPUT"
- name: Restore the Runner release crate (read only)
id: runner_release_crate
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
packages/paperclip-runner/runner/target/release/.fingerprint/paperclip-runner-core-*
packages/paperclip-runner/runner/target/release/deps/libpaperclip_runner_core-*
packages/paperclip-runner/runner/target/release/deps/paperclip_runner_core-*.d
packages/paperclip-runner/runner/target/release/deps/paperclip_runnerd-*
packages/paperclip-runner/runner/target/release/paperclip-runnerd
packages/paperclip-runner/runner/target/release/paperclip-runnerd.d
key: runner-release-crate-v1-${{ steps.runner_release_crate_key.outputs.scope }}-${{ hashFiles('packages/paperclip-runner/rust-toolchain.toml', 'packages/paperclip-runner/runner/Cargo.toml', 'packages/paperclip-runner/runner/Cargo.lock', 'packages/paperclip-runner/runner/crates/**', 'packages/paperclip-runner/protocol/**', 'packages/paperclip-runner/test-fixtures/**', 'packages/paperclip-runner/test/fixtures/**', 'tests/runner-e2e/fixtures/**') }}
- name: Mark the restored Runner release crate current
if: ${{ steps.runner_release_crate.outputs.cache-hit == 'true' }}
run: node scripts/runner-release-crate-cache.mjs mark-current
- name: Build Runner Evalbook viewer
run: pnpm --filter @paperclipai/paperclip-runner build:issue-thread
@@ -1259,6 +1376,42 @@ jobs:
# Restore only. Never let a pull request evict master's entry.
save-if: false
# The Runner's own crate is the one thing the entry above leaves out:
# rust-cache drops workspace crates when it saves, and a fresh checkout
# would invalidate cargo's mtime fingerprints for them anyway. So this
# job compiled paperclip-runner-core in release mode on every run,
# 28-49s (runs 38169436532 to 38183149920, 2026-10-11), while 221 of
# the 239 master commits of the preceding week left its inputs alone.
# The master Rust lane in release-verify.yml saves that crate's release
# artifacts (about 70 MB) under a key made from the compiler, the
# checkout root and a hash of every input the build reads. A pull
# request whose merge tree carries byte-identical inputs restores them
# and marks them newer than the checkout, so cargo keeps them; any
# other pull request misses and compiles exactly as before. Exact key
# only: a prefix match would hand cargo another revision's output. The
# path list and the key live in scripts/runner-release-crate-cache.mjs
# and must match the writer's; a guard test holds them together.
- name: Compute the Runner release crate cache key
id: runner_release_crate_key
run: echo "scope=$(node scripts/runner-release-crate-cache.mjs scope)" >> "$GITHUB_OUTPUT"
- name: Restore the Runner release crate (read only)
id: runner_release_crate
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
packages/paperclip-runner/runner/target/release/.fingerprint/paperclip-runner-core-*
packages/paperclip-runner/runner/target/release/deps/libpaperclip_runner_core-*
packages/paperclip-runner/runner/target/release/deps/paperclip_runner_core-*.d
packages/paperclip-runner/runner/target/release/deps/paperclip_runnerd-*
packages/paperclip-runner/runner/target/release/paperclip-runnerd
packages/paperclip-runner/runner/target/release/paperclip-runnerd.d
key: runner-release-crate-v1-${{ steps.runner_release_crate_key.outputs.scope }}-${{ hashFiles('packages/paperclip-runner/rust-toolchain.toml', 'packages/paperclip-runner/runner/Cargo.toml', 'packages/paperclip-runner/runner/Cargo.lock', 'packages/paperclip-runner/runner/crates/**', 'packages/paperclip-runner/protocol/**', 'packages/paperclip-runner/test-fixtures/**', 'packages/paperclip-runner/test/fixtures/**', 'tests/runner-e2e/fixtures/**') }}
- name: Mark the restored Runner release crate current
if: ${{ steps.runner_release_crate.outputs.cache-hit == 'true' }}
run: node scripts/runner-release-crate-cache.mjs mark-current
# `release.sh` always executes its Step 2/7 workspace build, even when
# `--skip-verify` bypasses the initial verification gate. release.sh
# also requires a clean working tree, and the install step may have
+55
View File
@@ -414,6 +414,61 @@ jobs:
if: ${{ matrix.lane == 'rust' && github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
run: pnpm --filter @paperclipai/paperclip-runner build:rust
# The dependency cache above deliberately leaves the Runner's own crate
# out, so every pull request build compiled paperclip-runner-core in
# release mode (28-49s) although most pushes leave its inputs alone.
# This master push saves that crate's release artifacts for the readers
# in pr-trusted.yml ("Restore the Runner release crate") under a key
# made from the compiler, the checkout root and a hash of every input
# the build reads, so a reader only ever restores output of inputs
# byte-identical to its own. Same writer gate as the dependency cache:
# the Rust lane of an own-repository master push. The entry is looked
# up first; a push that left the inputs alone already has it. On a
# miss the crate is cleaned and built with the exact command the
# readers run, so the artifacts carry no test-profile variants. The
# path list and the key live in scripts/runner-release-crate-cache.mjs
# and must match the readers'; a guard test holds them together.
- name: Compute the Runner release crate cache key
id: runner_release_crate_key
if: ${{ matrix.lane == 'rust' && github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
run: echo "scope=$(node scripts/runner-release-crate-cache.mjs scope)" >> "$GITHUB_OUTPUT"
- name: Check for this push's Runner release crate
id: runner_release_crate
if: ${{ matrix.lane == 'rust' && github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha }}
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
lookup-only: true
path: |
packages/paperclip-runner/runner/target/release/.fingerprint/paperclip-runner-core-*
packages/paperclip-runner/runner/target/release/deps/libpaperclip_runner_core-*
packages/paperclip-runner/runner/target/release/deps/paperclip_runner_core-*.d
packages/paperclip-runner/runner/target/release/deps/paperclip_runnerd-*
packages/paperclip-runner/runner/target/release/paperclip-runnerd
packages/paperclip-runner/runner/target/release/paperclip-runnerd.d
key: runner-release-crate-v1-${{ steps.runner_release_crate_key.outputs.scope }}-${{ hashFiles('packages/paperclip-runner/rust-toolchain.toml', 'packages/paperclip-runner/runner/Cargo.toml', 'packages/paperclip-runner/runner/Cargo.lock', 'packages/paperclip-runner/runner/crates/**', 'packages/paperclip-runner/protocol/**', 'packages/paperclip-runner/test-fixtures/**', 'packages/paperclip-runner/test/fixtures/**', 'tests/runner-e2e/fixtures/**') }}
- name: Build the Runner release crate for its cache
if: ${{ matrix.lane == 'rust' && github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha && steps.runner_release_crate.outputs.cache-hit != 'true' }}
working-directory: packages/paperclip-runner
run: |
set -euo pipefail
cargo clean --release --manifest-path runner/Cargo.toml -p paperclip-runner-core
pnpm build:binary
- name: Save the Runner release crate
if: ${{ matrix.lane == 'rust' && github.repository == 'paperclipai/paperclip' && github.event_name == 'push' && github.ref == 'refs/heads/master' && inputs.ref == github.sha && steps.runner_release_crate.outputs.cache-hit != 'true' }}
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
packages/paperclip-runner/runner/target/release/.fingerprint/paperclip-runner-core-*
packages/paperclip-runner/runner/target/release/deps/libpaperclip_runner_core-*
packages/paperclip-runner/runner/target/release/deps/paperclip_runner_core-*.d
packages/paperclip-runner/runner/target/release/deps/paperclip_runnerd-*
packages/paperclip-runner/runner/target/release/paperclip-runnerd
packages/paperclip-runner/runner/target/release/paperclip-runnerd.d
key: runner-release-crate-v1-${{ steps.runner_release_crate_key.outputs.scope }}-${{ hashFiles('packages/paperclip-runner/rust-toolchain.toml', 'packages/paperclip-runner/runner/Cargo.toml', 'packages/paperclip-runner/runner/Cargo.lock', 'packages/paperclip-runner/runner/crates/**', 'packages/paperclip-runner/protocol/**', 'packages/paperclip-runner/test-fixtures/**', 'packages/paperclip-runner/test/fixtures/**', 'tests/runner-e2e/fixtures/**') }}
build:
name: Build
runs-on: ${{ vars.AWS_POST_MERGE_CI_ENABLED == 'true' && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.ref == 'refs/heads/master' && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.sha != '' && inputs.ref == github.sha && 'runs-on/fleet=paperclip-post-merge-x64/env=public-ci' || 'ubuntu-latest' }}
+1
View File
@@ -71,6 +71,7 @@ tests/storybook-visual/.cache/
tests/storybook-visual/.snapshots/
tests/storybook-visual/baseline-review/
tests/storybook-visual/test-results/
tests/storybook-visual/repository-skill-results/
tests/storybook-visual/playwright-report/
.superset/
.superpowers/
+14
View File
@@ -5,6 +5,20 @@ must be attached to the Paperclip issue before the agent chooses a final
disposition. A local workspace path is not enough, because cloud users and
reviewers often cannot access the agent's disk.
## Upload files from a task
People can open the task's Artifacts tab and use the upload icon, or drop files
anywhere in the panel, including its header and existing artifact cards. Uploads
save directly to the task. They do not send a message, assign an agent, or
release a pause hold. Files from people and agents appear together in Artifacts.
Markdown imports create task documents. Repeated filenames get a new document
key instead of replacing an existing revision.
Hidden system keys and artifact-review prefixes are reserved; imports with
those filenames receive a visible upload document key. Other supported files use
the existing attachment API and its type and size limits. Failed uploads show
an inline Retry action while successful files remain available.
## Native runner
When `register_deliverable` is available, use it for files in the bound local or
+81 -3
View File
@@ -560,6 +560,41 @@ falls back to the fresh path automatically, so the cache can make a suite
slower but never fail it or skip it: a migration that cannot apply still fails
the suite on the fresh path.
#### Reused Runner build in the server build
The server build vendors `packages/paperclip-runner/dist`, so its
`prepare:runner-vendor` step has to make sure that directory is a current
build. It used to run the Runner's full `build` every time. Now the Runner's
`build` brackets itself with `scripts/record-build-inputs.mjs`: `--begin`, its
first step, forgets the previous stamp and records a digest of the build's
inputs (every file under `packages/paperclip-runner` except `dist`,
`dist-browser`, `runner/target`, `node_modules` and dot-directories; every
file under `packages/paperclip-eval-kernel` except `dist` and `node_modules`;
every file under the root `patches` directory, which the acpx profile check
hashes and the provider entrypoint bundler copies into `dist`; and the root
`package.json`, `pnpm-lock.yaml`, `pnpm-workspace.yaml` and
`tsconfig.base.json`). `--finish`, its last step, recomputes the digest and
writes the stamp
`packages/paperclip-runner/node_modules/.cache/paperclip-runner-build-inputs.json`
only if the inputs are still the ones the build began with, together with the
path and size of every file under `dist`. An edit that lands while the build
runs leaves no stamp, so the next consumer rebuilds.
`node server/scripts/prepare-runner-vendor.mjs` reuses the build when the
stamp is for this platform, every recorded output is still there at its
recorded size, and the current digest matches. Anything else (no stamp, a
changed input, a deleted or truncated output, a stamp from another platform)
runs `pnpm --filter @paperclipai/paperclip-runner build` exactly as before, so
a standalone `pnpm --filter @paperclipai/server build` on a fresh checkout
still builds the Runner itself. Inside `pnpm build` the Runner has just been
built by the same command, and the step skips a 12-16s rebuild. The step
prints one `paperclip.runner-vendor.timing.v1` line saying which it did.
Set `PAPERCLIP_RUNNER_VENDOR_REBUILD=1` to rebuild regardless of the stamp. A
stamp is only written after a Runner build that passed every step, and both
the Runner's first build step and the prepare step remove any existing stamp
before a rebuild starts, so a build that stops partway never leaves a stamp
that vouches for it.
### Task search evaluation
The task search relevance rubric and regression corpus are documented in
@@ -1278,14 +1313,57 @@ When a workspace service runs Paperclip for browser OAuth QA, configure its `exp
Keep relevant heartbeat extractions and their focused tests in `server/src/services/heartbeat/`.
Runtime-service coordination is in `server/src/services/heartbeat/runtime-services.ts`.
It validates service intents, starts configured services, records adapter-managed
services and primary URLs, and posts workspace-ready notices. Process registries
and leases stay in `workspace-runtime.ts`; the executor keeps release ordering in
its existing cleanup path. Public-path tests use real local service processes.
Instruction working-copy coordination is in
`server/src/services/heartbeat/instruction-working-copy.ts`. Its per-run controller
prepares copies, tracks save receipts, and coordinates warm-directory handoffs.
Live getters preserve updated copies and expose acquired reservations even if
preparation fails. The executor keeps release and environment-cleanup ordering;
a separate release closure defers cleanup failures without replacing run results.
See `tests/e2e/instruction-working-copy/README.md` for isolated browser acceptance.
Task assignment Markdown is rendered by `server/src/services/heartbeat/task-markdown.ts`.
`heartbeat.ts` calls the renderer and re-exports it for existing callers. Keep prompt
`heartbeat.ts` re-exports it for existing callers. Keep prompt
formatting changes in the renderer and its tests, separate from run orchestration.
Task context assembly is in `server/src/services/heartbeat/task-context.ts`.
`prepareHeartbeatTaskContext` rebuilds prompt fields on the current run context:
continuation summaries, wake comments, task plans, communication guidance, and
full/compact assignment Markdown. It preserves quarantine and secret redaction
before computing turn-context revisions. The executor supplies the already-loaded
issue, session, and trust policy, and retains admission, resource ownership, and
cleanup. Public-path characterization tests are in
`server/src/__tests__/heartbeat-task-context.test.ts`; the workspace-bootstrap
browser fixture also checks the persisted prompt variants and description revision
from a real process adapter before completing its task.
Execution configuration is assembled in
`server/src/services/heartbeat/execution-configuration.ts`: workspace and issue
overrides, scoped credentials, managed AI identity, and runtime skill delivery.
The executor receives credential ownership and failure-redaction updates as soon
as they are available, so later setup failures still clean up acquired resources.
Busy-account deferral returns explicitly before session/workspace preparation.
Public-path tests are in `server/src/__tests__/heartbeat-execution-configuration.test.ts`.
Session preparation is in `server/src/services/heartbeat/session-preparation.ts`.
It selects fresh or resumable sessions from effective config and managed-account
identity, then resolves adapter session identity and compaction after workspace
preparation. Both phases keep their original ordering in `heartbeat.ts`; they
acquire no runtime resources. Public-path characterization tests live in
`server/src/__tests__/heartbeat-session-preparation.test.ts`.
Run-log formatting is in `server/src/services/heartbeat/run-log.ts`. It bounds
stored event payloads, redacts and shortens log chunks, and caps stdout/stderr
excerpts. `heartbeat.ts` keeps event writes, current-user redaction, and live
event delivery. Existing public helpers remain available from `heartbeat.ts`.
excerpts. `run-output.ts` owns per-attempt log handles, current-user redaction,
output progress writes, live log delivery, and adapter event callbacks. The log
store remains service-scoped. Completion reads live excerpt getters so output
that arrives during finalization is retained. Existing public helpers and flush
interval constants remain available from `heartbeat.ts`.
Workspace preparation is in `server/src/services/heartbeat/workspaces.ts`. It
owns managed checkout materialization, workspace validation and reuse, referenced
+2
View File
@@ -340,6 +340,8 @@ Notes:
## General Notes
- The `docker-entrypoint.sh` adjusts the container `node` user UID/GID at startup to match the values passed via `USER_UID`/`USER_GID`, avoiding permission issues on bind-mounted volumes.
- Before it drops privileges, the entrypoint makes sure `PAPERCLIP_HOME` is owned by the `node` user. Each boot checks only the home directory and its first three levels. The first boot (and any boot after a UID/GID change) also checks the full tree, and then writes `PAPERCLIP_HOME/.paperclip-ownership-verified`. While that marker is current, later boots skip the full-tree walk, which can take minutes on homes with many files. A mismatch triggers a recursive `chown`. If files deep in the tree get the wrong owner (for example, after a restore as root), delete the marker and restart to force a full check. If a check fails (for example, with an I/O error), the entrypoint does not write the marker, so the next boot checks again.
- The entrypoint writes timestamped lines when it starts, around the full ownership check or any repair (with the duration), and just before it starts the server, so you can measure the pre-start time from the container log.
- Paperclip data persists via Docker volumes/bind mounts (compose) or at `~/.local/share/paperclip` (quadlet).
## Native Runner build cache
+53 -2
View File
@@ -363,14 +363,65 @@ paths and the PR lanes can restore master's entry. The guard tests under
`.github/scripts/tests/` require this step, with identical text, before every
Rust cache step in both workflows.
Workspace crates and installed Cargo binaries are excluded. Every run rebuilds
workspace code and runs all assigned checks, including on a cache hit. Only an
Workspace crates and installed Cargo binaries are excluded from this entry.
Every run runs all assigned checks, including on a cache hit. Only an
own-repository master-push run verifying that push's exact SHA can restore the
cache, and only a successful Rust lane saves it. PR, tag, and manual candidate
verification compile without this cache. A miss or eviction costs compilation
time but does not change the checks. To discard old dependency caches, increment
the shared-key version and let the next successful master verification warm it.
### Runner release crate cache
The dependency entry leaves the Runner's own crate out, and a fresh checkout
invalidates cargo's mtime fingerprints for workspace crates anyway, so every
pull request build compiled `paperclip-runner-core` in release mode from
scratch (28-49s per job on 2026-10-11) although most master commits leave its
inputs untouched. A second, content-addressed entry holds only that crate's
release artifacts: its fingerprints, the lib's rlib and rmeta, the
`paperclip-runnerd` binary and their dep-info, about 70 MB. The path list and
the key inputs live in `scripts/runner-release-crate-cache.mjs`, which both
workflows call, and `scripts/__tests__/runner-release-crate-cache.test.mjs`
holds the readers and the writer to them.
The key is `runner-release-crate-v1-<scope>-<hashFiles>`. The scope names the
platform, the pinned compiler (`rustc --version --verbose`: release, commit and
host) and a hash of the checkout root, because cargo's dep-info records the
protocol files the crate includes by relative path as absolute paths. The
`hashFiles` part covers every input the build reads: the toolchain pin, the
manifests and lockfile, the crate sources, the protocol schemas and fixtures,
and the three fixture trees the test modules include (two under the Runner
package and `tests/runner-e2e/fixtures` at the repository root). Any change
to any of them selects a new key. A guard test resolves every `include_str!`
and `include_bytes!` in the crate, whether its argument is a literal or a
`concat!` of literals, `env!("CARGO_MANIFEST_DIR")` and macro metavariables,
and fails on an include that lands outside these patterns or uses a form it
cannot resolve. Readers pass no `restore-keys`: a prefix match would hand
cargo another revision's output.
The writer is the Rust lane of the same master-push verification, after its
checks and the debug warm-up. It looks the key up first and, on a miss, runs
`cargo clean --release -p paperclip-runner-core` and then the package's
`build:binary` (the exact command the readers run) before saving, so the entry
carries no test-profile variants. In the trusted PR workflow, the Typecheck,
Build and Canary Dry Run jobs and the two Verify Paperclip Runner lanes that
build the crate (`rust` and the final vitest shard) restore the entry
read-only right after the dependency restore. On a hit, `mark-current` sets
the mtime of every restored artifact to now, so cargo's freshness check sees
every source as older than the build and keeps the artifacts. A restore that
lacks any file cargo reads or checks for either unit (the fingerprint hash
and dep-info of each unit, the lib's rlib and rmeta, the binary under
`deps`; measured with cargo 1.97.1, which recompiles without any one of
them and recreates the uplifted binary without any of the rest) is removed
instead, the job log names the missing file, and cargo compiles as on a
miss. `pnpm build`
then prints `Finished release profile` in well under a second for that crate.
On a miss nothing changes: cargo compiles the crate as before.
The trust boundary is unchanged: only master writes, a pull request can only
restore master's entry, and it restores only artifacts built from inputs
byte-identical to its own merge tree.
The trust boundary is the protected master branch, not the cache-key text.
GitHub does not let master restore caches created by a child branch, sibling
branch, tag, or PR merge ref. Both permitted restore scopes (current branch and
+22 -1
View File
@@ -2042,7 +2042,28 @@ unavailable. Preserve current ownership and newer-work fences. See
and declared `compatibility` requirements with source entries. Preview reads reauthorize
the current caller and audit the requested package at the scanned immutable commit;
discovery and source metadata contain no file contents. Reference checks are advisory,
do not establish complete runtime dependencies, and never expand package boundaries.
do not establish complete runtime dependencies, and never expand package boundaries automatically.
- New UI imports use `packageMode: repository` to preserve supported repository files
in one company-scoped immutable snapshot. Selected entrypoints become library skills;
other skills remain supporting files. The root `paperclip.skills.json` may declare
version 1, validated `skills` paths and `requirements` text. Invalid manifests fall
back to discovery. The dialog shows every skill and containing folder, selects all
discovered skills, and collapses package files. There is no import-mode chooser;
existing directory-mode sources and API callers remain compatible.
Unsupported links, oversized files and files rejected for unsafe execution or
secret disclosure are omitted with persistent warnings. Malformed entrypoints
stay selected with Needs setup warnings; usable selected skills still publish.
Limits are 9,990 repository entries, 1 MiB per file and 61 MiB total. The dialog
exposes included files and setup requirements without repair choices or another
confirmation. Installed file views read saved repository bytes and resolve links
from each instruction file's original directory.
- Each repository-backed version references its snapshot and original entrypoint path.
Runtime materialization adds a discovery wrapper and preserves the original tree under
`.paperclip-repository/`, including binaries, modes and credits. Original instructions
are unchanged. `agents/openai.yaml` discovery policy is retained beside the wrapper.
Existing adapters carry self-contained copies, so multiple entrypoints may duplicate
runtime bytes. Shared-file changes version all selected skills; pinned runs remain
on their snapshot. Importing does not install dependencies or configure host tools.
- Stage and audit complete packages before publishing. Scripts are allowed through
the existing content audit and never run on import. Persist binary bytes and executable
flags in immutable version inventories (legacy entries default to UTF-8/non-executable).
+9 -1
View File
@@ -715,7 +715,15 @@ A source tracks repository identity, branch, selected paths, and installed commi
an existing GitHub connection supplies caller-authorized reads. Manual refresh publishes
complete local immutable versions, preserving skill identity and assignments. Selection
operates on whole skill packages, with inspectable included files, declared runtime
requirements, and advisory warnings for missing or external references. New
requirements, and advisory warnings for missing or external references. A repository
package retains supported source files while the user selects which skill entrypoints
appear in the library. Authored content remains unchanged and shared files travel to
the runtime. A declarative manifest can suggest public entrypoints and setup requirements;
new imports keep repository files together and select all discovered skills. Missing
references, setup problems and omitted files produce persistent warnings. Files rejected
for unsafe execution or secret disclosure are excluded without blocking other files.
The installed file viewer exposes saved repository dependencies. Importing never runs
setup or configures host tools. New
upstream skills require reviewed selection; removed or deselected skills remain
installed. Editing starts with an independent copy. Write-back and PR publication
are a later milestone; exact path and commit provenance provide their base.
@@ -0,0 +1,29 @@
# Repository skill packages
Outcome: GitHub imports can preserve a complete repository while independently selecting library entrypoints. Original authored files remain unchanged. Existing skill-directory sources remain supported; new imports keep repository files together. Imports do not execute setup scripts.
Branch: `codex/skill-repository-packages`, based on `origin/master` at `7333e4eba8`.
Acceptance: evaluation prose is accepted; dynamic execution checks remain; repository snapshots preserve shared references, binaries, executable files and credits through runtime copying; shared-file updates produce coherent pinned versions; shared-file problems are disclosed without blocking the package; the existing import dialog explains selection, files and requirements and supports inspecting both.
Implementation: one immutable, company-scoped repository snapshot is referenced by each imported skill version. Runtime projections contain a generated discovery entrypoint and the untouched tree under `.paperclip-repository/`, so all existing adapters carry the dependencies without introducing host-only symlinks. Projections may duplicate package files for multiple enabled entrypoints; the database stores each package once. Optional `paperclip.skills.json` declares version 1, public `skills` entrypoint paths and `requirements` text. It never runs installation commands.
Verification: 55 focused scanner, delivery and UI tests pass. The full workspace typecheck, full build, and direct server/UI TypeScript checks pass. Design-token gates pass. Scanning the pinned example distribution accepts all 62 skills without changing their text. A real Codex discovery probe exposes only the selected entrypoint while retaining the complete repository as supporting files.
Follow-up verification: all 12 PostgreSQL lifecycle tests pass, including company deletion and both copy APIs. The alternate create-from-existing API copies the complete runtime projection and preserves its discovery wrapper. All 163 focused runner evidence, ownership, catalog and isolation tests pass, as does the runner E2E TypeScript check. The final full workspace build and typecheck pass.
Live qualification: the opt-in `repository-skills` suite imports the private Poteto repository through the production Sources UI, selects and assigns architect and bro, disables the source connection, then invokes those skills. Independent inspections of both delivered packages match all 234 Git blobs, sizes and executable modes at commit 37d2ddde97aaef8017ccef811e682948f421062d. Native Codex tasks complete successfully on local and Daytona. Both qualifying results have complete evidence, no credential leaks and successful resource cleanup. No test manually stages the skill repository. Local campaign: local-2026-10-09T22-27-38-635Z. Daytona campaign: local-2026-10-09T22-16-10-814Z.
The location oracle recognizes both provider-home copies and the immutable bundles supplied in the active runtime context. Local ownership includes the company, agent, workspace and session from the persisted, validated execution binding. Daytona ownership includes the active company/agent/task/run sandbox lease and its native-session directory. Earlier harness failures from missing report artifacts and incorrect location checks were corrected and rerun.
The first broad test attempt was stopped when PostgreSQL-dependent suites skipped on the exhausted host. The new full `pnpm test:run` was stopped after seven failures across six test files unchanged by this PR (native session recovery, agent conversations, workspace runtime exposure, workspace dispatch, connection removal, and skill utility locking), plus 21 skipped connection-intent database cases. The recovery assertion reproduces in isolation; the failing test files are unchanged from the starting commit and the recovery fixture omits skill runtime context. The broad suite remains incomplete and is not counted as passed. Live qualification verifies import, delivery and instruction access, not every Poteto workflow or adapter.
Review: [PR #15731](https://github.com/paperclipai/paperclip/pull/15731). The branch is based on master and has no unrelated checkout changes.
## October 12: import warnings
The importer saves available repository files without asking for repair decisions. Unsupported links and oversized files are omitted with persistent path-specific warnings. Missing references remain visible on the source; repository imports include shared files automatically. An invalid optional manifest falls back to discovered entrypoints. A skill that cannot be registered remains selected with a Needs setup warning; its original bytes still travel as repository support content for usable entrypoints. Content audits remain available, and import never executes setup commands.
The production dialog shows every skill by default, selects all discovered skills, and collapses individual package files. It uses one collapsed warnings disclosure and leaves Import repository available. The importer has no packaging option: new imports keep repository files together. Storybook covers omitted files, warning details, missing references, setup problems, warnings after import, and mobile review. Focused scanner/delivery/PostgreSQL tests and UI tests verify saved warnings and available bytes; browser fixtures exercise the production UI. This revision has not rerun the paid local/Daytona campaigns above.
Final acceptance also found and fixed duplicate saved warnings and an installed-file viewer that did not expose repository dependencies. The viewer now lists repository files and reads their saved bytes without GitHub access. Content errors for unsafe execution or secret disclosure omit the affected files with warnings; the remaining repository can still import. Database and scanner regressions verify these exclusions and file-reader company isolation.
@@ -0,0 +1,306 @@
# Chat channel service extraction sequence
## Scope and finish line
Apply the refactor-large-file workflow to `server/src/services/chat-channels.ts`.
Map five responsibilities, then extract and verify one at a time. This slice
extracts provider message normalization without changing provider behavior,
public service exports, database contracts, authentication, or worker ownership.
Subsequent slices require reassessment against the preceding result. No merge or
deployment is part of this slice.
## Baseline
- Base: `1100682261` on a clean, detached task worktree.
- File: 39,132 lines / 1,502,701 bytes; imports occupy 421 lines.
- `chatChannelService`: 36,152 lines, including nested functions and wiring.
This is not a measurement of orchestration alone. The first slice removes
top-level normalization; it does not simplify that closure yet.
- Largest nested functions: `processMessage` (2,164 lines), `handleAction`
(1,074), `applyProviderLifecycleEffect` (895), `configureWithCredentialLease`
(868), `processSelectedPublication` (728), `handleModalSubmit` (708),
`handleWebhook` (684), `failedChatRetrySource` (677), and
`processLifecycleDelivery` (657).
- Existing siblings already own SDK runtimes/state, provider inventory reads,
resource lifecycle parsing, GitHub app management, registration, publication
projection/streaming, Teams transfers, Telegram media, and interaction forms.
Extend those boundaries where appropriate instead of duplicating them.
- Callers include application assembly, chat routes, issue/agent routes, and
existing unit/integration tests. Keep their imports and service API stable.
## Ranked extraction candidates
Estimates are source lines moved / net reduction after wiring. Later estimates
are provisional: explicit dependency contracts must precede each extraction.
New paths below are relative to `server/src/services/`.
| Rank / responsibility | Destination | Approx. move / net | Inputs and outputs | State and effects owned | Risk and verification |
| --- | --- | --- | --- | --- | --- |
| 1. Provider message normalization | `chat-channels/provider-messages.ts` | 540 / 520 | Provider payload, author, Request, Teams parser, or retained delivery → actor, lifecycle event, message ID, revision digest, timestamp | No mutable service state or DB writes; consumes the supplied GitHub Request body | Preserve exact revision hashes, limits, native IDs, malformed-input handling, and raw clock provenance. Existing public service integration tests cover all four providers. |
| 2. Resource inventory reconciliation and availability | `chat-channels/resource-reconciliation.ts`, alongside `chat-provider-inventory.ts` and `chat-provider-lifecycle.ts` | 1,300–1,500 / 1,200–1,400 | Endpoint, inventory/lifecycle effect, credential lease, DB and explicit runtime callbacks → reconciled resources and availability | Company-scoped resource/conversation transactions; GitHub stable repository identity and provider-order checks | Keep advisory locks, lease checks and the complete availability transaction together. Verify repository rename, removal/reinstall, resource auditing, Telegram migration and stale callbacks. |
| 3. Identity linking | `chat-channels/identity-links.ts` | 430–600 / 390–550 | DB, endpoint lookup, live public URL getter, principal, token, authenticated user → link preview/confirmation/revocation | Token hashes/expiry, membership checks, join requests and transactional audit records | No cached authority or URL snapshot. Preserve endpoint/company binding, replay behavior and revoked/conflicting links; run identity route and DB integration tests. |
| 4. Publication replacement and supersession | `chat-channels/publication-replacement.ts` (the existing reconciliation module owns wake signals, a separate responsibility) | 1,000–1,150 / 950–1,100 | DB, publication, current endpoint/conversation/run → replacement candidate or suppression reason | Scoped reads and existing issue/run authorization snapshots; no transport or queue ownership | Preserve exact outbound link ownership, interaction precedence, terminal failure replacement, and issue→run lock order. Run publication reconciliation, progress and integration fixtures. |
| 5. Failed chat run retry coordination | `chat-channels/failed-run-retry.ts` | 1,600–1,900 / 1,400–1,700 | DB, source/action IDs, explicit admission/heartbeat/publication hooks → authorized retry, claim and completion | Per-service in-flight retry promises and authority registration/cleanup, persisted source evidence | Higher risk: preserve single retry ownership, no replay of provider calls, live source revalidation and shutdown ordering. Run reconstructed-service, edited-source, revocation and concurrent retry integration fixtures. |
Start with candidate 1: it establishes a domain folder and removes an independent
responsibility without introducing a mutable context object. It is deliberately
smaller than the later stateful slices. Resource lifecycle effects remain in the
existing parser; message edits/deletes are a different normalization contract.
Do not move runtime leases, webhook authentication, conversation admission,
delivery transactions, transport attempts, or shutdown piecemeal. The service
must continue to show who admits work and who releases resources. Candidate 2
needs a narrow callback contract; candidate 5 must preserve the same service
lifetime for registration and in-flight promises. Candidates are sequential,
not independent concurrent changes to this file.
## Provider folder convention (2026-10-11)
Provider-specific code belongs under `chat-channels/<provider>/`, using the
existing provider identifiers. The normalization extraction now lives in
`telegram/messages.ts`, `slack/messages.ts`, `discord/messages.ts`,
`microsoft-teams/messages.ts` and `github/messages.ts`. Shared actor normalization,
actor data and the inbound text limit live in `message-lifecycle.ts`. The service
imports the provider modules directly. Future extractions should follow this
layout; move older flat provider modules when their responsibility is touched.
All 22 declarations retain their exact bodies. The service changes imports only.
The same 48 provider integration cases passed before and after this move, and
server typecheck and module boundaries pass. The historical extraction sizes
below predate this folder change: direct imports add 10 service lines / 281 bytes.
The six message modules now total 566 lines / 19,534 bytes, including imports.
## Verification and progress
Before the production move, run existing service integration cases for message
lifecycle, file revision identity, provider timestamps, Telegram zero-message
handling and rich content. These use disposable PostgreSQL and fixture provider
transports, not live third-party execution. Keep those tests at their existing
public entry points. Add characterization only if inspection exposes a gap.
After the move, repeat that selection, compare extracted declarations and the
remaining file mechanically against the base, then run the relevant adjacent
suites and repository typecheck, full test and build gates. Record blockers and
unverified checks explicitly; do not treat skipped integration tests as passes.
- Dependencies: installed with the repository-pinned pnpm 9.15.4. The host's
newer pnpm ignored legacy patch settings, so verification uses the pinned tool.
- Original-base integration selection: 48 passed before and 48 passed after
extraction. Existing tests cover the boundary; no new test-only exports or
mock-only characterization were needed.
- Rebased onto `b1ebe3de3c` after master added fast-response acknowledgements
and action queue notifications. The only conflict was the import block;
retain upstream imports and add the normalization import. All 21 extracted
declarations match the new base, ignoring added exports and two narrowed
input types. Every remaining non-import declaration, including the entire
service body, matches the new base exactly.
- Current size: 39,155 lines / 1,509,090 bytes before extraction; 38,633 lines /
1,491,422 bytes after. Net reduction: 522 lines / 17,668 bytes. Destination:
552 lines / 19,028 bytes, including imports and the shared text-limit constant.
- On the rebased source, the service spans 36,175 lines: 287 direct nested
function declarations occupy 35,324 lines; 51 other statements occupy 504
lines of state, wiring, registration callbacks and the public API. The rest
is comments, whitespace and delimiters. Those counts remain unchanged by
this slice; they make the remaining orchestration work explicit.
- The module takes only the Teams parser method and the two delivery fields it
reads, rather than requiring an entire runtime or database row. These are
type-only changes. It imports sibling implementations directly, never the
service entry point. Public service exports remain unchanged.
- Initial full-test/typecheck runs were interrupted to rebase and are not
passes. Rebased integration selection: 48 passed (the other 1,185 cases were
excluded by the test-name filter). Module-boundary and whitespace checks pass.
Workspace `pnpm -r typecheck` and `pnpm build`: passed. The full suite and
remote checks were still running at PR opening; final results belong in the
PR's Verification section below.
Reproduce the focused service-path check with the repository-pinned pnpm:
```sh
pnpm exec vitest run server/src/__tests__/chat-channels.integration.test.ts \
-t 'records message edits and deletes durably|redacts lifecycle edits|Telegram edit|Teams edit, soft-delete|verified Telegram edited_message|verified GitHub comment edit|provider timestamp provenance|Telegram retained zero-message admission|Telegram current rich inbound compatibility|deduplicates Slack file-only revisions|preserves exact Discord source authority'
```
Branch: `codex/chat-channel-provider-messages`. The user authorized publishing
this slice to `paperclipai/paperclip`; subsequent extractions are not part of
this PR.
Review and final verification: [PR #15823](https://github.com/paperclipai/paperclip/pull/15823).
## Authorized stacked continuation
The user requested at least five stacked PRs on 2026-10-10. Continue the four
remaining boundaries in order, with each branch based on its predecessor. Keep
merge and deployment outside this series. Each PR carries its own current-head
verification and review results; do not transfer a prior head's pass to a new one.
### Slice 2: resource reconciliation and availability
The lifecycle handler also owns credential replacement, delivery claims and
runtime shutdown. Keep those responsibilities in the service. Extract the full
non-GitHub availability transaction, inventory reconciliation, repository rename
handling, provider ordering, resource listing and audited selection together.
The factory takes the database, endpoint lookup, existing credential lease
function, generation reader and error text limit. It creates no worker or cache.
- Source: 38,633 lines / 1,491,422 bytes → 37,720 lines / 1,454,762 bytes.
- Destination: 947 lines / 39,584 bytes in `resource-reconciliation.ts`.
- Mechanical comparison: 14 declarations and the complete availability
transaction are unchanged; all other function bodies remain unchanged.
- Baseline: all three complete chat CI shards passed on predecessor `cbba30a526`.
- Local baseline and after selections could not start PostgreSQL and skipped
the suite. They are not passes. Server typecheck and module boundaries pass.
- Require complete chat CI on this slice before starting the next extraction.
### 2026-10-12 channel-folder reassessment: slice 2
PR #15823 was merged at the user's request. Rebase this slice onto the merged
master, then retain the remaining four PRs for review in dependency order.
- GitHub payload parsing, repository coordinate repair, webhook recovery,
repository pagination and bulk-selection entry points live in
`chat-channels/github/resources.ts`.
- Telegram migration preparation and conversation migration live in
`telegram/resources.ts`. They receive the existing transaction and timestamp;
the shared coordinator still upserts the destination between those steps.
- Slack fallback labels, Teams conversation IDs and Photon participant parsing
live in each provider's `resources.ts`.
- Shared inventory replacement, availability settlement and audited selection
retain their transaction and credential-lease ownership. Small provider
dispatch predicates remain next to the shared operation they select.
- All eight moved declarations and three transaction blocks retain their logic.
The GitHub provider guard stays in the caller. No factory starts background work.
- Existing public-service tests: 15 resource cases and the Telegram migration
case passed before; the same 16 cases passed after. The name filter excludes
unrelated cases. No live third-party provider calls were made.
### Slice 3: identity linking
Extract six identity-link operations into `identity-links.ts`. Keep the live
public URL getter. Fetch confirmation runtime authority through a callback that
reads the existing service map after awaiting runtime creation. Keep the same
transaction locks, token consumption, membership checks and post-commit notice.
- Source: 37,720 lines / 1,454,762 bytes → 37,295 lines / 1,436,867 bytes.
- Destination: 459 lines / 19,970 bytes.
- Baseline: complete CI and 5/5 review on slice 2 (`c01f34ad2b`, PR #15834).
- Server typecheck, module boundaries and mechanical comparison pass. The only
body substitution moves the runtime lookup behind the supplied live callback.
All remaining statements are unchanged. Local PostgreSQL remains unavailable;
record this slice's complete CI and review in its PR before continuing.
### 2026-10-12 channel-folder reassessment: slice 3
Keep identity-token consumption, principal scope, membership checks, link locks
and revocation in shared `identity-links.ts`. Move Slack receipt queries,
self-service access requests and confirmation-notice staging into
`slack/identity-links.ts`. GitHub identity fields live in `github/identity-links.ts`.
The shared confirmation transaction still stages the Slack notice before commit
and schedules it only after commit. The live runtime and public URL callbacks
remain live. The Slack factory holds no cache or worker state.
Four existing public-service cases passed before and after, including the full
Slack connect journey: discovery, preview, outsider access request, duplicate
request, member confirmation, provider notice, consumed-token rejection and
endpoint isolation. Receipt queries, access-request transaction and notice
payload retain their statements with explicit parameter substitutions.
### Slice 4: publication replacement and supersession
Move 13 functions into `publication-replacement.ts`. The module selects the exact
provider message a publication can replace and suppresses obsolete progress. It
uses the database, endpoint lookup and retained-source authorization callback.
Transport execution, publication claims and receipt reactions remain coordinated
by the service. No state or registration moves in this slice.
- Source: 37,295 lines / 1,436,867 bytes → 36,393 lines / 1,400,636 bytes.
- Destination: 963 lines / 38,724 bytes.
- Baseline: all 51 checks and 5/5 review passed on slice 3 (`581dd656c1`, #15836).
- All 13 bodies and every remaining statement match the predecessor exactly.
Server typecheck, module boundaries and whitespace checks pass.
- Existing CI fixtures cover exact outbound ownership, interleaved control
messages, consumed progress, interactions, committed failure replacement and
issue-to-run lock order. Local database capacity remains blocked.
### 2026-10-12 channel-folder reassessment: slice 4
Publication replacement is shared ownership logic: exact outbound links, run and
wake scope, consumed progress messages, interaction prompts and retained-source
authorization. Keep those queries together. The close-message provider allowlist
is shared dispatch metadata, not four separate implementations.
Move the iMessage queued/working suppression policy and explanation to
`imessage-photon/publications.ts`. Preserve when endpoint lookup occurs and leave
all ownership queries unchanged. Baseline: 49 public-service publication and
Photon cases passed, including close-owned progress, interleaved Slack control,
Telegram confirmation and FIFO replacement. The same 49 cases passed after. Server typecheck and module boundaries pass.
### Slice 5: failed chat run retry coordination
Extract source validation, native failure evidence, retry preparation, claims,
wake authorization and retry notices into `failed-run-retry.ts`. Keep the
in-flight promise map, notice cursor, error class and two authority registrations
inside the factory so each service retains its own lifetime. Keep the live
shutdown and heartbeat callbacks. The service still owns cleanup order and waits
for the module's pending retries before unregistering authorities.
- Source: 36,393 lines / 1,400,636 bytes → 34,606 lines / 1,336,443 bytes.
Net reduction: 1,787 lines / 64,193 bytes.
- Destination: 1,936 lines / 68,329 bytes.
- Baseline: all 52 checks and 5/5 review passed on slice 4 (`166ea04ac6`, #15844).
- Added three tests through the existing public service before the move. All
three passed before and after: duplicate calls share a promise, shutdown
during a pending read prevents a new claim and waits for completion, rejected
work permits a later attempt, and old-service cleanup preserves replacement
authority. These tests use controlled reads and do not require PostgreSQL.
- All 21 moved declarations and every remaining statement match the predecessor
after restoring the documented live callbacks and pending-task accessor.
Server typecheck, module boundaries and whitespace checks pass.
- Existing CI database fixtures cover corrected/edited sources, lifecycle epochs,
native recovery evidence, recreated services and concurrent retries. Local
full database execution remains blocked by the host's shared-memory limit.
Current-head full CI, broad local gates and review results belong in the PR.
## Reassessment after five slices
The service falls from 39,155 lines / 1,509,090 bytes at the rebased baseline to
34,606 lines / 1,336,443 bytes: 4,549 lines and 172,647 bytes removed (11.6% of
source lines). The five modules total 4,857 lines / 185,635 bytes, including their
imports and typed boundaries. This is useful separation, not completion of the
larger decomposition.
The service function still spans 32,254 lines. Its 244 direct nested functions
occupy 31,525 lines; 48 other statements occupy 435 lines of state, wiring,
registration and public API. The remaining lines are whitespace, comments and
syntax. The largest functions are `processMessage` (2,166), `handleAction`
(1,074), `configureWithCredentialLease` (868), `processSelectedPublication` (729),
`handleModalSubmit` (708), `handleWebhook` (685), `processLifecycleDelivery` (657)
and `applyProviderLifecycleEffect` (593).
The best next planning candidate is inbound message admission and persistence,
starting from `processMessage` and its conversation/drain ownership. Define the
transaction, live-message retention and scheduling contracts before moving it;
do not move that entire function and claim its complexity is resolved. Action
resolution and endpoint configuration are later candidates. Keep runtime lease
acquisition, partial startup and shutdown together until their ownership contract
is explicit. This batch ends with five stacked PRs; merging is separate work.
### 2026-10-12 channel-folder reassessment: slice 5
Keep retry admission, complete source-batch authorization, ancestor checks,
in-flight deduplication, authority registration and shutdown coordination shared.
References to native runner providers describe execution recovery, not chat
channel implementations, and remain with that shared proof.
Move the exact Teams file-transfer continuation proof into
`microsoft-teams/file-transfer-source.ts`. It receives the caller's transaction,
explicit source identifiers and original denial factory; it returns only the
verified publication ID. The shared caller still validates the committed result,
all original inputs and other uncertain publications. All provider proof guards,
join predicates, row locks and attempt checks retain their statements.
Baseline: 59 retry cases plus six negative Teams authority cases passed. These
include service lifetime checks and transfer ambiguity without duplicate uploads.
The same 65 cases passed after. Server typecheck and module boundaries pass.
The remaining retry coordinator
has 1,825 lines; the Teams proof module has 149 lines.
@@ -0,0 +1,119 @@
# Chat channel workflow refactor, batch two
Base: `37913e4907324838ae2c8c94dc8155002b6d4223` (fresh master after the previous stack merged). The previous five PRs
(#15823, #15834, #15836, #15844, #15845) are merged. The service remains
34,603 lines. This batch creates five dependent PRs; merging is separate work.
| Order / responsibility | Destination | Approximate lines moved | Contract and ownership | Verification |
| --- | --- | --- | --- | --- |
| 1. Telegram maintenance and Stop subscription repair | `chat-channels/telegram/maintenance.ts` | 850 | Endpoint lookup, credential lease, credential resolution, runtime fence, live webhook URL, fetch. Own the per-service subscription scan cursor. Preserve claim/confirmation/settlement ordering. | Existing maintenance, rotation, lost-lease, subscription recovery and durable draft Stop integration cases before/after. |
| 2. Slack session stopping | `chat-channels/slack/session-stop.ts` | 750 | Stage and consume durable stop receipts; call shared principal authorization and task-control publication. Preserve captured target, event chronology and transaction locks. | Existing Slack stop, stale-session, authorization and failure-recovery cases before/after. |
| 3. GitHub durable webhook ingress and recovery | `chat-channels/github/webhook-ingress.ts` | 1,400 | Persist verified ingress, replay through the shared webhook entry point, recover provider failures. Keep persistence lease, live shutdown check and current endpoint/credential fences. | Existing signed webhook, restart/replay, recovery and stale-source integration cases before/after. |
| 4. Slack task-start admission | `chat-channels/slack/task-start.ts` | 920 | Recover task starter receipt, revalidate principal/resource/runtime and admit via the shared message path. Preserve lease ownership, synthetic message identity and claim barrier. | Existing slash-task concurrency, partial-post and recovery cases before/after. |
| 5. Discord native commands | `chat-channels/discord/commands.ts` | 550 | Reconcile command registration and admit native controls; shared service retains gateway ownership and authorization. Read gateway ownership live. | Existing command registration, stale runtime, gateway ownership and native command cases before/after. |
Keep message admission, shared control authority, provider runtime startup,
publication transport and service shutdown together. Each factory receives named,
typed dependencies. Do not copy mutable maps or snapshot live values. Shared
runtime record types may move to a type-only sibling module; do not introduce a
runtime import back to the original service.
For each slice: baseline existing public-service tests, mechanical extraction,
syntax/body parity, same tests afterward, server typecheck and module boundaries,
then PR checks and current-head review. Run broad local gates once at final
handoff. Record failures honestly; a retry does not erase the original failure.
## Slice 1: Telegram maintenance
- Service: 34,603 lines / 1,337,182 bytes → 33,718 lines / 1,306,801 bytes.
- `telegram/maintenance.ts`: 931 lines / 32,792 bytes; type-only shared runtime
records: 38 lines / 1,200 bytes.
- Twelve moved declarations and six shared types preserve token-level syntax;
all remaining service statements are unchanged. The module owns the existing
scan cursor and creates no timers or background work. Its live callbacks retain
credential-lease checks and runtime fencing around every provider mutation.
- The same 41 existing Telegram public-service cases passed before and after.
Server typecheck, module boundaries and whitespace checks passed. Full local
repository gates are deferred to the cumulative handoff; remote CI/review will
be recorded in this slice's PR.
- Reassessment: keep Telegram draft Stop admission and publication transport in
the orchestrator. Slack durable session stopping remains the next cohesive cut.
## Slice 2: Slack session stopping
- Service: 33,718 lines / 1,306,801 bytes → 32,928 lines / 1,279,554 bytes.
- `slack/session-stop.ts`: 858 lines / 30,166 bytes.
- All six moved declarations and all remaining service statements preserve
token-level syntax. A getter retains the original live heartbeat property
access. Shared principal authorization and publication staging stay in the
caller; the entire captured-target claim and settlement transactions move.
- Both existing public-service scenarios passed before and after. They cover
exact-run cancellation, retry/deduplication, guest denial, delayed events,
queued wakeups, a wakeup promoted after Stop, and cancellation losing to run
completion. Server typecheck passes. This factory owns no mutable state.
- Reassessment: GitHub ingress/recovery has a coherent durable inbox and replay
contract; preserve its shutdown getter and persistence compare-and-set lease.
## Slice 3: GitHub webhook ingress and recovery
- Service: 32,928 lines / 1,279,554 bytes → 31,438 lines / 1,225,374 bytes.
- `github/webhook-ingress.ts`: 1,579 lines / 58,233 bytes.
- All 26 moved declarations and remaining statements preserve syntax after the
explicit shutdown getter substitution. The replay barrier remains a live
getter. Persistence and credential leases remain the same service instances.
- The same 84 existing public-service cases passed before and after, including
authentication before retention, disabled-repository filtering, restart,
recovery epochs, concurrent scans, ambiguous redelivery, operator retries,
canonical-source changes and stale-runtime replay. Server typecheck passes.
- Reassessment: Slack task starter confirmation, recovery and failure settlement
form one boundary. Keep normal message admission shared and pass its exact
existing call contract; do not duplicate provider-visible posts during recovery.
## Slice 4: Slack task starts
- Service: 31,438 lines / 1,225,374 bytes → 30,449 lines / 1,190,643 bytes.
- `slack/task-start.ts`: 1,075 lines / 39,262 bytes.
- Nine moved declarations and all remaining statements preserve syntax. The
existing positional message-admission contract is unchanged. The claim barrier
remains live through a getter; no worker or mutable per-service state moves.
- The same 25 public-service cases passed before and after, covering slash-task
admission, provider-confirmed recovery without reposting, reclaimed admission,
stale credential ownership, ambiguous sends and explicit operator retries.
Server typecheck passes.
- Reassessment: Discord command registration and native control admission can
share a channel module. Expose only a live lease-key/token lookup; retain the
gateway ownership map, renewal, partial acquisition and shutdown in the service.
## Slice 5: Discord commands
- Service: 30,449 lines / 1,190,643 bytes → 29,897 lines / 1,170,080 bytes.
- `discord/commands.ts`: 639 lines / 25,036 bytes.
- Both moved function bodies and all remaining statements preserve syntax after
replacing the map read with a live lease-key/token callback. Gateway map,
renewal, runtime identity, partial startup and shutdown ownership stay shared.
- The same 53 registration and native-command integration cases passed before
and after. They cover registration failure, stale/runtime/gateway authority,
command receipts, exact target replay and current principal/resource access.
Server typecheck passes. Network work remains outside command transactions.
## Reassessment after batch two
The service falls from 34,603 lines / 1,337,182 bytes to 29,897 lines /
1,170,080 bytes: 4,706 lines and 167,102 bytes removed (13.6% of source lines).
The five provider modules total 5,082 lines, plus 38 shared type lines.
The service function itself spans 27,835 lines. Its 212 nested functions account
for 27,146 lines; 51 state/wiring/registration/public API statements account for
435 lines. Imports account for 423 lines. This batch reduces provider workflow
mixing; it does not resolve the large shared admission functions.
The largest remaining functions are `processMessage` (2,166 lines),
`handleAction` (1,074), `configureWithCredentialLease` (868),
`processSelectedPublication` (729), `handleModalSubmit` (708), `handleWebhook`
(685), `processLifecycleDelivery` (657), and `applyProviderLifecycleEffect` (593).
Future candidates include Teams file-transfer authority/protocol and shared
inbound persistence/drain ownership. Preserve the shared transaction and live
message contracts before attempting those cuts. Stop this batch at five PRs.
Cumulative verification: `pnpm -r typecheck` and `pnpm build` passed. Full local
test suite results and current-head CI/review are recorded in the final PR.
+404
View File
@@ -0,0 +1,404 @@
# Test audit: remove redundant proof and repeated fixtures
Date: 2026-10-12
Baseline: `fb21a3f30a`
Reference: [OpenClaw test audit](https://github.com/openclaw/openclaw/blob/main/.agents/skills/test-audit/SKILL.md)
The combined cleanup removes **11,976 net test/support lines, 41 unused
production lines, and 1,073 registered cases**. Four obsolete ACPX
characterization files and the old Codex driver monolith are retired. Unique
contracts remain in their current owners. The latest round passes all 1,421
affected/sibling cases, with caught mutations and restored passing controls.
Workspace typecheck and build pass. The previous round's completed owner and
chat proof is recorded separately below. This is a reviewed subset of the
repository, not an exhaustive review or a complete default-suite run.
## PR delivery verification
The cleanup is published in [PR #15978](https://github.com/paperclipai/paperclip/pull/15978).
The three cleanup commits rebased onto `914b70572c` without changing their
patches. At code revision `454e3a91ff`, fresh `pnpm -r typecheck` and
`pnpm build` pass.
The local `pnpm test:run` started at that code revision. During its first
general-server invocation, one unchanged tool-access case timed out at
15,035ms: `rejects unavailable Workspace gmail.read (other profile advertised:
true)`. The invocation was stopped after the reported failure and exited 130.
It did not complete the 882 general-server files or reach the remaining
workspace and 158 serialized suites. **There is no full local default-suite
pass.** The timing discrepancy remains unresolved; earlier passes alone do
not establish its cause.
Commit `37eb57418b` only corrects a coverage-guide link to the retained HTTP
route suite. It arrived during the local default invocation, so that run is
not an immutable run of the later head, although its executable sources are
unchanged. The complete tool-access owner then runs at immutable
`37eb57418b` and passes all **419 cases**, including the timed-out matrix row,
with its existing assertions, fixtures and timeout. UI token gates also pass.
No test was skipped, removed or given a longer budget in response to the failure.
At `37eb57418b`, all remote CI and security checks pass, Greptile completes a
fresh **5/5** review, and there are zero unresolved review threads. The review
fix and this delivery record change Markdown only. Check the PR for the final
head and its required CI/review results; the local failure remains disclosed
in the PR verification section.
## Second expansion
Baseline: `8538e024d7`. Refreshed `origin/master` remains at `fb21a3f30a`.
Three auditors nominated owner batches; original-source and proposal SHA-256
guards were checked before cutover. This round changes tests, one test-gate
file selector, documentation, and scoped test ownership guidance. It changes
zero application source lines.
| Owner batch | Net test/support lines removed | Net registered cases removed |
| --- | ---: | ---: |
| Codex monolith retirement and test-gate selector | 3,780 | 64 |
| Sandbox/SSH execution target fixtures | 976 | 0 |
| Tool request/audit and heartbeat recovery consolidation | 367 | 4 |
| Issue workspace inheritance duplicates | 175 | 2 |
| **New round** | **5,298** | **70** |
| Earlier rounds | 6,678 | 1,003 |
| **Combined** | **11,976** | **1,073** |
These figures exclude Markdown. The Codex row contains 3,779 test lines and
one removed prerequisite-selector line. The earlier 41 application lines
were the unused WorkspaceRuntimeControls compatibility path; no additional
application code was removed in this round. No measured speedup is claimed.
### New retired proof and keepers
- Codex: `06f8773097` (#12371) split the driver tests but retained the
4,153-line monolith. Its 70 declarations have 56 exact keepers, four
evolved stronger keepers, and ten unique migrations. All parameter rows
survive. Move tool refresh to recovery; continuation/effort/controls/goals
to context-control; direct-chat tools and unknown outcomes to
semantic-results. Preserve exact host-HOME equality as well as descendant
denial, the -32004 goal-policy denial and exact availability event, and
stronger diff/path and rejected/missing-result contracts. The monolith's
76 registered cases become 12 migrated rows; the nine-file baseline has
214 cases and the eight retained owners have 150.
- Test-gate routing: remove only SH-1's stale monolith filename. Its
existing lifecycle keeper still proves additive instructions. Required
patterns, source-digest receipts, other prerequisites and grading stay.
Update four docs to reachable suite filters/owners. The runner package
has its own Vitest config and is not in the root project list.
- Adapter fixtures: consolidate fresh roots/workspaces, repeated sandbox
targets, bridge defaults and SSH targets. Expand the actual new helpers
with their actual arguments/defaults and compare the entire executable
AST to the committed originals. Every case, all 523 expectation statements,
custom runner, body limit, timeout, callback, await and cleanup remains.
Helpers allocate fresh objects/runners; all 41 converted bridge calls
resolve target.remoteCwd to the original local remoteCwd expression.
- Tool requests: combine two audit-only cases with their exact same
behavioral request/actor/policy owners. Move both complete activity and
dedicated-audit assertion blocks. Preserve the missing-ID HTTP 404 in
the existing live lifecycle. Repair the misleading non-test-origin
negative, whose random missing ID never reached the origin guard, using
the real persisted agent-owned request in the connected-MCP gateway case.
Its company/connection/agent/run/issue preconditions reach the intended guard.
- Heartbeat: combine the weaker no-state accepted-plan recovery case with
the dropped-accept-time-wake keeper. Carry every full-context/retry
assertion. Consolidate only identical DB seed rows in five retained
siblings; retain historical-run, answered-question, board authority and
post-resolution timing scenarios.
- Issues: remove the second exact inheritance and explicit-override
declarations. Both groups initialize the same real DB/service/schema and
per-case experimental inputs. Keep every unique generic-project,
cross-project, source-inheritance, update and runtime-config case.
### Second expansion preservation and validation
Independent reviews cover all four boundary groups. They found no remaining
preservation blockers. Codex review independently compares all 70 mappings:
66 callback/table/budget ASTs are identical under the one workspace alias,
including the ten migrations; four evolved keepers preserve/strengthen the
original contracts. Eight shared helpers match; the fake transport's existing
optional providerIdentity is absent in these cases. The existing disposable,
canonical workspace preserves cwd identity and containment tests.
Adapter helper expansion, issue full-file comparison after the two exact
deletions, server audit-block comparison and recovery helper expansion retain
all surrounding executable statements. No generated expectation, fake owner
receipt, broad skip or longer timeout replaces behavioral proof.
- Baseline: **1,459 cases pass** across all nominated complete files and the
stock-harness gate test. The initial sandboxed attempt hit localhost
`listen EPERM` and consequent skips; unrestricted fixture runs pass every case.
- Candidate: **1,211 server/adapter cases, 182 Codex/transport/console cases,
and 28 stock-harness cases pass**, zero failed/skipped cases. The 182 include
all 150 retained Codex/diff owners plus 32 transport/console siblings.
- Mutation: bypass only getTestCallStatus's origin clause. The real persisted
request resolves and the new rejection assertion fails. Lookup/company/
connection preconditions remain valid. Restore source byte for byte; control
passes.
- Mutation: misclassify Codex goal policy denial as unsupported. The retained
real availability-event assertion fails. Restore source byte for byte;
control passes.
- Workspace `pnpm -r typecheck`, `pnpm build`, runner E2E-support typecheck,
and `git diff --check` pass. No UI source changed in this round.
- The full default repository suite remains unverified; this is complete
affected-owner/sibling proof. A full default run is still required before
claiming a full local pass. These commits were local at the end of that
round; later PR verification is recorded above.
### Retained second-round nominees
Keep native-session-runtime, runnerd transport, ACPX translation and browser
model suites: they exercise distinct lifecycle, authentication, protocol or
separate implementation owners. Keep sandbox queue versus real HTTP2 layers,
credential/stream/order guards, and unrelated tool/company/security cases.
No additional full-layer retirement was justified in those boundaries.
### Codex declaration ledger
Each exact case name identifies its protected contract; callback/table preservation was checked against the committed baseline. D deletes redundant old proof; C moves or consolidates it.
| Old line / exact case | Mark | Keeper | Evidence |
| --- | --- | --- | --- |
| 330: refreshes authorized tools on resume without replacing the provider thread | C move | `codex-app-server-driver.recovery.test.ts:1305` | identical body/table under cwd alias |
| 433: parses a complete Codex turn diff snapshot with bounded file statistics | D | `codex-turn-diff.test.ts:17` | identical body/table under cwd alias |
| 466: bounds aggregate Codex diffs and rejects unsafe workspace paths | C stronger keeper | `codex-turn-diff.test.ts:50` | explicit superset/migration above |
| 496: persists process ownership before sending provider requests | D | `codex-app-server-driver.lifecycle.test.ts:265` | identical body/table under cwd alias |
| 533: sends direct chat as plain text and permits a follow-up turn | D | `codex-app-server-driver.lifecycle.test.ts:417` | identical body/table under cwd alias |
| 579: forwards the persisted native model to the runner transport | D | `codex-app-server-driver.lifecycle.test.ts:463` | identical body/table under cwd alias |
| 600: adds Paperclip developer instructions without replacing the Codex base and enables selected skills | D | `codex-app-server-driver.lifecycle.test.ts:486` | identical body/table under cwd alias |
| 630: passes the common typed-event contract and reports one provider turn terminal | D | `codex-app-server-driver.lifecycle.test.ts:574` | identical body/table under cwd alias |
| 845: admits a strictly bound semantic result from the durable runner | D | `codex-app-server-driver.events.test.ts:47` | identical body/table under cwd alias |
| 879: correlates rehydrated notifications with the final canonical PRP event ids | D | `codex-app-server-driver.events.test.ts:81` | identical body/table under cwd alias |
| 935: maps canonical workspace snapshots with monotonic revisions and one terminal diff | D | `codex-app-server-driver.events.test.ts:137` | identical body/table under cwd alias |
| 1052: finalizes an authoritative empty workspace snapshot when a turn is interrupted | D | `codex-app-server-driver.events.test.ts:254` | identical body/table under cwd alias |
| 1095: accepts a thread usage snapshot replayed before a resumed turn starts | D | `codex-app-server-driver.events.test.ts:319` | identical body/table under cwd alias |
| 1129: normalizes provider message and reasoning phases onto every streamed item event | D | `codex-app-server-driver.events.test.ts:353` | identical body/table under cwd alias |
| 1264: captures an exact skillless model/environment snapshot with credentials absent | D | `codex-app-server-driver.context-control.test.ts:82` | identical body/table under cwd alias |
| 1356: pins approval policy %s for both thread start and resume | D | `codex-app-server-driver.context-control.test.ts:174` | identical body/table under cwd alias |
| 1383: requires trusted continuation metadata before omitting task context (%s) | C move | `codex-app-server-driver.context-control.test.ts:692` | identical body/table under cwd alias |
| 1402: allows eval fixtures to opt out of Codex collaboration instructions | D | `codex-app-server-driver.context-control.test.ts:201` | identical body/table under cwd alias |
| 1423: negotiates genuine plan mode with collaboration instructions and read-only workspace access | D | `codex-app-server-driver.context-control.test.ts:222` | identical body/table under cwd alias |
| 1466: sends the selected reasoning effort with the Codex turn | C move | `codex-app-server-driver.context-control.test.ts:711` | identical body/table under cwd alias |
| 1478: fails closed when the installed app-server does not confirm plan mode | D | `codex-app-server-driver.context-control.test.ts:265` | identical body/table under cwd alias |
| 1492: refuses to turn host credential roots into model-writable workspaces | C stronger keeper | `codex-app-server-driver.context-control.test.ts:279` | explicit superset/migration above |
| 1528: uses negotiated ACP controls through the Codex transport facade | C move | `codex-app-server-driver.context-control.test.ts:723` | identical body/table under cwd alias |
| 1549: keeps native follow-up disabled for the Codex app-server driver | C move | `codex-app-server-driver.context-control.test.ts:744` | identical body/table under cwd alias |
| 1560: steers and interrupts an active turn without replacing the session | D | `codex-app-server-driver.context-control.test.ts:315` | identical body/table under cwd alias |
| 1590: loads the deterministic Live console wire fixture | D | `codex-app-server-driver.context-control.test.ts:345` | identical body/table under cwd alias |
| 1612: holds browser-resolved upstream requests and returns the exact fixture responses | D | `codex-app-server-driver.context-control.test.ts:367` | identical body/table under cwd alias |
| 1673: acknowledges same-turn steering and rejects stale or child steering | D | `codex-app-server-driver.context-control.test.ts:428` | identical body/table under cwd alias |
| 1792: queues an interrupt before turn identity and reports a terminal race precisely | D | `codex-app-server-driver.context-control.test.ts:547` | identical body/table under cwd alias |
| 1827: maps all goal operations and advertises an exact unsupported state | C stronger keeper | `codex-app-server-driver.context-control.test.ts:582` | explicit superset/migration above |
| 1907: keeps goal support advertised when the probe fails transiently | D | `codex-app-server-driver.context-control.test.ts:662` | identical body/table under cwd alias |
| 1937: preserves an inactive goal status while editing its objective | C move | `codex-app-server-driver.context-control.test.ts:755` | identical body/table under cwd alias |
| 1964: rolls back only a definitely rejected idle goal autostart | C move | `codex-app-server-driver.context-control.test.ts:782` | identical body/table under cwd alias |
| 2014: preserves a provider-neutral goal action subset through the Codex transport facade | C move | `codex-app-server-driver.context-control.test.ts:832` | identical body/table under cwd alias |
| 2059: validates runtime request resolutions against the kind of request they answer | D | `codex-app-server-driver.runtime-requests.test.ts:47` | identical body/table under cwd alias |
| 2176: replays the DOT-185 requestUserInput shape as one canonical three-question request | D | `codex-app-server-driver.runtime-requests.test.ts:164` | identical body/table under cwd alias |
| 2282: rejects an explicit malformed Codex form without falling back to v1 | D | `codex-app-server-driver.runtime-requests.test.ts:317` | identical body/table under cwd alias |
| 2324: emits one canonical outcome payload for every terminal runtime request fact | D | `codex-app-server-driver.runtime-requests.test.ts:359` | identical body/table under cwd alias |
| 2410: expires a canonical input with its full question set when the provider transport is lost | D | `codex-app-server-driver.runtime-requests.test.ts:445` | identical body/table under cwd alias |
| 2469: hands a live canonical input off exactly once when the durable window expires | D | `codex-app-server-driver.runtime-requests.test.ts:504` | identical body/table under cwd alias |
| 2551: exposes and dispatches only explicit chat tools across fresh and resumed direct chat | C move | `codex-app-server-driver.semantic-results.test.ts:866` | identical body/table under cwd alias |
| 2683: lets an answer claimed before expiry win the terminal-event race | D | `codex-app-server-driver.runtime-requests.test.ts:644` | identical body/table under cwd alias |
| 2758: redacts browser-visible request details and diagnostics from the fixture markers | D | `codex-app-server-driver.runtime-requests.test.ts:719` | identical body/table under cwd alias |
| 2796: makes duplicate semantic completion idempotent and rejects changed payloads | D | `codex-app-server-driver.semantic-results.test.ts:151` | identical body/table under cwd alias |
| 2873: accepts a normalized runnerd echo of a tool-committed semantic result | D | `codex-app-server-driver.semantic-results.test.ts:226` | identical body/table under cwd alias |
| 2928: advertises and dispatches run-authorized control-plane tools | D | `codex-app-server-driver.semantic-results.test.ts:288` | identical body/table under cwd alias |
| 2988: preserves the dynamic tool outcome boundary (%s) | C move | `codex-app-server-driver.semantic-results.test.ts:998` | identical body/table under cwd alias |
| 3011: fails closed when an agent message changes a tool-committed result | D | `codex-app-server-driver.semantic-results.test.ts:348` | identical body/table under cwd alias |
| 3058: rejects a tool result that changes an agent-message commitment | D | `codex-app-server-driver.semantic-results.test.ts:450` | identical body/table under cwd alias |
| 3109: treats a canonically identical cross-channel result as a no-op | D | `codex-app-server-driver.semantic-results.test.ts:501` | identical body/table under cwd alias |
| 3153: fails closed when a live terminal embeds a changed semantic result | D | `codex-app-server-driver.semantic-results.test.ts:545` | identical body/table under cwd alias |
| 3205: rejects a semantic result for $label without committing it | D | `codex-app-server-driver.semantic-results.test.ts:597` | identical body/table under cwd alias |
| 3244: rejects pre-turn, cross-thread, and post-terminal notifications | D | `codex-app-server-driver.semantic-results.test.ts:636` | identical body/table under cwd alias |
| 3328: rejects duplicate and conflicting terminal facts | D | `codex-app-server-driver.semantic-results.test.ts:720` | identical body/table under cwd alias |
| 3362: rejects oversized semantic results without retaining provider payloads | D | `codex-app-server-driver.semantic-results.test.ts:754` | identical body/table under cwd alias |
| 3404: normalizes finish and block tools into one canonical result contract | D | `codex-app-server-driver.semantic-results.test.ts:796` | identical body/table under cwd alias |
| 3475: resumes and reconciles the exact provider thread after transport loss | D | `codex-app-server-driver.recovery.test.ts:297` | identical body/table under cwd alias |
| 3507: ignores an old terminal turn when the persisted active turn is still running | D | `codex-app-server-driver.recovery.test.ts:365` | identical body/table under cwd alias |
| 3548: preserves proposal idempotency and rejects a changed result after transport loss | D | `codex-app-server-driver.recovery.test.ts:406` | identical body/table under cwd alias |
| 3632: rejects changed terminal result content discovered during recovery | D | `codex-app-server-driver.recovery.test.ts:490` | identical body/table under cwd alias |
| 3703: does not re-emit an already observed terminal after recovery | D | `codex-app-server-driver.recovery.test.ts:561` | identical body/table under cwd alias |
| 3758: fails recovery explicitly when the persisted active turn is missing or replaced | D | `codex-app-server-driver.recovery.test.ts:1186` | identical body/table under cwd alias |
| 3799: reconciles a turn completed during transport loss without replacing either session identity | D | `codex-app-server-driver.recovery.test.ts:1227` | identical body/table under cwd alias |
| 3877: degrades unsupported operations with explicit redacted diagnostics | D | `codex-app-server-driver.conformance.test.ts:47` | identical body/table under cwd alias |
| 3912: rejects proposals that do not match the exact task envelope | D | `codex-app-server-driver.conformance.test.ts:82` | identical body/table under cwd alias |
| 3947: keeps a completed runtime successful for an accepted %s advisory proposal | D | `codex-app-server-driver.conformance.test.ts:117` | identical body/table under cwd alias |
| 4001: records rejected and missing proposals as recovery evidence instead of review status | C stronger keeper | `codex-app-server-driver.conformance.test.ts:171` | explicit superset/migration above |
| 4033: preserves four independent stable identities under controller ownership | D | `codex-app-server-driver.conformance.test.ts:210` | identical body/table under cwd alias |
| 4054: validates persisted identity, uniqueness, terminals, and line bounds before replay | D | `codex-app-server-driver.conformance.test.ts:231` | identical body/table under cwd alias |
| 4103: degrades declared unsupported capabilities without Codex-specific core branches | D | `codex-app-server-driver.conformance.test.ts:280` | identical body/table under cwd alias |
## Aggressive follow-up
The follow-up baseline is `3951d69f9b`, which contains the small first pass.
`origin/master` was refreshed and remained at `fb21a3f30a`. Three read-only
auditors examined adapter lifecycle, native execution, and UI ownership.
Proposals were prepared outside the checkout while baseline tests ran, then
applied with original-source SHA-256 guards.
| Owner batch | Net test/support lines removed | Production lines removed | Net registered cases removed |
| --- | ---: | ---: | ---: |
| ACPX characterization layer | 3,283 | 0 | 44 |
| Chat database read fixtures | 1,612 | 0 | 0 |
| Native executor, resume, and runner API | 802 | 0 | 943 |
| UI fixture consolidation and obsolete compatibility | 698 | 41 | 7 |
| Exact workspace/Daytona duplicate cases | 137 | 0 | 2 |
| Initial pass | 146 | 0 | 7 |
| **Total** | **6,678** | **41** | **1,003** |
These figures exclude documentation. Savings are measured from the Git diff,
not formatting estimates. The case counts include migrated/table-expanded
keepers, not just deleted declarations. No wall-time speedup is attributed to
the change: baseline and candidate runs had different cache and machine load.
### Retired layers and their keepers
- ACPX: remove startup, turn, settlement, and composed-run characterization
suites (3,643 gross lines). They copied the engine harness for #11461's
refactor; #11576 completed the coordinator extraction. The current
`execute.test.ts`, fault matrix, sequence, cancellation, session-store, and
workspace teardown suites own their remaining proof. Unique Stop-phase
regressions from #14990 remain, with actual callback/gate observations.
Preserve bridge credential isolation, current prompt/turn inputs, fingerprint
dimensions, retry order, close/cancel policy, bridge-stop before sync-back,
lease release, and native restore/last-finalize assertions in those keepers.
- Chat: replace 607 repeated real database query chains with 22 typed,
file-local query owners. Every table, column, parameter, caller await,
ordering, limit, polling loop, action, and assertion remains. Independent
expansion of the helpers produces a normalized AST identical to the entire
original program. All 1,237 integration cases execute before and after.
- Native: consolidate repeated scoped-runtime fixtures while retaining every
executor declaration and parameter row, including 1,010 complete assertion
chains. Replace six copied historical catalog constructors with independently
checked immutable digests in the existing resume table. Restore the valid
context before testing changed agent identity so that the negative cannot
pass through an earlier, unrelated instruction mismatch.
- Runner API: remove the second identical REST-route matrix. Baseline
collection executes 943 rows twice; the keeper executes each same row once.
The full file changes from 1,994 to 1,051 passing cases. No catalog entries
or public route/actor/response contract are removed.
- UI: remove five exact duplicate cases and two tests of an internal legacy
workspace API with no production callers or published source export. Both
shipped workspace pages use the retained `sections` interface. Consolidate
Secrets rendering, fresh Daytona capabilities, and four literal runtime
status rows without changing their actions, waits, or outcomes. Two AWS
error negatives now actually receive the sensitive provider details they
assert are absent.
- Workspace/Daytona: remove a second byte-equivalent worktree dependency case
and an overlapping-execute case whose body duplicated the retained lease
tombstone test. All other provisioning and teardown guards remain.
### Preservation review and final proof
Independent reviewers checked each of the four large owner proposals against
its original tests and production callers. Review caught and repaired lost
existing-file overwrite proof and a prompt guard that could accept
`String(undefined)`. Both fingerprints in the dimension matrix and the exact
session-key keeper now also require nonempty strings. No remaining preservation
blocker was found.
Three additional production mutations were caught at assertions: dropping the
runtime prompt, skipping existing-file writes during native restore, and
spreading unsafe secret-provider details into the UI. Every production file was
restored byte for byte. The two AWS negative cases fail on the unsafe-details
mutation. All four focused controls pass after restoration. The earlier HTTP
mutation proof remains below.
- Complete affected owners and siblings: **22 files, 2,817 cases pass**.
- Complete chat integration: **1,237 cases pass**, both baseline and candidate.
- ACPX baseline: 335 cases pass; native/UI baseline: 939 pass; runner API
baseline: 1,994 pass. Additional workspace/Daytona/UI baseline records 508
passes and 10 pending cases with a failed workspace file but no failed case.
That baseline inconsistency is not claimed repaired; the complete retained
workspace file subsequently passes all 175 cases with built prerequisites.
- `pnpm -r typecheck`, `pnpm build`, token gates, and diff whitespace checks pass.
- The full default repository suite remains unverified. The completed tests
above are full files at the affected owners; they are not a claim that all
repository tests ran. A full default run remains required for a PR-ready
handoff.
## Initial pass evidence
### Scope and findings
This is a repository inventory and a small evidence-backed cleanup, not a review
of every test. The baseline has 2,816 tracked `*.test.*` / `*.spec.*` files:
1,038 server, 725 UI, 697 packages, 216 test harnesses, 66 CLI, 50 scripts,
21 GitHub tooling, and 3 skill tooling files. These include opt-in suites;
the count does not describe the default Vitest run alone.
A text scan found 382 files that combine file reads with containment/matching
assertions: 156 packages, 133 server, 42 harnesses, 18 UI, 13 CLI, 13 scripts,
6 GitHub tooling, and 1 skill tooling file. The scan also matches fixture reads
and executable checks. **382 is a review queue, not a deletion estimate.**
The useful question is which independent regression a test catches that the
retained owner proof misses. File reads, snapshots, mocks, and slow tests are
not sufficient reasons to remove a test.
### Applied decisions
| Candidate | Decision and retained proof |
| --- | --- |
| `server/src/__tests__/issue-interactions-read-only-contract.test.ts`: route source check | Remove the file after adding terminal-expiry and activity-write assertions to the existing real HTTP read-path test in `issue-thread-interaction-routes.test.ts`. Check both `in_review` and `done`, the JSON response, listing, and absence of recovery. |
| Same file: two whitespace rows, three forbidden-name rows, two missing-boundary rows | Remove all seven synthetic cases. They test a local regex/assertion helper against hand-written route strings, not production behavior. Removing the helper removes their purpose. |
| Retry cases in `ui/src/api/chatEndpoints.test.ts` and `agents.test.ts` | Consolidate under `agents.test.ts`, the production API owner. Preserve the exact company-scoped request, ordinary and durable receipts, distinct stopped-successor errors, deferred dispatch, legacy receipts, both skipped-wake forms, and single-dispatch checks. |
| Slack manifest replay in `ui/src/pages/apps/chat/chat-ui-contract.test.ts` | Remove duplicate scope/event assertions already pinned by the shared owner snapshots. Move the unique agent-view, bot presence, and app-home assertions to the existing shared manifest test. Keep the UI's manifest integration checks. |
| Same UI file: sanitizer implementation substrings | Remove the checks for `replaceAll` and `encodeURIComponent`. Keep the UI integration assertions and `chat-setup-error.test.ts`, which exercises raw, escaped, encoded, and malformed UTF-16 secrets. |
| Same UI file: v8 wireframe generator/import strings | Remove the design-plan implementation checks. Keep the shipped GitHub setup assertions, wizard behavior suite, and PEM file/late-read tests. Design generator version names are not a product contract. |
The source-only route test originated in #10414; its synthetic parser cases
arrived in #13038, alongside the real HTTP test. The UI retry coverage came from
#13038 and #13765. The current chat source-check owner was last changed in
#15416. None of the removed support code has a production caller.
Change size before this report: **146 net test lines removed**, one test file
retired, seven fewer executed cases across the affected route suites, and zero
production lines changed. No runtime speedup is claimed: cold transforms,
prerequisite builds, and a baseline timeout prevent a fair wall-time comparison.
### Preservation evidence
- UI baseline: 48 cases pass across the two API files, chat UI source contract,
chooser routing, and setup-error suite. The shared manifest has four cases.
- After cleanup: those six UI/shared suites pass, 52 cases total.
- Route baseline: the eight source-check cases pass; 84 of 85 HTTP cases pass.
The unchanged standard-work plan-confirmation case times out at 15 seconds
during the full file run. Isolating it with the original read-path case passes
both. This does not establish a flake repair; its test and timeout remain intact.
- Three temporary mutations add historical expiry, terminal expiry, or an
activity write to the production GET handler. Each makes the retained HTTP
test fail at its side-effect assertion. Production source is restored byte
for byte after every mutation, including failure cleanup.
- UI token gates and `git diff --check` pass.
- Final complete HTTP route file: all 86 cases pass. Together with the 52
UI/shared cases, focused validation passes 138 cases.
- `pnpm -r typecheck` and `pnpm build` pass.
- `pnpm test:run` was started, then interrupted after about five minutes while
still running its first general-server file, the 75,178-line chat integration
suite. It exited 130; there is no full-suite pass or failure conclusion.
The default runner has 158 separately serialized suites after its general
lanes. A complete run remains a prerequisite for claiming a PR-ready handoff.
### Next review boundaries
1. Remaining `chat-ui-contract.test.ts` assertions: exact retry call syntax,
tabs, lifecycle handlers, and provider fields. Replace them only after
rendered tests exercise the selected run, navigation, error, or credential
outcome. Provider guidance and permission bytes need independent proof.
2. Component-only helper exports: `shouldRenderComposerHandoffPreview`,
`replaceResolvedInteraction`, and model-transition helpers. Their direct
tests currently protect distinct behavior. The existing rendered suites do
not cover every assertion, so deletion is not yet justified.
3. Server source guards: compare each with its route/service owner before
pruning. Authentication ordering and company boundaries deserve particular
care; a source guard can protect routes missing a behavioral negative case.
Retain protocol-schema cross-checks, package/build metadata, migration replay,
dependency singleton checks, provider permission snapshots, and theme contrast
tests. They compare independent contracts or outcomes. The telemetry README
guard is also retained; changing that path requires its own strict review.
For another batch, pin a fresh baseline, assign one production boundary,
record every deletion's keeper first, and prove the keeper catches the relevant
mutation. Do not optimize for test counts or remove baseline failures for green.
+63 -21
View File
@@ -135,12 +135,13 @@ opens the standard setup in Apps. Completing or cancelling setup returns you to
the importer with your draft retained; the repository list refreshes automatically.
Use the refresh button beside the repository count to reload accessible repositories.
The searchable tree starts with every discovered skill package checked, including
hidden and deeply nested directories. Each package shows its name and file count;
expand it to see the files included with it. Folder checkboxes select descendant
packages. A package checkbox selects that whole directory, stopping at nested
`SKILL.md` boundaries; nested skills remain independently selectable. Supporting
files cannot be deselected individually.
New imports keep repository files together. **View files** opens the included
repository inventory. The skill tree stays open, with every discovered skill
selected. Containing folders start expanded; individual package files start
collapsed. Folder checkboxes select descendant skills. Uncheck skills that should
not appear in the library; their files still accompany the selected skills as
repository support content. Supporting files cannot be deselected individually.
Existing sources retain their saved import mode.
Use the inspection button or click an included file to open a read-only preview.
The inspector shows the repository path, scanned commit, byte sizes, executable
@@ -150,22 +151,63 @@ download. Preview reads use the current caller's GitHub access and re-audit the
package at that exact commit. Preview failures do not alter selection or installed
content. Existing sources gain inspection metadata on their next successful refresh.
**Check references** flags detectable Markdown links and inline-code relative
resource paths that are missing or outside the package, including files belonging to
nested skills. This is a best-effort check, not a complete dependency analysis;
references do not cause external files to be imported automatically. Fix the source
or leave the package unchecked if it needs those files. Declared requirements are
shown separately from these warnings and from content-audit notices.
Shared files present in the repository are included automatically. Missing
references, omitted files, and setup problems appear in one expandable warnings
list. The same warnings remain on the saved source after import and refresh.
Reference detection is a best-effort check, not a complete dependency analysis.
Declared runtime requirements appear in the file inspector.
Validation errors appear beside affected skills; eligible selections import and
skipped skills are reported. These skills become available in the current company's
library and agent skill picker. Files are limited to 1 MiB each and a scan to 100 MiB
of expanded skill content, counting repeated file copies. A scan supports up to
1,000 skill packages and 10,000 package files. The repository path index includes
directories and is limited to 100,000 paths, with paths up to 4,096 characters and
64 levels deep. Discovery retains audited manifests without retaining every
package's contents. Importing never runs scripts, hooks, dependency installation,
or builds.
Warnings do not require a repair choice or confirmation. Import saves available
content and registers usable selected skills. A malformed entrypoint stays
selected with a **Needs setup** warning; its supported original files remain
repository content. Unsupported links, oversized files, and files rejected by
execution or secret-disclosure checks are omitted with path-specific warnings.
Importing never runs scripts, hooks, dependency installation, or builds.
Scans support up to 1,000 skill packages and 10,000 package files, counting repeated
file copies, with a 100 MiB scan budget. The repository path index supports up to
100,000 paths, each at most 4,096 characters and 64 levels deep. Discovery returns
manifests without retaining every package's content.
### Keep repository files together
This is the default for new imports, including skills that share agents, scripts,
policies, or sibling skills.
Paperclip saves supported repository files at the scanned commit. The skill checkboxes
control which entrypoints appear in the company library; unselected helper skills
remain available as supporting files. Add the imported skills to an agent to use
that repository in its runtime. **View files** shows exactly what
will be copied, including root license and credit files.
Repository packages support regular files up to 1 MiB each, 9,990 files, and
61 MiB of content. Omitted files produce warnings; they do not block the remaining
repository. Source-access or download failures retain
all previously installed versions. Changes to shared files create a new version
for each selected entrypoint; pinned versions retain their original repository.
Authors can optionally add `paperclip.skills.json` at the repository root:
```json
{
"version": 1,
"skills": ["skills/review/SKILL.md", "skills/research/SKILL.md"],
"requirements": "Requires Python 3.11, Git, and GitHub access for code reviews."
}
```
The listed paths must identify existing skill entrypoints. They do not change the
all-selected import default, remove other files, or prevent users from selecting
other entrypoints. The manifest only supplies
declarative metadata: it never runs setup commands. Dependencies, credentials,
subagent tools, and scheduled jobs still need to be configured in the agent's environment.
At runtime, each selected skill contains a generated discovery `SKILL.md` that
points to the untouched original in `.paperclip-repository/`. Relative references
resolve from that original directory. Original file bytes and executable flags
are preserved; discovery policy in `agents/openai.yaml` is also copied beside the
entrypoint. Existing adapters copy this self-contained package to their runtime.
Multiple enabled entrypoints currently repeat the repository files in the runtime,
while the company database stores each immutable repository snapshot once.
Repeated scans resolve the branch's current commit and reuse the caller's cached
snapshot when it is unchanged. Each server process allows one active scan per
+7
View File
@@ -0,0 +1,7 @@
# GitHub repository import UI
Production Skills UI in Storybook, captured on October 12, 2026. API responses are fixtures; these images do not establish a live GitHub import or sandbox delivery.
Desktop: 1280 × 900. Mobile: 375 × 812.
Reproduce with the repository-skill-packages Playwright configuration against a running Storybook. The tests cover the always-visible skill list, all-selected defaults, repository-file preview, warnings without import blocking, warning persistence, missing references, setup problems, and the mobile footer. There is no packaging-mode menu.
Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 99 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

+10 -4
View File
@@ -9,7 +9,7 @@ Product behavior is still governed by [the implementation spec](../doc/SPEC-impl
The map is organized by user capability, not by source file. It covers **35 feature
families** across the product, CLI, and operator workflows, checked against source
on 2026-10-05. Experimental and developer-only surfaces are labeled explicitly.
on 2026-10-12. Experimental and developer-only surfaces are labeled explicitly.
Each recipe includes sub-features, current entry points, automated evidence,
manual verification steps, and gotchas.
@@ -113,7 +113,10 @@ fixture integration. Record executed results separately from the map.
persisted state or continuation. Use read-only API checks to corroborate UI
evidence; creating state through the API does not prove the creation UI.
For agent-driven acceptance work, the existing
For agent-driven acceptance work, start with the
[Paperclip verification skill](../.agents/skills/verify-paperclip/SKILL.md) for an
isolated launch, ownership/readiness checks, browser driving, evidence, and cleanup.
It uses this directory as its feature map. The existing
[dev-workspace run/verify skill](../.agents/skills/paperclip-dev-workspace-run-verify-fix/SKILL.md)
and [evaluation skill](../.agents/skills/paperclip-evals/SKILL.md) describe runtime
ownership and evidence handling. Reuse them; this map introduces no environment
@@ -168,10 +171,13 @@ Each recipe starts with an H1 and a user-visible description, then exactly:
1. **Sub-features** — stable backticked IDs and observable behavior/states.
2. **How to get to it (user POV)** — one H3 backticked entry-point ID per surface.
3. **Driving it** — starts with `Preconditions:`; repeat each entry-point H3,
3. **Driving it with <harness>** — starts with `Preconditions:`; repeat each entry-point H3,
with `Automated:` and `Manual:` paragraphs. Name test scope and gaps honestly.
4. **Gotchas** — misleading look-alikes, feature gates, and invalid evidence.
The format is inspired by
New or updated entries follow
[Poteto's feature-map examples](../.agents/skills/create-verification-skill/references/feature-map-example/).
Existing entries with a `Driving it` heading can adopt the named harness when
their recipes are updated. The original format was inspired by
[Omnigent's feature map](https://github.com/omnigent-ai/omnigent/tree/91acfbbb59f6fc210ff95a9e9428aadd62e06582/feature-map).
Paperclip's recipes and checks follow its own task model and test infrastructure.
+8 -1
View File
@@ -21,12 +21,15 @@ Open `/auth`; first-owner bootstrap uses `/board-claim/:token` when configured.
### `invite-members`
Open `/invite/:token`; operators manage members in company settings and join requests at `/inbox/requests`.
Create/revoke human links at Members → Invites
(`/company/settings/members?tab=invites`). Human invitees receive the chosen
role after sign-in; agent invitations still use join-request approval.
### `cli-authorization`
Start the CLI auth/connect flow and approve its `/cli-auth/:id` browser page; instance access is under Settings when visible.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). For login tests use authenticated mode and two test users; local-trusted mode does not prove session enforcement. Use two companies for isolation checks.
@@ -48,6 +51,10 @@ Automated: [board auth client](../cli/src/__tests__/board-auth.test.ts) and [tok
Manual: Authorize the intended CLI context, make a read-only company request, then revoke its token and retry. Verify failure is explicit and does not silently fall back to another identity.
Use `auth login`, `auth whoami`, and `auth revoke-current` with explicit context.
Retry with the same bearer to prove revocation; deleting local credentials with
logout alone does not establish that server revocation succeeded.
## Gotchas
- Settings visibility is not authorization; direct requests still need permission checks.
+6 -1
View File
@@ -22,7 +22,7 @@ Open `/dashboard` or `/dashboard/live`.
Open `/activity`, `/activity/runs`, or `/activity/timeline` in the streamlined shell; legacy routes may redirect.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare known task mutations and completed/failed runs in a disposable company, recording their IDs and actors.
@@ -38,6 +38,11 @@ Automated: [activity service](../server/src/__tests__/activity-service.test.ts),
Manual: Find a known mutation by actor/resource and a known run by status. Change filters, load more, and follow the source links. Reload the chosen view and verify no company leakage or duplicate events. Inspect a routine run from its audit entry.
Audit-event Load more and run listings differ: Runs currently loads at most 200
entries without pagination. The tested RoutineAuditActivity component has no
registered activity-route host; inspect execution links actually offered by
the routine rather than treating that component test as a reachable journey.
## Gotchas
- Audit/activity events, run-log rows, first-party Telemetry, and operator OpenTelemetry traces are separate data paths.
+9 -2
View File
@@ -25,7 +25,7 @@ Ask for project work in Agent Chat and follow its created project/task links.
Open `/board-chat` only when its conference-room gate is enabled.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable the applicable chat feature, use a configured test agent, and distinguish model replies from a mocked conversation fixture.
@@ -33,7 +33,10 @@ Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enab
Automated: [chat landing](../ui/src/pages/AgentChats.test.tsx) checks recent-chat routing; [conversation service](../server/src/__tests__/agent-conversations.test.ts) covers persisted conversation behavior.
Manual: Start a conversation, send a recognizable request, and inspect its reply. Navigate away and reopen the same history. Test a new conversation, unavailable historical agent, and an inaccessible saved conversation; verify no cross-user or cross-company history appears.
Manual: Start a conversation, send a recognizable request, and inspect its reply.
Navigate away and reopen the same history. Use `/new` for a fresh session within
the same standing conversation. Test an unavailable historical agent and an
inaccessible saved conversation; verify no cross-user or cross-company history appears.
### `project-handoff`
@@ -45,6 +48,10 @@ Manual: Ask the configured agent to create bounded project work, inspect the pro
Automated: [board chat gate](../server/src/__tests__/board-chat-route-feature-flag.test.ts) proves route gating, not a live conversation.
Preconditions for this entry point: `local_trusted` mode and an installed,
authenticated `claude` CLI, in addition to the Conference Room flag. These are
separate from Agent Chat's configured-agent prerequisites.
Manual: Check disabled access first, enable in the disposable instance, then send and reload a conversation. Inspect the actual responding agent and persisted history; report this result separately from Agent Chat.
## Gotchas
+8 -2
View File
@@ -6,7 +6,8 @@ Implementation: [hiring](../ui/src/pages/NewAgent.tsx), [configuration](../ui/sr
## Sub-features
- `hire`: choose role, manager, adapter, model, and required account settings.
- `hire`: choose identity, adapter, model, and required account settings; edit
reporting relationships through the agent configuration afterward.
- `instructions`: edit agent configuration and instruction files with durable revisions.
- `organization`: inspect and change reporting relationships through supported controls.
- `lifecycle`: pause/resume or retire an agent while preserving history.
@@ -26,7 +27,7 @@ Open `/agents/:agentId` and the configuration/instruction surfaces it exposes.
Use the roster’s organization view; `/org` redirects to the roster in the streamlined shell.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable company and supported adapter; hiring may require a formal approval.
@@ -36,6 +37,11 @@ Automated: [new agent page](../ui/src/pages/NewAgent.test.tsx) and [configuratio
Manual: Hire an agent with a distinct name and role. Complete any required approval, reload the roster, and open its configuration. Confirm manager, adapter, and model persisted; run a small task to prove execution readiness.
The current hiring wizard assigns the first hire CEO and subsequent hires
General reporting to the CEO. Change manager/role afterward through supported
configuration. Process/HTTP agents can be created through API/CLI but are excluded
from the hiring picker; API fixture creation does not prove this UI journey.
### `agent-configuration`
Automated: [instruction service](../server/src/__tests__/agent-instructions-service.test.ts) and [instruction revisions](../server/src/__tests__/agent-instruction-revisions.test.ts) cover file/revision contracts.
+5 -2
View File
@@ -21,7 +21,7 @@ Use Activity → Costs/Budgets in the streamlined shell, `/costs` in the product
Open the governing budget configuration and any resulting budget incident card.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use synthetic low-cost events and small disposable budgets. Do not buy model usage merely to hit a threshold.
@@ -29,7 +29,10 @@ Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use
Automated: [costs service](../server/src/__tests__/costs-service.test.ts) covers cost contracts; [embedded costs UI](../ui/src/pages/Costs.test.tsx) has narrow Audit-host coverage.
Manual: Inspect a known fixture run and reconcile its recorded spend with the selected report period and resource. Switch company/project filters and reload. Check empty and unavailable cost data without treating missing data as zero spend.
Manual: Inspect a known fixture run and reconcile recorded spend with the chosen
period and resource. Switch the report period/company and inspect the **By
project** breakdown; it is not a project-selector filter. Reload and check empty
or unavailable cost data without treating missing data as zero spend.
### `budget-enforcement`
+9 -3
View File
@@ -21,7 +21,7 @@ Open `/cases` and switch between available grouped/tree views.
Open `/cases/:caseIdentifier` or a task’s linked case.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable cases and prepare related records with different types/statuses plus an associated task.
@@ -35,9 +35,15 @@ Manual: Search for a child, verify its parent context remains understandable, ch
Automated: [case routes](../server/src/__tests__/cases-routes.test.ts) covers API behavior; [case fields](../ui/src/components/CaseFieldsPanel.test.tsx) covers field UI separately.
Manual: Edit an offered field, reload, inspect its revision/activity, and follow child and task links. Verify cross-company and invalid field changes fail clearly. Distinguish a pipeline-owned item from a stand-alone case.
Manual: Change status or labels, reload, inspect activity, and follow child/task
links. Edit a case document and inspect its document revisions separately.
Arbitrary JSON fields are currently displayed without a general field editor.
Verify cross-company and invalid changes fail clearly. Distinguish a
pipeline-owned item from a stand-alone case.
## Gotchas
- A case is not a task execution; inspect linked tasks to prove work occurred.
- Feature gating and case-type schema determine which actions are available.
- Feature gating and offered controls determine available actions. The API
accepts a string case type and arbitrary JSON fields; do not infer typed
case-field validation or a field-editing UI from those values.
+9 -2
View File
@@ -22,9 +22,13 @@ Message the configured bot/address from its external provider and follow the cor
Open `/apps/chat/:endpointId/settings` and the endpoint’s other available tabs; identity confirmation uses `/chat-identity/confirm`.
## Driving it
## Driving it with Playwright and provider clients
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable chat connectors and use a test provider account/channel with permission to send messages. Setup is covered in [connection setup](./connection-setup.md).
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a
test provider account/channel with permission to send messages. Gates differ:
AgentMail bypasses the chat-connectors UI gate, GitHub uses
`enableGitHubReviewBots`, and other chat providers use `enableChatConnectors`.
Setup is covered in [connection setup](./connection-setup.md).
### `external-thread`
@@ -40,6 +44,9 @@ Manual: Inspect the bound agent and identity, recent activity, and delivery fail
## Gotchas
- Visibility flags do not pause existing delivery. Disable the disposable
endpoint itself to test stopped traffic.
- One provider’s passing fixture does not qualify another provider or live delivery.
- An accepted send request does not prove the recipient received the message or file.
- Use [steering](./steering.md) and [questions](./questions-and-approvals.md) for their channel-specific interaction checks.
+6 -1
View File
@@ -25,10 +25,15 @@ Use issue subresource, asset/skill, and run/heartbeat commands exposed by the CL
### `worktree-instance`
Use the documented worktree/test-drive commands in an owned checkout; inspect `paperclipai worktree --help` first.
Inspect root help too: `worktree:make/list/merge-history/cleanup` are top-level
commands rather than subcommands of `worktree`.
## Driving it
## Driving it with the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use `--help` from the checked-out CLI to discover exact arguments. Select a disposable company and isolated context; do not inherit an unknown API target.
Pass the server origin to `--api-base` (without appending `/api`); the commands
already include the API path. `issue create` requires explicit `--company-id`
even when a profile stores a company. A persona label is not authentication proof.
### `client-context`
+5 -1
View File
@@ -21,7 +21,7 @@ Use the company switcher or `/companies`, then enter company Settings.
Use `/company/export` and `/company/import` where enabled, or the corresponding CLI company commands.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use two disposable companies and a small package fixture. Record cloud-managed restrictions and visible settings gates.
@@ -39,6 +39,10 @@ Manual: Export a small company, inspect the package preview, and import into a d
## Gotchas
- Restore an archived company through `/companies` → its menu → **Unarchive**.
Archiving pauses eligible agents and cancels active runs; restoration handles
agents paused specifically for company archival.
- Package presence does not prove a runnable imported company; credentials and environment bindings need separate verification.
- Company deletion is a distinct gated operation; do not use it as the archive test.
- Settings hidden in a managed deployment should be recorded as unavailable, not bypassed.
+5 -1
View File
@@ -57,7 +57,7 @@ or open `/apps/chat/connect`. Complete that provider's endpoint setup, then
open its settings (`/apps/chat/:endpointId/settings`). Follow identity linking
as the intended sender before messaging the agent externally.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey).
Choose a disposable provider account/resource and record its auth method,
@@ -138,6 +138,10 @@ Slack setup does not qualify Discord, Telegram, GitHub, email, or iMessage.
## Gotchas
- Private endpoint rejection depends on deployment: owned loopback MCP servers
are supported in local-trusted/private instances. Public authenticated
deployments apply the private-network restriction.
- A successful OAuth callback is not proof of saved connection access or task
continuation. Verify all three checkpoints.
- Personal identity, shared account access, and per-agent access are distinct.
+7 -2
View File
@@ -21,7 +21,7 @@ Use `/design-guide`, registered `/ux-lab/...` routes, or Storybook; consult `ui/
Open `/tests/perf/long-thread` only in an appropriate development environment.
## Driving it
## Driving it with Playwright
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a development/test environment. Confirm the route is registered for the current build before claiming the page is available.
@@ -35,9 +35,14 @@ Manual: Open the specific fixture state, inspect controls and layout, then repea
Automated: [task thread component](../ui/src/components/IssueChatThread.test.tsx) does not establish a performance budget. Timing/scroll measurements need a dedicated browser run.
Manual: Record browser/build/data size, exercise scrolling and composing in the fixture, and retain timing or trace evidence. Confirm a representative real task does not regress before extrapolating fixture measurements.
Manual: Record browser/build/data size, exercise scrolling and rendering in the
long-thread fixture, and retain timing or trace evidence. Its composer is
disabled; test composing separately in a real task. Confirm representative
real-task behavior before extrapolating fixture measurements.
## Gotchas
- An unmounted or obsolete lab file remains in the conservative page inventory but is not a shipped entry point.
- Intended developer use is not a universal runtime gate: Design Guide and
global lab/performance routes are also registered in production builds.
- Screenshots of a lab do not prove live provider behavior, persistence, or authorization.
+12 -3
View File
@@ -21,12 +21,15 @@ Open a document from the task thread or document side panel.
### `task-output`
Open an attachment, output card, or file link from a task.
For direct uploads, use the task side panel's **Open a new tab → Artifacts →
Upload files** or drop files anywhere in that panel. On mobile, open
**More actions → Properties** first. Agent conversations do not expose this upload path.
### `artifact-library`
Open `/artifacts` and its grouping, search, and media controls.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a task with a text document, a small attachment, and a generated work product. Use nonsensitive fixture files.
@@ -38,9 +41,15 @@ Manual: Edit a paragraph, save, reload, and compare revisions. Add an annotation
### `task-output`
Automated: [attachment routes](../server/src/__tests__/issue-attachment-routes.test.ts) and [work products](../server/src/__tests__/work-products.test.ts) cover data/access behavior; [artifact arrival browser suite](../tests/e2e/artifact-tab-arrival.spec.ts) targets the arrival experience.
Automated: [attachment routes](../server/src/__tests__/issue-attachment-routes.test.ts) and [work products](../server/src/__tests__/work-products.test.ts) cover data/access behavior; [artifact arrival browser suite](../tests/e2e/artifact-tab-arrival.spec.ts) targets arrival. [Artifact uploads](../tests/e2e/artifact-uploads.spec.ts) drives file selection/drop, filename collisions, retry, and preservation of paused/unassigned tasks.
Manual: Upload a fixture and have the test task produce an output. Preview and download each, compare the contents, and follow the source task link. Reload and test an inaccessible user/company; verify missing workspace-only resources have an explicit state.
Manual: Upload a small Markdown file and CSV through Artifacts. Markdown becomes
a versioned task document; other files become attachments. Read the document,
download the CSV and compare bytes, then reload. Upload the same Markdown filename
again and verify a separate suffixed document without overwriting the original.
On a paused unassigned task, confirm upload does not start work or add comments.
Verify agent-produced work products separately using an eligible test run.
Test an inaccessible user/company and explicit missing workspace-only resources.
### `artifact-library`
+9 -1
View File
@@ -17,12 +17,16 @@ Implementation: [environment settings](../ui/src/pages/CompanyEnvironments.tsx),
### `environment-settings`
Open Settings → Environments, then create/edit a supported target.
The canonical URL is `/company/settings/instance/environments`. Enable
`enableEnvironments` first; keep `enableManagedSandboxOnly` off to inspect local
targets. The catalog is instance-scoped and mutation requires a local operator
or instance administrator.
### `remote-execution`
Choose the environment in agent/project/workspace configuration, then inspect the resulting task/run and workspace.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use an owned disposable target and a supported provider plugin. Record provider, target identity, account, and spending constraints before a live remote run.
@@ -32,6 +36,10 @@ Automated: [environment UI](../ui/src/pages/CompanyEnvironments.test.tsx), [driv
Manual: Save a disposable target, reload it, and run its available probe. Verify both success and missing-credential/reachability errors identify the target. Confirm unavailable providers are not offered as working run targets.
The local row has no UI Test button. Its API/CLI probe reports host/cwd
availability only; it does not validate an adapter binary, workspace, or task
execution. Remote-driver probes and a real bounded run need their own evidence.
### `remote-execution`
Automated: [execution target](../server/src/__tests__/environment-execution-target.test.ts) and [custom images](../server/src/__tests__/environment-custom-images-service.test.ts) cover contracts; [SSH live test](../server/src/__tests__/environment-live-ssh.test.ts) requires its own external setup.
+7 -2
View File
@@ -6,7 +6,7 @@ Implementation: [gateways](../ui/src/pages/apps/gateways/GatewaysList.tsx), [gat
## Sub-features
- `gateway`: create/edit an endpoint and select its exposed apps/tools.
- `gateway`: create/edit an endpoint backed by a tool access profile.
- `profiles`: define reusable tool access configuration and inspect draft/detail state.
- `client-connect`: copy the intended endpoint/configuration to an authorized test client.
- `tokens`: create/revoke client credentials and verify their scope.
@@ -22,10 +22,15 @@ Open `/apps/gateways`, then a gateway’s Overview, Apps/Tools, Tokens, and Acti
Open `/apps/advanced/profiles`, create a profile, or follow its detail/edit route.
## Driving it
## Driving it with Playwright and an MCP client
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a fixture tool connection and test client. Record gateway/profile, token owner, transport, and selected company.
Gateway creation selects a profile. Maintain that profile's app/action selection
through the Apps/Tools links; the gateway form is not a separate tool picker.
Use an active organization member for human-scoped access, and check the actual
endpoint authorization independently of whether the admin UI is visible.
### `gateway-client`
Automated: [gateway service](../server/src/__tests__/tool-gateway-service.test.ts), [connect client dialog](../ui/src/pages/apps/gateways/ConnectClientDialog.test.tsx), and [token panel](../ui/src/pages/apps/gateways/panels/TokensPanel.test.tsx) cover service/UI contracts.
+7 -2
View File
@@ -20,7 +20,7 @@ Open `/goals`, create a goal, and enter `/goals/:goalId`.
Inspect the goal context from a project or task and compare explicit versus inherited context.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare two goals and a disposable project/task in the selected company.
@@ -28,7 +28,12 @@ Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prep
Automated: [project-goal validation](../server/src/__tests__/project-goal-validation.test.ts) covers relationship validation; [goal detail control](../ui/src/pages/GoalDetail.test.tsx) only tests the properties toggle, not the whole creation journey.
Manual: Create a parent and child goal, save owner/status changes, and reload. Link a project and task through the offered controls, follow their goal context, and verify a cross-company or invalid relationship is rejected.
Manual: Choose **Add Goal/New Goal**, enter **Goal title**, select the level, and
create a parent. Open it and choose **Sub Goal** to create a child with the
parent preselected. Change status in Properties and reload; follow the Parent
Goal link and inspect Sub-Goals. Owner is currently display-only in this UI.
Link a project and task through the offered controls, follow their goal context,
and verify a cross-company or invalid relationship is rejected.
### `task-goal-context`
+5 -1
View File
@@ -25,7 +25,7 @@ Open `/decisions`, then a queue at `/decisions/queues/:key` and its source item.
Open `/search` or the navigation search entry.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Seed assigned, unread, blocked, and resolved tasks plus a pending decision. Record the combined Inbox/Tasks experimental setting.
@@ -49,6 +49,10 @@ Manual: Search a unique task title and content phrase, open the result, and comp
## Gotchas
- Combined Inbox/Tasks requires both `enableCombinedInboxTasks` and
`enableStreamlinedUi`. [Standalone Inbox tests](../ui/src/pages/Inbox.test.tsx)
render archive/read controls and toolbars; this is not live multi-user evidence.
- Personal archival is not task cancellation or completion.
- Legacy Inbox routes can redirect to the combined task view; record which host actually rendered.
- Search service tests do not establish the relevance quality of every query.
+6 -1
View File
@@ -22,7 +22,7 @@ Use `paperclipai install`, `run`, `service`, `doctor`, `configure`, `channels`,
Use currently visible instance settings for experiments/access and `paperclipai db:backup --help` for one-off backup. Some legacy general-settings URLs redirect.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable data directory/service identity. Do not update, restart, or restore over an unrelated live instance as a documentation test.
@@ -40,6 +40,11 @@ Manual: Change a harmless experimental setting, reload, and confirm the affected
## Gotchas
- Source-checkout update/check commands report manual update guidance; they do
not qualify a managed installation. Rollback restores retained CLI payloads,
not database migrations. Service status applies to its selected service
identity, not automatically to an owned foreground dev server.
- A successful backup write does not prove restoration works.
- Settings visibility and mutability differ for cloud-managed and self-hosted instances.
- Keep the three data paths distinct: first-party Telemetry, operator-configured OpenTelemetry, and local run logs.
+6 -1
View File
@@ -22,7 +22,7 @@ Use starred/recent/sidebar links and direct company-prefixed URLs.
Use Settings → Profile, `/u/:userSlug`, and the announcement well when a publication is available.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use two test users and companies; record the selected UI shell and any client-local preference storage.
@@ -32,6 +32,11 @@ Automated: [sidebar](../ui/src/components/Sidebar.test.tsx) and [sidebar prefere
Manual: Star and reorder supported resources, collapse a section, navigate and reload. Switch company/user and inspect the intended preference scope. Open an unavailable resource and a legacy route; verify clear fallback or correct redirect without leaking another company’s state.
Record persistence separately: section collapse is in-memory and reopens after
reload; global sidebar collapse is disabled. Streamlined starred projects sort
alphabetically. Recents are browser-local by user/company; stars and supported
ordering are saved server-side. Local-trusted mode does not prove two-user isolation.
### `profile-announcements`
Automated: [profile page](../ui/src/pages/ProfileSettings.test.tsx), [announcement hook](../ui/src/hooks/useAnnouncement.test.tsx), and [announcement routes](../server/src/__tests__/announcements-routes.test.ts) cover separate UI/persistence layers.
+18 -2
View File
@@ -17,12 +17,15 @@ Implementation: [web wizard](../ui/src/components/OnboardingWizard.tsx), [CLI se
### `web-wizard`
Open `/onboarding` or the onboarding launcher in an empty company context.
Use `/<company-prefix>/onboarding` to resume an existing company's agent setup;
the unprefixed route starts at organization naming on self-hosted instances.
Cloud-managed instances reuse their existing company for agent setup.
### `cli-setup`
Use `paperclipai onboard --help` or `paperclipai run --help` to select an isolated data directory before setup.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a new isolated data directory and record deployment mode. A configured adapter may still need account authentication.
@@ -30,7 +33,20 @@ Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use
Automated: [wizard tests](../ui/src/components/OnboardingWizard.test.tsx) cover mocked wizard behavior; [onboarding browser suite](../tests/e2e/onboarding.spec.ts) supplies its own test environment. Live model execution is a separate check.
Manual: Create a disposable company and first agent. Save and leave midway, reopen, and finish. Follow the created first task, verify the selected agent and company, and inspect its output or explicit authentication gate. Reload both company and task.
- **Create company and resume.** Start a fresh empty instance and run doctor using
the [verification skill](../.agents/skills/verify-paperclip/SKILL.md). Run
`node .agents/skills/verify-paperclip/scripts/onboarding-smoke.mjs --evidence-dir "$VERIFY_EVIDENCE_DIR"`.
The helper opens `/onboarding` in a fresh self-hosted instance, enters a unique
name in `e.g. Northwind Labs`, and chooses `Continue`. The agent-name field must
appear and the saved company must have no invented goal. Reopen the
company-prefixed onboarding route and reload; the agent step must appear
without creating another company. Screenshots and a trace capture the real UI action and result.
This proves company creation/resumption, not hiring or a first model run.
Manual: Create a disposable company and first agent. Leave midway, reopen the
company's onboarding route, and finish; drafts save automatically. Follow the
created first task, verify the selected agent and company, and inspect its output
or explicit authentication gate. Reload both company and task.
### `cli-setup`
+5 -1
View File
@@ -21,7 +21,7 @@ Open `/pipelines`, `/pipelines/:pipelineId`, and its Settings route.
Open `/pipelines/:pipelineId/items/:caseId`, `/review-queue`, or `/learnings`.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable pipelines and prepare a disposable pipeline with at least two stages and a fixture agent. Record any associated case type.
@@ -39,6 +39,10 @@ Manual: Open a pending review, inspect its source output and revision, accept or
## Gotchas
- Learnings is filtered case-event history, such as review decisions and forced
moves, not a separate authored learning-record system. The UI visibility flag
is separate from API authorization.
- These routes are experimental and may redirect or deny access when disabled.
- ReviewQueue and Learnings are hosted from the pipeline module; a page-file inventory alone misses them.
- Pipeline items and free-standing [cases](./cases.md) share records but have different navigation contexts.
+6 -1
View File
@@ -21,7 +21,7 @@ Open Settings → Plugins where permitted, or the CLI plugin commands.
Open `/plugins/:pluginId`, a registered plugin route, or its contributed company/project settings surface.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a known local fixture plugin with bounded capabilities. Record installed version and any declared external dependencies.
@@ -39,6 +39,11 @@ Manual: Open the contributed page by navigation and cold deep link, perform a ha
## Gotchas
- Installation/lifecycle is instance-level; configuration can be company-scoped.
Settings lives at `/company/settings/instance/plugins`. npm installation and
bundled-local installation are different controls. Developer file-watcher
reload is not a lifecycle action in the plugin manager.
- Installing a plugin does not authorize every host capability or external action.
- Plugin-provided routes are dynamic; this map covers the host lifecycle, not every third-party feature.
- Environment-driver plugins also need the [environment](./execution-environments.md) checks.
+8 -1
View File
@@ -21,7 +21,7 @@ Open `/projects`, create a project, and enter its overview or Issues view.
Open `/projects/:projectId/configuration` and its repository/workspace settings.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable project and a small repository you can access; record checkout and credential ownership.
@@ -39,6 +39,13 @@ Manual: Attach a small repository, save, and create a project task. Inspect the
## Gotchas
- `/overview` now selects Configuration, not a distinct overview page. Personal
join/leave and star controls have their own persistence and should be checked
when affected; **Star/Unstar {project name}** is the visible star action.
- Project name slugs change on rename and have no historical alias lookup.
Reopen a renamed project through its stable UUID route, wait for the current
Configuration form, and record the newly normalized URL before reloading.
- Project repository configuration and a provisioned execution workspace are different records.
- See [workspaces](./workspaces.md), [goals](./goals.md), and [budgets](./budgets-costs.md) for those features.
- Do not assume a deleted or archived project should erase its historical task evidence.
+1 -1
View File
@@ -148,7 +148,7 @@ Real provider and model checks follow [Task reviews](../doc/connections/TASK-REV
### `skill-test`
Automated: [read-only interaction contract](../server/src/__tests__/issue-interactions-read-only-contract.test.ts)
Automated: [read-only interaction contract](../server/src/__tests__/issue-thread-interaction-routes.test.ts)
checks that fetching interactions does not mutate them. Shared card suites cover
rendering. Neither proves the complete Skill Studio host or its permission matrix.
+8 -2
View File
@@ -20,9 +20,10 @@ Open `/routines`, create one, and inspect `/routines/:routineId` and its edit se
### `trigger-history`
Use the routine’s trigger controls and History/Audit links; lists also offer Run now.
Use the routine's trigger controls and Runs/Recent Runs/Audit links; lists also
offer Run now. **History** records configuration versions, not executions.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable routine, deterministic test agent, and harmless target. Record schedule timezone and enabled state.
@@ -32,6 +33,11 @@ Automated: [routine detail helper](../ui/src/pages/RoutineDetail.test.tsx) tests
Manual: Save an agent, request, project, and a required variable. Reload and inspect the saved definition. Run now with a distinguishable input, follow the resulting task, and verify its output and history entry. Check a missing required input.
Variables are detected from `{{name}}` placeholders; mark a detected variable
Required and verify missing input prevents Run routine. Configuration can be
checked without model credentials. A successful subprocess run alone does not
complete the generated task or establish a useful task output.
### `trigger-history`
Automated: [trigger wizard](../ui/src/components/routine-triggers/TriggerWizard.test.tsx) is component coverage; [routine integration](../server/src/__tests__/routines-e2e.test.ts) is server integration rather than a provider-wide browser proof.
+5 -1
View File
@@ -21,7 +21,7 @@ Open an agent’s model/adapter configuration or the instance adapter settings p
Open an agent run at `/agents/:agentId/runs/:runId` from the agent or task activity.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Record adapter, model, account, native/legacy runtime, and environment. Capabilities vary; one harness result is not a matrix-wide qualification.
@@ -39,6 +39,10 @@ Manual: Start a disposable task, open its running transcript, and follow it to a
## Gotchas
- Model accounts are managed in Apps and inline agent account controls.
`/company/settings/connections` configures decision/fast-response models,
not the general model-account library.
- The run log is instance database data; it is not the opt-out first-party Telemetry system or operator-configured OpenTelemetry tracing.
- Authentication/model discovery does not establish tool, image, steering, or resumption capability parity.
- Use [recovery](./recovery.md) for interrupted runs and [execution environments](./execution-environments.md) for local/SSH/sandbox differences.
+5 -1
View File
@@ -22,7 +22,7 @@ Open company Settings → Secrets and the company/personal secret views.
Use the Proposals tab or Import from vault where available.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use dummy credentials in an isolated company, two user identities, and a test agent. Never print actual secret values in evidence.
@@ -40,6 +40,10 @@ Manual: Review a dummy proposal, reject one and accept another, then inspect the
## Gotchas
- Vault import currently targets eligible AWS Secrets Manager providers.
Organization-secret checks in local-trusted mode do not establish two-user
personal-secret ownership or actual external-vault import.
- A displayed reference name is not proof that the actor can resolve its value at runtime.
- Company secrets and user-specific values have different ownership.
- Redaction needs verification in errors, activity, and run logs as well as the form.
+7 -1
View File
@@ -21,12 +21,15 @@ Open `/skills`, Discover, or `/skills/sources`; project import is available thro
### `studio`
Use New skill or `/skills/studio/:skillId`; writable copies and source-backed originals differ.
Direct authoring starts at `/skills/studio/new`: fill **Name** and **Slug**, then
**Create skill**. Authoring/save/reload requires no model credential; running a
Studio test is separate.
### `agent-enablement`
Use skill policy/agent selection in the library or an agent’s Skills tab.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a harmless skill fixture and test agent with skill support. Record whether the source is local, bundled, project, or remote.
@@ -50,6 +53,9 @@ Manual: Enable the skill for one test agent and leave another without access. Ru
## Gotchas
- The release picker requires `enableBetaSkills`. Do not invent an Instructions
accessibility label for the Studio body editor; inspect its current handles.
- Installed does not mean enabled for every agent.
- Read-only GitHub/bundled skills may require a writable copy rather than an in-place edit.
- Studio interaction coverage is in [questions and approvals](./questions-and-approvals.md); a passing mock is not a live harness test.
+5 -1
View File
@@ -21,10 +21,14 @@ Open the task’s parent/child and dependency links in task detail.
Inspect the task’s execution policy and available review, pause, resume, or stop controls.
## Driving it
## Driving it with Playwright and the Paperclip API
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Prepare a parent, child, and dependent task plus implementer/reviewer agents. Record execution policy and use a disposable tree.
Parent/dependency navigation and inherited tree holds can be checked on unassigned
tasks. Agents are required for the continuation and signoff portion. A tree pause
creates a hold; it does not change task statuses to a paused status.
### `task-relationships`
Automated: [dependency wakeups](../server/src/__tests__/issue-dependency-wakeups-routes.test.ts) checks server behavior; visual relationship navigation remains a manual check.
+11 -2
View File
@@ -26,9 +26,12 @@ Open `/issues` or a project’s Issues view; legacy `/tasks` links redirect.
Open `/issues/:issueId` from a row, mention, or direct link.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use an invokable test agent and a project; record whether a real or simulated harness handles the request.
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a
disposable project. Persistence checks can assign work to Me or leave it
unassigned; execution checks additionally need an invokable test agent. Record
whether a real or simulated harness handles execution.
### `new-task`
@@ -36,6 +39,10 @@ Automated: [creation dialog](../ui/src/components/NewIssueDialog.test.tsx) is co
Manual: Create a uniquely named task with an agent and project. Open its resulting link and reload. Confirm text, attachment, assignment, and project persisted, and verify execution started only when the task and agent are eligible.
Creation is request-first: enter **Describe a task…**, choose the assignee and
project, and submit **Create task**. A standalone title field is not always shown.
Use a human/unassigned task to verify persistence without invoking a model.
### `task-list`
Automated: [task-list helpers](../ui/src/pages/Issues.test.tsx) cover search URLs, pagination, deduplication, and presentation constants. The suite does not render the page or exercise filtering, grouping, opening a task, and returning; those interactions still need a running-browser check.
@@ -50,6 +57,8 @@ Manual: Add a comment and edit authorized properties. Reload and verify history,
## Gotchas
- Priority remains an API field, but ordinary priority UI editing is hidden.
- A status label alone does not prove that the requested work exists.
- Single-assignee and atomic checkout rules still apply when tasks are created from different hosts.
- Use [coordination](./task-coordination.md), [steering](./steering.md), and [documents](./documents-artifacts.md) for their distinct workflows.
+5 -1
View File
@@ -21,7 +21,7 @@ Use the CLI teams catalog/preview/install operations in the selected context.
The catalog components and install wizard are available in developer tests/design surfaces. No top-level `/teams` route is registered; use the CLI/API for the current installation workflow.
## Driving it
## Driving it with the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable company, a small catalog entry, and explicit company context. Check `paperclipai teams --help` for current subcommands.
@@ -39,6 +39,10 @@ Manual: When changing or mounting these components, exercise preview, manager/so
## Gotchas
- Catalog preview records an audit event but installs no agents/projects. Use
`teams browse/inspect/preview` with explicit company and origin API context;
the current product has no `/teams` route.
- Source files do not prove that a screen is reachable in the current app.
- Installed team configuration may still need credentials or environment setup.
- Company package import is a separate [portability](./companies.md) workflow.
+6 -1
View File
@@ -22,7 +22,7 @@ Use the task’s workspace card or Files panel and follow its workspace link.
Open `/workspaces`, a project workspace, or `/execution-workspaces/:workspaceId` and its Services/Configuration/Runtime logs views.
## Driving it
## Driving it with Playwright and the Paperclip CLI
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Enable isolated-workspace UI where needed. Use an owned disposable repository and environment; record workspace ID and branch.
@@ -40,6 +40,11 @@ Manual: Provision a disposable workspace, wait for readiness, start an available
## Gotchas
- UI visibility, default isolation policy, the local environment row, a workspace
record, and a running service are distinct prerequisites. A fresh instance
still needs an owned cwd/repository and runtime commands; `/workspaces` lists
existing work. Reopening depends on lifecycle, not a generic detail action.
- A link or running badge does not prove the service is reachable from the user’s browser.
- Workspace IDs, runtime leases, and repository paths are not interchangeable.
- Use [recovery](./recovery.md) for divergence/export repair and [CLI operations](./cli-operations.md) for local worktree tooling.
+1 -1
View File
@@ -61,7 +61,7 @@
"smoke:posthog-live": "node scripts/smoke/posthog-live.mjs",
"smoke:pipelines-tutorial": "./scripts/smoke/pipelines-tutorial-smoke.sh",
"smoke:terminal-bench-loop-skill": "node scripts/smoke/terminal-bench-loop-skill-smoke.mjs",
"test:release-registry": "node --test scripts/__tests__/canary-startup-playwright.test.mjs scripts/__tests__/cursor-public-install-sandbox.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/prefetch-child.test.mjs scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/pack-staged-packages.test.mjs scripts/__tests__/retain-runner-qualification-packages.test.mjs scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs scripts/release-lib.test.mjs scripts/release-registry-versions.test.mjs scripts/link-plugin-dev-sdk.test.js scripts/acpx-patch-packaging.test.mjs scripts/service-onboard-smoke.test.mjs scripts/docker-onboard-smoke.test.mjs scripts/preview-artifacts.test.mjs scripts/cloud-migrator-artifacts.test.mjs",
"test:release-registry": "node --test scripts/__tests__/canary-startup-playwright.test.mjs scripts/__tests__/cursor-public-install-sandbox.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/prefetch-child.test.mjs scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/pack-staged-packages.test.mjs scripts/__tests__/retain-runner-qualification-packages.test.mjs scripts/__tests__/runner-release-crate-cache.test.mjs scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs scripts/release-lib.test.mjs scripts/release-registry-versions.test.mjs scripts/link-plugin-dev-sdk.test.js scripts/acpx-patch-packaging.test.mjs scripts/service-onboard-smoke.test.mjs scripts/docker-onboard-smoke.test.mjs scripts/preview-artifacts.test.mjs scripts/cloud-migrator-artifacts.test.mjs",
"storybook-visual:baseline": "node scripts/storybook-visual-baseline.mjs",
"test:storybook-visual": "node scripts/storybook-visual-baseline.mjs download && node scripts/storybook-visual-baseline.mjs verify && pnpm build-storybook && npx playwright test --config tests/storybook-visual/playwright.config.ts",
"test:storybook-visual:update": "node scripts/storybook-visual-baseline.mjs download && pnpm build-storybook && npx playwright test --config tests/storybook-visual/playwright.config.ts --update-snapshots && node scripts/storybook-visual-baseline.mjs pack",
@@ -0,0 +1,22 @@
# ACPX lifecycle test ownership
The engine's primary behavioral boundary is `createAcpxEngineExecutor` in
`execute.test.ts`. `run-fault-matrix.test.ts` owns resource acquisition/failure
outcomes. Sequence, cancellation, session-store, and workspace restore tests
retain their separate ordering and persistence contracts.
- Extend the existing harness or literal case table before creating another
full executor/bridge/sandbox mock graph. Temporary characterization suites
must have a cutover plan; migrate their unique assertions when the refactor
finishes rather than keeping both layers indefinitely.
- A provider adapter suite needs a distinct provider configuration, credential,
prompt, parser, or transport contract. Do not replay shared lifecycle proof
merely under different adapter names.
- Verify actual close, cancel, bridge-stop, restore, and lease-release callbacks
when guarding physical cleanup. Ledger dispositions alone do not establish
that the cleanup ran or that ordering is correct.
- Keep stalled-phase/Stop tests at the live engine boundary. Pure sequence
tests cannot replace callback wiring while real awaits are held open.
- Before retiring a layer, map every unique assertion to a keeper, obtain an
independent preservation review, and use focused mutations for repaired gaps.
Never remove a baseline failure to make the suite green.
@@ -1,876 +0,0 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { AcpRuntimeOptions } from "acpx/runtime";
import type { AdapterExecutionContext, AdapterRuntimeMcpAccess } from "@paperclipai/adapter-utils";
import {
prepareAdapterExecutionTargetRuntime,
startAdapterExecutionTargetPaperclipBridge,
startAdapterExecutionTargetProcessSessionBridge,
} from "@paperclipai/adapter-utils/execution-target";
// This file is a characterization test. It pins the engine boundary's CURRENT
// behavior; it never changes production code. Each test states the observed
// contract of `executeAcpxEngine` as data, so a later refactor that alters an
// exit path's result form, its finalization set, or its warm-lane decision
// fails here.
//
// Adapter-boundary automatic CLI fallback note (File 1b): the codex, claude, and
// gemini adapters wrap this engine and fall back to their CLI lane when an
// AUTOMATIC ACP selection fails, but RETHROW when ACP was EXPLICITLY selected.
// Those three adapters live in higher packages that depend on this one, so an
// adapter-utils test cannot import them (a reverse dependency does not exist in
// package.json). That branch already has REAL coverage in each adapter package:
// - packages/adapters/codex-local/src/server/execute.acp-fallback.test.ts
// - packages/adapters/claude-local/src/server/execute.acp-fallback.test.ts
// - packages/adapters/gemini-local/src/server/execute.acp-fallback.test.ts
// and each adapter's `normalizeEngine` / `resolveXxxExecutionEngineForRun`
// classification (explicit acp/cli vs automatic acp) is pinned in each
// package's acp.test.ts. This file pins the engine-level exit paths those
// adapters wrap.
// Wrap the staging seam + both sandbox bridges in call-recording spies that
// still delegate to the real implementations (a runner-backed sandbox test
// exercises them end-to-end against a local runner). This lets the staging
// tests assert the exact `runtimeRootDir`/`workspaceLocalDir`/`assets` the
// engine threads without changing any real behavior for the other tests.
vi.mock("@paperclipai/adapter-utils/execution-target", async (importActual) => {
const actual = await importActual<typeof import("@paperclipai/adapter-utils/execution-target")>();
return {
...actual,
prepareAdapterExecutionTargetRuntime: vi.fn(actual.prepareAdapterExecutionTargetRuntime),
startAdapterExecutionTargetPaperclipBridge: vi.fn(actual.startAdapterExecutionTargetPaperclipBridge),
startAdapterExecutionTargetProcessSessionBridge: vi.fn(actual.startAdapterExecutionTargetProcessSessionBridge),
};
});
import {
createAcpxEngineExecutor,
type AcpxEngineExecutorOptions,
} from "./execute.js";
import { runChildProcess } from "../server-utils.js";
const tempRoots: string[] = [];
async function makeTempRoot() {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-acpx-skills-"));
tempRoots.push(root);
return root;
}
afterEach(async () => {
// A remote run stages a process-session bridge whose detached event writer can
// still be flushing a trailing event file into `.../process-sessions/<id>/events`
// when the run's own best-effort `client.remove(sessionDir)` (which production
// catch-wraps) has already returned. Under CI load that trailing write can land
// between this recursive delete's directory snapshot and its `rmdir`, surfacing as
// `ENOTEMPTY`. `maxRetries`/`retryDelay` make the cleanup ride out that window the
// same way production tolerates it, instead of failing the just-passed test.
await Promise.all(
tempRoots.splice(0).map((root) =>
fs.rm(root, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }),
),
);
});
function createLocalSandboxRunner(
onExecute?: (input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
}) => void,
) {
let counter = 0;
return {
execute: async (input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
stdin?: string;
timeoutMs?: number;
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
onSpawn?: (meta: { pid: number; startedAt: string }) => Promise<void>;
}) => {
counter += 1;
onExecute?.(input);
const command = input.command === "bash" ? "/bin/bash" : input.command;
return await runChildProcess(`acpx-sandbox-run-${counter}`, command, input.args ?? [], {
cwd: input.cwd ?? process.cwd(),
env: input.env ?? {},
stdin: input.stdin,
timeoutSec: Math.max(1, Math.ceil((input.timeoutMs ?? 30_000) / 1000)),
graceSec: 5,
onLog: input.onLog ?? (async () => {}),
onSpawn: input.onSpawn
? async (meta) => input.onSpawn?.({ pid: meta.pid, startedAt: meta.startedAt })
: undefined,
});
},
};
}
function buildRuntime(
onSetConfigOption?: (input: { key: string; value: string }) => void,
onEnsureSession?: (input: Record<string, unknown>) => void,
) {
return {
ensureSession: async (input: Record<string, unknown>) => {
onEnsureSession?.(input);
return ({
backendSessionId: "backend-session",
agentSessionId: "agent-session",
runtimeSessionName: "runtime-session",
});
},
startTurn: () => ({
events: (async function* () {
yield { type: "done", stopReason: "end_turn" };
})(),
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
cancel: async () => {},
}),
setConfigOption: async (input: { key: string; value: string }) => {
onSetConfigOption?.(input);
},
close: async () => {},
};
}
async function runExecutor(
config: Record<string, unknown>,
options: {
context?: Record<string, unknown>;
executionTransport?: Record<string, unknown>;
authToken?: string;
executionTarget?: Record<string, unknown>;
runtimeMcp?: AdapterRuntimeMcpAccess;
prepareRemoteManagedHome?: AcpxEngineExecutorOptions["prepareRemoteManagedHome"];
startupTraceContext?: AdapterExecutionContext["startupTraceContext"];
} = {},
) {
const runtimeOptions: Record<string, unknown>[] = [];
const configOptions: Array<{ key: string; value: string }> = [];
const sessionInputs: Record<string, unknown>[] = [];
const meta: Record<string, unknown>[] = [];
const logs: Array<{ stream: string; text: string }> = [];
const events: Array<{ eventType: string; payload?: Record<string, unknown> }> = [];
const execute = createAcpxEngineExecutor({
...(options.prepareRemoteManagedHome
? { prepareRemoteManagedHome: options.prepareRemoteManagedHome }
: {}),
createRuntime: (options) => {
runtimeOptions.push(options as unknown as Record<string, unknown>);
return buildRuntime(
({ key, value }) => configOptions.push({ key, value }),
(input) => sessionInputs.push(input),
) as never;
},
});
const result = await execute({
runId: "run-1",
agent: {
id: "agent-1",
companyId: "company-1",
},
runtime: {},
config,
context: options.context ?? {},
executionTransport: options.executionTransport,
authToken: options.authToken,
executionTarget: options.executionTarget,
runtimeMcp: options.runtimeMcp,
startupTraceContext: options.startupTraceContext,
onLog: async (stream: "stdout" | "stderr", text: string) => {
logs.push({ stream, text });
},
onMeta: async (payload: unknown) => {
meta.push(payload as Record<string, unknown>);
},
onEvent: async (event: { eventType: string; payload?: Record<string, unknown> }) => {
events.push(event);
},
} as never);
expect(result.exitCode).toBe(0);
return { logs, meta, events, runtimeOptions, configOptions, sessionInputs, result };
}
// A remote sandbox setup that stages the host worktree through the real local
// runner, so a run reaches the post-build window with live bridges and a held
// staging lease.
async function setupRemoteSandbox() {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const localCwd = path.join(root, "worktree");
const remoteCwd = path.join(root, "remote-workspace");
await fs.mkdir(localCwd, { recursive: true });
await fs.mkdir(remoteCwd, { recursive: true });
await fs.writeFile(path.join(localCwd, "hello.txt"), "hi", "utf8");
const executionTarget = {
kind: "remote",
transport: "sandbox",
providerKey: "fake-plugin",
remoteCwd,
runner: createLocalSandboxRunner(),
};
return { root, stateDir, localCwd, remoteCwd, executionTarget };
}
const okHandle = {
backendSessionId: "backend-session",
agentSessionId: "agent-session",
runtimeSessionName: "runtime-session",
};
function completedTurn() {
return {
events: (async function* () {})(),
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
cancel: async () => {},
};
}
function throwingTurn() {
return {
events: (async function* () {
throw new Error("turn upstream boom");
})(),
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
cancel: async () => {},
};
}
// A context whose only prompt-read field throws when read, so the prompt build
// fails after the session handshake succeeds and before the turn starts.
function throwingHandoffContext(): Record<string, unknown> {
const context: Record<string, unknown> = {};
Object.defineProperty(context, "paperclipSessionHandoffMarkdown", {
enumerable: false,
get() {
throw new Error("prompt build boom");
},
});
return context;
}
// Stub both sandbox bridges with stop spies collected per start, so a test can
// assert the bridges stopped without running the real bridge transport.
function stubBridges() {
const paperclipStops: Array<ReturnType<typeof vi.fn>> = [];
const processStops: Array<ReturnType<typeof vi.fn>> = [];
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementation(async () => {
const stop = vi.fn(async () => {});
paperclipStops.push(stop);
return { env: {}, stop } as never;
});
vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mockImplementation(async () => {
const stop = vi.fn(async () => {});
processStops.push(stop);
return { agentCommand: null, stop } as never;
});
const anyStopped = (stops: Array<ReturnType<typeof vi.fn>>) =>
stops.some((stop) => stop.mock.calls.length > 0);
const stoppedCount = (stops: Array<ReturnType<typeof vi.fn>>) =>
stops.filter((stop) => stop.mock.calls.length > 0).length;
return { paperclipStops, processStops, anyStopped, stoppedCount };
}
function remoteArgs(
stateDir: string,
localCwd: string,
executionTarget: unknown,
overrides: Record<string, unknown> = {},
) {
return {
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
context: {},
authToken: "real-run-jwt",
executionTarget,
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
...overrides,
};
}
describe("composed ACPX run: engine-boundary result form per exit path", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("returns (never throws) a create_runtime error result when the post-build runtime construction fails", async () => {
const root = await makeTempRoot();
const execute = createAcpxEngineExecutor({
createRuntime: () => {
throw new Error("createRuntime boom");
},
});
// The engine RETURNS a settled error result on this path; it does not throw.
const result = await execute({
runId: "boundary-create-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.signal).toBe(null);
expect(result.timedOut).toBe(false);
expect(result.resultJson?.phase).toBe("create_runtime");
expect(result.summary).toContain("createRuntime boom");
});
it("returns an ensure_session error result when the handshake fails", async () => {
const root = await makeTempRoot();
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
ensureSession: async () => {
throw new Error("ensureSession boom");
},
startTurn: () => completedTurn(),
close: async () => {},
}) as never,
});
const result = await execute({
runId: "boundary-ensure-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("ensure_session");
});
it("returns a runtime-error result with no clearSession and no errorMeta when ensureSession yields no handle", async () => {
const root = await makeTempRoot();
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
// A runtime that returns no session handle drives the missing-handle path.
ensureSession: async () => undefined,
startTurn: () => completedTurn(),
close: async () => {},
}) as never,
});
const result = await execute({
runId: "boundary-missing-handle",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("ensure_session");
expect(result.errorCode).toBe("acpx_runtime_error");
// The missing-handle result carries neither clearSession nor errorMeta, unlike
// the other pre-turn error paths.
expect(result.clearSession).toBeUndefined();
expect(result.errorMeta).toBeUndefined();
});
it("returns a configure_session error result that carries the requested session fields", async () => {
const root = await makeTempRoot();
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => completedTurn(),
// A custom agent with a requested model applies a session config option;
// a throwing setConfigOption drives the configure_session failure path.
setConfigOption: async () => {
throw new Error("setConfigOption boom");
},
close: async () => {},
}) as never,
});
const result = await execute({
runId: "boundary-configure-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: {
agent: "custom",
agentCommand: "node ./fake-acp.js",
model: "custom-model-x",
stateDir: path.join(root, "state"),
},
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("configure_session");
// The configure_session result echoes the requested session identity fields.
expect(result.resultJson).toHaveProperty("agent");
expect(result.resultJson?.requestedModel).toBe("custom-model-x");
expect(result.resultJson).toHaveProperty("requestedThinkingEffort");
expect(result.resultJson).toHaveProperty("fastMode");
});
it("returns a prepare_turn error result when the prompt build fails before the turn starts", async () => {
const root = await makeTempRoot();
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => completedTurn(),
close: async () => {},
}) as never,
});
const result = await execute({
runId: "boundary-prepare-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: throwingHandoffContext(),
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("prepare_turn");
});
it("returns a turn error result with acpx_turn_failed when the running turn fails", async () => {
const root = await makeTempRoot();
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => throwingTurn(),
close: async () => {},
}) as never,
});
const result = await execute({
runId: "boundary-turn-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("turn");
expect(result.errorCode).toBe("acpx_turn_failed");
});
it("returns the terminal turn-success shape (the one non-error-shaped return)", async () => {
const root = await makeTempRoot();
const execute = createAcpxEngineExecutor({
createRuntime: () => buildRuntime() as never,
});
const result = await execute({
runId: "boundary-success",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
// A completed, non-timed-out turn returns exitCode 0, no signal, no errorCode,
// and the session identity fields, with resultJson.status = "completed".
expect(result.exitCode).toBe(0);
expect(result.signal).toBe(null);
expect(result.timedOut).toBe(false);
expect(result.errorCode).toBe(null);
expect(result.resultJson?.status).toBe("completed");
expect(result.sessionId).toBe("backend-session");
expect(result.sessionDisplayId).toBe("agent-session");
expect(result.sessionParams).toBeTruthy();
});
it("THROWS (does not return) when buildRuntime fails before it settles", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
// A staging failure throws inside buildRuntime before it settles, so the engine
// rethrows through the `if (!buildRuntimeSettled) throw err` guard. This is one
// of exactly two engine-boundary throw paths.
vi.mocked(prepareAdapterExecutionTargetRuntime).mockImplementationOnce(async () => {
throw new Error("staging boom");
});
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
createRuntime: () => buildRuntime() as never,
});
await expect(
execute({
runId: "boundary-build-throw",
...remoteArgs(stateDir, localCwd, executionTarget),
} as never),
).rejects.toThrow("staging boom");
});
it("THROWS (does not return) on a partial bridge failure and stops the started sibling bridge once", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
// The paperclip bridge fails while the concurrently-started process-session
// bridge resolves a live handle. This is the second engine-boundary throw path;
// the abandon path must stop the started sibling so no bridge leaks.
const stop = vi.fn(async () => {});
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementationOnce(async () => {
throw new Error("paperclip bridge boom");
});
vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mockImplementationOnce(
async () => ({ agentCommand: null, stop }) as never,
);
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
createRuntime: () => buildRuntime() as never,
});
await expect(
execute({
runId: "boundary-bridge-throw",
...remoteArgs(stateDir, localCwd, executionTarget),
} as never),
).rejects.toThrow("paperclip bridge boom");
expect(stop).toHaveBeenCalledTimes(1);
});
});
describe("composed ACPX run: finalization set fires exactly once per exit path", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("stops both bridges once and releases the staging lease on every exit path", async () => {
const scenarios: Array<{
name: string;
createRuntime: AcpxEngineExecutorOptions["createRuntime"];
config?: Record<string, unknown>;
context?: Record<string, unknown>;
}> = [
{
name: "create_runtime",
createRuntime: () => {
throw new Error("createRuntime boom");
},
},
{
name: "ensure_session",
createRuntime: () =>
({
ensureSession: async () => {
throw new Error("ensure boom");
},
startTurn: () => completedTurn(),
close: async () => {},
}) as never,
},
{
name: "configure_session",
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => completedTurn(),
setConfigOption: async () => {
throw new Error("config boom");
},
close: async () => {},
}) as never,
config: { agent: "custom", agentCommand: "node ./fake-acp.js", model: "custom-model-x" },
},
{
name: "prepare_turn",
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => completedTurn(),
close: async () => {},
}) as never,
context: throwingHandoffContext(),
},
{
name: "turn",
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => throwingTurn(),
close: async () => {},
}) as never,
},
{
name: "success",
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => completedTurn(),
close: async () => {},
}) as never,
},
];
for (const scenario of scenarios) {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
const { paperclipStops, processStops, stoppedCount } = stubBridges();
const stagingLocks = new Map<string, Promise<unknown>>();
const execute = createAcpxEngineExecutor({
stagingLocks,
warmHandles: new Map(),
stagedRuntimes: new Map(),
createRuntime: scenario.createRuntime,
});
const overrides: Record<string, unknown> = {};
if (scenario.config) {
overrides.config = {
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir,
cwd: localCwd,
...scenario.config,
};
}
if (scenario.context) overrides.context = scenario.context;
await execute({
runId: `finalize-${scenario.name}`,
...remoteArgs(stateDir, localCwd, executionTarget, overrides),
} as never).catch(() => {});
// The finalization set fires exactly once: each bridge stops once and the
// per-session staging lease releases, so the lock map never strands the next
// same-session run.
expect(stoppedCount(paperclipStops), `paperclip bridge must stop once on ${scenario.name}`).toBe(1);
expect(stoppedCount(processStops), `process-session bridge must stop once on ${scenario.name}`).toBe(1);
expect(stagingLocks.size, `lease must release on ${scenario.name}`).toBe(0);
}
});
it("does not re-run the turn teardown when the result mapping throws after the close", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
const { paperclipStops, processStops, stoppedCount } = stubBridges();
let closeCount = 0;
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => ({
events: (async function* () {
yield { type: "done", stopReason: "end_turn" };
})(),
// A completed turn whose result mapping throws after the close is read.
result: Promise.resolve({
status: "completed",
get stopReason(): string {
throw new Error("mapping boom");
},
}),
cancel: async () => {},
}),
close: async () => {
closeCount += 1;
},
}) as never,
});
await execute({
runId: "finalize-map-throw",
...remoteArgs(stateDir, localCwd, executionTarget),
} as never).catch(() => {});
// The completed turn closed the runtime once; the mapping throw did not re-run
// the teardown through the turn catch.
expect(closeCount).toBe(1);
expect(stoppedCount(paperclipStops)).toBe(1);
expect(stoppedCount(processStops)).toBe(1);
});
});
describe("composed ACPX run: host-lane warm handle set", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("closes and relaunches a persistent local runtime, so the next compatible run re-creates it", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
let createCount = 0;
const warmHandles = new Map();
const execute = createAcpxEngineExecutor({
warmHandles,
createRuntime: () => {
createCount += 1;
return buildRuntime() as never;
},
});
const config = {
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir,
mode: "persistent",
warmHandleIdleMs: 60_000,
};
const first = await execute({
runId: "warm-save-1",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
} as never);
expect(first.exitCode).toBe(0);
// Amendment B: a persistent local completed turn closes and relaunches the
// runtime instead of warm-saving it (the run-minted API key is never revoked).
expect(warmHandles.size).toBe(0);
expect(createCount).toBe(1);
const second = await execute({
runId: "warm-save-2",
agent: { id: "agent-1", companyId: "company-1" },
runtime: { sessionParams: first.sessionParams },
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
} as never);
expect(second.exitCode).toBe(0);
// No warm runtime survived, so the compatible second run re-creates it.
expect(createCount).toBe(2);
});
it("closes (does not warm-save) a completed non-persistent runtime, so the next run re-creates", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
let createCount = 0;
let closeCount = 0;
const warmHandles = new Map();
const execute = createAcpxEngineExecutor({
warmHandles,
createRuntime: () => {
createCount += 1;
return {
ensureSession: async () => okHandle,
startTurn: () => completedTurn(),
setConfigOption: async () => {},
close: async () => {
closeCount += 1;
},
} as never;
},
});
// No persistent mode, so the completed turn closes the runtime instead of
// warm-saving it.
const config = { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir };
const first = await execute({
runId: "warm-none-1",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
} as never);
expect(first.exitCode).toBe(0);
expect(warmHandles.size).toBe(0);
expect(closeCount).toBe(1);
const second = await execute({
runId: "warm-none-2",
agent: { id: "agent-1", companyId: "company-1" },
runtime: { sessionParams: first.sessionParams },
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
} as never);
expect(second.exitCode).toBe(0);
// No warm handle survived, so the second run constructs a fresh runtime.
expect(createCount).toBe(2);
});
it("runs a fresh acp.handshake on the second run and re-creates the runtime (no warm reuse)", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
let createCount = 0;
const warmHandles = new Map();
const secondEvents: Array<{ eventType: string; payload?: Record<string, unknown> }> = [];
const execute = createAcpxEngineExecutor({
warmHandles,
createRuntime: () => {
createCount += 1;
return buildRuntime() as never;
},
});
const config = {
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir,
mode: "persistent",
warmHandleIdleMs: 60_000,
};
const first = await execute({
runId: "warm-hit-1",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
} as never);
await execute({
runId: "warm-hit-2",
agent: { id: "agent-1", companyId: "company-1" },
runtime: { sessionParams: (first as { sessionParams?: unknown }).sessionParams },
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onEvent: async (event: { eventType: string; payload?: Record<string, unknown> }) => {
secondEvents.push(event);
},
} as never);
// Amendment B: the first run closed and relaunched, so no warm handle survives.
// The second run runs a fresh acp.handshake (outcome ok, not skipped) and
// re-creates the runtime.
const handshakeEvents = secondEvents.filter(
(event) => event.eventType === "run.startup.step" && event.payload?.step === "acp.handshake",
);
expect(handshakeEvents).toHaveLength(1);
expect(handshakeEvents[0]!.payload?.outcome).not.toBe("skipped");
expect(createCount).toBe(2);
});
});
@@ -393,6 +393,54 @@ const ALLOWED_TURN_SPAN_ATTRIBUTE_KEYS = new Set<string>([
]);
describe("shared ACPX engine runtime behavior", () => {
it("passes the current prompt, session, cancellation and diagnostic inputs to the runtime turn", async () => {
const root = await makeTempRoot();
const { result, turnInputs, meta } = await runExecutor({
agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state"),
cwd: root, timeoutSec: 120,
});
expect(turnInputs).toHaveLength(1);
expect(turnInputs[0]).toMatchObject({
handle: { backendSessionId: "backend-session", agentSessionId: "agent-session", runtimeSessionName: "runtime-session" },
text: meta[0]?.prompt, mode: "prompt", requestId: "run-1", timeoutMs: 120_000,
onTerminalSessionFailure: expect.any(Function),
});
expect(meta[0]?.prompt).toBeTypeOf("string");
expect(turnInputs[0]?.text).toBeTypeOf("string");
expect((meta[0]?.prompt as string).length).toBeGreaterThan(0);
expect((turnInputs[0]?.text as string).length).toBeGreaterThan(0);
expect(turnInputs[0]?.signal).toBeInstanceOf(AbortSignal);
expect((turnInputs[0]?.signal as AbortSignal).aborted).toBe(false);
expect(result.summary).toBe("end_turn");
expect(result.sessionParams?.configFingerprint).toBeTypeOf("string");
expect((result.sessionParams?.configFingerprint as string).length).toBeGreaterThan(0);
expect(result.sessionParams?.sessionKey).toBe(`paperclip:company-1:agent-1:default:${result.sessionParams?.configFingerprint}`);
});
it.each(["agentCommand", "cwd", "model", "thinkingEffort", "mode", "fastMode"] as const)(
"folds configured %s into the engine session fingerprint",
async (dimension) => {
const root = await makeTempRoot();
const config = {
agent: dimension === "fastMode" ? "codex" : "custom", agentCommand: "node ./fake-acp.js",
stateDir: path.join(root, "state"), cwd: root,
env: { CODEX_HOME: path.join(root, "codex-home") },
paperclipRuntimeSkills: [], paperclipSkillSync: { desiredSkills: [] },
};
const changed: Record<string, unknown> = {
agentCommand: "node ./other-acp.js", cwd: path.join(root, "other-workspace"), model: "test-model",
thinkingEffort: "high", mode: "oneshot", fastMode: true,
};
const first = await runExecutor(config);
const second = await runExecutor({ ...config, [dimension]: changed[dimension] });
expect(first.result.sessionParams?.configFingerprint).toBeTypeOf("string");
expect(second.result.sessionParams?.configFingerprint).toBeTypeOf("string");
expect((first.result.sessionParams?.configFingerprint as string).length).toBeGreaterThan(0);
expect((second.result.sessionParams?.configFingerprint as string).length).toBeGreaterThan(0);
expect(second.result.sessionParams?.configFingerprint).not.toBe(first.result.sessionParams?.configFingerprint);
},
);
it("integrates the env-free store while delivering current credentials and options to the provider", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
@@ -1003,10 +1051,11 @@ describe("shared ACPX engine runtime behavior", () => {
events: (async function* () {
yield {
type: "text_delta",
text: "streamed hello",
text: " streamed ",
stream: "output",
tag: "agent_message_chunk",
};
yield { type: "text_delta", text: "hello ", stream: "output", tag: "agent_message_chunk" };
yield { type: "done", stopReason: "end_turn" };
})(),
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
@@ -1032,15 +1081,13 @@ describe("shared ACPX engine runtime behavior", () => {
} as never);
expect(result.exitCode).toBe(0);
expect(logs).toContainEqual({
stream: "stdout",
text: `${JSON.stringify({
type: "acpx.text_delta",
text: "streamed hello",
channel: "output",
tag: "agent_message_chunk",
})}\n`,
});
expect(result.summary).toBe("streamed hello");
for (const text of [" streamed ", "hello "]) {
expect(logs).toContainEqual({
stream: "stdout",
text: `${JSON.stringify({ type: "acpx.text_delta", text, channel: "output", tag: "agent_message_chunk" })}\n`,
});
}
});
it("defaults run summaries to the final output segment without thought text", async () => {
@@ -1620,6 +1667,11 @@ describe("shared ACPX engine runtime behavior", () => {
} as never);
expect(result.exitCode).not.toBe(0);
if (status === "failed") {
expect(result.errorCode).toBe("acpx_turn_failed");
expect(result.summary).toBe("fixture failure");
expect(result.resultJson?.stopReason).toBe("fixture failure");
}
expect(result.usageComplete).toBe(false);
expect(onUsage).toHaveBeenLastCalledWith(expect.objectContaining({ complete: false, costUsd: 0.31 }));
expect(result.usage).toEqual({ inputTokens: 40, outputTokens: 700, cachedInputTokens: 60 });
@@ -3267,6 +3319,7 @@ describe("gemini ACP flag selection", () => {
"Run exceeded the adapter execution timeout (timeoutSec=1, configured via adapterConfig.timeoutSec). " +
"Set adapterConfig.timeoutSec to raise it.";
expect(result.timedOut).toBe(true);
expect(result.signal).toBe("SIGTERM");
expect(result.errorCode).toBe("acpx_timeout");
expect(result.resultJson?.adapterExecutionTimeout).toEqual({ timeoutSec: 1, source: "configured" });
expect(result.errorMessage).toBe(expectedMessage);
@@ -3644,8 +3697,10 @@ describe("ACPX engine remote sandbox staging seam (PR 1: workspace + cwd)", () =
// Decode the process-session LAUNCH payload (the base64 command blob) — the
// in-sandbox process env is carried there, NOT in the exec's own `env`.
let launchPayload: Record<string, unknown> | null = null;
let bridgeExecEnv: Record<string, string> | undefined;
(executionTarget as { runner: unknown }).runner = createLocalSandboxRunner((input) => {
if (input.env?.PAPERCLIP_SANDBOX_EXEC_CHANNEL === "bridge") {
bridgeExecEnv = input.env;
const script = input.args?.[1] ?? "";
const match = script.match(/PAPERCLIP_PROCESS_SESSION_COMMAND_B64='([^']+)'/);
if (match) {
@@ -3680,6 +3735,8 @@ describe("ACPX engine remote sandbox staging seam (PR 1: workspace + cwd)", () =
expect(String(payloadEnv.PAPERCLIP_API_URL ?? "")).toMatch(/^http:\/\/127\.0\.0\.1:\d+$/);
expect(payloadEnv.PAPERCLIP_API_KEY).toBeTruthy();
expect(payloadEnv.PAPERCLIP_API_KEY).not.toBe("real-run-jwt");
expect(bridgeExecEnv?.PAPERCLIP_SANDBOX_EXEC_CHANNEL).toBe("bridge");
expect(bridgeExecEnv?.PAPERCLIP_API_KEY).toBeUndefined();
});
it("publishes referenced-project workspace hints repointed at their staged sandbox directories", async () => {
@@ -3816,12 +3873,17 @@ describe("ACPX engine remote sandbox staging seam (PR 1: workspace + cwd)", () =
expect(sessionInputs[0]?.cwd).toBe(remoteCwd);
});
it("test_remote_warm_handle_reused_after_cwd_change", async () => {
it("recreates the remote runtime while resuming after the cwd is staged", async () => {
const { stateDir, localCwd, remoteCwd, executionTarget } = await setupRemoteSandbox();
const ensureInputs: Array<Record<string, unknown>> = [];
let created = 0;
const warmHandles = new Map();
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
createRuntime: () => buildRuntime(undefined, (input) => ensureInputs.push(input)) as never,
warmHandles,
createRuntime: () => {
created += 1;
return buildRuntime(undefined, (input) => ensureInputs.push(input)) as never;
},
});
const base = {
agent: { id: "agent-1", companyId: "company-1" },
@@ -3849,6 +3911,9 @@ describe("ACPX engine remote sandbox staging seam (PR 1: workspace + cwd)", () =
expect(first.exitCode).toBe(0);
expect(second.exitCode).toBe(0);
expect(created).toBe(2);
expect(warmHandles.size).toBe(0);
expect(ensureInputs).toHaveLength(2);
// Both runs resolve session/new to the in-sandbox cwd...
expect(ensureInputs[0]?.cwd).toBe(remoteCwd);
expect(ensureInputs[1]?.cwd).toBe(remoteCwd);
@@ -3981,6 +4046,7 @@ describe("ACPX engine remote managed-home seam (PR 2: per-adapter home seed)", (
// The seam's asset was threaded through the shared staging seam...
const stageArgs = vi.mocked(prepareAdapterExecutionTargetRuntime).mock.calls[0]![0];
expect(stageArgs.workspaceLocalDir).toBe(localCwd);
expect(stageArgs.assets).toEqual([
{ key: "home", localDir: managedHomeDir, followSymlinks: true },
]);
@@ -6372,10 +6438,11 @@ describe("ACPX engine per-step startup timing (run.startup.step events)", () =>
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const warmHandles = new Map();
let created = 0;
const secondEvents: Array<{ eventType: string; payload?: Record<string, unknown> }> = [];
const execute = createAcpxEngineExecutor({
warmHandles,
createRuntime: () => buildRuntime() as never,
createRuntime: () => { created += 1; return buildRuntime() as never; },
});
const config = {
agent: "custom",
@@ -6413,6 +6480,7 @@ describe("ACPX engine per-step startup timing (run.startup.step events)", () =>
const handshakeEvents = secondEvents.filter(
(event) => event.eventType === "run.startup.step" && event.payload?.step === "acp.handshake",
);
expect(created).toBe(2);
expect(handshakeEvents).toHaveLength(1);
expect(handshakeEvents[0]!.payload?.outcome).not.toBe("skipped");
});
@@ -6950,6 +7018,229 @@ describe("ACPX engine run lifecycle corrections (F3: one teardown error policy)"
};
}
it.each(["close_session", "stop_transport", "instruction_collection", "workspace_restore", "phase_reporting", "close_error_reporting"])(
"keeps %s visible while its real settlement await is stalled",
async (blockedPhase) => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
stubBridges();
let resume!: () => void;
let entered!: () => void;
const gate = new Promise<void>((resolve) => { resume = resolve; });
const reached = new Promise<void>((resolve) => { entered = resolve; });
const block = () => { entered(); return gate; };
const scopes = new Map<symbol, string>();
if (blockedPhase === "stop_transport") {
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementation(async () => ({ env: {}, stop: block }) as never);
}
const execute = createAcpxEngineExecutor({
createRuntime: () => ({
ensureSession: async () => okHandle,
startTurn: () => blockedPhase === "close_error_reporting" ? throwingTurn() : completedTurn(),
close: blockedPhase === "close_session" ? block : async () => {
if (blockedPhase === "close_error_reporting") throw new Error("close fixture failed");
},
}) as never,
prepareRemoteManagedHome: async (input) => ({
stagedRuntime: await input.stage([]),
teardown: async () => {
if (blockedPhase === "workspace_restore") await block();
return { ok: true };
},
}),
});
const execution = execute({
runId: "pending-settlement-fixture",
...remoteArgs(stateDir, localCwd, executionTarget),
onProviderStopped: blockedPhase === "instruction_collection" ? block : async () => {},
onExecutionPhase: (phase: string) => {
const token = Symbol();
scopes.set(token, phase);
return () => { scopes.delete(token); };
},
onEvent: async (event: { eventType: string; payload?: Record<string, unknown> }) => {
if (blockedPhase === "phase_reporting" && event.eventType === "run.phase.timing" && event.payload?.phase === "end_session") await block();
},
onLog: async (_stream: string, text: string) => {
if (blockedPhase === "close_error_reporting" && text.includes("close fixture failed")) await block();
},
} as never);
try {
await reached;
expect([...scopes.values()].at(-1)).toBe(blockedPhase === "close_error_reporting" ? "phase_reporting" : blockedPhase);
} finally {
resume();
expect((await execution).exitCode).toBe(blockedPhase === "close_error_reporting" ? 1 : 0);
}
expect(scopes.size).toBe(0);
},
);
it("keeps the primary Stop cancellation visible until its await settles", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
stubBridges();
const controller = new AbortController();
let started!: () => void;
let cancelled!: () => void;
let resume!: () => void;
const turnStarted = new Promise<void>((resolve) => { started = resolve; });
const cancellationStarted = new Promise<void>((resolve) => { cancelled = resolve; });
const gate = new Promise<void>((resolve) => { resume = resolve; });
const scopes = new Map<symbol, string>();
const execute = createAcpxEngineExecutor({
createRuntime: () => ({
ensureSession: async () => okHandle,
startTurn: () => {
started();
return {
events: (async function* () { await gate; })(),
result: gate.then(() => ({ status: "cancelled", stopReason: "cancelled" })),
cancel: async () => { cancelled(); await gate; },
};
},
close: async () => {},
}) as never,
});
const execution = execute({
runId: "pending-cancel-fixture",
...remoteArgs(stateDir, localCwd, executionTarget),
signal: controller.signal,
onExecutionPhase: (phase: string) => {
const token = Symbol();
scopes.set(token, phase);
return () => { scopes.delete(token); };
},
} as never);
try {
await turnStarted;
controller.abort();
await cancellationStarted;
expect([...scopes.values()].at(-1)).toBe("cancel_turn");
} finally {
resume();
expect((await execution).exitCode).toBe(1);
}
expect(scopes.size).toBe(0);
});
it("test_clean_completed_remote_teardown_runs_bridge_stop_then_sync_back_then_lease_release", async () => {
// cleanupRemoteBridges (execute.ts:2306-2326) fixes the sub-order for a clean
// exit: stop both bridges (allSettled) → run the managed-home sync-back
// (remoteManagedHomeTeardown) → release the per-session staging lease LAST in
// a finally. Record the order through spies threaded into the stubbed bridges
// and the seam teardown, and read the still-held lease during the sync-back.
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
const order: string[] = [];
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementation(async () => ({
env: {},
stop: vi.fn(async () => {
order.push("bridge-stop");
}),
}) as never);
vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mockImplementation(async () => ({
agentCommand: null,
stop: vi.fn(async () => {
order.push("bridge-stop");
}),
}) as never);
const stagingLocks = new Map<string, Promise<unknown>>();
let leaseSizeDuringSyncBack = -1;
const execute = createAcpxEngineExecutor({
stagingLocks,
warmHandles: new Map(),
stagedRuntimes: new Map(),
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => completedTurn(),
close: vi.fn(async () => {}),
}) as never,
prepareRemoteManagedHome: async (input) => {
const stagedRuntime = await input.stage([]);
return {
stagedRuntime,
teardown: async () => {
order.push("sync-back");
// The lease is still held while the sync-back runs; it releases only
// afterward, in the cleanupRemoteBridges finally.
leaseSizeDuringSyncBack = stagingLocks.size;
return { ok: true };
},
};
},
});
const result = await execute({
runId: "clean-order",
...remoteArgs(stateDir, localCwd, executionTarget),
} as never);
expect(result.exitCode).toBe(0);
// Both bridge stops precede the sync-back; the sync-back is the last recorded
// step before the lease release.
expect(order).toEqual(["bridge-stop", "bridge-stop", "sync-back"]);
// The lease was still held during the sync-back...
expect(leaseSizeDuringSyncBack).toBe(1);
// ...and released last, so the lock map never strands the next run.
expect(stagingLocks.size).toBe(0);
});
it("test_terminal_close_reason_and_discard_persistent_state_per_outcome", async () => {
const cases: Array<{
status: "completed" | "failed" | "cancelled";
reason: string;
discard: boolean;
exitCode: number;
}> = [
{ status: "completed", reason: "paperclip completed turn cleanup", discard: false, exitCode: 0 },
{ status: "failed", reason: "paperclip turn failed", discard: false, exitCode: 1 },
{ status: "cancelled", reason: "paperclip turn cancelled", discard: true, exitCode: 1 },
];
for (const testCase of cases) {
const root = await makeTempRoot();
const closeSpy = vi.fn(async () => {});
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => ({
events: (async function* () {
yield { type: "done", stopReason: "end_turn" };
})(),
result:
testCase.status === "failed"
? Promise.resolve({ status: "failed", error: new Error("turn boom") })
: testCase.status === "cancelled"
? Promise.resolve({ status: "cancelled", stopReason: "cancelled" })
: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
cancel: async () => {},
}),
close: closeSpy,
}) as never,
});
const result = await execute({
runId: `terminal-${testCase.status}`,
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode, `${testCase.status} exit`).toBe(testCase.exitCode);
expect(closeSpy, `${testCase.status} closes once`).toHaveBeenCalledTimes(1);
expect((closeSpy.mock.calls[0]! as unknown[])[0], `${testCase.status} close args`).toMatchObject({
reason: testCase.reason,
discardPersistentState: testCase.discard,
});
}
});
it("collects instructions after confirmed close on a thrown provider turn, before workspace restore", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
const { paperclipStops, anyStopped } = stubBridges();
@@ -47,6 +47,14 @@ import { LedgerStateError, type ResourceId } from "./run-contracts.js";
import type { SettlementDispositionReport } from "./run-coordinator.js";
const tempRoots: string[] = [];
const bridgeStops: Array<ReturnType<typeof vi.fn>> = [];
const stagingLockMaps: Array<Map<string, Promise<unknown>>> = [];
function makeStagingLocks() {
const locks = new Map<string, Promise<unknown>>();
stagingLockMaps.push(locks);
return locks;
}
async function makeTempRoot() {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-acpx-fault-"));
@@ -151,16 +159,17 @@ function throwingHandoffContext(): Record<string, unknown> {
// handles without running the real bridge transport. `stopRejects` makes the
// bridge stop reject during settlement.
function stubBridges(options: { stopRejects?: boolean } = {}) {
const stop = options.stopRejects
? vi.fn(async () => {
throw new Error("bridge stop boom");
})
const makeStop = () => options.stopRejects
? vi.fn(async () => { throw new Error("bridge stop boom"); })
: vi.fn(async () => {});
const controlStop = makeStop();
const agentStop = makeStop();
bridgeStops.push(controlStop, agentStop);
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementation(async () => {
return { env: {}, stop } as never;
return { env: {}, stop: controlStop } as never;
});
vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mockImplementation(async () => {
return { agentCommand: null, stop } as never;
return { agentCommand: null, stop: agentStop } as never;
});
}
@@ -261,6 +270,10 @@ function assertDispositionReport(
expect(ids.length, "no resource is reported twice").toBe(new Set(ids).size);
// The report keys equal the acquired subset of the six resource ids.
expect(new Set(ids)).toEqual(new Set(expected.acquired));
// Reports alone can claim cleanup without invoking the actual cleanup callbacks.
if (ids.includes("control_bridge")) expect(bridgeStops[0]).toHaveBeenCalledTimes(1);
if (ids.includes("agent_bridge")) expect(bridgeStops[1]).toHaveBeenCalledTimes(1);
for (const locks of stagingLockMaps) expect(locks.size).toBe(0);
const transferred = expected.transferred ?? null;
for (const record of records) {
expect(["finalized", "transferred"]).toContain(record.disposition);
@@ -289,6 +302,8 @@ const SANDBOX_WITH_MANAGED_HOME: ResourceId[] = [
describe("composed ACPX run fault matrix", () => {
beforeEach(() => {
vi.clearAllMocks();
bridgeStops.length = 0;
stagingLockMaps.length = 0;
});
// Case 1 — a build failure throws before the ledger acquires any resource.
@@ -302,7 +317,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => completedRuntime(),
onSettlementDisposition: capture.onSettlementDisposition,
});
@@ -329,7 +344,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => completedRuntime(),
onSettlementDisposition: capture.onSettlementDisposition,
});
@@ -350,7 +365,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => {
throw new Error("createRuntime boom");
},
@@ -363,6 +378,9 @@ describe("composed ACPX run fault matrix", () => {
} as never);
expect(result.exitCode).toBe(1);
expect(result.signal).toBeNull();
expect(result.timedOut).toBe(false);
expect(result.summary).toContain("createRuntime boom");
expect(result.resultJson?.phase).toBe("create_runtime");
assertDispositionReport(capture.last(), { acquired: SANDBOX_PRE_RUNTIME });
});
@@ -372,17 +390,18 @@ describe("composed ACPX run fault matrix", () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
stubBridges();
const capture = captureDisposition();
const close = vi.fn(async () => {});
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () =>
({
ensureSession: async () => {
throw new Error("ensureSession boom");
},
startTurn: () => completedTurn(),
close: async () => {},
close,
}) as never,
onSettlementDisposition: capture.onSettlementDisposition,
});
@@ -393,6 +412,7 @@ describe("composed ACPX run fault matrix", () => {
} as never);
expect(result.exitCode).toBe(1);
expect(close).toHaveBeenCalledTimes(1);
expect(result.resultJson?.phase).toBe("ensure_session");
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
});
@@ -406,7 +426,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () =>
({
ensureSession: async () => undefined,
@@ -434,10 +454,11 @@ describe("composed ACPX run fault matrix", () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
stubBridges();
const capture = captureDisposition();
const close = vi.fn(async () => {});
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () =>
({
ensureSession: async () => okHandle,
@@ -445,7 +466,7 @@ describe("composed ACPX run fault matrix", () => {
setConfigOption: async () => {
throw new Error("setConfigOption boom");
},
close: async () => {},
close,
}) as never,
onSettlementDisposition: capture.onSettlementDisposition,
});
@@ -464,7 +485,11 @@ describe("composed ACPX run fault matrix", () => {
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("configure_session");
expect(close).toHaveBeenCalledTimes(1);
expect(result.resultJson).toMatchObject({
phase: "configure_session", agent: "custom", requestedModel: "custom-model-x",
requestedThinkingEffort: null, fastMode: false,
});
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
});
@@ -476,7 +501,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => completedRuntime(),
onSettlementDisposition: capture.onSettlementDisposition,
});
@@ -517,16 +542,20 @@ describe("composed ACPX run fault matrix", () => {
// succeeds. The runtime is created once, so acp_runtime registers once.
const capture = captureDisposition();
let created = 0;
let started = 0;
const ensureInputs: Record<string, unknown>[] = [];
const logs: string[] = [];
const secondExecute = createAcpxEngineExecutor({
onSettlementDisposition: capture.onSettlementDisposition,
createRuntime: () => {
created += 1;
return {
ensureSession: async (session: Record<string, unknown>) => {
ensureInputs.push(session);
if (session.resumeSessionId) throw new Error("resume session not found");
return okHandle;
},
startTurn: () => completedTurn(),
startTurn: () => { started += 1; return completedTurn(); },
setConfigOption: async () => {},
close: async () => {},
} as never;
@@ -538,13 +567,19 @@ describe("composed ACPX run fault matrix", () => {
runtime: { sessionParams: (first as { sessionParams?: unknown }).sessionParams },
config,
context: {},
onLog: async () => {},
onLog: async (_stream: string, text: string) => { logs.push(text); },
onMeta: async () => {},
} as never);
expect(second.exitCode).toBe(0);
// The runtime was created exactly once across the resume and the fresh retry.
expect(created).toBe(1);
expect(ensureInputs).toHaveLength(2);
expect(ensureInputs[0]?.resumeSessionId).toBe(first.sessionId);
expect(ensureInputs[1]?.resumeSessionId).toBeUndefined();
expect(started).toBe(1);
expect(second.clearSession).toBe(true);
expect(logs.some((text) => text.includes("is unavailable; retrying with a fresh session"))).toBe(true);
// The host lane acquires only the runtime, and it appears once.
assertDispositionReport(capture.last(), { acquired: ["acp_runtime"] });
});
@@ -555,15 +590,16 @@ describe("composed ACPX run fault matrix", () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
stubBridges();
const capture = captureDisposition();
const order: string[] = [];
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () =>
({
ensureSession: async () => okHandle,
startTurn: () => throwingTurn(),
close: async () => {},
startTurn: () => ({ ...throwingTurn(), cancel: async () => { order.push("cancel"); } }),
close: async () => { order.push("close"); },
}) as never,
onSettlementDisposition: capture.onSettlementDisposition,
});
@@ -574,6 +610,7 @@ describe("composed ACPX run fault matrix", () => {
} as never);
expect(result.exitCode).toBe(1);
expect(order).toEqual(["cancel", "close"]);
expect(result.resultJson?.phase).toBe("turn");
expect(result.errorCode).toBe("acpx_turn_failed");
assertDispositionReport(capture.last(), { acquired: SANDBOX_WITH_RUNTIME });
@@ -591,7 +628,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () =>
({
ensureSession: async () => okHandle,
@@ -629,7 +666,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () =>
({
ensureSession: async () => okHandle,
@@ -658,7 +695,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => completedRuntime(),
prepareRemoteManagedHome: managedHomeSeed(),
onSettlementDisposition: capture.onSettlementDisposition,
@@ -670,6 +707,13 @@ describe("composed ACPX run fault matrix", () => {
} as never);
expect(result.exitCode).toBe(0);
expect(result.signal).toBeNull();
expect(result.timedOut).toBe(false);
expect(result.errorCode).toBeNull();
expect(result.sessionId).toBe("backend-session");
expect(result.sessionDisplayId).toBe("agent-session");
expect(result.sessionParams).toBeTruthy();
expect(result.summary).toBe("end_turn");
expect(result.resultJson?.status).toBe("completed");
assertDispositionReport(capture.last(), {
acquired: SANDBOX_WITH_MANAGED_HOME,
@@ -686,7 +730,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => completedRuntime(),
onSettlementDisposition: capture.onSettlementDisposition,
});
@@ -714,7 +758,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => completedRuntime(),
prepareRemoteManagedHome: managedHomeSeed({ teardownRejects: true }),
onSettlementDisposition: capture.onSettlementDisposition,
@@ -742,7 +786,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => completedRuntime(),
prepareRemoteManagedHome: managedHomeSeed(),
});
@@ -770,7 +814,7 @@ describe("composed ACPX run fault matrix", () => {
const execute = createAcpxEngineExecutor({
warmHandles: new Map(),
stagedRuntimes: new Map(),
stagingLocks: new Map(),
stagingLocks: makeStagingLocks(),
createRuntime: () => completedRuntime(),
prepareRemoteManagedHome: managedHomeSeedWithClassifiedTeardownFailure(eaccesError),
});
@@ -915,6 +959,8 @@ describe("composed ACPX run fault matrix", () => {
describe("composed ACPX run per-phase telemetry", () => {
beforeEach(() => {
vi.clearAllMocks();
bridgeStops.length = 0;
stagingLockMaps.length = 0;
});
it("emits a run.phase.timing event for each lifecycle phase a clean host run reaches", async () => {
File diff suppressed because it is too large. Load diff
@@ -1,917 +0,0 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { AcpRuntimeOptions } from "acpx/runtime";
import type { AdapterExecutionContext, AdapterRuntimeMcpAccess } from "@paperclipai/adapter-utils";
import {
prepareAdapterExecutionTargetRuntime,
startAdapterExecutionTargetPaperclipBridge,
startAdapterExecutionTargetProcessSessionBridge,
} from "@paperclipai/adapter-utils/execution-target";
// Wrap the staging seam + both sandbox bridges in call-recording spies that
// still delegate to the real implementations. This copies the execute.test.ts
// harness verbatim so a startup test asserts the exact staging args and bridge
// hand-off the engine threads without changing any real behavior.
vi.mock("@paperclipai/adapter-utils/execution-target", async (importActual) => {
const actual = await importActual<typeof import("@paperclipai/adapter-utils/execution-target")>();
return {
...actual,
prepareAdapterExecutionTargetRuntime: vi.fn(actual.prepareAdapterExecutionTargetRuntime),
startAdapterExecutionTargetPaperclipBridge: vi.fn(actual.startAdapterExecutionTargetPaperclipBridge),
startAdapterExecutionTargetProcessSessionBridge: vi.fn(actual.startAdapterExecutionTargetProcessSessionBridge),
};
});
import { createAcpxEngineExecutor, type AcpxEngineExecutorOptions } from "./execute.js";
import { runChildProcess } from "../server-utils.js";
const tempRoots: string[] = [];
async function makeTempRoot() {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-acpx-skills-"));
tempRoots.push(root);
return root;
}
afterEach(async () => {
// A remote run stages a process-session bridge whose detached event writer can
// still be flushing a trailing event file into `.../process-sessions/<id>/events`
// when the run's own best-effort `client.remove(sessionDir)` (which production
// catch-wraps) has already returned. Under CI load that trailing write can land
// between this recursive delete's directory snapshot and its `rmdir`, surfacing as
// `ENOTEMPTY`. `maxRetries`/`retryDelay` make the cleanup ride out that window the
// same way production tolerates it, instead of failing the just-passed test.
await Promise.all(
tempRoots.splice(0).map((root) =>
fs.rm(root, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }),
),
);
});
async function pathExists(candidate: string): Promise<boolean> {
return fs.access(candidate).then(() => true).catch(() => false);
}
void pathExists;
function createLocalSandboxRunner(
onExecute?: (input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
}) => void,
) {
let counter = 0;
return {
execute: async (input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
stdin?: string;
timeoutMs?: number;
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
onSpawn?: (meta: { pid: number; startedAt: string }) => Promise<void>;
}) => {
counter += 1;
onExecute?.(input);
const command = input.command === "bash" ? "/bin/bash" : input.command;
return await runChildProcess(`acpx-sandbox-run-${counter}`, command, input.args ?? [], {
cwd: input.cwd ?? process.cwd(),
env: input.env ?? {},
stdin: input.stdin,
timeoutSec: Math.max(1, Math.ceil((input.timeoutMs ?? 30_000) / 1000)),
graceSec: 5,
onLog: input.onLog ?? (async () => {}),
onSpawn: input.onSpawn
? async (meta) => input.onSpawn?.({ pid: meta.pid, startedAt: meta.startedAt })
: undefined,
});
},
};
}
function buildRuntime(
onSetConfigOption?: (input: { key: string; value: string }) => void,
onEnsureSession?: (input: Record<string, unknown>) => void,
) {
return {
ensureSession: async (input: Record<string, unknown>) => {
onEnsureSession?.(input);
return ({
backendSessionId: "backend-session",
agentSessionId: "agent-session",
runtimeSessionName: "runtime-session",
});
},
startTurn: () => ({
events: (async function* () {
yield { type: "done", stopReason: "end_turn" };
})(),
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
cancel: async () => {},
}),
setConfigOption: async (input: { key: string; value: string }) => {
onSetConfigOption?.(input);
},
close: async () => {},
};
}
async function runExecutor(
config: Record<string, unknown>,
options: {
context?: Record<string, unknown>;
executionTransport?: Record<string, unknown>;
authToken?: string;
executionTarget?: Record<string, unknown>;
runtimeMcp?: AdapterRuntimeMcpAccess;
prepareRemoteManagedHome?: AcpxEngineExecutorOptions["prepareRemoteManagedHome"];
startupTraceContext?: AdapterExecutionContext["startupTraceContext"];
} = {},
) {
const runtimeOptions: Record<string, unknown>[] = [];
const configOptions: Array<{ key: string; value: string }> = [];
const sessionInputs: Record<string, unknown>[] = [];
const meta: Record<string, unknown>[] = [];
const logs: Array<{ stream: string; text: string }> = [];
const events: Array<{ eventType: string; payload?: Record<string, unknown> }> = [];
const execute = createAcpxEngineExecutor({
...(options.prepareRemoteManagedHome
? { prepareRemoteManagedHome: options.prepareRemoteManagedHome }
: {}),
createRuntime: (options) => {
runtimeOptions.push(options as unknown as Record<string, unknown>);
return buildRuntime(
({ key, value }) => configOptions.push({ key, value }),
(input) => sessionInputs.push(input),
) as never;
},
});
const result = await execute({
runId: "run-1",
agent: {
id: "agent-1",
companyId: "company-1",
},
runtime: {},
config,
context: options.context ?? {},
executionTransport: options.executionTransport,
authToken: options.authToken,
executionTarget: options.executionTarget,
runtimeMcp: options.runtimeMcp,
startupTraceContext: options.startupTraceContext,
onLog: async (stream: "stdout" | "stderr", text: string) => {
logs.push({ stream, text });
},
onMeta: async (payload: unknown) => {
meta.push(payload as Record<string, unknown>);
},
onEvent: async (event: { eventType: string; payload?: Record<string, unknown> }) => {
events.push(event);
},
} as never);
expect(result.exitCode).toBe(0);
return { logs, meta, events, runtimeOptions, configOptions, sessionInputs, result };
}
// The staging-seam describe helper from execute.test.ts (~:2152). It builds a
// runner-backed remote target: the local runner extracts the staged tar into
// `remoteCwd`, so the run really ships the HOST worktree into the sandbox.
async function setupRemoteSandbox() {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const localCwd = path.join(root, "worktree");
const remoteCwd = path.join(root, "remote-workspace");
await fs.mkdir(localCwd, { recursive: true });
await fs.mkdir(remoteCwd, { recursive: true });
// A file present only in the HOST worktree proves the workspace is shipped
// into the sandbox: the local runner extracts the staged tar into remoteCwd.
await fs.writeFile(path.join(localCwd, "hello.txt"), "hi", "utf8");
const runner = createLocalSandboxRunner();
const executionTarget = {
kind: "remote",
transport: "sandbox",
providerKey: "fake-plugin",
remoteCwd,
runner,
};
return { root, stateDir, localCwd, remoteCwd, executionTarget };
}
// Read the `configFingerprint` off a settled run result.
function fpOf(result: { sessionParams?: unknown }): string | undefined {
return (result.sessionParams as { configFingerprint?: string } | undefined)?.configFingerprint;
}
const okHandle = {
backendSessionId: "backend-session",
agentSessionId: "agent-session",
runtimeSessionName: "runtime-session",
};
function completedTurn() {
return {
events: (async function* () {
yield { type: "done", stopReason: "end_turn" };
})(),
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
cancel: async () => {},
};
}
describe("ACPX engine startup characterization", () => {
// Item 1 + 8: the remote launch env and its finalization point.
describe("remote launch environment values", () => {
beforeEach(() => vi.clearAllMocks());
it("mints the bridge launch env into the process-session command payload", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
// Decode the process-session LAUNCH payload (the base64 command blob). The
// in-sandbox process env rides there, not in the exec's own `env`.
let launchPayload: Record<string, unknown> | null = null;
(executionTarget as { runner: unknown }).runner = createLocalSandboxRunner((input) => {
if (input.env?.PAPERCLIP_SANDBOX_EXEC_CHANNEL === "bridge") {
const script = input.args?.[1] ?? "";
const match = script.match(/PAPERCLIP_PROCESS_SESSION_COMMAND_B64='([^']+)'/);
if (match) {
launchPayload = JSON.parse(Buffer.from(match[1]!, "base64").toString("utf8")) as Record<
string,
unknown
>;
}
}
});
await runExecutor(
{ agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
{ authToken: "real-run-jwt", executionTarget },
);
// The launch payload carries the MERGED paperclip bridge env: the queue
// transport mode, a loopback bridge base URL, and a minted bridge token.
const payloadEnv = ((launchPayload as Record<string, unknown> | null)?.env ?? {}) as Record<
string,
unknown
>;
expect(payloadEnv).toMatchObject({ PAPERCLIP_API_BRIDGE_MODE: "queue_v1" });
expect(String(payloadEnv.PAPERCLIP_API_URL ?? "")).toMatch(/^http:\/\/127\.0\.0\.1:\d+$/);
// The minted bridge token is present and is NOT the host run JWT.
expect(payloadEnv.PAPERCLIP_API_KEY).toBeTruthy();
expect(payloadEnv.PAPERCLIP_API_KEY).not.toBe("real-run-jwt");
});
it("finalizes the launch env at the bridge merge: the process env carries the merged bridge values", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
// Capture the launch payload AND the bridge-channel exec's own `env`.
let launchPayload: Record<string, unknown> | null = null;
let bridgeExecEnv: Record<string, string> | undefined;
(executionTarget as { runner: unknown }).runner = createLocalSandboxRunner((input) => {
if (input.env?.PAPERCLIP_SANDBOX_EXEC_CHANNEL === "bridge") {
bridgeExecEnv = input.env;
const script = input.args?.[1] ?? "";
const match = script.match(/PAPERCLIP_PROCESS_SESSION_COMMAND_B64='([^']+)'/);
if (match) {
launchPayload = JSON.parse(Buffer.from(match[1]!, "base64").toString("utf8")) as Record<
string,
unknown
>;
}
}
});
await runExecutor(
{ agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
{ authToken: "real-run-jwt", executionTarget },
);
// The launch payload is the finalized carrier of the process env. It already
// holds the merged bridge values at the point the run emits it, so no later
// write mutates the process env after the bridge merge.
const payloadEnv = ((launchPayload as Record<string, unknown> | null)?.env ?? {}) as Record<
string,
unknown
>;
expect(payloadEnv.PAPERCLIP_API_BRIDGE_MODE).toBe("queue_v1");
expect(payloadEnv.PAPERCLIP_API_KEY).toBeTruthy();
// The bridge-channel exec's OWN env is the sandbox transport channel, not the
// agent process env: it does not carry the minted agent bridge key. This pins
// that the merged agent env lives only in the finalized launch payload.
expect(bridgeExecEnv?.PAPERCLIP_SANDBOX_EXEC_CHANNEL).toBe("bridge");
expect(bridgeExecEnv?.PAPERCLIP_API_KEY).toBeUndefined();
});
});
// Item 2: the 17 fingerprint fields folded into `configFingerprint`, and the
// outer session key form that embeds it.
describe("session fingerprint and session key", () => {
beforeEach(() => vi.clearAllMocks());
it("forms the session key as paperclip:company:agent:taskKey:fingerprint and embeds the fingerprint", async () => {
const root = await makeTempRoot();
const { result } = await runExecutor({
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir: path.join(root, "state"),
cwd: path.join(root, "workspace"),
});
const fp = fpOf(result);
expect(fp).toBeTypeOf("string");
expect(fp).toBeTruthy();
// No taskId/issueId/workspaceId in the default context, so taskKey is "default".
const sessionKey = (result.sessionParams as { sessionKey?: string }).sessionKey;
expect(sessionKey).toBe(`paperclip:company-1:agent-1:default:${fp}`);
});
it("keeps the fingerprint stable across two identical runs and a same-config new wake", async () => {
const root = await makeTempRoot();
const baseConfig = {
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir: path.join(root, "state"),
cwd: path.join(root, "workspace"),
};
const first = await runExecutor(baseConfig, {
context: { taskId: "issue-1", wakeReason: "issue_assigned" },
});
const identical = await runExecutor(baseConfig, {
context: { taskId: "issue-1", wakeReason: "issue_assigned" },
});
// A fresh heartbeat with a different wake reason but the same config env.
const newWake = await runExecutor(baseConfig, {
context: { taskId: "issue-1", wakeReason: "comment", wakeCommentId: "c-9" },
});
expect(fpOf(first.result)).toBeTruthy();
expect(fpOf(identical.result)).toBe(fpOf(first.result));
// Per-wake PAPERCLIP_* churn does not reset the session fingerprint.
expect(fpOf(newWake.result)).toBe(fpOf(first.result));
});
it("busts the fingerprint when any representative folded dimension changes", async () => {
const root = await makeTempRoot();
const cwd = path.join(root, "workspace");
const stateDir = path.join(root, "state");
const context = { context: { taskId: "issue-1", wakeReason: "issue_assigned" } };
const base = { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd };
const baseFp = fpOf((await runExecutor(base, context)).result);
expect(baseFp).toBeTruthy();
// Each edit changes exactly one folded dimension and must bust the fingerprint.
// agentCommand.
expect(fpOf((await runExecutor({ ...base, agentCommand: "node ./other-acp.js" }, context)).result)).not.toBe(baseFp);
// cwd.
expect(fpOf((await runExecutor({ ...base, cwd: path.join(root, "other-cwd") }, context)).result)).not.toBe(baseFp);
// requestedModel.
expect(fpOf((await runExecutor({ ...base, model: "some-model" }, context)).result)).not.toBe(baseFp);
// requestedThinkingEffort.
expect(fpOf((await runExecutor({ ...base, thinkingEffort: "high" }, context)).result)).not.toBe(baseFp);
// mode.
expect(fpOf((await runExecutor({ ...base, mode: "oneshot" }, context)).result)).not.toBe(baseFp);
// adapterEnvHash (a resolved adapter env value).
expect(fpOf((await runExecutor({ ...base, env: { FOO: "bar" } }, context)).result)).not.toBe(baseFp);
// mcpServers identity (injected runtime MCP set).
expect(
fpOf(
(
await runExecutor(base, {
...context,
runtimeMcp: {
getServers: () => [
{ name: "github", url: "https://x.test/mcp", connectionId: "c-1", token: "t-1" },
],
},
})
).result,
),
).not.toBe(baseFp);
// secretManifestHash.
expect(
fpOf(
(
await runExecutor(base, {
...context,
context: {
taskId: "issue-1",
wakeReason: "issue_assigned",
paperclipSecrets: {
manifest: [
{
configPath: "env.API_TOKEN",
envKey: "API_TOKEN",
secretId: "secret-1",
bindingId: "binding-1",
secretKey: "api-token",
version: 1,
provider: "local_encrypted",
},
],
},
},
})
).result,
),
).not.toBe(baseFp);
// additionalSourcesIdentity (referenced-project set).
expect(
fpOf(
(
await runExecutor(base, {
...context,
context: {
taskId: "issue-1",
wakeReason: "issue_assigned",
paperclipWorkspace: {
cwd,
realization: {
additional: [
{
path: "/host/project-a",
projectId: "a",
projectWorkspaceId: "ws-a",
repoUrl: "https://example.test/a.git",
repoRef: "ref-a-1",
},
],
},
},
},
})
).result,
),
).not.toBe(baseFp);
// fastMode folds only for codex, so pin it with a codex-vs-codex pair.
const codexBase = { agent: "codex", agentCommand: "node ./fake-acp.js", stateDir, cwd };
const codexFp = fpOf((await runExecutor(codexBase, context)).result);
expect(fpOf((await runExecutor({ ...codexBase, fastMode: true }, context)).result)).not.toBe(codexFp);
});
});
// Item 3: the staging seam call, its arguments, and its order (workspace then
// assets, serial). Modeled on the PR-1 staging-seam tests.
describe("staging seam calls, arguments, and order", () => {
beforeEach(() => vi.clearAllMocks());
it("stages the host workspace with no assets, exactly once, before the process launch", async () => {
const { stateDir, localCwd, remoteCwd, executionTarget } = await setupRemoteSandbox();
const { sessionInputs, events } = await runExecutor(
{ agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
{ authToken: "real-run-jwt", executionTarget },
);
// The staging seam crossed exactly once.
expect(vi.mocked(prepareAdapterExecutionTargetRuntime)).toHaveBeenCalledTimes(1);
const stageArgs = vi.mocked(prepareAdapterExecutionTargetRuntime).mock.calls[0]![0];
// The HOST worktree is shipped first; no per-adapter home asset in this lane.
expect(stageArgs.workspaceLocalDir).toBe(localCwd);
expect(stageArgs.assets ?? []).toEqual([]);
expect(stageArgs.installCommand ?? null).toBeNull();
expect(stageArgs.target).toMatchObject({ kind: "remote", transport: "sandbox" });
// The workspace really landed in the sandbox workspace dir.
await expect(fs.readFile(path.join(remoteCwd, "hello.txt"), "utf8")).resolves.toBe("hi");
// A per-step timing event proves the sync ran inside its timed boundary.
const stageEvent = events.find(
(event) => event.eventType === "run.startup.step" && event.payload?.step === "stage.sync",
);
expect(stageEvent).toBeTruthy();
// And session/new binds to the in-sandbox workspace cwd the seam returned.
expect(sessionInputs[0]?.cwd).toBe(remoteCwd);
});
it("threads a managed-home asset through the same seam after the workspace", async () => {
const { root, stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
const managedHomeDir = path.join(root, "managed-home");
await fs.mkdir(managedHomeDir, { recursive: true });
await fs.writeFile(path.join(managedHomeDir, "config.json"), "{}", "utf8");
await runExecutor(
{ agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
{
authToken: "real-run-jwt",
executionTarget,
prepareRemoteManagedHome: async (input) => {
const stagedRuntime = await input.stage([
{ key: "home", localDir: managedHomeDir, followSymlinks: true },
]);
return { stagedRuntime };
},
},
);
// The seam's home asset is threaded through the SAME staging seam, keyed by
// the workspace-local dir. Workspace ships first; the asset rides alongside it.
const stageArgs = vi.mocked(prepareAdapterExecutionTargetRuntime).mock.calls[0]![0];
expect(stageArgs.workspaceLocalDir).toBe(localCwd);
expect(stageArgs.assets).toEqual([
{ key: "home", localDir: managedHomeDir, followSymlinks: true },
]);
});
});
// Item 4: the two-bridge overlap and the ACP-initialization ordering.
describe("two-bridge overlap and ACP initialization order", () => {
beforeEach(() => vi.clearAllMocks());
it("defers the process bridge env, shares one runtimeRootDir, and runs session/new on the sandbox cwd", async () => {
const { stateDir, localCwd, remoteCwd, executionTarget } = await setupRemoteSandbox();
const { sessionInputs, runtimeOptions } = await runExecutor(
{ agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
{ authToken: "real-run-jwt", executionTarget },
);
// The process-session bridge receives its launch env as a DEFERRED thunk, the
// seam that lets its env-independent setup overlap the paperclip bridge start.
const processArgs = vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mock.calls[0]![0];
expect(typeof processArgs.env).toBe("function");
// Both bridges receive the SAME real (non-null) runtimeRootDir from staging.
const paperclipArgs = vi.mocked(startAdapterExecutionTargetPaperclipBridge).mock.calls[0]![0];
expect(paperclipArgs.runtimeRootDir).toBeTruthy();
expect(String(paperclipArgs.runtimeRootDir)).toContain(".paperclip-runtime");
expect(processArgs.runtimeRootDir).toBe(paperclipArgs.runtimeRootDir);
// The ACP runtime + session/new both bind to the in-sandbox workspace cwd,
// which the run resolves only after the bridges bring the sandbox up.
expect(runtimeOptions[0]?.cwd).toBe(remoteCwd);
expect(sessionInputs[0]?.cwd).toBe(remoteCwd);
expect(sessionInputs[0]?.cwd).not.toBe(localCwd);
});
});
// Item 5 + 6: every startup exit path, its result phase, and the cleanup-call
// set (bridges stop / lease releases / runtime closes).
describe("startup exit paths and cleanup", () => {
beforeEach(() => vi.clearAllMocks());
it("create_runtime failure: settles an error result, stops both bridges, releases the lease", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
const paperclipStop = vi.fn(async () => {});
const processStop = vi.fn(async () => {});
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementationOnce(
async () => ({ env: {}, stop: paperclipStop }) as never,
);
vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mockImplementationOnce(
async () => ({ agentCommand: null, stop: processStop }) as never,
);
const stagingLocks = new Map<string, Promise<unknown>>();
const execute = createAcpxEngineExecutor({
stagingLocks,
warmHandles: new Map(),
stagedRuntimes: new Map(),
createRuntime: () => {
throw new Error("createRuntime boom");
},
});
const result = await execute({
runId: "run-create-fail-remote",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
context: {},
authToken: "real-run-jwt",
executionTarget,
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
} as never);
// The post-build runtime-creation failure returns a settled error result.
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("create_runtime");
// Both live bridges stop exactly once and the per-session lease releases.
expect(paperclipStop).toHaveBeenCalledTimes(1);
expect(processStop).toHaveBeenCalledTimes(1);
expect(stagingLocks.size).toBe(0);
});
it("partial-bridge failure: throws and stops the concurrently-started bridge exactly once", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
const stop = vi.fn(async () => {});
vi.mocked(startAdapterExecutionTargetPaperclipBridge).mockImplementationOnce(async () => {
throw new Error("paperclip bridge boom");
});
vi.mocked(startAdapterExecutionTargetProcessSessionBridge).mockImplementationOnce(
async () => ({ agentCommand: null, stop }) as never,
);
const execute = createAcpxEngineExecutor({
createRuntime: () => buildRuntime() as never,
});
// A partial bridge failure inside buildRuntime is one of the only two throw
// paths, so the run rethrows instead of settling a result.
await expect(
execute({
runId: "run-bridge-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd: localCwd },
context: {},
authToken: "real-run-jwt",
executionTarget,
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
} as never),
).rejects.toThrow("paperclip bridge boom");
// The concurrently-started process-session bridge was stopped exactly once.
expect(stop).toHaveBeenCalledTimes(1);
});
it("cold ensure_session throw: ensure_session error, and the runtime closes (the cold-handshake leak is fixed)", async () => {
const root = await makeTempRoot();
const closeSpy = vi.fn(async () => {});
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
ensureSession: async () => {
throw new Error("ensureSession boom");
},
startTurn: () => completedTurn(),
close: closeSpy,
}) as never,
});
const result = await execute({
runId: "handshake-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("ensure_session");
// The runtime enters the ledger the moment it is created, so a cold
// `ensureSession` throw (before any handle exists) still closes it through the
// settlement `endSession` step. This closes the former cold-handshake leak.
expect(closeSpy).toHaveBeenCalledTimes(1);
});
it("missing session handle: ensure_session error and the minimal runtime closes", async () => {
const root = await makeTempRoot();
const closeSpy = vi.fn(async () => {});
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
// A runtime that returns no session handle drives the missing-handle path.
ensureSession: async () => undefined,
startTurn: () => completedTurn(),
close: closeSpy,
}) as never,
});
const result = await execute({
runId: "missing-handle",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir: path.join(root, "state") },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("ensure_session");
expect(result.errorCode).toBe("acpx_runtime_error");
expect(closeSpy).toHaveBeenCalledTimes(1);
});
it("configure_session failure: configure_session error and the runtime closes", async () => {
const root = await makeTempRoot();
const closeSpy = vi.fn(async () => {});
const execute = createAcpxEngineExecutor({
createRuntime: () =>
({
ensureSession: async () => okHandle,
// Gemini model/effort drive a session config option; a throwing setter
// fails the configure_session phase after the handshake succeeds.
setConfigOption: async () => {
throw new Error("setConfigOption boom");
},
startTurn: () => completedTurn(),
close: closeSpy,
}) as never,
});
const result = await execute({
runId: "configure-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: {
agent: "gemini",
model: "gemini-2.5-pro",
thinkingEffort: "high",
stateDir: path.join(root, "state"),
},
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.resultJson?.phase).toBe("configure_session");
expect(closeSpy).toHaveBeenCalledTimes(1);
});
it("persistent host failure: the first run closes and relaunches, so a failing second run re-creates and closes", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const startedAt = "2026-01-01T00:00:00.000Z";
const closeSpy = vi.fn(async () => {});
let created = 0;
const warmHandles = new Map();
const execute = createAcpxEngineExecutor({
warmHandles,
createRuntime: (options) => {
created += 1;
const opts = options as AcpRuntimeOptions & {
onAgentSpawn?: (meta: { pid: number; startedAt: string }) => Promise<void>;
};
return {
ensureSession: async () => {
await opts.onAgentSpawn?.({ pid: 4242, startedAt });
return okHandle;
},
startTurn: () => completedTurn(),
close: closeSpy,
} as never;
},
});
const config = {
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir,
mode: "persistent",
warmHandleIdleMs: 60_000,
};
const first = await execute({
runId: "warm-1",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onSpawn: async () => {},
} as never);
expect(first.exitCode).toBe(0);
// Amendment B: the first clean persistent turn closes and relaunches instead
// of warm-saving the runtime, so no warm handle survives.
expect(warmHandles.size).toBe(0);
expect(created).toBe(1);
// The second run finds no warm handle, so it re-creates the runtime and fails
// while persisting process identity during ensure_session.
const second = await execute({
runId: "warm-2",
agent: { id: "agent-1", companyId: "company-1" },
runtime: { sessionParams: first.sessionParams },
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onSpawn: async () => {
throw new Error("onSpawn boom");
},
} as never);
// The second run built runtime #2 (no warm reuse), closed it on the failure,
// and reported the ensure_session phase. Both runtimes closed (one per run).
expect(created).toBe(2);
expect(second.exitCode).toBe(1);
expect(second.resultJson?.phase).toBe("ensure_session");
expect(closeSpy).toHaveBeenCalledTimes(2);
expect(warmHandles.size).toBe(0);
});
});
// Item 7: the per-lane resource set.
describe("per-lane resource set", () => {
beforeEach(() => vi.clearAllMocks());
it("local lane: crosses no staging seam, starts no bridge, keeps session/new on the host cwd", async () => {
const root = await makeTempRoot();
const localCwd = path.join(root, "worktree");
await fs.mkdir(localCwd, { recursive: true });
const { sessionInputs, runtimeOptions } = await runExecutor({
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir: path.join(root, "state"),
cwd: localCwd,
});
expect(vi.mocked(prepareAdapterExecutionTargetRuntime)).not.toHaveBeenCalled();
expect(vi.mocked(startAdapterExecutionTargetPaperclipBridge)).not.toHaveBeenCalled();
expect(vi.mocked(startAdapterExecutionTargetProcessSessionBridge)).not.toHaveBeenCalled();
expect(sessionInputs[0]?.cwd).toBe(localCwd);
expect(runtimeOptions[0]?.cwd).toBe(localCwd);
});
it("persistent host lane: closes and relaunches the handle, so a second run re-creates the runtime", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
let created = 0;
const warmHandles = new Map();
const execute = createAcpxEngineExecutor({
warmHandles,
createRuntime: () => {
created += 1;
return buildRuntime() as never;
},
});
const config = {
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir,
mode: "persistent",
warmHandleIdleMs: 60_000,
};
const base = {
agent: { id: "agent-1", companyId: "company-1" },
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
onSpawn: async () => {},
};
const first = await execute({ runId: "host-warm-1", runtime: {}, ...base } as never);
const second = await execute({
runId: "host-warm-2",
runtime: { sessionParams: first.sessionParams },
...base,
} as never);
expect(first.exitCode).toBe(0);
expect(second.exitCode).toBe(0);
// Amendment B: the host lane never warm-saves a live runtime (the run-minted
// API key is never revoked), so the first run closes and relaunches. No warm
// handle survives, so the second run re-creates runtime #2.
expect(created).toBe(2);
expect(warmHandles.size).toBe(0);
});
it("remote process-session lane: does NOT warm-save the handle, so a second run re-creates the runtime", async () => {
const { stateDir, localCwd, executionTarget } = await setupRemoteSandbox();
let created = 0;
const ensureInputs: Array<Record<string, unknown>> = [];
const warmHandles = new Map();
const execute = createAcpxEngineExecutor({
warmHandles,
stagedRuntimes: new Map(),
stagingLocks: new Map(),
createRuntime: () => {
created += 1;
return buildRuntime(undefined, (input) => ensureInputs.push(input)) as never;
},
});
const base = {
agent: { id: "agent-1", companyId: "company-1" },
config: {
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir,
cwd: localCwd,
mode: "persistent",
warmHandleIdleMs: 60_000,
},
context: {},
authToken: "real-run-jwt",
executionTarget,
onLog: async () => {},
onMeta: async () => {},
onEvent: async () => {},
};
const first = await execute({ runId: "remote-warm-1", runtime: {}, ...base } as never);
const second = await execute({
runId: "remote-warm-2",
runtime: { sessionParams: first.sessionParams },
...base,
} as never);
expect(first.exitCode).toBe(0);
expect(second.exitCode).toBe(0);
// The remote process-session lane never keeps the handle warm, so the second
// run re-creates the runtime and runs a fresh handshake instead of reusing one.
expect(created).toBe(2);
expect(warmHandles.size).toBe(0);
expect(ensureInputs).toHaveLength(2);
});
});
});
@@ -1,686 +0,0 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import type { AdapterExecutionContext, AdapterRuntimeMcpAccess } from "@paperclipai/adapter-utils";
import {
prepareAdapterExecutionTargetRuntime,
startAdapterExecutionTargetPaperclipBridge,
startAdapterExecutionTargetProcessSessionBridge,
} from "@paperclipai/adapter-utils/execution-target";
// Wrap the staging seam + both sandbox bridges in call-recording spies that
// still delegate to the real implementations. This mirrors the execute.test.ts
// harness so the turn characterization tests share the same mocked module graph.
vi.mock("@paperclipai/adapter-utils/execution-target", async (importActual) => {
const actual = await importActual<typeof import("@paperclipai/adapter-utils/execution-target")>();
return {
...actual,
prepareAdapterExecutionTargetRuntime: vi.fn(actual.prepareAdapterExecutionTargetRuntime),
startAdapterExecutionTargetPaperclipBridge: vi.fn(actual.startAdapterExecutionTargetPaperclipBridge),
startAdapterExecutionTargetProcessSessionBridge: vi.fn(actual.startAdapterExecutionTargetProcessSessionBridge),
};
});
import {
createAcpxEngineExecutor,
summarizeAcpxTurnUsage,
type AcpxEngineExecutorOptions,
} from "./execute.js";
import { runChildProcess } from "../server-utils.js";
const tempRoots: string[] = [];
async function makeTempRoot() {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-acpx-skills-"));
tempRoots.push(root);
return root;
}
afterEach(async () => {
// A remote run stages a process-session bridge whose detached event writer can
// still be flushing a trailing event file into `.../process-sessions/<id>/events`
// when the run's own best-effort `client.remove(sessionDir)` (which production
// catch-wraps) has already returned. Under CI load that trailing write can land
// between this recursive delete's directory snapshot and its `rmdir`, surfacing as
// `ENOTEMPTY`. `maxRetries`/`retryDelay` make the cleanup ride out that window the
// same way production tolerates it, instead of failing the just-passed test.
await Promise.all(
tempRoots.splice(0).map((root) =>
fs.rm(root, { recursive: true, force: true, maxRetries: 5, retryDelay: 50 }),
),
);
});
function createLocalSandboxRunner(
onExecute?: (input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
}) => void,
) {
let counter = 0;
return {
execute: async (input: {
command: string;
args?: string[];
cwd?: string;
env?: Record<string, string>;
stdin?: string;
timeoutMs?: number;
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
onSpawn?: (meta: { pid: number; startedAt: string }) => Promise<void>;
}) => {
counter += 1;
onExecute?.(input);
const command = input.command === "bash" ? "/bin/bash" : input.command;
return await runChildProcess(`acpx-sandbox-run-${counter}`, command, input.args ?? [], {
cwd: input.cwd ?? process.cwd(),
env: input.env ?? {},
stdin: input.stdin,
timeoutSec: Math.max(1, Math.ceil((input.timeoutMs ?? 30_000) / 1000)),
graceSec: 5,
onLog: input.onLog ?? (async () => {}),
onSpawn: input.onSpawn
? async (meta) => input.onSpawn?.({ pid: meta.pid, startedAt: meta.startedAt })
: undefined,
});
},
};
}
function buildRuntime(
onSetConfigOption?: (input: { key: string; value: string }) => void,
onEnsureSession?: (input: Record<string, unknown>) => void,
) {
return {
ensureSession: async (input: Record<string, unknown>) => {
onEnsureSession?.(input);
return ({
backendSessionId: "backend-session",
agentSessionId: "agent-session",
runtimeSessionName: "runtime-session",
});
},
startTurn: () => ({
events: (async function* () {
yield { type: "done", stopReason: "end_turn" };
})(),
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
cancel: async () => {},
}),
setConfigOption: async (input: { key: string; value: string }) => {
onSetConfigOption?.(input);
},
close: async () => {},
};
}
async function runExecutor(
config: Record<string, unknown>,
options: {
context?: Record<string, unknown>;
executionTransport?: Record<string, unknown>;
authToken?: string;
executionTarget?: Record<string, unknown>;
runtimeMcp?: AdapterRuntimeMcpAccess;
prepareRemoteManagedHome?: AcpxEngineExecutorOptions["prepareRemoteManagedHome"];
startupTraceContext?: AdapterExecutionContext["startupTraceContext"];
} = {},
) {
const runtimeOptions: Record<string, unknown>[] = [];
const configOptions: Array<{ key: string; value: string }> = [];
const sessionInputs: Record<string, unknown>[] = [];
const meta: Record<string, unknown>[] = [];
const logs: Array<{ stream: string; text: string }> = [];
const events: Array<{ eventType: string; payload?: Record<string, unknown> }> = [];
const execute = createAcpxEngineExecutor({
...(options.prepareRemoteManagedHome
? { prepareRemoteManagedHome: options.prepareRemoteManagedHome }
: {}),
createRuntime: (options) => {
runtimeOptions.push(options as unknown as Record<string, unknown>);
return buildRuntime(
({ key, value }) => configOptions.push({ key, value }),
(input) => sessionInputs.push(input),
) as never;
},
});
const result = await execute({
runId: "run-1",
agent: {
id: "agent-1",
companyId: "company-1",
},
runtime: {},
config,
context: options.context ?? {},
executionTransport: options.executionTransport,
authToken: options.authToken,
executionTarget: options.executionTarget,
runtimeMcp: options.runtimeMcp,
startupTraceContext: options.startupTraceContext,
onLog: async (stream: "stdout" | "stderr", text: string) => {
logs.push({ stream, text });
},
onMeta: async (payload: unknown) => {
meta.push(payload as Record<string, unknown>);
},
onEvent: async (event: { eventType: string; payload?: Record<string, unknown> }) => {
events.push(event);
},
} as never);
expect(result.exitCode).toBe(0);
return { logs, meta, events, runtimeOptions, configOptions, sessionInputs, result };
}
// A stub ACP runtime that yields a controlled event stream and terminal result.
// The tests supply the events/result/cancel behavior; the harness above only
// drives always-happy sessions, so the turn tests build their own runtime here.
function turnRuntime(input: {
events: () => AsyncGenerator<Record<string, unknown>>;
result: Promise<Record<string, unknown>>;
onStartTurn?: (turnInput: Record<string, unknown>) => void;
onCancel?: (reason: string) => void;
onClose?: () => void;
getStatus?: () => Promise<Record<string, unknown>>;
onEnsureSession?: (session: Record<string, unknown>) => void;
ensureSession?: (session: Record<string, unknown>) => Promise<Record<string, unknown>>;
}) {
const runtime: Record<string, unknown> = {
ensureSession:
input.ensureSession ??
(async (session: Record<string, unknown>) => {
input.onEnsureSession?.(session);
return {
backendSessionId: "backend-session",
agentSessionId: "agent-session",
runtimeSessionName: "runtime-session",
};
}),
startTurn: (turnInput: Record<string, unknown>) => {
input.onStartTurn?.(turnInput);
return {
events: input.events(),
result: input.result,
cancel: async ({ reason }: { reason: string }) => {
input.onCancel?.(reason);
},
};
},
close: async () => {
input.onClose?.();
},
};
if (input.getStatus) runtime.getStatus = input.getStatus;
return runtime;
}
describe("ACPX engine turn characterization", () => {
// The stub session handle every ensureSession returns. startTurn must receive
// this exact handle object.
const SESSION_HANDLE = {
backendSessionId: "backend-session",
agentSessionId: "agent-session",
runtimeSessionName: "runtime-session",
};
it("passes the turn inputs and terminal failure callback to startTurn", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
let captured: Record<string, unknown> | null = null;
let metaPrompt = "";
const execute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
onStartTurn: (turnInput) => {
captured = turnInput;
},
events: async function* () {
yield { type: "done", stopReason: "end_turn" };
},
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
}) as never,
});
const result = await execute({
runId: "run-six-inputs",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, timeoutSec: 120 },
context: {},
onLog: async () => {},
onMeta: async (payload: { prompt?: string }) => {
metaPrompt = payload.prompt ?? "";
},
} as never);
expect(result.exitCode).toBe(0);
const input = captured!;
// The handle is the exact session handle ensureSession returned.
expect(input.handle).toEqual(SESSION_HANDLE);
// The text is the built run prompt, the same string reported to onMeta.
expect(input.text).toBe(metaPrompt);
expect(typeof input.text).toBe("string");
expect((input.text as string).length).toBeGreaterThan(0);
// The turn always runs in prompt mode.
expect(input.mode).toBe("prompt");
// The request id is the run id.
expect(input.requestId).toBe("run-six-inputs");
// A positive timeoutSec becomes timeoutMs in milliseconds.
expect(input.timeoutMs).toBe(120_000);
// The abort signal is present and not yet aborted.
const signal = input.signal as AbortSignal;
expect(signal).toBeInstanceOf(AbortSignal);
expect(signal.aborted).toBe(false);
// Every adapter receives diagnostics, even without a failure classifier.
expect(input.onTerminalSessionFailure).toBeTypeOf("function");
expect(Object.keys(input).sort()).toEqual(
["handle", "mode", "onTerminalSessionFailure", "requestId", "signal", "text", "timeoutMs"].sort(),
);
});
it("joins text_delta events into the trimmed result summary", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const execute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
events: async function* () {
yield { type: "text_delta", text: " Hello, ", stream: "output", tag: "agent_message_chunk" };
yield { type: "text_delta", text: "world ", stream: "output", tag: "agent_message_chunk" };
yield { type: "done", stopReason: "end_turn" };
},
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
}) as never,
});
const result = await execute({
runId: "run-summary",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(0);
// The summary is the concatenation of the deltas, trimmed.
expect(result.summary).toBe("Hello, world");
});
it("falls back to the stop reason for the summary when no text streams", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const execute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
events: async function* () {
yield { type: "done", stopReason: "end_turn" };
},
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
}) as never,
});
const result = await execute({
runId: "run-no-text",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(0);
// With no streamed text, the summary is the terminal stop reason.
expect(result.summary).toBe("end_turn");
});
it("pins one event sequence across the log transcript and structured event transports", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const logs: Array<{ stream: string; text: string }> = [];
const events: Array<{ eventType: string; payload?: Record<string, unknown> }> = [];
const execute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
events: async function* () {
yield { type: "text_delta", text: "streamed hello", stream: "output", tag: "agent_message_chunk" };
yield { type: "done", stopReason: "end_turn" };
},
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
}) as never,
});
const result = await execute({
runId: "run-transports",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir },
context: {},
onLog: async (stream: "stdout" | "stderr", text: string) => {
logs.push({ stream, text });
},
onMeta: async () => {},
onEvent: async (event: { eventType: string; payload?: Record<string, unknown> }) => {
events.push(event);
},
} as never);
expect(result.exitCode).toBe(0);
// Transport 1 — the stdout transcript carries the text_delta as an acpx record.
expect(logs).toContainEqual({
stream: "stdout",
text: `${JSON.stringify({
type: "acpx.text_delta",
text: "streamed hello",
channel: "output",
tag: "agent_message_chunk",
})}\n`,
});
// The joined text also lands in the result summary.
expect(result.summary).toBe("streamed hello");
// Transport 2 — the structured onEvent stream reflects the run bring-up
// sequence as run.startup.step events, including the acp handshake.
const steps = events.filter((event) => event.eventType === "run.startup.step");
const stepNames = steps.map((event) => String(event.payload?.step));
expect(stepNames).toContain("acp.handshake");
expect(stepNames).toContain("workspace.resolve");
});
it("aborts a hung turn on the wall-clock timer and cancels with the timeout message", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const cwd = path.join(root, "worktree");
await fs.mkdir(cwd, { recursive: true });
const cancelReasons: string[] = [];
let releaseTurn: (() => void) | null = null;
const turnCancelled = new Promise<void>((resolve) => {
releaseTurn = resolve;
});
const execute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
// The stream never yields on its own. Only the wall-clock timer's cancel
// unblocks it, simulating a hung run.
events: async function* () {
await turnCancelled;
},
result: turnCancelled.then(() => ({ status: "cancelled", stopReason: "cancelled" })),
onCancel: (reason) => {
cancelReasons.push(reason);
releaseTurn?.();
},
}) as never,
});
const result = await execute({
runId: "run-timeout",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd, timeoutSec: 1 },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
const expectedMessage =
"Run exceeded the adapter execution timeout (timeoutSec=1, configured via adapterConfig.timeoutSec). " +
"Set adapterConfig.timeoutSec to raise it.";
expect(result.timedOut).toBe(true);
expect(result.signal).toBe("SIGTERM");
expect(result.errorCode).toBe("acpx_timeout");
expect(result.errorMessage).toBe(expectedMessage);
// The cancel ran with the formatted timeout message.
expect(cancelReasons).toContain(expectedMessage);
}, 15_000);
it("cancels the turn before closing the runtime when the turn throws", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const order: string[] = [];
const execute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
// The event stream throws mid-turn, so the catch path runs teardown.
events: async function* () {
throw new Error("turn boom");
},
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
onCancel: () => order.push("cancel"),
onClose: () => order.push("close"),
}) as never,
});
const result = await execute({
runId: "run-throw",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
// A failure after startTurn returned reports phase "turn".
expect((result.resultJson as Record<string, unknown>)?.phase).toBe("turn");
// Cancel runs before close, in that exact order.
expect(order).toEqual(["cancel", "close"]);
});
it("retries the session resume once and never retries the turn", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const cwd = path.join(root, "worktree");
await fs.mkdir(cwd, { recursive: true });
const config = { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir, cwd };
const ensureInputs: Array<Record<string, unknown>> = [];
let startTurnCalls = 0;
let resumeLogged = false;
// A first clean run mints the session params the second run resumes from.
const firstExecute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
onEnsureSession: (session) => ensureInputs.push(session),
onStartTurn: () => {
startTurnCalls += 1;
},
events: async function* () {
yield { type: "done", stopReason: "end_turn" };
},
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
}) as never,
});
const first = await firstExecute({
runId: "run-resume-a",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config,
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(first.exitCode).toBe(0);
// The second run resumes. Its first ensureSession (the resume) fails with a
// resume-shaped error; the fresh retry (no resumeSessionId) succeeds.
const secondExecute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
ensureSession: async (session: Record<string, unknown>) => {
ensureInputs.push(session);
if (session.resumeSessionId) {
throw new Error("resume session not found");
}
return {
backendSessionId: "backend-session",
agentSessionId: "agent-session",
runtimeSessionName: "runtime-session",
};
},
onStartTurn: () => {
startTurnCalls += 1;
},
events: async function* () {
yield { type: "done", stopReason: "end_turn" };
},
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
}) as never,
});
const second = await secondExecute({
runId: "run-resume-b",
agent: { id: "agent-1", companyId: "company-1" },
runtime: { sessionParams: (first as { sessionParams?: unknown }).sessionParams },
config,
context: {},
onLog: async (_stream: string, text: string) => {
if (text.includes("is unavailable; retrying with a fresh session")) resumeLogged = true;
},
onMeta: async () => {},
} as never);
expect(second.exitCode).toBe(0);
// The first run's single ensureSession, plus the second run's resume and
// fresh retry, make three ensureSession calls total; two on the second run.
expect(ensureInputs).toHaveLength(3);
expect(ensureInputs[1]?.resumeSessionId).toBe(first.sessionId);
expect(ensureInputs[2]?.resumeSessionId).toBeUndefined();
// The turn is never retried: one startTurn per run, two across both runs.
expect(startTurnCalls).toBe(2);
// The engine logged the resume fallback.
expect(resumeLogged).toBe(true);
// The fresh retry clears the stale session for the caller.
expect(second.clearSession).toBe(true);
});
it("maps a failed terminal to acpx_turn_failed and folds in the reported usage", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const execute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
events: async function* () {
yield {
type: "status",
text: "usage",
tag: "usage_update",
cost: { amount: 0.31, currency: "USD" },
breakdown: { inputTokens: 40, outputTokens: 700, cachedReadTokens: 60 },
};
yield { type: "done", stopReason: "failed" };
},
result: Promise.resolve({ status: "failed", error: new Error("boom") }),
}) as never,
});
const result = await execute({
runId: "run-failed",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir },
context: {},
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
expect(result.errorCode).toBe("acpx_turn_failed");
// The failed error message becomes the terminal stop reason and the summary.
expect(result.summary).toBe("boom");
expect((result.resultJson as Record<string, unknown>)?.stopReason).toBe("boom");
// The usage math folds the usage_update event into per-run usage and cost.
expect(result.usage).toEqual({ inputTokens: 40, outputTokens: 700, cachedInputTokens: 60 });
expect(result.usageBasis).toBe("per_run");
expect(result.costUsd).toBeCloseTo(0.31);
});
it("computes usage the same way summarizeAcpxTurnUsage does for the event fallback", () => {
// Pin the exported helper the turn path calls: with no getStatus snapshots the
// event breakdown and cost drive the per-run usage.
const summary = summarizeAcpxTurnUsage({
preStatus: null,
postStatus: null,
eventBreakdown: { inputTokens: 40, outputTokens: 700, cachedReadTokens: 60 },
eventCostUsd: 0.31,
});
expect(summary.usage).toEqual({ inputTokens: 40, outputTokens: 700, cachedInputTokens: 60 });
expect(summary.costUsd).toBeCloseTo(0.31);
});
it("returns a prepare_turn error result when the prompt build throws", async () => {
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
let startTurnCalls = 0;
let closes = 0;
const execute = createAcpxEngineExecutor({
createRuntime: () =>
turnRuntime({
onStartTurn: () => {
startTurnCalls += 1;
},
onClose: () => {
closes += 1;
},
events: async function* () {},
result: Promise.resolve({ status: "completed", stopReason: "end_turn" }),
}) as never,
});
// A throwing accessor on a field only buildPrompt reads makes the prompt build
// fail after the session handshake succeeds but before startTurn runs.
const context: Record<string, unknown> = {};
Object.defineProperty(context, "paperclipSessionHandoffMarkdown", {
enumerable: false,
get() {
throw new Error("prompt build boom");
},
});
const result = await execute({
runId: "run-prepare-fail",
agent: { id: "agent-1", companyId: "company-1" },
runtime: {},
config: { agent: "custom", agentCommand: "node ./fake-acp.js", stateDir },
context,
onLog: async () => {},
onMeta: async () => {},
} as never);
expect(result.exitCode).toBe(1);
// The pre-turn failure reports phase "prepare_turn".
expect((result.resultJson as Record<string, unknown>)?.phase).toBe("prepare_turn");
// The turn never started, and the runtime closed once.
expect(startTurnCalls).toBe(0);
expect(closes).toBe(1);
});
it("threads the same happy-path turn through the shared runExecutor harness", async () => {
// A smoke pin on the copied harness: the always-happy buildRuntime turn exits
// clean with the done stop reason as the summary.
const root = await makeTempRoot();
const stateDir = path.join(root, "state");
const { result } = await runExecutor({
agent: "custom",
agentCommand: "node ./fake-acp.js",
stateDir,
});
expect(result.exitCode).toBe(0);
expect(result.summary).toBe("end_turn");
});
});
File diff suppressed because it is too large. Load diff
@@ -11,8 +11,27 @@ import {
resolveAdapterExecutionTargetCwd,
runAdapterExecutionTargetProcess,
runAdapterExecutionTargetShellCommand,
type AdapterSshExecutionTarget,
} from "./execution-target.js";
function sshExecutionTarget(): AdapterSshExecutionTarget {
return {
kind: "remote",
transport: "ssh",
remoteCwd: "/srv/paperclip/workspace",
spec: {
host: "ssh.example.test",
port: 22,
username: "ssh-user",
remoteCwd: "/srv/paperclip/workspace",
remoteWorkspacePath: "/srv/paperclip/workspace",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: true,
},
};
}
describe("runAdapterExecutionTargetShellCommand", () => {
afterEach(() => {
vi.restoreAllMocks();
@@ -27,21 +46,7 @@ describe("runAdapterExecutionTargetShellCommand", () => {
await runAdapterExecutionTargetShellCommand(
"run-1",
{
kind: "remote",
transport: "ssh",
remoteCwd: "/srv/paperclip/workspace",
spec: {
host: "ssh.example.test",
port: 22,
username: "ssh-user",
remoteCwd: "/srv/paperclip/workspace",
remoteWorkspacePath: "/srv/paperclip/workspace",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: true,
},
},
sshExecutionTarget(),
`printf '%s\\n' "$HOME" && echo "it's ok"`,
{
cwd: "/tmp/local",
@@ -74,21 +79,7 @@ describe("runAdapterExecutionTargetShellCommand", () => {
await runAdapterExecutionTargetShellCommand(
"run-1b",
{
kind: "remote",
transport: "ssh",
remoteCwd: "/srv/paperclip/workspace",
spec: {
host: "ssh.example.test",
port: 22,
username: "ssh-user",
remoteCwd: "/srv/paperclip/workspace",
remoteWorkspacePath: "/srv/paperclip/workspace",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: true,
},
},
sshExecutionTarget(),
"env",
{
cwd: "/tmp/local",
@@ -122,21 +113,7 @@ describe("runAdapterExecutionTargetShellCommand", () => {
const result = await runAdapterExecutionTargetShellCommand(
"run-2",
{
kind: "remote",
transport: "ssh",
remoteCwd: "/srv/paperclip/workspace",
spec: {
host: "ssh.example.test",
port: 22,
username: "ssh-user",
remoteCwd: "/srv/paperclip/workspace",
remoteWorkspacePath: "/srv/paperclip/workspace",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: true,
},
},
sshExecutionTarget(),
"sleep 10",
{
cwd: "/tmp/local",
@@ -167,21 +144,7 @@ describe("runAdapterExecutionTargetShellCommand", () => {
const result = await runAdapterExecutionTargetShellCommand(
"run-3",
{
kind: "remote",
transport: "ssh",
remoteCwd: "/srv/paperclip/workspace",
spec: {
host: "ssh.example.test",
port: 22,
username: "ssh-user",
remoteCwd: "/srv/paperclip/workspace",
remoteWorkspacePath: "/srv/paperclip/workspace",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: true,
},
},
sshExecutionTarget(),
"false",
{
cwd: "/tmp/local",
@@ -272,21 +235,7 @@ describe("runAdapterExecutionTargetProcess", () => {
await runAdapterExecutionTargetProcess(
"run-ssh-process",
{
kind: "remote",
transport: "ssh",
remoteCwd: "/srv/paperclip/workspace",
spec: {
host: "ssh.example.test",
port: 22,
username: "ssh-user",
remoteCwd: "/srv/paperclip/workspace",
remoteWorkspacePath: "/srv/paperclip/workspace",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: true,
},
},
sshExecutionTarget(),
"agent-cli",
["--json"],
{
@@ -367,21 +316,7 @@ describe("ensureAdapterExecutionTargetRuntimeCommandInstalled", () => {
await ensureAdapterExecutionTargetRuntimeCommandInstalled({
runId: "run-skip",
target: {
kind: "remote",
transport: "ssh",
remoteCwd: "/srv/paperclip/workspace",
spec: {
host: "ssh.example.test",
port: 22,
username: "ssh-user",
remoteCwd: "/srv/paperclip/workspace",
remoteWorkspacePath: "/srv/paperclip/workspace",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: true,
},
},
target: sshExecutionTarget(),
installCommand: "npm install -g @google/gemini-cli",
cwd: "/tmp/local",
env: {},
@@ -392,21 +327,7 @@ describe("ensureAdapterExecutionTargetRuntimeCommandInstalled", () => {
});
describe("resolveAdapterExecutionTargetCwd", () => {
const sshTarget = {
kind: "remote" as const,
transport: "ssh" as const,
remoteCwd: "/srv/paperclip/workspace",
spec: {
host: "ssh.example.test",
port: 22,
username: "ssh-user",
remoteCwd: "/srv/paperclip/workspace",
remoteWorkspacePath: "/srv/paperclip/workspace",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: true,
},
};
const sshTarget = sshExecutionTarget();
it("falls back to the remote cwd when no adapter cwd is configured", () => {
expect(resolveAdapterExecutionTargetCwd(sshTarget, "", "/Users/host/repo/server")).toBe(
@@ -664,6 +664,7 @@ describe("sandbox managed runtime", () => {
},
};
const runtimeStatuses: string[] = [];
const run = vi.spyOn(client, "run");
attachFallbackSyncIn(client);
const prepared = await prepareSandboxManagedRuntime({
@@ -720,6 +721,7 @@ describe("sandbox managed runtime", () => {
expect.stringMatching(/^restore:Restoring workspace from environment: 100% \(\d+\.\d\/\d+\.\d MB\)$/),
]));
expect(runtimeStatuses.at(-1)).toBe("finalize:Finalizing workspace");
expect(run.mock.calls.some(([command]) => command.includes("workspace-download.tar"))).toBe(true);
});
it("falls back to the host-known byte count when the provider reports 0 for inbound workspace sync", async () => {
@@ -1057,6 +1059,7 @@ describe("sandbox managed runtime", () => {
await expect(readFile(path.join(localWorkspaceDir, "deleted.txt"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
await expect(readFile(path.join(localWorkspaceDir, "node_modules", "cache.bin"), "utf8")).resolves.toBe("do not upload\n");
expect(runtimeStatuses.at(-1)?.phase).toBe("finalize");
expect(downloadedTars).toHaveLength(1);
const downloadMembers = await listTarMembers(rootDir, "workspace-download-list.tar", downloadedTars[0]!.bytes);
expect(downloadMembers.some((entry) => entry === ".git" || entry.startsWith(".git/"))).toBe(false);
@@ -4096,6 +4099,26 @@ describe("sandbox managed runtime outbound coordinator", () => {
expect(control.settled).toEqual(expect.arrayContaining(["workspace", "home", "private-asset"]));
});
it("restores non-git workspace bytes through native syncOut without a fallback tar", async () => {
const { workspaceDir, remoteWorkspaceDir } = await makeOutboundDirs([]);
const client = makeGatedOutboundClient(false, async (_label, run) => run());
const syncOut = vi.spyOn(client, "syncOut");
const run = vi.spyOn(client, "run");
const prepared = await prepareSandboxManagedRuntime({
spec: makeSpec(remoteWorkspaceDir), adapterKey: "test-adapter", client, workspaceLocalDir: workspaceDir,
});
await writeFile(path.join(remoteWorkspaceDir, "file.txt"), "updated native bytes\n", "utf8");
await writeFile(path.join(remoteWorkspaceDir, "remote-only.txt"), "native sync\n", "utf8");
await prepared.restoreWorkspace();
expect(syncOut).toHaveBeenCalledTimes(1);
expect(syncOut.mock.calls[0]?.[0].flatMap((operation) => operation.files)).toEqual([
expect.objectContaining({ kind: "directory", sourcePath: remoteWorkspaceDir }),
]);
await expect(readFile(path.join(workspaceDir, "file.txt"), "utf8")).resolves.toBe("updated native bytes\n");
await expect(readFile(path.join(workspaceDir, "remote-only.txt"), "utf8")).resolves.toBe("native sync\n");
expect(run.mock.calls.some(([command]) => command.includes("workspace-download.tar"))).toBe(false);
});
it("with concurrency permitted, an asset restore starts while the workspace restore is held open", async () => {
const { workspaceDir, remoteWorkspaceDir, dirOf } = await makeOutboundDirs(["home"]);
const { control, gate } = makeOutboundControl();
@@ -1,10 +1,10 @@
import os from "node:os";
import path from "node:path";
import { chmod, mkdtemp, rm, writeFile } from "node:fs/promises";
import { chmod, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { expect, test } from "vitest";
import { HERMES_CLI } from "../shared/constants.js";
import { resolveHermesCommand } from "./execute.js";
import { execute, resolveHermesCommand } from "./execute.js";
import { testEnvironment } from "./test.js";
test("resolveHermesCommand prefers hermesCommand over command", () => {
@@ -47,6 +47,56 @@ test("testEnvironment accepts config.command when hermesCommand is absent", asyn
}
});
test("execute renders prompt API base from adapter env PAPERCLIP_API_URL", async () => {
const tempDir = await mkdtemp(path.join(os.tmpdir(), "hermes-prompt-api-url-"));
const cliPath = path.join(tempDir, "fake-hermes");
const argvPath = path.join(tempDir, "argv.txt");
const originalApiUrl = process.env.PAPERCLIP_API_URL;
try {
process.env.PAPERCLIP_API_URL = "https://public.example.com/api";
await writeFile(
cliPath,
[
"#!/bin/sh",
`printf '%s\\n' "$@" > ${JSON.stringify(argvPath)}`,
"printf 'done\\n\\nsession_id: test-session\\n'",
"",
].join("\n"),
"utf8",
);
await chmod(cliPath, 0o755);
await execute({
runId: "run-test",
agent: {
id: "agent-test",
name: "Hermes Test Agent",
companyId: "company-test",
},
config: {
hermesCommand: cliPath,
env: {
PAPERCLIP_API_URL: { type: "plain", value: "http://127.0.0.1:3101" },
},
},
runtime: {},
onLog: async () => {},
} as any);
const argv = await readFile(argvPath, "utf8");
expect(argv).toContain("- API base: http://127.0.0.1:3101/api");
expect(argv).not.toContain("https://public.example.com/api");
} finally {
if (originalApiUrl === undefined) {
delete process.env.PAPERCLIP_API_URL;
} else {
process.env.PAPERCLIP_API_URL = originalApiUrl;
}
await rm(tempDir, { recursive: true, force: true });
}
});
test("managed connections probe the selected environment without falling back to host keys", async () => {
const tempDir = await mkdtemp(path.join(os.tmpdir(), "hermes-managed-probe-"));
const cliPath = path.join(tempDir, "fake-hermes");
+18 -1
View File
@@ -72,6 +72,14 @@ function cfgStringArray(v: unknown): string[] | undefined {
? (v as string[])
: undefined;
}
function cfgEnvString(v: unknown): string | undefined {
if (typeof v === "string" && v.length > 0) return v;
if (v && typeof v === "object" && "value" in v) {
const value = (v as { value?: unknown }).value;
return typeof value === "string" && value.length > 0 ? value : undefined;
}
return undefined;
}
export function resolveHermesCommand(config: Record<string, unknown>): string {
return cfgString(config.hermesCommand) || cfgString(config.command) || HERMES_CLI;
@@ -432,8 +440,17 @@ export async function execute(
}
// ── Build prompt ───────────────────────────────────────────────────────
const userEnvForPrompt = config.env as Record<string, unknown> | undefined;
const promptConfig: Record<string, unknown> = { ...config };
if (!cfgString(promptConfig.paperclipApiUrl) && userEnvForPrompt && typeof userEnvForPrompt === "object") {
const configuredPaperclipApiUrl = cfgEnvString(userEnvForPrompt.PAPERCLIP_API_URL);
if (configuredPaperclipApiUrl) {
promptConfig.paperclipApiUrl = configuredPaperclipApiUrl;
}
}
const sessionId = persistSession ? prevSessionId : undefined;
const prompt = buildPrompt(ctx, config, { resumedSession: Boolean(sessionId) });
const prompt = buildPrompt(ctx, promptConfig, { resumedSession: Boolean(sessionId) });
// ── Build command args ─────────────────────────────────────────────────
// Use -Q (quiet) to get clean output: just response + session_id line
@@ -0,0 +1,15 @@
CREATE TABLE "company_skill_repository_snapshots" (
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
"company_id" uuid NOT NULL,
"content_hash" text NOT NULL,
"file_inventory" jsonb NOT NULL,
"created_at" timestamp with time zone DEFAULT now() NOT NULL
);
--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN "package_mode" text DEFAULT 'skills' NOT NULL;--> statement-breakpoint
ALTER TABLE "company_skill_sources" ADD COLUMN "repository_package" jsonb;--> statement-breakpoint
ALTER TABLE "company_skill_versions" ADD COLUMN "repository_snapshot_id" uuid;--> statement-breakpoint
ALTER TABLE "company_skill_versions" ADD COLUMN "repository_skill_path" text;--> statement-breakpoint
ALTER TABLE "company_skill_repository_snapshots" ADD CONSTRAINT "company_skill_repository_snapshots_company_id_companies_id_fk" FOREIGN KEY ("company_id") REFERENCES "public"."companies"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
CREATE UNIQUE INDEX "company_skill_repository_snapshots_content_idx" ON "company_skill_repository_snapshots" USING btree ("company_id","content_hash");--> statement-breakpoint
ALTER TABLE "company_skill_versions" ADD CONSTRAINT "company_skill_versions_repository_snapshot_id_company_skill_repository_snapshots_id_fk" FOREIGN KEY ("repository_snapshot_id") REFERENCES "public"."company_skill_repository_snapshots"("id") ON DELETE restrict ON UPDATE no action;
+7
View File
@@ -2325,6 +2325,13 @@
"when": 1791781882729,
"tag": "0333_normal_wild_pack",
"breakpoints": true
},
{
"idx": 334,
"version": "7",
"when": 1791807836288,
"tag": "0334_rare_slipstream",
"breakpoints": true
}
]
}
@@ -1,5 +1,5 @@
import { pgTable, uuid, text, timestamp, jsonb, integer, boolean, uniqueIndex, index } from 'drizzle-orm/pg-core';
import type { SkillPackageInspection } from '@paperclipai/shared';
import type { SkillPackageInspection, SkillRepositoryPackage } from '@paperclipai/shared';
import { companies } from './companies.js';
import { companySkills } from './company_skills.js';
import { toolConnections } from './tool_access.js';
@@ -12,6 +12,8 @@ export const companySkillSources = pgTable('company_skill_sources', {
fullName: text('full_name').notNull(),
trackingRef: text('tracking_ref').notNull(),
connectionId: uuid('connection_id').references(() => toolConnections.id, { onDelete: 'set null' }),
packageMode: text('package_mode').$type<'skills' | 'repository'>().notNull().default('skills'),
repositoryPackage: jsonb('repository_package').$type<SkillRepositoryPackage>(),
excludedFolders: jsonb('excluded_folders').$type<string[]>().notNull().default([]),
enabled: boolean('enabled').notNull().default(true),
revision: integer('revision').notNull().default(0),
+11
View File
@@ -68,12 +68,23 @@ export type CompanySkillVersionFileInventoryEntry = CompanySkillFileInventoryEnt
executable?: boolean;
};
/** Immutable repository bytes shared by all skill versions from the same package. */
export const companySkillRepositorySnapshots = pgTable("company_skill_repository_snapshots", {
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
contentHash: text("content_hash").notNull(),
fileInventory: jsonb("file_inventory").$type<CompanySkillVersionFileInventoryEntry[]>().notNull(),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
}, table => ({ content: uniqueIndex("company_skill_repository_snapshots_content_idx").on(table.companyId, table.contentHash) }));
export const companySkillVersions = pgTable(
"company_skill_versions",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
companySkillId: uuid("company_skill_id").notNull().references(() => companySkills.id, { onDelete: "cascade" }),
repositorySnapshotId: uuid("repository_snapshot_id").references(() => companySkillRepositorySnapshots.id, { onDelete: "restrict" }),
repositorySkillPath: text("repository_skill_path"),
revisionNumber: integer("revision_number").notNull(),
label: text("label"),
releaseId: text("release_id"),
+1 -1
View File
@@ -185,7 +185,7 @@ export {
} from "./tool_access.js";
export {
companySkills,
companySkillVersions,
companySkillVersions, companySkillRepositorySnapshots,
companySkillStars,
companySkillComments,
companySkillTestInputs,
@@ -283,7 +283,7 @@ model or rely on an LLM choosing the desired timing.
| Concurrent writes held at filesystem commit; authority restart | One write, one audit record, same receipt; revoked authorization rejected | Server `agent-instruction-tools.integration.test.ts` |
| Failure after file rename or rollback while saving the receipt | Durable attempt and audit evidence survive; no success claim and no second write | Server `agent-instruction-tools.integration.test.ts` |
| Receipt commits but its database acknowledgement is lost | Exact original result replays; one file-update audit and one attempt record | Server `agent-instruction-tools.integration.test.ts` |
| Tool handler throws an unknown-outcome error versus a validation error | Unknown propagates without a completed tool event; validation returns an ordinary failed response | `codex-app-server-driver.test.ts` |
| Tool handler throws an unknown-outcome error versus a validation error | Unknown propagates without a completed tool event; validation returns an ordinary failed response | `codex-app-server-driver.semantic-results.test.ts` |
| Invalid arguments, oversized input, or stale base; restart and replay | Original failure replays with zero writes, even after the base becomes valid | Server `agent-instruction-tools.integration.test.ts` |
| Failure receipt cannot be stored | No unrecorded final failure; attempt remains unsettled and never reexecutes | Server `agent-instruction-tools.integration.test.ts` |
| Long Unicode input and result; corrupted input digest | Exact bytes across persistence/wire; no copied input in result; bad proof blocks settlement | Rust durable-state tests; `durable-prp-control-plane.test.ts` |
@@ -130,6 +130,6 @@ Run deterministic conformance first, then the real Codex boundary:
```sh
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
src/drivers/codex/codex-app-server-driver.test.ts \
src/drivers/codex/codex-app-server-driver \
src/mock-core/live-console-demo-server.test.ts
```
@@ -42,7 +42,7 @@ use a separate clean operator workspace rather than bypassing that guard.
pnpm --filter @paperclipai/paperclip-runner typecheck:typescript
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
src/drivers/codex/app-server-transport.test.ts \
src/drivers/codex/codex-app-server-driver.test.ts
src/drivers/codex/codex-app-server-driver
pnpm --filter @paperclipai/paperclip-runner check:forbidden-imports
```
@@ -14,11 +14,11 @@ From the repository root, run:
```sh
pnpm --filter @paperclipai/paperclip-runner exec vitest run \
src/drivers/codex/codex-app-server-driver.test.ts \
src/drivers/codex/codex-app-server-driver \
src/mock-core/live-console-demo-server.test.ts
```
Expected result: both files pass. The tests cover requests, goals, lineage,
Expected result: the Codex driver suites and Live console server suite pass. The tests cover requests, goals, lineage,
steering, interrupt races, reconnect, replay, and redaction.
## Step 2: Start the server
+2 -2
View File
@@ -63,7 +63,7 @@
},
"sideEffects": false,
"scripts": {
"build": "pnpm run check:protocol-manifest && pnpm run build:typescript && pnpm run check:capability-contract && pnpm run check:capability-inventory && pnpm run check:protocol-coverage && node scripts/generate-semantic-contracts.mjs --check && node scripts/check-runner-workflow-traceability.mjs && pnpm run build:binary && node scripts/generate-replay-goldens.mjs --check && node scripts/generate-semantic-action-catalog.mjs --check",
"build": "node scripts/record-build-inputs.mjs --begin && pnpm run check:protocol-manifest && pnpm run build:typescript && pnpm run check:capability-contract && pnpm run check:capability-inventory && pnpm run check:protocol-coverage && node scripts/generate-semantic-contracts.mjs --check && node scripts/check-runner-workflow-traceability.mjs && pnpm run build:binary && node scripts/generate-replay-goldens.mjs --check && node scripts/generate-semantic-action-catalog.mjs --check && node scripts/record-build-inputs.mjs --finish",
"build:typescript": "pnpm run check:acpx-profiles && pnpm run ensure:eval-build-deps && pnpm run check:protocol-types && node ./node_modules/typescript/bin/tsc --version && node ./node_modules/typescript/bin/tsc -p tsconfig.json && node ./node_modules/typescript/bin/tsc -p tsconfig.surfaces.json && node scripts/build-verified-provider-entrypoints.mjs",
"build:rust": "cargo build --manifest-path runner/Cargo.toml --locked --workspace --bins",
"build:binary": "cargo build --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/stage-runner-binary.mjs",
@@ -81,7 +81,7 @@
"typecheck:browser": "tsc -p tsconfig.browser.json --noEmit",
"test": "pnpm run test:typescript && pnpm run test:rust",
"test:typescript": "pnpm run test:typescript:prep && vitest run",
"test:typescript:prep": "pnpm run ensure:eval-build-deps && pnpm run build:rust && node --test test/protocol-contract.test.mjs test/acpx-sidecar-contract.test.mjs test/acpx-codex-package-contract.test.mjs test/acpx-model-selection-package-contract.test.mjs test/acpx-message-boundaries-package-contract.test.mjs test/acpx-rich-extensions-package-contract.test.mjs test/acpx-pi-receipt-package-contract.test.mjs test/acpx-response-delivery-package-contract.test.mjs scripts/candidate-provider-pack.test.mjs scripts/cursor-provisioner-layout.test.mjs scripts/release-provider-pack.test.mjs scripts/aws-agentcore-provisioning.test.mjs scripts/build-verified-provider-entrypoints.test.mjs scripts/local-provider-smoke-environment.test.mjs scripts/materialize-opencode-binary.test.mjs scripts/installed-package-pack.test.mjs scripts/runner-package-executable.test.mjs scripts/stage-runner-binary.test.mjs",
"test:typescript:prep": "pnpm run ensure:eval-build-deps && pnpm run build:rust && node --test test/protocol-contract.test.mjs test/acpx-sidecar-contract.test.mjs test/acpx-codex-package-contract.test.mjs test/acpx-model-selection-package-contract.test.mjs test/acpx-message-boundaries-package-contract.test.mjs test/acpx-rich-extensions-package-contract.test.mjs test/acpx-pi-receipt-package-contract.test.mjs test/acpx-response-delivery-package-contract.test.mjs scripts/candidate-provider-pack.test.mjs scripts/cursor-provisioner-layout.test.mjs scripts/release-provider-pack.test.mjs scripts/aws-agentcore-provisioning.test.mjs scripts/build-verified-provider-entrypoints.test.mjs scripts/local-provider-smoke-environment.test.mjs scripts/materialize-opencode-binary.test.mjs scripts/installed-package-pack.test.mjs scripts/runner-package-executable.test.mjs scripts/stage-runner-binary.test.mjs scripts/build-inputs.test.mjs",
"test:typescript:vitest": "node ./scripts/run-pr-vitest-lane.mjs",
"test:rust": "cargo test --release --manifest-path runner/Cargo.toml --locked --workspace",
"test:codex": "cargo test --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --test codex_provider",
@@ -0,0 +1,444 @@
// Records which inputs produced the Runner package's current build output, so
// a consumer that would otherwise rebuild the Runner can reuse the build when
// nothing it depends on has changed.
//
// server's `prepare:runner-vendor` runs the Runner's full `build`, because a
// standalone `pnpm --filter @paperclipai/server build` (the Dockerfile's path)
// must produce a current vendored Runner on its own. Inside `pnpm -r build`
// the same command has just built the Runner a few seconds earlier, and the
// rebuild cost 12-16s on the CI Build job's critical path (six master runs,
// 2026-10-11), the same again in every other `pnpm build`.
//
// The Runner's `build` brackets itself with scripts/record-build-inputs.mjs:
// `--begin` removes any previous stamp and writes a pending record of the
// input digest before the first step reads anything; `--finish` recomputes
// the digest after the last check and writes the stamp under
// node_modules/.cache only if the inputs are still the ones the build started
// from, together with a manifest of every file the build left under dist. The
// server's prepare step (server/scripts/prepare-runner-vendor.mjs) reuses the
// build only when the stamp matches, every recorded output is still there at
// its recorded size, and the current input digest matches. Every other state
// rebuilds: a missing stamp, a changed input, a deleted or truncated output, a
// different platform, an edit that landed while the build was running. The
// slow direction is always the safe one.
//
// Each build chain has its own pending record, so two Runner builds running
// at once in one checkout (server `build` and `typecheck` side by side, say)
// do not consume each other's: `--begin` and `--finish` are separate
// processes, and what they share is the shell pnpm runs the chain in, so the
// record is keyed by that shell's pid. Both builds write the same dist, which
// the stamp cannot make safe; what it does guarantee is that a stamp is
// written only by a build whose own inputs did not move, and that the other
// build's `--finish` drops the stamp if its inputs did.
//
// Inputs: every file under packages/paperclip-runner except its build outputs
// (dist, dist-browser, runner/target), node_modules and dot-directories (tool
// caches); every file under packages/paperclip-eval-kernel except dist and
// node_modules, because build:typescript builds that package first; every
// file under the root patches directory, because check:acpx-profiles hashes
// the acpx patch and build-verified-provider-entrypoints.mjs copies the pi-acp
// and brace-expansion patches into dist; and the root files the Runner's
// tsconfig and dependency graph read. The toolchain pins live inside the
// package (rust-toolchain.toml, runner/Cargo.lock), and the TypeScript version
// comes from pnpm-lock.yaml, so both are covered.
//
// Outputs: the server build vendors packages/paperclip-runner/dist wholesale
// (`cp -Rf ../packages/paperclip-runner/dist/. dist/vendor/paperclip-runner/`),
// so the stamp lists every file under dist with its size and reuse needs all
// of them present; a helper such as dist/cli/acpx-runtime-sidecar.cjs is
// opened by path at runtime, so a tree that lost it must not count as current.
import { createHash } from "node:crypto";
import {
existsSync,
lstatSync,
mkdirSync,
readdirSync,
readFileSync,
readlinkSync,
renameSync,
rmSync,
writeFileSync,
} from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
export const BUILD_INPUTS_SCHEMA = "paperclip.runner-build-inputs.v2";
const RUNNER_PACKAGE_RELATIVE = "packages/paperclip-runner";
const EVAL_KERNEL_PACKAGE_RELATIVE = "packages/paperclip-eval-kernel";
const ROOT_INPUT_DIRECTORIES = ["patches"];
const ROOT_INPUT_FILES = [
"package.json",
"pnpm-lock.yaml",
"pnpm-workspace.yaml",
"tsconfig.base.json",
];
const OUTPUT_DIRECTORY_RELATIVE = "dist";
const SKIPPED_DIRECTORY_NAMES = new Set(["node_modules", "dist", "dist-browser"]);
const RUNNER_SKIPPED_DIRECTORIES = new Set([`${RUNNER_PACKAGE_RELATIVE}/runner/target`]);
export function runnerPackageRootFromScripts() {
return path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
}
export function repoRootFromRunnerPackage(packageRoot) {
return path.resolve(packageRoot, "../..");
}
export function buildInputsStampPath(packageRoot) {
return path.join(packageRoot, "node_modules", ".cache", "paperclip-runner-build-inputs.json");
}
/** One record per build in progress, written by `--begin` and consumed by that build's `--finish`. */
export function buildInputsPendingDirectory(packageRoot) {
return path.join(packageRoot, "node_modules", ".cache", "paperclip-runner-build-inputs.pending");
}
export function buildInputsPendingPath(packageRoot, buildId) {
return path.join(buildInputsPendingDirectory(packageRoot), `${buildId}.json`);
}
/**
* A pending record older than this belongs to a build that never reached
* `--finish` (a failed step, an interrupted build); `--begin` removes such
* records so they cannot pile up. No Runner build runs anywhere near this long.
*/
export const PENDING_RECORD_MAX_AGE_MS = 24 * 60 * 60 * 1000;
/**
* Identifies the build chain a `--begin` or `--finish` process belongs to.
* The two are separate processes; what they share is the shell pnpm runs the
* chain in, so the chain is named after its pid, which `--begin` sees as its
* parent. A shell that execs the last command of a `-c` string (bash can)
* turns `--finish` into the shell itself, so `--finish` also accepts a record
* named after its own pid.
*/
export function currentBuildId() {
return String(process.ppid);
}
function finishingBuildIds() {
return [String(process.ppid), String(process.pid)];
}
function readPendingRecord(pendingPath) {
const pending = readBuildInputsStamp(pendingPath);
if (!pending || pending.schema !== BUILD_INPUTS_SCHEMA || typeof pending.digest !== "string") return null;
return pending;
}
/** Drops pending records of builds that never finished, leaving the one for `buildId`. */
function prunePendingRecords(packageRoot, buildId, now) {
const directory = buildInputsPendingDirectory(packageRoot);
let entries;
try {
entries = readdirSync(directory);
} catch {
return;
}
const cutoff = now().getTime() - PENDING_RECORD_MAX_AGE_MS;
for (const name of entries) {
if (name === `${buildId}.json`) continue;
const absolute = path.join(directory, name);
let stat;
try {
stat = lstatSync(absolute);
} catch {
continue;
}
if (stat.mtimeMs < cutoff) rmSync(absolute, { force: true });
}
}
/**
* The outputs every consumer of the vendored Runner reads; a build that did
* not produce them is not recorded, and a tree missing any of them is not
* current whatever the stamp says.
*/
export function expectedBuildOutputs(packageRoot, platform = process.platform) {
const executable = platform === "win32" ? "paperclip-runnerd.exe" : "paperclip-runnerd";
return ["dist/index.js", "dist/live/index.js", "dist/testing.js", `dist/bin/${executable}`]
.map((relative) => path.join(packageRoot, ...relative.split("/")));
}
function walk(root, relativeDirectory, { skipNames, skippedDirectories, skipDotDirectories }, out) {
const absolute = path.join(root, ...relativeDirectory.split("/"));
if (!existsSync(absolute)) return;
const entries = readdirSync(absolute, { withFileTypes: true })
.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
for (const entry of entries) {
const relative = `${relativeDirectory}/${entry.name}`;
if (entry.isDirectory()) {
if (skipDotDirectories && entry.name.startsWith(".")) continue;
if (skipNames.has(entry.name)) continue;
if (skippedDirectories.has(relative)) continue;
walk(root, relative, { skipNames, skippedDirectories, skipDotDirectories }, out);
continue;
}
// Files and symlinks; a symlinked directory is hashed as its link target
// rather than followed, so a link into node_modules cannot pull a
// dependency tree into the input set.
out.push(relative);
}
}
const INPUT_WALK = {
skipNames: SKIPPED_DIRECTORY_NAMES,
skippedDirectories: new Set(),
skipDotDirectories: true,
};
/** Repo-relative POSIX paths of every build input, sorted. */
export function collectBuildInputFiles(repoRoot) {
const files = [];
walk(repoRoot, RUNNER_PACKAGE_RELATIVE, { ...INPUT_WALK, skippedDirectories: RUNNER_SKIPPED_DIRECTORIES }, files);
walk(repoRoot, EVAL_KERNEL_PACKAGE_RELATIVE, INPUT_WALK, files);
for (const directory of ROOT_INPUT_DIRECTORIES) walk(repoRoot, directory, INPUT_WALK, files);
for (const name of ROOT_INPUT_FILES) {
if (existsSync(path.join(repoRoot, name))) files.push(name);
}
return files.sort();
}
export function computeBuildInputsDigest(repoRoot) {
const files = collectBuildInputFiles(repoRoot);
const hash = createHash("sha256");
let byteCount = 0;
for (const relative of files) {
const absolute = path.join(repoRoot, ...relative.split("/"));
hash.update(relative);
hash.update("\0");
if (lstatSync(absolute).isSymbolicLink()) {
hash.update("link\0");
hash.update(readlinkSync(absolute));
} else {
const content = readFileSync(absolute);
byteCount += content.length;
hash.update(`file ${content.length}\0`);
hash.update(content);
}
hash.update("\0");
}
return {
schema: BUILD_INPUTS_SCHEMA,
digest: hash.digest("hex"),
fileCount: files.length,
byteCount,
};
}
/**
* Every file the build left under dist (the tree the server vendors), as
* package-relative POSIX paths with the size lstat reports, sorted. Nothing
* under dist is skipped: the server copies all of it.
*/
export function collectBuildOutputFiles(packageRoot) {
const files = [];
walk(packageRoot, OUTPUT_DIRECTORY_RELATIVE, { skipNames: new Set(), skippedDirectories: new Set(), skipDotDirectories: false }, files);
return files.sort().map((relative) => ({
path: relative,
size: lstatSync(path.join(packageRoot, ...relative.split("/"))).size,
}));
}
/** The first recorded output that is gone or has changed size, or null when all are intact. */
export function findChangedBuildOutput(packageRoot, outputs) {
for (const output of outputs) {
const absolute = path.join(packageRoot, ...output.path.split("/"));
let stat;
try {
stat = lstatSync(absolute);
} catch {
return { ...output, state: "missing" };
}
if (stat.size !== output.size) return { ...output, state: "changed size", size: stat.size, recordedSize: output.size };
}
return null;
}
export function readBuildInputsStamp(stampPath) {
try {
const parsed = JSON.parse(readFileSync(stampPath, "utf8"));
return parsed && typeof parsed === "object" ? parsed : null;
} catch {
return null;
}
}
export function writeBuildInputsStamp(stampPath, record) {
mkdirSync(path.dirname(stampPath), { recursive: true });
const staged = `${stampPath}.${process.pid}.tmp`;
writeFileSync(staged, `${JSON.stringify(record, null, 2)}\n`);
renameSync(staged, stampPath);
}
export function removeBuildInputsStamp(stampPath) {
rmSync(stampPath, { force: true });
}
/**
* First step of the Runner's build: forgets the previous build (a build in
* progress can no longer vouch for its outputs) and records the inputs the
* build is about to read, so that `recordRunnerBuild` can tell whether they
* changed while it ran.
*/
export function beginRunnerBuild({
repoRoot,
packageRoot,
platform = process.platform,
arch = process.arch,
buildId = currentBuildId(),
now = () => new Date(),
}) {
removeBuildInputsStamp(buildInputsStampPath(packageRoot));
prunePendingRecords(packageRoot, buildId, now);
const inputs = computeBuildInputsDigest(repoRoot);
const pending = { ...inputs, platform, arch, buildId, startedAt: now().toISOString() };
const pendingPath = buildInputsPendingPath(packageRoot, buildId);
writeBuildInputsStamp(pendingPath, pending);
return { ...pending, pendingPath };
}
/**
* Last step of the Runner's build, after every step and check has passed:
* records the inputs captured by `beginRunnerBuild` as the ones that produced
* the outputs now under dist. An edit that landed while the build ran means
* the outputs may not reflect the current inputs, so nothing is recorded and
* the next consumer rebuilds. A build that did not begin with
* `beginRunnerBuild` (no pending record for this build chain), or that left
* a required output missing, is an error.
*/
export function recordRunnerBuild({
repoRoot,
packageRoot,
platform = process.platform,
arch = process.arch,
buildId = null,
now = () => new Date(),
}) {
const stampPath = buildInputsStampPath(packageRoot);
const candidates = buildId === null ? finishingBuildIds() : [String(buildId)];
let pending = null;
let pendingPath = null;
for (const candidate of candidates) {
pendingPath = buildInputsPendingPath(packageRoot, candidate);
pending = readPendingRecord(pendingPath);
if (pending) break;
}
if (!pending) {
removeBuildInputsStamp(stampPath);
throw new Error(
`no pending build inputs for build ${candidates.join(" or ")} under ` +
`${buildInputsPendingDirectory(packageRoot)}: the Runner build must start with ` +
"`node scripts/record-build-inputs.mjs --begin` before it can record its result",
);
}
rmSync(pendingPath, { force: true });
const missing = expectedBuildOutputs(packageRoot, platform).filter((file) => !existsSync(file));
if (missing.length > 0) {
removeBuildInputsStamp(stampPath);
throw new Error(`the Runner build finished without producing ${path.relative(packageRoot, missing[0])}`);
}
const inputs = computeBuildInputsDigest(repoRoot);
if (inputs.digest !== pending.digest) {
removeBuildInputsStamp(stampPath);
return {
recorded: false,
code: "inputs_changed_during_build",
detail:
`inputs changed while the build ran (digest ${pending.digest.slice(0, 12)} at start, ` +
`${inputs.digest.slice(0, 12)} now); the next consumer rebuilds`,
startedAt: pending.startedAt,
stampPath,
};
}
const outputs = collectBuildOutputFiles(packageRoot);
const record = {
...inputs,
platform,
arch,
startedAt: pending.startedAt,
recordedAt: now().toISOString(),
outputCount: outputs.length,
outputs,
};
writeBuildInputsStamp(stampPath, record);
return { recorded: true, code: "recorded", ...record, stampPath };
}
/**
* Decides whether the Runner's outputs are current for the present inputs.
* `code` is a stable slug for timing lines; `detail` is for humans.
*/
export function assessRunnerBuild({
repoRoot,
packageRoot,
platform = process.platform,
arch = process.arch,
}) {
const stampPath = buildInputsStampPath(packageRoot);
const stamp = readBuildInputsStamp(stampPath);
if (!stamp) {
return { current: false, code: "no_stamp", detail: `no build stamp at ${stampPath}`, stampPath };
}
if (stamp.schema !== BUILD_INPUTS_SCHEMA || !Array.isArray(stamp.outputs)) {
return {
current: false,
code: "schema_mismatch",
detail: `stamp schema ${String(stamp.schema)} is not ${BUILD_INPUTS_SCHEMA}`,
stampPath,
};
}
if (stamp.platform !== platform || stamp.arch !== arch) {
return {
current: false,
code: "platform_mismatch",
detail: `stamp recorded on ${stamp.platform}/${stamp.arch}, this is ${platform}/${arch}`,
stampPath,
};
}
const missing = expectedBuildOutputs(packageRoot, platform).filter((file) => !existsSync(file));
if (missing.length > 0) {
return {
current: false,
code: "missing_output",
detail: `missing build output ${path.relative(packageRoot, missing[0])}`,
stampPath,
};
}
const changedOutput = findChangedBuildOutput(packageRoot, stamp.outputs);
if (changedOutput) {
return {
current: false,
code: "outputs_changed",
detail:
`build output ${changedOutput.path} is ${changedOutput.state} since the build recorded at ` +
`${stamp.recordedAt}; ${stamp.outputs.length} outputs checked`,
stampPath,
};
}
const inputs = computeBuildInputsDigest(repoRoot);
if (inputs.digest !== stamp.digest) {
return {
current: false,
code: "inputs_changed",
detail: `inputs changed since the recorded build; ${inputs.fileCount} files checked`,
stampPath,
inputs,
};
}
return {
current: true,
code: "inputs_unchanged",
detail:
`inputs unchanged since the build recorded at ${stamp.recordedAt}; ${inputs.fileCount} files ` +
`and ${stamp.outputs.length} outputs checked`,
stampPath,
inputs,
stamp,
};
}
@@ -0,0 +1,533 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { existsSync, readdirSync } from "node:fs";
import { mkdtemp, mkdir, readdir, readFile, rm, stat, symlink, truncate, utimes, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { test } from "node:test";
import { promisify } from "node:util";
import { fileURLToPath } from "node:url";
import {
BUILD_INPUTS_SCHEMA,
PENDING_RECORD_MAX_AGE_MS,
assessRunnerBuild,
beginRunnerBuild,
buildInputsPendingDirectory,
buildInputsPendingPath,
buildInputsStampPath,
collectBuildInputFiles,
collectBuildOutputFiles,
computeBuildInputsDigest,
currentBuildId,
expectedBuildOutputs,
readBuildInputsStamp,
recordRunnerBuild,
} from "./build-inputs.mjs";
const execFileAsync = promisify(execFile);
const recordScript = fileURLToPath(new URL("./record-build-inputs.mjs", import.meta.url));
/** A child of this process names its build after this process, as `--begin` names it after the shell. */
const childBuildId = String(process.pid);
async function waitFor(condition, label, timeoutMs = 10_000) {
const deadline = Date.now() + timeoutMs;
while (!condition()) {
if (Date.now() > deadline) throw new Error(`timed out waiting for ${label}`);
await new Promise((resolve) => setTimeout(resolve, 20));
}
}
/**
* The whole build chain the way pnpm runs it: one shell, `--begin` first,
* `--finish` last, with a stand-in for the build steps that waits for a
* marker file so two chains can be held open at the same time.
*/
function runChainInShell(shell, packageRoot, marker) {
const quote = (value) => `'${value.replace(/'/g, `'\\''`)}'`;
const wait =
`${quote(process.execPath)} -e ${quote(
"const fs=require('node:fs');const [marker]=process.argv.slice(1);const deadline=Date.now()+10000;" +
"(function poll(){if(fs.existsSync(marker))return;if(Date.now()>deadline)process.exit(3);setTimeout(poll,20)})()",
)} ${quote(marker)}`;
const phase = (flag) => `${quote(process.execPath)} ${quote(recordScript)} ${flag} --package-root ${quote(packageRoot)}`;
return execFileAsync(shell, ["-c", `${phase("--begin")} && ${wait} && ${phase("--finish")}`]);
}
async function writeTree(root, files) {
for (const [relative, content] of Object.entries(files)) {
const absolute = path.join(root, ...relative.split("/"));
await mkdir(path.dirname(absolute), { recursive: true });
await writeFile(absolute, content);
}
}
/**
* A throwaway repo with one Runner source file, one Rust source, two root
* patches, built outputs (the four every consumer reads plus one helper that
* is only opened by path), and the usual noise.
*/
async function makeRepo() {
const repoRoot = await mkdtemp(path.join(tmpdir(), "runner-build-inputs-"));
await writeTree(repoRoot, {
"package.json": "{}\n",
"pnpm-lock.yaml": "lockfileVersion: 9\n",
"pnpm-workspace.yaml": "packages:\n - packages/*\n",
"tsconfig.base.json": "{}\n",
"patches/acpx@0.13.1.patch": "--- a\n+++ b\n",
"patches/pi-acp@0.0.33.patch": "--- c\n+++ d\n",
"packages/paperclip-runner/package.json": '{"name":"@paperclipai/paperclip-runner"}\n',
"packages/paperclip-runner/src/index.ts": "export const one = 1;\n",
"packages/paperclip-runner/runner/src/main.rs": "fn main() {}\n",
"packages/paperclip-runner/runner/target/release/paperclip-runnerd": "binary\n",
"packages/paperclip-runner/dist/index.js": "export const one = 1;\n",
"packages/paperclip-runner/dist/live/index.js": "export {};\n",
"packages/paperclip-runner/dist/testing.js": "export {};\n",
"packages/paperclip-runner/dist/bin/paperclip-runnerd": "binary\n",
"packages/paperclip-runner/dist/cli/acpx-runtime-sidecar.cjs": "module.exports = {};\n",
"packages/paperclip-runner/dist-browser/index.js": "browser\n",
"packages/paperclip-runner/node_modules/dep/index.js": "dep\n",
"packages/paperclip-runner/.vite/cache.json": "{}\n",
"packages/paperclip-eval-kernel/src/index.ts": "export const kernel = 1;\n",
"packages/paperclip-eval-kernel/dist/index.js": "export const kernel = 1;\n",
});
const packageRoot = path.join(repoRoot, "packages", "paperclip-runner");
return { repoRoot, packageRoot };
}
const INPUT_FILE_COUNT = 10;
const OUTPUT_FILE_COUNT = 5;
async function withRepo(action) {
const repo = await makeRepo();
try {
await action(repo);
} finally {
await rm(repo.repoRoot, { recursive: true, force: true });
}
}
/** A complete, undisturbed build: begin, (nothing changes), finish. */
function recordCurrentBuild({ repoRoot, packageRoot, platform, arch, now }) {
beginRunnerBuild({ repoRoot, packageRoot, platform, arch, now });
return recordRunnerBuild({ repoRoot, packageRoot, platform, arch, now });
}
test("the input set covers sources, the eval kernel, root patches and root config, and skips outputs and caches", async () => {
await withRepo(async ({ repoRoot }) => {
const files = collectBuildInputFiles(repoRoot);
assert.deepEqual(files, [
"package.json",
"packages/paperclip-eval-kernel/src/index.ts",
"packages/paperclip-runner/package.json",
"packages/paperclip-runner/runner/src/main.rs",
"packages/paperclip-runner/src/index.ts",
"patches/acpx@0.13.1.patch",
"patches/pi-acp@0.0.33.patch",
"pnpm-lock.yaml",
"pnpm-workspace.yaml",
"tsconfig.base.json",
]);
assert.equal(files.length, INPUT_FILE_COUNT);
});
});
test("the output manifest lists every file under dist with its size, and nothing outside it", async () => {
await withRepo(async ({ packageRoot }) => {
const outputs = collectBuildOutputFiles(packageRoot);
assert.deepEqual(outputs, [
{ path: "dist/bin/paperclip-runnerd", size: 7 },
{ path: "dist/cli/acpx-runtime-sidecar.cjs", size: 21 },
{ path: "dist/index.js", size: 22 },
{ path: "dist/live/index.js", size: 11 },
{ path: "dist/testing.js", size: 11 },
]);
assert.equal(outputs.length, OUTPUT_FILE_COUNT);
});
});
test("the digest is stable, ignores outputs and caches, and tracks every kind of input", async () => {
await withRepo(async ({ repoRoot }) => {
const before = computeBuildInputsDigest(repoRoot);
assert.equal(before.schema, BUILD_INPUTS_SCHEMA);
assert.equal(before.fileCount, INPUT_FILE_COUNT);
assert.equal(computeBuildInputsDigest(repoRoot).digest, before.digest, "two reads disagree");
// Outputs, dependencies, caches and the Rust target directory may change freely.
await writeTree(repoRoot, {
"packages/paperclip-runner/dist/index.js": "export const one = 2;\n",
"packages/paperclip-runner/dist/cli/acpx-runtime-sidecar.cjs": "changed\n",
"packages/paperclip-runner/dist-browser/index.js": "changed\n",
"packages/paperclip-runner/node_modules/dep/index.js": "changed\n",
"packages/paperclip-runner/.vite/cache.json": '{"changed":true}\n',
"packages/paperclip-runner/runner/target/release/paperclip-runnerd": "rebuilt\n",
"packages/paperclip-eval-kernel/dist/index.js": "changed\n",
});
assert.equal(computeBuildInputsDigest(repoRoot).digest, before.digest, "an output changed the digest");
// Every input kind changes it: TypeScript, Rust, the eval kernel, a root
// patch, root config, a new file, a rename.
const seen = new Set([before.digest]);
const expectChange = async (label, files) => {
await writeTree(repoRoot, files);
const next = computeBuildInputsDigest(repoRoot).digest;
assert.ok(!seen.has(next), `${label} did not change the digest`);
seen.add(next);
};
await expectChange("a Runner source", { "packages/paperclip-runner/src/index.ts": "export const one = 2;\n" });
await expectChange("a Rust source", { "packages/paperclip-runner/runner/src/main.rs": "fn main() { }\n" });
await expectChange("an eval-kernel source", { "packages/paperclip-eval-kernel/src/index.ts": "export const kernel = 2;\n" });
await expectChange("the pi-acp patch the build copies into dist", { "patches/pi-acp@0.0.33.patch": "--- c\n+++ d\n+e\n" });
await expectChange("the acpx patch the profile check hashes", { "patches/acpx@0.13.1.patch": "--- a\n+++ b\n+c\n" });
await expectChange("a new patch", { "patches/brace-expansion@5.0.9.patch": "--- x\n+++ y\n" });
await expectChange("the lockfile", { "pnpm-lock.yaml": "lockfileVersion: 9\nchanged: true\n" });
await expectChange("the base tsconfig", { "tsconfig.base.json": '{"compilerOptions":{}}\n' });
await expectChange("a new file", { "packages/paperclip-runner/protocol/new.json": "{}\n" });
await rm(path.join(repoRoot, "packages/paperclip-runner/protocol/new.json"));
await expectChange("a renamed file", { "packages/paperclip-runner/protocol/renamed.json": "{}\n" });
});
});
test("a symlink is hashed as its target path, not followed", async () => {
await withRepo(async ({ repoRoot, packageRoot }) => {
const before = computeBuildInputsDigest(repoRoot);
await symlink("../node_modules/dep", path.join(packageRoot, "src", "linked"), "dir");
const linked = computeBuildInputsDigest(repoRoot);
assert.notEqual(linked.digest, before.digest, "the link itself is an input");
assert.equal(linked.fileCount, before.fileCount + 1, "the link target's files were followed");
await writeFile(path.join(packageRoot, "node_modules/dep/index.js"), "changed\n");
assert.equal(computeBuildInputsDigest(repoRoot).digest, linked.digest, "a change behind the link changed the digest");
});
});
test("a build is current only with a matching stamp and every recorded output intact", async () => {
await withRepo(async ({ repoRoot, packageRoot }) => {
const platform = "linux";
const arch = "x64";
const assess = () => assessRunnerBuild({ repoRoot, packageRoot, platform, arch });
assert.equal(assess().current, false);
assert.equal(assess().code, "no_stamp");
const record = recordCurrentBuild({ repoRoot, packageRoot, platform, arch, now: () => new Date(0) });
assert.equal(record.recorded, true);
assert.equal(record.stampPath, buildInputsStampPath(packageRoot));
const stamp = readBuildInputsStamp(record.stampPath);
assert.equal(stamp.schema, BUILD_INPUTS_SCHEMA);
assert.equal(stamp.digest, record.digest);
assert.equal(stamp.fileCount, INPUT_FILE_COUNT);
assert.equal(stamp.byteCount, record.byteCount);
assert.equal(stamp.platform, platform);
assert.equal(stamp.arch, arch);
assert.equal(stamp.startedAt, "1970-01-01T00:00:00.000Z");
assert.equal(stamp.recordedAt, "1970-01-01T00:00:00.000Z");
assert.equal(stamp.outputCount, OUTPUT_FILE_COUNT);
assert.deepEqual(stamp.outputs, collectBuildOutputFiles(packageRoot));
const current = assess();
assert.equal(current.current, true, current.detail);
assert.equal(current.code, "inputs_unchanged");
assert.match(current.detail, /10 files and 5 outputs checked/);
assert.equal(assessRunnerBuild({ repoRoot, packageRoot, platform: "win32", arch }).code, "platform_mismatch");
assert.equal(assessRunnerBuild({ repoRoot, packageRoot, platform, arch: "arm64" }).code, "platform_mismatch");
// The four outputs every consumer reads are checked by name.
for (const output of expectedBuildOutputs(packageRoot, platform)) {
const content = await readFile(output);
await rm(output);
const missing = assess();
assert.equal(missing.current, false, `${output} missing still counted as current`);
assert.equal(missing.code, "missing_output");
assert.match(missing.detail, new RegExp(path.basename(output)));
await writeFile(output, content);
}
assert.equal(assess().current, true);
// So is every other file the build left under dist: a helper that is only
// opened by path must be there, at the size the build wrote.
const helper = path.join(packageRoot, "dist", "cli", "acpx-runtime-sidecar.cjs");
const helperContent = await readFile(helper);
await rm(helper);
const gone = assess();
assert.equal(gone.current, false, "a missing helper still counted as current");
assert.equal(gone.code, "outputs_changed");
assert.match(gone.detail, /^build output dist\/cli\/acpx-runtime-sidecar\.cjs is missing since the build recorded at /);
await writeFile(helper, helperContent);
assert.equal(assess().current, true);
await truncate(helper, 3);
const truncated = assess();
assert.equal(truncated.current, false, "a truncated helper still counted as current");
assert.equal(truncated.code, "outputs_changed");
assert.match(truncated.detail, /^build output dist\/cli\/acpx-runtime-sidecar\.cjs is changed size since /);
await writeFile(helper, helperContent);
assert.equal(assess().current, true);
// A file the build did not produce may appear without invalidating it.
await writeFile(path.join(packageRoot, "dist", "extra.txt"), "later\n");
assert.equal(assess().current, true);
await writeFile(path.join(packageRoot, "src", "index.ts"), "export const one = 3;\n");
const changed = assess();
assert.equal(changed.current, false);
assert.equal(changed.code, "inputs_changed");
await writeFile(record.stampPath, JSON.stringify({ ...readBuildInputsStamp(record.stampPath), schema: "other" }));
assert.equal(assess().code, "schema_mismatch");
await writeFile(record.stampPath, JSON.stringify({ ...readBuildInputsStamp(record.stampPath), schema: BUILD_INPUTS_SCHEMA, outputs: undefined }));
assert.equal(assess().code, "schema_mismatch", "a stamp without an output manifest was trusted");
await writeFile(record.stampPath, "not json");
assert.equal(assess().code, "no_stamp");
});
});
test("an input edited while the build runs leaves no stamp, so the next consumer rebuilds", async () => {
await withRepo(async ({ repoRoot, packageRoot }) => {
const stampPath = buildInputsStampPath(packageRoot);
const pendingPath = buildInputsPendingPath(packageRoot, currentBuildId());
const assess = () => assessRunnerBuild({ repoRoot, packageRoot });
// A previous good build, then a new build begins: the old stamp is gone at
// once, because the outputs are about to be replaced.
recordCurrentBuild({ repoRoot, packageRoot });
assert.equal(assess().current, true);
const pending = beginRunnerBuild({ repoRoot, packageRoot, now: () => new Date(1000) });
assert.equal(pending.pendingPath, pendingPath);
assert.equal(pending.buildId, String(process.ppid), "a build is named after the shell that runs its chain");
assert.equal(readBuildInputsStamp(stampPath), null, "the old stamp survived the start of a new build");
assert.equal(readBuildInputsStamp(pendingPath).digest, pending.digest);
assert.equal(readBuildInputsStamp(pendingPath).buildId, pending.buildId);
assert.equal(assess().code, "no_stamp");
// The build "compiles" the old source, then an edit lands before it ends.
await writeFile(path.join(packageRoot, "src", "index.ts"), "export const one = 2;\n");
const result = recordRunnerBuild({ repoRoot, packageRoot });
assert.equal(result.recorded, false);
assert.equal(result.code, "inputs_changed_during_build");
assert.match(result.detail, /^inputs changed while the build ran \(digest [0-9a-f]{12} at start, [0-9a-f]{12} now\); the next consumer rebuilds$/);
assert.equal(result.startedAt, "1970-01-01T00:00:01.000Z");
assert.equal(readBuildInputsStamp(stampPath), null, "a stamp was written for a build whose inputs moved");
assert.equal(readBuildInputsStamp(pendingPath), null, "the pending record was left behind");
assert.equal(assess().code, "no_stamp");
// An undisturbed build records normally, and only the pending record's
// start time is kept.
const record = recordCurrentBuild({ repoRoot, packageRoot });
assert.equal(record.recorded, true);
assert.equal(readBuildInputsStamp(pendingPath), null);
assert.equal(assess().current, true);
});
});
test("a build that did not begin, or that left a required output missing, cannot be recorded", async () => {
await withRepo(async ({ repoRoot, packageRoot }) => {
const stampPath = buildInputsStampPath(packageRoot);
recordCurrentBuild({ repoRoot, packageRoot });
assert.throws(
() => recordRunnerBuild({ repoRoot, packageRoot }),
new RegExp(
`no pending build inputs for build ${process.ppid} or ${process.pid} under .*paperclip-runner-build-inputs\\.pending: ` +
"the Runner build must start with `node scripts/record-build-inputs\\.mjs --begin`",
),
);
assert.equal(readBuildInputsStamp(stampPath), null, "the previous stamp outlived a build that did not begin");
// Another build's record is not this build's: a chain that did not begin
// cannot finish on the strength of one that is still running.
beginRunnerBuild({ repoRoot, packageRoot, buildId: "other-shell" });
assert.throws(() => recordRunnerBuild({ repoRoot, packageRoot }), /no pending build inputs for build /);
assert.equal(
readBuildInputsStamp(buildInputsPendingPath(packageRoot, "other-shell")).digest,
computeBuildInputsDigest(repoRoot).digest,
"a build that did not begin consumed another build's record",
);
await rm(buildInputsPendingPath(packageRoot, "other-shell"));
beginRunnerBuild({ repoRoot, packageRoot });
await rm(path.join(packageRoot, "dist", "testing.js"));
assert.throws(
() => recordRunnerBuild({ repoRoot, packageRoot }),
/the Runner build finished without producing dist\/testing\.js/,
);
assert.equal(readBuildInputsStamp(stampPath), null);
assert.equal(assessRunnerBuild({ repoRoot, packageRoot }).code, "no_stamp");
});
});
test("record-build-inputs.mjs brackets the build for the package it is pointed at", async () => {
await withRepo(async ({ repoRoot, packageRoot }) => {
const run = (...args) => execFileAsync(process.execPath, [recordScript, ...args, "--package-root", packageRoot]);
const expected = computeBuildInputsDigest(repoRoot);
const short = expected.digest.slice(0, 12);
// --finish before any --begin is a wiring error, not a silent stamp.
await assert.rejects(run("--finish"), /no pending build inputs/);
assert.equal(readBuildInputsStamp(buildInputsStampPath(packageRoot)), null);
const begin = await run("--begin");
assert.match(begin.stdout, new RegExp(`^${BUILD_INPUTS_SCHEMA} begin: digest=${short} files=${INPUT_FILE_COUNT} bytes=\\d+\n$`));
assert.equal(readBuildInputsStamp(buildInputsPendingPath(packageRoot, childBuildId)).digest, expected.digest);
const finish = await run("--finish");
assert.match(
finish.stdout,
new RegExp(`^${BUILD_INPUTS_SCHEMA} recorded: digest=${short} files=${INPUT_FILE_COUNT} bytes=\\d+ outputs=${OUTPUT_FILE_COUNT} stamp=node_modules/.cache/paperclip-runner-build-inputs.json\n$`),
);
const stamp = readBuildInputsStamp(buildInputsStampPath(packageRoot));
assert.equal(stamp.digest, expected.digest);
assert.equal(stamp.platform, process.platform);
assert.equal(stamp.arch, process.arch);
assert.equal(stamp.outputCount, OUTPUT_FILE_COUNT);
assert.equal(assessRunnerBuild({ repoRoot, packageRoot }).current, true);
assert.equal((await stat(buildInputsStampPath(packageRoot))).isFile(), true);
// An edit between the two phases is reported and leaves nothing behind.
await run("--begin");
await writeFile(path.join(packageRoot, "src", "index.ts"), "export const one = 2;\n");
const moved = await run("--finish");
assert.match(moved.stdout, /^paperclip\.runner-build-inputs not recorded: inputs changed while the build ran /);
assert.equal(readBuildInputsStamp(buildInputsStampPath(packageRoot)), null);
await assert.rejects(run(), /one of --begin or --finish is required/);
await assert.rejects(run("--begin", "--finish"), /--finish cannot be combined with --begin/);
await assert.rejects(
execFileAsync(process.execPath, [recordScript, "--begin", "--bogus"]),
/unknown argument --bogus/,
);
});
});
test("two builds running at once in one checkout each keep their own pending record", async () => {
await withRepo(async ({ repoRoot, packageRoot }) => {
const stampPath = buildInputsStampPath(packageRoot);
const pendingDirectory = buildInputsPendingDirectory(packageRoot);
const first = { repoRoot, packageRoot, buildId: "shell-1" };
const second = { repoRoot, packageRoot, buildId: "shell-2" };
const digestNow = () => computeBuildInputsDigest(repoRoot).digest;
// Nothing changes: both builds record, in either finishing order, and the
// second to finish does not fail for want of a record.
beginRunnerBuild(first);
beginRunnerBuild(second);
assert.deepEqual((await readdir(pendingDirectory)).sort(), ["shell-1.json", "shell-2.json"]);
assert.equal(recordRunnerBuild(first).recorded, true);
assert.deepEqual(await readdir(pendingDirectory), ["shell-2.json"], "the first finish consumed the wrong record");
assert.equal(readBuildInputsStamp(stampPath).digest, digestNow());
assert.equal(recordRunnerBuild(second).recorded, true);
assert.deepEqual(await readdir(pendingDirectory), []);
assert.equal(assessRunnerBuild({ repoRoot, packageRoot }).current, true);
beginRunnerBuild(first);
beginRunnerBuild(second);
assert.equal(recordRunnerBuild(second).recorded, true);
assert.equal(recordRunnerBuild(first).recorded, true);
assert.equal(assessRunnerBuild({ repoRoot, packageRoot }).current, true);
// An edit lands between the two starts. The build that began before it
// must not leave a stamp whichever of the two finishes last: a stamp means
// every build that wrote dist read the inputs it names.
const source = path.join(packageRoot, "src", "index.ts");
beginRunnerBuild(first);
await writeFile(source, "export const one = 2;\n");
beginRunnerBuild(second);
const staleFirst = recordRunnerBuild(first);
assert.equal(staleFirst.recorded, false);
assert.equal(staleFirst.code, "inputs_changed_during_build");
assert.equal(readBuildInputsStamp(stampPath), null);
assert.equal(recordRunnerBuild(second).recorded, true, "the build that read the edited inputs was not recorded");
assert.equal(assessRunnerBuild({ repoRoot, packageRoot }).current, true);
beginRunnerBuild(first);
await writeFile(source, "export const one = 3;\n");
beginRunnerBuild(second);
assert.equal(recordRunnerBuild(second).recorded, true);
assert.equal(readBuildInputsStamp(stampPath).digest, digestNow());
const staleLast = recordRunnerBuild(first);
assert.equal(staleLast.recorded, false);
assert.equal(staleLast.code, "inputs_changed_during_build");
assert.equal(readBuildInputsStamp(stampPath), null, "a build that began before the edit left the other build's stamp standing");
assert.equal(assessRunnerBuild({ repoRoot, packageRoot }).code, "no_stamp");
assert.deepEqual(await readdir(pendingDirectory), []);
});
});
test("a build's start drops the pending records of builds that never finished, and no others", async () => {
await withRepo(async ({ repoRoot, packageRoot }) => {
const pendingDirectory = buildInputsPendingDirectory(packageRoot);
const now = new Date("2026-10-11T22:00:00.000Z");
const age = async (buildId, ageMs) => {
const moment = new Date(now.getTime() - ageMs);
await utimes(buildInputsPendingPath(packageRoot, buildId), moment, moment);
};
// A build that failed a step yesterday, one that failed a minute ago, and
// one still running, all left records; a new build starts.
beginRunnerBuild({ repoRoot, packageRoot, buildId: "crashed-yesterday", now: () => now });
await age("crashed-yesterday", PENDING_RECORD_MAX_AGE_MS + 60_000);
beginRunnerBuild({ repoRoot, packageRoot, buildId: "crashed-just-now", now: () => now });
await age("crashed-just-now", 60_000);
beginRunnerBuild({ repoRoot, packageRoot, buildId: "still-running", now: () => now });
await writeFile(path.join(pendingDirectory, "notes.txt"), "not a record\n");
await age("still-running", 0);
beginRunnerBuild({ repoRoot, packageRoot, buildId: "new-build", now: () => now });
assert.deepEqual(
(await readdir(pendingDirectory)).sort(),
["crashed-just-now.json", "new-build.json", "notes.txt", "still-running.json"],
"pruning removed the wrong records",
);
// The running build and the new one still finish on their own records.
assert.equal(recordRunnerBuild({ repoRoot, packageRoot, buildId: "still-running" }).recorded, true);
assert.equal(recordRunnerBuild({ repoRoot, packageRoot, buildId: "new-build" }).recorded, true);
// A record that fell out of the directory is simply absent.
await rm(pendingDirectory, { recursive: true, force: true });
beginRunnerBuild({ repoRoot, packageRoot, buildId: "after-wipe" });
assert.deepEqual(await readdir(pendingDirectory), ["after-wipe.json"]);
});
});
test("two chains run by separate shells at the same time both record", { skip: process.platform === "win32" }, async () => {
await withRepo(async ({ repoRoot, packageRoot }) => {
const pendingDirectory = buildInputsPendingDirectory(packageRoot);
const marker = path.join(repoRoot, "release-the-chains");
const records = () => (existsSync(pendingDirectory) ? readdirSync(pendingDirectory).sort() : []);
const first = runChainInShell("sh", packageRoot, marker);
await waitFor(() => records().length === 1, "the first chain's record");
const second = runChainInShell("sh", packageRoot, marker);
await waitFor(() => records().length === 2, "the second chain's record");
const [firstId, secondId] = records().map((name) => name.replace(/\.json$/, ""));
assert.notEqual(firstId, secondId);
for (const id of [firstId, secondId]) {
assert.match(id, /^\d+$/, "a chain's record is named after its shell's pid");
assert.equal(readBuildInputsStamp(buildInputsPendingPath(packageRoot, id)).buildId, id);
}
await writeFile(marker, "go\n");
const results = await Promise.all([first, second]);
for (const result of results) {
assert.match(result.stdout, new RegExp(`${BUILD_INPUTS_SCHEMA} begin: `));
assert.match(result.stdout, new RegExp(`${BUILD_INPUTS_SCHEMA} recorded: `), "a chain finished without recording");
}
assert.deepEqual(records(), []);
assert.equal(assessRunnerBuild({ repoRoot, packageRoot }).current, true);
// The same chain under bash, which may exec its last command: `--finish`
// is then the shell itself and still finds the record `--begin` wrote.
let bash;
try {
bash = (await execFileAsync("bash", ["-c", "echo ok"])).stdout.trim() === "ok";
} catch {
bash = false;
}
if (bash) {
await rm(marker);
const chain = runChainInShell("bash", packageRoot, marker);
await waitFor(() => records().length === 1, "the bash chain's record");
await writeFile(marker, "go\n");
assert.match((await chain).stdout, new RegExp(`${BUILD_INPUTS_SCHEMA} recorded: `));
assert.deepEqual(records(), []);
}
});
});
@@ -0,0 +1,79 @@
// Brackets the Runner package's `build` so server's prepare:runner-vendor can
// reuse this build instead of repeating it when nothing has changed. See
// build-inputs.mjs for what counts as an input and an output.
//
// --begin first step of the build: forgets the previous stamp and records
// the inputs the build is about to read
// --finish last step of the build, after every check: writes the stamp if
// the inputs are still the ones the build began with, with a
// manifest of every file under dist; otherwise writes nothing, so
// the next consumer rebuilds
//
// Each build chain has its own pending record, named after the shell that
// runs the chain (the parent of both phases), so two Runner builds running at
// once in one checkout do not consume each other's record.
//
// Usage: node scripts/record-build-inputs.mjs (--begin | --finish) [--package-root <dir>]
// The package root defaults to this package; the flag exists for tests that
// build a throwaway package layout.
import path from "node:path";
import { fileURLToPath } from "node:url";
import {
beginRunnerBuild,
recordRunnerBuild,
repoRootFromRunnerPackage,
runnerPackageRootFromScripts,
} from "./build-inputs.mjs";
export function parseArguments(argv) {
let packageRoot = null;
let phase = null;
for (let index = 0; index < argv.length; index += 1) {
const argument = argv[index];
if (argument === "--begin" || argument === "--finish") {
if (phase) throw new Error(`${argument} cannot be combined with --${phase}`);
phase = argument.slice(2);
continue;
}
if (argument === "--package-root") {
packageRoot = argv[index + 1];
if (!packageRoot) throw new Error("--package-root needs a directory");
index += 1;
continue;
}
throw new Error(`unknown argument ${argument}`);
}
if (!phase) throw new Error("one of --begin or --finish is required");
return { phase, packageRoot };
}
export function main(argv = process.argv.slice(2), log = console.log) {
const { phase, packageRoot: requested } = parseArguments(argv);
const packageRoot = requested ? path.resolve(requested) : runnerPackageRootFromScripts();
const repoRoot = repoRootFromRunnerPackage(packageRoot);
if (phase === "begin") {
const pending = beginRunnerBuild({ repoRoot, packageRoot });
log(
`${pending.schema} begin: digest=${pending.digest.slice(0, 12)} files=${pending.fileCount} ` +
`bytes=${pending.byteCount}`,
);
return pending;
}
const record = recordRunnerBuild({ repoRoot, packageRoot });
if (!record.recorded) {
log(`${record.code === "inputs_changed_during_build" ? "paperclip.runner-build-inputs" : record.code} not recorded: ${record.detail}`);
return record;
}
log(
`${record.schema} recorded: digest=${record.digest.slice(0, 12)} files=${record.fileCount} ` +
`bytes=${record.byteCount} outputs=${record.outputCount} stamp=${path.relative(packageRoot, record.stampPath)}`,
);
return record;
}
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
main();
}
@@ -0,0 +1,21 @@
# Codex driver test ownership
The driver tests are split by contract:
- lifecycle: startup, session lifetime, shutdown and provider admission
- context-control: instructions, models, permissions, goals and turn controls
- events: canonical event and provider-notification projection
- runtime-requests: question and approval request/response protocols
- semantic-results: dynamic tools, completion receipts and unknown outcomes
- recovery: snapshot, resume, reconciliation and current tool refresh
- conformance: replay validation and controller-owned result contracts
Use the existing suites and `codex-app-server-driver.test-support.ts`. The
retired `codex-app-server-driver.test.ts` duplicated the split suites; do not
recreate it or replay one complete contract in multiple files. When splitting
another test file, move its cases and retire the old copies in the same change.
Preserve table rows, waits, actions and assertions, including failure cases.
Keep provider protocol and native lifecycle layers independent when they test
different owners. Test workspaces must use the disposable real `WORKSPACE`
fixture; do not use a managed Codex checkout as model-writable workspace.
@@ -289,18 +289,20 @@ describe("Codex app-server Codex driver", () => {
workingDirectory: WORKSPACE,
}),
).rejects.toThrow("cannot overlap host CODEX_HOME");
await expect(
makeDriver([], {
environment: {
HOME: `${WORKSPACE}/host-home`,
CODEX_HOME: "/isolated/codex",
},
}).openSession({
runId: "run-host-home",
normalizedSessionId: "normalized-host-home",
workingDirectory: WORKSPACE,
}),
).rejects.toThrow("cannot contain the host HOME");
for (const hostHome of [WORKSPACE, `${WORKSPACE}/host-home`]) {
await expect(
makeDriver([], {
environment: {
HOME: hostHome,
CODEX_HOME: "/isolated/codex",
},
}).openSession({
runId: "run-host-home",
normalizedSessionId: "normalized-host-home",
workingDirectory: WORKSPACE,
}),
).rejects.toThrow("cannot contain the host HOME");
}
await expect(
makeDriver([]).openSession({
runId: "run-root",
@@ -614,15 +616,16 @@ describe("Codex app-server Codex driver", () => {
// Both denials a real app-server sends: the method is absent, and the
// build has the feature switched off.
for (const denial of [
new CodexRpcError(
'{"code":-32601,"message":"method not found"}',
CODEX_METHOD_NOT_FOUND,
),
new CodexRpcError(
'{"code":-32600,"message":"goals feature is disabled"}',
CODEX_INVALID_REQUEST,
),
for (const { denial, availability } of [
{ denial: new CodexRpcError(
'{"code":-32601,"message":"method not found"}', CODEX_METHOD_NOT_FOUND,
), availability: "unsupported" },
{ denial: new CodexRpcError(
'{"code":-32600,"message":"goals feature is disabled"}', CODEX_INVALID_REQUEST,
), availability: "policy_disabled" },
{ denial: new CodexRpcError(
'{"code":-32004,"message":"goals feature is disabled by policy"}', -32_004,
), availability: "policy_disabled" },
]) {
const unsupportedTransport = new FakeCodexTransport();
unsupportedTransport.rejectMethods.set("thread/goal/get", denial);
@@ -635,6 +638,11 @@ describe("Codex app-server Codex driver", () => {
expect((await unsupportedDriver.descriptor()).capabilities).toMatchObject(
{ goals: false },
);
const unsupportedEvents = unsupported.events()[Symbol.asyncIterator]();
await unsupportedEvents.next();
await expect(unsupportedEvents.next()).resolves.toMatchObject({
value: { eventType: "session.capabilities.updated", payload: { sessionGoals: { availability } } },
});
await expect(
unsupported.goal?.({ action: "get" }),
).rejects.toBeInstanceOf(HarnessCapabilityUnavailableError);
@@ -681,4 +689,189 @@ describe("Codex app-server Codex driver", () => {
}
});
it.each([false, true])("requires trusted continuation metadata before omitting task context (%s)", async (continuation) => {
const transport = new FakeCodexTransport();
const session = await makeDriver([transport], {
skillInputs: [{ type: "skill", name: "first-task", path: "/skills/first-task/SKILL.md" }],
}).openSession({ runId: "run-delta", normalizedSessionId: "session-delta", workingDirectory: WORKSPACE });
const text = JSON.stringify({ schema: "paperclip.native-continuation.v1", events: '{"messages":[{"body":"Go ahead"}]}', completion: { revision: "2", criterionIds: ["comment"] } });
await session.startTurn({ message: { role: "user", text }, ...(continuation ? { continuation: true as const } : {}) });
const params = transport.calls.find((call) => call.method === "turn/start")!.params;
if (continuation) {
expect(params.input).toEqual([{ type: "text", text, text_elements: [] }]);
expect(JSON.stringify(params.input)).not.toContain("constraints");
expect(JSON.stringify(params.input)).not.toContain("first-task");
} else {
expect(JSON.stringify(params.input)).toContain("constraints");
expect(params.input).toContainEqual({ type: "skill", name: "first-task", path: "/skills/first-task/SKILL.md" });
}
await session.close({ reason: "test complete" });
});
it("sends the selected reasoning effort with the Codex turn", async () => {
const transport = new FakeCodexTransport();
const session = await makeDriver([transport], { model: "gpt-6-astra", reasoningEffort: "ultra" }).openSession({
runId: "run-effort",
normalizedSessionId: "session-effort",
workingDirectory: WORKSPACE,
});
await session.startTurn({ message: { role: "user", text: "Continue" } });
expect(transport.calls.find((call) => call.method === "turn/start")?.params).toMatchObject({ effort: "ultra" });
await session.close({ reason: "test complete" });
});
it("uses negotiated ACP controls through the Codex transport facade", async () => {
let controls = { steering: false, queuedFollowUp: false };
const transport = Object.assign(new FakeCodexTransport(), { turnControlCapabilities: () => ({ ...controls }) });
const session = await makeDriver([transport], {
driverIdentity: { kind: "acpx_runtime", displayName: "Pi ACP", version: "test" },
capabilities: { steering: false },
}).openSession({ runId: "run-pi", normalizedSessionId: "session-pi", workingDirectory: WORKSPACE });
try {
expect(session.turnControlCapabilities?.()).toEqual(controls);
const { turnId } = await session.startTurn({ message: { role: "user", text: "Work" } });
await expect(session.steer?.({ turnId, message: { role: "user", text: "Change" } })).rejects.toThrow("not negotiated");
controls = { steering: true, queuedFollowUp: true };
expect(session.turnControlCapabilities?.()).toEqual(controls);
const input = { turnId, correlationId: "queued-1", mode: "follow_up" as const, message: { role: "user" as const, text: "Then validate" } };
await session.steer?.(input);
expect(transport.calls.find(call => call.method === "turn/steer")?.params).toMatchObject({ expectedTurnId: turnId, mode: "follow_up", correlationId: "queued-1" });
await expect(session.steer?.(input)).rejects.toThrow("already acknowledged");
expect(transport.calls.filter(call => call.method === "turn/steer")).toHaveLength(1);
} finally { await session.close({ reason: "verified" }); }
});
it("keeps native follow-up disabled for the Codex app-server driver", async () => {
const transport = Object.assign(new FakeCodexTransport(), { turnControlCapabilities: () => ({ steering: true, queuedFollowUp: true }) });
const session = await makeDriver([transport]).openSession({ runId: "run-codex", normalizedSessionId: "session-codex", workingDirectory: WORKSPACE });
try {
expect(session.turnControlCapabilities?.()).toEqual({ steering: true, queuedFollowUp: false });
const { turnId } = await session.startTurn({ message: { role: "user", text: "Work" } });
await expect(session.steer?.({ turnId, mode: "follow_up", message: { role: "user", text: "Queue" } })).rejects.toThrow("not negotiated");
expect(transport.calls.filter(call => call.method === "turn/steer")).toHaveLength(0);
} finally { await session.close({ reason: "verified" }); }
});
it("preserves an inactive goal status while editing its objective", async () => {
const transport = new FakeCodexTransport();
const session = await makeDriver([transport]).openSession({
runId: "run-goal-inactive-edit",
normalizedSessionId: "normalized-goal-inactive-edit",
workingDirectory: WORKSPACE,
});
await session.goal?.({
action: "set",
objective: "Updated while paused",
status: "paused",
});
expect(
transport.calls.filter(({ method }) => method === "thread/goal/set"),
).toContainEqual({
method: "thread/goal/set",
params: {
threadId: "thread-1",
objective: "Updated while paused",
status: "paused",
},
});
await session.close({ reason: "fixture complete" });
});
it("rolls back only a definitely rejected idle goal autostart", async () => {
const definiteTransport = new FakeCodexTransport();
const definiteSession = await makeDriver([definiteTransport]).openSession({
runId: "run-goal-autostart-definite-rejection",
normalizedSessionId: "normalized-goal-autostart-definite-rejection",
workingDirectory: WORKSPACE,
});
definiteTransport.rejectMethods.set(
"thread/goal/set",
new CodexRpcError('{"code":-32603,"message":"internal error"}', -32_603),
);
await expect(
definiteSession.goal?.({
action: "set",
objective: "Rejected before a turn starts",
}),
).rejects.toBeInstanceOf(HarnessCapabilityUnavailableError);
definiteTransport.rejectMethods.delete("thread/goal/set");
await expect(
definiteSession.startTurn({
message: { role: "user", text: "A normal turn may still start." },
}),
).resolves.toMatchObject({ turnId: "turn-1" });
await definiteSession.close({ reason: "fixture complete" });
const ambiguousTransport = new FakeCodexTransport();
const ambiguousSession = await makeDriver([ambiguousTransport]).openSession({
runId: "run-goal-autostart-ambiguous",
normalizedSessionId: "normalized-goal-autostart-ambiguous",
workingDirectory: WORKSPACE,
});
ambiguousTransport.rejectMethods.set(
"thread/goal/set",
new Error("codex app-server transport closed"),
);
await expect(
ambiguousSession.goal?.({
action: "set",
objective: "The provider may have started this goal",
}),
).rejects.toBeInstanceOf(HarnessCapabilityUnavailableError);
ambiguousTransport.rejectMethods.delete("thread/goal/set");
await expect(
ambiguousSession.startTurn({
message: { role: "user", text: "Do not create competing work." },
}),
).rejects.toBeInstanceOf(HarnessCapabilityUnavailableError);
await ambiguousSession.close({ reason: "fixture complete" });
});
it("preserves a provider-neutral goal action subset through the Codex transport facade", async () => {
const transport = new FakeCodexTransport();
const driver = makeDriver([transport], {
goalCapability: {
actions: ["set", "clear"],
autonomousUpdates: true,
persistentAcrossResume: true,
maxObjectiveChars: 4_000,
tokenBudgetControl: false,
usageReporting: true,
},
});
const session = await driver.openSession({
runId: "run-goals-action-subset",
normalizedSessionId: "normalized-goals-action-subset",
workingDirectory: WORKSPACE,
});
const events = session.events()[Symbol.asyncIterator]();
await events.next();
await expect(events.next()).resolves.toMatchObject({
value: {
eventType: "session.capabilities.updated",
payload: {
sessionGoals: {
availability: "available",
actions: ["set", "clear"],
tokenBudgetControl: false,
},
},
},
});
await expect(
session.goal?.({ action: "pause" }),
).rejects.toBeInstanceOf(HarnessCapabilityUnavailableError);
expect(
transport.calls.filter(({ method }) => method === "thread/goal/set"),
).toHaveLength(0);
await expect(
session.goal?.({ action: "set", objective: "Finish the task" }),
).resolves.toMatchObject({ objective: "Finish the task" });
await expect(session.goal?.({ action: "clear" })).resolves.toBeNull();
await session.close({ reason: "fixture complete" });
});
});
@@ -1302,4 +1302,29 @@ describe("Codex app-server Codex driver", () => {
});
});
it("refreshes authorized tools on resume without replacing the provider thread", async () => {
const first = new FakeCodexTransport();
const second = new FakeCodexTransport();
const original = await makeDriver([first], { conversationMode: "prepared", dynamicTools: [] }).openSession({
runId: "run-connect", normalizedSessionId: "normalized-connect", workingDirectory: WORKSPACE,
});
const snapshot = await original.snapshot();
await original.close({ reason: "waiting for connection" });
const githubTool = { name: "github_search", description: "Search authorized repositories", inputSchema: { type: "object" } };
const handler = vi.fn(async () => ({ repositories: ["paperclip"] }));
const resumedDriver = makeDriver([second], { conversationMode: "prepared", dynamicTools: [githubTool], dynamicToolHandler: handler });
expect((await resumedDriver.descriptor()).capabilities.toolRefreshOnResume).toBe(true);
const recovery = await resumedDriver.recoverSession(snapshot);
expect(recovery.recovered).toBe(true);
await recovery.session!.startTurn({ message: { role: "user", text: "Continue with GitHub connected" } });
expect(second.calls.some(call => call.method === "thread/start")).toBe(false);
expect(second.calls.find(call => call.method === "thread/resume")?.params).toMatchObject({ threadId: "thread-1", dynamicTools: expect.arrayContaining([githubTool]) });
const result = await second.invoke({ id: "rpc-github", method: "item/tool/call", params: {
threadId: "thread-1", turnId: "turn-1", callId: "github-call", tool: "github_search", arguments: { query: "paperclip" },
} });
expect(result).toMatchObject({ success: true });
expect(handler).toHaveBeenCalledWith(expect.objectContaining({ tool: "github_search", threadId: "thread-1" }));
await recovery.session?.close({ reason: "test done" });
});
});
@@ -1,3 +1,4 @@
import { SemanticToolOutcomeUnknownError } from "../../contracts/native-session-backend.js";
import {
CODEX_BLOCK_RESULT_OUTPUT_SCHEMA,
CODEX_INVALID_REQUEST,
@@ -862,4 +863,159 @@ describe("Codex app-server Codex driver", () => {
?.payload,
).toMatchObject({ reportedWorkDisposition: "blocked" });
});
it("exposes and dispatches only explicit chat tools across fresh and resumed direct chat", async () => {
const first = new FakeCodexTransport();
const second = new FakeCodexTransport();
const registerDeliverable = {
name: "register_deliverable",
description: "Prepare one requested file.",
inputSchema: { type: "object", properties: {} },
};
const readCurrentWakeComments = {
name: "read_current_wake_comments",
description: "Read only comments bound into the current wake.",
inputSchema: { type: "object", properties: {} },
};
const requestHumanInput = {
name: "request_human_input",
description: "Ask one structured question through Paperclip.",
inputSchema: { type: "object", properties: {} },
};
const listChatAttachments = {
name: "list_chat_attachments",
description: "List same-conversation attachment metadata.",
inputSchema: { type: "object", properties: {} },
};
const reuseChatAttachment = {
name: "reuse_chat_attachment",
description: "Prepare one same-conversation attachment again.",
inputSchema: { type: "object", properties: {} },
};
const readChatAttachment = {
name: "read_chat_attachment",
description: "Read one same-conversation file without resending it.",
inputSchema: { type: "object", properties: {} },
};
const handler = vi.fn(async (call) => ({
interaction: { id: "interaction-direct-question", status: "pending" },
callId: call.callId,
}));
const driver = makeDriver([first, second], {
conversationMode: "direct",
dynamicTools: [
registerDeliverable,
readCurrentWakeComments,
requestHumanInput,
listChatAttachments,
reuseChatAttachment,
readChatAttachment,
{
name: "report_progress",
description: "Must remain unavailable in direct chat.",
inputSchema: { type: "object", properties: {} },
},
],
dynamicToolHandler: handler,
});
const original = await driver.openSession({
runId: "run-direct-file",
normalizedSessionId: "normalized-direct-file",
workingDirectory: WORKSPACE,
});
await original.startTurn({
message: { role: "user", text: "Please return one file." },
});
const snapshot = await original.snapshot();
await original.close({ reason: "transport lost" });
expect(
first.calls.find((call) => call.method === "thread/start")?.params
.dynamicTools,
).toEqual([
registerDeliverable,
readCurrentWakeComments,
requestHumanInput,
listChatAttachments,
reuseChatAttachment,
readChatAttachment,
]);
const freshQuestion = await first.invoke({
id: "rpc-direct-question-fresh",
method: "item/tool/call",
params: {
threadId: "thread-1",
turnId: "turn-1",
callId: "call-direct-question-fresh",
tool: "request_human_input",
arguments: { interactionKind: "questions" },
},
});
expect(freshQuestion).toMatchObject({ success: true });
expect(handler).toHaveBeenCalledWith(
expect.objectContaining({
tool: "request_human_input",
callId: "call-direct-question-fresh",
arguments: { interactionKind: "questions" },
}),
);
const recovery = await driver.recoverSession?.(snapshot);
expect(recovery).toMatchObject({ recovered: true });
expect(
second.calls.find((call) => call.method === "thread/resume")?.params
.dynamicTools,
).toEqual([
registerDeliverable,
readCurrentWakeComments,
requestHumanInput,
listChatAttachments,
reuseChatAttachment,
readChatAttachment,
]);
const resumedQuestion = await second.invoke({
id: "rpc-direct-question-resumed",
method: "item/tool/call",
params: {
threadId: "thread-1",
turnId: "turn-1",
callId: "call-direct-question-resumed",
tool: "request_human_input",
arguments: { interactionKind: "confirmation" },
},
});
expect(resumedQuestion).toMatchObject({ success: true });
expect(handler).toHaveBeenCalledWith(
expect.objectContaining({
tool: "request_human_input",
callId: "call-direct-question-resumed",
arguments: { interactionKind: "confirmation" },
}),
);
await recovery?.session?.close({ reason: "test complete" });
});
it.each(["unknown", "validation"])("preserves the dynamic tool outcome boundary (%s)", async (outcome) => {
const transport = new FakeCodexTransport();
const error = outcome === "unknown"
? new SemanticToolOutcomeUnknownError("write may have committed")
: new Error("invalid instruction input");
const handler = vi.fn(async () => { throw error; });
const session = await makeDriver([transport], {
dynamicTools: [{ name: "update_agent_instructions", description: "Write instructions.", inputSchema: { type: "object" } }],
dynamicToolHandler: handler,
}).openSession({ runId: "run-uncertain-write", normalizedSessionId: "session-uncertain-write", workingDirectory: WORKSPACE });
await session.startTurn({ message: { role: "user", text: "Update instructions." } });
const response = transport.invoke({ id: "rpc-write", method: "item/tool/call", params: {
threadId: "thread-1", turnId: "turn-1", callId: "uncertain-write", tool: "update_agent_instructions", arguments: {},
} });
if (outcome === "unknown") await expect(response).rejects.toBe(error);
else await expect(response).resolves.toMatchObject({ success: false });
transport.push("turn/completed", { threadId: "thread-1", turn: { id: "turn-1", status: "completed", items: [] } });
const events = await collectUntilTerminal(session.events());
const completions = events.filter((event) => event.eventType === "item.completed" && event.itemId === "uncertain-write");
expect(completions).toHaveLength(outcome === "unknown" ? 0 : 1);
expect(handler).toHaveBeenCalledTimes(1);
});
});
Loaded 100 of 251 files, more files were not shown because too many files have changed in this diff. Show more