mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 01:41:42 +02:00
352153b5ed
## Thinking Path Post-merge of #13557, the slowest check on the freshest fully-green PR run ([35246999382](https://github.com/paperclipai/paperclip/actions/runs/35246999382)) was `ci / General tests (server (1/12))` at **339s**. The cause is one suite: `server/src/services/native-runtime/native-codex-runner.integration.test.ts` runs 1 test in **277s of a 291s vitest step (95%)** because its `beforeAll` cargo-builds the Runner release binaries, and the general-server shards carry no Rust cache — every PR run cold-compiles the full third-party crate graph. The other 19 suites in that shard finish in under 70ms each. The obvious fix (a dedicated Rust-cached matrix lane) requires editing workflow files, which the available GitHub App credentials cannot push (`workflows` permission). But the `Verify Paperclip Runner` lanes **already restore the shared `release-runner-v1` Rust cache read-only**, and their commands are `pnpm --filter @paperclipai/paperclip-runner <package script>` — so the suite can move into a Rust-cached lane purely through script changes. ## What Changed - `scripts/run-vitest-stable.mjs`: new `general-server-native-runner` group carrying exactly that suite. Under the PR workflow (`GITHUB_WORKFLOW == "PR"`, inherited from `pr.yml` by the reusable `pr-trusted.yml`) the without-chat server shards exclude it and rebalance to ~211s of tests each. Every other caller — local runs, `release-verify.yml` under the Release / Cloud readiness workflows — keeps the suite in the shards, so a renamed or unknown workflow degrades to today's slower-but-covered behavior instead of dropping coverage. - `packages/paperclip-runner`: `test:typescript:vitest` now routes through `scripts/run-pr-vitest-lane.mjs` — the identical `ensure:eval-build-deps && build:rust && vitest run` chain (shard flags passed through), plus the native-runner group on the **final PR shard only** (`--shard=N/M` with `N == M`, i.e. today's `vitest 2/2`, the 122s lane). With the restored cache the suite's cargo build becomes an incremental rebuild. - `scripts/__tests__/run-vitest-stable-shard.test.mjs`: guards pin the whole contract — PR 12-shard coverage (shards + chat + native-runner = full server group exactly), Release/local 10-shard runs keep the suite, `pr.yml` is named `PR`, the vitest lanes partition with exactly one final shard, the package-script wiring, and the wrapper's shard/workflow gating via its `--dry-run` plan output. No workflow files change. `.github/workflows/*` are untouched. ## Verification - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs scripts/__tests__/release-verify-workflow.test.mjs`: **36/36 pass** locally on this branch (includes the new coverage, wiring, and wrapper-gating guards). Both files run in CI's `Test general-server shard partition` / `Test release verify workflow wiring` steps. - Wrapper `--dry-run` plan matrix verified for all six shard/workflow combinations plus malformed-shard rejection (pinned as a guard test). - The executing proof is this PR's own CI: `ci / Verify Paperclip Runner (vitest 2/2)` must go green while running the native-runner suite (its log will show the `general-server-native-runner` group after the package vitest shard), and the 12 `ci / General tests (server (x/12))` shards must go green without it. ## Risks - The exclusion keys on `GITHUB_WORKFLOW == "PR"`. Failure mode of a rename is safe (suite falls back into the server shards, slower but covered) and the guard test on `pr.yml`'s name makes it loud. - `vitest 2/2` grows from ~122s to an expected ~210–260s — still well under the ~306s `vitest 1/2` and ~326s e2e shards, and inside the 20-minute lane timeout. If the cache misses (key drift), the lane pays a cold compile like the server shard does today; a miss is slow, never wrong. - Double-run/coverage-loss combinations are enumerated in the wrapper header and pinned by tests: each caller runs the suite exactly once. ## Model Used Claude (Bender agent, Paperclip) — Fable 5. --- Expected savings once merged: the 339s `server (1/12)` check drops to ~265s-equivalent shard levels (~211s of tests), the slowest `ci /` check becomes the ~326s e2e shard (~13–33s off PR wall time), and every PR run stops paying ~4.5 min of billed cold Rust compile. For the merger (squash): please keep the trailer below in the squash body to preserve authorship. `Co-Authored-By: Bender (Fable) <Paperclip-Paperclip@users.noreply.github.com>` ## Related PRs Searched the GitHub PR list for prior work on this surface — related groundwork, none duplicate this change: - #13457 — restored master's Rust dependency cache on the PR runner lane (the read-only cache this PR relies on) - #13500 — made that cache key image-toolchain-independent so GitHub-hosted PR runners actually hit it - #13521 — rebalanced PR shards and split the Verify Paperclip Runner lanes this PR extends - #13557 — previous health-check iteration (split the runnerd transport suite); this PR targets the next slowest check ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above Co-authored-by: Bender (Fable) <Paperclip-Paperclip@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
204 lines
16 KiB
JSON
204 lines
16 KiB
JSON
{
|
|
"name": "@paperclipai/paperclip-runner",
|
|
"version": "0.0.0",
|
|
"private": true,
|
|
"license": "MIT",
|
|
"engines": {
|
|
"node": ">=24.11.0"
|
|
},
|
|
"type": "module",
|
|
"bin": {
|
|
"paperclip-runner-eval-session": "./dist/cli/eval-session.js",
|
|
"paperclip-runner-codex-proxy": "./dist/cli/codex-app-server-unix-proxy.js",
|
|
"paperclip-runner-opencode-proxy": "./dist/cli/opencode-app-server-proxy.js",
|
|
"paperclip-runner-acpx-sidecar": "./dist/cli/acpx-runtime-sidecar.js"
|
|
},
|
|
"exports": {
|
|
".": {
|
|
"types": "./dist/index.d.ts",
|
|
"import": "./dist/index.js"
|
|
},
|
|
"./testing": {
|
|
"types": "./dist/testing.d.ts",
|
|
"import": "./dist/testing.js"
|
|
},
|
|
"./evals": {
|
|
"types": "./dist/evals/index.d.ts",
|
|
"import": "./dist/evals/index.js"
|
|
},
|
|
"./live": {
|
|
"types": "./dist/live/index.d.ts",
|
|
"import": "./dist/live/index.js"
|
|
},
|
|
"./devtools": {
|
|
"types": "./dist/devtools/index.d.ts",
|
|
"import": "./dist/devtools/index.js"
|
|
},
|
|
"./browser": {
|
|
"types": "./dist/browser/index.d.ts",
|
|
"import": "./dist/browser/index.js"
|
|
},
|
|
"./react": {
|
|
"types": "./dist/react/index.d.ts",
|
|
"import": "./dist/react/index.js"
|
|
},
|
|
"./standalone": {
|
|
"types": "./dist/standalone/index.d.ts",
|
|
"import": "./dist/standalone/index.js"
|
|
},
|
|
"./styles.css": "./styles.css"
|
|
},
|
|
"files": [
|
|
"dist",
|
|
"dist-issue-thread",
|
|
"protocol",
|
|
"styles.css",
|
|
"README.md"
|
|
],
|
|
"sideEffects": false,
|
|
"scripts": {
|
|
"build": "pnpm run check:protocol-manifest && pnpm run build:typescript && pnpm run check:capability-contract && pnpm run check:capability-inventory && pnpm run check:protocol-coverage && pnpm run check:semantic-contracts && pnpm run check:runner-workflow-traceability && pnpm run build:binary && node scripts/generate-replay-goldens.mjs --check && node scripts/generate-semantic-action-catalog.mjs --check",
|
|
"build:typescript": "pnpm run ensure:eval-build-deps && pnpm run check:protocol-types && node ./node_modules/typescript/bin/tsc --version && node ./node_modules/typescript/bin/tsc -p tsconfig.json && node ./node_modules/typescript/bin/tsc -p tsconfig.surfaces.json && node scripts/build-verified-provider-entrypoints.mjs",
|
|
"build:rust": "cargo build --manifest-path runner/Cargo.toml --locked --workspace --bins",
|
|
"build:binary": "cargo build --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/stage-runner-binary.mjs",
|
|
"build:provider-pack": "pnpm run build:typescript && node scripts/build-provider-pack.mjs",
|
|
"build:runner-binaries": "cargo build --manifest-path runner/Cargo.toml --locked --workspace --bins",
|
|
"build:browser": "vite build --config vite.config.ts",
|
|
"build:sdk": "vite build --config vite.sdk.config.ts",
|
|
"build:standalone": "vite build --config vite.standalone.config.ts",
|
|
"build:scenarios": "vite build --config vite.scenarios.config.ts && node scripts/check-capability-csp-bundle.mjs",
|
|
"build:issue-thread": "vite build --config vite.issue-thread.config.ts",
|
|
"typecheck": "pnpm run typecheck:typescript && pnpm run typecheck:rust",
|
|
"typecheck:typescript": "pnpm run ensure:eval-build-deps && node --check scripts/protocol-contract.mjs && node --check scripts/generate-protocol-manifest.mjs && node --check scripts/generate-protocol-schema-module.mjs && node --check scripts/generate-acpx-sidecar-contract.mjs && node --check scripts/generate-replay-goldens.mjs && node --check scripts/generate-semantic-action-catalog.mjs && pnpm run check:protocol-types && node ./node_modules/typescript/bin/tsc --version && node ./node_modules/typescript/bin/tsc -p tsconfig.json --noEmit && node ./node_modules/typescript/bin/tsc -p tsconfig.surfaces.json --noEmit",
|
|
"typecheck:rust": "cargo fmt --manifest-path runner/Cargo.toml --all -- --check && cargo check --manifest-path runner/Cargo.toml --locked --workspace",
|
|
"typecheck:browser": "tsc -p tsconfig.browser.json --noEmit",
|
|
"test": "pnpm run test:typescript && pnpm run test:rust",
|
|
"test:typescript": "pnpm run test:typescript:prep && vitest run",
|
|
"test:typescript:prep": "pnpm run ensure:eval-build-deps && pnpm run build:rust && node --test test/protocol-contract.test.mjs test/acpx-sidecar-contract.test.mjs test/acpx-codex-package-contract.test.mjs scripts/aws-agentcore-provisioning.test.mjs scripts/build-verified-provider-entrypoints.test.mjs scripts/local-provider-smoke-environment.test.mjs scripts/materialize-opencode-binary.test.mjs",
|
|
"test:typescript:vitest": "node ./scripts/run-pr-vitest-lane.mjs",
|
|
"test:rust": "cargo test --release --manifest-path runner/Cargo.toml --locked --workspace",
|
|
"test:codex": "cargo test --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --test codex_provider",
|
|
"test:durable": "cargo test --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core durable::",
|
|
"test:browser": "pnpm run build:typescript && pnpm run build:runner-binaries && pnpm run build:browser && playwright test --config devtools/browser/playwright.config.ts",
|
|
"test:sdk": "vitest run src/browser/client.test.ts src/browser/transcript-model.test.ts src/react/public-contract.test.tsx",
|
|
"test:scenarios": "vitest run src/scenarios src/live src/semantic-tools/semantic-tools.test.ts src/mock-core/capability-mock-control-plane-adapter.test.ts src/issue-thread devtools/issue-thread/src examples/scenario-explorer",
|
|
"test:browser:sdk": "pnpm run build:typescript && playwright test --config examples/playwright.config.ts",
|
|
"test:browser:scenarios": "playwright test --config examples/playwright.scenarios.config.ts && pnpm run test:browser:issue-thread",
|
|
"test:browser:issue-thread": "pnpm run build:typescript && pnpm run build:issue-thread && playwright test --config devtools/issue-thread/playwright.config.ts",
|
|
"record:sdk:codex": "pnpm run build:typescript && playwright test --config examples/playwright.real.config.ts",
|
|
"generate:protocol-manifest": "node scripts/generate-protocol-manifest.mjs",
|
|
"check:protocol-manifest": "node scripts/generate-protocol-manifest.mjs --check",
|
|
"generate:protocol-types": "node scripts/generate-protocol-schema-module.mjs && node scripts/generate-acpx-sidecar-contract.mjs",
|
|
"check:protocol-types": "node scripts/generate-protocol-schema-module.mjs --check && node scripts/generate-acpx-sidecar-contract.mjs --check",
|
|
"generate:capability-contract": "node scripts/generate-capability-contract.mjs",
|
|
"check:capability-contract": "node scripts/generate-capability-contract.mjs --check",
|
|
"generate:semantic-contracts": "pnpm run build:typescript && node scripts/generate-semantic-contracts.mjs",
|
|
"check:semantic-contracts": "pnpm run build:typescript && node scripts/generate-semantic-contracts.mjs --check",
|
|
"generate:semantic-action-catalog": "pnpm run build:typescript && node scripts/generate-semantic-action-catalog.mjs",
|
|
"check:semantic-action-catalog": "pnpm run build:typescript && node scripts/generate-semantic-action-catalog.mjs --check",
|
|
"check:protocol": "pnpm run typecheck:typescript && pnpm run check:protocol-manifest && pnpm run test:typescript && pnpm run check:replay-goldens",
|
|
"check:protocol-without-vitest": "pnpm run typecheck:typescript && pnpm run check:protocol-manifest && pnpm run test:typescript:prep && pnpm run check:replay-goldens",
|
|
"check:runner": "pnpm run typecheck:rust && pnpm run test:rust && pnpm run check:conformance-parity && pnpm run check:replay-parity",
|
|
"check:all": "pnpm run check:eval-kernel && pnpm run check:protocol && pnpm run check:runner && pnpm run check:api-authority",
|
|
"check:all-without-vitest": "pnpm run check:eval-kernel && pnpm run check:protocol-without-vitest && pnpm run check:runner && pnpm run check:api-authority",
|
|
"check:static": "pnpm run check:eval-kernel && pnpm run check:protocol-without-vitest && pnpm run check:api-authority",
|
|
"check:api-authority": "node scripts/check-api-authority.mjs",
|
|
"generate:replay-goldens": "pnpm run build:typescript && node scripts/generate-replay-goldens.mjs",
|
|
"check:replay-goldens": "pnpm run build:typescript && node scripts/generate-replay-goldens.mjs --check",
|
|
"check:browser-tokens": "node scripts/check-browser-tokens.mjs",
|
|
"check:forbidden-imports": "node scripts/check-forbidden-imports.mjs",
|
|
"check:tracked-imports": "node scripts/check-tracked-imports.mjs",
|
|
"check:numbered-milestones": "node scripts/check-numbered-milestones.mjs",
|
|
"check:package-boundaries": "node scripts/check-package-boundaries.mjs",
|
|
"check:clean-consumers": "node scripts/check-clean-consumers.mjs",
|
|
"check:eval-kernel": "pnpm --filter @paperclipai/paperclip-eval-kernel test",
|
|
"ensure:eval-build-deps": "pnpm --filter @paperclipai/paperclip-eval-kernel build",
|
|
"generate:capability-inventory": "node scripts/generate-capability-inventory.mjs",
|
|
"generate:protocol-coverage": "pnpm run build:typescript && node scripts/generate-protocol-coverage.mjs",
|
|
"check:protocol-coverage": "node scripts/generate-protocol-coverage.mjs --check",
|
|
"check:capability-inventory": "node scripts/check-capability-inventory.mjs",
|
|
"test:capability-inventory": "node scripts/check-capability-inventory.test.mjs",
|
|
"test:capability-evals": "vitest run src/conformance/capability-eval-suite.test.ts",
|
|
"test:eval-slice": "pnpm run ensure:eval-build-deps && vitest run src/eval",
|
|
"test:runner-workflow-evals": "pnpm run ensure:eval-build-deps && node --test scripts/render-runner-workflow-evalbook.test.mjs && vitest run src/eval/workflow-evals.test.ts src/eval/live-workflow-executor.test.ts",
|
|
"test:runner-protocol-eval-publish": "node --test scripts/runner-protocol-eval-campaign.test.mjs scripts/publish-runner-protocol-eval-history.test.mjs scripts/runner-protocol-eval-metrics.test.mjs scripts/runner-protocol-eval-workflow-security.test.mjs scripts/public-eval-chat.test.mjs",
|
|
"check:runner-workflow-traceability": "pnpm run build:typescript && node scripts/check-runner-workflow-traceability.mjs",
|
|
"report:capability-evals": "pnpm run build:typescript && node scripts/run-capability-eval-suite.mjs",
|
|
"report:capability-live-evals": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/run-capability-live-eval-matrix.mjs",
|
|
"report:runner-workflow-evals": "pnpm run build:typescript && node scripts/run-runner-workflow-evals.mjs",
|
|
"report:runner-live-evals": "pnpm run build:typescript && pnpm run build:runner-binaries && node scripts/run-runner-live-eval-schedule.mjs --mode nightly --execute",
|
|
"report:runner-protocol-eval:catalog": "node scripts/runner-protocol-eval-campaign.mjs catalog",
|
|
"report:runner-protocol-eval:publish": "node scripts/publish-runner-protocol-eval-history.mjs",
|
|
"report:runner-chaos-evals": "pnpm run build:typescript && node scripts/run-runner-live-eval-schedule.mjs --mode chaos",
|
|
"check:conformance-parity": "cargo test --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core runs_the_mock_core_path_with_stable_output",
|
|
"check:replay-parity": "cargo test --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core replay_fixture_parity",
|
|
"docs:validate": "node scripts/validate-doc-links.mjs",
|
|
"trace:conformance": "pnpm run trace:conformance:rust",
|
|
"trace:conformance:rust": "cargo run --quiet --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin conformance-tracer",
|
|
"trace:conformance:typescript": "pnpm run build:typescript && node dist/tracer/conformance.js",
|
|
"replay:fixture": "pnpm run build:typescript && node dist/cli/replay.js",
|
|
"trace:local-runner": "pnpm run build:typescript && pnpm run build:runner-binaries && node dist/cli/local-runner.js",
|
|
"trace:codex": "pnpm run build:typescript && node dist/cli/codex.js",
|
|
"trace:standalone": "pnpm run build:typescript && node dist/cli/paperclip-adapter.js",
|
|
"trace:live-runner": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node dist/cli/live-runner.js",
|
|
"demo:standalone": "pnpm run build:typescript && vite --config vite.standalone.config.ts --host 127.0.0.1 --port 4182",
|
|
"demo:scenarios": "pnpm run build:typescript && vite --config vite.scenarios.config.ts --host 127.0.0.1 --port 4183",
|
|
"console:issue-thread": "pnpm run build:typescript && vite --config vite.issue-thread.config.ts --host 127.0.0.1 --port 4184",
|
|
"aws-agentcore:provision": "bash scripts/aws-agentcore.sh provision",
|
|
"aws-agentcore:probe": "bash scripts/aws-agentcore.sh probe",
|
|
"aws-agentcore:lab": "bash scripts/aws-agentcore.sh lab",
|
|
"aws-agentcore:destroy": "bash scripts/aws-agentcore.sh destroy",
|
|
"test:aws-agentcore-provisioning": "node --test scripts/aws-agentcore-provisioning.test.mjs",
|
|
"smoke:capability:aws-agentcore": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/capability-aws-agentcore-smoke.mjs",
|
|
"demo:live-console": "pnpm run build:typescript && node scripts/live-console-demo-server.mjs",
|
|
"smoke:capability:ui": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/capability-issue-thread-smoke.mjs",
|
|
"smoke:capability:cleanroom": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/capability-clean-room-smoke.mjs",
|
|
"smoke:local-provider": "node scripts/run-local-provider-smoke.mjs",
|
|
"console:live-console": "pnpm run build:typescript && vite --config vite.config.ts --host 127.0.0.1 --port 4180",
|
|
"console:sdk": "pnpm run build:typescript && vite --config vite.sdk.config.ts --host 127.0.0.1 --port 4181",
|
|
"browser:dev": "pnpm run build:typescript && pnpm run build:runner-binaries && vite --config vite.config.ts",
|
|
"browser:preview": "vite preview --config vite.config.ts",
|
|
"verify": "pnpm run build && pnpm run typecheck && pnpm run check:replay-goldens && pnpm run test && pnpm run test:sdk && pnpm run test:scenarios && pnpm run check:browser-tokens && pnpm run test:browser && pnpm run test:browser:sdk && pnpm run test:browser:scenarios && pnpm run check:forbidden-imports && pnpm run check:tracked-imports && pnpm run check:numbered-milestones && pnpm run check:package-boundaries && pnpm run check:clean-consumers && pnpm run docs:validate && pnpm run check:conformance-parity && pnpm run check:replay-parity && pnpm run trace:conformance && pnpm run replay:fixture && pnpm run trace:local-runner -- --quiet",
|
|
"verify:rootless": "bash scripts/verify-rootless-linux.sh"
|
|
},
|
|
"dependencies": {
|
|
"@agentclientprotocol/claude-agent-acp": "0.73.0",
|
|
"@agentclientprotocol/codex-acp": "1.6.2",
|
|
"@openai/codex": "0.153.4",
|
|
"acpx": "0.13.1",
|
|
"ajv": "^8.20.0",
|
|
"json-schema-to-ts": "^3.1.1",
|
|
"opencode-ai": "1.18.29",
|
|
"react-markdown": "^10.1.0",
|
|
"remark-gfm": "^4.0.1",
|
|
"smol-toml": "^1.4.2"
|
|
},
|
|
"peerDependencies": {
|
|
"react": ">=18",
|
|
"react-dom": ">=18"
|
|
},
|
|
"peerDependenciesMeta": {
|
|
"react": {
|
|
"optional": true
|
|
},
|
|
"react-dom": {
|
|
"optional": true
|
|
}
|
|
},
|
|
"devDependencies": {
|
|
"@paperclipai/paperclip-eval-kernel": "workspace:*",
|
|
"@playwright/test": "^1.61.1",
|
|
"@types/node": "^24.0.0",
|
|
"@types/react": "^19.2.17",
|
|
"@types/react-dom": "^19.2.5",
|
|
"@vitejs/plugin-react": "^4.7.0",
|
|
"axe-core": "^4.12.1",
|
|
"react": "^19.2.7",
|
|
"react-dom": "^19.2.7",
|
|
"tsx": "^4.23.12",
|
|
"typescript": "^7.0.2",
|
|
"vite": "^6.1.0",
|
|
"vitest": "^4.1.11"
|
|
}
|
|
}
|