mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 08:22:20 +02:00
DottaandPaperclip
5bddff0920
## Thinking Path > - Paperclip manages AI agents and their work. > - The new runner gives agents dedicated tools for common tasks. > - Some API operations and parameters have no dedicated tool. > - Agents need a controlled way to find and use those operations. > - This pull request adds API search and calls through the real server routes. > - Existing tools remain the preferred path. The new tools are disabled by default. > - Paired tests measure correctness, tool choice, cost and time. ## Linked Issues or Issue Description **Subsystem affected** Paperclip Runner contracts, production tool authority and the server API catalog. **Problem or motivation** The runner cannot use much of the API described by the old Paperclip skill. A generic HTTP client would also let agents bypass runner control rules. **Proposed solution** Add `search_api` and `call_api`. Resolve calls from the mounted API catalog. Use server-held, run-bound credentials. Preserve route checks and runner lifecycle rules. Keep the tools disabled until an operator enables selected companies. **Alternatives considered** A dedicated tool for every endpoint would add a large initial prompt. An unrestricted HTTP tool would weaken authorization and replay controls. **Roadmap alignment** This extends the native runner tooling. The repository owner requested this design and implementation. The roadmap and related open PRs were checked. No duplicate API escape-hatch PR was found. ## What Changed - Register two compact fallback tools in canonical contracts and provider projections. - Build deterministic API discovery from OpenAPI, mounted experimental routes and the old skill reference. - Execute bounded JSON, text, file and download requests through authenticated HTTP routes. - Recheck active runs, company access and work modes. Block runner lifecycle, scheduling, credential and approval bypasses. Keep routine annotation collaboration available. - Retain mutation receipts. Report uncertain outcomes without blindly repeating writes. - Add a company rollout gate and a durable eval worker with complete cost accounting checks. - Record child-task creation in the activity log with the agent and run. - Add contract, authorization, file, replay and real runnerd/PRP/HTTP tests. - Document rollout gates and paid coverage limits. The companion eval repository retains immutable attempts and reports. ## Verification - Final app commit `da58370524c3626a744eec20164397c5fb6ba9ef`: all 32 checks passed; the unrelated Storybook visual check was skipped. Greptile 5/5; no unresolved review threads. - Full Linux build and recursive typecheck passed. Repository tests were run by project and serialized shard; all 143 serialized server suites passed. - Runner TypeScript: 1,599 passed, two skipped. Rust release: 451 passing test reports. Conformance and replay parity passed. The required API check passed 837 tests, including runnerd → PRP → authority → real HTTP. - Bindings cannot enable API tools without the explicit deployment flag. Unit and real-authority tests prove the default-off boundary. - The standalone API check builds and stages its own binary. It passed after existing staged and debug binaries were removed from the test container. - UI and CLI tests passed. Initial environment failures (missing jq, Docker overlay file identity, and parallel linker memory pressure) and focused passing reruns are retained. The macOS full runner suite has platform-specific failures; Linux is the qualified full-check platform. - Eval harness: 27 tests passed; existing CI discovery ran 86 tests with two unrelated skips. Credential export rejection is tested against the actual report command. - Luna and OpenRouter Sonnet each passed 60 common-workflow runs: ten workflows, three repetitions per arm, zero unnecessary API fallback. - Sonnet passed 11 selected capability/contract cases after fixes. Gemini passed three smoke cases. DeepSeek exceeded the 120-second limit and remains unqualified. - Luna's two cost flags received focused follow-up. The original flags and a later n=1 latency flag remain visible. Sonnet had no cost or latency increase above 20%. - The catalog contains 785 entries; 58 were exercised across all stages. Most operation probes remain unrun and some need additional fixtures. Authored probes do not establish successful coverage. - Total conservative accounted cost: $9.875960. Active paid-campaign time: 88.16/90 minutes. No missing accounting. Later security and harness fixes have provider-free verification; no paid validation is claimed for those revisions. - Inspect the [qualification report](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/READINESS.md) and [verification record](https://github.com/paperclipai/paperclip-evals/blob/codex/seach-call-api-tools/evals/runner-api-tools/reports/2026-09-07-production/verification.json). ## Risks - This is a broad authenticated API surface. Keep the default-off gate until an operator selects initial rollout companies. - Paid coverage is incomplete. Small regression samples do not prove all workflows are unchanged. - A timeout or server failure can follow a committed mutation. The result reports an unknown outcome and requires state inspection. - The new definitions add prompt tokens. The report retains cost flags and cache variation. - No database migration is required. - Repository rules require code-owner approval before merge. Technical CI and automated review are complete. ## Model Used OpenAI Codex based on GPT-6 assisted with code, tests and review. The exact serving model ID and context window are not exposed in this session. It used reasoning, tool calls and code execution. Eval models: `gpt-5.6-luna` with low reasoning, `openrouter/anthropic/claude-sonnet-5`, `openrouter/google/gemini-3.8-flash`, and `openrouter/deepseek/deepseek-v4-flash-0731`. Attempts retain runtime versions, model identity, usage and source provenance. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
198 lines
15 KiB
JSON
198 lines
15 KiB
JSON
{
|
|
"name": "@paperclipai/paperclip-runner",
|
|
"version": "0.0.0",
|
|
"private": true,
|
|
"license": "MIT",
|
|
"engines": {
|
|
"node": ">=24.11.0"
|
|
},
|
|
"type": "module",
|
|
"bin": {
|
|
"paperclip-runner-eval-session": "./dist/cli/eval-session.js",
|
|
"paperclip-runner-codex-proxy": "./dist/cli/codex-app-server-unix-proxy.js",
|
|
"paperclip-runner-opencode-proxy": "./dist/cli/opencode-app-server-proxy.js",
|
|
"paperclip-runner-acpx-sidecar": "./dist/cli/acpx-runtime-sidecar.js"
|
|
},
|
|
"exports": {
|
|
".": {
|
|
"types": "./dist/index.d.ts",
|
|
"import": "./dist/index.js"
|
|
},
|
|
"./testing": {
|
|
"types": "./dist/testing.d.ts",
|
|
"import": "./dist/testing.js"
|
|
},
|
|
"./evals": {
|
|
"types": "./dist/evals/index.d.ts",
|
|
"import": "./dist/evals/index.js"
|
|
},
|
|
"./live": {
|
|
"types": "./dist/live/index.d.ts",
|
|
"import": "./dist/live/index.js"
|
|
},
|
|
"./devtools": {
|
|
"types": "./dist/devtools/index.d.ts",
|
|
"import": "./dist/devtools/index.js"
|
|
},
|
|
"./browser": {
|
|
"types": "./dist/browser/index.d.ts",
|
|
"import": "./dist/browser/index.js"
|
|
},
|
|
"./react": {
|
|
"types": "./dist/react/index.d.ts",
|
|
"import": "./dist/react/index.js"
|
|
},
|
|
"./standalone": {
|
|
"types": "./dist/standalone/index.d.ts",
|
|
"import": "./dist/standalone/index.js"
|
|
},
|
|
"./styles.css": "./styles.css"
|
|
},
|
|
"files": [
|
|
"dist",
|
|
"dist-issue-thread",
|
|
"protocol",
|
|
"styles.css",
|
|
"README.md"
|
|
],
|
|
"sideEffects": false,
|
|
"scripts": {
|
|
"build": "pnpm run check:protocol-manifest && pnpm run build:typescript && pnpm run check:capability-contract && pnpm run check:capability-inventory && pnpm run check:protocol-coverage && pnpm run check:semantic-contracts && pnpm run check:runner-workflow-traceability && pnpm run build:binary && node scripts/generate-replay-goldens.mjs --check && node scripts/generate-semantic-action-catalog.mjs --check",
|
|
"build:typescript": "pnpm run ensure:eval-build-deps && pnpm run check:protocol-types && node ./node_modules/typescript/bin/tsc --version && node ./node_modules/typescript/bin/tsc -p tsconfig.json && node ./node_modules/typescript/bin/tsc -p tsconfig.surfaces.json && node scripts/build-verified-provider-entrypoints.mjs",
|
|
"build:rust": "cargo build --manifest-path runner/Cargo.toml --locked --workspace --bins",
|
|
"build:binary": "cargo build --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/stage-runner-binary.mjs",
|
|
"build:provider-pack": "pnpm run build:typescript && node scripts/build-provider-pack.mjs",
|
|
"build:runner-binaries": "cargo build --manifest-path runner/Cargo.toml --locked --workspace --bins",
|
|
"build:browser": "vite build --config vite.config.ts",
|
|
"build:sdk": "vite build --config vite.sdk.config.ts",
|
|
"build:standalone": "vite build --config vite.standalone.config.ts",
|
|
"build:scenarios": "vite build --config vite.scenarios.config.ts && node scripts/check-capability-csp-bundle.mjs",
|
|
"build:issue-thread": "vite build --config vite.issue-thread.config.ts",
|
|
"typecheck": "pnpm run typecheck:typescript && pnpm run typecheck:rust",
|
|
"typecheck:typescript": "pnpm run ensure:eval-build-deps && node --check scripts/protocol-contract.mjs && node --check scripts/generate-protocol-manifest.mjs && node --check scripts/generate-protocol-schema-module.mjs && node --check scripts/generate-acpx-sidecar-contract.mjs && node --check scripts/generate-replay-goldens.mjs && node --check scripts/generate-semantic-action-catalog.mjs && pnpm run check:protocol-types && node ./node_modules/typescript/bin/tsc --version && node ./node_modules/typescript/bin/tsc -p tsconfig.json --noEmit && node ./node_modules/typescript/bin/tsc -p tsconfig.surfaces.json --noEmit",
|
|
"typecheck:rust": "cargo fmt --manifest-path runner/Cargo.toml --all -- --check && cargo check --manifest-path runner/Cargo.toml --locked --workspace",
|
|
"typecheck:browser": "tsc -p tsconfig.browser.json --noEmit",
|
|
"test": "pnpm run test:typescript && pnpm run test:rust",
|
|
"test:typescript": "pnpm run ensure:eval-build-deps && pnpm run build:rust && node --test test/protocol-contract.test.mjs test/acpx-sidecar-contract.test.mjs test/acpx-codex-package-contract.test.mjs scripts/aws-agentcore-provisioning.test.mjs scripts/build-verified-provider-entrypoints.test.mjs scripts/local-provider-smoke-environment.test.mjs scripts/materialize-opencode-binary.test.mjs && vitest run",
|
|
"test:rust": "cargo test --release --manifest-path runner/Cargo.toml --locked --workspace",
|
|
"test:codex": "cargo test --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --test codex_provider",
|
|
"test:durable": "cargo test --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core durable::",
|
|
"test:browser": "pnpm run build:typescript && pnpm run build:runner-binaries && pnpm run build:browser && playwright test --config devtools/browser/playwright.config.ts",
|
|
"test:sdk": "vitest run src/browser/client.test.ts src/browser/transcript-model.test.ts src/react/public-contract.test.tsx",
|
|
"test:scenarios": "vitest run src/scenarios src/live src/semantic-tools/semantic-tools.test.ts src/mock-core/capability-mock-control-plane-adapter.test.ts src/issue-thread devtools/issue-thread/src examples/scenario-explorer",
|
|
"test:browser:sdk": "pnpm run build:typescript && playwright test --config examples/playwright.config.ts",
|
|
"test:browser:scenarios": "playwright test --config examples/playwright.scenarios.config.ts && pnpm run test:browser:issue-thread",
|
|
"test:browser:issue-thread": "pnpm run build:typescript && pnpm run build:issue-thread && playwright test --config devtools/issue-thread/playwright.config.ts",
|
|
"record:sdk:codex": "pnpm run build:typescript && playwright test --config examples/playwright.real.config.ts",
|
|
"generate:protocol-manifest": "node scripts/generate-protocol-manifest.mjs",
|
|
"check:protocol-manifest": "node scripts/generate-protocol-manifest.mjs --check",
|
|
"generate:protocol-types": "node scripts/generate-protocol-schema-module.mjs && node scripts/generate-acpx-sidecar-contract.mjs",
|
|
"check:protocol-types": "node scripts/generate-protocol-schema-module.mjs --check && node scripts/generate-acpx-sidecar-contract.mjs --check",
|
|
"generate:capability-contract": "node scripts/generate-capability-contract.mjs",
|
|
"check:capability-contract": "node scripts/generate-capability-contract.mjs --check",
|
|
"generate:semantic-contracts": "pnpm run build:typescript && node scripts/generate-semantic-contracts.mjs",
|
|
"check:semantic-contracts": "pnpm run build:typescript && node scripts/generate-semantic-contracts.mjs --check",
|
|
"generate:semantic-action-catalog": "pnpm run build:typescript && node scripts/generate-semantic-action-catalog.mjs",
|
|
"check:semantic-action-catalog": "pnpm run build:typescript && node scripts/generate-semantic-action-catalog.mjs --check",
|
|
"check:protocol": "pnpm run typecheck:typescript && pnpm run check:protocol-manifest && pnpm run test:typescript && pnpm run check:replay-goldens",
|
|
"check:runner": "pnpm run typecheck:rust && pnpm run test:rust && pnpm run check:conformance-parity && pnpm run check:replay-parity",
|
|
"check:all": "pnpm run check:eval-kernel && pnpm run check:protocol && pnpm run check:runner && pnpm run check:api-authority",
|
|
"check:api-authority": "node scripts/check-api-authority.mjs",
|
|
"generate:replay-goldens": "pnpm run build:typescript && node scripts/generate-replay-goldens.mjs",
|
|
"check:replay-goldens": "pnpm run build:typescript && node scripts/generate-replay-goldens.mjs --check",
|
|
"check:browser-tokens": "node scripts/check-browser-tokens.mjs",
|
|
"check:forbidden-imports": "node scripts/check-forbidden-imports.mjs",
|
|
"check:tracked-imports": "node scripts/check-tracked-imports.mjs",
|
|
"check:numbered-milestones": "node scripts/check-numbered-milestones.mjs",
|
|
"check:package-boundaries": "node scripts/check-package-boundaries.mjs",
|
|
"check:clean-consumers": "node scripts/check-clean-consumers.mjs",
|
|
"check:eval-kernel": "pnpm --filter @paperclipai/paperclip-eval-kernel test",
|
|
"ensure:eval-build-deps": "pnpm --filter @paperclipai/paperclip-eval-kernel build",
|
|
"generate:capability-inventory": "node scripts/generate-capability-inventory.mjs",
|
|
"generate:protocol-coverage": "pnpm run build:typescript && node scripts/generate-protocol-coverage.mjs",
|
|
"check:protocol-coverage": "node scripts/generate-protocol-coverage.mjs --check",
|
|
"check:capability-inventory": "node scripts/check-capability-inventory.mjs",
|
|
"test:capability-inventory": "node scripts/check-capability-inventory.test.mjs",
|
|
"test:capability-evals": "vitest run src/conformance/capability-eval-suite.test.ts",
|
|
"test:eval-slice": "pnpm run ensure:eval-build-deps && vitest run src/eval",
|
|
"test:runner-workflow-evals": "pnpm run ensure:eval-build-deps && node --test scripts/render-runner-workflow-evalbook.test.mjs && vitest run src/eval/workflow-evals.test.ts src/eval/live-workflow-executor.test.ts",
|
|
"test:runner-protocol-eval-publish": "node --test scripts/runner-protocol-eval-campaign.test.mjs scripts/publish-runner-protocol-eval-history.test.mjs scripts/runner-protocol-eval-metrics.test.mjs scripts/runner-protocol-eval-workflow-security.test.mjs scripts/public-eval-chat.test.mjs",
|
|
"check:runner-workflow-traceability": "pnpm run build:typescript && node scripts/check-runner-workflow-traceability.mjs",
|
|
"report:capability-evals": "pnpm run build:typescript && node scripts/run-capability-eval-suite.mjs",
|
|
"report:capability-live-evals": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/run-capability-live-eval-matrix.mjs",
|
|
"report:runner-workflow-evals": "pnpm run build:typescript && node scripts/run-runner-workflow-evals.mjs",
|
|
"report:runner-live-evals": "pnpm run build:typescript && pnpm run build:runner-binaries && node scripts/run-runner-live-eval-schedule.mjs --mode nightly --execute",
|
|
"report:runner-protocol-eval:catalog": "node scripts/runner-protocol-eval-campaign.mjs catalog",
|
|
"report:runner-protocol-eval:publish": "node scripts/publish-runner-protocol-eval-history.mjs",
|
|
"report:runner-chaos-evals": "pnpm run build:typescript && node scripts/run-runner-live-eval-schedule.mjs --mode chaos",
|
|
"check:conformance-parity": "cargo test --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core runs_the_mock_core_path_with_stable_output",
|
|
"check:replay-parity": "cargo test --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core replay_fixture_parity",
|
|
"docs:validate": "node scripts/validate-doc-links.mjs",
|
|
"trace:conformance": "pnpm run trace:conformance:rust",
|
|
"trace:conformance:rust": "cargo run --quiet --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin conformance-tracer",
|
|
"trace:conformance:typescript": "pnpm run build:typescript && node dist/tracer/conformance.js",
|
|
"replay:fixture": "pnpm run build:typescript && node dist/cli/replay.js",
|
|
"trace:local-runner": "pnpm run build:typescript && pnpm run build:runner-binaries && node dist/cli/local-runner.js",
|
|
"trace:codex": "pnpm run build:typescript && node dist/cli/codex.js",
|
|
"trace:standalone": "pnpm run build:typescript && node dist/cli/paperclip-adapter.js",
|
|
"trace:live-runner": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node dist/cli/live-runner.js",
|
|
"demo:standalone": "pnpm run build:typescript && vite --config vite.standalone.config.ts --host 127.0.0.1 --port 4182",
|
|
"demo:scenarios": "pnpm run build:typescript && vite --config vite.scenarios.config.ts --host 127.0.0.1 --port 4183",
|
|
"console:issue-thread": "pnpm run build:typescript && vite --config vite.issue-thread.config.ts --host 127.0.0.1 --port 4184",
|
|
"aws-agentcore:provision": "bash scripts/aws-agentcore.sh provision",
|
|
"aws-agentcore:probe": "bash scripts/aws-agentcore.sh probe",
|
|
"aws-agentcore:lab": "bash scripts/aws-agentcore.sh lab",
|
|
"aws-agentcore:destroy": "bash scripts/aws-agentcore.sh destroy",
|
|
"test:aws-agentcore-provisioning": "node --test scripts/aws-agentcore-provisioning.test.mjs",
|
|
"smoke:capability:aws-agentcore": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/capability-aws-agentcore-smoke.mjs",
|
|
"demo:live-console": "pnpm run build:typescript && node scripts/live-console-demo-server.mjs",
|
|
"smoke:capability:ui": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/capability-issue-thread-smoke.mjs",
|
|
"smoke:capability:cleanroom": "pnpm run build:typescript && cargo build --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd && node scripts/capability-clean-room-smoke.mjs",
|
|
"smoke:local-provider": "node scripts/run-local-provider-smoke.mjs",
|
|
"console:live-console": "pnpm run build:typescript && vite --config vite.config.ts --host 127.0.0.1 --port 4180",
|
|
"console:sdk": "pnpm run build:typescript && vite --config vite.sdk.config.ts --host 127.0.0.1 --port 4181",
|
|
"browser:dev": "pnpm run build:typescript && pnpm run build:runner-binaries && vite --config vite.config.ts",
|
|
"browser:preview": "vite preview --config vite.config.ts",
|
|
"verify": "pnpm run build && pnpm run typecheck && pnpm run check:replay-goldens && pnpm run test && pnpm run test:sdk && pnpm run test:scenarios && pnpm run check:browser-tokens && pnpm run test:browser && pnpm run test:browser:sdk && pnpm run test:browser:scenarios && pnpm run check:forbidden-imports && pnpm run check:tracked-imports && pnpm run check:numbered-milestones && pnpm run check:package-boundaries && pnpm run check:clean-consumers && pnpm run docs:validate && pnpm run check:conformance-parity && pnpm run check:replay-parity && pnpm run trace:conformance && pnpm run replay:fixture && pnpm run trace:local-runner -- --quiet",
|
|
"verify:rootless": "bash scripts/verify-rootless-linux.sh"
|
|
},
|
|
"dependencies": {
|
|
"@agentclientprotocol/claude-agent-acp": "0.70.0",
|
|
"@agentclientprotocol/codex-acp": "1.6.2",
|
|
"@openai/codex": "0.153.4",
|
|
"acpx": "0.13.1",
|
|
"ajv": "^8.20.0",
|
|
"json-schema-to-ts": "^3.1.1",
|
|
"opencode-ai": "1.18.29",
|
|
"react-markdown": "^10.1.0",
|
|
"remark-gfm": "^4.0.1"
|
|
},
|
|
"peerDependencies": {
|
|
"react": ">=18",
|
|
"react-dom": ">=18"
|
|
},
|
|
"peerDependenciesMeta": {
|
|
"react": {
|
|
"optional": true
|
|
},
|
|
"react-dom": {
|
|
"optional": true
|
|
}
|
|
},
|
|
"devDependencies": {
|
|
"@paperclipai/paperclip-eval-kernel": "workspace:*",
|
|
"@playwright/test": "^1.61.1",
|
|
"@types/node": "^24.0.0",
|
|
"@types/react": "^19.2.17",
|
|
"@types/react-dom": "^19.2.5",
|
|
"@vitejs/plugin-react": "^4.7.0",
|
|
"axe-core": "^4.12.1",
|
|
"react": "^19.2.7",
|
|
"react-dom": "^19.2.7",
|
|
"tsx": "^4.23.12",
|
|
"typescript": "^7.0.2",
|
|
"vite": "^6.1.0",
|
|
"vitest": "^4.1.11"
|
|
}
|
|
}
|