mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 11:01:16 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Runner starts ACP providers (Claude, Codex, Pi and others) through the ACPX sidecar. `session.open` starts the provider session. > - When the provider session does not start, Runner reports `classification=session_ensure_failed`. If the provider failed with a plain error and not with an ACP `RequestError`, no provider cause crosses the sidecar. Then the run error, the run events and the run log do not say why the session did not start. > - A provider can also type the failure, for example with ACP's authentication-required code. Runner reported that failure with the same generic classification. The server could not tell a limit or a sign-in failure at session start from other startup failures. > - On one instance, a Claude agent hit its provider usage limit. The first run failed in the turn with a typed limit failure. Each later run for many hours failed at `session.open` with only `session_ensure_failed`. Operators could only guess the cause from the timing. > - This pull request keeps a redacted, bounded cause for every session ensure rejection. It names a typed provider category in the closed classification. > - The benefit: an operator can read why a session did not start, and a later recovery change can treat a typed startup failure by its category. ## Linked Issues or Issue Description Refs #18622, which added the `ProviderFailure` cause for startup and turn failures. This change closes the remaining startup gaps. Refs #17543, which is a related open change to the same classifier. It keeps the JSON-RPC code range as a closed code. The two changes edit the same lines, so the second one to merge needs a small rebase. **What happened?** A native ACPX run failed before its first prompt with `ACPX sidecar command session.open was rejected (retryable=false, classification=session_ensure_failed)`. When the ensure rejection was not an ACP `RequestError`, the run had no other cause data. When the provider typed the failure (for example ACP `-32000` "Authentication required"), the classification stayed the same generic value. **Expected behavior** Every session ensure rejection keeps a redacted provider cause that an operator can read on the run. A typed provider failure keeps its category in the cause and in the closed classification. Provider text never goes into the closed error text. **Steps to reproduce** 1. Make `runtime.ensureSession` reject with a plain `Error`, or with an ACP error that has code `-32000`. 2. Open a qualified ACPX runtime through the sidecar. 3. Before this change, the plain error has no `providerFailure`, and the `-32000` error has category `unknown` and `classification=session_ensure_failed`. **Paperclip version or commit** `master` at `e0276e0166c`. ## What Changed - `acpxProviderFailure` reads ACP's reserved authentication-required code (`-32000`) as category `authentication`. It also reads the `errorKind` that Claude ACP puts in ACP error data (`rate_limit`, `billing_error`, `authentication_failed`, `overloaded` and others). The category table is now a `Map`, so an `errorKind` such as `constructor` cannot select an inherited object key. - The session ensure path in `codex-runtime-adapter.ts` keeps a cause for every rejection. A rejection without an ACP error keeps its own message, redacted with the launch environment and the existing credential patterns, with category `unknown`. An abort reason is Runner cancellation, so it does not become a provider cause. A non-`Error` rejection now keeps its cause on the wrapper error. - `acpx_sidecar_transport.rs`: for `ACPX_SESSION_ENSURE_FAILED`, a schema-valid cause with a typed category gives `classification=provider_<category>` (for example `provider_quota_exhausted`, `provider_rate_limit`, `provider_authentication`). An untyped or invalid cause keeps `session_ensure_failed`. Only the validated enum value is used. The cause's text never goes into the error string. - `runnerd-codex-transport.ts`: when a command result has a provider cause, the reported error now keeps Runner's closed ACPX rejection text, so the classification stays in the run error. Before, it changed that text to "Provider request failed". Any other message (for example a Codex `RPC failed: ...` message) still gets the generic text. Provider text still travels only in `providerFailure`. - The credential-free Pi probe (`pi-closed-startup.test.mjs`, `scripts/pi-public-install-probe.mjs`) now expects `provider_authentication`. Pi reports its missing credential with ACP's authentication-required error. - Docs: `doc/run-log-events.md` and `doc/architecture/runner-pi-capabilities.md`. **What the Claude ACP agent returns at session start** (`@agentclientprotocol/claude-agent-acp` 0.73.0 with the repository patch, and the ACPX 0.13.1 patch): - `newSession` and `loadSession` call `createSession`. It throws `RequestError.authRequired` (`-32000`) only for a claude.ai subscription when gateway auth is hidden. It throws `invalidParams` for a bad `cwd` and `resourceNotFound` for a missing resumed session. - Any other error, for example a failed SDK `initializationResult()`, becomes a generic `-32603 Internal error`. The ACP SDK puts the raw message in `data.details`, or the parsed JSON body in `data` when the message is JSON. - The typed AIR `sessionFailure` (category `limit`, `access` and others) is sent only on prompt responses and on `session_info_update` notifications. `loadSession` history replay also sends one for each earlier usage-limit message, but that notification does not show whether the limit is still active. The `errorKind` data is attached only by the prompt loop. - Thus a usage limit at session start has no typed signal on the ACP error. The cause keeps the provider's message, and the existing `API error: NNN` status parse can type it. This change does not add more free-text matching. **Shape for a recovery consumer:** the server reads the cause with `providerFailureFromError(error)`, as it does now. A startup cause has `phase: "startup"`, a `category` from the `ProviderFailure` schema and its `retryability`. The run error keeps `classification=provider_<category>` for a typed ensure failure, or `session_ensure_failed` when the category is `unknown`. ## Verification - `npx vitest run src/drivers/acpx src/cli src/drivers/provider-failure.test.ts src/live/runnerd-codex-transport.test.ts src/live/runnerd-codex-transport-settlement.test.ts` in `packages/paperclip-runner`: 103 of 104 files passed, 2214 tests. The 2 failures are in `installation-integrity.test.ts`. They fail the same way on unmodified `origin/master` in this checkout (a temporary `server.js` cannot resolve a module). - New tests: a plain ensure error keeps its message with the launch-environment secret and an `sk-ant-` value redacted; a `-32000` ensure error gives category `authentication`; a non-`Error` rejection with `errorKind: "billing_error"` gives `quota_exhausted`; an aborted handshake gets no cause; typed and untyped ACP errors; a closed ACPX rejection keeps its classification and a changed one gets the generic text. - `npx vitest run src/control-plane/acpx-controller-recovery.test.ts`: 8 passed. - `cargo test -p paperclip-runner-core --no-fail-fast`: all passed. New unit test: typed categories give `provider_<category>`; `unknown`, no cause, a category outside the schema and an extra field each keep `session_ensure_failed`; other closed codes keep their own classification. New pipe test through the fake sidecar: the cause survives the rejection, the classification is typed or generic, and the cause text is not in the error string. In the first full run, two `stdio_mcp` relay tests timed out. They passed in the second full run. They do not use the changed code. - `node --test test/acpx-sidecar-contract.test.mjs test/protocol-contract.test.mjs test/pi-closed-startup.test.mjs`: 29 passed. The Pi probe is skipped without the Pi distribution. - `pnpm --filter @paperclipai/paperclip-runner typecheck` (TypeScript, `cargo fmt --check`, `cargo check`): passed. `pnpm --filter @paperclipai/paperclip-runner build`: passed. - Server, after the runner build: native provider failure, run completion, task interruption presentation, Antigravity probe and the related native session executor tests: 639 passed. - Not verified: a live Claude account at its usage limit. I could not run the credential-free Pi probe, because this checkout has no Pi distribution. Its new expectation follows from the Pi patch, which raises `RequestError.authRequired` for the missing credential. ## Risks - The run error for an ACPX startup rejection with a cause now shows Runner's closed rejection text, not "Provider request failed". This text has only closed codes and is the same text that the run error showed before #18622. Provider text stays in `providerFailure`. - A plain ensure error message now becomes a displayed provider cause. Redaction applies before the 4,000-character bound. A new credential format that the patterns do not know can still need a new pattern. The same risk applies to all provider causes. - The classification of a typed ensure failure changes from `session_ensure_failed` to `provider_<category>`. No server code matches `session_ensure_failed`. The Pi probe expectation is updated. - The category and retryability are evidence only. This change does not change retry policy. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use (shell, file edits, test runs). The runtime does not expose the context window or reasoning configuration. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
112 lines
7.1 KiB
JavaScript
112 lines
7.1 KiB
JavaScript
#!/usr/bin/env node
|
|
// Run in a credential-free public npm consumer after `paperclipai runtime setup pi`.
|
|
// Use the server's normal packaged daemon resolver; never submit a prompt.
|
|
import assert from 'node:assert/strict';
|
|
import { createHash } from 'node:crypto';
|
|
import { mkdir, mkdtemp, readFile, realpath, rm } from 'node:fs/promises';
|
|
import { tmpdir } from 'node:os';
|
|
import { join, resolve } from 'node:path';
|
|
import { fileURLToPath, pathToFileURL } from 'node:url';
|
|
|
|
export async function verifyPiPackagedDaemon(server, daemon, target, runnerBinaryTarget) {
|
|
assert.match(target, /^(?:linux|darwin)-(?:x64|arm64)$/);
|
|
const output = join(server, 'dist/vendor/paperclip-runner'), manifestPath = join(output, 'bin/release-manifest.json');
|
|
let manifest;
|
|
try { manifest = JSON.parse(await readFile(manifestPath, 'utf8')); }
|
|
catch (error) { if (error.code !== 'ENOENT') throw error; }
|
|
let expectedPath = join(output, 'bin/paperclip-runnerd');
|
|
if (manifest !== undefined) {
|
|
assert.ok(manifest && typeof manifest === 'object' && !Array.isArray(manifest), 'Release manifest must be an object');
|
|
assert.equal(await realpath(manifestPath), manifestPath, 'Release manifest must remain inside the installed package');
|
|
assert.equal(manifest.schema, 'paperclip.runner.release-binaries.v1');
|
|
assert.match(manifest.sourceRevision, /^[a-f0-9]{40}$/);
|
|
const build = JSON.parse(await readFile(join(server, 'dist/build-info.json'), 'utf8'));
|
|
assert.equal(manifest.sourceRevision, build.commit, 'Packaged daemon source must match the installed server build');
|
|
const artifact = manifest.platforms?.[target];
|
|
assert.equal(artifact?.path, `${target}/paperclip-runnerd`, 'Packaged daemon must use the known host target path');
|
|
assert.match(artifact.sha256, /^sha256:[a-f0-9]{64}$/);
|
|
expectedPath = join(output, 'bin', artifact.path);
|
|
assert.equal(await realpath(daemon), expectedPath, 'Production must select the installed host daemon');
|
|
assert.equal(await realpath(expectedPath), expectedPath, 'Packaged daemon cannot escape through a symlink');
|
|
const bytes = await readFile(expectedPath);
|
|
assert.equal(`sha256:${createHash('sha256').update(bytes).digest('hex')}`, artifact.sha256);
|
|
assert.equal(runnerBinaryTarget(bytes), target);
|
|
} else {
|
|
assert.equal(await realpath(daemon), expectedPath);
|
|
assert.equal(runnerBinaryTarget(await readFile(expectedPath)), target);
|
|
}
|
|
}
|
|
|
|
async function main() {
|
|
|
|
const [serverDirectory, ...extra] = process.argv.slice(2);
|
|
assert.ok(serverDirectory && extra.length === 0, 'Usage: pi-public-install-probe.mjs INSTALLED_SERVER_ROOT');
|
|
for (const key of Object.keys(process.env)) {
|
|
assert.ok(!/(?:API_KEY|TOKEN|SECRET|PASSWORD)$/.test(key), `Unexpected credential variable: ${key}`);
|
|
}
|
|
assert.equal(process.env.PAPERCLIP_RUNNER_BINARY, undefined);
|
|
assert.equal(process.env.NODE_OPTIONS, undefined);
|
|
assert.equal(process.env.NODE_PATH, undefined);
|
|
const server = await realpath(serverDirectory);
|
|
assert.equal(JSON.parse(await readFile(join(server, 'package.json'), 'utf8')).name, '@paperclipai/server');
|
|
const { resolvePaperclipRunnerBinary } = await import(pathToFileURL(join(server, 'dist/services/native-runtime/native-codex-runner.js')));
|
|
const { createCapabilityRunnerdCodexTransport } = await import(pathToFileURL(join(server, 'dist/vendor/paperclip-runner/live/runnerd-codex-transport.js')));
|
|
const { QUALIFIED_ACPX_PROFILES } = await import(pathToFileURL(join(server, 'dist/vendor/paperclip-runner/drivers/acpx/qualified-profiles.js')));
|
|
assert.equal(QUALIFIED_ACPX_PROFILES.pi.agentProfileVersion, 37);
|
|
assert.equal(QUALIFIED_ACPX_PROFILES.pi.commandDigest, 'sha256:42e186b921f36accf3087cf64986a88346dcdb0bdf78c2ef44a2c4f8abd3453a');
|
|
assert.equal(Object.hasOwn(QUALIFIED_ACPX_PROFILES.pi, 'qualificationModel'), false);
|
|
assert.equal(Object.hasOwn(QUALIFIED_ACPX_PROFILES.pi, 'reportedModelId'), false);
|
|
const daemon = resolvePaperclipRunnerBinary();
|
|
const { runnerBinaryTarget } = await import(pathToFileURL(join(server, 'dist/vendor/paperclip-runner/live/runner-binary.js')));
|
|
await verifyPiPackagedDaemon(server, daemon, `${process.platform}-${process.arch}`, runnerBinaryTarget);
|
|
const root = await mkdtemp(join(tmpdir(), 'pi-public-install-probe-'));
|
|
const evidence = [];
|
|
const started = performance.now();
|
|
const watchdog = setTimeout(() => { console.error('Pi public-install probe exceeded its cleanup deadline'); process.exit(1); }, 85_000);
|
|
let bundle;
|
|
let failure;
|
|
let commandFailure;
|
|
let settledMs;
|
|
try {
|
|
const workspace = join(root, 'workspace'); await mkdir(workspace);
|
|
bundle = createCapabilityRunnerdCodexTransport({
|
|
provider: 'acpx', acpxAgent: 'pi', piThinkingLevel: 'low', acpxPermissionMode: 'deny-all',
|
|
runnerBinary: daemon, stateDirectory: join(root, 'state'),
|
|
environment: { PATH: '/usr/bin:/bin', LANG: 'C.UTF-8' },
|
|
onEvidence: value => evidence.push(value),
|
|
});
|
|
try {
|
|
await bundle.transport.request('initialize', { clientInfo: { name: 'pi-public-install-verification', version: '1' } });
|
|
await bundle.transport.request('thread/start', {
|
|
cwd: workspace, model: 'openrouter/deepseek/deepseek-v4-flash-0731',
|
|
baseInstructions: 'Credential-free admission probe. No prompt is submitted.',
|
|
permissions: 'paperclip-runner-workspace-read-only', dynamicTools: [],
|
|
});
|
|
} catch (error) { failure = String(error); commandFailure = error.runnerCommandResult; }
|
|
settledMs = Math.round(performance.now() - started);
|
|
await bundle.transport.close();
|
|
assert.match(failure ?? '', /session\.open failed/);
|
|
assert.match(commandFailure?.result?.message ?? '', /retryable=false, classification=provider_authentication/);
|
|
assert.equal(commandFailure?.result?.providerFailure?.schema, 'paperclip.provider-failure.v1');
|
|
assert.equal(commandFailure.result.providerFailure.provider, 'pi');
|
|
assert.equal(commandFailure.result.providerFailure.phase, 'startup');
|
|
assert.equal(commandFailure.result.providerFailure.category, 'authentication');
|
|
assert.doesNotMatch(failure, /timed out/);
|
|
assert.ok(settledMs < 60_000, `Pi admission took ${settledMs} ms`);
|
|
assert.ok(evidence.some(item => item.runnerExited === true && item.runnerExitCode === 0));
|
|
for (const item of evidence) assert.deepEqual(item.childEnvironmentKeys, ['LANG', 'PATH']);
|
|
const state = JSON.parse(await readFile(join(root, 'state/runner/acpx-provider-state.json'), 'utf8'));
|
|
assert.equal(state.descriptor.agent, 'pi');
|
|
assert.equal(state.descriptor.agentRuntimeVersion, '1.0.0');
|
|
assert.equal(state.descriptor.piThinkingLevel, 'low');
|
|
assert.equal(state.activeTurnId, null);
|
|
assert.equal(state.identity, null);
|
|
assert.equal(state.providerExitUnconfirmed, false);
|
|
console.log(JSON.stringify({ schema: 'paperclip.pi.public-npm-install.v1', target: `${process.platform}-${process.arch}`, normalPackagedDaemon: true, exactPiProfile: QUALIFIED_ACPX_PROFILES.pi.agentProfileVersion, runtime: '1.0.0', credentials: 'none', promptCalls: 0, settledMs, cleanRunnerExit: true }));
|
|
} finally {
|
|
try { await bundle?.transport.close(); }
|
|
finally { clearTimeout(watchdog); await rm(root, { recursive: true, force: true }); }
|
|
}
|
|
}
|
|
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) await main();
|