mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 11:01:16 +01:00
fix(runner): explain ACPX session-start failures and name typed causes (#18847)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Runner starts ACP providers (Claude, Codex, Pi and others) through the ACPX sidecar. `session.open` starts the provider session. > - When the provider session does not start, Runner reports `classification=session_ensure_failed`. If the provider failed with a plain error and not with an ACP `RequestError`, no provider cause crosses the sidecar. Then the run error, the run events and the run log do not say why the session did not start. > - A provider can also type the failure, for example with ACP's authentication-required code. Runner reported that failure with the same generic classification. The server could not tell a limit or a sign-in failure at session start from other startup failures. > - On one instance, a Claude agent hit its provider usage limit. The first run failed in the turn with a typed limit failure. Each later run for many hours failed at `session.open` with only `session_ensure_failed`. Operators could only guess the cause from the timing. > - This pull request keeps a redacted, bounded cause for every session ensure rejection. It names a typed provider category in the closed classification. > - The benefit: an operator can read why a session did not start, and a later recovery change can treat a typed startup failure by its category. ## Linked Issues or Issue Description Refs #18622, which added the `ProviderFailure` cause for startup and turn failures. This change closes the remaining startup gaps. Refs #17543, which is a related open change to the same classifier. It keeps the JSON-RPC code range as a closed code. The two changes edit the same lines, so the second one to merge needs a small rebase. **What happened?** A native ACPX run failed before its first prompt with `ACPX sidecar command session.open was rejected (retryable=false, classification=session_ensure_failed)`. When the ensure rejection was not an ACP `RequestError`, the run had no other cause data. When the provider typed the failure (for example ACP `-32000` "Authentication required"), the classification stayed the same generic value. **Expected behavior** Every session ensure rejection keeps a redacted provider cause that an operator can read on the run. A typed provider failure keeps its category in the cause and in the closed classification. Provider text never goes into the closed error text. **Steps to reproduce** 1. Make `runtime.ensureSession` reject with a plain `Error`, or with an ACP error that has code `-32000`. 2. Open a qualified ACPX runtime through the sidecar. 3. Before this change, the plain error has no `providerFailure`, and the `-32000` error has category `unknown` and `classification=session_ensure_failed`. **Paperclip version or commit** `master` at `e0276e0166c`. ## What Changed - `acpxProviderFailure` reads ACP's reserved authentication-required code (`-32000`) as category `authentication`. It also reads the `errorKind` that Claude ACP puts in ACP error data (`rate_limit`, `billing_error`, `authentication_failed`, `overloaded` and others). The category table is now a `Map`, so an `errorKind` such as `constructor` cannot select an inherited object key. - The session ensure path in `codex-runtime-adapter.ts` keeps a cause for every rejection. A rejection without an ACP error keeps its own message, redacted with the launch environment and the existing credential patterns, with category `unknown`. An abort reason is Runner cancellation, so it does not become a provider cause. A non-`Error` rejection now keeps its cause on the wrapper error. - `acpx_sidecar_transport.rs`: for `ACPX_SESSION_ENSURE_FAILED`, a schema-valid cause with a typed category gives `classification=provider_<category>` (for example `provider_quota_exhausted`, `provider_rate_limit`, `provider_authentication`). An untyped or invalid cause keeps `session_ensure_failed`. Only the validated enum value is used. The cause's text never goes into the error string. - `runnerd-codex-transport.ts`: when a command result has a provider cause, the reported error now keeps Runner's closed ACPX rejection text, so the classification stays in the run error. Before, it changed that text to "Provider request failed". Any other message (for example a Codex `RPC failed: ...` message) still gets the generic text. Provider text still travels only in `providerFailure`. - The credential-free Pi probe (`pi-closed-startup.test.mjs`, `scripts/pi-public-install-probe.mjs`) now expects `provider_authentication`. Pi reports its missing credential with ACP's authentication-required error. - Docs: `doc/run-log-events.md` and `doc/architecture/runner-pi-capabilities.md`. **What the Claude ACP agent returns at session start** (`@agentclientprotocol/claude-agent-acp` 0.73.0 with the repository patch, and the ACPX 0.13.1 patch): - `newSession` and `loadSession` call `createSession`. It throws `RequestError.authRequired` (`-32000`) only for a claude.ai subscription when gateway auth is hidden. It throws `invalidParams` for a bad `cwd` and `resourceNotFound` for a missing resumed session. - Any other error, for example a failed SDK `initializationResult()`, becomes a generic `-32603 Internal error`. The ACP SDK puts the raw message in `data.details`, or the parsed JSON body in `data` when the message is JSON. - The typed AIR `sessionFailure` (category `limit`, `access` and others) is sent only on prompt responses and on `session_info_update` notifications. `loadSession` history replay also sends one for each earlier usage-limit message, but that notification does not show whether the limit is still active. The `errorKind` data is attached only by the prompt loop. - Thus a usage limit at session start has no typed signal on the ACP error. The cause keeps the provider's message, and the existing `API error: NNN` status parse can type it. This change does not add more free-text matching. **Shape for a recovery consumer:** the server reads the cause with `providerFailureFromError(error)`, as it does now. A startup cause has `phase: "startup"`, a `category` from the `ProviderFailure` schema and its `retryability`. The run error keeps `classification=provider_<category>` for a typed ensure failure, or `session_ensure_failed` when the category is `unknown`. ## Verification - `npx vitest run src/drivers/acpx src/cli src/drivers/provider-failure.test.ts src/live/runnerd-codex-transport.test.ts src/live/runnerd-codex-transport-settlement.test.ts` in `packages/paperclip-runner`: 103 of 104 files passed, 2214 tests. The 2 failures are in `installation-integrity.test.ts`. They fail the same way on unmodified `origin/master` in this checkout (a temporary `server.js` cannot resolve a module). - New tests: a plain ensure error keeps its message with the launch-environment secret and an `sk-ant-` value redacted; a `-32000` ensure error gives category `authentication`; a non-`Error` rejection with `errorKind: "billing_error"` gives `quota_exhausted`; an aborted handshake gets no cause; typed and untyped ACP errors; a closed ACPX rejection keeps its classification and a changed one gets the generic text. - `npx vitest run src/control-plane/acpx-controller-recovery.test.ts`: 8 passed. - `cargo test -p paperclip-runner-core --no-fail-fast`: all passed. New unit test: typed categories give `provider_<category>`; `unknown`, no cause, a category outside the schema and an extra field each keep `session_ensure_failed`; other closed codes keep their own classification. New pipe test through the fake sidecar: the cause survives the rejection, the classification is typed or generic, and the cause text is not in the error string. In the first full run, two `stdio_mcp` relay tests timed out. They passed in the second full run. They do not use the changed code. - `node --test test/acpx-sidecar-contract.test.mjs test/protocol-contract.test.mjs test/pi-closed-startup.test.mjs`: 29 passed. The Pi probe is skipped without the Pi distribution. - `pnpm --filter @paperclipai/paperclip-runner typecheck` (TypeScript, `cargo fmt --check`, `cargo check`): passed. `pnpm --filter @paperclipai/paperclip-runner build`: passed. - Server, after the runner build: native provider failure, run completion, task interruption presentation, Antigravity probe and the related native session executor tests: 639 passed. - Not verified: a live Claude account at its usage limit. I could not run the credential-free Pi probe, because this checkout has no Pi distribution. Its new expectation follows from the Pi patch, which raises `RequestError.authRequired` for the missing credential. ## Risks - The run error for an ACPX startup rejection with a cause now shows Runner's closed rejection text, not "Provider request failed". This text has only closed codes and is the same text that the run error showed before #18622. Provider text stays in `providerFailure`. - A plain ensure error message now becomes a displayed provider cause. Redaction applies before the 4,000-character bound. A new credential format that the patterns do not know can still need a new pattern. The same risk applies to all provider causes. - The classification of a typed ensure failure changes from `session_ensure_failed` to `provider_<category>`. No server code matches `session_ensure_failed`. The Pi probe expectation is updated. - The category and retryability are evidence only. This change does not change retry policy. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use (shell, file edits, test runs). The runtime does not expose the context window or reasoning configuration. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
80c308f872
commit
9fb06afbfa
13 files changed
+287
-42
No files matched your search
@@ -1247,8 +1247,10 @@ This is trusted controller timing policy, outside the immutable Pi wrapper/profi
|
||||
closure; it does not change profile 13, runtime 1.0.0, or the explicit model.
|
||||
|
||||
The credential-free closed-startup regression requires the real
|
||||
`session_ensure_failed` rejection within 60 seconds, with no prompt, no provider
|
||||
identity and confirmed cleanup. Its child watchdog is 80 seconds (20 seconds for
|
||||
cleanup after admission), inside a 90-second test timeout. Failed evidence stays
|
||||
`provider_authentication` rejection within 60 seconds, with no prompt, no provider
|
||||
identity and confirmed cleanup. Pi reports the missing credential with ACP's
|
||||
authentication-required error, so the rejection names that typed category instead
|
||||
of the generic `session_ensure_failed`. Its child watchdog is 80 seconds (20
|
||||
seconds for cleanup after admission), inside a 90-second test timeout. Failed evidence stays
|
||||
in its reported temporary directory. These margins do not extend production
|
||||
command deadlines or qualify a target platform without a fresh run.
|
||||
+12
-2
@@ -100,8 +100,18 @@ adapters extract these fields; arbitrary exception objects, request bodies,
|
||||
headers, environment variables, and stacks are excluded. Unknown categories retain
|
||||
their explanation. The cause stays in company-scoped run records and the run log;
|
||||
it is not added to first-party Telemetry, Observability attributes, or Sentry
|
||||
failure reports. The reportable run error keeps its existing generic text. Startup
|
||||
receipts retain it across controller restart before any provider turn exists.
|
||||
failure reports. The reportable run error keeps its existing generic text, except
|
||||
that an ACPX startup rejection keeps Runner's closed rejection text and its
|
||||
classification. Startup receipts retain the cause across controller restart before
|
||||
any provider turn exists.
|
||||
|
||||
Every ACPX session ensure rejection carries a startup cause. When the provider
|
||||
returned no ACP error, the cause keeps the rejection's own redacted message with
|
||||
category `unknown`. A provider can type the failure with ACP's
|
||||
authentication-required code, or with a known error type or `errorKind` in the ACP
|
||||
error data. The cause then uses that category, and the rejection classification is
|
||||
`provider_<category>`, such as `provider_quota_exhausted`, instead of
|
||||
`session_ensure_failed`.
|
||||
|
||||
Completed OpenCode builtin `bash` parts preserve the numeric process exit from
|
||||
`state.metadata.exit` in `tool.execution.completed.payload.exitCode`. The tool
|
||||
|
||||
@@ -14,6 +14,7 @@ use crate::local_runner::LocalRunnerError;
|
||||
use crate::process_supervisor::{
|
||||
BoundedLogBuffer, ProcessOutput, SupervisedProcess, VerifiedProcessLaunch,
|
||||
};
|
||||
use crate::provider_failure::validated_provider_failure;
|
||||
use crate::stable_identity::{is_stable_id, DURABLE_STABLE_ID_CHARS, SHORT_STABLE_ID_CHARS};
|
||||
|
||||
pub const ACPX_SIDECAR_MAX_FRAME_BYTES: usize = 1024 * 1024;
|
||||
@@ -908,6 +909,24 @@ fn stderr_diagnostic_categories(value: &str) -> BTreeSet<&'static str> {
|
||||
categories
|
||||
}
|
||||
|
||||
// A typed ensure rejection names its provider-failure category, so a limit at
|
||||
// session start is distinguishable from other startup failures. Only the
|
||||
// schema-validated enum crosses here, never the cause's text.
|
||||
fn typed_provider_classification(error: &ResponseError) -> Option<&'static str> {
|
||||
let failure = validated_provider_failure(error.provider_failure.clone())?;
|
||||
Some(match failure.get("category")?.as_str()? {
|
||||
"rate_limit" => "provider_rate_limit",
|
||||
"quota_exhausted" => "provider_quota_exhausted",
|
||||
"authentication" => "provider_authentication",
|
||||
"permission" => "provider_permission",
|
||||
"model_unavailable" => "provider_model_unavailable",
|
||||
"invalid_request" => "provider_invalid_request",
|
||||
"overloaded" => "provider_overloaded",
|
||||
"timeout" => "provider_timeout",
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
|
||||
fn response_error_classification(error: &ResponseError) -> &'static str {
|
||||
match error.code.as_str() {
|
||||
"GROK_EFFORT_ACKNOWLEDGEMENT_MISSING" => return "grok_effort_acknowledgement_missing",
|
||||
@@ -955,7 +974,9 @@ fn response_error_classification(error: &ResponseError) -> &'static str {
|
||||
"SESSION_CONFIG_OPTION_REPLAY_FAILED" => return "session_config_option_replay_failed",
|
||||
"CLAUDE_ACP_SESSION_CREATE_TIMEOUT" => return "claude_session_create_timeout",
|
||||
"ACPX_SESSION_HANDSHAKE_TIMEOUT" => return "session_handshake_timeout",
|
||||
"ACPX_SESSION_ENSURE_FAILED" => return "session_ensure_failed",
|
||||
"ACPX_SESSION_ENSURE_FAILED" => {
|
||||
return typed_provider_classification(error).unwrap_or("session_ensure_failed")
|
||||
}
|
||||
"ACPX_SESSION_ENSURE_TYPE_ERROR" => return "session_ensure_type_error",
|
||||
"ACPX_SESSION_ENSURE_NON_ERROR" => return "session_ensure_non_error",
|
||||
"ACP_SESSION_INIT_FAILED" => return "acp_session_init_failed",
|
||||
@@ -1188,6 +1209,55 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn session_ensure_names_only_a_typed_provider_category() {
|
||||
let ensure = |provider_failure: Option<Value>| ResponseError {
|
||||
code: "ACPX_SESSION_ENSURE_FAILED".to_owned(),
|
||||
message: "private-token-canary".to_owned(),
|
||||
retryable: false,
|
||||
provider_failure,
|
||||
};
|
||||
let cause = |category: &str| {
|
||||
json!({ "schema": "paperclip.provider-failure.v1", "provider": "anthropic", "phase": "startup",
|
||||
"category": category, "message": "private-token-canary", "retryability": "unknown" })
|
||||
};
|
||||
for (category, expected) in [
|
||||
("quota_exhausted", "provider_quota_exhausted"),
|
||||
("rate_limit", "provider_rate_limit"),
|
||||
("authentication", "provider_authentication"),
|
||||
("unknown", "session_ensure_failed"),
|
||||
] {
|
||||
assert_eq!(
|
||||
response_error_classification(&ensure(Some(cause(category)))),
|
||||
expected
|
||||
);
|
||||
}
|
||||
assert_eq!(
|
||||
response_error_classification(&ensure(None)),
|
||||
"session_ensure_failed"
|
||||
);
|
||||
// A cause outside the schema cannot select a classification.
|
||||
let mut invented = cause("quota_exhausted");
|
||||
invented["category"] = json!("private-token-canary");
|
||||
assert_eq!(
|
||||
response_error_classification(&ensure(Some(invented))),
|
||||
"session_ensure_failed"
|
||||
);
|
||||
let mut extra = cause("quota_exhausted");
|
||||
extra["headers"] = json!({ "authorization": "private-token-canary" });
|
||||
assert_eq!(
|
||||
response_error_classification(&ensure(Some(extra))),
|
||||
"session_ensure_failed"
|
||||
);
|
||||
// Other closed codes keep their own classification.
|
||||
let mut admission = ensure(Some(cause("quota_exhausted")));
|
||||
admission.code = "ACPX_ADMISSION_CREDENTIAL".to_owned();
|
||||
assert_eq!(
|
||||
response_error_classification(&admission),
|
||||
"admission_credential"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classifies_only_allowlisted_internal_sidecar_failures() {
|
||||
let error = |code: &str, message: &str| ResponseError {
|
||||
|
||||
@@ -256,17 +256,21 @@ fn run() -> Result<(), Box<dyn std::error::Error>> {
|
||||
write_json(&mut stdout, &success(id + 1, command, &request))?;
|
||||
}
|
||||
"remote-error" => {
|
||||
let mut error = json!({
|
||||
"code": request["params"]["errorCode"].as_str().unwrap_or("Q7Z9"),
|
||||
"message": "violet-circuit-4821",
|
||||
"retryable": false,
|
||||
});
|
||||
if let Some(cause) = request["params"].get("providerFailure") {
|
||||
error["providerFailure"] = cause.clone();
|
||||
}
|
||||
write_json(
|
||||
&mut stdout,
|
||||
&json!({
|
||||
"protocolVersion": GENERATED_ACPX_SIDECAR_PROTOCOL_VERSION,
|
||||
"id": id,
|
||||
"ok": false,
|
||||
"error": {
|
||||
"code": request["params"]["errorCode"].as_str().unwrap_or("Q7Z9"),
|
||||
"message": "violet-circuit-4821",
|
||||
"retryable": false,
|
||||
},
|
||||
"error": error,
|
||||
}),
|
||||
)?;
|
||||
}
|
||||
|
||||
@@ -247,6 +247,35 @@ fn preserves_closed_admission_categories_over_the_sidecar_pipe() {
|
||||
transport.shutdown().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn session_ensure_rejection_carries_its_provider_cause_outside_the_error_text() {
|
||||
let mut transport = transport("remote-error", Duration::from_secs(1));
|
||||
let cause = |category: &str| {
|
||||
json!({ "schema": "paperclip.provider-failure.v1", "provider": "anthropic", "phase": "startup",
|
||||
"category": category, "message": "You've hit your limit [REDACTED] amber-cause-5512",
|
||||
"retryability": "requires_action" })
|
||||
};
|
||||
for (category, classification) in [
|
||||
("quota_exhausted", "provider_quota_exhausted"),
|
||||
("unknown", "session_ensure_failed"),
|
||||
] {
|
||||
let error = transport
|
||||
.request(
|
||||
GeneratedAcpxSidecarCommand::SessionOpen,
|
||||
json!({ "errorCode": "ACPX_SESSION_ENSURE_FAILED", "providerFailure": cause(category) }),
|
||||
)
|
||||
.unwrap_err();
|
||||
let message = error.to_string();
|
||||
assert!(message.contains(&format!(
|
||||
"retryable=false, classification={classification})"
|
||||
)));
|
||||
assert!(!message.contains("amber-cause-5512"));
|
||||
assert!(!message.contains("violet-circuit-4821"));
|
||||
assert_eq!(error.provider_failure(), Some(cause(category)));
|
||||
}
|
||||
transport.shutdown().unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn redacts_sidecar_stderr_when_the_process_exits() {
|
||||
let mut transport = transport("exit-secret", Duration::from_secs(1));
|
||||
|
||||
@@ -49,6 +49,56 @@ describe("Codex ACPX runtime adapter", () => {
|
||||
});
|
||||
expect(runtime.startTurn).not.toHaveBeenCalled();
|
||||
});
|
||||
it("keeps a redacted explanation for an ensure rejection that carries no ACP error", async () => {
|
||||
const runtime = fakeRuntime();
|
||||
const options = openOptions(fakeCommand());
|
||||
options.profile = { ...options.profile, agent: "claude" };
|
||||
options.launchEnvironment = { ANTHROPIC_API_KEY: "fixture-anthropic-secret" };
|
||||
const rejection = new Error("Claude Code exited during startup for fixture-anthropic-secret (sk-ant-api03-fixtureprivate)");
|
||||
vi.mocked(runtime.ensureSession).mockRejectedValue(rejection);
|
||||
const failure = await openCodexAcpxRuntime(options, { createRegistry: () => registry(), createStore: () => store(),
|
||||
createRuntime: () => runtime }).catch((error: unknown) => error);
|
||||
expect(failure).toBe(rejection);
|
||||
expect(failure).toMatchObject({ code: "ACPX_SESSION_ENSURE_FAILED", providerFailure: { provider: "anthropic",
|
||||
phase: "startup", category: "unknown", message: "Claude Code exited during startup for [REDACTED] ([REDACTED])" } });
|
||||
expect(JSON.stringify((failure as { providerFailure: unknown }).providerFailure)).not.toMatch(/fixture-anthropic-secret|fixtureprivate/);
|
||||
});
|
||||
it("classifies an ACP authentication-required ensure rejection", async () => {
|
||||
const runtime = fakeRuntime();
|
||||
const message = "Authentication required: This integration does not support using claude.ai subscriptions.";
|
||||
vi.mocked(runtime.ensureSession).mockRejectedValue(Object.assign(new Error(message), { code: -32000 }));
|
||||
const options = openOptions(fakeCommand());
|
||||
options.profile = { ...options.profile, agent: "claude" };
|
||||
await expect(openCodexAcpxRuntime(options, { createRegistry: () => registry(), createStore: () => store(),
|
||||
createRuntime: () => runtime })).rejects.toMatchObject({ code: "ACPX_SESSION_ENSURE_FAILED",
|
||||
providerFailure: { category: "authentication", message, phase: "startup", retryability: "requires_action" } });
|
||||
});
|
||||
it("attaches a provider cause to a non-error ensure rejection", async () => {
|
||||
const runtime = fakeRuntime();
|
||||
vi.mocked(runtime.ensureSession).mockRejectedValue({ code: -32603, message: "Internal error", data: { errorKind: "billing_error" } });
|
||||
const options = openOptions(fakeCommand());
|
||||
options.profile = { ...options.profile, agent: "claude" };
|
||||
await expect(openCodexAcpxRuntime(options, { createRegistry: () => registry(), createStore: () => store(),
|
||||
createRuntime: () => runtime })).rejects.toMatchObject({ code: "ACPX_SESSION_ENSURE_NON_ERROR",
|
||||
providerFailure: { category: "quota_exhausted", providerCode: "billing_error", phase: "startup" } });
|
||||
});
|
||||
it("does not report Runner cancellation as a provider explanation", async () => {
|
||||
const runtime = fakeRuntime();
|
||||
const controller = new AbortController();
|
||||
const reason = new Error("runtime admission cancelled");
|
||||
const rejection = new Error("ACP connection closed");
|
||||
vi.mocked(runtime.ensureSession).mockImplementation(async () => {
|
||||
controller.abort(reason);
|
||||
throw rejection;
|
||||
});
|
||||
const options = openOptions(fakeCommand());
|
||||
options.signal = controller.signal;
|
||||
await expect(openCodexAcpxRuntime(options, { createRegistry: () => registry(), createStore: () => store(),
|
||||
createRuntime: () => runtime })).rejects.toBe(reason);
|
||||
await vi.waitFor(() => expect(rejection).toHaveProperty("code", "ACPX_SESSION_ENSURE_FAILED"));
|
||||
expect(rejection).not.toHaveProperty("providerFailure");
|
||||
expect(reason).not.toHaveProperty("providerFailure");
|
||||
});
|
||||
it.each([false, true])("acknowledges configured Claude effort before admission (restoring=%s)", async restoring => {
|
||||
const runtime = fakeRuntime();
|
||||
const child = fakeChild();
|
||||
|
||||
@@ -5,6 +5,7 @@ import { resolveGrokDisableWebSearch } from "./grok-web-search.js";
|
||||
import { resolveGrokEffort, createGrokEffortAdmission } from "./grok-effort.js";
|
||||
import { resolveClaudeEffort, createClaudeEffortAdmission } from "./claude-effort.js";
|
||||
import { acpxProviderFailure } from "./provider-failure.js";
|
||||
import type { ProviderFailure } from "../../contracts/provider-failure.generated.js";
|
||||
import { createPiThinkingAdmission, resolvePiThinkingLevel } from "./pi-thinking.js";
|
||||
import { usesPreconfiguredHome } from "../preconfigured-home.js";
|
||||
import type { ChildProcess } from "node:child_process";
|
||||
@@ -600,11 +601,12 @@ export async function openQualifiedAcpxRuntime(
|
||||
return ensuredHandle;
|
||||
}) : ensuredSession)
|
||||
.catch((error: unknown) => {
|
||||
const providerFailure = acpxProviderFailure(error, options.profile.agent, "startup", options.launchEnvironment);
|
||||
if (providerFailure && error instanceof Error) {
|
||||
Object.defineProperty(error, "providerFailure", { value: providerFailure, enumerable: false });
|
||||
}
|
||||
throw classifySessionEnsureFailure(error);
|
||||
// The closed code below cannot say why the provider session did not
|
||||
// start, so every ensure rejection keeps its redacted explanation. An
|
||||
// abort reason is Runner cancellation, not a provider explanation.
|
||||
const providerFailure = acpxProviderFailure(error, options.profile.agent, "startup",
|
||||
options.launchEnvironment, options.signal?.aborted !== true);
|
||||
throw classifySessionEnsureFailure(error, providerFailure);
|
||||
});
|
||||
let lateCleanup: Promise<void> | null = null;
|
||||
try {
|
||||
@@ -706,20 +708,21 @@ export async function openQualifiedAcpxRuntime(
|
||||
/** Backward-compatible name retained for existing Codex-only consumers. */
|
||||
export const openCodexAcpxRuntime = openQualifiedAcpxRuntime;
|
||||
|
||||
function classifySessionEnsureFailure(error: unknown): Error {
|
||||
if (error instanceof Error) {
|
||||
const details = error as Error & Record<string, unknown>;
|
||||
if (typeof details.code !== "string" || details.code.length === 0) {
|
||||
details.code =
|
||||
error instanceof TypeError
|
||||
? "ACPX_SESSION_ENSURE_TYPE_ERROR"
|
||||
: "ACPX_SESSION_ENSURE_FAILED";
|
||||
}
|
||||
return error;
|
||||
function classifySessionEnsureFailure(error: unknown, providerFailure: ProviderFailure | null): Error {
|
||||
const failure = error instanceof Error ? error
|
||||
: Object.assign(new Error("ACPX session ensure rejected a non-error"), { code: "ACPX_SESSION_ENSURE_NON_ERROR" });
|
||||
const details = failure as Error & Record<string, unknown>;
|
||||
if (typeof details.code !== "string" || details.code.length === 0) {
|
||||
details.code =
|
||||
failure instanceof TypeError
|
||||
? "ACPX_SESSION_ENSURE_TYPE_ERROR"
|
||||
: "ACPX_SESSION_ENSURE_FAILED";
|
||||
}
|
||||
return Object.assign(new Error("ACPX session ensure rejected a non-error"), {
|
||||
code: "ACPX_SESSION_ENSURE_NON_ERROR",
|
||||
});
|
||||
// Keep a cause that an inner layer already attached.
|
||||
if (providerFailure && !Object.hasOwn(failure, "providerFailure")) {
|
||||
Object.defineProperty(failure, "providerFailure", { value: providerFailure, enumerable: false });
|
||||
}
|
||||
return failure;
|
||||
}
|
||||
|
||||
function raceRuntimeHandshakeWithAbort<T>(
|
||||
|
||||
@@ -17,27 +17,41 @@ function acpError(value: unknown, depth = 0): Record<string, unknown> | null {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Provider error types, plus the `errorKind` values Claude ACP attaches to
|
||||
// ACP error data so clients need not match message text.
|
||||
const CATEGORIES = new Map<string, ProviderFailure["category"]>([
|
||||
["authentication_error", "authentication"], ["permission_error", "permission"], ["overloaded_error", "overloaded"],
|
||||
["rate_limit_error", "rate_limit"], ["billing_error", "quota_exhausted"], ["invalid_request_error", "invalid_request"],
|
||||
["insufficient_quota", "quota_exhausted"], ["model_not_found", "model_unavailable"],
|
||||
["authentication_failed", "authentication"], ["oauth_org_not_allowed", "authentication"],
|
||||
["account_on_hold", "quota_exhausted"], ["rate_limit", "rate_limit"], ["overloaded", "overloaded"],
|
||||
["invalid_request", "invalid_request"],
|
||||
]);
|
||||
// ACP reserves this JSON-RPC code for "Authentication required".
|
||||
const ACP_AUTH_REQUIRED = -32000;
|
||||
|
||||
/**
|
||||
* `ownMessage` keeps an error's own message when it carries no ACP error. Use
|
||||
* it only where every rejection belongs to the provider: a failed terminal
|
||||
* result or a provider session ensure.
|
||||
*/
|
||||
export function acpxProviderFailure(error: unknown, agent: string, phase: ProviderFailure["phase"],
|
||||
environment: NodeJS.ProcessEnv = {}, terminal = false): ProviderFailure | null {
|
||||
environment: NodeJS.ProcessEnv = {}, ownMessage = false): ProviderFailure | null {
|
||||
const acp = acpError(error);
|
||||
const data = record(acp?.data);
|
||||
const detail = record(data.error);
|
||||
const fallback = terminal ? record(error) : {};
|
||||
const fallback = ownMessage ? record(error) : {};
|
||||
const message = typeof detail.message === "string" ? detail.message
|
||||
: typeof data.details === "string" ? data.details
|
||||
: typeof acp?.message === "string" ? acp.message : fallback.message;
|
||||
if (typeof message !== "string") return null;
|
||||
const status = data.status ?? data.statusCode ?? detail.status
|
||||
?? (agent === "claude" ? Number(/^API error: (\d{3})\b/.exec(message)?.[1]) : undefined);
|
||||
const code = detail.type ?? detail.code ?? data.errorType;
|
||||
const categories: Record<string, ProviderFailure["category"]> = {
|
||||
authentication_error: "authentication", permission_error: "permission", overloaded_error: "overloaded",
|
||||
rate_limit_error: "rate_limit", billing_error: "quota_exhausted", invalid_request_error: "invalid_request",
|
||||
insufficient_quota: "quota_exhausted", model_not_found: "model_unavailable",
|
||||
};
|
||||
const code = detail.type ?? detail.code ?? data.errorType ?? data.errorKind;
|
||||
const category = (typeof code === "string" ? CATEGORIES.get(code) : undefined)
|
||||
?? (acp?.code === ACP_AUTH_REQUIRED ? "authentication" : undefined);
|
||||
return createProviderFailure({ provider: agent === "claude" ? "anthropic" : agent === "codex" ? "openai" : agent,
|
||||
phase, message, httpStatus: status, providerCode: code, requestId: data.request_id,
|
||||
category: typeof code === "string" ? categories[code] : undefined, environment });
|
||||
phase, message, httpStatus: status, providerCode: code, requestId: data.request_id, category, environment });
|
||||
}
|
||||
|
||||
export function acpxTerminalResult(result: unknown, agent: string, environment: NodeJS.ProcessEnv = {}): Record<string, unknown> {
|
||||
|
||||
@@ -22,6 +22,40 @@ describe("provider failure boundary", () => {
|
||||
expect(acpxProviderFailure(new Error("internal runtime failure"), "claude", "startup")).toBeNull();
|
||||
});
|
||||
|
||||
it("classifies typed ACP errors and leaves untyped ones unknown", () => {
|
||||
const rejected = (code: number, message: string, data?: unknown) =>
|
||||
Object.assign(new Error(message), { code, ...(data === undefined ? {} : { data }) });
|
||||
// Claude ACP attaches the SDK error kind to ACP error data.
|
||||
expect(acpxProviderFailure(rejected(-32603, "Internal error", { errorKind: "billing_error" }), "claude", "startup"))
|
||||
.toMatchObject({ category: "quota_exhausted", providerCode: "billing_error", message: "Internal error", retryability: "requires_action" });
|
||||
expect(acpxProviderFailure(rejected(-32603, "Internal error", { errorKind: "rate_limit" }), "claude", "startup"))
|
||||
.toMatchObject({ category: "rate_limit", providerCode: "rate_limit" });
|
||||
// A JSON provider body becomes ACP error data on the agent side.
|
||||
expect(acpxProviderFailure(rejected(-32603, "Internal error",
|
||||
{ type: "error", error: { type: "rate_limit_error", message: "Usage limit reached." } }), "claude", "startup"))
|
||||
.toMatchObject({ category: "rate_limit", providerCode: "rate_limit_error", message: "Usage limit reached." });
|
||||
// ACP's reserved authentication-required code, as Pi and Claude raise it.
|
||||
const auth = "Authentication required: Bind the configured OpenRouter API credential to this Paperclip runtime.";
|
||||
expect(acpxProviderFailure(rejected(-32000, auth, { authMethods: [{ id: "openrouter" }] }), "pi", "startup"))
|
||||
.toEqual({ schema: "paperclip.provider-failure.v1", provider: "pi", phase: "startup", category: "authentication",
|
||||
message: auth, retryability: "requires_action" });
|
||||
for (const data of [{ details: "Query closed before response received" }, { errorKind: "server_error" }, { errorKind: "constructor" }]) {
|
||||
expect(acpxProviderFailure(rejected(-32603, "Internal error", data), "claude", "startup"))
|
||||
.toMatchObject({ category: "unknown", retryability: "unknown" });
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps a plain error's own redacted message only when asked", () => {
|
||||
const secret = "fixture-launch-secret";
|
||||
const error = new Error(`Claude Code initialization failed for ${secret} with sk-ant-api03-fixtureprivatecredential`);
|
||||
expect(acpxProviderFailure(error, "claude", "startup", { ANTHROPIC_API_KEY: secret })).toBeNull();
|
||||
const failure = acpxProviderFailure(error, "claude", "startup", { ANTHROPIC_API_KEY: secret }, true);
|
||||
expect(failure).toMatchObject({ provider: "anthropic", phase: "startup", category: "unknown",
|
||||
message: "Claude Code initialization failed for [REDACTED] with [REDACTED]" });
|
||||
expect(JSON.stringify(failure)).not.toMatch(/fixture-launch-secret|fixtureprivatecredential/);
|
||||
expect(acpxProviderFailure(new Error(""), "claude", "startup", {}, true)).toBeNull();
|
||||
});
|
||||
|
||||
it("keeps the ACPX terminal cause when the raw result exceeds its payload limit", () => {
|
||||
const terminal = acpxTerminalResult({ status: "failed", error: { ...acp,
|
||||
data: { ...acp.data, responseBody: "private-body".repeat(7000) } } }, "claude");
|
||||
|
||||
@@ -9909,6 +9909,24 @@ it.each(["RPC failed: Private provider explanation.", "failed to start ACPX prov
|
||||
expect(failure).toHaveProperty("runnerCommandResult.result.message", message);
|
||||
expect((failure as Error).message).not.toContain(providerFailure.message);
|
||||
expect(JSON.stringify(failure)).not.toContain(providerFailure.message);
|
||||
// Runner composes the ACPX rejection from closed codes, so its classification stays reportable.
|
||||
if (message.startsWith("failed to start ACPX provider")) expect((failure as Error).message).toContain(message);
|
||||
else expect((failure as Error).message).toContain('"message":"Provider request failed"');
|
||||
});
|
||||
|
||||
it("reports a generic failure when an ACPX rejection does not match Runner's closed text", async () => {
|
||||
const providerFailure = { schema: "paperclip.provider-failure.v1", provider: "anthropic", phase: "startup",
|
||||
category: "unknown", message: "Private provider explanation.", retryability: "unknown" };
|
||||
const message = "failed to start ACPX provider: ACPX sidecar command session.open was rejected (retryable=false, classification=session_ensure_failed) Private provider explanation.";
|
||||
let failure: unknown;
|
||||
try {
|
||||
await waitForRunnerCommand({ type: "session.open", deadline: Date.now() + 1000, abortOnClose: true,
|
||||
isClosed: () => false, throwIfFailed: () => {},
|
||||
command: () => ({ status: "failed", result: { result: { code: "command_execution_failed", message, providerFailure } } }),
|
||||
runnerHasExited: async () => false, failureCode: () => "startup_failed" });
|
||||
} catch (error) { failure = error; }
|
||||
expect((failure as Error).message).toContain('"message":"Provider request failed"');
|
||||
expect((failure as Error).message).not.toContain("Private provider explanation.");
|
||||
});
|
||||
|
||||
|
||||
|
||||
@@ -8114,6 +8114,14 @@ export interface RunnerCommandFailure extends Error {
|
||||
readonly runnerCommandResult: unknown;
|
||||
}
|
||||
|
||||
// Runner composes this whole ACPX rejection from closed codes; provider text
|
||||
// travels only in providerFailure. Keep its classification in the reported error.
|
||||
function closedAcpxRejection(message: unknown): string | null {
|
||||
return typeof message === "string"
|
||||
&& /^failed to start ACPX provider: ACPX sidecar command [a-z.]+ was rejected \(retryable=(?:true|false), classification=[a-z0-9_]+\)$/.test(message)
|
||||
? message : null;
|
||||
}
|
||||
|
||||
/** Shared polling path keeps startup cancellation separate from owned cleanup commands. */
|
||||
export async function waitForRunnerCommand(input: {
|
||||
type: string;
|
||||
@@ -8135,7 +8143,8 @@ export async function waitForRunnerCommand(input: {
|
||||
if (command !== undefined && command.status !== "pending") {
|
||||
const detail = record(record(command.result).result);
|
||||
const reportedResult = readProviderFailure(detail.providerFailure)
|
||||
? { ...record(command.result), result: { code: detail.code, message: "Provider request failed" } } : command.result;
|
||||
? { ...record(command.result), result: { code: detail.code,
|
||||
message: closedAcpxRejection(detail.message) ?? "Provider request failed" } } : command.result;
|
||||
const error = new Error(`PRP command ${input.type} ${command.status}: ${JSON.stringify(reportedResult,
|
||||
(key, value) => key === "providerFailure" ? undefined : value)}`) as RunnerCommandFailure;
|
||||
Object.defineProperty(error, "runnerCommandResult", { value: structuredClone(command.result), enumerable: false });
|
||||
|
||||
@@ -55,10 +55,11 @@ if (process.argv[2] === "--pi-no-key-probe") {
|
||||
const report = JSON.parse(await readFile(join(root, "report.json"), "utf8"));
|
||||
assert.equal(report.closeError, null, report.closeError);
|
||||
assert.match(report.error, /session\.open failed/);
|
||||
assert.match(report.commandFailure?.result?.message ?? "", /retryable=false, classification=session_ensure_failed/);
|
||||
assert.match(report.commandFailure?.result?.message ?? "", /retryable=false, classification=provider_authentication/);
|
||||
assert.equal(report.commandFailure?.result?.providerFailure?.schema, "paperclip.provider-failure.v1");
|
||||
assert.equal(report.commandFailure.result.providerFailure.provider, "pi");
|
||||
assert.equal(report.commandFailure.result.providerFailure.phase, "startup");
|
||||
assert.equal(report.commandFailure.result.providerFailure.category, "authentication");
|
||||
assert.doesNotMatch(report.error, /timed out/);
|
||||
assert.ok(report.settledMs < 60_000, `session.open took ${report.settledMs} ms`);
|
||||
assert.ok(report.evidence.some(item => item.runnerExited === true && item.runnerExitCode === 0));
|
||||
|
||||
@@ -86,10 +86,11 @@ try {
|
||||
settledMs = Math.round(performance.now() - started);
|
||||
await bundle.transport.close();
|
||||
assert.match(failure ?? '', /session\.open failed/);
|
||||
assert.match(commandFailure?.result?.message ?? '', /retryable=false, classification=session_ensure_failed/);
|
||||
assert.match(commandFailure?.result?.message ?? '', /retryable=false, classification=provider_authentication/);
|
||||
assert.equal(commandFailure?.result?.providerFailure?.schema, 'paperclip.provider-failure.v1');
|
||||
assert.equal(commandFailure.result.providerFailure.provider, 'pi');
|
||||
assert.equal(commandFailure.result.providerFailure.phase, 'startup');
|
||||
assert.equal(commandFailure.result.providerFailure.category, 'authentication');
|
||||
assert.doesNotMatch(failure, /timed out/);
|
||||
assert.ok(settledMs < 60_000, `Pi admission took ${settledMs} ms`);
|
||||
assert.ok(evidence.some(item => item.runnerExited === true && item.runnerExitCode === 0));
|
||||
|
||||
Reference in new issue
Block a user