mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 01:41:42 +02:00
## Thinking Path > - Paperclip manages AI agents that perform work. > - Paperclip Runner connects durable task runs to local provider processes. > - The full-stack paid matrix exposed failures after the runner integrity repair. > - Verified JavaScript entrypoints lost their relative module graph when Linux executed them through descriptor paths. > - Returned provider startup errors also remained pending and became indeterminate after recovery. > - Sparse Codex tool lifecycle events lost the `write_document` identity before task transcript projection. > - This pull request repairs those three boundaries and makes the structured-question fixture deterministic. > - The benefit is repeatable provider startup, exact failure replay, and correct inline Plan placement. ## Linked Issues or Issue Description Refs #12721 and #12700. **What happened?** The paid runner matrix failed ACPX and OpenCode startup before provider session creation. The runner journal then replaced the original startup error with an indeterminate recovery result. Native Codex saved a Plan but rendered it only as a fallback card. A legacy Claude waiting reply could also echo the reserved terminal marker before the answer arrived. **Expected behavior** Verified JavaScript providers must start from immutable descriptor-backed artifacts. Returned startup failures must persist as terminal failed command results. Native tool lifecycle updates must preserve the `write_document` boundary. Pre-answer fixture output must not contain the reserved terminal marker. **Steps to reproduce** 1. Run the local provider cells in the Runner Full-Stack E2E workflow. 2. Observe ACPX and OpenCode fail during `session.open` before provider execution. 3. Observe recovery report `execution_indeterminate` instead of the original startup error. 4. Run the native Codex Plan cell and observe the fallback Plan card after the tool activity row. 5. Run the legacy Claude structured-question resume cell and observe an early marker echo in waiting prose. **Paperclip version or commit** `0f9452101740835ce0b1488a204bf48acd5bafc3` **Deployment mode** Local development with the paid GitHub Actions acceptance workflow. ## What Changed - Bundle the ACPX sidecar and OpenCode proxy as self-contained Node ESM entrypoints before hashing and verified descriptor launch. - Anchor ACPX dynamic provider package resolution at a controller-derived provider-pack root and keep that root out of the provider child environment. - Persist executor-returned startup errors as redacted durable failed command results while retaining indeterminate recovery for true process death. - Coalesce sparse native tool items by stable ID so a late `write_document` name, input, and result reach the transcript boundary once. - Forbid the structured-question fixture from spelling or announcing its reserved terminal marker before the user answers. ## Verification - Rust and TypeScript regression tests cover durable failed replay, true crash ambiguity, bundle closure, package-root derivation, environment filtering, exact Codex tool lifecycle coalescing, and prompt determinism. - Local execution is intentionally limited to formatters and static diff checks. GitHub Actions will run tests, type checks, builds, and security checks. - After ordinary CI is green, scoped paid cells will validate one ACPX launch, one OpenCode launch, native Codex Plan projection, and legacy Claude structured resume before a complete matrix rerun. - Prior failing matrix: https://github.com/paperclipai/paperclip/actions/runs/33682434315 ## Risks - Bundling changes the bytes covered by provider launch hashes. Provider-pack generation already hashes the final built files. - ACPX still loads qualified provider packages dynamically. The controller supplies a normalized package root, while existing version, digest, path, and descriptor checks remain active. - Durable `failed` is terminal. Replays return the same redacted result and do not execute the provider effect twice. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex based on GPT-5 with agentic reasoning, repository inspection, code editing, Git, parallel subagents, and GitHub Actions coordination. The exact deployed snapshot and context-window size are not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked related public work or described the bug in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [ ] I have run tests locally and they pass (intentionally deferred to GitHub Actions) - [x] I have added or updated tests where applicable - [x] No documentation change is required for this runtime repair - [x] I have considered and documented the risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
184 lines
5.4 KiB
TypeScript
184 lines
5.4 KiB
TypeScript
import path from "node:path";
|
|
import { CREDENTIAL_NAMES } from "./types.js";
|
|
import type { MatrixExecution } from "./types.js";
|
|
|
|
const DATABASE_KEYS = ["DATABASE_URL", "DATABASE_MIGRATION_URL"] as const;
|
|
const AMBIENT_PAPERCLIP_CREDENTIAL_KEYS = [
|
|
"PAPERCLIP_API_KEY",
|
|
"PAPERCLIP_AGENT_API_KEY",
|
|
"PAPERCLIP_TASK_BRIDGE_TOKEN",
|
|
"PAPERCLIP_SETUP_TOKEN",
|
|
"PAPERCLIP_SECRETS_MASTER_KEY",
|
|
"PAPERCLIP_SECRETS_MASTER_KEY_FILE",
|
|
] as const;
|
|
const GENERATED_SERVER_SECRET_KEYS = [
|
|
"PAPERCLIP_AGENT_JWT_SECRET",
|
|
"PAPERCLIP_DECISION_SIGNING_SECRET",
|
|
"PAPERCLIP_TOOL_ACTION_SIGNING_SECRET",
|
|
"BETTER_AUTH_SECRET",
|
|
] as const;
|
|
const AMBIENT_EXTERNAL_STATE_KEYS = [
|
|
"PAPERCLIP_STORAGE_S3_BUCKET",
|
|
"PAPERCLIP_STORAGE_S3_REGION",
|
|
"PAPERCLIP_STORAGE_S3_ENDPOINT",
|
|
"PAPERCLIP_STORAGE_S3_PREFIX",
|
|
"PAPERCLIP_STORAGE_S3_FORCE_PATH_STYLE",
|
|
] as const;
|
|
const PROVIDER_SECRET_KEY = /^(?:OPENAI|ANTHROPIC|OPENROUTER|DAYTONA)(?:_|$)/;
|
|
|
|
export function runnerE2EServerControlPaths(temporaryRoot: string) {
|
|
const controlDirectory = path.join(temporaryRoot, "control");
|
|
return {
|
|
controlDirectory,
|
|
restartRequestPath: path.join(
|
|
controlDirectory,
|
|
"server-restart.request.json",
|
|
),
|
|
restartAcknowledgementPath: path.join(
|
|
controlDirectory,
|
|
"server-restart.ack.json",
|
|
),
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Native cells use the debug binary produced once by build:runner-binaries.
|
|
* Preserve an explicit override for release builds and developer workflows.
|
|
*/
|
|
export function resolvePaperclipRunnerBinaryForHarness(
|
|
executions: readonly MatrixExecution[],
|
|
repositoryRoot: string,
|
|
configuredPath = process.env.PAPERCLIP_RUNNER_BINARY,
|
|
platform: NodeJS.Platform = process.platform,
|
|
): string | undefined {
|
|
if (configuredPath?.trim()) return configuredPath;
|
|
if (
|
|
!executions.some((execution) => execution.profile.generation === "native")
|
|
) {
|
|
return undefined;
|
|
}
|
|
|
|
return path.join(
|
|
repositoryRoot,
|
|
"packages",
|
|
"paperclip-runner",
|
|
"runner",
|
|
"target",
|
|
"debug",
|
|
platform === "win32" ? "paperclip-runnerd.exe" : "paperclip-runnerd",
|
|
);
|
|
}
|
|
|
|
/**
|
|
* Remote native cells stage the same controller-owned binary whose digest is
|
|
* authorized by the PRP control plane. Local cells launch it directly.
|
|
*/
|
|
export function resolvePaperclipRemoteRunnerBinaryForHarness(
|
|
executions: readonly MatrixExecution[],
|
|
runnerBinary: string | undefined,
|
|
): string | undefined {
|
|
if (!runnerBinary) return undefined;
|
|
return executions.some(
|
|
(execution) =>
|
|
execution.profile.generation === "native" &&
|
|
execution.environment.expectedExecutionTarget.kind === "remote",
|
|
)
|
|
? runnerBinary
|
|
: undefined;
|
|
}
|
|
|
|
/**
|
|
* Keep fixture-only provider switches scoped to the one isolated harness that
|
|
* needs them. In particular, the pinned legacy OpenCode model is routed by the
|
|
* paid gateway and may not appear in OpenCode's public model catalog.
|
|
*/
|
|
export function buildRunnerE2EProcessEnvironment(
|
|
source: NodeJS.ProcessEnv,
|
|
executions: readonly MatrixExecution[],
|
|
): NodeJS.ProcessEnv {
|
|
const result = { ...source };
|
|
delete result.OPENCODE_ALLOW_ALL_MODELS;
|
|
if (
|
|
executions.length > 0 &&
|
|
executions.every(
|
|
(execution) =>
|
|
execution.profile.generation === "legacy" &&
|
|
execution.profile.provider === "opencode",
|
|
)
|
|
) {
|
|
result.OPENCODE_ALLOW_ALL_MODELS = "true";
|
|
}
|
|
return result;
|
|
}
|
|
|
|
/**
|
|
* Build the environment inherited by the Paperclip server. Paid credentials
|
|
* deliberately stay in the launcher/Playwright process and cross the server
|
|
* boundary only once, in the encrypted company-secrets API request.
|
|
*/
|
|
export function buildPaperclipServerEnvironment(
|
|
source: NodeJS.ProcessEnv,
|
|
overrides: NodeJS.ProcessEnv = {},
|
|
): NodeJS.ProcessEnv {
|
|
const result = { ...source };
|
|
for (const key of Object.keys(result)) {
|
|
if (PROVIDER_SECRET_KEY.test(key)) delete result[key];
|
|
}
|
|
for (const key of [
|
|
...CREDENTIAL_NAMES,
|
|
...DATABASE_KEYS,
|
|
...AMBIENT_PAPERCLIP_CREDENTIAL_KEYS,
|
|
...AMBIENT_EXTERNAL_STATE_KEYS,
|
|
]) {
|
|
delete result[key];
|
|
}
|
|
for (const key of GENERATED_SERVER_SECRET_KEYS) delete result[key];
|
|
Object.assign(result, overrides);
|
|
return result;
|
|
}
|
|
|
|
export function assertIsolatedServerEnvironment(
|
|
env: NodeJS.ProcessEnv,
|
|
expected: {
|
|
temporaryRoot: string;
|
|
paperclipHome: string;
|
|
configPath: string;
|
|
},
|
|
) {
|
|
const home = env.PAPERCLIP_HOME;
|
|
const config = env.PAPERCLIP_CONFIG;
|
|
if (home !== expected.paperclipHome || config !== expected.configPath) {
|
|
throw new Error(
|
|
"Paperclip server environment does not use the allocated home/config paths",
|
|
);
|
|
}
|
|
if (
|
|
!home.startsWith(`${expected.temporaryRoot}/`) ||
|
|
!config.startsWith(`${expected.temporaryRoot}/`)
|
|
) {
|
|
throw new Error(
|
|
"Paperclip server paths escape the isolated temporary root",
|
|
);
|
|
}
|
|
if (env.XDG_CACHE_HOME !== path.join(expected.temporaryRoot, "xdg-cache")) {
|
|
throw new Error(
|
|
"Paperclip server cache does not use the allocated temporary root",
|
|
);
|
|
}
|
|
for (const key of [
|
|
...CREDENTIAL_NAMES,
|
|
...DATABASE_KEYS,
|
|
...AMBIENT_PAPERCLIP_CREDENTIAL_KEYS,
|
|
...AMBIENT_EXTERNAL_STATE_KEYS,
|
|
]) {
|
|
if (env[key])
|
|
throw new Error(
|
|
`Paperclip server environment unexpectedly contains ${key}`,
|
|
);
|
|
}
|
|
for (const key of GENERATED_SERVER_SECRET_KEYS) {
|
|
if (!env[key])
|
|
throw new Error(`Paperclip server environment is missing ${key}`);
|
|
}
|
|
}
|