Files
PaperClipAI/tests/runner-e2e/harness-env.ts
T
Dotta af3023f1e3 fix(runner): repair paid provider startup paths (#12769)
## Thinking Path

> - Paperclip manages AI agents that perform work.
> - Paperclip Runner connects durable task runs to local provider
processes.
> - The full-stack paid matrix exposed failures after the runner
integrity repair.
> - Verified JavaScript entrypoints lost their relative module graph
when Linux executed them through descriptor paths.
> - Returned provider startup errors also remained pending and became
indeterminate after recovery.
> - Sparse Codex tool lifecycle events lost the `write_document`
identity before task transcript projection.
> - This pull request repairs those three boundaries and makes the
structured-question fixture deterministic.
> - The benefit is repeatable provider startup, exact failure replay,
and correct inline Plan placement.

## Linked Issues or Issue Description

Refs #12721 and #12700.

**What happened?**

The paid runner matrix failed ACPX and OpenCode startup before provider
session creation. The runner journal then replaced the original startup
error with an indeterminate recovery result. Native Codex saved a Plan
but rendered it only as a fallback card. A legacy Claude waiting reply
could also echo the reserved terminal marker before the answer arrived.

**Expected behavior**

Verified JavaScript providers must start from immutable
descriptor-backed artifacts. Returned startup failures must persist as
terminal failed command results. Native tool lifecycle updates must
preserve the `write_document` boundary. Pre-answer fixture output must
not contain the reserved terminal marker.

**Steps to reproduce**

1. Run the local provider cells in the Runner Full-Stack E2E workflow.
2. Observe ACPX and OpenCode fail during `session.open` before provider
execution.
3. Observe recovery report `execution_indeterminate` instead of the
original startup error.
4. Run the native Codex Plan cell and observe the fallback Plan card
after the tool activity row.
5. Run the legacy Claude structured-question resume cell and observe an
early marker echo in waiting prose.

**Paperclip version or commit**

`0f9452101740835ce0b1488a204bf48acd5bafc3`

**Deployment mode**

Local development with the paid GitHub Actions acceptance workflow.

## What Changed

- Bundle the ACPX sidecar and OpenCode proxy as self-contained Node ESM
entrypoints before hashing and verified descriptor launch.
- Anchor ACPX dynamic provider package resolution at a
controller-derived provider-pack root and keep that root out of the
provider child environment.
- Persist executor-returned startup errors as redacted durable failed
command results while retaining indeterminate recovery for true process
death.
- Coalesce sparse native tool items by stable ID so a late
`write_document` name, input, and result reach the transcript boundary
once.
- Forbid the structured-question fixture from spelling or announcing its
reserved terminal marker before the user answers.

## Verification

- Rust and TypeScript regression tests cover durable failed replay, true
crash ambiguity, bundle closure, package-root derivation, environment
filtering, exact Codex tool lifecycle coalescing, and prompt
determinism.
- Local execution is intentionally limited to formatters and static diff
checks. GitHub Actions will run tests, type checks, builds, and security
checks.
- After ordinary CI is green, scoped paid cells will validate one ACPX
launch, one OpenCode launch, native Codex Plan projection, and legacy
Claude structured resume before a complete matrix rerun.
- Prior failing matrix:
https://github.com/paperclipai/paperclip/actions/runs/33682434315

## Risks

- Bundling changes the bytes covered by provider launch hashes.
Provider-pack generation already hashes the final built files.
- ACPX still loads qualified provider packages dynamically. The
controller supplies a normalized package root, while existing version,
digest, path, and descriptor checks remain active.
- Durable `failed` is terminal. Replays return the same redacted result
and do not execute the provider effect twice.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex based on GPT-5 with agentic reasoning, repository
inspection, code editing, Git, parallel subagents, and GitHub Actions
coordination. The exact deployed snapshot and context-window size are
not exposed to this task.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked related public work or described the bug in
this PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket id
- [ ] I have run tests locally and they pass (intentionally deferred to
GitHub Actions)
- [x] I have added or updated tests where applicable
- [x] No documentation change is required for this runtime repair
- [x] I have considered and documented the risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-04 07:58:44 -05:00

184 lines
5.4 KiB
TypeScript

import path from "node:path";
import { CREDENTIAL_NAMES } from "./types.js";
import type { MatrixExecution } from "./types.js";
const DATABASE_KEYS = ["DATABASE_URL", "DATABASE_MIGRATION_URL"] as const;
const AMBIENT_PAPERCLIP_CREDENTIAL_KEYS = [
"PAPERCLIP_API_KEY",
"PAPERCLIP_AGENT_API_KEY",
"PAPERCLIP_TASK_BRIDGE_TOKEN",
"PAPERCLIP_SETUP_TOKEN",
"PAPERCLIP_SECRETS_MASTER_KEY",
"PAPERCLIP_SECRETS_MASTER_KEY_FILE",
] as const;
const GENERATED_SERVER_SECRET_KEYS = [
"PAPERCLIP_AGENT_JWT_SECRET",
"PAPERCLIP_DECISION_SIGNING_SECRET",
"PAPERCLIP_TOOL_ACTION_SIGNING_SECRET",
"BETTER_AUTH_SECRET",
] as const;
const AMBIENT_EXTERNAL_STATE_KEYS = [
"PAPERCLIP_STORAGE_S3_BUCKET",
"PAPERCLIP_STORAGE_S3_REGION",
"PAPERCLIP_STORAGE_S3_ENDPOINT",
"PAPERCLIP_STORAGE_S3_PREFIX",
"PAPERCLIP_STORAGE_S3_FORCE_PATH_STYLE",
] as const;
const PROVIDER_SECRET_KEY = /^(?:OPENAI|ANTHROPIC|OPENROUTER|DAYTONA)(?:_|$)/;
export function runnerE2EServerControlPaths(temporaryRoot: string) {
const controlDirectory = path.join(temporaryRoot, "control");
return {
controlDirectory,
restartRequestPath: path.join(
controlDirectory,
"server-restart.request.json",
),
restartAcknowledgementPath: path.join(
controlDirectory,
"server-restart.ack.json",
),
};
}
/**
* Native cells use the debug binary produced once by build:runner-binaries.
* Preserve an explicit override for release builds and developer workflows.
*/
export function resolvePaperclipRunnerBinaryForHarness(
executions: readonly MatrixExecution[],
repositoryRoot: string,
configuredPath = process.env.PAPERCLIP_RUNNER_BINARY,
platform: NodeJS.Platform = process.platform,
): string | undefined {
if (configuredPath?.trim()) return configuredPath;
if (
!executions.some((execution) => execution.profile.generation === "native")
) {
return undefined;
}
return path.join(
repositoryRoot,
"packages",
"paperclip-runner",
"runner",
"target",
"debug",
platform === "win32" ? "paperclip-runnerd.exe" : "paperclip-runnerd",
);
}
/**
* Remote native cells stage the same controller-owned binary whose digest is
* authorized by the PRP control plane. Local cells launch it directly.
*/
export function resolvePaperclipRemoteRunnerBinaryForHarness(
executions: readonly MatrixExecution[],
runnerBinary: string | undefined,
): string | undefined {
if (!runnerBinary) return undefined;
return executions.some(
(execution) =>
execution.profile.generation === "native" &&
execution.environment.expectedExecutionTarget.kind === "remote",
)
? runnerBinary
: undefined;
}
/**
* Keep fixture-only provider switches scoped to the one isolated harness that
* needs them. In particular, the pinned legacy OpenCode model is routed by the
* paid gateway and may not appear in OpenCode's public model catalog.
*/
export function buildRunnerE2EProcessEnvironment(
source: NodeJS.ProcessEnv,
executions: readonly MatrixExecution[],
): NodeJS.ProcessEnv {
const result = { ...source };
delete result.OPENCODE_ALLOW_ALL_MODELS;
if (
executions.length > 0 &&
executions.every(
(execution) =>
execution.profile.generation === "legacy" &&
execution.profile.provider === "opencode",
)
) {
result.OPENCODE_ALLOW_ALL_MODELS = "true";
}
return result;
}
/**
* Build the environment inherited by the Paperclip server. Paid credentials
* deliberately stay in the launcher/Playwright process and cross the server
* boundary only once, in the encrypted company-secrets API request.
*/
export function buildPaperclipServerEnvironment(
source: NodeJS.ProcessEnv,
overrides: NodeJS.ProcessEnv = {},
): NodeJS.ProcessEnv {
const result = { ...source };
for (const key of Object.keys(result)) {
if (PROVIDER_SECRET_KEY.test(key)) delete result[key];
}
for (const key of [
...CREDENTIAL_NAMES,
...DATABASE_KEYS,
...AMBIENT_PAPERCLIP_CREDENTIAL_KEYS,
...AMBIENT_EXTERNAL_STATE_KEYS,
]) {
delete result[key];
}
for (const key of GENERATED_SERVER_SECRET_KEYS) delete result[key];
Object.assign(result, overrides);
return result;
}
export function assertIsolatedServerEnvironment(
env: NodeJS.ProcessEnv,
expected: {
temporaryRoot: string;
paperclipHome: string;
configPath: string;
},
) {
const home = env.PAPERCLIP_HOME;
const config = env.PAPERCLIP_CONFIG;
if (home !== expected.paperclipHome || config !== expected.configPath) {
throw new Error(
"Paperclip server environment does not use the allocated home/config paths",
);
}
if (
!home.startsWith(`${expected.temporaryRoot}/`) ||
!config.startsWith(`${expected.temporaryRoot}/`)
) {
throw new Error(
"Paperclip server paths escape the isolated temporary root",
);
}
if (env.XDG_CACHE_HOME !== path.join(expected.temporaryRoot, "xdg-cache")) {
throw new Error(
"Paperclip server cache does not use the allocated temporary root",
);
}
for (const key of [
...CREDENTIAL_NAMES,
...DATABASE_KEYS,
...AMBIENT_PAPERCLIP_CREDENTIAL_KEYS,
...AMBIENT_EXTERNAL_STATE_KEYS,
]) {
if (env[key])
throw new Error(
`Paperclip server environment unexpectedly contains ${key}`,
);
}
for (const key of GENERATED_SERVER_SECRET_KEYS) {
if (!env[key])
throw new Error(`Paperclip server environment is missing ${key}`);
}
}