mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native runner must support project work, delegation, hiring, and service access. > - Browser tests exposed lost connection access, rejected helper events, and stalled recovery. > - Some eval failures also came from incorrect fixtures and decision controls. > - This pull request fixes those paths and adds eight everyday workflow stories. > - The tests retain observed failures and verify delivered files independently. > - The benefit is repeatable evidence for common user tasks and their remaining gaps. ## Linked Issues or Issue Description Related work: #13404 contains earlier workflow fixes. #13300 and #13470 changed the CI contracts used by the harness security tests. Merged companion: [paperclip-evals#22](https://github.com/paperclipai/paperclip-evals/pull/22). **What happened?** Native ACPX sessions did not receive the assigned connection gateway. Codex helper events could arrive before their spawn receipt and fail thread validation. A parent continuation could take a shared workspace before its child retried. A failed native continuation could leave the task status without a clear recovery blocker. The eval harness also confused tool approvals with new connection requests and could reject a valid delegated download. **Expected behavior** Keep assigned gateway access and its approval checks. Verify helper lineage before accepting helper progress. Let a waiting child proceed before automatic parent recovery. Preserve a failed task's recovery ownership. Grade the actual requested workflow and its delivered files. **Steps to reproduce** Run the everyday workflow suite with the native Codex and Claude profiles. Exercise service approval, connection refusal, delegated project work, and teammate reuse. The commands and case requirements are in `tests/runner-e2e/EVERYDAY-WORKFLOWS.md`. Use `pnpm test:runner-recovery` for controlled crash and replacement cases. ## What Changed - Pass the scoped connection gateway binding through the native ACPX host and sidecar. - Recognize Codex helper lineage from parent metadata and spawn receipts. Verify early helper events with `thread/read`. Keep helper events separate from root completion authority. - Guide agents to use persistent hiring, child tasks, dependency records, and a blocked handoff while waiting for a child. - Defer automatic parent recovery while a child has an active execution path in the same shared workspace. Allow parent recovery when the child needs review. - Record Blocked status and recovery evidence when a failed native continuation needs reconciliation, including existing active or escalated incidents. Preserve their owner and retry budget. - Add eight browser-driven workflow cases. Use real decision controls, explicit child feedback delivery, managed hiring credentials, and independent ZIP checks inside a bounded Docker sandbox. Verify sandbox availability before task creation. Record screenshot SHA-256 at capture. - Keep runner crash probes in controlled recovery tests. Preserve the original failure when cleanup also fails. - Display missing accounting and replay revisions as unavailable. Align harness security assertions with the approved CI changes. - Make the channel-rejection browser fixture bind its file after the send captures its payload. This prevents live refresh from removing the file before the simulated race. ## Verification - Full workspace `pnpm -r typecheck` passed after merging current master. - Runner E2E typecheck passed. Harness unit tests passed: 216/216. - Wake-queue database tests passed: 55/55. The two added existing-incident tests failed before the fix and pass after it. - Docker artifact calibration passed: 12/12. Host-file and host-loopback isolation tests failed before the fix and pass after it. Read-only delivery and output limits are also verified. - Full `pnpm build` passed. Targeted recovery tests passed: 83/83. - The channel-rejection browser test passed five consecutive runs after fixing the fixture race found in CI. - Local general-server (12,351 tests), UI (6,250), CLI (485), and workspace package groups passed. The monolithic run stopped at an unchanged lock-heartbeat fixture race; the isolated workspace group passed on rerun (shared: 747/747). A separate local serialized run passed 97 files before two socket errors in the unchanged issue-list route suite; that suite passed 15/15 on isolated rerun. These local full commands did not finish uninterrupted; the complete CI matrix below covers the remaining suites. - Final head `0fb293733fe307be7e6667ae8f1364077d0c6455`: **34 successful checks, 2 expected skips**, including every server/workspace shard, browser shard, native runner verification, build, and typecheck. [Final CI run](https://github.com/paperclipai/paperclip/actions/runs/34989136700). - Greptile reviewed this exact head at **5/5**; all review threads are resolved. Both Superagent security checks are successful. - ACPX credential-boundary tests passed: 118/118. Superagent accepted the runner/sidecar versus provider-environment trace and cleared its finding. - The latest paid local campaign on source `f6a2fdf7ac2af859826a2ae627ff4125a5478529` passed 22/24 cases: Sol 8/8, Claude 7/8, Mini 7/8. These results predate the merge with current master. - The two remaining failures are in `hire-reuse`: Claude exceeded the attempt deadline during final review; Mini made invalid deliverable tool calls and remained Blocked. - Six Daytona cases were not run because the matching immutable runner image was unavailable. This PR does not claim new remote model results. ## Risks The changes affect connection admission, helper identity, and recovery scheduling. Assigned gateway grants and user approval still govern service calls. The workspace admission gate still exists; the broader folder-sync design is separate work. Provider behavior can still cause the two recorded hiring failures. No database migration is required. Paid cases are opt-in and have bounded attempt deadlines. Project stories now require Docker and the documented pinned Python image on the harness host. ## Model Used OpenAI `gpt-6-astra` performed implementation, diagnosis, and substantive review. OpenAI `gpt-5.6-luna` assisted with verification, PR preparation, and review tracking. Both used repository tools and code execution. Context-window sizes were not recorded. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks and isolated reruns; full-run limitations are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: OpenAI GPT-5.6 Luna <noreply@openai.com> Co-authored-by: Paperclip <noreply@paperclip.ing>
196 lines
6.1 KiB
TypeScript
196 lines
6.1 KiB
TypeScript
import path from "node:path";
|
|
import { CREDENTIAL_NAMES } from "./types.js";
|
|
import type { MatrixExecution } from "./types.js";
|
|
|
|
const DATABASE_KEYS = ["DATABASE_URL", "DATABASE_MIGRATION_URL"] as const;
|
|
const AMBIENT_PAPERCLIP_CREDENTIAL_KEYS = [
|
|
"PAPERCLIP_API_KEY",
|
|
"PAPERCLIP_AGENT_API_KEY",
|
|
"PAPERCLIP_TASK_BRIDGE_TOKEN",
|
|
"PAPERCLIP_SETUP_TOKEN",
|
|
"PAPERCLIP_SECRETS_MASTER_KEY",
|
|
"PAPERCLIP_SECRETS_MASTER_KEY_FILE",
|
|
] as const;
|
|
const GENERATED_SERVER_SECRET_KEYS = [
|
|
"PAPERCLIP_AGENT_JWT_SECRET",
|
|
"PAPERCLIP_DECISION_SIGNING_SECRET",
|
|
"PAPERCLIP_TOOL_ACTION_SIGNING_SECRET",
|
|
"BETTER_AUTH_SECRET",
|
|
] as const;
|
|
const AMBIENT_EXTERNAL_STATE_KEYS = [
|
|
"PAPERCLIP_STORAGE_S3_BUCKET",
|
|
"PAPERCLIP_STORAGE_S3_REGION",
|
|
"PAPERCLIP_STORAGE_S3_ENDPOINT",
|
|
"PAPERCLIP_STORAGE_S3_PREFIX",
|
|
"PAPERCLIP_STORAGE_S3_FORCE_PATH_STYLE",
|
|
] as const;
|
|
const PROVIDER_SECRET_KEY = /^(?:OPENAI|ANTHROPIC|OPENROUTER|DAYTONA)(?:_|$)/;
|
|
|
|
export function runnerE2EServerControlPaths(temporaryRoot: string) {
|
|
const controlDirectory = path.join(temporaryRoot, "control");
|
|
return {
|
|
controlDirectory,
|
|
restartRequestPath: path.join(
|
|
controlDirectory,
|
|
"server-restart.request.json",
|
|
),
|
|
restartAcknowledgementPath: path.join(
|
|
controlDirectory,
|
|
"server-restart.ack.json",
|
|
),
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Native cells use the debug binary produced once by build:runner-binaries.
|
|
* Preserve an explicit override for release builds and developer workflows.
|
|
*/
|
|
export function resolvePaperclipRunnerBinaryForHarness(
|
|
executions: readonly MatrixExecution[],
|
|
repositoryRoot: string,
|
|
configuredPath = process.env.PAPERCLIP_RUNNER_BINARY,
|
|
platform: NodeJS.Platform = process.platform,
|
|
): string | undefined {
|
|
if (configuredPath?.trim()) return configuredPath;
|
|
if (
|
|
!executions.some((execution) => execution.profile.generation === "native")
|
|
) {
|
|
return undefined;
|
|
}
|
|
|
|
return path.join(
|
|
repositoryRoot,
|
|
"packages",
|
|
"paperclip-runner",
|
|
"runner",
|
|
"target",
|
|
"debug",
|
|
platform === "win32" ? "paperclip-runnerd.exe" : "paperclip-runnerd",
|
|
);
|
|
}
|
|
|
|
/**
|
|
* Remote native cells stage the same controller-owned binary whose digest is
|
|
* authorized by the PRP control plane. Local cells launch it directly.
|
|
*/
|
|
export function resolvePaperclipRemoteRunnerBinaryForHarness(
|
|
executions: readonly MatrixExecution[],
|
|
runnerBinary: string | undefined,
|
|
configuredPath = process.env.PAPERCLIP_RUNNER_REMOTE_BINARY_PATH,
|
|
platform: NodeJS.Platform = process.platform,
|
|
): string | undefined {
|
|
if (configuredPath?.trim()) return configuredPath;
|
|
if (!runnerBinary) return undefined;
|
|
// Daytona runs Linux. A default debug binary built by a macOS developer is
|
|
// Mach-O and cannot be staged into that sandbox. Leave the remote override
|
|
// unset so the pinned Daytona image's verified runnerd is discovered instead.
|
|
if (platform !== "linux") return undefined;
|
|
return executions.some(
|
|
(execution) =>
|
|
execution.profile.generation === "native" &&
|
|
execution.environment.expectedExecutionTarget.kind === "remote",
|
|
)
|
|
? runnerBinary
|
|
: undefined;
|
|
}
|
|
|
|
/**
|
|
* Keep fixture-only provider switches scoped to the one isolated harness that
|
|
* needs them. In particular, the pinned legacy OpenCode model is routed by the
|
|
* paid gateway and may not appear in OpenCode's public model catalog.
|
|
*/
|
|
export function buildRunnerE2EProcessEnvironment(
|
|
source: NodeJS.ProcessEnv,
|
|
executions: readonly MatrixExecution[],
|
|
): NodeJS.ProcessEnv {
|
|
const result = { ...source };
|
|
delete result.OPENCODE_ALLOW_ALL_MODELS;
|
|
// Hiring needs the opt-in native API surface. Scope this to the explicit
|
|
// manual hiring story; production and other suites retain their defaults.
|
|
if (executions.some((e) => e.suite.id === "everyday-workflows" && e.task.id === "hire-reuse")) {
|
|
result.PAPERCLIP_RUNNER_API_TOOLS_ENABLED = "true";
|
|
}
|
|
if (
|
|
executions.length > 0 &&
|
|
executions.every(
|
|
(execution) =>
|
|
execution.profile.generation === "legacy" &&
|
|
execution.profile.provider === "opencode",
|
|
)
|
|
) {
|
|
result.OPENCODE_ALLOW_ALL_MODELS = "true";
|
|
}
|
|
return result;
|
|
}
|
|
|
|
/**
|
|
* Build the environment inherited by the Paperclip server. Paid credentials
|
|
* deliberately stay in the launcher/Playwright process and cross the server
|
|
* boundary only once, in the encrypted company-secrets API request.
|
|
*/
|
|
export function buildPaperclipServerEnvironment(
|
|
source: NodeJS.ProcessEnv,
|
|
overrides: NodeJS.ProcessEnv = {},
|
|
): NodeJS.ProcessEnv {
|
|
const result = { ...source };
|
|
for (const key of Object.keys(result)) {
|
|
if (PROVIDER_SECRET_KEY.test(key)) delete result[key];
|
|
}
|
|
for (const key of [
|
|
...CREDENTIAL_NAMES,
|
|
...DATABASE_KEYS,
|
|
...AMBIENT_PAPERCLIP_CREDENTIAL_KEYS,
|
|
...AMBIENT_EXTERNAL_STATE_KEYS,
|
|
]) {
|
|
delete result[key];
|
|
}
|
|
for (const key of GENERATED_SERVER_SECRET_KEYS) delete result[key];
|
|
Object.assign(result, overrides);
|
|
return result;
|
|
}
|
|
|
|
export function assertIsolatedServerEnvironment(
|
|
env: NodeJS.ProcessEnv,
|
|
expected: {
|
|
temporaryRoot: string;
|
|
paperclipHome: string;
|
|
configPath: string;
|
|
},
|
|
) {
|
|
const home = env.PAPERCLIP_HOME;
|
|
const config = env.PAPERCLIP_CONFIG;
|
|
if (home !== expected.paperclipHome || config !== expected.configPath) {
|
|
throw new Error(
|
|
"Paperclip server environment does not use the allocated home/config paths",
|
|
);
|
|
}
|
|
if (
|
|
!home.startsWith(`${expected.temporaryRoot}/`) ||
|
|
!config.startsWith(`${expected.temporaryRoot}/`)
|
|
) {
|
|
throw new Error(
|
|
"Paperclip server paths escape the isolated temporary root",
|
|
);
|
|
}
|
|
if (env.XDG_CACHE_HOME !== path.join(expected.temporaryRoot, "xdg-cache")) {
|
|
throw new Error(
|
|
"Paperclip server cache does not use the allocated temporary root",
|
|
);
|
|
}
|
|
for (const key of [
|
|
...CREDENTIAL_NAMES,
|
|
...DATABASE_KEYS,
|
|
...AMBIENT_PAPERCLIP_CREDENTIAL_KEYS,
|
|
...AMBIENT_EXTERNAL_STATE_KEYS,
|
|
]) {
|
|
if (env[key])
|
|
throw new Error(
|
|
`Paperclip server environment unexpectedly contains ${key}`,
|
|
);
|
|
}
|
|
for (const key of GENERATED_SERVER_SECRET_KEYS) {
|
|
if (!env[key])
|
|
throw new Error(`Paperclip server environment is missing ${key}`);
|
|
}
|
|
}
|