mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 21:49:18 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote OpenCode and ACPX runs need a provider pack from the application build. > - Cloud now builds its application image from the standard production image. > - The provider pack was added only to the legacy cloud image target. > - The standard image therefore cannot supply the pack to downstream Cloud images. > - This pull request adds the pack to the production image and lets the cloud target inherit it. > - Remote runs can then use the pack that matches the application source commit. ## Linked Issues or Issue Description Refs #13827. Refs #14024. The standard production image does not include the remote provider pack. Downstream Cloud images inherit that omission. Remote OpenCode and ACPX runs fail with `runner_remote_provider_artifact_incompatible` and ask for `PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH`. ## What Changed - Build and copy the provider pack into the standard production image. - Set the pack path and check that an unprivileged user can read its artifacts and execute Node. - Let the legacy cloud target inherit the pack from production. - Add regression checks for production packaging and cloud inheritance. - Document stamped image behavior and the default pack path. ## Verification - The 12 focused Docker stamp and provider-pack reuse tests pass on commit `4a11f8d52aead55f85527c8e82c6d7f2644ce0da`. - The new packaging regression failed against the old Dockerfile and passed with the fix. - On the current commit, `pnpm build` and `pnpm -r typecheck` pass. All seven standard-image contract tests also pass. - The current-head CI build, typecheck, test, browser, and native Runner checks passed. The local full suite hit one chat-channel assertion failure; that exact test passed in isolation. The remaining local run was stopped after CI completed to avoid duplicating its full suite. An earlier run on the pre-rebase base had a heartbeat comment batching timeout; the external chat wait integration suite passed all 142 tests in isolation. - [The stamped preview image build passed](https://github.com/paperclipai/paperclip/actions/runs/36885002850/job/110446106393), including the production-stage provider pack build, copy, and unprivileged artifact readability/executable check. Publication, compatibility validation, and deployment of this exact commit to a staging QA instance passed. - Reproduced the exact missing-pack error on an existing staging image with Paperclip Runner, ACPX, and Claude in a remote Daytona computer. The legacy Claude adapter succeeds with the same account and computer. After deploying this commit, the same native task succeeded: it computed `5050` with a real remote shell command, wrote a proof file, read it back in a separate call, uploaded the file as a deliverable, and completed the task. The uploaded file contents and Done state persisted after a page reload. The run trace confirms Paperclip Runner, ACPX, and Claude. The first run took 2m 59s, including approximately 97s of remote artifact preparation. A second native run read the unchanged file from the prior run and completed successfully. Its startup took about 120s; this verifies repeated execution and file persistence, not fast provider-pack reuse. ## Risks - Stamped standard images now include the provider pack and its build cost. A pack build failure now fails the production image build. - Unstamped local builds still skip pack generation. Setting the path alone does not create a pack. - No database, provider authentication, or runner verification rules change. ## Model Used OpenAI Codex, GPT-6. The exact serving model identifier and context window are not exposed in this session. Capabilities used: repository inspection, code editing, shell verification, and browser testing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
138 lines
7.5 KiB
TypeScript
138 lines
7.5 KiB
TypeScript
import { readFileSync } from "node:fs";
|
|
import path from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
import { describe, expect, it } from "vitest";
|
|
|
|
/**
|
|
* Drift guard for the Docker build-stamp wiring.
|
|
*
|
|
* The server build runs scripts/write-build-stamp.mjs, which stamps the built
|
|
* commit into dist/build-info.json. The build context has no .git, so the
|
|
* script reads PAPERCLIP_BUILD_COMMIT instead. Docker exposes an ARG to the
|
|
* next RUN as an environment variable, but an ARG goes out of scope at the end
|
|
* of its stage. So the build stage must declare `ARG PAPERCLIP_BUILD_COMMIT`
|
|
* before the server build; the production ARG alone stamps nothing, because
|
|
* the server build already ran in the earlier stage.
|
|
*
|
|
* This guard fails if a refactor drops the build-stage ARG, moves it after the
|
|
* server build, or removes the build-arg the docker workflow passes.
|
|
*/
|
|
|
|
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..", "..");
|
|
const dockerfile = readFileSync(path.join(repoRoot, "Dockerfile"), "utf8");
|
|
const workflow = readFileSync(path.join(repoRoot, ".github", "workflows", "docker.yml"), "utf8");
|
|
const previewWorkflow = readFileSync(path.join(repoRoot, ".github", "workflows", "release.yml"), "utf8");
|
|
|
|
/**
|
|
* Return the text of the Dockerfile stage that starts at the named target.
|
|
* A stage runs from its `FROM ... AS <name>` line to the next `FROM` line.
|
|
*/
|
|
function stageBody(source: string, stageName: string): string {
|
|
const froms = [...source.matchAll(/^FROM .*$/gm)];
|
|
const startIdx = froms.findIndex((m) => new RegExp(`\\bAS ${stageName}\\s*$`).test(m[0]));
|
|
expect(startIdx, `Dockerfile must declare a '${stageName}' stage`).toBeGreaterThanOrEqual(0);
|
|
const start = froms[startIdx].index ?? 0;
|
|
const end = froms[startIdx + 1]?.index ?? source.length;
|
|
return source.slice(start, end);
|
|
}
|
|
|
|
it("keeps per-build runtime metadata out of the weekly CLI-install cache", () => {
|
|
const production = stageBody(dockerfile, "production");
|
|
const tools = production.search(/^RUN echo "cli-tools-epoch:/m);
|
|
const entrypoint = production.search(/^RUN chmod \+x \/usr\/local\/bin\/docker-entrypoint\.sh/m);
|
|
const runtime = production.search(/^ENV NODE_ENV=production/m);
|
|
const epoch = production.search(/^ARG CLI_TOOLS_CACHE_EPOCH\b/m);
|
|
expect(tools).toBeGreaterThanOrEqual(0);
|
|
expect(entrypoint).toBeGreaterThan(tools);
|
|
expect(epoch).toBeGreaterThanOrEqual(0);
|
|
expect(epoch).toBeLessThan(tools);
|
|
for (const name of ["PAPERCLIP_BUILD_VERSION", "PAPERCLIP_BUILD_COMMIT"]) {
|
|
const declarations = [...production.matchAll(new RegExp(`^ARG ${name}\\b`, "gm"))];
|
|
expect(declarations).toHaveLength(1);
|
|
expect(declarations[0].index).toBeGreaterThan(entrypoint);
|
|
expect(declarations[0].index).toBeLessThan(runtime);
|
|
expect(production.slice(runtime)).toContain(`${name}=\${${name}}`);
|
|
}
|
|
});
|
|
|
|
describe("docker build-stamp wiring", () => {
|
|
it("declares PAPERCLIP_BUILD_COMMIT in the build stage before the server build", () => {
|
|
const build = stageBody(dockerfile, "build");
|
|
const argIdx = build.search(/^ARG PAPERCLIP_BUILD_COMMIT\b/m);
|
|
const serverBuildIdx = build.search(/^RUN pnpm --filter @paperclipai\/server build\b/m);
|
|
expect(argIdx, "build stage must declare ARG PAPERCLIP_BUILD_COMMIT").toBeGreaterThanOrEqual(0);
|
|
expect(serverBuildIdx, "build stage must run the server build").toBeGreaterThanOrEqual(0);
|
|
expect(
|
|
argIdx,
|
|
"ARG PAPERCLIP_BUILD_COMMIT must precede the server build so the stamp script reads it",
|
|
).toBeLessThan(serverBuildIdx);
|
|
});
|
|
|
|
it("passes PAPERCLIP_BUILD_COMMIT as a build-arg for standard and explicit preview builds", () => {
|
|
for (const [name, source] of [["standard", workflow], ["preview", previewWorkflow]]) {
|
|
expect(source, `${name} must pass the source commit into the image build`)
|
|
.toMatch(/^\s*PAPERCLIP_BUILD_COMMIT=\$\{\{ (?:github.sha|inputs.source_ref) \}\}$/m);
|
|
}
|
|
});
|
|
});
|
|
|
|
|
|
describe("Docker Rust dependency cache", () => {
|
|
it("caches the locked dependency recipe separately from source and per-build metadata", () => {
|
|
const chef = stageBody(dockerfile, "rust-chef");
|
|
const planner = stageBody(dockerfile, "runner-plan");
|
|
const dependencies = stageBody(dockerfile, "runner-deps");
|
|
expect(chef).toContain("FROM rust-toolchain AS rust-chef");
|
|
expect(chef).toMatch(/cargo install cargo-chef --version \d+\.\d+\.\d+ --locked/);
|
|
expect(planner).toContain("COPY packages/paperclip-runner/runner ./runner");
|
|
expect(planner).toContain("cargo chef prepare --recipe-path /tmp/runner-recipe.json");
|
|
expect(dependencies).toContain("FROM rust-chef AS runner-deps");
|
|
expect(dependencies).toContain("COPY --from=runner-plan /tmp/runner-recipe.json /tmp/runner-recipe.json");
|
|
expect(dependencies).toContain("cargo chef cook --release --locked --package paperclip-runner-core --bin paperclip-runnerd");
|
|
expect(dependencies).not.toMatch(/COPY .*\.\/runner|COPY .*\.\/protocol|COPY \. \.|PAPERCLIP_BUILD_COMMIT/);
|
|
});
|
|
|
|
it("rebuilds real workspace code and embedded protocol inputs after cooking dependencies", () => {
|
|
const native = stageBody(dockerfile, "runner-build");
|
|
expect(native).toContain("FROM runner-deps AS runner-build");
|
|
for (const source of ["runner", "protocol"]) {
|
|
expect(native.indexOf(`COPY packages/paperclip-runner/${source} ./${source}`))
|
|
.toBeLessThan(native.indexOf("cargo build --release"));
|
|
expect(native).toContain(`COPY packages/paperclip-runner/${source} ./${source}`);
|
|
}
|
|
expect(native).toContain("cargo build --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd");
|
|
expect(stageBody(dockerfile, "build")).toContain("FROM runner-build AS build");
|
|
});
|
|
});
|
|
|
|
|
|
describe("Standard image remote provider pack", () => {
|
|
it("ships the build-owned pack in the production base used by downstream images", () => {
|
|
const pack = stageBody(dockerfile, "runner-provider-pack");
|
|
const production = stageBody(dockerfile, "production");
|
|
expect(pack).toContain("FROM build AS runner-provider-pack");
|
|
expect(pack).toContain('PAPERCLIP_RUNNER_SOURCE_REVISION="${PAPERCLIP_BUILD_COMMIT}"');
|
|
expect(pack).toContain("build-provider-pack.mjs /provider-pack");
|
|
expect(pack).toContain('if [ -n "${PAPERCLIP_BUILD_COMMIT}" ]; then');
|
|
expect(pack).toContain("mkdir -p /provider-pack");
|
|
expect(pack).toContain("Skipping remote provider pack");
|
|
expect(production).toContain("--from=runner-provider-pack /provider-pack /opt/paperclip-runner/provider-pack");
|
|
expect(production).not.toContain("--chown=node:node --from=runner-provider-pack");
|
|
expect(production).toContain("chmod -R a+rX /opt/paperclip-runner/provider-pack");
|
|
expect(production).toContain("gosu 65534:65534 node");
|
|
expect(production).toContain("Object.values(manifest.payload.artifacts)");
|
|
expect(production).toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH=/opt/paperclip-runner/provider-pack");
|
|
expect(workflow).toMatch(/^\s*target: production$/m);
|
|
expect(dockerfile).not.toContain("provision-grok.mjs");
|
|
});
|
|
|
|
it("lets explicit cloud preview builds inherit the same pack and configuration", () => {
|
|
const cloud = stageBody(dockerfile, "cloud");
|
|
expect(cloud).toContain("FROM production AS cloud");
|
|
expect(cloud).not.toContain("/opt/paperclip-runner/provider-pack");
|
|
expect(cloud).not.toContain("PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH");
|
|
expect(dockerfile.match(/build-provider-pack\.mjs \/provider-pack/g)).toHaveLength(1);
|
|
expect(previewWorkflow).toMatch(/^\s*target: cloud$/m);
|
|
});
|
|
});
|