mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runner qualification uses a Daytona runner image and a nightly Runner Full-Stack E2E workflow > - The nightly workflow fails at "Validate catalog and select cells", and a local image build fails at `sha256sum -c` > - Operators cannot build a current Daytona image for restart tests until both failures are understood > - This pull request removes a stale Dockerfile lock digest default, fixes the local recipes, and makes the nightly failure name its real cause > - The benefit is that a local build fails clearly when the digest is missing, and an expired overhead allowance is obvious from the log ## Linked Issues or Issue Description **What happened?** 1. The nightly `Runner Full-Stack E2E` runs 41997383733 and 42190298486 fail in "Validate catalog and select cells". The log shows `digest-mismatch: error`, but that line is only the input list of `actions/download-artifact`. The real error is `Operating overhead allowance needs review before paid admission`. The repository variable `RUNNER_E2E_OPERATING_OVERHEAD_JSON` has `expiresAt: 2026-11-03T19:49:53Z`. `scheduledOperatingOverhead` rejects an allowance that expires within 4 hours. Manual dispatch (run 42046147744) does not use the scheduled overhead path, so it passed this job. 2. `docker/daytona-runner/Dockerfile` set `ARG PAPERCLIP_RUNNER_LOCK_SHA256=342a01e4…`. That value went stale when `pnpm-lock.yaml` changed (#19443). The local recipes in `tests/runner-e2e/README.md` and `docker/daytona-runner/README.md` did not pass the argument, so local builds failed with a confusing checksum mismatch. **Expected behavior** A local build computes and passes the lockfile digest. A missing digest fails by name. An expired overhead allowance tells the operator which allowance expired and which variable to renew. **Steps to reproduce** 1. Run `gh run view 42190298486 --log` and read the "Validate catalog and select cells" job. 2. Follow the old "## Daytona image" recipe in `tests/runner-e2e/README.md` on current master. ## What Changed - `docker/daytona-runner/Dockerfile`: remove the fixed `PAPERCLIP_RUNNER_LOCK_SHA256` default. Reject a missing or non-SHA-256 value by name before `sha256sum -c` and before installation. CI already passes the real value. - `tests/runner-e2e/README.md` and `docker/daytona-runner/README.md`: compute `shasum -a 256 pnpm-lock.yaml` and pass `PAPERCLIP_RUNNER_LOCK_SHA256` in the local build recipes. Replace the text about a "fixed Dockerfile default". - `tests/runner-e2e/daytona-image.test.ts`: new test. It requires a default-free `ARG`, the shape check before the checksum, and the argument in every documented local recipe. - `tests/runner-e2e/operating-overhead.ts`: the scheduled admission error now names the allowance reference, its validity window, the 4-hour margin, and `RUNNER_E2E_OPERATING_OVERHEAD_JSON`. The check itself is unchanged. ## Verification - `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/operating-overhead.test.ts tests/runner-e2e/daytona-image.test.ts tests/runner-e2e/scheduled-operations.test.ts tests/runner-e2e/cost-ledger.test.ts`: 55 passed. - `pnpm test:e2e:runner:typecheck`: passed. - `pnpm verify`: exit 2 (README is a global CI input). Required CI supplies broad coverage. - `tests/runner-e2e/workflow-security.test.ts` has 5 local failures on files that this PR does not touch (`pr-trusted.yml`, `chat-catalog-testops-case.yml`). They are not caused by this change. - I did not run a Docker build or a paid campaign. ## Risks - Low risk. Any external script that builds this Dockerfile without `PAPERCLIP_RUNNER_LOCK_SHA256` now fails with a clear message. Before, it failed with a checksum mismatch whenever the lockfile changed. Both CI build paths already pass the argument. - The Dockerfile is an image content-ID input, so the next image gets a new content tag. Current master needs a new image anyway. - The nightly workflow stays red until an operator reviews and renews `RUNNER_E2E_OPERATING_OVERHEAD_JSON`. This PR does not renew the allowance, because that is a reviewed spend decision. - Scheduled runs set `needs_daytona=false`, so the nightly run does not publish a Daytona image even when the catalog passes. A current image comes from a manual dispatch that selects Daytona cells. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
670 lines
30 KiB
TypeScript
670 lines
30 KiB
TypeScript
import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
|
import { tmpdir } from "node:os";
|
|
import path from "node:path";
|
|
import { describe, expect, it } from "vitest";
|
|
import {
|
|
computeDaytonaImageContentId,
|
|
DAYTONA_IMAGE_DOCKERFILE_PATH,
|
|
DAYTONA_IMAGE_INPUT_PATHS,
|
|
extractDaytonaBaseImages,
|
|
extractDaytonaDockerfileFrontendDigest,
|
|
} from "./daytona-image-content.js";
|
|
|
|
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
|
|
function lastMatchingDockerignoreRule(dockerignore: string, fixturePath: string) {
|
|
const segments = fixturePath.split("/");
|
|
const ancestors = segments.map((_, index) => segments.slice(0, index + 1).join("/"));
|
|
return dockerignore.split(/\r?\n/).filter(rule => rule && !rule.startsWith("#")).filter(rule => {
|
|
const pattern = rule.replace(/^!/, "").replace(/^\//, "");
|
|
return ancestors.some(candidate => path.matchesGlob(candidate, pattern));
|
|
}).at(-1);
|
|
}
|
|
|
|
describe("runner E2E Daytona image contract", () => {
|
|
// ACPX generation recomputes these release attestation inputs in the image build.
|
|
const attestationInputs = [
|
|
"acpx-profiles.json",
|
|
"scripts/acpx-release-attestations.mjs",
|
|
"scripts/generate-acpx-profiles.mjs",
|
|
"src/drivers/acpx/copilot-closure-pins.ts",
|
|
"src/drivers/acpx/pi-closure-pins.ts",
|
|
"src/drivers/acpx/pi-node-pins.ts",
|
|
"src/drivers/acpx/kimi-closure-pins.ts",
|
|
"src/providers/grok/launcher.cjs",
|
|
].map(input => `packages/paperclip-runner/${input}`);
|
|
|
|
it("retains the ACPX release attestation inputs in the Docker context", async () => {
|
|
const dockerignore = await readFile(path.join(repositoryRoot, ".dockerignore"), "utf8");
|
|
for (const input of attestationInputs) {
|
|
await expect(readFile(path.join(repositoryRoot, input), "utf8")).resolves.toBeTruthy();
|
|
// Docker's last matching rule wins, including the parent-directory exclusion.
|
|
expect(lastMatchingDockerignoreRule(dockerignore, input)?.startsWith("!") ?? true, input).toBe(true);
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS.some(inputPath => input === inputPath || input.startsWith(`${inputPath}/`)), input).toBe(true);
|
|
}
|
|
});
|
|
|
|
it("invalidates the image content ID when a release attestation input changes", async () => {
|
|
const manifestPath = "packages/paperclip-runner/acpx-profiles.json";
|
|
const root = await mkdtemp(path.join(tmpdir(), "paperclip-daytona-attestation-"));
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths: DAYTONA_IMAGE_INPUT_PATHS.filter(inputPath => inputPath === manifestPath),
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
} as const;
|
|
try {
|
|
expect(options.inputPaths).toEqual([manifestPath]);
|
|
await mkdir(path.dirname(path.join(root, manifestPath)), { recursive: true });
|
|
await writeFile(path.join(root, manifestPath), '{"commandDigest":"before"}\n');
|
|
const before = await computeDaytonaImageContentId(options);
|
|
expect(await computeDaytonaImageContentId(options)).toBe(before);
|
|
const withoutManifest = { ...options, inputPaths: [] };
|
|
const unrelatedBefore = await computeDaytonaImageContentId(withoutManifest);
|
|
await writeFile(path.join(root, manifestPath), '{"commandDigest":"after"}\n');
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(before);
|
|
expect(await computeDaytonaImageContentId(withoutManifest)).toBe(unrelatedBefore);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("binds each Antigravity input consumed by the merged provider entrypoint builder", async () => {
|
|
const required = [
|
|
"packages/paperclip-runner/antigravity-distributions.json",
|
|
"packages/paperclip-runner/scripts/provision-antigravity.mjs",
|
|
"packages/paperclip-runner/scripts/antigravity-provisioner-layout.mjs",
|
|
"packages/paperclip-runner/scripts/materialize-antigravity-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/patch-antigravity-runtime.mjs",
|
|
"packages/paperclip-runner/scripts/patch-antigravity-runtime.admission.py",
|
|
];
|
|
const root = await mkdtemp(path.join(tmpdir(), "paperclip-daytona-builder-inputs-"));
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths: DAYTONA_IMAGE_INPUT_PATHS.filter(input => required.includes(input)),
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
} as const;
|
|
try {
|
|
for (const input of required) {
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).toContain(input);
|
|
await mkdir(path.dirname(path.join(root, input)), { recursive: true });
|
|
await writeFile(path.join(root, input), await readFile(path.join(repositoryRoot, input)));
|
|
}
|
|
const baseline = await computeDaytonaImageContentId(options);
|
|
for (const input of required) {
|
|
const original = await readFile(path.join(root, input));
|
|
await writeFile(path.join(root, input), Buffer.concat([original, Buffer.from("\nchanged-build-input\n")]));
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
await rm(path.join(root, input));
|
|
await expect(computeDaytonaImageContentId(options)).rejects.toThrow();
|
|
await writeFile(path.join(root, input), original);
|
|
expect(await computeDaytonaImageContentId(options)).toBe(baseline);
|
|
}
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("keeps the qualified native Grok binary separate from the legacy command", async () => {
|
|
const [dockerfile, packBuilder, runnerPackage] = await Promise.all([
|
|
readFile(path.join(repositoryRoot, "docker/daytona-runner/Dockerfile"), "utf8"),
|
|
readFile(path.join(repositoryRoot, "packages/paperclip-runner/scripts/build-provider-pack.mjs"), "utf8"),
|
|
readFile(path.join(repositoryRoot, "packages/paperclip-runner/package.json"), "utf8"),
|
|
]);
|
|
expect(dockerfile).toMatch(/@xai-official\/grok@\d+\.\d+\.\d+/);
|
|
expect(dockerfile).not.toMatch(/for cli in[^;]*\bgrok\b/);
|
|
expect(packBuilder).not.toMatch(/writePortable\w+Shim\("grok"/);
|
|
expect(JSON.parse(runnerPackage).dependencies).not.toHaveProperty("@paperclipai/grok-acp");
|
|
expect(packBuilder).toContain('path: "dist/providers/grok/launcher.cjs"');
|
|
expect(dockerfile).toContain("scripts/provision-grok.mjs");
|
|
expect(dockerfile).toContain("/opt/paperclip/providers/grok/1.0.13/grok");
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).toContain("packages/paperclip-runner/scripts/provision-grok.mjs");
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).not.toContain("packages/grok-acp/package.json");
|
|
});
|
|
|
|
it("requires every build, including documented local recipes, to pass the lockfile digest", async () => {
|
|
const [dockerfile, e2eReadme, imageReadme] = await Promise.all([
|
|
readFile(path.join(repositoryRoot, "docker/daytona-runner/Dockerfile"), "utf8"),
|
|
readFile(path.join(repositoryRoot, "tests/runner-e2e/README.md"), "utf8"),
|
|
readFile(path.join(repositoryRoot, "docker/daytona-runner/README.md"), "utf8"),
|
|
]);
|
|
// A fixed default goes stale with every lockfile change and then fails as a
|
|
// misleading checksum mismatch; a missing argument must fail by name.
|
|
expect(dockerfile).toMatch(/^ARG PAPERCLIP_RUNNER_LOCK_SHA256$/m);
|
|
const normalizedDockerfile = dockerfile.replace(/\\\r?\n\s*/g, " ");
|
|
const shapeCheck = normalizedDockerfile.indexOf(`grep -Eqx '[0-9a-f]{64}'`);
|
|
expect(shapeCheck).toBeGreaterThan(-1);
|
|
expect(shapeCheck).toBeLessThan(normalizedDockerfile.indexOf("sha256sum -c /tmp/provider-lock.sha256"));
|
|
expect(normalizedDockerfile.slice(shapeCheck)).toMatch(
|
|
/^grep -Eqx '\[0-9a-f\]\{64\}';\s+then\s+echo 'PAPERCLIP_RUNNER_LOCK_SHA256 build arg must be the SHA-256 of pnpm-lock\.yaml[^']*' >&2;\s+exit 1;\s+fi/,
|
|
);
|
|
for (const readme of [e2eReadme, imageReadme]) {
|
|
const recipes = [...readme.matchAll(/```(?:bash|sh)\n([\s\S]*?)```/g)]
|
|
.map(match => match[1]!)
|
|
.filter(block => block.includes("--file docker/daytona-runner/Dockerfile"));
|
|
expect(recipes.length).toBeGreaterThan(0);
|
|
for (const recipe of recipes) {
|
|
// The computed digest itself must be the value passed to the build.
|
|
const passed = recipe.match(
|
|
/--build-arg "?PAPERCLIP_RUNNER_LOCK_SHA256="?(\$\{lock_sha\}|\$\(shasum -a 256 pnpm-lock\.yaml \| cut -d ' ' -f 1\))"/,
|
|
);
|
|
expect(passed, recipe).not.toBeNull();
|
|
if (passed![1] === "${lock_sha}") {
|
|
expect(recipe).toContain(`lock_sha="$(shasum -a 256 pnpm-lock.yaml | cut -d ' ' -f 1)"`);
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
it("builds runnerd and the provider pack and verifies every required transport", async () => {
|
|
const [dockerfile, dockerignore, workflow] = await Promise.all([
|
|
readFile(
|
|
path.join(repositoryRoot, "docker/daytona-runner/Dockerfile"),
|
|
"utf8",
|
|
),
|
|
readFile(path.join(repositoryRoot, ".dockerignore"), "utf8"),
|
|
readFile(
|
|
path.join(
|
|
repositoryRoot,
|
|
".github/workflows/runner-full-stack-e2e.yml",
|
|
),
|
|
"utf8",
|
|
),
|
|
]);
|
|
const normalizedDockerfile = dockerfile.replace(/\\\r?\n\s*/g, " ");
|
|
const daytonaImageJob = workflow.match(/^ daytona_image:\n[\s\S]*?(?=^ \w+:)/m)?.[0];
|
|
expect(daytonaImageJob).toBeDefined();
|
|
expect(dockerfile).toContain("--bin paperclip-runnerd");
|
|
expect(dockerfile).toContain("build-provider-pack.mjs /provider-pack");
|
|
expect(normalizedDockerfile).not.toContain(
|
|
"COPY packages/paperclip-eval-kernel ./packages/paperclip-eval-kernel",
|
|
);
|
|
expect(normalizedDockerfile).not.toContain(
|
|
"COPY packages/paperclip-runner ./packages/paperclip-runner",
|
|
);
|
|
expect(dockerfile).toContain(
|
|
"COPY packages ./packages",
|
|
);
|
|
expect(dockerfile).toContain(
|
|
"/opt/paperclip-runner/provider-pack/provider-pack.json",
|
|
);
|
|
expect(dockerfile).toContain(
|
|
"${PAPERCLIP_RUNNER_PROVIDER_PACK_ROOT}/node_modules/.bin",
|
|
);
|
|
for (const command of ["acpx", "claude-agent-acp", "codex-acp"]) {
|
|
expect(dockerfile).toContain(command);
|
|
}
|
|
for (const transport of ["dial_ws_loopback", "dial_wss", "listen_ws"]) {
|
|
expect(dockerfile).toContain(transport);
|
|
}
|
|
expect(dockerfile).toContain(
|
|
'metadata="$(paperclip-runnerd --build-metadata)"',
|
|
);
|
|
expect(dockerfile).toContain("provider-pack.json");
|
|
expect(dockerfile).toContain("io.paperclip.runner.content-id");
|
|
expect(dockerfile).toContain("org.opencontainers.image.revision");
|
|
expect(extractDaytonaDockerfileFrontendDigest(dockerfile)).toBe(
|
|
"sha256:a57df69d0ea827fb7266491f2813635de6f17269be881f696fbfdf2d83dda33e",
|
|
);
|
|
expect(extractDaytonaBaseImages(dockerfile)).toEqual([
|
|
"rust:1.97-bookworm@sha256:408fe88047cef61a2087653b0c5255fa51c0f2d6d94ddedd7a2562a9b91a46f6",
|
|
"node:24.21.0-bookworm@sha256:5a750d3be5e5c80275f8c9a5367c3aed99c2875656590c8d0701c7ee687f5f0a",
|
|
"daytonaio/sandbox:0.8.0@sha256:eadf88e4391072b7ad4bed27d9cadfc9fe9d8ed375d9219d34c2ccb518f213e3",
|
|
]);
|
|
expect(dockerignore).toContain("**/node_modules");
|
|
expect(dockerignore).toContain("packages/paperclip-runner/dist");
|
|
expect(dockerignore).toContain("packages/paperclip-runner/runner/target");
|
|
expect(await readFile(path.join(repositoryRoot, ".github/docker-context-checks.Dockerfile"), "utf8")).toContain("node packages/paperclip-runner/scripts/generate-acpx-profiles.mjs --check");
|
|
expect(lastMatchingDockerignoreRule(dockerignore, "packages/paperclip-runner/test-fixtures/pi-acp/private-live-session.json")?.startsWith("!")).toBe(false);
|
|
for (const developmentOnlyInput of [
|
|
"packages/paperclip-runner/devtools",
|
|
"packages/paperclip-runner/docs",
|
|
"packages/paperclip-runner/examples",
|
|
"packages/paperclip-runner/test",
|
|
"packages/paperclip-runner/test-fixtures",
|
|
"packages/paperclip-runner/test-support",
|
|
"packages/paperclip-runner/**/*.md",
|
|
"packages/paperclip-runner/**/*.test.ts",
|
|
"packages/paperclip-runner/runner/crates/*/tests",
|
|
"packages/paperclip-runner/scripts/*-smoke.mjs",
|
|
]) {
|
|
expect(dockerignore).toContain(developmentOnlyInput);
|
|
}
|
|
for (const developmentOnlyFile of [
|
|
"devtools/probe.mjs",
|
|
"docs/development.md",
|
|
"examples/demo.ts",
|
|
"test/fixtures/copilot-live-denial.json",
|
|
"test-fixtures/pi-acp/live-evidence.json",
|
|
"test-support/helpers.ts",
|
|
"runner/crates/runner/tests/contract.rs",
|
|
"scripts/pi-smoke.mjs",
|
|
]) {
|
|
expect(lastMatchingDockerignoreRule(dockerignore, `packages/paperclip-runner/${developmentOnlyFile}`)?.startsWith("!")).toBe(false);
|
|
}
|
|
expect(workflow).toContain("--platform linux/amd64");
|
|
expect(workflow).toContain(
|
|
"Compute Daytona image content ID with pinned bases",
|
|
);
|
|
expect(workflow).toContain(
|
|
"e2e-content-${{ needs.catalog.outputs.daytona_image_content_id }}",
|
|
);
|
|
expect(workflow).toContain(
|
|
'--build-arg "PAPERCLIP_RUNNER_CONTENT_ID=${IMAGE_CONTENT_ID}"',
|
|
);
|
|
expect(daytonaImageJob).toContain(
|
|
"TARGET_LOCK_SHA256: ${{ needs.target_lock.outputs.lock_sha256 }}",
|
|
);
|
|
expect(daytonaImageJob).toContain(
|
|
'[[ "$TARGET_LOCK_SHA256" =~ ^[0-9a-f]{64}$ ]]',
|
|
);
|
|
expect(daytonaImageJob).toContain(
|
|
'--build-arg "PAPERCLIP_RUNNER_LOCK_SHA256=${TARGET_LOCK_SHA256}"',
|
|
);
|
|
expect(
|
|
daytonaImageJob!.indexOf('[[ "$TARGET_LOCK_SHA256" =~ ^[0-9a-f]{64}$ ]]'),
|
|
).toBeLessThan(
|
|
daytonaImageJob!.indexOf(
|
|
'--build-arg "PAPERCLIP_RUNNER_LOCK_SHA256=${TARGET_LOCK_SHA256}"',
|
|
),
|
|
);
|
|
expect(workflow).toContain(
|
|
"IMAGE_CACHE: ghcr.io/paperclipai/paperclip-daytona-runner:e2e-buildcache-amd64",
|
|
);
|
|
expect(workflow).toContain(
|
|
'--cache-from "type=registry,ref=${IMAGE_CACHE}"',
|
|
);
|
|
expect(workflow).toContain(
|
|
'--cache-to "type=registry,ref=${IMAGE_CACHE},mode=max"',
|
|
);
|
|
expect(workflow).toContain(
|
|
'if [ "$TARGET_REF" = "refs/heads/$DEFAULT_BRANCH" ]; then',
|
|
);
|
|
expect(workflow).not.toContain("e2e-git-${{ github.sha }}");
|
|
expect(workflow).toContain("cosign sign --yes");
|
|
expect(workflow).toContain("docker logout ghcr.io");
|
|
expect(workflow).toContain(`docker buildx imagetools inspect "$immutable"`);
|
|
expect(workflow).toContain(`--format '{{json .Image}}'`);
|
|
expect(workflow).not.toContain(`docker --config "$anonymous_config" pull`);
|
|
// Remote Daytona manifests must use registry inspection. Local oracle
|
|
// images in the test job can still use the Docker daemon.
|
|
expect(daytonaImageJob).not.toContain("docker image inspect");
|
|
expect(workflow).not.toContain("docker buildx prune --all --force");
|
|
expect(workflow).not.toContain("docker system prune --all --force");
|
|
expect(workflow).toContain('.architecture == "amd64"');
|
|
expect(workflow).toContain('.os == "linux"');
|
|
expect(workflow).toContain('.config.User == "daytona"');
|
|
expect(workflow).toContain("PAPERCLIP_RUNNER_PROVIDER_PACK_ROOT=");
|
|
expect(workflow).toContain(
|
|
"node packages/paperclip-runner/scripts/build-provider-pack.mjs packages/paperclip-runner/provider-pack",
|
|
);
|
|
expect(workflow).toContain(
|
|
"PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH: ${{ github.workspace }}/packages/paperclip-runner/provider-pack",
|
|
);
|
|
expect(workflow).toContain(
|
|
"PAPERCLIP_RUNNER_SOURCE_REVISION: ${{ needs.daytona_image.outputs.source_revision }}",
|
|
);
|
|
expect(workflow.indexOf("cosign verify")).toBeLessThan(
|
|
workflow.indexOf("docker logout ghcr.io"),
|
|
);
|
|
expect(workflow.indexOf("docker logout ghcr.io")).toBeLessThan(
|
|
workflow.indexOf(`--format '{{json .Image}}'`),
|
|
);
|
|
const providerInstall = dockerfile.indexOf(
|
|
"pnpm install --frozen-lockfile --filter '@paperclipai/paperclip-runner...'",
|
|
);
|
|
const runnerSourceCopy = dockerfile.indexOf(
|
|
"COPY packages ./packages",
|
|
);
|
|
const providerRevisionArg = dockerfile.indexOf(
|
|
"ARG PAPERCLIP_RUNNER_SOURCE_REVISION",
|
|
);
|
|
const cliInstall = dockerfile.indexOf("npm install -g");
|
|
const finalMetadataArgs = dockerfile.lastIndexOf(
|
|
"ARG PAPERCLIP_RUNNER_CONTENT_ID",
|
|
);
|
|
expect(providerInstall).toBeGreaterThan(0);
|
|
expect(runnerSourceCopy).toBeGreaterThan(0);
|
|
expect(runnerSourceCopy).toBeLessThan(providerInstall);
|
|
// The source-bound install and portable-pack build now share a layer so
|
|
// their disposable dependency tree can be removed before image export.
|
|
expect(providerRevisionArg).toBeGreaterThan(0);
|
|
expect(providerRevisionArg).toBeLessThan(providerInstall);
|
|
expect(cliInstall).toBeGreaterThan(0);
|
|
expect(cliInstall).toBeLessThan(finalMetadataArgs);
|
|
});
|
|
|
|
it("hashes the audited image dependency closure rather than the repository revision", async () => {
|
|
for (const requiredPath of [
|
|
".dockerignore",
|
|
"docker/daytona-runner/Dockerfile",
|
|
"pnpm-lock.yaml",
|
|
"patches",
|
|
"packages/paperclip-eval-kernel/src",
|
|
"packages/paperclip-runner/package.json",
|
|
"packages/paperclip-runner/scripts/candidate-provider-pack.mjs",
|
|
"packages/paperclip-runner/scripts/build-copilot-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/copilot-inner-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/acpx-release-attestations.mjs",
|
|
"packages/paperclip-runner/scripts/generate-acpx-profiles.mjs",
|
|
"packages/paperclip-runner/scripts/materialize-copilot-binary.mjs",
|
|
"packages/paperclip-runner/scripts/install-copilot-assets.mjs",
|
|
"packages/paperclip-runner/scripts/materialize-cursor-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/cursor-runtime-patch.mjs",
|
|
"packages/paperclip-runner/cursor-distributions.json",
|
|
"packages/paperclip-runner/runner/crates",
|
|
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/pi-distribution",
|
|
"packages/paperclip-runner/scripts/materialize-kimi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/patch-kimi-runtime.mjs",
|
|
"packages/paperclip-runner/scripts/kimi-instructions-runtime.mjs",
|
|
"packages/paperclip-runner/scripts/provision-kimi.mjs",
|
|
"packages/paperclip-runner/scripts/kimi-distribution",
|
|
"packages/paperclip-runner/src",
|
|
]) {
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).toContain(requiredPath);
|
|
}
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).not.toContain(
|
|
"packages/paperclip-eval-kernel",
|
|
);
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).not.toContain(
|
|
"packages/paperclip-runner",
|
|
);
|
|
expect(DAYTONA_IMAGE_DOCKERFILE_PATH).toBe(
|
|
"docker/daytona-runner/Dockerfile",
|
|
);
|
|
|
|
const contentId = await computeDaytonaImageContentId();
|
|
expect(contentId).toMatch(/^[0-9a-f]{64}$/);
|
|
});
|
|
|
|
it("changes for runtime source, package, lockfile, Dockerfile, frontend, base, or platform inputs", async () => {
|
|
const root = await mkdtemp(
|
|
path.join(tmpdir(), "paperclip-daytona-image-id-"),
|
|
);
|
|
const inputPaths = [
|
|
"docker/daytona-runner/Dockerfile",
|
|
"package.json",
|
|
"pnpm-lock.yaml",
|
|
"packages/paperclip-runner/package.json",
|
|
"packages/paperclip-runner/src",
|
|
"packages/paperclip-runner/runner/crates",
|
|
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/pi-distribution",
|
|
] as const;
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths,
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
} as const;
|
|
try {
|
|
await mkdir(path.join(root, "docker/daytona-runner"), {
|
|
recursive: true,
|
|
});
|
|
await mkdir(path.join(root, "packages/paperclip-runner/src"), {
|
|
recursive: true,
|
|
});
|
|
await mkdir(
|
|
path.join(
|
|
root,
|
|
"packages/paperclip-runner/runner/crates/runner-core/src",
|
|
),
|
|
{ recursive: true },
|
|
);
|
|
await writeFile(
|
|
path.join(root, "docker/daytona-runner/Dockerfile"),
|
|
"FROM pinned\n",
|
|
);
|
|
await writeFile(path.join(root, "package.json"), '{"private":true}\n');
|
|
await writeFile(
|
|
path.join(root, "pnpm-lock.yaml"),
|
|
"lockfileVersion: 9\n",
|
|
);
|
|
await writeFile(
|
|
path.join(root, "packages/paperclip-runner/package.json"),
|
|
'{"name":"@paperclipai/paperclip-runner"}\n',
|
|
);
|
|
await writeFile(
|
|
path.join(root, "packages/paperclip-runner/src/runner.ts"),
|
|
"version one\n",
|
|
);
|
|
await writeFile(
|
|
path.join(
|
|
root,
|
|
"packages/paperclip-runner/runner/crates/runner-core/src/lib.rs",
|
|
),
|
|
'pub const VERSION: &str = "one";\n',
|
|
);
|
|
await mkdir(path.join(root, "packages/paperclip-runner/scripts/pi-distribution"), { recursive: true });
|
|
for (const relativePath of [
|
|
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/pi-distribution/package.json",
|
|
"packages/paperclip-runner/scripts/pi-distribution/package-lock.json",
|
|
]) await writeFile(path.join(root, relativePath), "version one\n");
|
|
const baseline = await computeDaytonaImageContentId(options);
|
|
const candidate = await computeDaytonaImageContentId({ ...options, candidateProviders: ["pi"] });
|
|
expect(candidate).not.toBe(baseline);
|
|
expect(await computeDaytonaImageContentId({ ...options, candidateProviders: ["kimi"] })).not.toBe(baseline);
|
|
expect(await computeDaytonaImageContentId({ ...options, candidateProviders: ["copilot", "pi"] }))
|
|
.toBe(await computeDaytonaImageContentId({ ...options, candidateProviders: ["pi", "copilot"] }));
|
|
await expect(computeDaytonaImageContentId({ ...options, candidateProviders: ["pi", "pi"] }))
|
|
.rejects.toThrow("distinct known");
|
|
await expect(computeDaytonaImageContentId({ ...options, candidateProviders: ["unknown"] }))
|
|
.rejects.toThrow("distinct known");
|
|
expect(
|
|
await computeDaytonaImageContentId({
|
|
...options,
|
|
baseImages: [`example.test/base:1@sha256:${"b".repeat(64)}`],
|
|
}),
|
|
).not.toBe(baseline);
|
|
expect(
|
|
await computeDaytonaImageContentId({
|
|
...options,
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"d".repeat(64)}`,
|
|
}),
|
|
).not.toBe(baseline);
|
|
|
|
await writeFile(
|
|
path.join(root, "unrelated.txt"),
|
|
"does not enter the image\n",
|
|
);
|
|
expect(await computeDaytonaImageContentId(options)).toBe(baseline);
|
|
|
|
for (const relativePath of [
|
|
"docker/daytona-runner/Dockerfile",
|
|
"package.json",
|
|
"pnpm-lock.yaml",
|
|
"packages/paperclip-runner/package.json",
|
|
"packages/paperclip-runner/src/runner.ts",
|
|
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/pi-distribution/package.json",
|
|
"packages/paperclip-runner/scripts/pi-distribution/package-lock.json",
|
|
"packages/paperclip-runner/runner/crates/runner-core/src/lib.rs",
|
|
]) {
|
|
const absolutePath = path.join(root, relativePath);
|
|
const original = await readFile(absolutePath, "utf8");
|
|
await writeFile(absolutePath, `${original}changed\n`);
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
await writeFile(absolutePath, original);
|
|
}
|
|
expect(
|
|
await computeDaytonaImageContentId({
|
|
...options,
|
|
platform: "linux/arm64",
|
|
}),
|
|
).not.toBe(baseline);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it.each([
|
|
"packages/paperclip-runner/scripts/cursor-runtime-patch.mjs",
|
|
"packages/paperclip-runner/scripts/provider-pack-executable-shims.mjs",
|
|
"packages/paperclip-runner/scripts/materialize-kimi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/patch-kimi-runtime.mjs",
|
|
"packages/paperclip-runner/scripts/kimi-instructions-runtime.mjs",
|
|
"packages/paperclip-runner/scripts/provision-kimi.mjs",
|
|
"packages/paperclip-runner/scripts/kimi-distribution/package.json",
|
|
"packages/paperclip-runner/scripts/kimi-distribution/package-lock.json",
|
|
])("invalidates the image when %s changes", async (patchPath) => {
|
|
const root = await mkdtemp(path.join(tmpdir(), "paperclip-provider-image-id-"));
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS.some(inputPath => patchPath === inputPath || patchPath.startsWith(`${inputPath}/`))).toBe(true);
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths: [patchPath],
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
candidateProviders: ["cursor"],
|
|
} as const;
|
|
try {
|
|
await mkdir(path.dirname(path.join(root, patchPath)), { recursive: true });
|
|
await writeFile(path.join(root, patchPath), "export const policy = 'one';\n");
|
|
const baseline = await computeDaytonaImageContentId(options);
|
|
await writeFile(path.join(root, patchPath), "export const policy = 'two';\n");
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("reuses the image for runner-only tests and documentation", async () => {
|
|
const root = await mkdtemp(
|
|
path.join(tmpdir(), "paperclip-daytona-runner-development-inputs-"),
|
|
);
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths: ["packages/paperclip-runner"],
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
} as const;
|
|
try {
|
|
const runnerRoot = path.join(root, "packages/paperclip-runner");
|
|
await mkdir(path.join(runnerRoot, "src/live"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "docs"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "spec"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "scripts"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "test-fixtures"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "runner/crates/runner-core/src"), {
|
|
recursive: true,
|
|
});
|
|
await mkdir(path.join(runnerRoot, "runner/crates/runner-core/tests"), {
|
|
recursive: true,
|
|
});
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.ts"),
|
|
"export const runtime = 'one';\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "runner/crates/runner-core/src/lib.rs"),
|
|
'pub const RUNTIME: &str = "one";\n',
|
|
);
|
|
await writeFile(path.join(runnerRoot, "README.md"), "first readme\n");
|
|
await writeFile(
|
|
path.join(runnerRoot, "docs/local-runner.md"),
|
|
"first documentation\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "spec/architecture.md"),
|
|
"first architecture note\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.test.ts"),
|
|
"first TypeScript test\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "test-fixtures/provider.json"),
|
|
'{"fixture":"one"}\n',
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "scripts/capability-clean-room-smoke.mjs"),
|
|
"first smoke probe\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "runner/crates/runner-core/tests/recovery.rs"),
|
|
"// first Rust integration test\n",
|
|
);
|
|
|
|
const baseline = await computeDaytonaImageContentId(options);
|
|
await mkdir(path.join(runnerRoot, "src/new-test-only-directory"));
|
|
await writeFile(
|
|
path.join(
|
|
runnerRoot,
|
|
"src/new-test-only-directory/transport-edge.test.ts",
|
|
),
|
|
"new TypeScript test\n",
|
|
);
|
|
await writeFile(path.join(runnerRoot, "README.md"), "second readme\n");
|
|
await writeFile(
|
|
path.join(runnerRoot, "docs/local-runner.md"),
|
|
"second documentation\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "spec/architecture.md"),
|
|
"second architecture note\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.test.ts"),
|
|
"second TypeScript test\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "test-fixtures/provider.json"),
|
|
'{"fixture":"two"}\n',
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "scripts/capability-clean-room-smoke.mjs"),
|
|
"second smoke probe\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "runner/crates/runner-core/tests/recovery.rs"),
|
|
"// second Rust integration test\n",
|
|
);
|
|
expect(await computeDaytonaImageContentId(options)).toBe(baseline);
|
|
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.ts"),
|
|
"export const runtime = 'two';\n",
|
|
);
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.ts"),
|
|
"export const runtime = 'one';\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "runner/crates/runner-core/src/lib.rs"),
|
|
'pub const RUNTIME: &str = "two";\n',
|
|
);
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("rejects mutable Docker base references", () => {
|
|
expect(() => extractDaytonaBaseImages("FROM node:24-bookworm\n")).toThrow(
|
|
"must use an immutable sha256 digest",
|
|
);
|
|
});
|
|
|
|
it("rejects a mutable or missing Dockerfile syntax frontend", async () => {
|
|
expect(() =>
|
|
extractDaytonaDockerfileFrontendDigest(
|
|
"# syntax=docker/dockerfile:1.7\nFROM scratch\n",
|
|
),
|
|
).toThrow("must pin its syntax frontend");
|
|
expect(() =>
|
|
extractDaytonaDockerfileFrontendDigest("FROM scratch\n"),
|
|
).toThrow("must pin its syntax frontend");
|
|
await expect(
|
|
computeDaytonaImageContentId({
|
|
inputPaths: [],
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: "sha256:mutable",
|
|
}),
|
|
).rejects.toThrow("must use an immutable sha256 digest");
|
|
});
|
|
});
|