mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-25 02:26:12 +02:00
## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests check complete user journeys. > - The current harness prepares self-hosted instances. > - Hosted workspaces need separate authentication, setup, and cleanup. > - The same in-app story should run on either deployment. > - This pull request adds deployment targets and explicit prerequisites. > - The private Cloud adapter owns fleet access and lifecycle operations. ## Linked Issues or Issue Description **Subsystem affected** Product E2E harness, fixture registry, and reports. **Problem or motivation** Self-hosted test setup cannot qualify a deployed Cloud workspace. Copying each story into a second harness would make its assertions diverge. **Proposed solution** Add explicit deployment targets, setup recipes, and ordered campaigns. Reuse a no-model task-properties story on both deployments. Let fresh Cloud onboarding provide verified context for that same story. Keep Cloud lifecycle and credentials behind a private target provider. **Roadmap alignment** This supports Cloud deployment qualification and the existing agent eval system. It adds no application endpoint or scheduler. Related work: #17112. The separate onboarding gateway UI prerequisite is #17253. ## What Changed - Add explicit target configuration and a credential-free preparation preview. Keep historical selectors and the current `--all` matrix unchanged. - Resolve requested fixture prerequisites, including a run-owned backlog task prepared before measurement. Reuse verified onboarding context without counting recipe setup as onboarding coverage. - Add task assignment persistence and the actual onboarding parent/child first-task journey, with an independent artifact oracle and native trace requirement. - Preserve failed boundaries and blocked dependents, deployment identity, costs, phase outcomes, and private evidence in the existing dashboard. - Validate success assertions and accounting before successful cleanup. Bound collection by the campaign deadline and cleanup/evidence hooks by a separate ten-minute grace, with cancellation and stop-only fallback. - Stage dashboard rendering separately so a timed-out renderer cannot overwrite retained failure results. Add evidence finalization hooks so the private provider can save evidence before target deletion and lease release. - Reject private journey bundles in the public history publisher. Browser-only results have no model or provider. ## Verification - The unchanged task-properties story passed on managed self-hosted and a registered testing Cloud workspace. Authentication, setup, journey, collection, and cleanup passed. Model calls: zero. Earlier attempts remain private and retained. - Final journey and report tests: 96 passed in eight files. Catalog compatibility: 39 passed. The wider focused run exposed three stale master catalog expectations; these now match the already shipped case-handoff v4 metadata and four-cell matrix. The default 385-cell selection is unchanged. - Review regressions reproduced premature success for a zero-request receipt and a missing required assertion. Both now fail before success cleanup while retaining cost receipts. The initial lifecycle fixes passed 33 checks. A subsequent review exposed a late renderer write that could erase saved failures; rendering now writes into isolated staging and publishes HTML only after it settles. Combined campaign, fixture-contract, and retained-report checks: 67 passed, including late writes, delayed timer callbacks, stalled collection, release, stop, evidence hooks, and browser close. Harness typecheck passed after the fixes. - A broader harness unit attempt completed 3,601 passing cases, two failures, and two skips. The failures exposed a stale Pi count and an omitted active Copilot fixture in the Daytona image input closure. The focused compatibility fixes passed 83 tests. After all source fixes, the full harness support suite passed: 160 files, 3,621 tests, two skips, plus 128 native-source checks. The earlier failures remain recorded. - Harness typecheck, repository typecheck, repository build, map checks, and Storybook catalog checks passed on the rebased branch. - The required full `pnpm test:run` attempt timed out after 35 minutes. It had 1,195 completed passing cases and one chat integration failure before its first suite completed. That case and nine adjacent cases pass in isolation. The original failure remains unresolved and retained; this is not a full-suite pass. - Current-head remote checks (`39b2a1ed8c49222fabd1ba8e225cac81bff5c6f7`): 53 passed, two skipped, none pending or failing. Greptile completed successfully at 5/5 with no unresolved review threads. - Fresh paid Cloud onboarding, dependent reuse, and disposable cleanup have not been qualified live. They require deployment of the separate UI prerequisite and actual Cloud gateway budget admission. The trusted Cloud workflow remains disabled. - Final live verification at `39b2a1ed8c`: the unchanged shared story passed on managed self-hosted and the registered testing Cloud stack using the prepared backlog-task recipe. Both cleaned up successfully with zero model calls. Independent private S3 checks confirmed final evidence and released QA identity/stack leases. Cloud evidence remains private. ## Risks - A deployed UI or Cloud control-plane change can invalidate selectors or provider contracts. Version drift fails the attempt. - Cloud evidence and lifecycle recovery are private-provider responsibilities. A pending cleanup or evidence upload must keep the target unavailable for reuse. - No release is deployed or pinned by this change. No automatic story retry is added. The initial pilot retains its explicit $10, one disposable workspace, and 30-minute bounds. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and delegated agents. The runtime does not expose a more specific deployment model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
584 lines
25 KiB
TypeScript
584 lines
25 KiB
TypeScript
import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises";
|
|
import { tmpdir } from "node:os";
|
|
import path from "node:path";
|
|
import { describe, expect, it } from "vitest";
|
|
import {
|
|
computeDaytonaImageContentId,
|
|
DAYTONA_IMAGE_DOCKERFILE_PATH,
|
|
DAYTONA_IMAGE_INPUT_PATHS,
|
|
extractDaytonaBaseImages,
|
|
extractDaytonaDockerfileFrontendDigest,
|
|
} from "./daytona-image-content.js";
|
|
|
|
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
|
|
|
|
describe("runner E2E Daytona image contract", () => {
|
|
it("retains the active Copilot identity declaration in the Docker context", async () => {
|
|
const [generator, dockerignore] = await Promise.all([
|
|
readFile(path.join(repositoryRoot, "packages/paperclip-runner/scripts/copilot-profile-identity.mjs"), "utf8"),
|
|
readFile(path.join(repositoryRoot, ".dockerignore"), "utf8"),
|
|
]);
|
|
const fixture = generator.match(/const fixture = resolve\(root, "([^"]+)"\)/)?.[1];
|
|
expect(fixture).toBeDefined();
|
|
const fixturePath = `packages/paperclip-runner/${fixture}`;
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).toContain(fixturePath);
|
|
await expect(readFile(path.join(repositoryRoot, fixturePath), "utf8")).resolves.toContain('"commandDigest"');
|
|
// Docker's last matching rule wins, including the parent-directory exclusion.
|
|
const rules = dockerignore.split(/\r?\n/).filter(rule => rule && !rule.startsWith("#"));
|
|
const lastRule = rules.filter(rule => {
|
|
const pattern = rule.startsWith("!") ? rule.slice(1) : rule;
|
|
return path.matchesGlob(fixturePath, pattern) || fixturePath.startsWith(`${pattern}/`);
|
|
}).at(-1);
|
|
expect(lastRule?.startsWith("!")).toBe(true);
|
|
// The exception also covers the next version without exposing other live evidence.
|
|
const exception = lastRule!.slice(1);
|
|
expect(path.matchesGlob("packages/paperclip-runner/test/fixtures/copilot-profile-v999-identity.json", exception)).toBe(true);
|
|
expect(path.matchesGlob("packages/paperclip-runner/test/fixtures/copilot-live-denial-2026-09-28.json", exception)).toBe(false);
|
|
});
|
|
|
|
it("invalidates the image content ID when the active Copilot identity changes", async () => {
|
|
const generator = await readFile(
|
|
path.join(repositoryRoot, "packages/paperclip-runner/scripts/copilot-profile-identity.mjs"),
|
|
"utf8",
|
|
);
|
|
const fixture = generator.match(/const fixture = resolve\(root, "([^"]+)"\)/)?.[1];
|
|
expect(fixture).toBeDefined();
|
|
const fixturePath = `packages/paperclip-runner/${fixture}`;
|
|
const root = await mkdtemp(path.join(tmpdir(), "paperclip-daytona-copilot-identity-"));
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths: DAYTONA_IMAGE_INPUT_PATHS.filter(inputPath => inputPath === fixturePath),
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
} as const;
|
|
try {
|
|
await mkdir(path.dirname(path.join(root, fixturePath)), { recursive: true });
|
|
await writeFile(path.join(root, fixturePath), '{"commandDigest":"before"}\n');
|
|
const before = await computeDaytonaImageContentId(options);
|
|
await writeFile(path.join(root, fixturePath), '{"commandDigest":"after"}\n');
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(before);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("keeps the qualified native Grok binary separate from the legacy command", async () => {
|
|
const [dockerfile, packBuilder, runnerPackage] = await Promise.all([
|
|
readFile(path.join(repositoryRoot, "docker/daytona-runner/Dockerfile"), "utf8"),
|
|
readFile(path.join(repositoryRoot, "packages/paperclip-runner/scripts/build-provider-pack.mjs"), "utf8"),
|
|
readFile(path.join(repositoryRoot, "packages/paperclip-runner/package.json"), "utf8"),
|
|
]);
|
|
expect(dockerfile).toMatch(/@xai-official\/grok@\d+\.\d+\.\d+/);
|
|
expect(dockerfile).not.toMatch(/for cli in[^;]*\bgrok\b/);
|
|
expect(packBuilder).not.toMatch(/writePortable\w+Shim\("grok"/);
|
|
expect(JSON.parse(runnerPackage).dependencies).not.toHaveProperty("@paperclipai/grok-acp");
|
|
expect(packBuilder).toContain('path: "dist/providers/grok/launcher.cjs"');
|
|
expect(dockerfile).toContain("scripts/provision-grok.mjs");
|
|
expect(dockerfile).toContain("/opt/paperclip/providers/grok/1.0.13/grok");
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).toContain("packages/paperclip-runner/scripts/provision-grok.mjs");
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).not.toContain("packages/grok-acp/package.json");
|
|
});
|
|
|
|
it("builds runnerd and the provider pack and verifies every required transport", async () => {
|
|
const [dockerfile, dockerignore, workflow] = await Promise.all([
|
|
readFile(
|
|
path.join(repositoryRoot, "docker/daytona-runner/Dockerfile"),
|
|
"utf8",
|
|
),
|
|
readFile(path.join(repositoryRoot, ".dockerignore"), "utf8"),
|
|
readFile(
|
|
path.join(
|
|
repositoryRoot,
|
|
".github/workflows/runner-full-stack-e2e.yml",
|
|
),
|
|
"utf8",
|
|
),
|
|
]);
|
|
const normalizedDockerfile = dockerfile.replace(/\\\r?\n\s*/g, " ");
|
|
const daytonaImageJob = workflow.match(/^ daytona_image:\n[\s\S]*?(?=^ \w+:)/m)?.[0];
|
|
expect(daytonaImageJob).toBeDefined();
|
|
expect(dockerfile).toContain("--bin paperclip-runnerd");
|
|
expect(dockerfile).toContain("build-provider-pack.mjs /provider-pack");
|
|
expect(normalizedDockerfile).not.toContain(
|
|
"COPY packages/paperclip-eval-kernel ./packages/paperclip-eval-kernel",
|
|
);
|
|
expect(normalizedDockerfile).not.toContain(
|
|
"COPY packages/paperclip-runner ./packages/paperclip-runner",
|
|
);
|
|
expect(dockerfile).toContain(
|
|
"COPY packages ./packages",
|
|
);
|
|
expect(dockerfile).toContain(
|
|
"/opt/paperclip-runner/provider-pack/provider-pack.json",
|
|
);
|
|
expect(dockerfile).toContain(
|
|
"${PAPERCLIP_RUNNER_PROVIDER_PACK_ROOT}/node_modules/.bin",
|
|
);
|
|
for (const command of ["acpx", "claude-agent-acp", "codex-acp"]) {
|
|
expect(dockerfile).toContain(command);
|
|
}
|
|
for (const transport of ["dial_ws_loopback", "dial_wss", "listen_ws"]) {
|
|
expect(dockerfile).toContain(transport);
|
|
}
|
|
expect(dockerfile).toContain(
|
|
'metadata="$(paperclip-runnerd --build-metadata)"',
|
|
);
|
|
expect(dockerfile).toContain("provider-pack.json");
|
|
expect(dockerfile).toContain("io.paperclip.runner.content-id");
|
|
expect(dockerfile).toContain("org.opencontainers.image.revision");
|
|
expect(extractDaytonaDockerfileFrontendDigest(dockerfile)).toBe(
|
|
"sha256:a57df69d0ea827fb7266491f2813635de6f17269be881f696fbfdf2d83dda33e",
|
|
);
|
|
expect(extractDaytonaBaseImages(dockerfile)).toEqual([
|
|
"rust:1.97-bookworm@sha256:408fe88047cef61a2087653b0c5255fa51c0f2d6d94ddedd7a2562a9b91a46f6",
|
|
"node:24.21.0-bookworm@sha256:5a750d3be5e5c80275f8c9a5367c3aed99c2875656590c8d0701c7ee687f5f0a",
|
|
"daytonaio/sandbox:0.8.0@sha256:eadf88e4391072b7ad4bed27d9cadfc9fe9d8ed375d9219d34c2ccb518f213e3",
|
|
]);
|
|
expect(dockerignore).toContain("**/node_modules");
|
|
expect(dockerignore).toContain("packages/paperclip-runner/dist");
|
|
expect(dockerignore).toContain("packages/paperclip-runner/runner/target");
|
|
expect(await readFile(path.join(repositoryRoot, ".github/docker-context-checks.Dockerfile"), "utf8")).toContain("node packages/paperclip-runner/scripts/generate-acpx-profiles.mjs --check");
|
|
for (const declaration of [
|
|
"test-fixtures/pi-acp/profile-v19-identity.json",
|
|
"test/fixtures/copilot-profile-v16-identity.json",
|
|
]) {
|
|
expect(dockerignore).toContain(`!packages/paperclip-runner/${declaration}`);
|
|
await expect(readFile(path.join(repositoryRoot, "packages/paperclip-runner", declaration), "utf8")).resolves.toBeTruthy();
|
|
}
|
|
for (const developmentOnlyInput of [
|
|
"packages/paperclip-runner/devtools",
|
|
"packages/paperclip-runner/docs",
|
|
"packages/paperclip-runner/examples",
|
|
"packages/paperclip-runner/test",
|
|
"packages/paperclip-runner/test-fixtures",
|
|
"packages/paperclip-runner/test-support",
|
|
"packages/paperclip-runner/**/*.md",
|
|
"packages/paperclip-runner/**/*.test.ts",
|
|
"packages/paperclip-runner/runner/crates/*/tests",
|
|
"packages/paperclip-runner/scripts/*-smoke.mjs",
|
|
]) {
|
|
expect(dockerignore).toContain(developmentOnlyInput);
|
|
}
|
|
expect(workflow).toContain("--platform linux/amd64");
|
|
expect(workflow).toContain(
|
|
"Compute Daytona image content ID with pinned bases",
|
|
);
|
|
expect(workflow).toContain(
|
|
"e2e-content-${{ needs.catalog.outputs.daytona_image_content_id }}",
|
|
);
|
|
expect(workflow).toContain(
|
|
'--build-arg "PAPERCLIP_RUNNER_CONTENT_ID=${IMAGE_CONTENT_ID}"',
|
|
);
|
|
expect(daytonaImageJob).toContain(
|
|
"TARGET_LOCK_SHA256: ${{ needs.target_lock.outputs.lock_sha256 }}",
|
|
);
|
|
expect(daytonaImageJob).toContain(
|
|
'[[ "$TARGET_LOCK_SHA256" =~ ^[0-9a-f]{64}$ ]]',
|
|
);
|
|
expect(daytonaImageJob).toContain(
|
|
'--build-arg "PAPERCLIP_RUNNER_LOCK_SHA256=${TARGET_LOCK_SHA256}"',
|
|
);
|
|
expect(
|
|
daytonaImageJob!.indexOf('[[ "$TARGET_LOCK_SHA256" =~ ^[0-9a-f]{64}$ ]]'),
|
|
).toBeLessThan(
|
|
daytonaImageJob!.indexOf(
|
|
'--build-arg "PAPERCLIP_RUNNER_LOCK_SHA256=${TARGET_LOCK_SHA256}"',
|
|
),
|
|
);
|
|
expect(workflow).toContain(
|
|
"IMAGE_CACHE: ghcr.io/paperclipai/paperclip-daytona-runner:e2e-buildcache-amd64",
|
|
);
|
|
expect(workflow).toContain(
|
|
'--cache-from "type=registry,ref=${IMAGE_CACHE}"',
|
|
);
|
|
expect(workflow).toContain(
|
|
'--cache-to "type=registry,ref=${IMAGE_CACHE},mode=max"',
|
|
);
|
|
expect(workflow).toContain(
|
|
'if [ "$TARGET_REF" = "refs/heads/$DEFAULT_BRANCH" ]; then',
|
|
);
|
|
expect(workflow).not.toContain("e2e-git-${{ github.sha }}");
|
|
expect(workflow).toContain("cosign sign --yes");
|
|
expect(workflow).toContain("docker logout ghcr.io");
|
|
expect(workflow).toContain(`docker buildx imagetools inspect "$immutable"`);
|
|
expect(workflow).toContain(`--format '{{json .Image}}'`);
|
|
expect(workflow).not.toContain(`docker --config "$anonymous_config" pull`);
|
|
// Remote Daytona manifests must use registry inspection. Local oracle
|
|
// images in the test job can still use the Docker daemon.
|
|
expect(daytonaImageJob).not.toContain("docker image inspect");
|
|
expect(workflow).not.toContain("docker buildx prune --all --force");
|
|
expect(workflow).not.toContain("docker system prune --all --force");
|
|
expect(workflow).toContain('.architecture == "amd64"');
|
|
expect(workflow).toContain('.os == "linux"');
|
|
expect(workflow).toContain('.config.User == "daytona"');
|
|
expect(workflow).toContain("PAPERCLIP_RUNNER_PROVIDER_PACK_ROOT=");
|
|
expect(workflow).toContain(
|
|
"node packages/paperclip-runner/scripts/build-provider-pack.mjs packages/paperclip-runner/provider-pack",
|
|
);
|
|
expect(workflow).toContain(
|
|
"PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH: ${{ github.workspace }}/packages/paperclip-runner/provider-pack",
|
|
);
|
|
expect(workflow).toContain(
|
|
"PAPERCLIP_RUNNER_SOURCE_REVISION: ${{ needs.daytona_image.outputs.source_revision }}",
|
|
);
|
|
expect(workflow.indexOf("cosign verify")).toBeLessThan(
|
|
workflow.indexOf("docker logout ghcr.io"),
|
|
);
|
|
expect(workflow.indexOf("docker logout ghcr.io")).toBeLessThan(
|
|
workflow.indexOf(`--format '{{json .Image}}'`),
|
|
);
|
|
const providerInstall = dockerfile.indexOf(
|
|
"pnpm install --frozen-lockfile --filter '@paperclipai/paperclip-runner...'",
|
|
);
|
|
const runnerSourceCopy = dockerfile.indexOf(
|
|
"COPY packages ./packages",
|
|
);
|
|
const providerRevisionArg = dockerfile.indexOf(
|
|
"ARG PAPERCLIP_RUNNER_SOURCE_REVISION",
|
|
);
|
|
const cliInstall = dockerfile.indexOf("npm install -g");
|
|
const finalMetadataArgs = dockerfile.lastIndexOf(
|
|
"ARG PAPERCLIP_RUNNER_CONTENT_ID",
|
|
);
|
|
expect(providerInstall).toBeGreaterThan(0);
|
|
expect(runnerSourceCopy).toBeGreaterThan(0);
|
|
expect(runnerSourceCopy).toBeLessThan(providerInstall);
|
|
expect(providerInstall).toBeLessThan(providerRevisionArg);
|
|
expect(cliInstall).toBeGreaterThan(0);
|
|
expect(cliInstall).toBeLessThan(finalMetadataArgs);
|
|
});
|
|
|
|
it("hashes the audited image dependency closure rather than the repository revision", async () => {
|
|
for (const requiredPath of [
|
|
".dockerignore",
|
|
"docker/daytona-runner/Dockerfile",
|
|
"pnpm-lock.yaml",
|
|
"patches",
|
|
"packages/paperclip-eval-kernel/src",
|
|
"packages/paperclip-runner/package.json",
|
|
"packages/paperclip-runner/scripts/candidate-provider-pack.mjs",
|
|
"packages/paperclip-runner/scripts/build-copilot-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/copilot-inner-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/copilot-profile-identity.mjs",
|
|
"packages/paperclip-runner/scripts/materialize-copilot-binary.mjs",
|
|
"packages/paperclip-runner/scripts/install-copilot-assets.mjs",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v12-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v14-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v21-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v24-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v25-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v26-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v35-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v40-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v42-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v46-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v47-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v48-identity.json",
|
|
"packages/paperclip-runner/test/fixtures/copilot-profile-v49-identity.json",
|
|
"packages/paperclip-runner/scripts/materialize-cursor-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/cursor-runtime-patch.mjs",
|
|
"packages/paperclip-runner/cursor-distributions.json",
|
|
"packages/paperclip-runner/runner/crates",
|
|
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/pi-distribution",
|
|
"packages/paperclip-runner/src",
|
|
]) {
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).toContain(requiredPath);
|
|
}
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).not.toContain(
|
|
"packages/paperclip-eval-kernel",
|
|
);
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).not.toContain(
|
|
"packages/paperclip-runner",
|
|
);
|
|
expect(DAYTONA_IMAGE_DOCKERFILE_PATH).toBe(
|
|
"docker/daytona-runner/Dockerfile",
|
|
);
|
|
|
|
const contentId = await computeDaytonaImageContentId();
|
|
expect(contentId).toMatch(/^[0-9a-f]{64}$/);
|
|
});
|
|
|
|
it("changes for runtime source, package, lockfile, Dockerfile, frontend, base, or platform inputs", async () => {
|
|
const root = await mkdtemp(
|
|
path.join(tmpdir(), "paperclip-daytona-image-id-"),
|
|
);
|
|
const inputPaths = [
|
|
"docker/daytona-runner/Dockerfile",
|
|
"package.json",
|
|
"pnpm-lock.yaml",
|
|
"packages/paperclip-runner/package.json",
|
|
"packages/paperclip-runner/src",
|
|
"packages/paperclip-runner/runner/crates",
|
|
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/pi-distribution",
|
|
] as const;
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths,
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
} as const;
|
|
try {
|
|
await mkdir(path.join(root, "docker/daytona-runner"), {
|
|
recursive: true,
|
|
});
|
|
await mkdir(path.join(root, "packages/paperclip-runner/src"), {
|
|
recursive: true,
|
|
});
|
|
await mkdir(
|
|
path.join(
|
|
root,
|
|
"packages/paperclip-runner/runner/crates/runner-core/src",
|
|
),
|
|
{ recursive: true },
|
|
);
|
|
await writeFile(
|
|
path.join(root, "docker/daytona-runner/Dockerfile"),
|
|
"FROM pinned\n",
|
|
);
|
|
await writeFile(path.join(root, "package.json"), '{"private":true}\n');
|
|
await writeFile(
|
|
path.join(root, "pnpm-lock.yaml"),
|
|
"lockfileVersion: 9\n",
|
|
);
|
|
await writeFile(
|
|
path.join(root, "packages/paperclip-runner/package.json"),
|
|
'{"name":"@paperclipai/paperclip-runner"}\n',
|
|
);
|
|
await writeFile(
|
|
path.join(root, "packages/paperclip-runner/src/runner.ts"),
|
|
"version one\n",
|
|
);
|
|
await writeFile(
|
|
path.join(
|
|
root,
|
|
"packages/paperclip-runner/runner/crates/runner-core/src/lib.rs",
|
|
),
|
|
'pub const VERSION: &str = "one";\n',
|
|
);
|
|
await mkdir(path.join(root, "packages/paperclip-runner/scripts/pi-distribution"), { recursive: true });
|
|
for (const relativePath of [
|
|
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/pi-distribution/package.json",
|
|
"packages/paperclip-runner/scripts/pi-distribution/package-lock.json",
|
|
]) await writeFile(path.join(root, relativePath), "version one\n");
|
|
const baseline = await computeDaytonaImageContentId(options);
|
|
const candidate = await computeDaytonaImageContentId({ ...options, candidateProviders: ["pi"] });
|
|
expect(candidate).not.toBe(baseline);
|
|
expect(await computeDaytonaImageContentId({ ...options, candidateProviders: ["copilot", "pi"] }))
|
|
.toBe(await computeDaytonaImageContentId({ ...options, candidateProviders: ["pi", "copilot"] }));
|
|
await expect(computeDaytonaImageContentId({ ...options, candidateProviders: ["pi", "pi"] }))
|
|
.rejects.toThrow("distinct known");
|
|
await expect(computeDaytonaImageContentId({ ...options, candidateProviders: ["unknown"] }))
|
|
.rejects.toThrow("distinct known");
|
|
expect(
|
|
await computeDaytonaImageContentId({
|
|
...options,
|
|
baseImages: [`example.test/base:1@sha256:${"b".repeat(64)}`],
|
|
}),
|
|
).not.toBe(baseline);
|
|
expect(
|
|
await computeDaytonaImageContentId({
|
|
...options,
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"d".repeat(64)}`,
|
|
}),
|
|
).not.toBe(baseline);
|
|
|
|
await writeFile(
|
|
path.join(root, "unrelated.txt"),
|
|
"does not enter the image\n",
|
|
);
|
|
expect(await computeDaytonaImageContentId(options)).toBe(baseline);
|
|
|
|
for (const relativePath of [
|
|
"docker/daytona-runner/Dockerfile",
|
|
"package.json",
|
|
"pnpm-lock.yaml",
|
|
"packages/paperclip-runner/package.json",
|
|
"packages/paperclip-runner/src/runner.ts",
|
|
"packages/paperclip-runner/scripts/materialize-pi-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/pi-distribution/package.json",
|
|
"packages/paperclip-runner/scripts/pi-distribution/package-lock.json",
|
|
"packages/paperclip-runner/runner/crates/runner-core/src/lib.rs",
|
|
]) {
|
|
const absolutePath = path.join(root, relativePath);
|
|
const original = await readFile(absolutePath, "utf8");
|
|
await writeFile(absolutePath, `${original}changed\n`);
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
await writeFile(absolutePath, original);
|
|
}
|
|
expect(
|
|
await computeDaytonaImageContentId({
|
|
...options,
|
|
platform: "linux/arm64",
|
|
}),
|
|
).not.toBe(baseline);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it.each([
|
|
"packages/paperclip-runner/scripts/cursor-runtime-patch.mjs",
|
|
"packages/paperclip-runner/scripts/provider-pack-executable-shims.mjs",
|
|
])("invalidates the image when %s changes", async (patchPath) => {
|
|
const root = await mkdtemp(path.join(tmpdir(), "paperclip-provider-image-id-"));
|
|
expect(DAYTONA_IMAGE_INPUT_PATHS).toContain(patchPath);
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths: [patchPath],
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
candidateProviders: ["cursor"],
|
|
} as const;
|
|
try {
|
|
await mkdir(path.dirname(path.join(root, patchPath)), { recursive: true });
|
|
await writeFile(path.join(root, patchPath), "export const policy = 'one';\n");
|
|
const baseline = await computeDaytonaImageContentId(options);
|
|
await writeFile(path.join(root, patchPath), "export const policy = 'two';\n");
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("reuses the image for runner-only tests and documentation", async () => {
|
|
const root = await mkdtemp(
|
|
path.join(tmpdir(), "paperclip-daytona-runner-development-inputs-"),
|
|
);
|
|
const options = {
|
|
repositoryRoot: root,
|
|
inputPaths: ["packages/paperclip-runner"],
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: `sha256:${"c".repeat(64)}`,
|
|
} as const;
|
|
try {
|
|
const runnerRoot = path.join(root, "packages/paperclip-runner");
|
|
await mkdir(path.join(runnerRoot, "src/live"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "docs"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "spec"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "scripts"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "test-fixtures"), { recursive: true });
|
|
await mkdir(path.join(runnerRoot, "runner/crates/runner-core/src"), {
|
|
recursive: true,
|
|
});
|
|
await mkdir(path.join(runnerRoot, "runner/crates/runner-core/tests"), {
|
|
recursive: true,
|
|
});
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.ts"),
|
|
"export const runtime = 'one';\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "runner/crates/runner-core/src/lib.rs"),
|
|
'pub const RUNTIME: &str = "one";\n',
|
|
);
|
|
await writeFile(path.join(runnerRoot, "README.md"), "first readme\n");
|
|
await writeFile(
|
|
path.join(runnerRoot, "docs/local-runner.md"),
|
|
"first documentation\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "spec/architecture.md"),
|
|
"first architecture note\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.test.ts"),
|
|
"first TypeScript test\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "test-fixtures/provider.json"),
|
|
'{"fixture":"one"}\n',
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "scripts/capability-clean-room-smoke.mjs"),
|
|
"first smoke probe\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "runner/crates/runner-core/tests/recovery.rs"),
|
|
"// first Rust integration test\n",
|
|
);
|
|
|
|
const baseline = await computeDaytonaImageContentId(options);
|
|
await mkdir(path.join(runnerRoot, "src/new-test-only-directory"));
|
|
await writeFile(
|
|
path.join(
|
|
runnerRoot,
|
|
"src/new-test-only-directory/transport-edge.test.ts",
|
|
),
|
|
"new TypeScript test\n",
|
|
);
|
|
await writeFile(path.join(runnerRoot, "README.md"), "second readme\n");
|
|
await writeFile(
|
|
path.join(runnerRoot, "docs/local-runner.md"),
|
|
"second documentation\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "spec/architecture.md"),
|
|
"second architecture note\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.test.ts"),
|
|
"second TypeScript test\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "test-fixtures/provider.json"),
|
|
'{"fixture":"two"}\n',
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "scripts/capability-clean-room-smoke.mjs"),
|
|
"second smoke probe\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "runner/crates/runner-core/tests/recovery.rs"),
|
|
"// second Rust integration test\n",
|
|
);
|
|
expect(await computeDaytonaImageContentId(options)).toBe(baseline);
|
|
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.ts"),
|
|
"export const runtime = 'two';\n",
|
|
);
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
|
|
await writeFile(
|
|
path.join(runnerRoot, "src/live/transport.ts"),
|
|
"export const runtime = 'one';\n",
|
|
);
|
|
await writeFile(
|
|
path.join(runnerRoot, "runner/crates/runner-core/src/lib.rs"),
|
|
'pub const RUNTIME: &str = "two";\n',
|
|
);
|
|
expect(await computeDaytonaImageContentId(options)).not.toBe(baseline);
|
|
} finally {
|
|
await rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("rejects mutable Docker base references", () => {
|
|
expect(() => extractDaytonaBaseImages("FROM node:24-bookworm\n")).toThrow(
|
|
"must use an immutable sha256 digest",
|
|
);
|
|
});
|
|
|
|
it("rejects a mutable or missing Dockerfile syntax frontend", async () => {
|
|
expect(() =>
|
|
extractDaytonaDockerfileFrontendDigest(
|
|
"# syntax=docker/dockerfile:1.7\nFROM scratch\n",
|
|
),
|
|
).toThrow("must pin its syntax frontend");
|
|
expect(() =>
|
|
extractDaytonaDockerfileFrontendDigest("FROM scratch\n"),
|
|
).toThrow("must pin its syntax frontend");
|
|
await expect(
|
|
computeDaytonaImageContentId({
|
|
inputPaths: [],
|
|
baseImages: [`example.test/base:1@sha256:${"a".repeat(64)}`],
|
|
frontendDigest: "sha256:mutable",
|
|
}),
|
|
).rejects.toThrow("must use an immutable sha256 digest");
|
|
});
|
|
});
|