Files
PaperClipAI/tests/runner-e2e/daytona-image-content.ts
Devin FoleyandPaperclip 0ac194450a fix: make Copilot provider-pack wrappers portable (#15586)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Container images include a movable provider pack for agent
execution.
> - pnpm executable wrappers can contain the temporary build directory.
> - New optional Copilot packages add wrappers that the build does not
replace.
> - This pull request gives those installed wrappers relative executable
paths.
> - Image publication can finish while the existing path check remains
enforced.

## Linked Issues or Issue Description

Refs #15572 and #15560. The small portability loop comes from
cryppadotta's larger Copilot runtime PR #15560. This separate fix
repairs image publication without waiting for that feature's
qualification and runtime changes. That PR can remove its duplicate loop
after this lands.

**What happened?**

The standard Docker build stops with `Provider pack shim
copilot-linux-x64 retains its temporary build path`. The failure
occurred before and after #15522. See [the failed master
build](https://github.com/paperclipai/paperclip/actions/runs/37802065316).

**Expected behavior**

Installed native Copilot wrappers resolve their pinned executable after
the provider pack moves. Optional packages that are absent do not gain a
command. The builder still rejects wrappers with temporary paths.

**Steps to reproduce**

1. Run the provider-pack build from the affected master revision on
Linux x64.
2. Let `pnpm deploy --prod` install the optional Copilot package.
3. The wrapper scan rejects its temporary `NODE_PATH`.

**Paperclip version or commit**

Master `3367b75ccce34d02f355cda1f1ed3fe0b34cf93d`.

**Deployment mode**

Docker and provider-pack builds.

## What Changed

- Replace installed Copilot platform wrappers with relative native
executable launchers.
- Move the existing executable wrapper writer into an importable helper.
Retain the same launch behavior for Node, Claude and OpenCode.
- Test relocation, argument handling, exit status, absent optional
packages and missing executable packages. Register the tests in the
existing Runner test preparation command.
- Hash the helper in Daytona image identity and prove that helper
changes invalidate the image cache.
- Document the packaging rule. Keep dependencies, provider qualification
and the final temporary-path check unchanged.

## Verification

- 17 Node packaging tests passed across the new wrapper tests,
provider-pack release tests and candidate selection tests.
- Six existing bundled remote-provider-pack tests passed using the
Runner Vitest configuration.
- Nine Daytona image identity tests and Runner E2E typecheck passed
after the cache-input correction.
- `pnpm -r typecheck`: passed.
- `pnpm build`: passed.
- Real `pnpm deploy --prod` reproduction on macOS ARM64: the original
Copilot wrapper contained the temporary path. After the rewrite and
directory relocation, the actual executable returned Copilot CLI 1.0.88
with exit code zero using only `/usr/bin:/bin` in `PATH`.
- Syntax checks, `git diff --check` and the pre-push secret scan passed.
- The full local `pnpm test:run` reproduced the same five
skill/connector fixture failures observed earlier in this workspace. It
was stopped after current-head clean-checkout CI passed; later local
phases were not run. This local run is not claimed as passing. Focused
packaging tests, workspace typecheck/build and all hosted CI passed.
- [Hosted Docker verification
passed](https://github.com/paperclipai/paperclip/actions/runs/37804902678):
Linux AMD64 and ARM64 image builds, multi-architecture publication and
the process-reaping smoke check. This run tested
`779d94989c54d9abbeba0838194c951186af66a6`; the only later changes are
Daytona cache identity and its regression test. Provider-pack build code
is identical. Local Docker did not respond within the bounded probe.
- Current head `db2c19e270b8d5a7bab5db39d3de0e4761cf57c3`: 54 successful
checks, two conditional skips, no failures and no merge conflicts. Apex
is 5/5 with no unresolved comments.

## Risks

The helper uses each installed package's exported executable. An
installed wrapper with a missing package still fails the build. This
change does not execute Copilot during image construction, alter
dependency pins, change runtime admission or weaken the temporary-path
check.

## Model Used

OpenAI GPT-6 via Codex, with reasoning, repository inspection, tool use
and code execution. The exact serving model identifier and context
window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run focused tests locally and they pass; the full
local-suite limitation is documented above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-08 09:33:24 -07:00

318 lines
11 KiB
TypeScript

import { createHash } from "node:crypto";
import { lstat, readdir, readFile, readlink } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
export const DAYTONA_IMAGE_CONTENT_SCHEMA =
"paperclip-daytona-runner-image-content/v5";
export const DAYTONA_IMAGE_PLATFORM = "linux/amd64";
export const DAYTONA_IMAGE_DOCKERFILE_PATH = "docker/daytona-runner/Dockerfile";
// This mirrors the explicit repository-local build inputs copied by
// docker/daytona-runner/Dockerfile. Broad package-tree COPYs are forbidden by
// the contract test so development-only files cannot silently enter the image
// without first changing this content-identity contract.
export const DAYTONA_IMAGE_INPUT_PATHS = [
".dockerignore",
".npmrc",
"docker/daytona-runner/Dockerfile",
"package.json",
"patches",
"pnpm-lock.yaml",
"pnpm-workspace.yaml",
"scripts/link-plugin-dev-sdk.mjs",
"tsconfig.base.json",
"packages/paperclip-eval-kernel/package.json",
"packages/paperclip-eval-kernel/src",
"packages/paperclip-eval-kernel/tsconfig.json",
"packages/paperclip-runner/scripts/provision-grok.mjs",
"packages/paperclip-runner/package.json",
"packages/paperclip-runner/cursor-distributions.json",
"packages/paperclip-runner/cursor-contract.json",
"packages/paperclip-runner/protocol",
"packages/paperclip-runner/runner/Cargo.lock",
"packages/paperclip-runner/runner/Cargo.toml",
"packages/paperclip-runner/runner/crates",
"packages/paperclip-runner/scripts/acpx-sidecar-contract.mjs",
"packages/paperclip-runner/scripts/build-provider-pack.mjs",
"packages/paperclip-runner/scripts/provider-pack-executable-shims.mjs",
"packages/paperclip-runner/scripts/build-node-startup-timeout.mjs",
"packages/paperclip-runner/scripts/candidate-provider-pack.mjs",
"packages/paperclip-runner/scripts/materialize-cursor-distribution.mjs",
"packages/paperclip-runner/scripts/cursor-runtime-patch.mjs",
"packages/paperclip-runner/scripts/cursor-native-usage.mjs",
"packages/paperclip-runner/scripts/cursor-native-usage-source.json",
"packages/paperclip-runner/scripts/provision-cursor.mjs",
"packages/paperclip-runner/cursor-distributions.json",
"packages/paperclip-runner/cursor-contract.json",
"packages/paperclip-runner/scripts/build-verified-provider-entrypoints.mjs",
"packages/paperclip-runner/scripts/generate-acpx-sidecar-contract.mjs",
"packages/paperclip-runner/scripts/generate-protocol-schema-module.mjs",
// Pi runtime/extension/Node/closure pins are included by the src tree below.
"packages/paperclip-runner/src",
"packages/paperclip-runner/styles.css",
"packages/paperclip-runner/tsconfig.json",
"packages/paperclip-runner/tsconfig.surfaces.json",
] as const;
const ignoredRunnerDevelopmentDirectoryPaths = new Set([
"packages/paperclip-runner/devtools",
"packages/paperclip-runner/docs",
"packages/paperclip-runner/examples",
"packages/paperclip-runner/test",
"packages/paperclip-runner/test-fixtures",
"packages/paperclip-runner/test-support",
]);
const runnerDocumentationFilePattern = /\.md$/;
const runnerTestFilePattern = /\.(?:spec|test)\.(?:[cm]?[jt]sx?)$/;
const runnerRustIntegrationTestPathPattern =
/^packages\/paperclip-runner\/runner\/crates\/[^/]+\/tests(?:\/|$)/;
const runnerSmokeScriptPattern =
/^packages\/paperclip-runner\/scripts\/[^/]+-smoke\.mjs$/;
export interface DaytonaImageContentOptions {
repositoryRoot?: string;
inputPaths?: readonly string[];
platform?: string;
baseImages?: readonly string[];
frontendDigest?: string;
/** Must match the Docker PAPERCLIP_RUNNER_CANDIDATE_PROVIDERS build argument. */
candidateProviders?: readonly string[];
}
export function normalizedDaytonaCandidateProviders(values: readonly string[]): string[] {
const selected = [...values].sort();
if (selected.some(value => !["cursor", "copilot", "pi"].includes(value))
|| new Set(selected).size !== selected.length) {
throw new Error("Daytona candidate providers must be distinct known ACP profiles");
}
return selected;
}
function compareNames(left: string, right: string): number {
return left < right ? -1 : left > right ? 1 : 0;
}
function normalizedRelativePath(value: string): string {
return value.split(path.sep).join("/");
}
function shouldIgnore(relativePath: string): boolean {
return relativePath.split("/").includes("node_modules");
}
function shouldIgnoreRunnerDevelopmentInput(relativePath: string): boolean {
if (!relativePath.startsWith("packages/paperclip-runner/")) return false;
if (ignoredRunnerDevelopmentDirectoryPaths.has(relativePath)) return true;
return (
runnerDocumentationFilePattern.test(relativePath) ||
runnerTestFilePattern.test(relativePath) ||
runnerRustIntegrationTestPathPattern.test(relativePath) ||
runnerSmokeScriptPattern.test(relativePath)
);
}
function updateRecord(
hash: ReturnType<typeof createHash>,
kind: string,
relativePath: string,
payload = "",
): void {
hash.update(kind);
hash.update("\0");
hash.update(relativePath);
hash.update("\0");
hash.update(payload);
hash.update("\0");
}
function assertPinnedBaseImage(reference: string): void {
if (!/@sha256:[0-9a-f]{64}$/.test(reference)) {
throw new Error(
`Daytona image base must use an immutable sha256 digest: ${reference}`,
);
}
}
function assertPinnedFrontendDigest(digest: string): void {
if (!/^sha256:[0-9a-f]{64}$/.test(digest)) {
throw new Error(
`Daytona Dockerfile frontend must use an immutable sha256 digest: ${digest}`,
);
}
}
export function extractDaytonaDockerfileFrontendDigest(
dockerfile: string,
): string {
const firstLine = dockerfile.split(/\r?\n/, 1)[0] ?? "";
const match = /^#\s*syntax=\S+@(sha256:[0-9a-f]{64})\s*$/.exec(firstLine);
if (!match) {
throw new Error(
"Daytona Dockerfile first line must pin its syntax frontend to an immutable sha256 digest",
);
}
const digest = match[1]!;
assertPinnedFrontendDigest(digest);
return digest;
}
export function extractDaytonaBaseImages(dockerfile: string): string[] {
const baseImages: string[] = [];
const stageAliases = new Set<string>();
for (const line of dockerfile.split(/\r?\n/)) {
if (!/^\s*FROM\s/i.test(line)) continue;
const match = /^\s*FROM(?:\s+--\S+)*\s+(\S+)(?:\s+AS\s+(\S+))?\s*$/i.exec(
line,
);
if (!match) {
throw new Error(`Cannot parse Daytona Dockerfile FROM line: ${line}`);
}
const reference = match[1]!;
if (!stageAliases.has(reference)) {
assertPinnedBaseImage(reference);
baseImages.push(reference);
}
if (match[2]) stageAliases.add(match[2]);
}
if (baseImages.length === 0) {
throw new Error("Daytona Dockerfile does not declare a base image");
}
return baseImages;
}
async function resolveBaseImages(
root: string,
explicitBaseImages?: readonly string[],
): Promise<string[]> {
const baseImages = explicitBaseImages
? [...explicitBaseImages]
: extractDaytonaBaseImages(
await readFile(path.join(root, DAYTONA_IMAGE_DOCKERFILE_PATH), "utf8"),
);
if (baseImages.length === 0) {
throw new Error("Daytona image content identity requires a base image");
}
for (const reference of baseImages) assertPinnedBaseImage(reference);
return baseImages.sort(compareNames);
}
async function resolveFrontendDigest(
root: string,
explicitFrontendDigest?: string,
): Promise<string> {
const digest =
explicitFrontendDigest ??
extractDaytonaDockerfileFrontendDigest(
await readFile(path.join(root, DAYTONA_IMAGE_DOCKERFILE_PATH), "utf8"),
);
assertPinnedFrontendDigest(digest);
return digest;
}
async function hashEntry(
hash: ReturnType<typeof createHash>,
root: string,
relativePath: string,
): Promise<void> {
const normalizedPath = normalizedRelativePath(relativePath);
if (
shouldIgnore(normalizedPath) ||
shouldIgnoreRunnerDevelopmentInput(normalizedPath)
) {
return;
}
const absolutePath = path.resolve(root, relativePath);
const relativeFromRoot = path.relative(root, absolutePath);
if (
relativeFromRoot.startsWith(`..${path.sep}`) ||
relativeFromRoot === ".." ||
path.isAbsolute(relativeFromRoot)
) {
throw new Error(
`Daytona image input escapes repository root: ${relativePath}`,
);
}
const stats = await lstat(absolutePath);
if (stats.isDirectory()) {
const entries = await readdir(absolutePath, { withFileTypes: true });
entries.sort((left, right) => compareNames(left.name, right.name));
for (const entry of entries) {
await hashEntry(hash, root, path.join(relativePath, entry.name));
}
return;
}
if (stats.isSymbolicLink()) {
updateRecord(hash, "symlink", normalizedPath, await readlink(absolutePath));
return;
}
if (stats.isFile()) {
const executable =
(stats.mode & 0o111) === 0 ? "non-executable" : "executable";
updateRecord(hash, "file", normalizedPath, executable);
hash.update(await readFile(absolutePath));
hash.update("\0");
return;
}
throw new Error(`Unsupported Daytona image input type: ${relativePath}`);
}
export async function computeDaytonaImageContentId(
options: DaytonaImageContentOptions = {},
): Promise<string> {
const root = path.resolve(options.repositoryRoot ?? repositoryRoot);
const inputPaths = [
...(options.inputPaths ?? DAYTONA_IMAGE_INPUT_PATHS),
].sort(compareNames);
const hash = createHash("sha256");
updateRecord(
hash,
"contract",
DAYTONA_IMAGE_CONTENT_SCHEMA,
options.platform ?? DAYTONA_IMAGE_PLATFORM,
);
updateRecord(
hash,
"dockerfile-frontend",
await resolveFrontendDigest(root, options.frontendDigest),
);
for (const baseImage of await resolveBaseImages(root, options.baseImages)) {
updateRecord(hash, "base-image", baseImage);
}
for (const candidate of normalizedDaytonaCandidateProviders(options.candidateProviders ?? [])) {
updateRecord(hash, "candidate-provider", candidate);
}
for (const inputPath of inputPaths) {
await hashEntry(hash, root, inputPath);
}
return hash.digest("hex");
}
const invokedPath = process.argv[1] ? path.resolve(process.argv[1]) : null;
if (invokedPath && import.meta.url === pathToFileURL(invokedPath).href) {
const arguments_ = process.argv.slice(2);
const candidateFlag = arguments_[0];
if (arguments_.length > 1 || (candidateFlag !== undefined && !candidateFlag.startsWith("--candidate-providers="))) {
throw new Error("Expected only --candidate-providers=cursor,copilot,pi");
}
const candidateProviders = candidateFlag?.slice("--candidate-providers=".length).split(",").filter(Boolean) ?? [];
computeDaytonaImageContentId({ candidateProviders })
.then((contentId) => process.stdout.write(`${contentId}\n`))
.catch((error: unknown) => {
process.stderr.write(
`${error instanceof Error ? error.message : String(error)}\n`,
);
process.exitCode = 1;
});
}
export const DAYTONA_IMAGE_CONTENT_SCRIPT = fileURLToPath(import.meta.url);