mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-19 01:46:15 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Container images include a movable provider pack for agent execution. > - pnpm executable wrappers can contain the temporary build directory. > - New optional Copilot packages add wrappers that the build does not replace. > - This pull request gives those installed wrappers relative executable paths. > - Image publication can finish while the existing path check remains enforced. ## Linked Issues or Issue Description Refs #15572 and #15560. The small portability loop comes from cryppadotta's larger Copilot runtime PR #15560. This separate fix repairs image publication without waiting for that feature's qualification and runtime changes. That PR can remove its duplicate loop after this lands. **What happened?** The standard Docker build stops with `Provider pack shim copilot-linux-x64 retains its temporary build path`. The failure occurred before and after #15522. See [the failed master build](https://github.com/paperclipai/paperclip/actions/runs/37802065316). **Expected behavior** Installed native Copilot wrappers resolve their pinned executable after the provider pack moves. Optional packages that are absent do not gain a command. The builder still rejects wrappers with temporary paths. **Steps to reproduce** 1. Run the provider-pack build from the affected master revision on Linux x64. 2. Let `pnpm deploy --prod` install the optional Copilot package. 3. The wrapper scan rejects its temporary `NODE_PATH`. **Paperclip version or commit** Master `3367b75ccce34d02f355cda1f1ed3fe0b34cf93d`. **Deployment mode** Docker and provider-pack builds. ## What Changed - Replace installed Copilot platform wrappers with relative native executable launchers. - Move the existing executable wrapper writer into an importable helper. Retain the same launch behavior for Node, Claude and OpenCode. - Test relocation, argument handling, exit status, absent optional packages and missing executable packages. Register the tests in the existing Runner test preparation command. - Hash the helper in Daytona image identity and prove that helper changes invalidate the image cache. - Document the packaging rule. Keep dependencies, provider qualification and the final temporary-path check unchanged. ## Verification - 17 Node packaging tests passed across the new wrapper tests, provider-pack release tests and candidate selection tests. - Six existing bundled remote-provider-pack tests passed using the Runner Vitest configuration. - Nine Daytona image identity tests and Runner E2E typecheck passed after the cache-input correction. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Real `pnpm deploy --prod` reproduction on macOS ARM64: the original Copilot wrapper contained the temporary path. After the rewrite and directory relocation, the actual executable returned Copilot CLI 1.0.88 with exit code zero using only `/usr/bin:/bin` in `PATH`. - Syntax checks, `git diff --check` and the pre-push secret scan passed. - The full local `pnpm test:run` reproduced the same five skill/connector fixture failures observed earlier in this workspace. It was stopped after current-head clean-checkout CI passed; later local phases were not run. This local run is not claimed as passing. Focused packaging tests, workspace typecheck/build and all hosted CI passed. - [Hosted Docker verification passed](https://github.com/paperclipai/paperclip/actions/runs/37804902678): Linux AMD64 and ARM64 image builds, multi-architecture publication and the process-reaping smoke check. This run tested `779d94989c54d9abbeba0838194c951186af66a6`; the only later changes are Daytona cache identity and its regression test. Provider-pack build code is identical. Local Docker did not respond within the bounded probe. - Current head `db2c19e270b8d5a7bab5db39d3de0e4761cf57c3`: 54 successful checks, two conditional skips, no failures and no merge conflicts. Apex is 5/5 with no unresolved comments. ## Risks The helper uses each installed package's exported executable. An installed wrapper with a missing package still fails the build. This change does not execute Copilot during image construction, alter dependency pins, change runtime admission or weaken the temporary-path check. ## Model Used OpenAI GPT-6 via Codex, with reasoning, repository inspection, tool use and code execution. The exact serving model identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass; the full local-suite limitation is documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
318 lines
11 KiB
TypeScript
318 lines
11 KiB
TypeScript
import { createHash } from "node:crypto";
|
|
import { lstat, readdir, readFile, readlink } from "node:fs/promises";
|
|
import path from "node:path";
|
|
import { fileURLToPath, pathToFileURL } from "node:url";
|
|
|
|
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
|
|
|
|
export const DAYTONA_IMAGE_CONTENT_SCHEMA =
|
|
"paperclip-daytona-runner-image-content/v5";
|
|
export const DAYTONA_IMAGE_PLATFORM = "linux/amd64";
|
|
export const DAYTONA_IMAGE_DOCKERFILE_PATH = "docker/daytona-runner/Dockerfile";
|
|
|
|
// This mirrors the explicit repository-local build inputs copied by
|
|
// docker/daytona-runner/Dockerfile. Broad package-tree COPYs are forbidden by
|
|
// the contract test so development-only files cannot silently enter the image
|
|
// without first changing this content-identity contract.
|
|
export const DAYTONA_IMAGE_INPUT_PATHS = [
|
|
".dockerignore",
|
|
".npmrc",
|
|
"docker/daytona-runner/Dockerfile",
|
|
"package.json",
|
|
"patches",
|
|
"pnpm-lock.yaml",
|
|
"pnpm-workspace.yaml",
|
|
"scripts/link-plugin-dev-sdk.mjs",
|
|
"tsconfig.base.json",
|
|
"packages/paperclip-eval-kernel/package.json",
|
|
"packages/paperclip-eval-kernel/src",
|
|
"packages/paperclip-eval-kernel/tsconfig.json",
|
|
"packages/paperclip-runner/scripts/provision-grok.mjs",
|
|
"packages/paperclip-runner/package.json",
|
|
"packages/paperclip-runner/cursor-distributions.json",
|
|
"packages/paperclip-runner/cursor-contract.json",
|
|
"packages/paperclip-runner/protocol",
|
|
"packages/paperclip-runner/runner/Cargo.lock",
|
|
"packages/paperclip-runner/runner/Cargo.toml",
|
|
"packages/paperclip-runner/runner/crates",
|
|
"packages/paperclip-runner/scripts/acpx-sidecar-contract.mjs",
|
|
"packages/paperclip-runner/scripts/build-provider-pack.mjs",
|
|
"packages/paperclip-runner/scripts/provider-pack-executable-shims.mjs",
|
|
"packages/paperclip-runner/scripts/build-node-startup-timeout.mjs",
|
|
"packages/paperclip-runner/scripts/candidate-provider-pack.mjs",
|
|
"packages/paperclip-runner/scripts/materialize-cursor-distribution.mjs",
|
|
"packages/paperclip-runner/scripts/cursor-runtime-patch.mjs",
|
|
"packages/paperclip-runner/scripts/cursor-native-usage.mjs",
|
|
"packages/paperclip-runner/scripts/cursor-native-usage-source.json",
|
|
"packages/paperclip-runner/scripts/provision-cursor.mjs",
|
|
"packages/paperclip-runner/cursor-distributions.json",
|
|
"packages/paperclip-runner/cursor-contract.json",
|
|
"packages/paperclip-runner/scripts/build-verified-provider-entrypoints.mjs",
|
|
"packages/paperclip-runner/scripts/generate-acpx-sidecar-contract.mjs",
|
|
"packages/paperclip-runner/scripts/generate-protocol-schema-module.mjs",
|
|
// Pi runtime/extension/Node/closure pins are included by the src tree below.
|
|
"packages/paperclip-runner/src",
|
|
"packages/paperclip-runner/styles.css",
|
|
"packages/paperclip-runner/tsconfig.json",
|
|
"packages/paperclip-runner/tsconfig.surfaces.json",
|
|
] as const;
|
|
|
|
const ignoredRunnerDevelopmentDirectoryPaths = new Set([
|
|
"packages/paperclip-runner/devtools",
|
|
"packages/paperclip-runner/docs",
|
|
"packages/paperclip-runner/examples",
|
|
"packages/paperclip-runner/test",
|
|
"packages/paperclip-runner/test-fixtures",
|
|
"packages/paperclip-runner/test-support",
|
|
]);
|
|
|
|
const runnerDocumentationFilePattern = /\.md$/;
|
|
const runnerTestFilePattern = /\.(?:spec|test)\.(?:[cm]?[jt]sx?)$/;
|
|
const runnerRustIntegrationTestPathPattern =
|
|
/^packages\/paperclip-runner\/runner\/crates\/[^/]+\/tests(?:\/|$)/;
|
|
const runnerSmokeScriptPattern =
|
|
/^packages\/paperclip-runner\/scripts\/[^/]+-smoke\.mjs$/;
|
|
|
|
export interface DaytonaImageContentOptions {
|
|
repositoryRoot?: string;
|
|
inputPaths?: readonly string[];
|
|
platform?: string;
|
|
baseImages?: readonly string[];
|
|
frontendDigest?: string;
|
|
/** Must match the Docker PAPERCLIP_RUNNER_CANDIDATE_PROVIDERS build argument. */
|
|
candidateProviders?: readonly string[];
|
|
}
|
|
|
|
export function normalizedDaytonaCandidateProviders(values: readonly string[]): string[] {
|
|
const selected = [...values].sort();
|
|
if (selected.some(value => !["cursor", "copilot", "pi"].includes(value))
|
|
|| new Set(selected).size !== selected.length) {
|
|
throw new Error("Daytona candidate providers must be distinct known ACP profiles");
|
|
}
|
|
return selected;
|
|
}
|
|
|
|
function compareNames(left: string, right: string): number {
|
|
return left < right ? -1 : left > right ? 1 : 0;
|
|
}
|
|
|
|
function normalizedRelativePath(value: string): string {
|
|
return value.split(path.sep).join("/");
|
|
}
|
|
|
|
function shouldIgnore(relativePath: string): boolean {
|
|
return relativePath.split("/").includes("node_modules");
|
|
}
|
|
|
|
function shouldIgnoreRunnerDevelopmentInput(relativePath: string): boolean {
|
|
if (!relativePath.startsWith("packages/paperclip-runner/")) return false;
|
|
if (ignoredRunnerDevelopmentDirectoryPaths.has(relativePath)) return true;
|
|
return (
|
|
runnerDocumentationFilePattern.test(relativePath) ||
|
|
runnerTestFilePattern.test(relativePath) ||
|
|
runnerRustIntegrationTestPathPattern.test(relativePath) ||
|
|
runnerSmokeScriptPattern.test(relativePath)
|
|
);
|
|
}
|
|
|
|
function updateRecord(
|
|
hash: ReturnType<typeof createHash>,
|
|
kind: string,
|
|
relativePath: string,
|
|
payload = "",
|
|
): void {
|
|
hash.update(kind);
|
|
hash.update("\0");
|
|
hash.update(relativePath);
|
|
hash.update("\0");
|
|
hash.update(payload);
|
|
hash.update("\0");
|
|
}
|
|
|
|
function assertPinnedBaseImage(reference: string): void {
|
|
if (!/@sha256:[0-9a-f]{64}$/.test(reference)) {
|
|
throw new Error(
|
|
`Daytona image base must use an immutable sha256 digest: ${reference}`,
|
|
);
|
|
}
|
|
}
|
|
|
|
function assertPinnedFrontendDigest(digest: string): void {
|
|
if (!/^sha256:[0-9a-f]{64}$/.test(digest)) {
|
|
throw new Error(
|
|
`Daytona Dockerfile frontend must use an immutable sha256 digest: ${digest}`,
|
|
);
|
|
}
|
|
}
|
|
|
|
export function extractDaytonaDockerfileFrontendDigest(
|
|
dockerfile: string,
|
|
): string {
|
|
const firstLine = dockerfile.split(/\r?\n/, 1)[0] ?? "";
|
|
const match = /^#\s*syntax=\S+@(sha256:[0-9a-f]{64})\s*$/.exec(firstLine);
|
|
if (!match) {
|
|
throw new Error(
|
|
"Daytona Dockerfile first line must pin its syntax frontend to an immutable sha256 digest",
|
|
);
|
|
}
|
|
const digest = match[1]!;
|
|
assertPinnedFrontendDigest(digest);
|
|
return digest;
|
|
}
|
|
|
|
export function extractDaytonaBaseImages(dockerfile: string): string[] {
|
|
const baseImages: string[] = [];
|
|
const stageAliases = new Set<string>();
|
|
|
|
for (const line of dockerfile.split(/\r?\n/)) {
|
|
if (!/^\s*FROM\s/i.test(line)) continue;
|
|
const match = /^\s*FROM(?:\s+--\S+)*\s+(\S+)(?:\s+AS\s+(\S+))?\s*$/i.exec(
|
|
line,
|
|
);
|
|
if (!match) {
|
|
throw new Error(`Cannot parse Daytona Dockerfile FROM line: ${line}`);
|
|
}
|
|
|
|
const reference = match[1]!;
|
|
if (!stageAliases.has(reference)) {
|
|
assertPinnedBaseImage(reference);
|
|
baseImages.push(reference);
|
|
}
|
|
if (match[2]) stageAliases.add(match[2]);
|
|
}
|
|
|
|
if (baseImages.length === 0) {
|
|
throw new Error("Daytona Dockerfile does not declare a base image");
|
|
}
|
|
return baseImages;
|
|
}
|
|
|
|
async function resolveBaseImages(
|
|
root: string,
|
|
explicitBaseImages?: readonly string[],
|
|
): Promise<string[]> {
|
|
const baseImages = explicitBaseImages
|
|
? [...explicitBaseImages]
|
|
: extractDaytonaBaseImages(
|
|
await readFile(path.join(root, DAYTONA_IMAGE_DOCKERFILE_PATH), "utf8"),
|
|
);
|
|
if (baseImages.length === 0) {
|
|
throw new Error("Daytona image content identity requires a base image");
|
|
}
|
|
for (const reference of baseImages) assertPinnedBaseImage(reference);
|
|
return baseImages.sort(compareNames);
|
|
}
|
|
|
|
async function resolveFrontendDigest(
|
|
root: string,
|
|
explicitFrontendDigest?: string,
|
|
): Promise<string> {
|
|
const digest =
|
|
explicitFrontendDigest ??
|
|
extractDaytonaDockerfileFrontendDigest(
|
|
await readFile(path.join(root, DAYTONA_IMAGE_DOCKERFILE_PATH), "utf8"),
|
|
);
|
|
assertPinnedFrontendDigest(digest);
|
|
return digest;
|
|
}
|
|
|
|
async function hashEntry(
|
|
hash: ReturnType<typeof createHash>,
|
|
root: string,
|
|
relativePath: string,
|
|
): Promise<void> {
|
|
const normalizedPath = normalizedRelativePath(relativePath);
|
|
if (
|
|
shouldIgnore(normalizedPath) ||
|
|
shouldIgnoreRunnerDevelopmentInput(normalizedPath)
|
|
) {
|
|
return;
|
|
}
|
|
|
|
const absolutePath = path.resolve(root, relativePath);
|
|
const relativeFromRoot = path.relative(root, absolutePath);
|
|
if (
|
|
relativeFromRoot.startsWith(`..${path.sep}`) ||
|
|
relativeFromRoot === ".." ||
|
|
path.isAbsolute(relativeFromRoot)
|
|
) {
|
|
throw new Error(
|
|
`Daytona image input escapes repository root: ${relativePath}`,
|
|
);
|
|
}
|
|
|
|
const stats = await lstat(absolutePath);
|
|
if (stats.isDirectory()) {
|
|
const entries = await readdir(absolutePath, { withFileTypes: true });
|
|
entries.sort((left, right) => compareNames(left.name, right.name));
|
|
for (const entry of entries) {
|
|
await hashEntry(hash, root, path.join(relativePath, entry.name));
|
|
}
|
|
return;
|
|
}
|
|
if (stats.isSymbolicLink()) {
|
|
updateRecord(hash, "symlink", normalizedPath, await readlink(absolutePath));
|
|
return;
|
|
}
|
|
if (stats.isFile()) {
|
|
const executable =
|
|
(stats.mode & 0o111) === 0 ? "non-executable" : "executable";
|
|
updateRecord(hash, "file", normalizedPath, executable);
|
|
hash.update(await readFile(absolutePath));
|
|
hash.update("\0");
|
|
return;
|
|
}
|
|
throw new Error(`Unsupported Daytona image input type: ${relativePath}`);
|
|
}
|
|
|
|
export async function computeDaytonaImageContentId(
|
|
options: DaytonaImageContentOptions = {},
|
|
): Promise<string> {
|
|
const root = path.resolve(options.repositoryRoot ?? repositoryRoot);
|
|
const inputPaths = [
|
|
...(options.inputPaths ?? DAYTONA_IMAGE_INPUT_PATHS),
|
|
].sort(compareNames);
|
|
const hash = createHash("sha256");
|
|
updateRecord(
|
|
hash,
|
|
"contract",
|
|
DAYTONA_IMAGE_CONTENT_SCHEMA,
|
|
options.platform ?? DAYTONA_IMAGE_PLATFORM,
|
|
);
|
|
updateRecord(
|
|
hash,
|
|
"dockerfile-frontend",
|
|
await resolveFrontendDigest(root, options.frontendDigest),
|
|
);
|
|
for (const baseImage of await resolveBaseImages(root, options.baseImages)) {
|
|
updateRecord(hash, "base-image", baseImage);
|
|
}
|
|
for (const candidate of normalizedDaytonaCandidateProviders(options.candidateProviders ?? [])) {
|
|
updateRecord(hash, "candidate-provider", candidate);
|
|
}
|
|
for (const inputPath of inputPaths) {
|
|
await hashEntry(hash, root, inputPath);
|
|
}
|
|
return hash.digest("hex");
|
|
}
|
|
|
|
const invokedPath = process.argv[1] ? path.resolve(process.argv[1]) : null;
|
|
if (invokedPath && import.meta.url === pathToFileURL(invokedPath).href) {
|
|
const arguments_ = process.argv.slice(2);
|
|
const candidateFlag = arguments_[0];
|
|
if (arguments_.length > 1 || (candidateFlag !== undefined && !candidateFlag.startsWith("--candidate-providers="))) {
|
|
throw new Error("Expected only --candidate-providers=cursor,copilot,pi");
|
|
}
|
|
const candidateProviders = candidateFlag?.slice("--candidate-providers=".length).split(",").filter(Boolean) ?? [];
|
|
computeDaytonaImageContentId({ candidateProviders })
|
|
.then((contentId) => process.stdout.write(`${contentId}\n`))
|
|
.catch((error: unknown) => {
|
|
process.stderr.write(
|
|
`${error instanceof Error ? error.message : String(error)}\n`,
|
|
);
|
|
process.exitCode = 1;
|
|
});
|
|
}
|
|
|
|
export const DAYTONA_IMAGE_CONTENT_SCRIPT = fileURLToPath(import.meta.url);
|