mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every master push runs a canary, and only one canary runs at a time, so its duration sets how often merged work ships > - `publish_canary` takes about 30 min. 8 min of it is `pnpm build`, and 4 min 40 s of that is an uncached `cargo build --release` of `paperclip-runnerd` > - The release then replaces that binary with the exact-source daemons that `release_runner_binaries` already built and verified for all four targets > - This pull request stages the verified daemon for the publisher's own target during a release, instead of compiling it > - The benefit is about 4.5 min less per canary, with the same verified bytes published ## Linked Issues or Issue Description **What happened?** In canary run 43487884481 (2026-11-13), the `Publish canary` step took 28.6 min. Its `pnpm build` ran 19:42-19:50. The `packages/paperclip-runner build` took 5.3 min, and its `cargo build` reported `Finished release profile ... in 4m 40s`. Every other package waited for it. `release.sh` then runs `assemble-daemons`, which overwrites `dist/bin` with the four exact-source release daemons. **Expected behavior** A release does not compile a binary that it discards. The Runner build stamp stays valid, so the server build still reuses the Runner build. Refs #21554 (the other canary speed-up) ## What Changed - `packages/paperclip-runner/scripts/build-runner-binary.mjs` (new): when `PAPERCLIP_RELEASE_RUNNER_ARTIFACTS_DIR` is set, it checks the release daemons with the existing `releaseDaemonManifest` (source revision equal to `HEAD`, digest, architecture, build metadata) and stages the host target's daemon at `dist/bin` with the existing `stageRunnerBinary`. Otherwise it runs `pnpm run build:binary` as before. - The Runner `build` chain calls this script in place of `pnpm run build:binary`. `build:binary` itself and the image build are unchanged. - `stage-release-runner-binaries.mjs` exports `metadataContract`. - Tests: `stage-runner-binary.test.mjs` (staging per target, wrong source, bad digest, missing target, non-release fallback); `runner-image-identity.test.ts` step list. - `doc/RELEASING.md` describes the behavior. ## Verification - `node --test packages/paperclip-runner/scripts/stage-runner-binary.test.mjs`: 9 of 9 pass. - `server`: `vitest run src/__tests__/runner-image-identity.test.ts`: 42 of 42 pass. - With the real daemon artifacts of canary run 43487884481 (receipts pointed at this commit): `PAPERCLIP_RELEASE_RUNNER_ARTIFACTS_DIR=... pnpm run build` in `packages/paperclip-runner` finished in 15 s with no `cargo` line. The build stamp was recorded, and `server/scripts/prepare-runner-vendor.mjs` then reported `action:reuse,reason:inputs_unchanged`. The staged daemon runs. - After merge: the next canary's `packages/paperclip-runner build` log shows "Staged the verified exact-source linux-x64 release daemon" and no `Compiling` lines. ## Risks - Low. A release still fails closed: a daemon with the wrong source, digest, architecture or metadata stops the build, and `assemble-daemons` checks all four again afterwards. - Local, Docker and CI builds without the variable are unchanged. - On macOS, staging re-signs the binary, as it does for a compiled one. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
53 lines
2.7 KiB
JavaScript
53 lines
2.7 KiB
JavaScript
import { execFileSync, spawnSync } from "node:child_process";
|
|
import path from "node:path";
|
|
import { fileURLToPath } from "node:url";
|
|
import { stageRunnerBinary } from "./stage-runner-binary.mjs";
|
|
|
|
export const RELEASE_ARTIFACTS_VARIABLE = "PAPERCLIP_RELEASE_RUNNER_ARTIFACTS_DIR";
|
|
|
|
export function hostRunnerTarget(platform = process.platform, arch = process.arch) {
|
|
return `${platform}-${arch}`;
|
|
}
|
|
|
|
export async function stageReleaseHostBinary({ artifactDirectory, sourceRevision, packageRoot, verifyMetadata, target = hostRunnerTarget(), stage = stageRunnerBinary }) {
|
|
const { releaseDaemonManifest } = await import("./stage-release-runner-binaries.mjs");
|
|
const manifest = releaseDaemonManifest(artifactDirectory, sourceRevision, verifyMetadata);
|
|
const artifact = manifest.platforms[target];
|
|
if (!artifact) throw new Error(`No exact-source release daemon for ${target}`);
|
|
const executable = target.startsWith("win32-") ? "paperclip-runnerd.exe" : "paperclip-runnerd";
|
|
await stage(artifact.path, path.join(packageRoot, "dist", "bin", executable));
|
|
return { target, sha256: artifact.sha256 };
|
|
}
|
|
|
|
function pnpmInvocation(env) {
|
|
const npmExecPath = env.npm_execpath?.trim();
|
|
if (npmExecPath && npmExecPath.toLowerCase().includes("pnpm")) {
|
|
return /\.(c|m)?js$/i.test(npmExecPath) ? { command: process.execPath, args: [npmExecPath] } : { command: npmExecPath, args: [] };
|
|
}
|
|
return { command: "pnpm", args: [] };
|
|
}
|
|
|
|
export async function main({ env = process.env, packageRoot, log = console.log } = {}) {
|
|
const root = packageRoot ?? path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
|
|
const artifactDirectory = env[RELEASE_ARTIFACTS_VARIABLE]?.trim();
|
|
if (artifactDirectory) {
|
|
const { metadataContract } = await import("./stage-release-runner-binaries.mjs");
|
|
const sourceRevision = execFileSync("git", ["rev-parse", "HEAD"], { cwd: root, encoding: "utf8" }).trim();
|
|
const staged = await stageReleaseHostBinary({ artifactDirectory, sourceRevision, packageRoot: root, verifyMetadata: await metadataContract() });
|
|
log(`Staged the verified exact-source ${staged.target} release daemon (${staged.sha256}) instead of compiling it.`);
|
|
return { reused: true, ...staged };
|
|
}
|
|
const { command, args } = pnpmInvocation(env);
|
|
const result = spawnSync(command, [...args, "run", "build:binary"], { cwd: root, env, stdio: "inherit", shell: process.platform === "win32" && command === "pnpm" });
|
|
if (result.error) throw result.error;
|
|
if (result.status !== 0) throw new Error(`build:binary exited with ${result.status ?? result.signal}`);
|
|
return { reused: false };
|
|
}
|
|
|
|
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
|
main().catch((error) => {
|
|
console.error(error.message);
|
|
process.exitCode = 1;
|
|
});
|
|
}
|