mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-28 04:02:18 +01:00
test(runner): qualify the OpenCode discovery fixture on Linux (#17757)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Runner tests must exercise the same verified launch boundary as real providers. > - The assigned-tool test from #17740 used an executable JavaScript script as its OpenCode fixture. > - Linux launches verified executable bytes from a sealed, unlinked memory file. > - Node cannot reopen that deleted script path. Cleanup then hid the startup error. > - This pull request uses the native fixture wrapper already used by the adjacent OpenCode integration test. ## Linked Issues or Issue Description Refs #17740. Related fixture qualification work: #14529. **What happened?** The merged-source Cloud readiness job failed twice in the new assigned-tool integration test. Linux could not start the script fixture from the verified executable file. The later cleanup error hid the original `ENOENT` error for `/memfd:paperclip-verified-launch (deleted)`. **Expected behavior** The test must pass through the real Rust Runner, OpenCode proxy, and HTTP MCP bridge on Linux. It must also report the original error if cleanup fails. **Steps to reproduce** Run `vitest run src/live/runnerd-opencode-assigned-tools.test.ts` in the Runner package with its Linux binary and built proxy. The original fixture fails during startup. The native wrapper passes the assigned-tool assertions and durable close. ## What Changed - Launch the existing JavaScript server fixture through a native wrapper. - Qualify an owned Node copy on Linux with strict file permissions. - Bundle the current production proxy into the test's temporary directory. The test no longer skips when shared `dist` output is absent. - Preserve both the original error and a cleanup error. - Remove the temporary fixture even when setup fails. ## Verification - Reproduced the failure in a local Linux container with the exact merged Runner binary from the published Core image. - The same Linux test passes after this change. It checks the assigned discovery call, the completion call, and successful durable close. - The updated test passes on macOS. - A real Linux compiler-failure probe leaves zero fixture directories behind. - `pnpm -r typecheck` and `pnpm build`: passed. - Runner TypeScript checks pass after the cleanup review change. - Full technical CI passed on `6ee1865af3310f13523b11407f9e41879832f543`. - Fresh Greptile review: 5/5, with no unresolved findings. - Fresh Paperclip Review published 5/5 on `6ee1865af3310f13523b11407f9e41879832f543`, with no actionable findings. - The optional Entire assessment remains pending because this branch has no assessed Entire Change. The maintainer approved merging after CI and both reviews pass. - The full local test suite was not repeated. Earlier attempts hit host PostgreSQL semaphore limits. CI provides the full suite check. ## Risks Low risk. This changes one test fixture. Runtime behavior and launch verification stay the same. The test now needs the C compiler already required by the adjacent native fixture tests. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The runtime does not expose the exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
8c702d59db
commit
f1922d95e1
1 file changed
+77
-42
@@ -1,6 +1,7 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { chmod, copyFile, mkdir, mkdtemp, rm } from "node:fs/promises";
|
||||
import { chmod, cp, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { expect, it } from "vitest";
|
||||
@@ -8,56 +9,90 @@ import { codexSemanticToolSpecs, dynamicToolResponse } from "../drivers/codex/co
|
||||
import { createRunnerdCodexTransport, defaultCapabilityRunnerdBinary } from "./runnerd-codex-transport.js";
|
||||
|
||||
const runner = defaultCapabilityRunnerdBinary();
|
||||
const proxy = resolve("dist/cli/opencode-app-server-proxy.cjs");
|
||||
const digest = (path: string) => `sha256:${createHash("sha256").update(readFileSync(path)).digest("hex")}`;
|
||||
|
||||
it.skipIf(process.platform === "win32" || !existsSync(runner) || !existsSync(proxy))(
|
||||
it.skipIf(process.platform === "win32" || !existsSync(runner))(
|
||||
"retains assigned discovery authority across Rust, the proxy, and HTTP MCP",
|
||||
async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), "runnerd-opencode-assigned-"));
|
||||
const workspace = join(root, "workspace");
|
||||
const executable = join(root, "fake-opencode-server.mjs");
|
||||
await mkdir(workspace);
|
||||
await copyFile(resolve("test/fixtures/fake-opencode-server.mjs"), executable);
|
||||
await chmod(executable, 0o755);
|
||||
const operation = "paperclip_search_assigned_tools";
|
||||
const seen: string[] = [];
|
||||
const { transport } = createRunnerdCodexTransport({
|
||||
provider: "opencode", runnerBinary: runner, stateDirectory: join(root, "state"), sourceCodexHome: "",
|
||||
providerNodeCommand: process.execPath, providerNodeCommandSha256: digest(process.execPath),
|
||||
opencodeCommand: executable, opencodeCommandSha256: digest(executable),
|
||||
opencodeProxyPath: proxy, opencodeProxySha256: digest(proxy),
|
||||
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "synthetic-fixture-key" },
|
||||
});
|
||||
transport.setServerRequestHandler(async request => {
|
||||
expect(request.method).toBe("item/tool/call");
|
||||
const tool = request.params.tool;
|
||||
expect([operation, "paperclip_finish"]).toContain(tool);
|
||||
seen.push(String(tool));
|
||||
return dynamicToolResponse(tool === operation ? { tools: [] } : "Accepted completion.");
|
||||
});
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
try {
|
||||
await transport.request("thread/start", {
|
||||
cwd: workspace, model: "openrouter/deepseek/deepseek-v4.1-flash-1017",
|
||||
completionContract: { revision: "assigned-tools-v1", criterionIds: ["objective"] },
|
||||
dynamicTools: [{ name: operation, description: "Search assigned synthetic tools.", inputSchema: { type: "object" } }, ...codexSemanticToolSpecs()],
|
||||
const workspace = join(root, "workspace");
|
||||
const executable = join(root, "fake-opencode");
|
||||
await mkdir(workspace);
|
||||
// Linux launches verified executables from an unlinked, sealed memfd. Use
|
||||
// a native wrapper like the real OpenCode binary: Node cannot reopen a
|
||||
// shebang script through that deleted executable path.
|
||||
const providerNode = process.platform === "linux" ? join(root, "node") : process.execPath;
|
||||
if (process.platform === "linux") {
|
||||
await cp(process.execPath, providerNode, { dereference: true });
|
||||
await chmod(providerNode, 0o500);
|
||||
}
|
||||
const fixture = resolve("test/fixtures/fake-opencode-server.mjs");
|
||||
execFileSync("cc", ["-x", "c", "-o", executable, "-"], {
|
||||
input: `#include <unistd.h>\n#include <stdlib.h>\nint main(int argc, char **argv) { char **args = calloc(argc + 2, sizeof(char *)); args[0] = ${JSON.stringify(providerNode)}; args[1] = ${JSON.stringify(fixture)}; for (int i = 1; i < argc; i++) args[i + 1] = argv[i]; execv(args[0], args); return 127; }`,
|
||||
});
|
||||
const completed = (async () => {
|
||||
for await (const notification of transport.notifications()) {
|
||||
if (notification.method === "turn/completed") return;
|
||||
if (notification.method === "turn/failed") throw new Error("Native OpenCode turn failed");
|
||||
await chmod(executable, 0o755);
|
||||
// Exercise the current production bundle even in a Rust-only test lane;
|
||||
// neither depend on nor overwrite another test's shared dist artifacts.
|
||||
const proxy = join(root, "opencode-app-server-proxy.cjs");
|
||||
const proxyBytes = execFileSync(process.execPath, ["--input-type=module", "-e", `
|
||||
import { bundleVerifiedProviderEntrypoints } from "./scripts/build-verified-provider-entrypoints.mjs";
|
||||
const entries = await bundleVerifiedProviderEntrypoints({ write: false });
|
||||
const proxy = entries.find(({ entrypoint }) => entrypoint.name === "opencode-app-server-proxy");
|
||||
process.stdout.write(proxy.verifiedResult.outputFiles[0].contents);
|
||||
`], { maxBuffer: 16 * 1024 * 1024 });
|
||||
await writeFile(proxy, proxyBytes, { mode: 0o755 });
|
||||
const operation = "paperclip_search_assigned_tools";
|
||||
const seen: string[] = [];
|
||||
const { transport } = createRunnerdCodexTransport({
|
||||
provider: "opencode", runnerBinary: runner, stateDirectory: join(root, "state"), sourceCodexHome: "",
|
||||
providerNodeCommand: providerNode, providerNodeCommandSha256: digest(providerNode),
|
||||
opencodeCommand: executable, opencodeCommandSha256: digest(executable),
|
||||
opencodeProxyPath: proxy, opencodeProxySha256: digest(proxy),
|
||||
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "synthetic-fixture-key" },
|
||||
});
|
||||
transport.setServerRequestHandler(async request => {
|
||||
expect(request.method).toBe("item/tool/call");
|
||||
const tool = request.params.tool;
|
||||
expect([operation, "paperclip_finish"]).toContain(tool);
|
||||
seen.push(String(tool));
|
||||
return dynamicToolResponse(tool === operation ? { tools: [] } : "Accepted completion.");
|
||||
});
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
let primaryError: unknown;
|
||||
try {
|
||||
await transport.request("thread/start", {
|
||||
cwd: workspace, model: "openrouter/deepseek/deepseek-v4.1-flash-1017",
|
||||
completionContract: { revision: "assigned-tools-v1", criterionIds: ["objective"] },
|
||||
dynamicTools: [{ name: operation, description: "Search assigned synthetic tools.", inputSchema: { type: "object" } }, ...codexSemanticToolSpecs()],
|
||||
});
|
||||
const completed = (async () => {
|
||||
for await (const notification of transport.notifications()) {
|
||||
if (notification.method === "turn/completed") return;
|
||||
if (notification.method === "turn/failed") throw new Error("Native OpenCode turn failed");
|
||||
}
|
||||
throw new Error("Native OpenCode notifications ended before completion");
|
||||
})();
|
||||
await transport.request("turn/start", { input: [{ type: "text", text: "Discover assigned synthetic tools." }] });
|
||||
await Promise.race([completed, new Promise<never>((_, reject) => {
|
||||
timer = setTimeout(() => reject(new Error("Native assigned-tool smoke timed out")), 20_000);
|
||||
})]);
|
||||
expect(seen).toEqual([operation, "paperclip_finish"]);
|
||||
} catch (error) {
|
||||
primaryError = error;
|
||||
throw error;
|
||||
} finally {
|
||||
if (timer) clearTimeout(timer);
|
||||
try {
|
||||
await transport.close();
|
||||
} catch (cleanupError) {
|
||||
if (primaryError) {
|
||||
throw new AggregateError([primaryError, cleanupError], "Native OpenCode assigned-tool probe and cleanup failed", { cause: primaryError });
|
||||
}
|
||||
throw cleanupError;
|
||||
}
|
||||
throw new Error("Native OpenCode notifications ended before completion");
|
||||
})();
|
||||
await transport.request("turn/start", { input: [{ type: "text", text: "Discover assigned synthetic tools." }] });
|
||||
await Promise.race([completed, new Promise<never>((_, reject) => {
|
||||
timer = setTimeout(() => reject(new Error("Native assigned-tool smoke timed out")), 20_000);
|
||||
})]);
|
||||
expect(seen).toEqual([operation, "paperclip_finish"]);
|
||||
}
|
||||
} finally {
|
||||
if (timer) clearTimeout(timer);
|
||||
await transport.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
},
|
||||
|
||||
Reference in new issue
Block a user