mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-15 18:56:05 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The Paperclip runner drives OpenCode agents. It sends their
OpenRouter and Paperclip-connection model calls through a local provider
proxy.
> - An agent often takes a screenshot of its own work and reads it to
verify the result.
> - OpenCode 1.18.34 cannot put images in OpenAI-compatible tool
results. It moves them into a synthetic user message ("Attached media
from tool result:"). For a text-only model, it replaces the image with
"ERROR: Cannot read image (this model does not support image input).
Inform the user."
> - The model reads that text as a message from the person. In a chat
eval, a DeepSeek worker redid its verification for about 5 minutes,
registered its deliverable twice, and ran out of the 15-minute attempt
clock.
> - This pull request moves that notice into the tool result that
produced the image, worded as a runner note, and removes the synthetic
user message.
> - The benefit is that a text-only agent sees a clear tool outcome and
continues its work. It does not mistake the notice for the person.
## Linked Issues or Issue Description
**What happened?**
An OpenCode agent on `openrouter/deepseek/deepseek-v4.1-flash-1017` read
a PNG screenshot that it made. The next model request contained a
user-role message with the text "Attached media from tool result: ERROR:
Cannot read image (this model does not support image input). Inform the
user." The model reasoned "The previous input from the user was ..." and
repeated finished work.
**Steps to reproduce**
1. Run OpenCode 1.18.34 with an `@ai-sdk/openai-compatible` provider and
a model that has no image input.
2. Have the model call the `read` tool on a PNG file.
3. Inspect the next chat-completions request. The last message is the
synthetic user message above.
**Expected behavior**
The model must get the notice as part of the tool result, not as a
message from the person.
**Environment**
Paperclip runner, OpenCode 1.18.34 (qualified pin), OpenRouter and
Paperclip-connection model routes.
## What Changed
- Add
`packages/paperclip-runner/src/drivers/opencode/tool-media-attribution.ts`.
It finds the OpenCode synthetic attachment message only when it directly
follows tool results. It moves each "Cannot read ..." or "empty or
corrupted" notice into the last tool result of that step as a runner
note. It keeps readable media in the synthetic message. It removes the
message when no media remains.
- The OpenCode provider proxy applies this rewrite before it forwards a
request. It re-serializes the body only when something changed.
- Messages written by a person are never changed. A person who attaches
an image to a text-only model still gets "Inform the user", which is
correct for that case.
- Add unit tests for the rewrite and an integration test through the
real provider proxy.
## Verification
- Red: the new proxy test failed before the change (`expected ... to
have a length of 4 but got 5`).
- Green: `npx vitest run
src/drivers/opencode/opencode-server-driver.test.ts
src/drivers/opencode/tool-media-attribution.test.ts` (82 tests pass).
`tsc --noEmit -p packages/paperclip-runner` is clean.
- Real binary: I ran the real OpenCode 1.18.34 binary with a mock
OpenAI-compatible server and had the model read a PNG. The captured
request has exactly the message shape that the test fixture uses.
- `pnpm verify` returned exit 2 (test files outside the resolved
graphs). Required CI supplies broad coverage.
- A paid live run of
`chat-features.runner-opencode.local.task-coordination.delegate-with-context`
will follow in a PR comment.
## Risks
- Low risk. The rewrite matches the exact OpenCode 1.18.34 strings. If a
future OpenCode version changes them, the rewrite does nothing and the
old behavior returns. It does not fail the request.
- The rewrite is deterministic, so prompt-cache prefixes stay stable
across requests.
- No persisted state changes. OpenCode history stays as it is. Sessions
saved before this change also get the rewrite when they resume.
- Providers that do not use the runner provider proxy are not covered by
this change.
## Model Used
- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through
Claude Code with tool use (shell, file edits, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
3412 lines
145 KiB
TypeScript
3412 lines
145 KiB
TypeScript
import {
|
|
chmod,
|
|
copyFile,
|
|
lstat,
|
|
mkdir,
|
|
mkdtemp,
|
|
readFile,
|
|
readdir,
|
|
realpath,
|
|
rm,
|
|
stat,
|
|
symlink,
|
|
writeFile,
|
|
} from "node:fs/promises";
|
|
import { tmpdir } from "node:os";
|
|
import { createServer as createHttpServer } from "node:http";
|
|
import { execFile, type ChildProcess } from "node:child_process";
|
|
import { EventEmitter } from "node:events";
|
|
import { promisify } from "node:util";
|
|
import { getCACertificates } from "node:tls";
|
|
import { basename, dirname, join, resolve } from "node:path";
|
|
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
|
|
|
|
import {
|
|
NATIVE_RUNTIME_ASSET_SCHEMA,
|
|
PAPERCLIP_EXECUTION_PROMPT,
|
|
PAPERCLIP_EXECUTION_PROMPT_REVISION,
|
|
canonicalNativeRuntimeContextDigest,
|
|
nativeRuntimePromptDigest,
|
|
type NativeRuntimeContextSnapshot,
|
|
} from "../../contracts/runtime-context.js";
|
|
import { createCodexTaskEnvelope } from "../../contracts/codex.js";
|
|
import { CONFIGURED_ENVIRONMENT_KEYS, configuredEnvironmentProjection } from "../../configured-environment.js";
|
|
import { localIntegrityBoundaryGolden } from "../../../test-support/local-integrity-boundary-golden.js";
|
|
import type { PrpEvent } from "../../protocol/replay-contract.js";
|
|
import {
|
|
OpenCodeServerDriver,
|
|
openCodeServerDriverInternals,
|
|
} from "./opencode-server-driver.js";
|
|
|
|
const upstreamTransport = vi.hoisted(() => ({ endpoint: "", urls: [] as string[] }));
|
|
vi.mock("node:https", async importOriginal => {
|
|
const actual = await importOriginal<typeof import("node:https")>();
|
|
const http = await import("node:http");
|
|
return { ...actual, request: (...args: Parameters<typeof actual.request>) => {
|
|
if (!upstreamTransport.endpoint) return actual.request(...args);
|
|
upstreamTransport.urls.push(String(args[0]));
|
|
const options = args[1] as import("node:http").RequestOptions;
|
|
return http.request(upstreamTransport.endpoint, { ...options, agent: undefined }, args[2]);
|
|
} };
|
|
});
|
|
|
|
const roots: string[] = [];
|
|
let fixtureRoot: string | undefined;
|
|
let fixture: string;
|
|
|
|
beforeAll(async () => {
|
|
fixtureRoot = await mkdtemp(join(tmpdir(), "paperclip-opencode-fixture-"));
|
|
fixture = join(fixtureRoot, "fake-opencode-server.mjs");
|
|
await copyFile(resolve("test/fixtures/fake-opencode-server.mjs"), fixture);
|
|
// Tests launch the fixture directly; it must not depend on run order.
|
|
await chmod(fixture, 0o755);
|
|
});
|
|
|
|
const TURN_TERMINAL_EVENT_TYPES = new Set([
|
|
"turn.completed",
|
|
"turn.failed",
|
|
"turn.interrupted",
|
|
"turn.cancelled",
|
|
]);
|
|
|
|
/**
|
|
* The session event stream stays open past a turn boundary so a session can
|
|
* run more turns. Read one turn's events by stopping at its terminal event,
|
|
* the same rule the production consumer applies.
|
|
*/
|
|
async function collectTurnEvents(
|
|
events: AsyncIterable<PrpEvent>,
|
|
): Promise<PrpEvent[]> {
|
|
const collected: PrpEvent[] = [];
|
|
for await (const event of events) {
|
|
collected.push(event);
|
|
if (TURN_TERMINAL_EVENT_TYPES.has(event.eventType)) break;
|
|
}
|
|
return collected;
|
|
}
|
|
|
|
function runtimeContext(
|
|
skillRoot: string,
|
|
instructionRoot: string,
|
|
): NativeRuntimeContextSnapshot {
|
|
const digest = "0".repeat(64);
|
|
const value = {
|
|
prompt: {
|
|
revision: PAPERCLIP_EXECUTION_PROMPT_REVISION,
|
|
text: PAPERCLIP_EXECUTION_PROMPT,
|
|
digest: nativeRuntimePromptDigest(),
|
|
},
|
|
instructions: {
|
|
entryPath: "AGENTS.md",
|
|
bundle: {
|
|
schema: NATIVE_RUNTIME_ASSET_SCHEMA,
|
|
digest,
|
|
manifestDigest: digest,
|
|
rootPath: instructionRoot,
|
|
fileCount: 2,
|
|
totalBytes: 2,
|
|
},
|
|
},
|
|
skills: [
|
|
{
|
|
key: "company/assigned",
|
|
runtimeName: "assigned",
|
|
versionId: "version-1",
|
|
bundle: {
|
|
schema: NATIVE_RUNTIME_ASSET_SCHEMA,
|
|
digest,
|
|
manifestDigest: digest,
|
|
rootPath: skillRoot,
|
|
fileCount: 2,
|
|
totalBytes: 2,
|
|
},
|
|
},
|
|
],
|
|
mcp: { assignmentSetId: "assigned", digest, bindingId: "binding" },
|
|
} satisfies Omit<NativeRuntimeContextSnapshot, "aggregateDigest">;
|
|
return {
|
|
...value,
|
|
aggregateDigest: canonicalNativeRuntimeContextDigest(value),
|
|
};
|
|
}
|
|
|
|
async function makeWritable(root: string): Promise<void> {
|
|
const info = await lstat(root).catch(() => null);
|
|
if (!info) return;
|
|
if (info.isDirectory()) {
|
|
await chmod(root, 0o700);
|
|
for (const entry of await readdir(root))
|
|
await makeWritable(join(root, entry));
|
|
} else {
|
|
await chmod(root, 0o600);
|
|
}
|
|
}
|
|
|
|
afterEach(async () => {
|
|
await Promise.all(
|
|
roots.splice(0).map(async (root) => {
|
|
await makeWritable(root);
|
|
await rm(root, { recursive: true, force: true });
|
|
}),
|
|
);
|
|
});
|
|
|
|
afterAll(async () => {
|
|
if (fixtureRoot) await rm(fixtureRoot, { recursive: true, force: true });
|
|
});
|
|
|
|
describe("OpenCodeServerDriver", () => {
|
|
it("bounds schema rejection errors in events and transcript without changing the MCP receipt", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-large-rejection-")); roots.push(root);
|
|
const handler = vi.fn();
|
|
const driver = new OpenCodeServerDriver({ model: "openrouter/test-model", command: fixture, runtimeDirectory: root,
|
|
environment: { PATH: process.env.PATH },
|
|
dynamicTools: [{ name: "large_call", inputSchema: { type: "array", items: { type: "string" } } }],
|
|
dynamicToolHandler: handler,
|
|
});
|
|
const session = await driver.openSession({ runId: "large-rejection", normalizedSessionId: "large-rejection", workingDirectory: root });
|
|
try {
|
|
await session.startTurn({ message: { role: "user", text: "oversized-schema-rejection" } });
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(events.at(-1)?.eventType).toBe("turn.completed");
|
|
const receipt = JSON.parse(await readFile(join(root, "fake-large-rejection-receipt.json"), "utf8"));
|
|
expect(receipt.result.isError).toBe(true);
|
|
expect(receipt.result.content[0].text.length).toBeGreaterThan(8_192);
|
|
expect(handler).not.toHaveBeenCalled();
|
|
if (!session.transcript) throw new Error("OpenCode session must expose its transcript");
|
|
for (const recorded of [events, (await session.transcript()).events]) {
|
|
const errors = recorded.filter(event => event.eventType === "item.completed")
|
|
.map(event => event.payload.item as { is_error?: boolean; error?: string }).filter(item => item?.is_error);
|
|
expect(errors).toHaveLength(1);
|
|
expect(errors[0]!.error).toContain("Invalid tool input");
|
|
expect(errors[0]!.error).toHaveLength(8_192);
|
|
}
|
|
} finally { await session.close({ reason: "test" }); }
|
|
});
|
|
|
|
it.each(["1", "0"])("exports safe dispatch argument diagnostics with tool content=%s", async capture => {
|
|
await chmod(fixture, 0o755);
|
|
const bodies: any[] = [];
|
|
const collector = createHttpServer((request, response) => {
|
|
const chunks: Buffer[] = [];
|
|
request.on("data", chunk => chunks.push(chunk));
|
|
request.on("end", () => { bodies.push(JSON.parse(Buffer.concat(chunks).toString("utf8"))); response.end("{}"); });
|
|
});
|
|
await new Promise<void>(resolve => collector.listen(0, "127.0.0.1", resolve));
|
|
const address = collector.address();
|
|
if (!address || typeof address === "string") throw new Error("collector did not bind");
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-argument-diagnostics-")); roots.push(root);
|
|
const calls: unknown[] = [];
|
|
const driver = new OpenCodeServerDriver({ model: "openrouter/test-model", command: fixture, runtimeDirectory: root,
|
|
environment: { PATH: process.env.PATH, PAPERCLIP_OBSERVABILITY_OPENCODE: "1",
|
|
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: `http://127.0.0.1:${address.port}/v1/traces`,
|
|
OTEL_LOG_TOOL_DETAILS: "1", OTEL_LOG_TOOL_CONTENT: capture },
|
|
taskEnvelope: createCodexTaskEnvelope({ objective: "argument-diagnostics" }),
|
|
dynamicTools: [{ name: "paperclip_call_assigned_tool", inputSchema: { type: "object" } }],
|
|
dynamicToolHandler: async call => {
|
|
calls.push(call.arguments);
|
|
if (typeof (call.arguments as { name?: unknown }).name !== "string") throw new Error("name must be a string");
|
|
return { receipt: "benign-receipt" };
|
|
},
|
|
});
|
|
let session: Awaited<ReturnType<typeof driver.openSession>> | undefined;
|
|
try {
|
|
session = await driver.openSession({ runId: "first", normalizedSessionId: "diagnostics", workingDirectory: root });
|
|
for (const runId of ["first", "second"]) {
|
|
if (runId === "second") session.attachRun!({ runId });
|
|
await session.startTurn({ message: { role: "user", text: JSON.stringify({ message: "argument-diagnostics", completionContract: createCodexTaskEnvelope({ objective: "argument-diagnostics" }).completionContract }) } });
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(events.at(-1)?.eventType).toBe("turn.completed");
|
|
expect(events.filter(event => event.eventType === "run.result.proposed")).toHaveLength(1);
|
|
const receipts = JSON.parse(await readFile(join(root, "fake-diagnostic-receipts.json"), "utf8"));
|
|
expect(receipts[0].result.isError).toBe(true);
|
|
expect(receipts[1].result.isError).toBe(true);
|
|
expect(receipts[2].result.content[0].text).toContain("benign-receipt");
|
|
}
|
|
await session.close({ reason: "test" }); session = undefined;
|
|
expect(calls).toHaveLength(4);
|
|
expect(bodies).toHaveLength(2);
|
|
for (const [index, body] of bodies.entries()) {
|
|
const spans = body.resourceSpans[0].scopeSpans[0].spans;
|
|
const attrs = (span: any) => Object.fromEntries(span.attributes.map((a: any) => [a.key, a.value.stringValue ?? a.value.boolValue]));
|
|
const tools = spans.filter((span: any) => span.name === "tool paperclip_call_assigned_tool");
|
|
expect(tools).toHaveLength(3);
|
|
const native = spans.filter((span: any) => span.name === "opencode tool paperclip_call_assigned_tool");
|
|
expect(native).toHaveLength(1);
|
|
expect(attrs(native[0])["paperclip.tool.source"]).toBe("provider");
|
|
expect(attrs(native[0])).not.toHaveProperty("braintrust.input");
|
|
expect(attrs(tools[0])["paperclip.tool.source"]).toBe("dispatch");
|
|
|
|
expect(tools.map((span: any) => span.status.code)).toEqual([2, 2, 1]);
|
|
expect(attrs(tools[0])["paperclip.run.id"]).toBe(index === 0 ? "first" : "second");
|
|
if (capture === "1") {
|
|
expect(JSON.parse(attrs(tools[0])["braintrust.input"])).toMatchObject({ argumentType: "array", fields: {} });
|
|
expect(JSON.parse(attrs(tools[1])["braintrust.input"])).toMatchObject({ schema: "paperclip.opencode.tool-diagnostics.v1", method: "tools/call", argumentType: "object", fields: { name: "number", arguments: "object" } });
|
|
expect(JSON.parse(attrs(tools[2])["braintrust.input"]).fields.name).toBe("string");
|
|
} else for (const tool of tools) expect(attrs(tool)).not.toHaveProperty("braintrust.input");
|
|
}
|
|
for (const excluded of ["private-argument-canary", "private-key-canary", "private-value-canary", "benign-operation"]) expect(JSON.stringify(bodies)).not.toContain(excluded);
|
|
} finally {
|
|
await session?.close({ reason: "test" });
|
|
await new Promise<void>(resolve => { collector.close(() => resolve()); collector.closeAllConnections(); });
|
|
}
|
|
});
|
|
|
|
it("forwards configured bindings to the provider shell and redacts their values", async () => {
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-configured-env-"));
|
|
roots.push(root);
|
|
const bindings = { PROBE_PLAIN: "fixture-plain-binding", PROBE_SECRET: 'fixture-"quoted"\\path\nmultiline-binding' };
|
|
const command = join(root, "probe-opencode.mjs");
|
|
await writeFile(command, [
|
|
"#!/usr/bin/env node",
|
|
'import { execFileSync } from "node:child_process";',
|
|
'import { writeFile } from "node:fs/promises";',
|
|
'import { join } from "node:path";',
|
|
'const lengths = execFileSync("/bin/sh", ["-c", \'printf "%s,%s" "${#PROBE_PLAIN}" "${#PROBE_SECRET}"\'], { encoding: "utf8" });',
|
|
'await writeFile(join(process.env.XDG_DATA_HOME, "shell-probe.json"), JSON.stringify({ lengths, keys: JSON.parse(process.env.PAPERCLIP_CONFIGURED_ENV_KEYS ?? "[]") }));',
|
|
'process.stderr.write(`plain binding: ${process.env.PROBE_PLAIN}\\nresolved binding: ${process.env.PROBE_SECRET}\\n`);',
|
|
`await import(${JSON.stringify(fixture)});`,
|
|
].join("\n"), { mode: 0o755 });
|
|
const diagnostics: string[] = [];
|
|
const tracePath = join(root, "provider-trace.ndjson");
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/test-model", command, runtimeDirectory: root,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
...configuredEnvironmentProjection({
|
|
...bindings,
|
|
OPENCODE_CONFIG: "/untrusted/opencode.json",
|
|
OPENCODE_CONFIG_CONTENT: '{"permission":{"*":"allow"}}',
|
|
OPENCODE_CONFIG_DIR: "/untrusted/opencode",
|
|
OPENCODE_PERMISSION: '{"*":"allow"}',
|
|
OPENCODE_DISABLE_PROJECT_CONFIG: "false",
|
|
XDG_CONFIG_HOME: "/untrusted/config",
|
|
XDG_DATA_HOME: "/untrusted/data",
|
|
XDG_CACHE_HOME: "/untrusted/cache",
|
|
XDG_STATE_HOME: "/untrusted/state",
|
|
}),
|
|
DATABASE_URL: "unbound-host-database",
|
|
PAPERCLIP_API_KEY: "controller-key",
|
|
NODE_OPTIONS: "--require=/untrusted/bootstrap.js",
|
|
PAPERCLIP_PROVIDER_TRACE_PATH: tracePath,
|
|
},
|
|
onDiagnostic: message => diagnostics.push(message),
|
|
});
|
|
const session = await driver.openSession({ runId: "bindings", normalizedSessionId: "bindings", workingDirectory: root });
|
|
let persisted = "";
|
|
try {
|
|
const probe = JSON.parse(await readFile(join(root, "bindings", "data", "shell-probe.json"), "utf8"));
|
|
expect(probe).toEqual({ lengths: `${bindings.PROBE_PLAIN.length},${bindings.PROBE_SECRET.length}`, keys: Object.keys(bindings).sort() });
|
|
const child = JSON.parse(await readFile(join(root, "bindings", "data", "fake-environment.json"), "utf8"));
|
|
expect(child.keys).toEqual(expect.arrayContaining(Object.keys(bindings)));
|
|
for (const key of ["DATABASE_URL", "PAPERCLIP_API_KEY", "NODE_OPTIONS", "OPENCODE_CONFIG",
|
|
"OPENCODE_CONFIG_CONTENT", "OPENCODE_CONFIG_DIR", "OPENCODE_PERMISSION", "XDG_STATE_HOME"]) {
|
|
expect(child.keys).not.toContain(key);
|
|
}
|
|
expect(child.projectConfigDisabled).toBe("true");
|
|
await session.startTurn({ message: { role: "user", text: "configured-environment-output" } });
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(events.at(-1)?.eventType).toBe("turn.completed");
|
|
persisted = JSON.stringify(await session.snapshot())
|
|
+ await readFile(join(root, "bindings", "config", "opencode", "opencode.json"), "utf8");
|
|
} finally {
|
|
await session.close({ reason: "test" });
|
|
}
|
|
const trace = await readFile(tracePath, "utf8");
|
|
const frames = trace.trim().split("\n").map(line => JSON.parse(line))
|
|
.filter(entry => entry.kind === "frame")
|
|
.map(entry => Buffer.from(entry.rawBase64, "base64").toString("utf8"));
|
|
expect(frames.find(frame => frame.includes("part-configured-binding-output"))).toContain('"output":"[REDACTED]"');
|
|
persisted += trace + frames.join("\n");
|
|
expect(diagnostics.join("\n")).toContain("plain binding: [REDACTED]");
|
|
expect(diagnostics.join("\n")).toContain("resolved binding: [REDACTED]");
|
|
for (const value of Object.values(bindings)) {
|
|
expect(diagnostics.join("\n")).not.toContain(value);
|
|
expect(persisted).not.toContain(value);
|
|
expect(persisted).not.toContain(JSON.stringify(value).slice(1, -1));
|
|
}
|
|
});
|
|
|
|
it("preserves empty arrays in traces when all configured bindings are runtime-owned", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-empty-projection-"));
|
|
roots.push(root);
|
|
const tracePath = join(root, "provider-trace.ndjson");
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/test-model", command: fixture, runtimeDirectory: root,
|
|
environment: { PATH: process.env.PATH, PAPERCLIP_PROVIDER_TRACE_PATH: tracePath,
|
|
...configuredEnvironmentProjection({ OPENCODE_CONFIG_CONTENT: "{}" }) },
|
|
});
|
|
const session = await driver.openSession({ runId: "empty", normalizedSessionId: "empty", workingDirectory: root });
|
|
try {
|
|
expect(await session.read!()).toMatchObject({ messages: [] });
|
|
} finally {
|
|
await session.close({ reason: "test" });
|
|
}
|
|
const entries = (await readFile(tracePath, "utf8")).trim().split("\n").map(line => JSON.parse(line));
|
|
const frame = entries.find(entry => entry.kind === "frame" && entry.direction === "provider_to_client"
|
|
&& entry.nativeMethod === "GET /session/ses_fake_1/message 200");
|
|
expect(frame).toBeDefined();
|
|
expect(Buffer.from(frame.rawBase64, "base64").toString("utf8")).toBe("[]");
|
|
});
|
|
|
|
it.each([false, true])("ignores an ambient configured-environment projection (explicit environment: %s)", async explicit => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-ambient-env-"));
|
|
roots.push(root);
|
|
vi.stubEnv(CONFIGURED_ENVIRONMENT_KEYS, '["PROBE_SECRET"]');
|
|
vi.stubEnv("PROBE_SECRET", "ambient-binding-must-not-cross");
|
|
try {
|
|
const driver = new OpenCodeServerDriver({ model: "openrouter/test-model", command: fixture, runtimeDirectory: root,
|
|
...(explicit ? { environment: { PATH: process.env.PATH } } : {}) });
|
|
const session = await driver.openSession({ runId: "ambient", normalizedSessionId: "ambient", workingDirectory: root });
|
|
try {
|
|
const child = JSON.parse(await readFile(join(root, "ambient", "data", "fake-environment.json"), "utf8"));
|
|
expect(child.keys).not.toContain("PROBE_SECRET");
|
|
expect(child.keys).not.toContain(CONFIGURED_ENVIRONMENT_KEYS);
|
|
} finally {
|
|
await session.close({ reason: "test" });
|
|
}
|
|
} finally {
|
|
vi.unstubAllEnvs();
|
|
}
|
|
});
|
|
|
|
it.each([false, true])("uses the native MCP client ceiling only for durable tool requests (%s)", async durableToolRequests => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-tool-budget-"));
|
|
roots.push(root);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/test-model", command: fixture, runtimeDirectory: root, durableToolRequests,
|
|
});
|
|
const session = await driver.openSession({ runId: "budget", normalizedSessionId: "budget", workingDirectory: root });
|
|
try {
|
|
const config = JSON.parse(await readFile(join(root, "budget", "config", "opencode", "opencode.json"), "utf8"));
|
|
expect(config.mcp.paperclip.timeout).toBe(durableToolRequests ? 60 * 60 * 1_000 : 30_000);
|
|
} finally {
|
|
await session.close({ reason: "test" });
|
|
}
|
|
});
|
|
|
|
it("exports company traces across warm runs without giving the ingestion key to the provider", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const bodies: Array<Record<string, any>> = [];
|
|
const collector = createHttpServer((request, response) => {
|
|
const chunks: Buffer[] = [];
|
|
request.on("data", chunk => chunks.push(chunk));
|
|
request.on("end", () => { bodies.push(JSON.parse(Buffer.concat(chunks).toString("utf8"))); response.end("{}"); });
|
|
});
|
|
await new Promise<void>(resolve => collector.listen(0, "127.0.0.1", resolve));
|
|
const address = collector.address();
|
|
if (!address || typeof address === "string") throw new Error("collector did not bind");
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-company-export-")); roots.push(root);
|
|
const driver = new OpenCodeServerDriver({ model: "openrouter/test-model", command: fixture, runtimeDirectory: root,
|
|
environment: { PATH: process.env.PATH, PAPERCLIP_OBSERVABILITY_OPENCODE: "1",
|
|
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: `http://127.0.0.1:${address.port}/v1/traces`,
|
|
OTEL_EXPORTER_OTLP_TRACES_HEADERS: "authorization=Bearer%20ingestion-canary",
|
|
OTEL_RESOURCE_ATTRIBUTES: "paperclip.company.id=fixture-company", OTEL_LOG_USER_PROMPTS: "1", OTEL_LOG_ASSISTANT_RESPONSES: "1" } });
|
|
let session: Awaited<ReturnType<typeof driver.openSession>> | undefined;
|
|
try {
|
|
session = await driver.openSession({ runId: "first", normalizedSessionId: "export", workingDirectory: root });
|
|
const child = await readFile(join(root, "export", "data", "fake-environment.json"), "utf8");
|
|
expect(child).not.toContain("OTEL_EXPORTER_OTLP_TRACES_HEADERS");
|
|
expect(child).not.toContain("ingestion-canary");
|
|
for (const runId of ["first", "second"]) {
|
|
if (runId === "second") session.attachRun!({ runId });
|
|
await session.startTurn({ message: { role: "user", text: "finish" } });
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(events.at(-1)).toMatchObject({ runId, eventType: "turn.completed" });
|
|
}
|
|
await session.close({ reason: "test" }); session = undefined;
|
|
expect(bodies).toHaveLength(2);
|
|
const roots = bodies.map(body => body.resourceSpans[0].scopeSpans[0].spans[0]);
|
|
expect(roots.map(span => span.attributes.find((a: any) => a.key === "paperclip.run.id").value.stringValue)).toEqual(["first", "second"]);
|
|
expect(JSON.stringify(bodies)).not.toContain("ingestion-canary");
|
|
} finally {
|
|
await session?.close({ reason: "test" });
|
|
await new Promise<void>(resolve => { collector.close(() => resolve()); collector.closeAllConnections(); });
|
|
}
|
|
});
|
|
|
|
it("advertises within-turn plans as unsupported", async () => {
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: "/tmp/paperclip-opencode-capabilities",
|
|
command: fixture,
|
|
});
|
|
const descriptor = await driver.descriptor();
|
|
expect(
|
|
descriptor.capabilities.typedEventFamilies?.find(
|
|
(family) => family.family === "plan",
|
|
),
|
|
).toMatchObject({ availability: "unsupported" });
|
|
});
|
|
|
|
it("normalizes raw OpenCode boundary events into the local-integrity golden identities", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const profile = localIntegrityBoundaryGolden.profiles.find(
|
|
(candidate) => candidate.id === "runner-opencode",
|
|
);
|
|
if (!profile) throw new Error("missing runner-opencode golden profile");
|
|
const goldenEvent = (sourceEventId: string) => {
|
|
const event = localIntegrityBoundaryGolden.events.find(
|
|
(candidate) => candidate.sourceEventId === sourceEventId,
|
|
);
|
|
if (!event) throw new Error(`missing golden event ${sourceEventId}`);
|
|
return event;
|
|
};
|
|
const progress = localIntegrityBoundaryGolden.expected.assistantItems.find(
|
|
(item) => item.channel === "progress",
|
|
);
|
|
const final = localIntegrityBoundaryGolden.expected.assistantItems.find(
|
|
(item) => item.channel === "final",
|
|
);
|
|
if (!progress || !final)
|
|
throw new Error("missing golden assistant messages");
|
|
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-boundary-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-boundary-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const tracePath = join(root, "provider-trace.ndjson");
|
|
const providerSessionId = "provider-session-boundary";
|
|
const nativeEvents: Array<Record<string, unknown>> = [];
|
|
const queuedFrames: Uint8Array[] = [];
|
|
const encoder = new TextEncoder();
|
|
let streamController: ReadableStreamDefaultController<Uint8Array> | null =
|
|
null;
|
|
let streamClosed = false;
|
|
const emitNative = (event: Record<string, unknown>) => {
|
|
nativeEvents.push(structuredClone(event));
|
|
const frame = encoder.encode(`data: ${JSON.stringify(event)}\n\n`);
|
|
if (streamController) streamController.enqueue(frame);
|
|
else queuedFrames.push(frame);
|
|
};
|
|
const closeNativeStream = () => {
|
|
streamClosed = true;
|
|
streamController?.close();
|
|
};
|
|
const nativeQuestion = {
|
|
type: "question.asked",
|
|
id: "native-question-created",
|
|
properties: {
|
|
id: localIntegrityBoundaryGolden.expected.questionRequestId,
|
|
sessionID: providerSessionId,
|
|
title: "Boundary mode",
|
|
questions: [
|
|
{
|
|
id: "boundary-mode",
|
|
question: "Which mode should continue?",
|
|
required: true,
|
|
options: [{ id: "lossless", label: "Lossless" }],
|
|
},
|
|
],
|
|
},
|
|
} satisfies Record<string, unknown>;
|
|
const emitBeforeQuestion = () => {
|
|
emitNative({
|
|
type: "message.updated",
|
|
id: "native-progress-message-role",
|
|
properties: {
|
|
info: {
|
|
id: "provider-message-progress",
|
|
sessionID: providerSessionId,
|
|
role: "assistant",
|
|
},
|
|
},
|
|
});
|
|
emitNative({
|
|
type: "message.part.updated",
|
|
id: "native-progress-part",
|
|
properties: {
|
|
sessionID: providerSessionId,
|
|
part: {
|
|
id: progress.itemId,
|
|
messageID: "provider-message-progress",
|
|
type: "text",
|
|
text: progress.text,
|
|
time: { start: 1, end: 2 },
|
|
},
|
|
},
|
|
});
|
|
emitNative({
|
|
type: "message.part.updated",
|
|
id: "native-reasoning-part",
|
|
properties: {
|
|
sessionID: providerSessionId,
|
|
part: {
|
|
id: localIntegrityBoundaryGolden.expected.reasoningItemId,
|
|
messageID: "provider-message-progress",
|
|
type: "reasoning",
|
|
text: "Verified event identity.",
|
|
time: { start: 2, end: 3 },
|
|
},
|
|
},
|
|
});
|
|
for (const [id, status, state] of [
|
|
["native-tool-started", "pending", { title: "Inspecting" }],
|
|
["native-tool-progressed", "running", { title: "Inspecting" }],
|
|
[
|
|
"native-tool-completed",
|
|
"completed",
|
|
{ output: "boundary-ok", time: 12, exit: 0 },
|
|
],
|
|
] as const) {
|
|
emitNative({
|
|
type: "message.part.updated",
|
|
id,
|
|
properties: {
|
|
sessionID: providerSessionId,
|
|
part: {
|
|
id: localIntegrityBoundaryGolden.expected.toolExecutionId,
|
|
messageID: "provider-message-progress",
|
|
type: "tool",
|
|
tool: "boundary-check",
|
|
state: { status, ...state },
|
|
},
|
|
},
|
|
});
|
|
}
|
|
emitNative(nativeQuestion);
|
|
};
|
|
const emitAfterQuestion = () => {
|
|
emitNative({
|
|
...nativeQuestion,
|
|
id: "native-question-resolved",
|
|
type: "question.replied",
|
|
});
|
|
emitNative({
|
|
type: "message.updated",
|
|
id: "native-final-message-role",
|
|
properties: {
|
|
info: {
|
|
id: "provider-message-final",
|
|
sessionID: providerSessionId,
|
|
role: "assistant",
|
|
},
|
|
},
|
|
});
|
|
emitNative({
|
|
type: "message.part.updated",
|
|
id: "native-final-part",
|
|
properties: {
|
|
sessionID: providerSessionId,
|
|
part: {
|
|
id: final.itemId,
|
|
messageID: "provider-message-final",
|
|
type: "text",
|
|
text: final.text,
|
|
time: { start: 4, end: 5 },
|
|
},
|
|
},
|
|
});
|
|
emitNative({
|
|
type: "message.updated",
|
|
id: "native-usage",
|
|
properties: {
|
|
info: {
|
|
id: "provider-message-final",
|
|
sessionID: providerSessionId,
|
|
role: "assistant",
|
|
tokens: {
|
|
input: 40,
|
|
output: 12,
|
|
cache: { read: 3, write: 0 },
|
|
},
|
|
cost: 0.02,
|
|
},
|
|
},
|
|
});
|
|
emitNative({
|
|
type: "session.idle",
|
|
id: "native-turn-completed",
|
|
properties: { sessionID: providerSessionId },
|
|
});
|
|
closeNativeStream();
|
|
};
|
|
let submittedQuestionReply: unknown = null;
|
|
const fetcher: typeof globalThis.fetch = async (input, init) => {
|
|
const url = new URL(String(input));
|
|
const method = String(init?.method ?? "GET").toUpperCase();
|
|
const json = (value: unknown, status = 200) =>
|
|
new Response(JSON.stringify(value), {
|
|
status,
|
|
headers: { "Content-Type": "application/json" },
|
|
});
|
|
if (url.pathname === "/global/health")
|
|
return json({ healthy: true, version: "1.18.34" });
|
|
if (url.pathname === "/event") {
|
|
return new Response(
|
|
new ReadableStream<Uint8Array>({
|
|
start(controller) {
|
|
streamController = controller;
|
|
for (const frame of queuedFrames.splice(0))
|
|
controller.enqueue(frame);
|
|
if (streamClosed) controller.close();
|
|
},
|
|
}),
|
|
{ status: 200, headers: { "Content-Type": "text/event-stream" } },
|
|
);
|
|
}
|
|
if (method === "POST" && url.pathname === "/session")
|
|
return json({ id: providerSessionId });
|
|
if (method === "GET" && url.pathname === "/question") return json([]);
|
|
if (method === "GET" && url.pathname === "/permission") return json([]);
|
|
if (
|
|
method === "POST" &&
|
|
url.pathname === `/session/${providerSessionId}/prompt_async`
|
|
) {
|
|
emitBeforeQuestion();
|
|
return new Response(null, { status: 204 });
|
|
}
|
|
if (
|
|
method === "POST" &&
|
|
url.pathname ===
|
|
`/question/${localIntegrityBoundaryGolden.expected.questionRequestId}/reply`
|
|
) {
|
|
submittedQuestionReply = JSON.parse(String(init?.body));
|
|
emitAfterQuestion();
|
|
return json(true);
|
|
}
|
|
if (
|
|
method === "POST" &&
|
|
url.pathname === `/session/${providerSessionId}/abort`
|
|
)
|
|
return json(true);
|
|
throw new Error(`unexpected OpenCode boundary request ${method} ${url}`);
|
|
};
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
runnerInstanceId: profile.sourceInstanceId,
|
|
fetch: fetcher,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
PAPERCLIP_PROVIDER_TRACE_PATH: tracePath,
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: profile.runId,
|
|
normalizedSessionId: profile.normalizedSessionId,
|
|
workingDirectory: workspace,
|
|
});
|
|
const { turnId } = await session.startTurn({
|
|
message: { role: "user", text: "exercise native golden boundary" },
|
|
});
|
|
const events = [];
|
|
const iterator = session.events()[Symbol.asyncIterator]();
|
|
let requestCreated = false;
|
|
while (!requestCreated) {
|
|
const next = await iterator.next();
|
|
if (next.done)
|
|
throw new Error("OpenCode ended before asking its question");
|
|
events.push(next.value);
|
|
requestCreated = next.value.eventType === "runtime_request.created";
|
|
}
|
|
await session.resolveRuntimeRequest?.({
|
|
requestId: localIntegrityBoundaryGolden.expected.questionRequestId,
|
|
turnId,
|
|
resolution: {
|
|
action: "submit",
|
|
response: {
|
|
schema: "paperclip.question_response.v1",
|
|
answers: {
|
|
"boundary-mode": { selectedOptionIds: ["lossless"] },
|
|
},
|
|
},
|
|
},
|
|
});
|
|
for (;;) {
|
|
const next = await iterator.next();
|
|
if (next.done) break;
|
|
events.push(next.value);
|
|
if (TURN_TERMINAL_EVENT_TYPES.has(next.value.eventType)) break;
|
|
}
|
|
await session.close({ reason: "boundary-golden-test" });
|
|
|
|
expect(submittedQuestionReply).toEqual({ answers: [["Lossless"]] });
|
|
expect(
|
|
events.every(
|
|
(event) =>
|
|
event.runId === profile.runId &&
|
|
event.normalizedSessionId === profile.normalizedSessionId,
|
|
),
|
|
).toBe(true);
|
|
expect(events).toContainEqual(
|
|
expect.objectContaining({
|
|
itemId: progress.itemId,
|
|
eventType: "item.delta",
|
|
payload: expect.objectContaining({
|
|
kind: "agentMessage",
|
|
channel: progress.channel,
|
|
providerPhase: "commentary",
|
|
text: progress.text,
|
|
}),
|
|
}),
|
|
);
|
|
expect(events).toContainEqual(
|
|
expect.objectContaining({
|
|
itemId: localIntegrityBoundaryGolden.expected.reasoningItemId,
|
|
eventType: "item.completed",
|
|
payload: expect.objectContaining({
|
|
kind: "reasoning",
|
|
channel: "detail",
|
|
providerPhase: "reasoning",
|
|
text: String(goldenEvent("reasoning-completed").payload.text),
|
|
}),
|
|
}),
|
|
);
|
|
expect(
|
|
events
|
|
.filter(
|
|
(event) =>
|
|
event.itemId ===
|
|
localIntegrityBoundaryGolden.expected.toolExecutionId &&
|
|
event.eventType.startsWith("tool.execution."),
|
|
)
|
|
.map((event) => [event.eventType, event.payload.status]),
|
|
).toEqual([
|
|
["tool.execution.started", "running"],
|
|
["tool.execution.progressed", "running"],
|
|
["tool.execution.completed", "completed"],
|
|
]);
|
|
expect(events).toContainEqual(
|
|
expect.objectContaining({
|
|
itemId: localIntegrityBoundaryGolden.expected.questionRequestId,
|
|
eventType: "runtime_request.created",
|
|
payload: {
|
|
request: expect.objectContaining({
|
|
requestId: localIntegrityBoundaryGolden.expected.questionRequestId,
|
|
origin: {
|
|
adapter: profile.adapter,
|
|
provider: profile.originProvider,
|
|
method: profile.originMethod,
|
|
},
|
|
input: expect.objectContaining({
|
|
schema: "paperclip.question_set.v1",
|
|
title: "Boundary mode",
|
|
questions: [
|
|
expect.objectContaining({
|
|
id: "boundary-mode",
|
|
answerMode: "single_select",
|
|
}),
|
|
],
|
|
}),
|
|
}),
|
|
},
|
|
}),
|
|
);
|
|
expect(
|
|
events.filter(
|
|
(event) =>
|
|
event.itemId ===
|
|
localIntegrityBoundaryGolden.expected.questionRequestId &&
|
|
event.eventType === "runtime_request.resolved",
|
|
),
|
|
).toHaveLength(1);
|
|
expect(events).toContainEqual(
|
|
expect.objectContaining({
|
|
itemId: final.itemId,
|
|
eventType: "item.completed",
|
|
payload: expect.objectContaining({
|
|
kind: "agentMessage",
|
|
channel: final.channel,
|
|
providerPhase: "final_answer",
|
|
text: final.text,
|
|
}),
|
|
}),
|
|
);
|
|
expect(
|
|
events.find(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "usage",
|
|
),
|
|
).toMatchObject({
|
|
itemId: `${turnId}:usage`,
|
|
payload: {
|
|
usageMessageId: "provider-message-final",
|
|
usage: {
|
|
input: 40,
|
|
output: 12,
|
|
cache: { read: 3, write: 0 },
|
|
costUsd: 0.02,
|
|
},
|
|
},
|
|
});
|
|
expect(events).toContainEqual(
|
|
expect.objectContaining({
|
|
eventType: "turn.completed",
|
|
turnId,
|
|
payload: { status: "completed" },
|
|
}),
|
|
);
|
|
|
|
// OpenCode 1.18 does not expose a qualified structured Plan event, and
|
|
// run.terminal is owned by the native-session layer above this driver.
|
|
// Keep both absences explicit instead of synthesizing Codex-shaped input.
|
|
expect(
|
|
(await driver.descriptor()).capabilities.typedEventFamilies?.find(
|
|
(family) => family.family === "plan",
|
|
),
|
|
).toMatchObject({ availability: "unsupported" });
|
|
expect(events.some((event) => event.eventType === "plan.updated")).toBe(
|
|
false,
|
|
);
|
|
expect(events.some((event) => event.eventType === "run.terminal")).toBe(
|
|
false,
|
|
);
|
|
|
|
const traceEntries = (await readFile(tracePath, "utf8"))
|
|
.trim()
|
|
.split("\n")
|
|
.map((line) => JSON.parse(line) as Record<string, unknown>);
|
|
const traceFrames = traceEntries.filter(
|
|
(entry) =>
|
|
entry.kind === "frame" && entry.direction === "provider_to_client",
|
|
);
|
|
const normalization = traceEntries.filter(
|
|
(entry) =>
|
|
entry.kind === "interpretation" &&
|
|
entry.stage === "typescript_opencode_driver_normalization",
|
|
);
|
|
expect(nativeEvents.map((event) => event.type)).toEqual([
|
|
"message.updated",
|
|
"message.part.updated",
|
|
"message.part.updated",
|
|
"message.part.updated",
|
|
"message.part.updated",
|
|
"message.part.updated",
|
|
"question.asked",
|
|
"question.replied",
|
|
"message.updated",
|
|
"message.part.updated",
|
|
"message.updated",
|
|
"session.idle",
|
|
]);
|
|
for (const nativeEvent of nativeEvents) {
|
|
const frame = traceFrames.find((entry) => {
|
|
if (typeof entry.rawBase64 !== "string") return false;
|
|
const raw = Buffer.from(entry.rawBase64, "base64").toString("utf8");
|
|
return raw.includes(`\"id\":\"${String(nativeEvent.id)}\"`);
|
|
});
|
|
expect(frame, String(nativeEvent.id)).toBeDefined();
|
|
const interpretation = normalization.find(
|
|
(entry) => entry.frameId === frame?.frameId,
|
|
);
|
|
expect(interpretation, String(nativeEvent.id)).toMatchObject({
|
|
disposition: expect.stringMatching(/^(mapped|ignored)$/),
|
|
});
|
|
if (interpretation?.disposition === "ignored") {
|
|
expect(
|
|
[
|
|
"native-progress-message-role",
|
|
"native-final-message-role",
|
|
"native-question-resolved",
|
|
],
|
|
`${String(nativeEvent.id)} must be a structural or duplicate echo`,
|
|
).toContain(nativeEvent.id);
|
|
}
|
|
}
|
|
});
|
|
|
|
it("projects a custom connection into the isolated OpenCode config", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-routing-"));
|
|
roots.push(root);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "paperclip/team/model-alias",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: { PATH: process.env.PATH, PAPERCLIP_AI_PROVIDER_URL: "https://gateway.example/v1", PAPERCLIP_AI_PROVIDER_KEY: "selected-gateway-key" },
|
|
});
|
|
const session = await driver.openSession({ runId: "routing", normalizedSessionId: "routing", workingDirectory: root });
|
|
try {
|
|
const configPath = join(root, "routing", "config", "opencode", "opencode.json");
|
|
expect(JSON.parse(await readFile(configPath, "utf8"))).toMatchObject({
|
|
model: "paperclip/team/model-alias", small_model: "paperclip/team/model-alias", plugin: [],
|
|
provider: { paperclip: { npm: "@ai-sdk/openai-compatible", options: { baseURL: expect.stringMatching(/^http:\/\/127\.0\.0\.1:\d+\/v1$/), apiKey: expect.any(String) }, models: { "team/model-alias": { name: "team/model-alias" } } } },
|
|
});
|
|
expect((await stat(configPath)).mode & 0o777).toBe(0o600);
|
|
const shell = await promisify(execFile)("sh", ["-c", 'cat "$1"', "sh", configPath]);
|
|
expect(shell.stdout).not.toContain("selected-gateway-key");
|
|
const environment = JSON.parse(await readFile(join(root, "routing", "data", "fake-environment.json"), "utf8"));
|
|
expect(environment.keys).not.toContain("PAPERCLIP_AI_PROVIDER_KEY");
|
|
} finally {
|
|
await session.close({ reason: "test" });
|
|
}
|
|
});
|
|
|
|
it("forwards an unreadable tool-result image as a tool outcome, not as a user message", async () => {
|
|
// Shape OpenCode 1.18.34 sends through @ai-sdk/openai-compatible after a
|
|
// text-only model reads a screenshot (chat-eval run 42704461144).
|
|
const received: unknown[] = [];
|
|
const upstream = createHttpServer((request, response) => {
|
|
void (async () => {
|
|
const chunks = [];
|
|
for await (const chunk of request) chunks.push(Buffer.from(chunk));
|
|
received.push(JSON.parse(Buffer.concat(chunks).toString("utf8")));
|
|
response.writeHead(200, { "content-type": "text/event-stream" });
|
|
response.end("data: [DONE]\n\n");
|
|
})().catch(() => response.destroy());
|
|
});
|
|
await new Promise<void>(resolve => upstream.listen(0, "127.0.0.1", resolve));
|
|
const address = upstream.address();
|
|
if (!address || typeof address === "string") throw new Error("Missing fixture address");
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-media-"));
|
|
roots.push(root);
|
|
let session: Awaited<ReturnType<OpenCodeServerDriver["openSession"]>> | undefined;
|
|
try {
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "paperclip/team/model-alias", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, PAPERCLIP_AI_PROVIDER_URL: `http://127.0.0.1:${address.port}/v1`, PAPERCLIP_AI_PROVIDER_KEY: "gateway-key" },
|
|
});
|
|
session = await driver.openSession({ runId: "media", normalizedSessionId: "media", workingDirectory: root });
|
|
const config = JSON.parse(await readFile(join(root, "media", "config", "opencode", "opencode.json"), "utf8"));
|
|
const { baseURL, apiKey } = config.provider.paperclip.options;
|
|
const prefix = [
|
|
{ role: "system", content: "system" },
|
|
{ role: "user", content: "Build the launch page and verify it." },
|
|
{ role: "assistant", content: "", tool_calls: [{ id: "call_read", type: "function", function: { name: "read", arguments: "{\"filePath\":\"/tmp/opencode/website-preview.png\"}" } }] },
|
|
{ role: "tool", tool_call_id: "call_read", content: "Image read successfully" },
|
|
];
|
|
const payload = {
|
|
model: "team/model-alias",
|
|
stream: true,
|
|
messages: [...prefix, {
|
|
role: "user",
|
|
content: [
|
|
{ type: "text", text: "Attached media from tool result:" },
|
|
{ type: "text", text: "ERROR: Cannot read image (this model does not support image input). Inform the user." },
|
|
],
|
|
}],
|
|
};
|
|
const result = await fetch(`${baseURL}/chat/completions`, {
|
|
method: "POST",
|
|
headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" },
|
|
body: JSON.stringify(payload),
|
|
});
|
|
expect(result.status).toBe(200);
|
|
await result.text();
|
|
const forwarded = received[0] as { messages: Array<{ role: string; content: unknown }> };
|
|
expect(forwarded.messages.slice(0, 3)).toEqual(prefix.slice(0, 3));
|
|
expect(forwarded.messages).toHaveLength(prefix.length);
|
|
expect(forwarded.messages.at(-1)?.role).toBe("tool");
|
|
const toolContent = String(forwarded.messages.at(-1)?.content);
|
|
expect(toolContent).toMatch(/^Image read successfully\n\n\[Paperclip runner: This tool result included an image attachment/);
|
|
expect(toolContent).toContain("not from the user");
|
|
expect(JSON.stringify(forwarded)).not.toContain("Inform the user");
|
|
} finally {
|
|
await session?.close({ reason: "test" });
|
|
await new Promise<void>(resolve => { upstream.close(() => resolve()); upstream.closeAllConnections(); });
|
|
}
|
|
});
|
|
|
|
it.each(["gateway-key", ""])("forwards selected-model streams without exposing the reusable key and revokes the proxy on close (%s)", async key => {
|
|
const received: Array<{ path: string; authorization?: string; body: unknown }> = [];
|
|
const upstream = createHttpServer((request, response) => {
|
|
void (async () => {
|
|
const chunks = [];
|
|
for await (const chunk of request) chunks.push(Buffer.from(chunk));
|
|
const body = JSON.parse(Buffer.concat(chunks).toString("utf8"));
|
|
received.push({ path: request.url!, authorization: request.headers.authorization, body });
|
|
response.writeHead(200, { "content-type": "text/event-stream", "x-provider-private-header": "private" });
|
|
response.write('data: {"choices":[]}\n\n');
|
|
if (body.messages[0].content !== "hold") response.end('data: [DONE]\n\n');
|
|
})().catch(() => response.destroy());
|
|
});
|
|
await new Promise<void>(resolve => upstream.listen(0, "127.0.0.1", resolve));
|
|
const address = upstream.address();
|
|
if (!address || typeof address === "string") throw new Error("Missing fixture address");
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-proxy-"));
|
|
roots.push(root);
|
|
let session: Awaited<ReturnType<OpenCodeServerDriver["openSession"]>> | undefined;
|
|
try {
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "paperclip/team/model-alias", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, PAPERCLIP_AI_PROVIDER_URL: `http://127.0.0.1:${address.port}/custom/v1`, PAPERCLIP_AI_PROVIDER_KEY: key },
|
|
});
|
|
session = await driver.openSession({ runId: "proxy", normalizedSessionId: "proxy", workingDirectory: root });
|
|
const config = JSON.parse(await readFile(join(root, "proxy", "config", "opencode", "opencode.json"), "utf8"));
|
|
const { baseURL, apiKey } = config.provider.paperclip.options;
|
|
expect(apiKey).not.toBe(key);
|
|
const url = `${baseURL}/chat/completions`;
|
|
const headers = { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" };
|
|
const payload = { model: "team/model-alias", stream: true, messages: [{ role: "user", content: "test" }] };
|
|
expect((await fetch(url, { method: "POST", body: JSON.stringify(payload) })).status).toBe(401);
|
|
expect((await fetch(`${baseURL}/models`, { headers })).status).toBe(404);
|
|
expect((await fetch(url, { method: "POST", headers, body: JSON.stringify({ ...payload, model: "other" }) })).status).toBe(400);
|
|
expect(received).toHaveLength(0);
|
|
const result = await fetch(url, { method: "POST", headers, body: JSON.stringify(payload) });
|
|
expect(result.headers.get("x-provider-private-header")).toBeNull();
|
|
expect(await result.text()).toBe('data: {"choices":[]}\n\ndata: [DONE]\n\n');
|
|
expect(received).toEqual([{ path: "/custom/v1/chat/completions", authorization: key ? `Bearer ${key}` : undefined, body: payload }]);
|
|
const pending = await fetch(url, { method: "POST", headers, body: JSON.stringify({ ...payload, messages: [{ role: "user", content: "hold" }] }) });
|
|
const pendingText = pending.text().then(() => "unexpected completion", () => "aborted");
|
|
await session.close({ reason: "test" });
|
|
session = undefined;
|
|
expect(await pendingText).toBe("aborted");
|
|
await expect(fetch(url, { method: "POST", headers, body: JSON.stringify(payload) })).rejects.toThrow();
|
|
} finally {
|
|
await session?.close({ reason: "test" });
|
|
await new Promise<void>(resolve => { upstream.close(() => resolve()); upstream.closeAllConnections(); });
|
|
}
|
|
});
|
|
|
|
it.each(["explicit", "ambient"])("keeps OpenRouter upstream credentials out of the child while preserving exact provider/model routing (%s)", async source => {
|
|
await chmod(fixture, 0o755);
|
|
const key = "synthetic-upstream-openrouter-key";
|
|
const model = "deepseek/deepseek-v4.1-flash-1017";
|
|
const received: Array<{ authorization?: string; body: unknown; referer?: string }> = [];
|
|
let redirect = false;
|
|
const upstream = createHttpServer((request, response) => {
|
|
void (async () => {
|
|
const chunks = [];
|
|
for await (const chunk of request) chunks.push(Buffer.from(chunk));
|
|
received.push({ authorization: request.headers.authorization, body: JSON.parse(Buffer.concat(chunks).toString("utf8")), referer: request.headers["http-referer"] as string });
|
|
if (redirect) response.writeHead(302, { location: "https://must-not-follow.invalid/" }).end();
|
|
else response.writeHead(200, { "content-type": "text/event-stream" }).end("data: [DONE]\n\n");
|
|
})().catch(() => response.destroy());
|
|
});
|
|
await new Promise<void>(resolve => upstream.listen(0, "127.0.0.1", resolve));
|
|
const address = upstream.address();
|
|
if (!address || typeof address === "string") throw new Error("Missing fixture address");
|
|
upstreamTransport.endpoint = `http://127.0.0.1:${address.port}/chat/completions`;
|
|
upstreamTransport.urls = [];
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-openrouter-custody-"));
|
|
roots.push(root);
|
|
let session: Awaited<ReturnType<OpenCodeServerDriver["openSession"]>> | undefined;
|
|
try {
|
|
if (source === "ambient") {
|
|
vi.stubEnv("OPENROUTER_API_KEY", key);
|
|
vi.stubEnv("no_proxy", undefined);
|
|
vi.stubEnv("NO_PROXY", "custody-bypass.invalid");
|
|
}
|
|
const driver = new OpenCodeServerDriver({ model: `openrouter/${model}`, runtimeDirectory: root, command: fixture,
|
|
...(source === "explicit" ? { environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: key, NO_PROXY: "custody-bypass.invalid" } } : {}) });
|
|
session = await driver.openSession({ runId: "custody", normalizedSessionId: "custody", workingDirectory: root });
|
|
const configText = await readFile(join(root, "custody", "config", "opencode", "opencode.json"), "utf8");
|
|
const config = JSON.parse(configText);
|
|
expect(config.model).toBe(`openrouter/${model}`);
|
|
expect(config.small_model).toBe(`openrouter/${model}`);
|
|
expect(Object.keys(config.provider)).toEqual(["openrouter"]);
|
|
expect(Object.keys(config.provider.openrouter.models)).toEqual([model]);
|
|
const { baseURL, apiKey } = config.provider.openrouter.options;
|
|
expect(configText).not.toContain(key);
|
|
const environmentText = await readFile(join(root, "custody", "data", "fake-environment.json"), "utf8");
|
|
expect(JSON.parse(environmentText).keys).not.toContain("OPENROUTER_API_KEY");
|
|
expect(environmentText).not.toContain(key);
|
|
expect(JSON.parse(environmentText).noProxy).toBe("custody-bypass.invalid,127.0.0.1,localhost");
|
|
const persisted = await readFile(join(root, "custody", "data", "fake-provider-custody.json"), "utf8");
|
|
expect(persisted).not.toContain(key);
|
|
expect(JSON.parse(persisted)).toMatchObject({ providerOptions: { baseURL, apiKey } });
|
|
expect(baseURL).toMatch(/^http:\/\/127\.0\.0\.1:\d+\/v1$/);
|
|
expect(apiKey).not.toBe(key);
|
|
const url = `${baseURL}/chat/completions`;
|
|
const headers = { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" };
|
|
const payload = { model, messages: [{ role: "user", content: "synthetic" }] };
|
|
expect((await fetch(url, { method: "POST", headers, body: JSON.stringify({ ...payload, model: "wrong-model" }) })).status).toBe(400);
|
|
expect(received).toHaveLength(0);
|
|
for (const [field, value] of Object.entries({ models: ["wrong-model"], route: "fallback", plugins: [{ id: "web" }], transforms: ["middle-out"] })) {
|
|
expect((await fetch(url, { method: "POST", headers, body: JSON.stringify({ ...payload, [field]: value }) })).status).toBe(400);
|
|
expect(received).toHaveLength(0);
|
|
}
|
|
expect((await fetch(url, { method: "POST", headers, body: JSON.stringify(payload) })).status).toBe(200);
|
|
expect(received).toEqual([{ authorization: `Bearer ${key}`, body: payload, referer: "https://paperclip.ing" }]);
|
|
expect(upstreamTransport.urls).toEqual(["https://openrouter.ai/api/v1/chat/completions"]);
|
|
redirect = true;
|
|
expect((await fetch(url, { method: "POST", headers, body: JSON.stringify(payload) })).status).toBe(502);
|
|
expect(upstreamTransport.urls).toHaveLength(2);
|
|
await session.close({ reason: "test" });
|
|
session = undefined;
|
|
await expect(fetch(url, { method: "POST", headers, body: JSON.stringify(payload) })).rejects.toThrow();
|
|
} finally {
|
|
await session?.close({ reason: "test" });
|
|
upstreamTransport.endpoint = "";
|
|
vi.unstubAllEnvs();
|
|
await new Promise<void>(resolve => { upstream.close(() => resolve()); upstream.closeAllConnections(); });
|
|
}
|
|
});
|
|
|
|
it.each([["HTTP_PROXY", "file"], ["ALL_PROXY", "directory"]] as const)("uses the runtime's %s, %s trust, and NO_PROXY without changing global transport", async (proxySetting, trust) => {
|
|
const requests: string[] = [];
|
|
const outgoingProxy = createHttpServer((request, response) => {
|
|
requests.push(request.url!);
|
|
request.resume();
|
|
response.writeHead(200, { "content-type": "application/json" }).end('{"choices":[]}');
|
|
});
|
|
await new Promise<void>(resolve => outgoingProxy.listen(0, "127.0.0.1", resolve));
|
|
const address = outgoingProxy.address();
|
|
if (!address || typeof address === "string") throw new Error("Missing proxy fixture address");
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-outgoing-proxy-"));
|
|
roots.push(root);
|
|
const certificateDir = join(root, "certificates");
|
|
await mkdir(certificateDir);
|
|
const certificatePath = join(certificateDir, "runtime-ca.pem");
|
|
await writeFile(certificatePath, getCACertificates("default")[0]!);
|
|
await writeFile(join(certificateDir, "README"), "Non-certificate directory entries must be ignored.");
|
|
const trustEnvironment = trust === "file" ? { SSL_CERT_FILE: certificatePath } : { SSL_CERT_DIR: certificateDir };
|
|
const headersAndUrl = async (sessionId: string, noProxy: string) => {
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "paperclip/team/model-alias", runtimeDirectory: root, command: fixture,
|
|
environment: { ...trustEnvironment, PATH: process.env.PATH, PAPERCLIP_AI_PROVIDER_URL: "http://gateway.invalid/v1", PAPERCLIP_AI_PROVIDER_KEY: "fixture-key", [proxySetting]: `http://127.0.0.1:${address.port}`, NO_PROXY: noProxy },
|
|
});
|
|
const session = await driver.openSession({ runId: sessionId, normalizedSessionId: sessionId, workingDirectory: root });
|
|
const config = JSON.parse(await readFile(join(root, sessionId, "config", "opencode", "opencode.json"), "utf8"));
|
|
return { session, url: `${config.provider.paperclip.options.baseURL}/chat/completions`, headers: { Authorization: `Bearer ${config.provider.paperclip.options.apiKey}`, "Content-Type": "application/json" } };
|
|
};
|
|
try {
|
|
const proxied = await headersAndUrl("proxied", "");
|
|
try {
|
|
const response = await fetch(proxied.url, { method: "POST", headers: proxied.headers, body: JSON.stringify({ model: "team/model-alias", messages: [] }) });
|
|
expect(response.status).toBe(200);
|
|
expect(await response.json()).toEqual({ choices: [] });
|
|
expect(requests).toEqual(["http://gateway.invalid/v1/chat/completions"]);
|
|
} finally { await proxied.session.close({ reason: "test" }); }
|
|
const bypassed = await headersAndUrl("bypassed", "gateway.invalid");
|
|
try {
|
|
const response = await fetch(bypassed.url, { method: "POST", headers: bypassed.headers, body: JSON.stringify({ model: "team/model-alias", messages: [] }) });
|
|
expect(response.status).toBe(502);
|
|
expect(requests).toHaveLength(1);
|
|
} finally { await bypassed.session.close({ reason: "test" }); }
|
|
} finally {
|
|
await new Promise<void>(resolve => { outgoingProxy.close(() => resolve()); outgoingProxy.closeAllConnections(); });
|
|
}
|
|
});
|
|
|
|
it("starts an authenticated isolated server, creates a session, streams usage, aborts, and cleans up", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const tracePath = join(root, "provider-trace.ndjson");
|
|
const spawns: Array<{ pid: number; processGroupId: number | null }> = [];
|
|
const diagnostics: string[] = [];
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "test-openrouter-key",
|
|
PAPERCLIP_RUNTIME_NOTIFICATION_CONTEXT: join(root, "runtime-notifications.json"),
|
|
PAPERCLIP_RUNTIME_NOTIFICATION_HELPER: join(root, "runnerd"),
|
|
PAPERCLIP_API_KEY: "must-not-leak",
|
|
UNRELATED_SECRET: "must-not-leak",
|
|
PAPERCLIP_PROVIDER_TRACE_PATH: tracePath,
|
|
PAPERCLIP_PROVIDER_TRACE_MAX_BYTES: String(64 * 1024 * 1024),
|
|
},
|
|
onSpawn: async (meta) => {
|
|
spawns.push(meta);
|
|
},
|
|
onDiagnostic: (message) => diagnostics.push(message),
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-1",
|
|
normalizedSessionId: "normalized/1",
|
|
workingDirectory: workspace,
|
|
});
|
|
expect(session.ids()).toMatchObject({ providerSessionId: "ses_fake_1" });
|
|
expect(spawns).toHaveLength(1);
|
|
const turn = await session.startTurn({
|
|
message: { role: "user", text: "finish" },
|
|
});
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(events.map((event) => event.eventType)).toContain("turn.completed");
|
|
expect(events).toContainEqual(
|
|
expect.objectContaining({ eventType: "run.result.proposed" }),
|
|
);
|
|
expect(
|
|
events.filter((event) => event.eventType === "item.delta"),
|
|
).toHaveLength(1);
|
|
expect(
|
|
events.find((event) => event.eventType === "item.delta")?.payload.text,
|
|
).toBe("done [guide](guide.md)");
|
|
expect(
|
|
events.find(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "agentMessage",
|
|
)?.payload,
|
|
).toMatchObject({
|
|
channel: "final",
|
|
providerPhase: "final_answer",
|
|
text: "done [guide](guide.md)",
|
|
});
|
|
const usageEvents = events.filter(
|
|
(event) =>
|
|
event.eventType === "item.completed" && event.payload.kind === "usage",
|
|
);
|
|
expect(usageEvents).toHaveLength(1);
|
|
expect(usageEvents[0]?.payload).toMatchObject({
|
|
usageMessageId: expect.any(String),
|
|
});
|
|
expect(
|
|
events.find((event) => event.eventType === "workspace.change.updated")
|
|
?.payload,
|
|
).toMatchObject({
|
|
schema: "paperclip.workspace.diff.v1",
|
|
source: "harness_reported",
|
|
totals: { files: 1 },
|
|
});
|
|
expect(
|
|
events.find((event) => event.eventType === "workspace.diff.recorded")
|
|
?.payload,
|
|
).toMatchObject({ schema: "paperclip.workspace.diff.v1", complete: true });
|
|
expect(
|
|
events.find((event) => event.eventType === "workspace.file.referenced")
|
|
?.payload,
|
|
).toMatchObject({
|
|
schema: "paperclip.workspace.file_reference.v1",
|
|
path: "guide.md",
|
|
});
|
|
expect(JSON.stringify(events)).not.toContain(
|
|
"submitted prompt must not become assistant text",
|
|
);
|
|
expect(await session.usage!()).toMatchObject({
|
|
input: 3,
|
|
output: 2,
|
|
costUsd: 0.001,
|
|
provider: "openrouter",
|
|
driverVersion: "1.18.34",
|
|
});
|
|
await session.interrupt?.({ turnId: turn.turnId });
|
|
const snapshot = await session.snapshot();
|
|
expect(snapshot.driverKind).toBe("opencode_server");
|
|
const sessionRoot = join(root, "normalized_1");
|
|
expect((await stat(sessionRoot)).mode & 0o777).toBe(0o700);
|
|
expect(
|
|
(await stat(join(sessionRoot, "config", "opencode", "opencode.json")))
|
|
.mode & 0o777,
|
|
).toBe(0o600);
|
|
const config = await readFile(
|
|
join(sessionRoot, "config", "opencode", "opencode.json"),
|
|
"utf8",
|
|
);
|
|
await session.close({ reason: "test" });
|
|
const recovered = await driver.recoverSession?.(snapshot);
|
|
expect(recovered).toMatchObject({ recovered: true });
|
|
await expect(recovered?.session?.read?.()).resolves.toMatchObject({
|
|
sessionId: "ses_fake_1",
|
|
messages: [],
|
|
});
|
|
await recovered?.session?.close({ reason: "recovery-test" });
|
|
|
|
const traceEntries = (await readFile(tracePath, "utf8"))
|
|
.trim()
|
|
.split("\n")
|
|
.map((line) => JSON.parse(line) as Record<string, unknown>);
|
|
expect(JSON.stringify(traceEntries)).not.toContain("test-openrouter-key");
|
|
const traceFrames = traceEntries.filter((entry) => entry.kind === "frame");
|
|
expect(traceFrames.map((entry) => entry.direction)).toEqual(
|
|
expect.arrayContaining([
|
|
"client_to_provider",
|
|
"provider_to_client",
|
|
"provider_stderr",
|
|
]),
|
|
);
|
|
expect(
|
|
traceFrames.some((entry) => entry.nativeMethod === "SSE /event"),
|
|
).toBe(true);
|
|
expect(
|
|
traceEntries.some(
|
|
(entry) =>
|
|
entry.stage === "typescript_opencode_driver_normalization" &&
|
|
Array.isArray(entry.emittedEventIds) &&
|
|
entry.emittedEventIds.length > 0,
|
|
),
|
|
).toBe(true);
|
|
expect(traceEntries.at(-1)).toMatchObject({
|
|
kind: "trace_status",
|
|
debugChannel: "typescript_opencode_native",
|
|
status: "complete",
|
|
});
|
|
|
|
const environment = JSON.parse(
|
|
await readFile(
|
|
join(sessionRoot, "data", "fake-environment.json"),
|
|
"utf8",
|
|
),
|
|
);
|
|
const mcpEvidence = JSON.parse(
|
|
await readFile(
|
|
join(sessionRoot, "data", "fake-mcp-evidence.json"),
|
|
"utf8",
|
|
),
|
|
);
|
|
expect(environment.notificationContext).toBe(join(root, "runtime-notifications.json"));
|
|
expect(environment.notificationHelper).toBe(join(root, "runnerd"));
|
|
expect(environment.keys).not.toContain("OPENROUTER_API_KEY");
|
|
expect(environment.keys).not.toContain("PAPERCLIP_API_KEY");
|
|
expect(environment.keys).not.toContain("UNRELATED_SECRET");
|
|
expect(environment.keys).not.toContain("PAPERCLIP_PROVIDER_TRACE_PATH");
|
|
expect(environment.projectConfigDisabled).toBe("true");
|
|
expect(environment.modelsFetchDisabled).toBe("true");
|
|
expect(environment.defaultPluginsDisabled).toBe("true");
|
|
expect(mcpEvidence.tools).toEqual(
|
|
expect.arrayContaining(["paperclip_finish", "paperclip_block"]),
|
|
);
|
|
expect(config).toContain("openrouter/deepseek/deepseek-v4-flash-0731");
|
|
expect(config).toContain('"*": "allow"');
|
|
expect(JSON.parse(config).permission.external_directory).toMatchObject({ "*": "deny", [`${workspace}/**`]: "allow" });
|
|
expect(
|
|
events.some((event) => event.eventType === "runtime_request.created"),
|
|
).toBe(false);
|
|
expect(config).not.toContain("test-openrouter-key");
|
|
expect(diagnostics.join("\n")).not.toContain("test-openrouter-key");
|
|
expect(diagnostics.join("\n")).toContain("[REDACTED]");
|
|
});
|
|
|
|
it.each(["role-first", "role-last", "role-last-long"])("preserves incremental text and completion with identity checks (%s)", async (roleOrder) => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-stream-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-stream-workspace-"));
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: { PATH: process.env.PATH },
|
|
});
|
|
const session = await driver.openSession({ runId: "stream", normalizedSessionId: "stream", workingDirectory: workspace });
|
|
try {
|
|
await session.startTurn({ message: { role: "user", text: `incremental-stream ${roleOrder}` } });
|
|
const deltas: string[] = [];
|
|
const reasoning: string[] = [];
|
|
const events: PrpEvent[] = [];
|
|
for await (const event of session.events()) {
|
|
events.push(event);
|
|
if (event.eventType === "item.delta" && event.payload.kind === "reasoning") reasoning.push(String(event.payload.text));
|
|
if (event.eventType === "item.delta" && event.payload.kind === "agentMessage") {
|
|
deltas.push(String(event.payload.text));
|
|
// The fixture cannot finish its turn until a chunk reaches us.
|
|
if (deltas.length === 1) {
|
|
expect(await readFile(join(root, "stream", "data", "stream-released"), "utf8").catch(() => null)).toBeNull();
|
|
}
|
|
await writeFile(join(root, "stream", "data", "release-stream"), "ready");
|
|
}
|
|
if (TURN_TERMINAL_EVENT_TYPES.has(event.eventType)) break;
|
|
}
|
|
const expectedText = `Hi ${"👋".repeat(roleOrder === "role-last-long" ? 120 : 2)}`;
|
|
expect(deltas).toEqual(roleOrder === "role-first" ? ["Hi ", "👋", "👋"] : [expectedText]);
|
|
expect(reasoning).toEqual(["Thinking"]);
|
|
expect(events.filter(event => event.eventType === "item.completed" && event.payload.kind === "agentMessage")
|
|
.map(event => event.payload.text)).toEqual([expectedText]);
|
|
} finally {
|
|
await session.close({ reason: "stream test complete" });
|
|
}
|
|
});
|
|
|
|
it("maps OpenCode's normal abort error to cancellation without a false provider failure notice", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-abort-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-abort-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-aborted",
|
|
normalizedSessionId: "aborted",
|
|
workingDirectory: workspace,
|
|
});
|
|
await session.startTurn({
|
|
message: { role: "user", text: "session-aborted" },
|
|
});
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(events.map((event) => event.eventType)).toContain("turn.cancelled");
|
|
expect(events.map((event) => event.eventType)).not.toContain("turn.failed");
|
|
expect(events.map((event) => event.eventType)).not.toContain(
|
|
"provider.notice.recorded",
|
|
);
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("runs two consecutive turns on one session and delivers exactly one terminal event per turn", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-multi-turn-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-multi-turn-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-multi-turn",
|
|
normalizedSessionId: "multi-turn",
|
|
workingDirectory: workspace,
|
|
});
|
|
|
|
const firstTurn = await session.startTurn({
|
|
message: { role: "user", text: "finish" },
|
|
});
|
|
const firstTurnEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
firstTurnEvents.filter((event) => event.eventType === "turn.completed"),
|
|
).toMatchObject([{ turnId: firstTurn.turnId }]);
|
|
expect(
|
|
firstTurnEvents.find(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "agentMessage",
|
|
)?.payload,
|
|
).toMatchObject({ text: "done [guide](guide.md)" });
|
|
|
|
const secondTurn = await session.startTurn({
|
|
message: { role: "user", text: "finish" },
|
|
});
|
|
expect(secondTurn.turnId).not.toBe(firstTurn.turnId);
|
|
const secondTurnEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
secondTurnEvents.filter((event) => event.eventType === "turn.completed"),
|
|
).toMatchObject([{ turnId: secondTurn.turnId }]);
|
|
expect(
|
|
secondTurnEvents.find(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "agentMessage",
|
|
)?.payload,
|
|
).toMatchObject({ text: "done [guide](guide.md)" });
|
|
expect(
|
|
secondTurnEvents.some((event) => event.turnId === firstTurn.turnId),
|
|
).toBe(false);
|
|
expect(
|
|
secondTurnEvents.some((event) => event.eventType === "harness.diagnostic"),
|
|
).toBe(false);
|
|
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("drops a distinct late frame for a completed turn instead of attributing it to the next turn", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-late-frame-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-late-frame-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-late-frame",
|
|
normalizedSessionId: "late-frame",
|
|
workingDirectory: workspace,
|
|
});
|
|
|
|
const firstTurn = await session.startTurn({
|
|
message: { role: "user", text: "late-straggler-source" },
|
|
});
|
|
const firstTurnEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
firstTurnEvents.filter((event) => event.eventType === "turn.completed"),
|
|
).toMatchObject([{ turnId: firstTurn.turnId }]);
|
|
|
|
// The fixture holds a distinct frame for the first turn's message and
|
|
// delivers it only once this second turn's prompt has been accepted,
|
|
// simulating a provider frame that arrives after its own turn is
|
|
// already sealed.
|
|
const secondTurn = await session.startTurn({
|
|
message: { role: "user", text: "finish" },
|
|
});
|
|
expect(secondTurn.turnId).not.toBe(firstTurn.turnId);
|
|
const secondTurnEvents = await collectTurnEvents(session.events());
|
|
|
|
expect(
|
|
secondTurnEvents.some((event) =>
|
|
JSON.stringify(event.payload).includes(
|
|
"late straggler text must not reach the next turn",
|
|
),
|
|
),
|
|
).toBe(false);
|
|
expect(
|
|
secondTurnEvents.some((event) => event.turnId === firstTurn.turnId),
|
|
).toBe(false);
|
|
expect(
|
|
secondTurnEvents.filter((event) => event.eventType === "turn.completed"),
|
|
).toMatchObject([{ turnId: secondTurn.turnId }]);
|
|
expect(
|
|
secondTurnEvents.find(
|
|
(event) => event.eventType === "harness.diagnostic",
|
|
)?.payload,
|
|
).toMatchObject({
|
|
code: "opencode_late_terminal_turn_event_dropped",
|
|
turnId: firstTurn.turnId,
|
|
});
|
|
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("keeps rejecting a late frame for a sealed turn no matter how many later turns have already sealed", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-sealed-many-turns-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-sealed-many-turns-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-sealed-many-turns",
|
|
normalizedSessionId: "sealed-many-turns",
|
|
workingDirectory: workspace,
|
|
});
|
|
|
|
// Turn A schedules its own late frame to arrive four prompts later, so
|
|
// it lands only after three more turns have sealed. The late-frame gate
|
|
// in `#emit` compares directly against the current active turn, not a
|
|
// bounded history, so it must still reject this frame no matter how
|
|
// many turns sealed in between.
|
|
const turnA = await session.startTurn({
|
|
message: { role: "user", text: "late-straggler-source-delay-4" },
|
|
});
|
|
await collectTurnEvents(session.events());
|
|
|
|
for (let index = 0; index < 3; index += 1) {
|
|
await session.startTurn({ message: { role: "user", text: "finish" } });
|
|
await collectTurnEvents(session.events());
|
|
}
|
|
|
|
// This turn's prompt request is what finally delivers turn A's scheduled
|
|
// late frame, well after turn A's own turn sealed.
|
|
const turnE = await session.startTurn({
|
|
message: { role: "user", text: "finish" },
|
|
});
|
|
const turnEEvents = await collectTurnEvents(session.events());
|
|
|
|
expect(
|
|
turnEEvents.some((event) =>
|
|
JSON.stringify(event.payload).includes(
|
|
"late straggler text must not reach the next turn",
|
|
),
|
|
),
|
|
).toBe(false);
|
|
expect(turnEEvents.some((event) => event.turnId === turnA.turnId)).toBe(
|
|
false,
|
|
);
|
|
expect(
|
|
turnEEvents.find((event) => event.eventType === "harness.diagnostic")
|
|
?.payload,
|
|
).toMatchObject({
|
|
code: "opencode_late_terminal_turn_event_dropped",
|
|
turnId: turnA.turnId,
|
|
});
|
|
expect(
|
|
turnEEvents.filter((event) => event.eventType === "turn.completed"),
|
|
).toMatchObject([{ turnId: turnE.turnId }]);
|
|
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("delivers the settlement event for a runtime request whose turn fails through the same single-pass consumer that reads the turn", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-pending-then-fail-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-pending-then-fail-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-pending-then-fail",
|
|
normalizedSessionId: "pending-then-fail",
|
|
workingDirectory: workspace,
|
|
});
|
|
|
|
const { turnId } = await session.startTurn({
|
|
message: { role: "user", text: "pending-request-then-turn-fails" },
|
|
});
|
|
// The fixture asks a native question, which leaves a runtime request
|
|
// pending, then fails the same turn through `session.error` without
|
|
// ever resolving that request. `collectTurnEvents` reads exactly one
|
|
// pass and stops at the turn's terminal event, the same rule the
|
|
// production consumer applies (see its doc comment above). The
|
|
// settlement event must arrive inside that same pass: a production
|
|
// consumer that stops at `turn.failed` never opens a second read
|
|
// afterward, so a settlement event that only `close()` produced later
|
|
// would never reach it.
|
|
const turnEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
turnEvents.some(
|
|
(event) => event.eventType === "runtime_request.created",
|
|
),
|
|
).toBe(true);
|
|
const settlementIndex = turnEvents.findIndex(
|
|
(event) => event.eventType === "runtime_request.expired",
|
|
);
|
|
const terminalIndex = turnEvents.findIndex(
|
|
(event) => event.eventType === "turn.failed",
|
|
);
|
|
expect(settlementIndex).toBeGreaterThanOrEqual(0);
|
|
expect(terminalIndex).toBeGreaterThanOrEqual(0);
|
|
// The settlement fact must precede the turn's terminal event, or a
|
|
// consumer that stops reading at that terminal event misses it.
|
|
expect(settlementIndex).toBeLessThan(terminalIndex);
|
|
expect(turnEvents[settlementIndex]).toMatchObject({
|
|
turnId,
|
|
itemId: "question-native-1",
|
|
});
|
|
expect(turnEvents[terminalIndex]).toMatchObject({ turnId });
|
|
expect(
|
|
turnEvents.some(
|
|
(event) =>
|
|
event.eventType === "harness.diagnostic" &&
|
|
event.payload.code === "opencode_late_terminal_turn_event_dropped" &&
|
|
event.payload.turnId === turnId,
|
|
),
|
|
).toBe(false);
|
|
// The request already settled with the turn; nothing is left pending
|
|
// for `close()` to settle a second time.
|
|
expect(session.pendingRuntimeRequests?.()).toHaveLength(0);
|
|
|
|
await session.close({ reason: "test" });
|
|
const closeEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
closeEvents.some((event) =>
|
|
["runtime_request.expired", "runtime_request.cancelled"].includes(
|
|
event.eventType,
|
|
),
|
|
),
|
|
).toBe(false);
|
|
});
|
|
|
|
it("keeps the session usable after a cancelled turn so the next turn on the same session still completes", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-cancel-then-finish-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-cancel-then-finish-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-cancel-then-finish",
|
|
normalizedSessionId: "cancel-then-finish",
|
|
workingDirectory: workspace,
|
|
});
|
|
|
|
const cancelledTurn = await session.startTurn({
|
|
message: { role: "user", text: "session-aborted" },
|
|
});
|
|
const cancelledTurnEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
cancelledTurnEvents.filter(
|
|
(event) => event.eventType === "turn.cancelled",
|
|
),
|
|
).toMatchObject([{ turnId: cancelledTurn.turnId }]);
|
|
|
|
const secondTurn = await session.startTurn({
|
|
message: { role: "user", text: "finish" },
|
|
});
|
|
const secondTurnEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
secondTurnEvents.filter((event) => event.eventType === "turn.completed"),
|
|
).toMatchObject([{ turnId: secondTurn.turnId }]);
|
|
expect(
|
|
secondTurnEvents.some((event) => event.turnId === cancelledTurn.turnId),
|
|
).toBe(false);
|
|
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("keeps the session usable after a failed turn so the next turn on the same session still completes", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-fail-then-finish-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-fail-then-finish-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-fail-then-finish",
|
|
normalizedSessionId: "fail-then-finish",
|
|
workingDirectory: workspace,
|
|
});
|
|
|
|
const failedTurn = await session.startTurn({
|
|
message: { role: "user", text: "session-failed" },
|
|
});
|
|
const failedTurnEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
failedTurnEvents.filter((event) => event.eventType === "turn.failed"),
|
|
).toMatchObject([{ turnId: failedTurn.turnId }]);
|
|
|
|
const secondTurn = await session.startTurn({
|
|
message: { role: "user", text: "finish" },
|
|
});
|
|
const secondTurnEvents = await collectTurnEvents(session.events());
|
|
expect(
|
|
secondTurnEvents.filter((event) => event.eventType === "turn.completed"),
|
|
).toMatchObject([{ turnId: secondTurn.turnId }]);
|
|
expect(
|
|
secondTurnEvents.some((event) => event.turnId === failedTurn.turnId),
|
|
).toBe(false);
|
|
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("still ends the event stream once the session closes", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-close-ends-stream-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-close-ends-stream-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-close-ends-stream",
|
|
normalizedSessionId: "close-ends-stream",
|
|
workingDirectory: workspace,
|
|
});
|
|
|
|
await session.startTurn({ message: { role: "user", text: "finish" } });
|
|
await collectTurnEvents(session.events());
|
|
await session.close({ reason: "test" });
|
|
|
|
const eventsAfterClose: PrpEvent[] = [];
|
|
for await (const event of session.events()) eventsAfterClose.push(event);
|
|
expect(eventsAfterClose).toEqual([]);
|
|
});
|
|
|
|
it("keeps OpenCode in an outer supervisor process group when requested", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
let processGroupId: number | null | undefined;
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
isolateProcessGroup: false,
|
|
onSpawn: async (meta) => {
|
|
processGroupId = meta.processGroupId;
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-supervised",
|
|
normalizedSessionId: "supervised",
|
|
workingDirectory: workspace,
|
|
});
|
|
expect(processGroupId).toBeNull();
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it.each([false, true])("uses the native system channel, selected skill tree, instruction root, and assigned MCP gateway after startup retry=%s", async (retryStartup) => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-context-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-context-workspace-"),
|
|
);
|
|
const skillRoot = join(root, "skill-source");
|
|
const instructionRoot = join(root, "instruction-source");
|
|
roots.push(root, workspace);
|
|
await Promise.all([
|
|
mkdir(join(skillRoot, "references"), { recursive: true }),
|
|
mkdir(instructionRoot, { recursive: true }),
|
|
]);
|
|
await Promise.all([
|
|
writeFile(
|
|
join(skillRoot, "SKILL.md"),
|
|
"# Assigned\nRead references/support.md\n",
|
|
),
|
|
writeFile(join(skillRoot, "references", "support.md"), "skill support\n"),
|
|
writeFile(join(instructionRoot, "AGENTS.md"), "Read sibling.md\n"),
|
|
writeFile(join(instructionRoot, "sibling.md"), "instruction sibling\n"),
|
|
]);
|
|
const systemInstructions = `${PAPERCLIP_EXECUTION_PROMPT}\n\nRead sibling.md\n\nRead-only instruction sibling root: ${instructionRoot}`;
|
|
let submittedPrompt: Record<string, unknown> | null = null;
|
|
let startupFailurePending = retryStartup;
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
systemInstructions,
|
|
runtimeContext: runtimeContext(skillRoot, instructionRoot),
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
PAPERCLIP_NATIVE_MCP_NAME: "paperclip-assigned",
|
|
PAPERCLIP_NATIVE_MCP_URL: "https://paperclip.example/mcp",
|
|
PAPERCLIP_NATIVE_MCP_TOKEN: "x".repeat(40),
|
|
},
|
|
fetch: async (input, init) => {
|
|
if (startupFailurePending && String(input).endsWith("/session") && init?.method === "POST") {
|
|
startupFailurePending = false;
|
|
return new Response("Fixture startup failure", { status: 503 });
|
|
}
|
|
if (String(input).endsWith("/prompt_async"))
|
|
submittedPrompt = JSON.parse(String(init?.body));
|
|
return fetch(input, init);
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-context",
|
|
normalizedSessionId: "context",
|
|
workingDirectory: workspace,
|
|
});
|
|
await session.startTurn({ message: { role: "user", text: "finish" } });
|
|
for await (const event of session.events()) {
|
|
if (event.eventType === "turn.completed") break;
|
|
}
|
|
expect(submittedPrompt).toMatchObject({
|
|
system: systemInstructions,
|
|
});
|
|
// A prompt tools map replaces OpenCode's session permissions. Question is
|
|
// enabled in config without overwriting the allow/ask/deny or path policy.
|
|
expect(submittedPrompt).not.toHaveProperty("tools");
|
|
expect(JSON.stringify((submittedPrompt as Record<string, unknown> | null)?.parts ?? null)).not.toContain(
|
|
PAPERCLIP_EXECUTION_PROMPT,
|
|
);
|
|
const sessionRoot = join(root, "context");
|
|
const runtimeEnvironment = JSON.parse(
|
|
await readFile(join(sessionRoot, "data", "fake-environment.json"), "utf8"),
|
|
);
|
|
const isolatedHome = runtimeEnvironment.home;
|
|
expect(startupFailurePending).toBe(false);
|
|
expect(dirname(isolatedHome)).toBe(join(sessionRoot, "homes"));
|
|
expect(basename(isolatedHome)).toMatch(/^home-/);
|
|
await expect(
|
|
readFile(
|
|
join(
|
|
isolatedHome,
|
|
".claude",
|
|
"skills",
|
|
"assigned",
|
|
"references",
|
|
"support.md",
|
|
),
|
|
"utf8",
|
|
),
|
|
).resolves.toBe("skill support\n");
|
|
const config = JSON.parse(
|
|
await readFile(
|
|
join(sessionRoot, "config", "opencode", "opencode.json"),
|
|
"utf8",
|
|
),
|
|
);
|
|
expect(config).toMatchObject({
|
|
instructions: [],
|
|
plugin: [],
|
|
tools: { question: true },
|
|
permission: {
|
|
external_directory: { "*": "deny", [`${instructionRoot}/**`]: "allow" },
|
|
},
|
|
mcp: {
|
|
paperclip: { enabled: true },
|
|
"paperclip-assigned": {
|
|
url: "https://paperclip.example/mcp",
|
|
enabled: true,
|
|
},
|
|
},
|
|
});
|
|
await expect(
|
|
readFile(join(instructionRoot, "sibling.md"), "utf8"),
|
|
).resolves.toBe("instruction sibling\n");
|
|
await session.close({ reason: "test" });
|
|
await expect(readdir(join(sessionRoot, "homes"))).resolves.toEqual([]);
|
|
});
|
|
|
|
it("removes isolated homes left by earlier launches when a session reopens", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-legacy-home-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-legacy-home-workspace-"),
|
|
);
|
|
const skillRoot = join(root, "skill-source");
|
|
const instructionRoot = join(root, "instruction-source");
|
|
roots.push(root, workspace);
|
|
await Promise.all([
|
|
mkdir(skillRoot, { recursive: true }),
|
|
mkdir(instructionRoot, { recursive: true }),
|
|
]);
|
|
await writeFile(join(skillRoot, "SKILL.md"), "# Assigned\n");
|
|
await writeFile(join(instructionRoot, "AGENTS.md"), "instructions\n");
|
|
const sessionRoot = join(root, "legacy");
|
|
const legacyHome = join(sessionRoot, "home-AbC123");
|
|
const sealedSkill = join(legacyHome, ".claude", "skills", "assigned");
|
|
await mkdir(sealedSkill, { recursive: true });
|
|
await mkdir(join(legacyHome, ".npm", "_cacache"), { recursive: true });
|
|
await writeFile(join(legacyHome, ".npm", "_cacache", "entry"), "cache");
|
|
await writeFile(join(sealedSkill, "SKILL.md"), "# Assigned\n");
|
|
await chmod(join(sealedSkill, "SKILL.md"), 0o444);
|
|
await chmod(sealedSkill, 0o555);
|
|
await mkdir(join(sessionRoot, "data", "opencode"), { recursive: true });
|
|
await writeFile(join(sessionRoot, "data", "opencode", "opencode.db"), "history");
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
runtimeContext: runtimeContext(skillRoot, instructionRoot),
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-legacy-home",
|
|
normalizedSessionId: "legacy",
|
|
workingDirectory: workspace,
|
|
});
|
|
await expect(lstat(legacyHome)).rejects.toMatchObject({ code: "ENOENT" });
|
|
await session.close({ reason: "test" });
|
|
expect(
|
|
(await readdir(sessionRoot)).filter((entry) => entry.startsWith("home-")),
|
|
).toEqual([]);
|
|
await expect(readdir(join(sessionRoot, "homes"))).resolves.toEqual([]);
|
|
await expect(
|
|
readFile(join(sessionRoot, "data", "opencode", "opencode.db"), "utf8"),
|
|
).resolves.toBe("history");
|
|
});
|
|
|
|
it("resends system instructions without repeating the task envelope on recovery", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-resume-context-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-resume-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
let submittedPrompt: Record<string, unknown> | null = null;
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
systemInstructions: "large original system context",
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
fetch: async (input, init) => {
|
|
if (String(input).endsWith("/prompt_async"))
|
|
submittedPrompt = JSON.parse(String(init?.body));
|
|
return fetch(input, init);
|
|
},
|
|
});
|
|
const seed = await driver.openSession({
|
|
runId: "run-seed",
|
|
normalizedSessionId: "resume-context",
|
|
workingDirectory: workspace,
|
|
});
|
|
const snapshot = await seed.snapshot();
|
|
await seed.close({ reason: "seed complete" });
|
|
|
|
const recovered = await driver.recoverSession?.(snapshot);
|
|
expect(recovered).toMatchObject({ recovered: true });
|
|
await recovered!.session!.startTurn({
|
|
message: {
|
|
role: "user",
|
|
text: '{"interactionResponses":[{"answer":"friendly"}]}',
|
|
},
|
|
});
|
|
for await (const event of recovered!.session!.events()) {
|
|
if (event.eventType === "turn.completed") break;
|
|
}
|
|
|
|
expect(submittedPrompt).toMatchObject({
|
|
providerID: "openrouter",
|
|
modelID: "deepseek/deepseek-v4-flash-0731",
|
|
parts: [
|
|
{
|
|
type: "text",
|
|
text: '{"interactionResponses":[{"answer":"friendly"}]}',
|
|
},
|
|
],
|
|
});
|
|
expect(submittedPrompt).not.toHaveProperty("tools");
|
|
expect(submittedPrompt).toHaveProperty("system", "large original system context");
|
|
await recovered!.session!.close({ reason: "recovery-test" });
|
|
});
|
|
|
|
it("preserves omitted native custom defaults and replies with ordered OpenCode answer arrays", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-question-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-question-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-native-question",
|
|
normalizedSessionId: "native-question",
|
|
workingDirectory: workspace,
|
|
});
|
|
const { turnId } = await session.startTurn({
|
|
message: { role: "user", text: "native-question" },
|
|
});
|
|
const iterator = session.events()[Symbol.asyncIterator]();
|
|
let requestEvent:
|
|
Awaited<ReturnType<typeof iterator.next>>["value"] | null = null;
|
|
for (let count = 0; count < 30; count += 1) {
|
|
const event = await iterator.next();
|
|
if (event.done) break;
|
|
if (event.value.eventType === "runtime_request.created") {
|
|
requestEvent = event.value;
|
|
break;
|
|
}
|
|
}
|
|
expect(requestEvent?.payload).toMatchObject({
|
|
request: {
|
|
schema: "paperclip.runtime_request.v2",
|
|
requestKind: "runtime",
|
|
type: "input",
|
|
input: {
|
|
schema: "paperclip.question_set.v1",
|
|
questions: [
|
|
{
|
|
id: "environment",
|
|
answerMode: "single_select",
|
|
required: false,
|
|
customAnswer: { enabled: true },
|
|
},
|
|
{ id: "regions", answerMode: "multi_select", required: false },
|
|
],
|
|
},
|
|
},
|
|
});
|
|
await new Promise((resolveDelay) => setTimeout(resolveDelay, 20));
|
|
expect(session.pendingRuntimeRequests?.()).toHaveLength(1);
|
|
await session.resolveRuntimeRequest?.({
|
|
requestId: "question-native-1",
|
|
turnId,
|
|
resolution: {
|
|
action: "submit",
|
|
response: {
|
|
schema: "paperclip.question_response.v1",
|
|
answers: {
|
|
environment: { customText: "Canary" },
|
|
regions: { selectedOptionIds: ["option-1", "option-2"] },
|
|
},
|
|
},
|
|
},
|
|
});
|
|
const reply = JSON.parse(
|
|
await readFile(
|
|
join(root, "native-question", "data", "fake-question-reply.json"),
|
|
"utf8",
|
|
),
|
|
);
|
|
expect(reply).toEqual({
|
|
url: expect.stringContaining(
|
|
`directory=${encodeURIComponent(workspace)}`,
|
|
),
|
|
body: { answers: [["Canary"], ["US", "EU"]] },
|
|
});
|
|
const terminalQuestionEvents =
|
|
(await session.transcript?.())?.events.filter(
|
|
(event) =>
|
|
event.payload.requestId === "question-native-1" &&
|
|
["runtime_request.resolved", "runtime_request.cancelled"].includes(
|
|
event.eventType,
|
|
),
|
|
) ?? [];
|
|
expect(terminalQuestionEvents).toHaveLength(1);
|
|
expect(terminalQuestionEvents[0]?.payload).toMatchObject({
|
|
action: "submit",
|
|
response: {
|
|
schema: "paperclip.question_response.v1",
|
|
answers: {
|
|
environment: { customText: "Canary" },
|
|
regions: { selectedOptionIds: ["option-1", "option-2"] },
|
|
},
|
|
},
|
|
});
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("hands a native question to the durable interaction path without touching permissions", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-question-handoff-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-question-handoff-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-native-question-handoff",
|
|
normalizedSessionId: "native-question-handoff",
|
|
workingDirectory: workspace,
|
|
});
|
|
const { turnId } = await session.startTurn({
|
|
message: { role: "user", text: "native-question" },
|
|
});
|
|
const iterator = session.events()[Symbol.asyncIterator]();
|
|
let created: Awaited<ReturnType<typeof iterator.next>>["value"] | null =
|
|
null;
|
|
for (let count = 0; count < 30; count += 1) {
|
|
const next = await iterator.next();
|
|
if (next.done) break;
|
|
if (next.value.eventType === "runtime_request.created") {
|
|
created = next.value;
|
|
break;
|
|
}
|
|
}
|
|
expect(created).toMatchObject({
|
|
payload: { request: { requestId: "question-native-1", type: "input" } },
|
|
});
|
|
|
|
const firstHandoff = session.handoffRuntimeRequest?.({
|
|
requestId: "question-native-1",
|
|
turnId,
|
|
reason: "durable_handoff",
|
|
signal: new AbortController().signal,
|
|
});
|
|
expect(firstHandoff?.result).toBe("handed_off");
|
|
await firstHandoff?.cleanup;
|
|
const repeatedHandoff = session.handoffRuntimeRequest?.({
|
|
requestId: "question-native-1",
|
|
turnId,
|
|
reason: "durable_handoff",
|
|
signal: new AbortController().signal,
|
|
});
|
|
expect(repeatedHandoff?.result).toBe("already_settled");
|
|
await repeatedHandoff?.cleanup;
|
|
|
|
let expired: Awaited<ReturnType<typeof iterator.next>>["value"] | null =
|
|
null;
|
|
for (let count = 0; count < 20; count += 1) {
|
|
const next = await iterator.next();
|
|
if (next.done) break;
|
|
if (next.value.eventType === "runtime_request.expired") {
|
|
expired = next.value;
|
|
break;
|
|
}
|
|
}
|
|
expect(expired).toMatchObject({
|
|
payload: {
|
|
requestId: "question-native-1",
|
|
reason: "durable_handoff",
|
|
replayAllowed: false,
|
|
requestType: "input",
|
|
request: {
|
|
schema: "paperclip.runtime_request.v2",
|
|
requestId: "question-native-1",
|
|
type: "input",
|
|
},
|
|
},
|
|
});
|
|
expect(session.pendingRuntimeRequests?.()).toEqual([]);
|
|
const terminal = (await session.transcript?.())?.events.filter(
|
|
(event) =>
|
|
event.payload.requestId === "question-native-1" &&
|
|
[
|
|
"runtime_request.resolved",
|
|
"runtime_request.cancelled",
|
|
"runtime_request.expired",
|
|
].includes(event.eventType),
|
|
);
|
|
expect(terminal).toHaveLength(1);
|
|
await session.close({ reason: "handoff test complete" });
|
|
});
|
|
|
|
it("recovers a missed pending question from the list endpoint and rejects it", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-question-recovery-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-question-recovery-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const normalizedSessionId = "question-recovery";
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const seed = await driver.openSession({
|
|
runId: "run-question-recovery",
|
|
normalizedSessionId,
|
|
workingDirectory: workspace,
|
|
});
|
|
const snapshot = await seed.snapshot();
|
|
await seed.close({ reason: "simulate reconnect" });
|
|
const dataRoot = join(root, normalizedSessionId, "data");
|
|
await mkdir(dataRoot, { recursive: true });
|
|
await writeFile(
|
|
join(dataRoot, "fake-pending-question.json"),
|
|
JSON.stringify({
|
|
id: "question-native-1",
|
|
sessionID: "ses_fake_1",
|
|
questions: [
|
|
{
|
|
id: "environment",
|
|
header: "Environment",
|
|
question: "Where should we deploy?",
|
|
options: [{ label: "Staging" }, { label: "Production" }],
|
|
custom: true,
|
|
},
|
|
],
|
|
}),
|
|
);
|
|
|
|
const recovered = await driver.recoverSession?.({
|
|
...snapshot,
|
|
activeTurnId: "turn-recovered-question",
|
|
});
|
|
expect(recovered).toMatchObject({ recovered: true });
|
|
const session = recovered!.session!;
|
|
const iterator = session.events()[Symbol.asyncIterator]();
|
|
let created: Awaited<ReturnType<typeof iterator.next>>["value"] | null =
|
|
null;
|
|
for (let count = 0; count < 20; count += 1) {
|
|
const next = await iterator.next();
|
|
if (next.done) break;
|
|
if (next.value.eventType === "runtime_request.created") {
|
|
created = next.value;
|
|
break;
|
|
}
|
|
}
|
|
expect(created).toMatchObject({
|
|
payload: { request: { requestId: "question-native-1", type: "input" } },
|
|
});
|
|
await session.resolveRuntimeRequest?.({
|
|
requestId: "question-native-1",
|
|
turnId: "turn-recovered-question",
|
|
resolution: { action: "decline" },
|
|
});
|
|
expect(session.pendingRuntimeRequests?.()).toEqual([]);
|
|
await session.close({ reason: "recovery test complete" });
|
|
});
|
|
|
|
it.each([
|
|
{ style: "legacy", action: "accept" as const, reply: "once" },
|
|
{ style: "v2", action: "accept_for_session" as const, reply: "always" },
|
|
{ style: "v2", action: "decline" as const, reply: "reject" },
|
|
])(
|
|
"normalizes $style permission events and maps $action to $reply",
|
|
async ({ style, action, reply }) => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), `paperclip-opencode-permission-${style}-`),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), `paperclip-opencode-permission-${style}-workspace-`),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
permissionMode: "ask",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const normalizedSessionId = `native-permission-${style}-${action}`;
|
|
const session = await driver.openSession({
|
|
runId: `run-${normalizedSessionId}`,
|
|
normalizedSessionId,
|
|
workingDirectory: workspace,
|
|
});
|
|
const { turnId } = await session.startTurn({
|
|
message: { role: "user", text: `native-permission-${style}` },
|
|
});
|
|
const iterator = session.events()[Symbol.asyncIterator]();
|
|
let created: Awaited<ReturnType<typeof iterator.next>>["value"] | null =
|
|
null;
|
|
for (let count = 0; count < 30; count += 1) {
|
|
const next = await iterator.next();
|
|
if (next.done) break;
|
|
if (next.value.eventType === "runtime_request.created") {
|
|
created = next.value;
|
|
break;
|
|
}
|
|
}
|
|
expect(created).toMatchObject({
|
|
payload: {
|
|
request: {
|
|
requestId: "permission-native-1",
|
|
type: "permission",
|
|
actions: ["accept", "accept_for_session", "decline", "cancel"],
|
|
},
|
|
},
|
|
});
|
|
await session.resolveRuntimeRequest?.({
|
|
requestId: "permission-native-1",
|
|
turnId,
|
|
resolution: { action },
|
|
});
|
|
const persisted = JSON.parse(
|
|
await readFile(
|
|
join(root, normalizedSessionId, "data", "fake-permission-reply.json"),
|
|
"utf8",
|
|
),
|
|
);
|
|
expect(persisted).toEqual({
|
|
url: expect.stringContaining(
|
|
`directory=${encodeURIComponent(workspace)}`,
|
|
),
|
|
body: { reply },
|
|
});
|
|
expect(session.pendingRuntimeRequests?.()).toEqual([]);
|
|
await session.close({ reason: "permission test complete" });
|
|
},
|
|
);
|
|
|
|
it.each(["allow", "ask", "deny"] as const)(
|
|
"pins the %s permission mode while retaining external-directory denial",
|
|
async (permissionMode) => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), `paperclip-opencode-mode-${permissionMode}-`),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), `paperclip-opencode-mode-${permissionMode}-workspace-`),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
permissionMode,
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: `run-${permissionMode}`,
|
|
normalizedSessionId: permissionMode,
|
|
workingDirectory: workspace,
|
|
});
|
|
const config = JSON.parse(
|
|
await readFile(
|
|
join(root, permissionMode, "config", "opencode", "opencode.json"),
|
|
"utf8",
|
|
),
|
|
);
|
|
expect(config.permission).toMatchObject({
|
|
"*": permissionMode,
|
|
external_directory: { "*": "deny", [`${workspace}/**`]: "allow" },
|
|
});
|
|
expect(config.autoupdate).toBe(false);
|
|
expect(config.provider.openrouter.options.headers).toMatchObject({
|
|
"HTTP-Referer": "https://paperclip.ing", "X-Title": "Paperclip", "X-OpenRouter-Title": "Paperclip",
|
|
});
|
|
expect(config.provider.openrouter.models).toHaveProperty(
|
|
"deepseek/deepseek-v4-flash-0731",
|
|
);
|
|
expect(config.provider.openrouter.models["deepseek/deepseek-v4-flash-0731"].options).toBeUndefined();
|
|
expect(config.permission.paperclip_finish).toBeUndefined();
|
|
expect(config.permission["paperclip_*"]).toBe("allow");
|
|
await session.close({ reason: "permission mode test complete" });
|
|
},
|
|
);
|
|
|
|
it.skipIf(process.platform === "win32")("allows the selected workspace alias and its canonical path while denying other directories", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-workspace-alias-"));
|
|
roots.push(root);
|
|
const workspace = join(root, "actual-workspace");
|
|
const alias = join(root, "workspace-alias");
|
|
await mkdir(workspace);
|
|
await symlink(workspace, alias, "dir");
|
|
const canonical = await realpath(workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
permissionMode: "allow",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-workspace-alias", normalizedSessionId: "alias-session",
|
|
workingDirectory: alias,
|
|
});
|
|
try {
|
|
const config = JSON.parse(await readFile(join(root, "alias-session", "config", "opencode", "opencode.json"), "utf8"));
|
|
expect(config.permission.external_directory).toEqual({
|
|
"*": "deny", [alias]: "allow", [`${alias}/**`]: "allow",
|
|
[canonical]: "allow", [`${canonical}/**`]: "allow",
|
|
});
|
|
} finally { await session.close({ reason: "workspace alias test complete" }); }
|
|
});
|
|
|
|
it("clears a stale active turn that already has a persisted terminal fingerprint", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-stale-turn-"),
|
|
);
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-stale-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const seed = await driver.openSession({
|
|
runId: "run-stale-turn",
|
|
normalizedSessionId: "stale-turn",
|
|
workingDirectory: workspace,
|
|
});
|
|
const snapshot = await seed.snapshot();
|
|
await seed.close({ reason: "seed complete" });
|
|
|
|
const recoveryDriver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const recovery = await recoveryDriver.recoverSession?.({
|
|
...snapshot,
|
|
activeTurnId: "turn-already-terminal",
|
|
terminalTurns: [
|
|
{
|
|
turnId: "turn-already-terminal",
|
|
fingerprint: "terminal-fingerprint",
|
|
},
|
|
],
|
|
});
|
|
expect(recovery).toMatchObject({ recovered: true });
|
|
await expect(recovery!.session!.snapshot()).resolves.toMatchObject({
|
|
activeTurnId: null,
|
|
});
|
|
await recovery!.session!.close({ reason: "recovery-test" });
|
|
});
|
|
|
|
it("validates provider/model form", async () => {
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
roots.push(root);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "bad",
|
|
runtimeDirectory: root,
|
|
});
|
|
await expect(
|
|
driver.validateConfig?.({ model: "bad" }),
|
|
).resolves.toMatchObject({ ok: false });
|
|
await expect(
|
|
driver.validateConfig?.({ model: "openrouter/model" }),
|
|
).resolves.toMatchObject({ ok: true });
|
|
});
|
|
|
|
it.each(["wrong", "missing", "duplicate"])("returns a repairable tool error for %s criteria without committing a bad semantic result", async (mode) => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-criteria-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-workspace-"));
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({ model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture, environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" }, taskEnvelope: createCodexTaskEnvelope({ objective: "Apply the accepted decision", contractRevision: "approval-v2", criteria: [{ id: "human_response", requirement: "Apply the approved response" }] }) });
|
|
const session = await driver.openSession({ runId: "criteria-repair", normalizedSessionId: "criteria-repair", workingDirectory: workspace });
|
|
await session.startTurn({ message: { role: "user", text: `repair-criteria-${mode}` } });
|
|
const events = await collectTurnEvents(session.events());
|
|
const results = events.filter((event) => event.eventType === "run.result.proposed");
|
|
expect(results).toHaveLength(1);
|
|
expect(events.some((event) => event.eventType === "item.completed" &&
|
|
(event.payload as { item?: { is_error?: boolean } }).item?.is_error === true)).toBe(true);
|
|
expect(results[0].payload).toMatchObject({ completionClaim: { criteria: [{ criterionId: "human_response" }] } });
|
|
const files = await readdir(root, { recursive: true });
|
|
const evidence = files.find((name) => name.endsWith("fake-criteria-repair.json"));
|
|
expect(evidence).toBeDefined();
|
|
expect(JSON.parse(await readFile(join(root, evidence!), "utf8"))).toMatchObject({ result: { isError: true, content: [{ text: expect.stringContaining('"human_response"') }] } });
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("names the expected revision in a repairable stale-revision tool error", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-revision-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-workspace-"));
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({ model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture, environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" }, taskEnvelope: createCodexTaskEnvelope({ objective: "Apply the accepted decision", contractRevision: "approval-v2", criteria: [{ id: "human_response", requirement: "Apply the approved response" }] }) });
|
|
const session = await driver.openSession({ runId: "revision-repair", normalizedSessionId: "revision-repair", workingDirectory: workspace });
|
|
await session.startTurn({ message: { role: "user", text: "repair-revision" } });
|
|
const events = await collectTurnEvents(session.events());
|
|
const results = events.filter((event) => event.eventType === "run.result.proposed");
|
|
expect(results).toHaveLength(1);
|
|
expect(results[0].payload).toMatchObject({ completionClaim: { contractRevision: "approval-v2" } });
|
|
const files = await readdir(root, { recursive: true });
|
|
const evidence = files.find((name) => name.endsWith("fake-revision-repair.json"));
|
|
expect(evidence).toBeDefined();
|
|
expect(JSON.parse(await readFile(join(root, evidence!), "utf8"))).toMatchObject({ result: { isError: true, content: [{ text: expect.stringContaining('Use contractRevision "approval-v2"') }] } });
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it.each(["monitor", "response_wake"] as const)("resolves an omitted %s through the real OpenCode MCP boundary only from current server state", async kind => {
|
|
for (const eligible of [true, false]) {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-monitor-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-monitor-workspace-"));
|
|
roots.push(root, workspace);
|
|
const continuation = { kind, summary: "Wait for the saved task path.", idempotencyKey: "task-wait:saved" };
|
|
const reads: string[] = [];
|
|
const accepted: unknown[] = [];
|
|
const diagnostics: string[] = [];
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
|
|
dynamicToolHandler: async call => {
|
|
reads.push(call.tool);
|
|
return { run: { id: "monitor-default" }, activeTask: { executionRunId: "monitor-default" },
|
|
completionContinuation: eligible ? continuation : null };
|
|
},
|
|
completionFeedback: async result => { accepted.push(result); return "Accepted monitor wait."; },
|
|
onDiagnostic: message => diagnostics.push(message),
|
|
});
|
|
const session = await driver.openSession({ runId: "monitor-default", normalizedSessionId: "monitor-default", workingDirectory: workspace });
|
|
try {
|
|
await session.startTurn({ message: { role: "user", text: "monitor-default" } });
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(diagnostics).toContainEqual(expect.stringContaining('"schema":"paperclip.opencode.finish-input-diagnostic/v1","stage":"offered"'));
|
|
expect(reads).toEqual(["get_task_context"]);
|
|
if (eligible) {
|
|
expect(accepted).toMatchObject([{ reportedWorkDisposition: "yielded", continuation }]);
|
|
expect(events.filter(event => event.eventType === "run.result.proposed")).toHaveLength(1);
|
|
expect((await session.snapshot()).semanticResult?.result).toMatchObject({ continuation });
|
|
} else {
|
|
expect(accepted).toEqual([]);
|
|
expect(events.filter(event => event.eventType === "run.result.proposed")).toEqual([]);
|
|
expect((await session.snapshot()).semanticResult).toBeNull();
|
|
}
|
|
} finally { await session.close({ reason: "saved wait default test complete" }); }
|
|
}
|
|
});
|
|
|
|
it("keeps assigned discovery operation IDs at the authenticated MCP and controller boundary", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-assigned-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-workspace-"));
|
|
roots.push(root, workspace);
|
|
const calls: string[] = [];
|
|
const operation = "paperclip_search_assigned_tools";
|
|
const driver = new OpenCodeServerDriver({ model: "openrouter/deepseek/deepseek-v4.1-flash-1017", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
|
|
dynamicTools: [{ name: operation, inputSchema: { type: "object" } }],
|
|
dynamicToolHandler: async call => { calls.push(call.tool); if (call.tool !== operation) throw new Error("unauthorized operation"); return { tools: [] }; },
|
|
});
|
|
const session = await driver.openSession({ runId: "assigned-name", normalizedSessionId: "assigned-name", workingDirectory: workspace });
|
|
try {
|
|
await session.startTurn({ message: { role: "user", text: "Discover assigned tools." } });
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(calls).toEqual([operation]);
|
|
expect(events.some(event => event.eventType === "item.completed" && (event.payload as { item?: { is_error?: boolean } }).item?.is_error)).toBe(false);
|
|
} finally { await session.close({ reason: "assigned operation identity test" }); }
|
|
});
|
|
|
|
it("normalizes a structured block result", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-block",
|
|
normalizedSessionId: "block",
|
|
workingDirectory: workspace,
|
|
});
|
|
await session.startTurn({
|
|
message: { role: "user", text: "block-result" },
|
|
});
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(
|
|
events.find((event) => event.eventType === "run.result.proposed")
|
|
?.payload,
|
|
).toMatchObject({
|
|
reportedWorkDisposition: "blocked",
|
|
blocker: { reasonCode: "test_dependency" },
|
|
});
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("designates the substantive pre-finish response instead of a trailing acknowledgement", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-final-selection",
|
|
normalizedSessionId: "final-selection",
|
|
workingDirectory: workspace,
|
|
});
|
|
await session.startTurn({
|
|
message: { role: "user", text: "text-before-finish" },
|
|
});
|
|
const events = await collectTurnEvents(session.events());
|
|
const finalMessages = events.filter(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "agentMessage" &&
|
|
event.payload.channel === "final",
|
|
);
|
|
expect(finalMessages).toHaveLength(1);
|
|
expect(finalMessages[0]?.payload.text).toBe(
|
|
"Substantive answer before finish.",
|
|
);
|
|
expect(
|
|
events
|
|
.filter((event) => event.eventType === "item.delta")
|
|
.map((event) => event.payload.text),
|
|
).toEqual(
|
|
expect.arrayContaining(["Substantive answer before finish.", "Done."]),
|
|
);
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("does not promote opening commentary to the final response when work follows it", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-commentary-only",
|
|
normalizedSessionId: "commentary-only",
|
|
workingDirectory: workspace,
|
|
});
|
|
await session.startTurn({
|
|
message: { role: "user", text: "commentary-only-before-work" },
|
|
});
|
|
const events = await collectTurnEvents(session.events());
|
|
expect(
|
|
events.filter(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "agentMessage" &&
|
|
event.payload.channel === "final",
|
|
),
|
|
).toHaveLength(0);
|
|
expect(
|
|
events
|
|
.filter((event) => event.eventType === "item.delta")
|
|
.map((event) => event.payload.text),
|
|
).toContain("I will inspect the workspace first.");
|
|
expect(events).toContainEqual(
|
|
expect.objectContaining({ eventType: "run.result.proposed" }),
|
|
);
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("correlates the final response to the native message containing the semantic tool", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-correlated-final-selection",
|
|
normalizedSessionId: "correlated-final-selection",
|
|
workingDirectory: workspace,
|
|
});
|
|
await session.startTurn({
|
|
message: { role: "user", text: "correlated-final-message" },
|
|
});
|
|
const events = await collectTurnEvents(session.events());
|
|
const finalMessages = events.filter(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "agentMessage" &&
|
|
event.payload.channel === "final",
|
|
);
|
|
expect(finalMessages).toHaveLength(1);
|
|
expect(finalMessages[0]?.payload.text).toBe(
|
|
"Correlated substantive final answer.",
|
|
);
|
|
const assistantDeltas = events.filter(
|
|
(event) =>
|
|
event.eventType === "item.delta" &&
|
|
event.payload.kind === "agentMessage",
|
|
);
|
|
expect(assistantDeltas.map((event) => event.payload.text)).toEqual([
|
|
"I will write the result now.",
|
|
"Correlated substantive final answer.",
|
|
"Done.",
|
|
]);
|
|
expect(assistantDeltas).toHaveLength(3);
|
|
expect(
|
|
assistantDeltas.every(
|
|
(event) =>
|
|
event.payload.channel === "progress" &&
|
|
event.payload.providerPhase === "commentary",
|
|
),
|
|
).toBe(true);
|
|
expect(assistantDeltas.map((event) => event.payload.item)).toEqual([
|
|
expect.objectContaining({
|
|
type: "agentMessage",
|
|
channel: "progress",
|
|
phase: "commentary",
|
|
}),
|
|
expect.objectContaining({
|
|
type: "agentMessage",
|
|
channel: "progress",
|
|
phase: "commentary",
|
|
}),
|
|
expect.objectContaining({
|
|
type: "agentMessage",
|
|
channel: "progress",
|
|
phase: "commentary",
|
|
}),
|
|
]);
|
|
expect(finalMessages[0]?.itemId).toBe(
|
|
assistantDeltas.find(
|
|
(event) =>
|
|
event.payload.text === "Correlated substantive final answer.",
|
|
)?.itemId,
|
|
);
|
|
expect(finalMessages[0]?.payload).toMatchObject({
|
|
kind: "agentMessage",
|
|
channel: "final",
|
|
providerPhase: "final_answer",
|
|
});
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("never settles a post-completion response truncated at the output limit", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-output-limit-after-tool",
|
|
normalizedSessionId: "output-limit-after-tool",
|
|
workingDirectory: workspace,
|
|
});
|
|
await session.startTurn({
|
|
message: { role: "user", text: "output-limit-after-tool" },
|
|
});
|
|
const events = await collectTurnEvents(session.events());
|
|
// The accepted completion report's summary resolves the durable reply;
|
|
// the runaway text is incomplete by the provider's own finish reason.
|
|
expect(events.some((event) => event.eventType === "run.result.proposed")).toBe(true);
|
|
expect(events.at(-1)?.eventType).toBe("turn.completed");
|
|
expect(
|
|
events.filter(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "agentMessage" &&
|
|
event.payload.channel === "final",
|
|
),
|
|
).toHaveLength(0);
|
|
expect(
|
|
events.some(
|
|
(event) =>
|
|
event.eventType === "harness.diagnostic" &&
|
|
event.payload.code === "opencode_output_limit_reached",
|
|
),
|
|
).toBe(true);
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it("uses provider structure rather than prose length to select a post-tool answer", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-post-tool-final-selection",
|
|
normalizedSessionId: "post-tool-final-selection",
|
|
workingDirectory: workspace,
|
|
});
|
|
await session.startTurn({
|
|
message: { role: "user", text: "final-after-tool-commentary" },
|
|
});
|
|
const events = await collectTurnEvents(session.events());
|
|
const finalMessages = events.filter(
|
|
(event) =>
|
|
event.eventType === "item.completed" &&
|
|
event.payload.kind === "agentMessage" &&
|
|
event.payload.channel === "final",
|
|
);
|
|
expect(finalMessages).toHaveLength(1);
|
|
expect(finalMessages[0]?.payload.text).toBe(
|
|
"This is the complete substantive answer emitted after the accepted completion tool call.",
|
|
);
|
|
await session.close({ reason: "test" });
|
|
});
|
|
|
|
it.each(["session.idle", "session.status", "session.error", "aborted"])("settles accepted completion before a racing %s event", async terminalType => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-completion-race-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-completion-race-workspace-"));
|
|
roots.push(root, workspace);
|
|
let stream: ReadableStreamDefaultController<Uint8Array>;
|
|
let releaseFeedback!: () => void;
|
|
let feedbackStarted!: () => void;
|
|
const started = new Promise<void>(resolve => { feedbackStarted = resolve; });
|
|
const release = new Promise<void>(resolve => { releaseFeedback = resolve; });
|
|
const feedback = "Accepted. Keep the exact saved document link in the final response.";
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
|
|
fetch: async (input, init) => String(input).endsWith("/event")
|
|
? new Response(new ReadableStream<Uint8Array>({ start(controller) { stream = controller; } }), { headers: { "Content-Type": "text/event-stream" } })
|
|
: fetch(input, init),
|
|
completionFeedback: async () => { feedbackStarted(); await release; return feedback; },
|
|
});
|
|
const session = await driver.openSession({ runId: "completion-race", normalizedSessionId: "completion-race", workingDirectory: workspace });
|
|
try {
|
|
const events = collectTurnEvents(session.events());
|
|
await session.startTurn({ message: { role: "user", text: "completion-feedback" } });
|
|
await started;
|
|
stream!.enqueue(new TextEncoder().encode(`data: ${JSON.stringify({ type: terminalType === "aborted" ? "session.error" : terminalType, id: "terminal-during-feedback", properties: { sessionID: session.ids().providerSessionId, status: { type: "idle" }, error: terminalType === "aborted" ? { name: "MessageAbortedError", data: { message: "Aborted" } } : { name: "FixtureError", message: "Provider ended during feedback." } } })}\n\n`));
|
|
// Drain the queued SSE frame before releasing the controller response.
|
|
await new Promise<void>(resolve => setImmediate(resolve));
|
|
releaseFeedback();
|
|
const observed = await events;
|
|
const proposed = observed.findIndex(event => event.eventType === "run.result.proposed");
|
|
const completed = observed.findIndex(event => event.eventType === "item.completed" && event.payload.kind === "dynamicToolCall" && (event.payload.item as { result?: unknown })?.result === feedback);
|
|
const terminal = observed.findIndex(event => TURN_TERMINAL_EVENT_TYPES.has(event.eventType));
|
|
expect(proposed).toBeGreaterThanOrEqual(0);
|
|
expect(completed).toBeGreaterThan(proposed);
|
|
expect(terminal).toBeGreaterThan(completed);
|
|
expect(observed.filter(event => event.eventType === "run.result.proposed")).toHaveLength(1);
|
|
expect((await session.snapshot()).semanticResult).not.toBeNull();
|
|
const sessionRoots = (await readdir(root, { withFileTypes: true })).filter(entry => entry.isDirectory());
|
|
await expect.poll(async () => JSON.parse(await readFile(join(root, sessionRoots[0]!.name, "data/fake-completion-feedback.json"), "utf8"))).toMatchObject([
|
|
{ result: { content: [{ text: expect.stringContaining(feedback) }] } },
|
|
]);
|
|
} finally {
|
|
releaseFeedback();
|
|
await session.close({ reason: "test complete" });
|
|
}
|
|
});
|
|
|
|
it.each(["close", "interrupt"] as const)("settles bound completion before explicit %s", async operation => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-completion-stop-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-completion-stop-workspace-"));
|
|
roots.push(root, workspace);
|
|
let releaseFeedback!: () => void, feedbackStarted!: () => void;
|
|
const started = new Promise<void>(resolve => { feedbackStarted = resolve; });
|
|
const release = new Promise<void>(resolve => { releaseFeedback = resolve; });
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
|
|
completionFeedback: async () => { feedbackStarted(); await release; return "Accepted completion."; },
|
|
});
|
|
const session = await driver.openSession({ runId: "completion-stop", normalizedSessionId: "completion-stop", workingDirectory: workspace });
|
|
try {
|
|
await session.startTurn({ message: { role: "user", text: "finish" } });
|
|
await started;
|
|
let stopped = false;
|
|
const stop = session[operation]!({ reason: "fixture stop" }).then(() => { stopped = true; });
|
|
await new Promise<void>(resolve => setImmediate(resolve));
|
|
expect(stopped).toBe(false);
|
|
releaseFeedback();
|
|
await stop;
|
|
expect((await session.snapshot()).semanticResult).not.toBeNull();
|
|
const transcript = await session.transcript!();
|
|
expect(transcript.events.filter(event => event.eventType === "run.result.proposed")).toHaveLength(1);
|
|
expect(transcript.events.some(event => event.eventType === "item.completed" && (event.payload.item as { is_error?: boolean })?.is_error)).toBe(false);
|
|
} finally {
|
|
releaseFeedback();
|
|
await session.close({ reason: "test complete" });
|
|
}
|
|
});
|
|
|
|
it("rejects malformed SSE frames", async () => {
|
|
const stream = (value: string) =>
|
|
new ReadableStream<Uint8Array>({
|
|
start(controller) {
|
|
controller.enqueue(new TextEncoder().encode(value));
|
|
controller.close();
|
|
},
|
|
});
|
|
const consume = async (value: string) => {
|
|
for await (const _event of openCodeServerDriverInternals.parseSse(
|
|
stream(value),
|
|
)) {
|
|
// Consume the complete stream so parse failures are observed.
|
|
}
|
|
};
|
|
await expect(consume("data: {bad json}\n\n")).rejects.toThrow();
|
|
});
|
|
|
|
it("keeps the turn and the tool outcome when OpenCode sends an event above the payload limit", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-oversized-sse-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-oversized-sse-workspace-"));
|
|
roots.push(root, workspace);
|
|
let stream!: ReadableStreamDefaultController<Uint8Array>;
|
|
let streamOpened!: () => void;
|
|
const opened = new Promise<void>(resolve => { streamOpened = resolve; });
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
|
|
fetch: async (input, init) => String(input).endsWith("/event")
|
|
? new Response(new ReadableStream<Uint8Array>({ start(controller) { stream = controller; streamOpened(); } }), { headers: { "Content-Type": "text/event-stream" } })
|
|
: fetch(input, init),
|
|
});
|
|
const session = await driver.openSession({ runId: "oversized-sse", normalizedSessionId: "oversized-sse", workingDirectory: workspace });
|
|
try {
|
|
const sessionID = session.ids().providerSessionId;
|
|
const events = collectTurnEvents(session.events());
|
|
await session.startTurn({ message: { role: "user", text: "run the full suite" } });
|
|
await opened;
|
|
const tail = "\nTests: 1 failed, 812 passed\nexit status 1";
|
|
const output = "test output line\n".repeat(150_000) + tail;
|
|
const send = (value: unknown) => {
|
|
const bytes = new TextEncoder().encode(`data: ${JSON.stringify(value)}\n\n`);
|
|
for (let index = 0; index < bytes.length; index += 64 * 1024)
|
|
stream.enqueue(bytes.subarray(index, index + 64 * 1024));
|
|
};
|
|
send({ type: "message.updated", properties: { info: { id: "msg_oversized", role: "assistant", sessionID } } });
|
|
send({ type: "message.part.updated", properties: { part: {
|
|
id: "prt_oversized", messageID: "msg_oversized", sessionID, type: "tool", callID: "call_oversized", tool: "bash",
|
|
state: { status: "completed", input: { command: "pnpm test" }, output, title: "pnpm test", metadata: { exit: 1, output }, time: { start: 1, end: 2 } },
|
|
} } });
|
|
send({ type: "message.part.updated", properties: { part: {
|
|
id: "prt_final", messageID: "msg_oversized", sessionID, type: "text",
|
|
text: "The suite completed: one test failed; 812 passed.", time: { start: 3, end: 4 },
|
|
} } });
|
|
send({ type: "session.idle", properties: { sessionID } });
|
|
const observed = await events;
|
|
expect(observed.at(-1)?.eventType).toBe("turn.completed");
|
|
const diagnostics = observed.filter(event => event.eventType === "harness.diagnostic");
|
|
expect(diagnostics.map(event => event.payload.code)).toEqual(["opencode_sse_event_compacted"]);
|
|
const tool = observed.find(event => event.eventType === "tool.execution.completed" && event.payload.executionId === "prt_oversized");
|
|
expect(tool?.payload).toMatchObject({
|
|
name: "bash",
|
|
status: "completed",
|
|
outputBytes: Buffer.byteLength(output),
|
|
outputTruncated: true,
|
|
outputDigest: null,
|
|
});
|
|
expect(String(tool?.payload.output).endsWith(tail)).toBe(true);
|
|
} finally {
|
|
await session.close({ reason: "test complete" });
|
|
}
|
|
});
|
|
|
|
it("keeps the provider explanation when an API error response body exceeds the payload limit", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-large-error-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-large-error-workspace-"));
|
|
roots.push(root, workspace);
|
|
let stream!: ReadableStreamDefaultController<Uint8Array>;
|
|
let streamOpened!: () => void;
|
|
const opened = new Promise<void>(resolve => { streamOpened = resolve; });
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
|
|
fetch: async (input, init) => String(input).endsWith("/event")
|
|
? new Response(new ReadableStream<Uint8Array>({ start(controller) { stream = controller; streamOpened(); } }), { headers: { "Content-Type": "text/event-stream" } })
|
|
: fetch(input, init),
|
|
});
|
|
const session = await driver.openSession({ runId: "large-error", normalizedSessionId: "large-error", workingDirectory: workspace });
|
|
try {
|
|
const events = collectTurnEvents(session.events());
|
|
await session.startTurn({ message: { role: "user", text: "diagnostic check" } });
|
|
await opened;
|
|
stream.enqueue(new TextEncoder().encode(`data: ${JSON.stringify({ type: "session.error", properties: {
|
|
sessionID: session.ids().providerSessionId, error: { name: "APIError", data: { message: "Provider is busy.", statusCode: 503,
|
|
responseBody: "private-body".repeat(7000) } },
|
|
} })}\n\n`));
|
|
const observed = await events;
|
|
const failure = observed.find(event => event.eventType === "turn.failed");
|
|
expect(failure?.payload.error).toMatchObject({ providerFailure: { provider: "openrouter", category: "overloaded", message: "Provider is busy." } });
|
|
// Provider text stays in display fields: the notice summary, never the failure message.
|
|
expect((failure?.payload.error as Record<string, unknown>).message).toBeUndefined();
|
|
expect(observed.find(event => event.eventType === "provider.notice.recorded")?.payload.summary).toBe("Provider is busy.");
|
|
expect(JSON.stringify(failure)).not.toContain("private-body");
|
|
} finally { await session.close({ reason: "test complete" }); }
|
|
});
|
|
|
|
it("fails the running turn with the exit cause when OpenCode stops mid-turn", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-server-lost-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-server-lost-workspace-"));
|
|
roots.push(root, workspace);
|
|
let stream!: ReadableStreamDefaultController<Uint8Array>;
|
|
let streamOpened!: () => void;
|
|
const opened = new Promise<void>(resolve => { streamOpened = resolve; });
|
|
let serverPid: number | undefined;
|
|
let eventRequests = 0;
|
|
const refused = () => new TypeError("fetch failed", {
|
|
cause: Object.assign(new Error("connect ECONNREFUSED 127.0.0.1"), { code: "ECONNREFUSED" }),
|
|
});
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731", runtimeDirectory: root, command: fixture,
|
|
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
|
|
onSpawn: async ({ pid }) => { serverPid = pid; },
|
|
// Own the event stream so the fixture cannot finish the turn on its own,
|
|
// then report the refused reconnects a dead server produces.
|
|
fetch: async (input, init) => {
|
|
if (!String(input).endsWith("/event")) return fetch(input, init);
|
|
eventRequests += 1;
|
|
if (eventRequests > 1) throw refused();
|
|
return new Response(new ReadableStream<Uint8Array>({ start(controller) { stream = controller; streamOpened(); } }),
|
|
{ headers: { "Content-Type": "text/event-stream" } });
|
|
},
|
|
});
|
|
const session = await driver.openSession({ runId: "server-lost", normalizedSessionId: "server-lost", workingDirectory: workspace });
|
|
try {
|
|
const events = collectTurnEvents(session.events());
|
|
const turn = await session.startTurn({ message: { role: "user", text: "install dependencies" } });
|
|
await opened;
|
|
process.kill(serverPid!, "SIGKILL");
|
|
stream.error(refused());
|
|
const observed = await events;
|
|
const failed = observed.at(-1);
|
|
expect(failed).toMatchObject({
|
|
eventType: "turn.failed",
|
|
turnId: turn.turnId,
|
|
payload: {
|
|
status: "failed",
|
|
providerTerminalObserved: false,
|
|
error: {
|
|
code: "opencode_server_exited",
|
|
opencodeFailure: { kind: "server_exited", signal: "SIGKILL", connectionError: "ECONNREFUSED" },
|
|
},
|
|
},
|
|
});
|
|
const diagnostic = observed.find(event => event.eventType === "harness.diagnostic" && event.payload.code === "opencode_sse_failed");
|
|
expect(diagnostic?.payload.failure).toEqual({ kind: "server_exited", signal: "SIGKILL", connectionError: "ECONNREFUSED" });
|
|
expect(eventRequests).toBe(4);
|
|
} finally {
|
|
await session.close({ reason: "test complete" });
|
|
}
|
|
});
|
|
|
|
it("classifies a lost event stream from closed error fields only", async () => {
|
|
const { openCodeEventStreamFailure } = openCodeServerDriverInternals;
|
|
const running = Object.assign(new EventEmitter(), { exitCode: null, signalCode: null }) as unknown as ChildProcess;
|
|
const exited = Object.assign(new EventEmitter(), { exitCode: 137, signalCode: null }) as unknown as ChildProcess;
|
|
const reset = new TypeError("fetch failed", { cause: Object.assign(new Error("private socket detail"), { code: "ECONNRESET" }) });
|
|
expect(await openCodeEventStreamFailure(reset, exited)).toEqual({ kind: "server_exited", exitCode: 137, connectionError: "ECONNRESET" });
|
|
expect(await openCodeEventStreamFailure(Object.assign(new Error("HTTP 502"), { httpStatus: 502 }), running))
|
|
.toEqual({ kind: "http_error", httpStatus: 502 });
|
|
expect(await openCodeEventStreamFailure(Object.assign(new Error("closed"), { streamClosed: true }), running))
|
|
.toEqual({ kind: "stream_closed" });
|
|
expect(await openCodeEventStreamFailure(new Error("private provider text"), running)).toEqual({ kind: "stream_error" });
|
|
const unknownCode = new TypeError("fetch failed", { cause: Object.assign(new Error("x"), { code: "EPRIVATE" }) });
|
|
const started = Date.now();
|
|
expect(await openCodeEventStreamFailure(unknownCode, running)).toEqual({ kind: "connection_failed", connectionError: "unrecognized" });
|
|
expect(Date.now() - started).toBeGreaterThanOrEqual(900);
|
|
});
|
|
|
|
it("restarts OpenCode when session startup fails transiently", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
let failSessionCreate = true;
|
|
const spawns: number[] = [];
|
|
const commandLifecycle: string[] = [];
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
},
|
|
commandLifecycle: {
|
|
beforeSpawn: () => {
|
|
commandLifecycle.push("before");
|
|
},
|
|
afterSpawn: () => {
|
|
commandLifecycle.push("after");
|
|
},
|
|
afterExit: () => {
|
|
commandLifecycle.push("exit");
|
|
},
|
|
},
|
|
fetch: async (input, init) => {
|
|
if (
|
|
failSessionCreate &&
|
|
String(input).endsWith("/session") &&
|
|
init?.method === "POST"
|
|
) {
|
|
failSessionCreate = false;
|
|
throw new Error("transient session initialization failure");
|
|
}
|
|
return fetch(input, init);
|
|
},
|
|
onSpawn: async ({ pid }) => {
|
|
spawns.push(pid);
|
|
},
|
|
});
|
|
const session = await driver.openSession({
|
|
runId: "run-retry",
|
|
normalizedSessionId: "retry",
|
|
workingDirectory: workspace,
|
|
});
|
|
expect(session.ids().providerSessionId).toBe("ses_fake_1");
|
|
expect(spawns.length).toBeGreaterThanOrEqual(2);
|
|
expect(spawns.length).toBeLessThanOrEqual(3);
|
|
expect(commandLifecycle).toEqual(
|
|
[...Array.from({ length: spawns.length - 1 }, () => ["before", "after", "exit"]).flat(), "before", "after"],
|
|
);
|
|
await session.close({ reason: "test" });
|
|
expect(commandLifecycle).toEqual(
|
|
Array.from({ length: spawns.length }, () => ["before", "after", "exit"]).flat(),
|
|
);
|
|
});
|
|
|
|
it("releases the executable binding when spawning fails before health checks", async () => {
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-spawn-error-"));
|
|
roots.push(root);
|
|
const lifecycle: string[] = [];
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
runtimeDirectory: root,
|
|
command: join(root, "missing-executable"),
|
|
commandLifecycle: {
|
|
beforeSpawn: () => { lifecycle.push("before"); },
|
|
afterSpawn: () => { lifecycle.push("spawned"); },
|
|
afterExit: () => { lifecycle.push("released"); },
|
|
},
|
|
});
|
|
await expect(driver.openSession({ runId: "spawn-error", normalizedSessionId: "spawn-error",
|
|
workingDirectory: root })).rejects.toThrow("ENOENT");
|
|
expect(lifecycle).toEqual(["before", "released"]);
|
|
});
|
|
|
|
it("reports startup exit details with stderr captured after spawn and redacted", async () => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(
|
|
join(tmpdir(), "paperclip-opencode-workspace-"),
|
|
);
|
|
roots.push(root, workspace);
|
|
const exitingFixture = join(root, "exit-before-health.mjs");
|
|
await writeFile(
|
|
exitingFixture,
|
|
"#!/usr/bin/env node\nimport { readFileSync } from 'node:fs';\nconst config = JSON.parse(readFileSync(`${process.env.XDG_CONFIG_HOME}/opencode/opencode.json`, 'utf8'));\nprocess.stderr.write(`credential=${process.env.OPENROUTER_API_KEY}\\ngateway=${config.provider.paperclip.options.apiKey}\\nauthorization=super-secret-opencode-token\\n`);\nprocess.exit(17);\n",
|
|
{ mode: 0o755 },
|
|
);
|
|
const diagnostics: string[] = [];
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "paperclip/team/model-alias",
|
|
runtimeDirectory: root,
|
|
command: exitingFixture,
|
|
onDiagnostic: (message) => { diagnostics.push(message); },
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
OPENROUTER_API_KEY: "fixture-key",
|
|
PAPERCLIP_AI_PROVIDER_KEY: "fixture-custom-gateway-key",
|
|
PAPERCLIP_AI_PROVIDER_URL: "https://gateway.example/v1",
|
|
},
|
|
});
|
|
const error = await driver
|
|
.openSession({
|
|
runId: "run-startup-exit",
|
|
normalizedSessionId: "startup-exit",
|
|
workingDirectory: workspace,
|
|
})
|
|
.then(
|
|
() => "provider unexpectedly started",
|
|
(cause: unknown) => String(cause),
|
|
);
|
|
expect(error).toContain("provider_process_exited");
|
|
expect(error).toContain("provider=opencode");
|
|
expect(error).toContain("stage=health");
|
|
expect(error).toContain("[REDACTED]");
|
|
expect(error).not.toContain("fixture-key");
|
|
expect(error).not.toContain("fixture-custom-gateway-key");
|
|
expect(error).not.toContain("super-secret-opencode-token");
|
|
expect(diagnostics.join("")).toContain("gateway=[REDACTED]");
|
|
expect(diagnostics.join("")).not.toContain("fixture-custom-gateway-key");
|
|
});
|
|
|
|
it.each(["network", "response"])("redacts custom gateway keys in %s errors", async (failure) => {
|
|
await chmod(fixture, 0o755);
|
|
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
|
|
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-workspace-"));
|
|
roots.push(root, workspace);
|
|
const key = "fixture-custom-gateway-key";
|
|
const driver = new OpenCodeServerDriver({
|
|
model: "paperclip/team/model-alias",
|
|
runtimeDirectory: root,
|
|
command: fixture,
|
|
environment: {
|
|
PATH: process.env.PATH,
|
|
PAPERCLIP_AI_PROVIDER_KEY: key,
|
|
PAPERCLIP_AI_PROVIDER_URL: "https://gateway.example/v1",
|
|
},
|
|
fetch: async (input, init) => {
|
|
if (String(input).endsWith("/session") && init?.method === "POST") {
|
|
if (failure === "network") throw new Error(`Gateway rejected ${key}`);
|
|
return new Response(`Gateway rejected ${key}`, { status: 400 });
|
|
}
|
|
return fetch(input, init);
|
|
},
|
|
});
|
|
const error = await driver.openSession({
|
|
runId: "run-gateway-error",
|
|
normalizedSessionId: "gateway-error",
|
|
workingDirectory: workspace,
|
|
}).then(() => "provider unexpectedly started", (cause: unknown) => String(cause));
|
|
expect(error).toContain("Gateway rejected [REDACTED]");
|
|
expect(error).not.toContain(key);
|
|
});
|
|
});
|