mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-19 10:36:26 +02:00
## Thinking Path > - Paperclip manages AI agents that perform work. > - Paperclip Runner connects durable task runs to local provider processes. > - The full-stack paid matrix exposed failures after the runner integrity repair. > - Verified JavaScript entrypoints lost their relative module graph when Linux executed them through descriptor paths. > - Returned provider startup errors also remained pending and became indeterminate after recovery. > - Sparse Codex tool lifecycle events lost the `write_document` identity before task transcript projection. > - This pull request repairs those three boundaries and makes the structured-question fixture deterministic. > - The benefit is repeatable provider startup, exact failure replay, and correct inline Plan placement. ## Linked Issues or Issue Description Refs #12721 and #12700. **What happened?** The paid runner matrix failed ACPX and OpenCode startup before provider session creation. The runner journal then replaced the original startup error with an indeterminate recovery result. Native Codex saved a Plan but rendered it only as a fallback card. A legacy Claude waiting reply could also echo the reserved terminal marker before the answer arrived. **Expected behavior** Verified JavaScript providers must start from immutable descriptor-backed artifacts. Returned startup failures must persist as terminal failed command results. Native tool lifecycle updates must preserve the `write_document` boundary. Pre-answer fixture output must not contain the reserved terminal marker. **Steps to reproduce** 1. Run the local provider cells in the Runner Full-Stack E2E workflow. 2. Observe ACPX and OpenCode fail during `session.open` before provider execution. 3. Observe recovery report `execution_indeterminate` instead of the original startup error. 4. Run the native Codex Plan cell and observe the fallback Plan card after the tool activity row. 5. Run the legacy Claude structured-question resume cell and observe an early marker echo in waiting prose. **Paperclip version or commit** `0f9452101740835ce0b1488a204bf48acd5bafc3` **Deployment mode** Local development with the paid GitHub Actions acceptance workflow. ## What Changed - Bundle the ACPX sidecar and OpenCode proxy as self-contained Node ESM entrypoints before hashing and verified descriptor launch. - Anchor ACPX dynamic provider package resolution at a controller-derived provider-pack root and keep that root out of the provider child environment. - Persist executor-returned startup errors as redacted durable failed command results while retaining indeterminate recovery for true process death. - Coalesce sparse native tool items by stable ID so a late `write_document` name, input, and result reach the transcript boundary once. - Forbid the structured-question fixture from spelling or announcing its reserved terminal marker before the user answers. ## Verification - Rust and TypeScript regression tests cover durable failed replay, true crash ambiguity, bundle closure, package-root derivation, environment filtering, exact Codex tool lifecycle coalescing, and prompt determinism. - Local execution is intentionally limited to formatters and static diff checks. GitHub Actions will run tests, type checks, builds, and security checks. - After ordinary CI is green, scoped paid cells will validate one ACPX launch, one OpenCode launch, native Codex Plan projection, and legacy Claude structured resume before a complete matrix rerun. - Prior failing matrix: https://github.com/paperclipai/paperclip/actions/runs/33682434315 ## Risks - Bundling changes the bytes covered by provider launch hashes. Provider-pack generation already hashes the final built files. - ACPX still loads qualified provider packages dynamically. The controller supplies a normalized package root, while existing version, digest, path, and descriptor checks remain active. - Durable `failed` is terminal. Replays return the same redacted result and do not execute the provider effect twice. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex based on GPT-5 with agentic reasoning, repository inspection, code editing, Git, parallel subagents, and GitHub Actions coordination. The exact deployed snapshot and context-window size are not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked related public work or described the bug in this PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [ ] I have run tests locally and they pass (intentionally deferred to GitHub Actions) - [x] I have added or updated tests where applicable - [x] No documentation change is required for this runtime repair - [x] I have considered and documented the risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
319 lines
9.9 KiB
TypeScript
319 lines
9.9 KiB
TypeScript
import { afterEach, describe, expect, it, vi } from "vitest";
|
|
import * as serverUtils from "@paperclipai/adapter-utils/server-utils";
|
|
import {
|
|
discoverOpenCodeModels,
|
|
ensureOpenCodeModelConfiguredAndAvailable,
|
|
listOpenCodeModels,
|
|
requireOpenCodeModelId,
|
|
resetOpenCodeModelsCacheForTests,
|
|
} from "./models.js";
|
|
|
|
describe("openCode models", () => {
|
|
afterEach(() => {
|
|
delete process.env.PAPERCLIP_OPENCODE_COMMAND;
|
|
delete process.env.OPENCODE_ALLOW_ALL_MODELS;
|
|
resetOpenCodeModelsCacheForTests();
|
|
vi.restoreAllMocks();
|
|
vi.useRealTimers();
|
|
});
|
|
|
|
it("returns an empty list when discovery command is unavailable", async () => {
|
|
process.env.PAPERCLIP_OPENCODE_COMMAND =
|
|
"__paperclip_missing_opencode_command__";
|
|
await expect(listOpenCodeModels()).resolves.toEqual([]);
|
|
});
|
|
|
|
it("rejects when model is missing", async () => {
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({ model: "" }),
|
|
).rejects.toThrow("OpenCode requires `adapterConfig.model`");
|
|
});
|
|
|
|
it("accepts a provider/model id without running discovery", () => {
|
|
expect(requireOpenCodeModelId("openai/gpt-5.2-codex")).toBe(
|
|
"openai/gpt-5.2-codex",
|
|
);
|
|
});
|
|
|
|
it("rejects malformed provider/model ids before discovery", () => {
|
|
expect(() => requireOpenCodeModelId("gpt-5.2-codex")).toThrow(
|
|
"OpenCode requires `adapterConfig.model`",
|
|
);
|
|
expect(() => requireOpenCodeModelId("openai/")).toThrow(
|
|
"OpenCode requires `adapterConfig.model`",
|
|
);
|
|
});
|
|
|
|
it("proceeds with the configured model when discovery cannot run (probe is best-effort, never fatal)", async () => {
|
|
process.env.PAPERCLIP_OPENCODE_COMMAND =
|
|
"__paperclip_missing_opencode_command__";
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({
|
|
model: "openai/gpt-5",
|
|
}),
|
|
).resolves.toEqual([{ id: "openai/gpt-5", label: "openai/gpt-5" }]);
|
|
});
|
|
|
|
it("skips the availability check when OPENCODE_ALLOW_ALL_MODELS is set in the run env", async () => {
|
|
process.env.PAPERCLIP_OPENCODE_COMMAND =
|
|
"__paperclip_missing_opencode_command__";
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({
|
|
model: "anthropic/tensorix/deepseek/deepseek-chat-v3.1",
|
|
env: { OPENCODE_ALLOW_ALL_MODELS: "true" },
|
|
}),
|
|
).resolves.toEqual([
|
|
{
|
|
id: "anthropic/tensorix/deepseek/deepseek-chat-v3.1",
|
|
label: "anthropic/tensorix/deepseek/deepseek-chat-v3.1",
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("honours OPENCODE_ALLOW_ALL_MODELS from the process env", async () => {
|
|
process.env.PAPERCLIP_OPENCODE_COMMAND =
|
|
"__paperclip_missing_opencode_command__";
|
|
process.env.OPENCODE_ALLOW_ALL_MODELS = "1";
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({
|
|
model: "anthropic/gateway/some-model",
|
|
}),
|
|
).resolves.toEqual([
|
|
{
|
|
id: "anthropic/gateway/some-model",
|
|
label: "anthropic/gateway/some-model",
|
|
},
|
|
]);
|
|
});
|
|
|
|
it("still enforces provider/model format when OPENCODE_ALLOW_ALL_MODELS is set", async () => {
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({
|
|
model: "not-a-valid-id",
|
|
env: { OPENCODE_ALLOW_ALL_MODELS: "true" },
|
|
}),
|
|
).rejects.toThrow("OpenCode requires `adapterConfig.model`");
|
|
});
|
|
|
|
it("retries a transient `opencode models` failure with backoff before succeeding", async () => {
|
|
vi.useFakeTimers();
|
|
const spy = vi
|
|
.spyOn(serverUtils, "runChildProcess")
|
|
.mockResolvedValueOnce({
|
|
exitCode: 1,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "",
|
|
stderr: "queued behind another opencode run",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockResolvedValueOnce({
|
|
exitCode: null,
|
|
signal: null,
|
|
timedOut: true,
|
|
stdout: "",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "ollama/qwen2.5-coder:7b\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
});
|
|
|
|
const promise = discoverOpenCodeModels();
|
|
await vi.runAllTimersAsync();
|
|
|
|
await expect(promise).resolves.toEqual([
|
|
{ id: "ollama/qwen2.5-coder:7b", label: "ollama/qwen2.5-coder:7b" },
|
|
]);
|
|
expect(spy).toHaveBeenCalledTimes(3);
|
|
});
|
|
|
|
it("refreshes a stale non-empty catalog before rejecting the configured model", async () => {
|
|
const spy = vi
|
|
.spyOn(serverUtils, "runChildProcess")
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "openrouter/example/stale-model\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "Models cache refreshed\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout:
|
|
"openrouter/example/current-model\nopenrouter/deepseek/deepseek-v4-flash-0731\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
});
|
|
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
}),
|
|
).resolves.toContainEqual({
|
|
id: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
label: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
});
|
|
expect(spy).toHaveBeenCalledTimes(3);
|
|
expect(spy.mock.calls[0]?.[2]).toEqual(["models"]);
|
|
expect(spy.mock.calls[1]?.[2]).toEqual(["models", "--refresh"]);
|
|
expect(spy.mock.calls[2]?.[2]).toEqual(["models"]);
|
|
});
|
|
|
|
it("still rejects when a refreshed non-empty catalog omits the configured model", async () => {
|
|
const spy = vi
|
|
.spyOn(serverUtils, "runChildProcess")
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "openrouter/example/stale-model\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "Models cache refreshed\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "openrouter/example/current-model\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
});
|
|
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
}),
|
|
).rejects.toThrow(
|
|
"Configured OpenCode model is unavailable: openrouter/deepseek/deepseek-v4-flash-0731",
|
|
);
|
|
expect(spy).toHaveBeenCalledTimes(3);
|
|
expect(spy.mock.calls[1]?.[2]).toEqual(["models", "--refresh"]);
|
|
expect(spy.mock.calls[2]?.[2]).toEqual(["models"]);
|
|
});
|
|
|
|
it("still rejects from the original catalog when post-refresh enumeration returns no models", async () => {
|
|
const spy = vi
|
|
.spyOn(serverUtils, "runChildProcess")
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "openrouter/example/stale-model\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "Models cache refreshed\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
});
|
|
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
}),
|
|
).rejects.toThrow("Available models: openrouter/example/stale-model");
|
|
expect(spy).toHaveBeenCalledTimes(3);
|
|
expect(spy.mock.calls[1]?.[2]).toEqual(["models", "--refresh"]);
|
|
expect(spy.mock.calls[2]?.[2]).toEqual(["models"]);
|
|
});
|
|
|
|
it("still rejects from the original catalog when refresh fails", async () => {
|
|
const warning = vi.spyOn(console, "warn").mockImplementation(() => {});
|
|
const spy = vi
|
|
.spyOn(serverUtils, "runChildProcess")
|
|
.mockResolvedValueOnce({
|
|
exitCode: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "openrouter/example/stale-model\n",
|
|
stderr: "",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
})
|
|
.mockRejectedValueOnce(new Error("refresh unavailable"));
|
|
|
|
await expect(
|
|
ensureOpenCodeModelConfiguredAndAvailable({
|
|
model: "openrouter/deepseek/deepseek-v4-flash-0731",
|
|
}),
|
|
).rejects.toThrow("Available models: openrouter/example/stale-model");
|
|
expect(spy).toHaveBeenCalledTimes(2);
|
|
expect(spy.mock.calls[1]?.[2]).toEqual(["models", "--refresh"]);
|
|
expect(warning).toHaveBeenCalledWith(
|
|
expect.stringContaining(
|
|
'refresh failed for "openrouter/deepseek/deepseek-v4-flash-0731"',
|
|
),
|
|
);
|
|
});
|
|
|
|
it("surfaces the last error once retries are exhausted", async () => {
|
|
vi.useFakeTimers();
|
|
const spy = vi.spyOn(serverUtils, "runChildProcess").mockResolvedValue({
|
|
exitCode: 1,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: "",
|
|
stderr: "queued behind another opencode run",
|
|
pid: 1,
|
|
startedAt: new Date().toISOString(),
|
|
});
|
|
|
|
const promise = discoverOpenCodeModels();
|
|
const assertion = expect(promise).rejects.toThrow(
|
|
"`opencode models` failed: queued behind another opencode run",
|
|
);
|
|
await vi.runAllTimersAsync();
|
|
await assertion;
|
|
expect(spy).toHaveBeenCalledTimes(3);
|
|
});
|
|
});
|