Files
PaperClipAI/packages/plugins/sdk/tests/environment-task-release-resume.test.ts
T
Nicky LeachandClaude Opus 5.5 dce4d47f60 feat(plugins): carry Grok, Copilot and OpenCode credentials to plugin Runners (#22359)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Environment-driver plugins can run `paperclip_runner` tasks on hosts
that the plugin owns
> - Core transfers the selected managed AI credential to the plugin as a
typed task credential
> - The credential contract has only two types: a Codex auth file and a
direct Claude credential
> - Grok, GitHub Copilot and OpenCode agents in a task-capable plugin
environment therefore fail closed, although the Runner already reads
their credentials from its environment
> - This pull request adds one generic, allowlisted environment
credential for these three harnesses and accepts the Runner's Grok
launch settings
> - The benefit is that plugin-provided Runner execution can run Grok,
Copilot and OpenCode agents with their selected connection

## Linked Issues or Issue Description

**Subsystem affected**

Plugin environment tasks (`packages/plugins/sdk` task contract) and
plugin-provided Runner execution in `server`.

**Problem or motivation**

An agent with the Grok, Copilot or OpenCode harness cannot run in a
plugin environment that runs Runner tasks. Core stops at credential
preparation with "Plugin Runner does not support the selected credential
provider". A Grok agent fails before that point too. The Runner emits
`grokEffort` and `grokDisableWebSearch` in its launch requirements (Core
turns off Grok web search by default), but the strict `settings` schema
of `environmentRunnerLaunchRequirementsSchema` rejects both keys.

**Proposed solution**

Core transfers the selected connection's credential variables for these
harnesses through one typed `provider-env` credential. Each provider has
a fixed allowlist of complete variable sets. The plugin sets the
variables in the Runner process environment.

**Alternatives considered**

One credential type per harness (`grok-env`, `copilot-env`,
`opencode-env`). This adds a type for each new harness. One generic type
with an exported allowlist keeps the same strictness with less surface.

**Roadmap alignment**

The roadmap lists Grok and OpenCode as supported harnesses. This change
lets plugin environments run them. It adds no new product concept.

**Additional context**

Grok subscription login (an auth file with refresh and writeback) is out
of scope. It still fails closed, with an error that names the xAI API
key as the supported option.

## What Changed

- `packages/plugins/sdk`: Added the `provider-env` task credential: `{
type: "provider-env", provider: "grok" | "copilot" | "opencode",
environment }`. The environment must contain exactly one complete
variable set for the provider and no other variable. Values are
non-empty, NUL-free and at most 65,536 characters, as for the other
credential types.
  - `grok`: `XAI_API_KEY`
  - `copilot`: `COPILOT_GITHUB_TOKEN`
- `opencode`: `OPENROUTER_API_KEY`, or `PAPERCLIP_AI_PROVIDER_URL` with
`PAPERCLIP_AI_PROVIDER_KEY`
- `packages/plugins/sdk`: Exported the allowlist as
`ENVIRONMENT_TASK_PROVIDER_CREDENTIAL_KEY_SETS` and the
`PluginEnvironmentTaskCredentialProvider` type. `claude-env` and
`codex-auth-json` do not change.
- `packages/plugins/sdk`: Added `grokEffort` (`minimal` to `xhigh`) and
`grokDisableWebSearch` (boolean) to the launch requirements `settings`,
with the types that the Runner uses.
- `server`: `preparePluginRunnerCredentials` now transfers the selected
connection's variables for the `grok`, `copilot` and `opencode`
harnesses, with the same account identity semantics as the Claude
branch. It refuses a Grok subscription, a Grok base URL override, and an
incomplete or ambiguous set. The error never contains a credential
value.
- `server`: The submit check now matches a `provider-env` credential to
the task harness. Before, it treated every non-Codex credential as
Anthropic.
- Drift test:
`server/src/__tests__/plugin-runner-credential-names.test.ts` checks the
SDK key sets against the Runner. The SDK list stays written by hand, and
the Runner keeps its own lists of forwarded names. For `grok` and
`copilot`, each name must be in `ACPX_CREDENTIAL_NAMES`, survive sidecar
admission with the session binding that the Runner mints, and reach the
agent process. The binding requirement must match the Rust sidecar's
candidate agents. For `opencode`, each name must be in the OpenCode
proxy environment set. Each name must also be in the Rust sidecar
credential keys or the Rust OpenCode provider keys. The test reads those
from the Runner source, as the Codex security test does. A provider
without a Runner route fails the test.
`OPEN_CODE_RUNNER_ENVIRONMENT_KEYS` is now exported for this test.
- Docs: Added a "Typed credentials" section to
`doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md`. It also documents the ACPX
credential binding that the Runner requires with a Copilot token.
Updated `doc/plugins/PLUGIN_RUNNER_EXECUTION.md`.

## Verification

- `npx vitest run tests/environment-task-release-resume.test.ts
tests/environment-tasks.test.ts` in `packages/plugins/sdk`: 96 passed.
The tests accept each valid shape. They reject mixed sets, two sets,
extra variables, another provider's variable, empty values, NUL bytes,
oversized values, unknown providers and extra fields, on `submit` and on
`resume`. They also check that every published allowlist set parses.
- `npx vitest run
src/services/native-runtime/plugin-runner-credentials.test.ts
src/__tests__/environment-task-runtime.test.ts` in `server`: 79 passed.
The new tests cover each provider's transfer, Grok subscription refusal,
Grok routing refusal, the fail-closed cases without value leaks, harness
matching at submit, and Runner-built Grok requirements
(`runnerProviderLaunchRequirements`) that parse through the task
contract and produce a Grok launch.
- `npx vitest run src/__tests__/plugin-runner-credential-names.test.ts`
in `server`: 4 passed. I broke each list locally to confirm that the
test fails: a renamed SDK key, an SDK provider without a Runner route,
and a dropped name in the Rust sidecar keys, the Rust candidate agents,
the Rust OpenCode keys and the TypeScript OpenCode keys. Each change
failed the test.
- `node scripts/typecheck-layers.mjs` in `server` (includes
`src/__tests__`): no errors.
- `npx vitest run
server/src/services/native-runtime/plugin-runner-probe.test.ts` (from
the root): 15 passed.
`server/src/services/native-runtime/plugin-runner-task.test.ts`: passed.
- `tsc --noEmit` for `packages/plugins/sdk` and `server`: no errors.
- I did not run the full suite locally. CI supplies that coverage.

## Risks

- Low risk for existing plugins. The change only adds a credential type
and two optional settings. A plugin that does not handle `provider-env`
gets a typed value that it does not recognize. Before this change, Core
failed these runs before it called the plugin.
- A plugin must set `PAPERCLIP_ACPX_CREDENTIAL_BINDING` for a Copilot
task. Otherwise the Runner refuses the token. The new doc section shows
the value.
- An OpenCode gateway connection without a key (`auth: none`) still
fails closed, because every value must be non-empty.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, through Claude Code
with tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-20 08:47:18 -08:00

213 lines
15 KiB
TypeScript

import { createInterface } from "node:readline";
import { PassThrough } from "node:stream";
import { describe, expect, it } from "vitest";
import { definePlugin } from "../src/define-plugin.js";
import {
environmentTaskOperationSchema,
environmentTaskResultSchema,
parseEnvironmentTaskResult,
type PluginEnvironmentTaskOperation,
type PluginEnvironmentTaskParams,
type PluginEnvironmentTaskResult,
} from "../src/environment-tasks.js";
import { createRequest, isJsonRpcResponse, parseMessage, serializeMessage, type JsonRpcResponse } from "../src/protocol.js";
import { startWorkerRpcHost } from "../src/worker-rpc-host.js";
const identity = { runnerInstanceId: "runner", environmentLeaseId: "lease", runId: "source-run",
normalizedSessionId: "session", turnId: "turn", itemId: "item" };
const release = { kind: "release", identity } as const;
const resume = { kind: "resume", identity, bootstrapTicket: "fixture-ticket" } as const;
const ownership = {
kind: "ensure_ownership", operationId: "9df4526f-36c5-4fba-9cf6-3c4fd317f531", identity,
sourceTaskId: "source-task", sourceGeneration: "source-generation", targetGeneration: "target-generation", policy: "reclaim_if_needed",
} as const;
const description = { kind: "description", taskId: "task", runner: { version: "fixture", digest: `sha256:${"a".repeat(64)}` } };
const submit = { kind: "submit", projectIds: [], bootstrapTicket: "fixture-ticket", runner: { harness: "claude", protocolMin: 1, protocolMax: 3,
runnerId: "runner", leaseId: "lease", runId: "run", sessionId: "session", turnId: "turn", itemId: "item" } } as const;
const credentials: [string, unknown, boolean][] = [
["a Claude OAuth token", { type: "claude-env", environment: { CLAUDE_CODE_OAUTH_TOKEN: "fixture-oauth" } }, true],
["a Claude API key", { type: "claude-env", environment: { ANTHROPIC_API_KEY: "fixture-key" } }, true],
["a Codex auth file", { type: "codex-auth-json", authJson: "{\"tokens\":{}}" }, true],
["two Claude credentials", { type: "claude-env", environment: { CLAUDE_CODE_OAUTH_TOKEN: "a", ANTHROPIC_API_KEY: "b" } }, false],
["no Claude credential", { type: "claude-env", environment: {} }, false],
["an unsupported Claude variable", { type: "claude-env", environment: { ANTHROPIC_BASE_URL: "https://example.test" } }, false],
["an empty value", { type: "claude-env", environment: { CLAUDE_CODE_OAUTH_TOKEN: "" } }, false],
["a NUL byte", { type: "claude-env", environment: { CLAUDE_CODE_OAUTH_TOKEN: "a\0b" } }, false],
["an oversized value", { type: "codex-auth-json", authJson: "x".repeat(65_537) }, false],
["an unknown type", { type: "gemini-env", environment: { GEMINI_API_KEY: "fixture" } }, false],
["an extra field", { type: "codex-auth-json", authJson: "{}", path: "/home/agent/.codex" }, false],
["an xAI API key for Grok", { type: "provider-env", provider: "grok", environment: { XAI_API_KEY: "fixture-xai" } }, true],
["a Copilot token", { type: "provider-env", provider: "copilot", environment: { COPILOT_GITHUB_TOKEN: "fixture-copilot" } }, true],
["an OpenRouter key for OpenCode", { type: "provider-env", provider: "opencode", environment: { OPENROUTER_API_KEY: "fixture-openrouter" } }, true],
["an OpenCode gateway", { type: "provider-env", provider: "opencode",
environment: { PAPERCLIP_AI_PROVIDER_URL: "https://gateway.example.test/v1", PAPERCLIP_AI_PROVIDER_KEY: "fixture-gateway" } }, true],
["half of an OpenCode gateway", { type: "provider-env", provider: "opencode", environment: { PAPERCLIP_AI_PROVIDER_URL: "https://gateway.example.test/v1" } }, false],
["a mixed OpenCode set", { type: "provider-env", provider: "opencode",
environment: { OPENROUTER_API_KEY: "fixture-openrouter", PAPERCLIP_AI_PROVIDER_KEY: "fixture-gateway" } }, false],
["two OpenCode sets", { type: "provider-env", provider: "opencode", environment: { OPENROUTER_API_KEY: "fixture-openrouter",
PAPERCLIP_AI_PROVIDER_URL: "https://gateway.example.test/v1", PAPERCLIP_AI_PROVIDER_KEY: "fixture-gateway" } }, false],
["another provider's variable", { type: "provider-env", provider: "grok", environment: { COPILOT_GITHUB_TOKEN: "fixture-copilot" } }, false],
["an extra provider variable", { type: "provider-env", provider: "grok", environment: { XAI_API_KEY: "fixture-xai", XAI_BASE_URL: "https://router.example.test" } }, false],
["no provider variable", { type: "provider-env", provider: "copilot", environment: {} }, false],
["an empty provider value", { type: "provider-env", provider: "copilot", environment: { COPILOT_GITHUB_TOKEN: "" } }, false],
["a NUL byte in a provider value", { type: "provider-env", provider: "grok", environment: { XAI_API_KEY: "a\0b" } }, false],
["an oversized provider value", { type: "provider-env", provider: "grok", environment: { XAI_API_KEY: "x".repeat(65_537) } }, false],
["an unsupported provider", { type: "provider-env", provider: "gemini", environment: { GEMINI_API_KEY: "fixture-gemini" } }, false],
["a provider credential without a provider", { type: "provider-env", environment: { XAI_API_KEY: "fixture-xai" } }, false],
["an extra provider credential field", { type: "provider-env", provider: "grok", environment: { XAI_API_KEY: "fixture-xai" }, home: "/home/agent" }, false],
];
async function invoke(operation: unknown, handler: (params: PluginEnvironmentTaskParams) => Promise<PluginEnvironmentTaskResult>) {
const stdin = new PassThrough(), stdout = new PassThrough(), lines = createInterface({ input: stdout });
const worker = startWorkerRpcHost({ stdin, stdout, plugin: definePlugin({ async setup() {}, onEnvironmentTask: handler }) });
try {
const response = new Promise<JsonRpcResponse>(resolve => lines.on("line", line => {
const message = parseMessage(line);
if (isJsonRpcResponse(message)) resolve(message);
}));
stdin.write(serializeMessage(createRequest("environmentTask", { companyId: "company", taskId: "task",
runId: "claimant-run", lease: { providerLeaseId: "task" }, operation }, "lifecycle")));
return await response;
} finally { worker.stop(); lines.close(); stdin.destroy(); stdout.destroy(); }
}
describe("release and resume task contract", () => {
it("keeps the capability optional and independent of warm and source recovery support", () => {
for (const capabilities of [{}, { releaseResumeVersion: 1 }, { warmTaskVersion: 1 },
{ codexSourceRecoveryVersion: 1 }, { acpxSourceRecoveryVersion: 1 }]) {
expect(environmentTaskResultSchema.parse({ ...description, ...capabilities })).toEqual({ ...description, ...capabilities });
}
expect(environmentTaskResultSchema.safeParse({ ...description, releaseResumeVersion: 2 }).success).toBe(false);
expect(environmentTaskResultSchema.parse({ ...description, resumeLaunchInputsVersion: 1 })).toEqual({ ...description, resumeLaunchInputsVersion: 1 });
expect(environmentTaskResultSchema.safeParse({ ...description, resumeLaunchInputsVersion: 2 }).success).toBe(false);
expect(environmentTaskResultSchema.parse({ ...description, resumeRelaunchedRunnerVersion: 1 })).toEqual({ ...description, resumeRelaunchedRunnerVersion: 1 });
expect(environmentTaskResultSchema.safeParse({ ...description, resumeRelaunchedRunnerVersion: 2 }).success).toBe(false);
});
it("reports the Runner build a resume relaunched on only when asked", () => {
expect(environmentTaskOperationSchema.parse({ ...resume, relaunchedRunnerVersion: 1 })).toEqual({ ...resume, relaunchedRunnerVersion: 1 });
expect(environmentTaskOperationSchema.safeParse({ ...resume, relaunchedRunnerVersion: 2 }).success).toBe(false);
const runner = { version: "0.4.1", digest: `sha256:${"b".repeat(64)}` };
const result = { kind: "resume", taskId: "task", outcome: "resumed", runner };
expect(parseEnvironmentTaskResult(resume, "task", result)).toEqual(result);
expect(environmentTaskResultSchema.safeParse({ ...result, runner: { ...runner, digest: "sha256:short" } }).success).toBe(false);
expect(environmentTaskResultSchema.safeParse({ ...result, runner: { ...runner, extra: true } }).success).toBe(false);
});
it("relaunches as the claimant's identity only when asked, and reports the archived state", () => {
expect(environmentTaskResultSchema.parse({ ...description, resumeIdentityVersion: 1 })).toEqual({ ...description, resumeIdentityVersion: 1 });
expect(environmentTaskResultSchema.safeParse({ ...description, resumeIdentityVersion: 2 }).success).toBe(false);
const runner = { runId: "claimant-run", turnId: "turn-claimant-run", itemId: "item-claimant-run" };
const relaunch = { ...resume, runner };
expect(environmentTaskOperationSchema.parse(relaunch)).toEqual(relaunch);
for (const changed of [{ ...runner, sessionId: "other" }, { ...runner, runId: undefined }, { ...runner, turnId: "" }]) {
expect(environmentTaskOperationSchema.safeParse({ ...resume, runner: changed }).success).toBe(false);
}
const build = { version: "0.4.1", digest: `sha256:${"b".repeat(64)}` };
const archivedState = { schema: "paperclip.runner.durable.state.v1", ...identity, lifecycle: "ready" };
const resumed = { kind: "resume", taskId: "task", outcome: "resumed", runner: build, archivedState };
expect(parseEnvironmentTaskResult(relaunch, "task", resumed)).toEqual(resumed);
expect(() => parseEnvironmentTaskResult(relaunch, "task", { ...resumed, archivedState: undefined })).toThrow();
expect(() => parseEnvironmentTaskResult(relaunch, "task", { ...resumed, runner: undefined })).toThrow();
expect(() => parseEnvironmentTaskResult(resume, "task", resumed)).toThrow();
expect(parseEnvironmentTaskResult(relaunch, "task", { kind: "resume", taskId: "task", outcome: "unavailable" }))
.toEqual({ kind: "resume", taskId: "task", outcome: "unavailable" });
});
it.each(credentials)("validates %s on resume as on submit", (_label, credential, valid) => {
expect(environmentTaskOperationSchema.safeParse({ ...submit, credential }).success).toBe(valid);
expect(environmentTaskOperationSchema.safeParse({ ...resume, credential }).success).toBe(valid);
});
it("dispatches a resume credential through the serialized worker RPC", async () => {
const operation = { ...resume, credential: { type: "claude-env", environment: { CLAUDE_CODE_OAUTH_TOKEN: "fixture-oauth" } } } as const;
const result = { kind: "resume", taskId: "task", outcome: "resumed" } as const;
expect(await invoke(operation, async params => {
expect(params.operation).toEqual(operation);
return result;
})).toMatchObject({ result });
});
it.each([release, resume])("requires the exact durable identity for $kind", operation => {
expect(environmentTaskOperationSchema.parse(operation)).toEqual(operation);
for (const field of Object.keys(identity)) {
expect(environmentTaskOperationSchema.safeParse({ ...operation, identity: { ...identity, [field]: undefined } }).success).toBe(false);
}
expect(environmentTaskOperationSchema.safeParse({ ...operation, identity: { ...identity, targetRunId: "other" } }).success).toBe(false);
expect(environmentTaskOperationSchema.safeParse({ ...operation, cancelActiveWork: true }).success).toBe(false);
});
it.each([undefined, "", "\0", "x".repeat(65_537)])("rejects a missing or invalid resume ticket", bootstrapTicket => {
expect(environmentTaskOperationSchema.safeParse({ ...resume, bootstrapTicket }).success).toBe(false);
});
it("matches each operation to its own closed result and task", () => {
const released = { kind: "released", taskId: "task" };
expect(parseEnvironmentTaskResult(release, "task", released)).toEqual(released);
for (const outcome of ["resumed", "unavailable"]) {
const result = { kind: "resume", taskId: "task", outcome };
expect(parseEnvironmentTaskResult(resume, "task", result)).toEqual(result);
expect(() => parseEnvironmentTaskResult(release, "task", result)).toThrow();
}
for (const operation of [release, resume]) {
expect(() => parseEnvironmentTaskResult(operation, "task", { kind: "accepted", taskId: "task" })).toThrow();
const result = operation.kind === "release" ? released : { kind: "resume", taskId: "task", outcome: "resumed" };
expect(() => parseEnvironmentTaskResult(operation, "other-task", result)).toThrow();
}
expect(() => parseEnvironmentTaskResult(resume, "task", { kind: "resume", taskId: "task", outcome: "unknown" })).toThrow();
expect(environmentTaskResultSchema.safeParse({ ...released, retained: true }).success).toBe(false);
expect(environmentTaskResultSchema.safeParse({ kind: "resume", taskId: "task", outcome: "resumed", bootstrapTicket: "fixture-ticket" }).success).toBe(false);
});
it.each([release, resume])("dispatches $kind through the serialized worker RPC", async operation => {
const result: PluginEnvironmentTaskResult = operation.kind === "release" ? { kind: "released", taskId: "task" }
: { kind: "resume", taskId: "task", outcome: "resumed" };
const response = await invoke(operation, async params => {
expect(params.operation).toEqual(operation);
expect(params.runId).toBe("claimant-run");
return result;
});
expect(response).toMatchObject({ result });
});
it("passes known absence through RPC without treating it as a successful reservation", async () => {
expect(await invoke(resume, async () => ({ kind: "resume", taskId: "task", outcome: "unavailable" })))
.toMatchObject({ result: { kind: "resume", taskId: "task", outcome: "unavailable" } });
});
it("binds a declarative ownership response to the exact worker request", async () => {
for (const status of ["pending", "ready"] as const) {
const result = { kind: "ownership", status, taskId: "task", operationId: ownership.operationId,
targetGeneration: ownership.targetGeneration } as const;
expect(await invoke(ownership, async params => {
expect(params.operation).toEqual(ownership);
return result;
})).toMatchObject({ result });
for (const changes of [
{ taskId: "another-task" },
{ operationId: "1f083ad0-a357-4f14-ad46-e9df981845b5" },
{ targetGeneration: "stale-generation" },
]) {
expect(await invoke(ownership, async () => ({ ...result, ...changes }))).toHaveProperty("error");
}
}
});
it.each(["source mismatch", "claim conflict", "authentication failed", "transport timeout"])("keeps %s as an RPC error", async message => {
expect(await invoke(resume, async () => { throw new Error(message); })).toHaveProperty("error");
});
it("rejects cross-task and wrong-kind replies at the worker boundary", async () => {
for (const result of [{ kind: "resume", taskId: "other", outcome: "resumed" }, { kind: "accepted", taskId: "task" }] as PluginEnvironmentTaskResult[]) {
expect(await invoke(resume, async () => result)).toHaveProperty("error");
}
});
it("preserves legacy describe and stop replies through RPC", async () => {
expect(await invoke({ kind: "describe" }, async () => description as PluginEnvironmentTaskResult)).toMatchObject({ result: description });
const operation: PluginEnvironmentTaskOperation = { kind: "stop" };
expect(await invoke(operation, async () => ({ kind: "accepted", taskId: "task" })))
.toMatchObject({ result: { kind: "accepted", taskId: "task" } });
});
});