Files
PaperClipAI/packages/plugins/sdk/tests/environment-tasks.test.ts
T
Nicky LeachandClaude Opus 5.5 2483cb46c2 feat(plugins): deliver configured environment to plugin Runner tasks (#21280)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - An agent gets environment variables from its execution environment,
its own configuration, its project, and its routine. Secrets are
resolved for each run.
> - A plugin environment can run the agent as a native Runner task on
its own host, through the environment task contract.
> - That contract has no field for the agent's variables. Runs on a
plugin environment get only the variables that the plugin sets itself.
> - This pull request adds a versioned contract field for the values,
sends them on a session handoff, lets Codex commands inherit the
variables in an external sandbox, and lets a plugin protect the
variables that it sets itself.
> - The benefit is that plain and secret variables work on plugin
environments in the same way as on local environments.

## Linked Issues or Issue Description

Refs #21233. That pull request by @lucktastic finds the same gap and
proposes the same capability and field names
(`configuredEnvironmentVersion: 1`, `configuredEnvironment`). This pull
request uses the same names, so providers implement one contract. It
differs in these places:

- Codex commands get the variables through an external-sandbox denylist,
not through key names sent at `prepare`. A retained Runner does not need
a new preparation when a key is added or removed.
- A session handoff `resume` gets the values, because it starts a new
Runner on another computer. Value changes on the same computer need no
new field: Core's process reuse key includes the values, so Core starts
a new task.
- A driver can declare `protectedEnvironmentKeys`. Paperclip rejects or
warns when one is saved.
- The run log names skipped variables only for an older provider.

Thank you to @lucktastic for the diagnosis and the contract shape.

**What happened?**
An agent has environment variables, plain and secret. Its runs on a
plugin task environment do not get them. `PAPERCLIP_CONFIGURED_ENV_KEYS`
in the run lists only the variables that the plugin sets.

**Expected behavior**
Each run gets the agent's configured variables.

**Steps to reproduce**
1. Set a plain and a secret environment variable on an agent.
2. Assign the agent to a plugin environment that provides Runner tasks.
3. Start a run. Read the names in `PAPERCLIP_CONFIGURED_ENV_KEYS` in a
command that the agent runs.
4. The agent's names are not in the list.

## What Changed

- **SDK contract** (`packages/plugins/sdk/src/environment-tasks.ts`):
- A provider advertises `configuredEnvironmentVersion: 1` on `describe`.
- `submit` gets `configuredEnvironment`. `resume` gets it only for a
session handoff.
- Names, count, and sizes have the same limits as the Runner projection.
- **Server**:
- `createRunnerdBackendWithinSessionClaim` gives the run's configured
projection to the plugin task owner. The projection has no provider
credentials and no generated runtime keys.
- `createPluginRunnerTaskOwner` sends the fields only to a provider with
the capability. An older provider gets no new field, and the run log
names the variables that were not set. The log never prints values.
- **Codex** (`codex-security-config.ts`):
- An externally sandboxed launch uses `inherit="all"`,
`ignore_default_excludes=true`, and an `exclude` list of Codex's own
credentials, instead of an `include_only` list fixed at preparation
time.
- A test reads the Runner's Codex environment keys, and fails if a new
key is not classified.
  - Host launches keep the bounded allowlist.
- **runnerd** (`codex_provider.rs`): `runtime_notification_codex_args`
still grants the notification files to read-only profiles when no
allowlist is present. Plan mode in an external sandbox needs these
grants.
- **Protected keys**:
- An environment driver can declare `protectedEnvironmentKeys` in its
manifest.
- Paperclip rejects a protected key on an agent or environment that uses
the driver. The 422 has code `protected_environment_variable`.
- On a project, a routine, or another agent, Paperclip saves the key.
The response has a `configurationWarnings` notice that names the
environments. The project and routine pages show the notice as a toast.
The agent pages already show configuration warnings.
- **Docs**: `doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md` describes the
provider obligations.
- `tests/runner-e2e/source-sdk-public-inputs.json`: refresh the reviewed
SDK source pin.

## Verification

- `packages/plugins/sdk`: `vitest run tests/environment-tasks.test.ts`
(38 pass). Accepts bounded values. Rejects bad names, NUL bytes, too
many variables, and oversized values.
- `server`: `vitest run
src/services/native-runtime/plugin-runner-task.test.ts` (95 pass).
Values go only to a provider with the capability, never in `prepare`.
`resume` sends values only for a handoff.
- `server`: `vitest run src/services/protected-environment-keys.test.ts
src/__tests__/project-routes-env.test.ts` (10 pass). Rejection, notices,
and the project route response.
- `server`: all route suites that import the project, routine,
environment, or agent routes (45 files, 1689 pass).
- `packages/shared`: `vitest run src/validators/plugin.test.ts` (64
pass).
- `packages/paperclip-runner`: `vitest run src/drivers/codex
src/configured-environment.test.ts
src/live/runner-provider-launch.test.ts` (444 pass).
- `packages/paperclip-runner/runner`: `cargo test -p
paperclip-runner-core --lib runtime_notification` (9 pass, 2 ignored as
before).
- `pnpm typecheck` in `packages/shared`, `packages/plugins/sdk`,
`packages/paperclip-runner`, `server`, and `ui`.
- Not run locally: `pnpm verify --full` and browser suites. Required CI
covers the final revision.

## Risks

- **Existing providers:** low risk. Core sends no new field to a
provider without the capability. Their runs behave as before, with one
added log line when variables were skipped.
- **Codex in an external sandbox:** commands now see every variable in
the Runner environment except Codex's own credentials. The execution
environment builds that environment from an allowlist. If the Runner
gives Codex a new credential, the drift test fails until the key is
classified.
- **Saving env:** new rejections happen only when an installed driver
declares `protectedEnvironmentKeys`. No driver declares them today.
- **Providers:** they must keep the values transient and never save or
log them. The doc lists these obligations.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 08:11:23 +00:00

127 lines
7.7 KiB
TypeScript

import {describe,expect,it} from "vitest";
import {environmentTaskOperationSchema,environmentTaskResultSchema} from "../src/environment-tasks.js";
const submit={kind:"submit",projectIds:[],bootstrapTicket:"synthetic",runner:{harness:"codex",protocolMin:1,protocolMax:3,runnerId:"runner",leaseId:"lease",runId:"run",sessionId:"session",turnId:"turn",itemId:"item"},
launch:{runnerVersion:"fixture",runnerDigest:`sha256:${"a".repeat(64)}`,maxOutboxBytes:262144,p0ReserveBytes:65536,lifecyclePolicy:{mode:"warm",idleTimeoutMs:300000}}};
describe("task lifecycle policy",()=>{
it("accepts bounded warm policy and the existing per-turn policy",()=>{
expect(environmentTaskOperationSchema.parse(submit)).toEqual(submit);
expect(environmentTaskOperationSchema.safeParse({...submit,launch:{...submit.launch,lifecyclePolicy:{mode:"per_turn",idleTimeoutMs:null}}}).success).toBe(true);
});
it.each([0,-1,86400001,null,1.5])("rejects invalid warm idle timeout %s",idleTimeoutMs=>{
expect(environmentTaskOperationSchema.safeParse({...submit,launch:{...submit.launch,lifecyclePolicy:{mode:"warm",idleTimeoutMs}}}).success).toBe(false);
});
});
const prepare = {
kind: "prepare", projectIds: [], requirements: {
schema: "paperclip.runner.launch-requirements.v1",
identity: { runnerInstanceId: "runner", environmentLeaseId: "lease", runId: "run",
normalizedSessionId: "session", turnId: "turn", itemId: "item" },
provider: "codex", model: "selected-model", instructions: "Verify the selected account.",
runtimeContext: null, settings: {}, approvalPolicy: "never", readOnly: true,
preparedConversation: false, includeCollaborationModeInstructions: false,
},
};
describe("retained source launch fields", () => {
it("uses the existing source identity and resume reference without a recovery mode", () => {
for (const operation of [prepare, submit]) {
const recovery = { ...operation, accountIdentity: "selected-account", resumeFrom: "source-task" };
expect(environmentTaskOperationSchema.parse(recovery)).toEqual(recovery);
expect(environmentTaskOperationSchema.safeParse({ ...recovery, recoveryPurpose: "source_reconcile" }).success).toBe(false);
}
});
});
describe("provider-specific source recovery capability", () => {
const description = { kind: "description", taskId: "task", runner: { version: "fixture", digest: `sha256:${"a".repeat(64)}` } };
it.each(["codexSourceRecoveryVersion", "acpxSourceRecoveryVersion"] as const)("accepts %s independently", key => {
const advertised = { ...description, [key]: 1 };
expect(environmentTaskResultSchema.parse(advertised)).toEqual(advertised);
expect(environmentTaskResultSchema.safeParse({ ...description, [key]: 2 }).success).toBe(false);
});
it("accepts both or neither and rejects an unqualified recovery flag", () => {
const both = { ...description, codexSourceRecoveryVersion: 1, acpxSourceRecoveryVersion: 1 };
expect(environmentTaskResultSchema.parse(both)).toEqual(both);
expect(environmentTaskResultSchema.parse(description)).toEqual(description);
expect(environmentTaskResultSchema.safeParse({ ...description, sourceRecoveryVersion: 1 }).success).toBe(false);
});
});
describe("task hosted search restriction", () => {
it("preserves Core's explicit restriction through plugin launch preparation", () => {
const operation = { ...prepare, requirements: { ...prepare.requirements,
settings: { disableCodexWebSearch: true } } };
expect(environmentTaskOperationSchema.parse(operation)).toEqual(operation);
});
it("keeps the provider default when the restriction is absent", () => {
expect(environmentTaskOperationSchema.parse(prepare)).toEqual(prepare);
});
it.each([false, null, 0, "true", "disabled"])("rejects a non-restricting flag value %s", disableCodexWebSearch => {
expect(environmentTaskOperationSchema.safeParse({ ...prepare, requirements: { ...prepare.requirements,
settings: { disableCodexWebSearch } } }).success).toBe(false);
});
it("continues rejecting unreviewed launch settings", () => {
expect(environmentTaskOperationSchema.safeParse({ ...prepare, requirements: { ...prepare.requirements,
settings: { disableCodexWebSearch: true, hostedSearchBillingMode: "unmetered" } } }).success).toBe(false);
});
});
describe("Codex probe project-instruction policy", () => {
it("preserves explicit suppression alongside the hosted-search restriction", () => {
const operation = { ...prepare, requirements: { ...prepare.requirements,
settings: { disableCodexProjectInstructions: true, disableCodexWebSearch: true } } };
expect(environmentTaskOperationSchema.parse(operation)).toEqual(operation);
});
it.each([false, null, 0, "true"])("rejects an invalid suppression value %s", disableCodexProjectInstructions => {
expect(environmentTaskOperationSchema.safeParse({ ...prepare, requirements: { ...prepare.requirements,
settings: { disableCodexProjectInstructions } } }).success).toBe(false);
});
});
describe("implementation-independent GitHub credential broker", () => {
it("accepts capability data without an executable, script, or filesystem path", () => {
const operation = { ...prepare, githubBroker: { url: "https://core.example.test/api", token: "runtime-capability" } };
expect(environmentTaskOperationSchema.parse(operation)).toEqual(operation);
});
it.each(["invalid", "file:///tmp/broker", "https://user:password@core.example.test", "https://core.example.test?token=secret"])("rejects broker transport %s", url => {
expect(environmentTaskOperationSchema.safeParse({ ...prepare, githubBroker: { url, token: "runtime-capability" } }).success).toBe(false);
});
it("rejects executable delivery as part of this contract", () => {
expect(environmentTaskOperationSchema.safeParse({ ...prepare, githubBroker: {
url: "https://core.example.test", token: "runtime-capability", script: "run-this" } }).success).toBe(false);
});
});
describe("configured environment", () => {
const configured = { ...submit, configuredEnvironment: { STRIPE_SECRET_KEY: "fixture-value", EMPTY_OK: "" } };
it("accepts bounded variables on submit and on a handoff resume", () => {
expect(environmentTaskOperationSchema.parse(configured)).toEqual(configured);
const identity = { runnerInstanceId: "runner", environmentLeaseId: "lease", runId: "run", normalizedSessionId: "session", turnId: "turn", itemId: "item" };
const resume = { kind: "resume", identity, bootstrapTicket: "synthetic", configuredEnvironment: { A: "1" } };
expect(environmentTaskOperationSchema.parse(resume)).toEqual(resume);
});
it.each([
["an invalid name", { "1BAD": "x" }],
["a dashed name", { "BAD-NAME": "x" }],
["a NUL byte", { GOOD: "a\0b" }],
["too many variables", Object.fromEntries(Array.from({ length: 129 }, (_, index) => [`K${index}`, "x"]))],
["an oversized value", { BIG: "x".repeat(65_536) }],
["an oversized total", Object.fromEntries(Array.from({ length: 5 }, (_, index) => [`K${index}`, "x".repeat(60_000)]))],
])("rejects %s", (_label, configuredEnvironment) => {
expect(environmentTaskOperationSchema.safeParse({ ...configured, configuredEnvironment }).success).toBe(false);
});
it("has no digest field", () => {
expect(environmentTaskOperationSchema.safeParse({ ...configured, configuredEnvironmentDigest: `hmac-sha256:${"b".repeat(64)}` }).success).toBe(false);
});
it("lets a provider advertise support", () => {
const description = { kind: "description", taskId: "task", runner: { version: "fixture", digest: `sha256:${"a".repeat(64)}` }, configuredEnvironmentVersion: 1 };
expect(environmentTaskResultSchema.parse(description)).toEqual(description);
expect(environmentTaskResultSchema.safeParse({ ...description, configuredEnvironmentVersion: 2 }).success).toBe(false);
});
});