Files
PaperClipAI/server/src/services/native-runtime/runtime-mode.test.ts
T
Dotta ee2a190626 Unify Paperclip Runner experimental controls (#12666)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Runner is an experimental execution adapter.
> - The adapter and its required sandbox ingress had separate settings.
> - A user could enable one setting and still have an unusable runner
configuration.
> - The runtime already makes one durable native or legacy decision for
each run.
> - This pull request uses that runtime decision for ingress
authorization.
> - The benefit is one clear opt-in with safe recovery for existing
native runs.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

This improves the experimental settings and transport authorization for
Paperclip Runner.

**Subsystem affected**

Cross-cutting. This change affects the React settings UI, shared
settings contracts, adapter utilities, and server runtime selection.

**Current behavior**

Settings shows separate Paperclip Runner and Runner Preview Ingress
controls. A user can enable the runner but leave required sandbox
ingress disabled.

**Proposed behavior**

Settings shows only Paperclip Runner. Its native runtime decision also
authorizes provider WebSocket ingress when the execution target requires
it. A persisted native run keeps its recovery transport after the
setting is disabled.

**Reason and benefit**

Paperclip Runner is one experimental capability. One opt-in removes an
invalid partial configuration and makes the rollout boundary easier to
understand.

**Breaking changes**

The Runner Preview Ingress card is removed. The old
`enableRunnerPreviewIngress` key remains accepted in stored settings and
managed configuration, but it has no server runtime effect. The public
adapter-utils input remains compatible through a deprecated alias.

**Additional context**

Refs: #12638, #12641, #12656.

## What Changed

- Removed the separate Runner Preview Ingress card from Experimental
Settings.
- Made resolved native runtime selection authorize required provider
ingress.
- Preserved ingress recovery for persisted native runs after the rollout
flag is disabled.
- Kept the old settings key and adapter-utils input as deprecated
compatibility contracts.
- Added focused UI, runtime policy, transport, stored-settings, and
managed-config regression tests.
- Updated deployment documentation and feature descriptions.

## Verification

- GitHub Actions will run typecheck, tests, build, policy, and browser
shards.
- Focused tests cover the single settings control, runtime
authorization, fail-closed transport selection, the deprecated public
input, and old managed configuration.
- No local tests were run, per the maintainer request to use GitHub
Actions for verification.
- `git diff --check` passes.

## Risks

Low to moderate risk. The effective ingress gate changes from a separate
stored flag to the resolved native run decision. Fresh runs still
require `enableNativeRunner`. Persisted native runs remain recoverable.
Legacy adapters never receive ingress authorization.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5, with reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-01 09:21:23 -05:00

340 lines
12 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { BUILTIN_ADAPTER_TYPES } from "../../adapters/builtin-adapter-types.js";
import {
NativeRunnerSelectionError,
NativeRuntimeEligibilityError,
isRunnerIngressAuthorized,
resolveHeartbeatNativeRuntimeMode,
resolveHeartbeatRuntimeMode,
resolveNativeRuntimeMode,
} from "./runtime-mode.js";
const eligible = {
enabled: true,
runtimeConfig: {},
adapterConfig: { provider: "codex" },
agent: { status: "running", adapterType: "paperclip_runner" },
issue: { id: "issue", workMode: "standard" },
target: { kind: "local" },
workspaceId: "workspace",
} as const;
describe("resolveNativeRuntimeMode", () => {
it("keeps every direct built-in adapter outside native arbitration", () => {
for (const adapterType of BUILTIN_ADAPTER_TYPES) {
if (adapterType === "paperclip_runner") continue;
expect(resolveNativeRuntimeMode({
...eligible,
enabled: false,
runtimeConfig: {
nativeRunner: {
mode: "native",
backend: "codex_app_server",
protocolVersion: 1,
},
},
agent: { ...eligible.agent, adapterType },
})).toEqual({
kind: "legacy",
resolverVersion: "phase6-v1",
reason: "direct_adapter",
});
}
});
it("rejects a fresh Paperclip Runner start while the rollout flag is disabled", () => {
expect(() => resolveNativeRuntimeMode({
...eligible,
enabled: false,
})).toThrow(expect.objectContaining({
code: "paperclip_runner_rollout_disabled",
}));
});
it("rejects unknown Paperclip Runner providers", () => {
expect(() => resolveNativeRuntimeMode({
...eligible,
runtimeConfig: {},
adapterConfig: { provider: "claude" },
agent: { ...eligible.agent, adapterType: "paperclip_runner" },
})).toThrow(expect.objectContaining({
code: "paperclip_runner_provider_unsupported",
}));
});
it("rejects fresh OpenCode and ACPX starts until their app profiles are activated", () => {
expect(() => resolveNativeRuntimeMode({
...eligible,
adapterConfig: { provider: "opencode", model: "openrouter/deepseek/deepseek-v4-flash-0731" },
})).toThrow(expect.objectContaining({
code: "paperclip_runner_provider_unsupported",
}));
expect(() => resolveNativeRuntimeMode({
...eligible,
adapterConfig: { provider: "acpx", acpxAgent: "claude", model: "claude-sonnet-5" },
})).toThrow(expect.objectContaining({
code: "paperclip_runner_provider_unsupported",
}));
});
it("preserves legacy as the default and as the kill-switch behavior", () => {
const direct = {
...eligible,
agent: { ...eligible.agent, adapterType: "codex_local" },
runtimeConfig: { nativeRunner: { mode: "native", backend: "codex_app_server", protocolVersion: 1 } },
};
expect(resolveNativeRuntimeMode(direct)).toEqual(expect.objectContaining({
kind: "legacy",
reason: "direct_adapter",
}));
expect(resolveNativeRuntimeMode({ ...direct, enabled: false })).toEqual(expect.objectContaining({
kind: "legacy",
reason: "direct_adapter",
}));
});
it("selects native only for an eligible explicit profile", () => {
expect(resolveNativeRuntimeMode(eligible)).toEqual(expect.objectContaining({
kind: "native",
reason: "eligible_opt_in",
}));
});
it("keeps a persisted active run native while the global flag rejects a fresh runner start", () => {
const disabled = { ...eligible, enabled: false };
expect(resolveHeartbeatNativeRuntimeMode({
...disabled,
persisted: {
runtimeMode: "native",
runtimeModeReason: "eligible_opt_in",
runtimeModeResolvedAt: new Date(),
},
})).toEqual(expect.objectContaining({
kind: "native",
reason: "eligible_opt_in",
authorityDecision: expect.objectContaining({ reasonCode: "live_continuation_registered" }),
}));
expect(() => resolveHeartbeatNativeRuntimeMode({
...disabled,
persisted: { runtimeMode: null, runtimeModeReason: null, runtimeModeResolvedAt: null },
})).toThrow(expect.objectContaining({
code: "paperclip_runner_rollout_disabled",
}));
});
it("authorizes ingress from the native runtime decision without a second flag", () => {
const freshNative = resolveHeartbeatNativeRuntimeMode({
...eligible,
persisted: {
runtimeMode: null,
runtimeModeReason: null,
runtimeModeResolvedAt: null,
},
});
const persistedNative = resolveHeartbeatNativeRuntimeMode({
...eligible,
enabled: false,
persisted: {
runtimeMode: "native",
runtimeModeReason: "eligible_opt_in",
runtimeModeResolvedAt: new Date(),
},
});
const directLegacy = resolveHeartbeatNativeRuntimeMode({
...eligible,
enabled: false,
agent: { ...eligible.agent, adapterType: "codex_local" },
persisted: {
runtimeMode: null,
runtimeModeReason: null,
runtimeModeResolvedAt: null,
},
});
expect(isRunnerIngressAuthorized(freshNative)).toBe(true);
expect(isRunnerIngressAuthorized(persistedNative)).toBe(true);
expect(isRunnerIngressAuthorized(directLegacy)).toBe(false);
});
it.each(["paused", "terminated", "pending_approval"])(
"refuses persisted native recovery for a %s agent",
(status) => {
expect(() => resolveHeartbeatNativeRuntimeMode({
...eligible,
enabled: false,
agent: { ...eligible.agent, status },
persisted: {
runtimeMode: "native",
runtimeModeReason: "eligible_opt_in",
runtimeModeResolvedAt: new Date(),
driverKind: "codex_app_server",
},
})).toThrow(expect.objectContaining({
code: "paperclip_runner_agent_ineligible",
}));
},
);
it("fails closed for an unknown persisted driver", () => {
expect(() => resolveHeartbeatNativeRuntimeMode({
...eligible,
enabled: false,
persisted: {
runtimeMode: "native",
runtimeModeReason: "eligible_opt_in",
runtimeModeResolvedAt: new Date(),
driverKind: "unknown_driver",
},
})).toThrow(expect.objectContaining({
code: "paperclip_runner_driver_unsupported",
}));
});
it("does not recover a native run through a direct adapter", () => {
expect(() => resolveHeartbeatNativeRuntimeMode({
...eligible,
enabled: false,
agent: { ...eligible.agent, adapterType: "codex_local" },
persisted: {
runtimeMode: "native",
runtimeModeReason: "eligible_opt_in",
runtimeModeResolvedAt: new Date(),
driverKind: "codex_app_server",
},
})).toThrow(expect.objectContaining({
code: "paperclip_runner_adapter_binding_mismatch",
}));
});
it("keeps a persisted OpenCode recovery on its immutable driver", () => {
expect(resolveHeartbeatNativeRuntimeMode({
...eligible,
enabled: false,
persisted: {
runtimeMode: "native",
runtimeModeReason: "eligible_opt_in",
runtimeModeResolvedAt: new Date(),
driverKind: "opencode_server",
},
})).toEqual(expect.objectContaining({
profile: { mode: "native", backend: "opencode_server", protocolVersion: 1 },
}));
});
it("rejects an explicit native profile outside the approved boundary", () => {
expect(resolveNativeRuntimeMode({ ...eligible, agent: { ...eligible.agent, adapterType: "claude_local" } }))
.toEqual(expect.objectContaining({ kind: "legacy", reason: "direct_adapter" }));
expect(() => resolveNativeRuntimeMode({ ...eligible, issue: { id: "issue", workMode: "skill_test" } }))
.toThrow(NativeRuntimeEligibilityError);
});
it("admits remote targets only through paperclip_runner", () => {
expect(resolveNativeRuntimeMode({
...eligible,
target: { kind: "remote" },
runtimeConfig: {},
adapterConfig: { provider: "codex" },
agent: { ...eligible.agent, adapterType: "paperclip_runner" },
})).toMatchObject({ kind: "native" });
expect(resolveNativeRuntimeMode({
...eligible,
target: { kind: "remote" },
agent: { ...eligible.agent, adapterType: "codex_local" },
})).toMatchObject({ kind: "legacy", reason: "direct_adapter" });
});
it("allows paperclip_runner to use a transient local workspace for projectless issues", () => {
expect(resolveNativeRuntimeMode({
...eligible,
workspaceId: null,
agent: { ...eligible.agent, adapterType: "paperclip_runner" },
runtimeConfig: {},
adapterConfig: { provider: "codex" },
})).toEqual(expect.objectContaining({ kind: "native" }));
});
it("admits planning only through paperclip_runner", () => {
expect(resolveNativeRuntimeMode({
...eligible,
issue: { id: "plan-issue", workMode: "planning" },
runtimeConfig: {},
adapterConfig: { provider: "codex" },
agent: { ...eligible.agent, adapterType: "paperclip_runner" },
})).toMatchObject({ kind: "native", profile: { backend: "codex_app_server" } });
expect(resolveNativeRuntimeMode({
...eligible,
issue: { id: "plan-issue", workMode: "planning" },
agent: { ...eligible.agent, adapterType: "codex_local" },
})).toEqual(expect.objectContaining({ kind: "legacy", reason: "direct_adapter" }));
});
it("admits ask mode through paperclip_runner while preserving the legacy native boundary", () => {
expect(resolveNativeRuntimeMode({
...eligible,
issue: { id: "ask-issue", workMode: "ask" },
runtimeConfig: {},
adapterConfig: { provider: "codex" },
agent: { ...eligible.agent, adapterType: "paperclip_runner" },
})).toMatchObject({ kind: "native", profile: { backend: "codex_app_server" } });
expect(resolveNativeRuntimeMode({
...eligible,
issue: { id: "ask-issue", workMode: "ask" },
agent: { ...eligible.agent, adapterType: "codex_local" },
})).toEqual(expect.objectContaining({ kind: "legacy", reason: "direct_adapter" }));
});
});
const compatibilityInput = {
persisted: { runtimeMode: "legacy", runtimeModeResolvedAt: null },
enabled: true,
adapterConfig: { provider: "codex" },
agentStatus: "running",
issue: { workMode: "standard" },
executionTarget: { kind: "local" },
} as const;
describe("resolveHeartbeatRuntimeMode compatibility", () => {
it("keeps every direct built-in adapter on the legacy path", () => {
for (const adapterType of BUILTIN_ADAPTER_TYPES) {
if (adapterType === "paperclip_runner") continue;
expect(resolveHeartbeatRuntimeMode({
...compatibilityInput,
adapterType,
})).toEqual({
kind: "legacy",
resolverVersion: "paperclip-runner-v1",
reason: "direct_adapter",
});
}
});
it("preserves the original public result and error contracts", () => {
expect(resolveHeartbeatRuntimeMode({
...compatibilityInput,
adapterType: "paperclip_runner",
})).toEqual({
kind: "native",
resolverVersion: "paperclip-runner-v1",
reason: "explicit_paperclip_runner",
provider: "codex",
});
expect(resolveHeartbeatRuntimeMode({
...compatibilityInput,
enabled: false,
adapterType: "paperclip_runner",
persisted: { runtimeMode: "native", runtimeModeResolvedAt: new Date() },
})).toEqual({
kind: "native",
resolverVersion: "paperclip-runner-v1",
reason: "persisted_native_selection",
provider: "codex",
});
expect(() => resolveHeartbeatRuntimeMode({
...compatibilityInput,
enabled: false,
adapterType: "paperclip_runner",
})).toThrow(NativeRunnerSelectionError);
});
});