Files
Dotta 397de98193 feat(runner): add flagged Codex execution adapter (#12188)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Paperclip Runner now has protocol, provider, tool, package,
persistence, and hidden server boundaries.
> - The server still cannot select that path for a real agent heartbeat.
> - A new runtime must not change any existing direct adapter.
> - An experimental runtime must fail closed when its rollout flag is
off.
> - This pull request adds one guarded Codex vertical slice through
runnerd.
> - The benefit is a production-built runner path that users cannot
start by default.

## Linked Issues or Issue Description

Refs #11962

Refs #12111

Refs #12169

Refs #12176

**Subsystem affected**

Cross-cutting. The change affects the runner package, server
orchestration, shared settings, and adapter configuration UI.

**Problem or motivation**

The hidden PRP coordinator cannot execute a real heartbeat. The
application also needs an explicit rollout boundary before it can expose
the experimental runner. Existing direct adapters must keep their
current execution and finalization behavior.

**Proposed solution**

Add `paperclip_runner` as a Codex-only adapter behind the default-off
`enableNativeRunner` instance flag. Select the native runtime only for
that adapter. Persist the run binding before runnerd starts. Wait for
the durable PRP result and terminal event. Resume the real Codex
provider thread on later heartbeats. Keep persisted native runs readable
and recoverable after the flag changes.

**Alternatives considered**

The server could route `codex_local` through runnerd. That option would
change an existing adapter and weaken rollback safety. The server could
expose all providers now. That option would add unreviewed provider
behavior. The build could depend on a prebuilt runner binary. That
option would make source builds architecture-dependent and difficult to
verify.

**Roadmap alignment**

This work supports the shipped enforced-outcomes, governed-tool, and
self-healing-run milestones. It does not add a new roadmap surface. It
is the guarded execution step after the merged hidden runner boundaries.

**Additional context**

This is the next replacement for the closed large runner pull request.
Task-thread presentation remains a separate follow-up so this change can
preserve the current direct-adapter UI.

## What Changed

- Add `paperclip_runner` as an explicit Codex-only adapter.
- Add the default-off `enableNativeRunner` instance flag.
- Reject fresh create, hire, import, switch, and execution requests
while the flag is off.
- Allow edits to persisted runner agents while the flag is off.
- Recover an already persisted native run even after the flag is
disabled.
- Keep every built-in direct adapter on its existing runtime path.
- Persist an immutable native run binding and revisioned completion
contract before runnerd starts.
- Execute server to PRP to runnerd to Codex to server through the hidden
coordinator.
- Validate the durable result against the terminal event and exact
completion criteria before finalization.
- Preserve the Codex provider thread ID and use `thread/resume` on the
next heartbeat.
- Strip unsupported Codex configuration fields from the experimental
adapter.
- Build a target-native release runner binary from source and vendor it
into the server distribution.
- Install Rust only in the Docker build stage. Do not add a workflow or
lockfile change.
- Stop the runner process group on completion, cancellation, and forced
shutdown.

## Verification

- Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All 69
TypeScript tests and 58 Rust tests pass. Protocol, conformance, replay,
formatting, and generated-file checks pass.
- Run the 12 focused adapter, settings, runtime-selection, coordinator,
direct-isolation, and real Codex integration test files. All 186 tests
pass.
- The real integration test uses PostgreSQL, HTTP, WebSocket, runnerd,
and a fake Codex app server. It proves one `thread/start` followed by
one `thread/resume`.
- Run `pnpm -r typecheck`.
- Run `pnpm build`.
- Run `pnpm check:token-gates`.
- Build the Docker `build` target from a clean context. Confirm that the
server distribution contains an executable `paperclip-runnerd` built
with Debian Rust 1.85.
- Start the server through the source-mode tsx entry point with the
package `dist` directory absent. Confirm the vendor shim resolves source
exports and the server boots.
- Run `pnpm test:run` twice. On this macOS host, 405 files pass and 1
file skips. Eight untouched workspace and loopback tests fail because
macOS resolves `/tmp` and `/var` through `/private` and because
PID-derived test ports exceed 65535. Linux CI must pass the full suite.
- Confirm that the diff contains 52 files. Confirm that it contains no
`.github` or `pnpm-lock.yaml` change.

## Risks

- The feature flag is off by default. A fresh native start fails with a
stable error while the flag is off.
- A persisted native run remains recoverable after the flag changes.
This prevents rollout changes from corrupting recorded work.
- Only local Codex execution is accepted. Other providers and remote
work modes fail closed.
- Existing direct adapters do not start runnerd, create native rows, use
native status arbitration, or enter native finalization.
- The runner receives its one-use bootstrap ticket through the child
environment. The server does not put the ticket in command arguments or
logs.
- The server validates the company, task, agent, run, runner, session,
completion contract, result, and terminal binding before it accepts
completion.
- The build compiles a target-native Rust binary. Cross-platform release
packaging remains a later concern. Source builds and Docker builds
compile for their current target.
- Docker needs enough build memory for the existing server TypeScript
compile. The Docker build stage sets a 4 GB V8 heap limit.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment ID and context-window
size are not exposed. The model used agentic reasoning, repository
tools, code execution, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and applicable tests pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-08-25 16:03:41 -05:00

273 lines
10 KiB
TypeScript

import type { UIAdapterModule } from "./types";
import { claudeLocalUIAdapter } from "./claude-local";
import { codexLocalUIAdapter } from "./codex-local";
import { paperclipRunnerUIAdapter } from "./paperclip-runner";
import { cursorCloudUIAdapter } from "./cursor-cloud";
import { cursorLocalUIAdapter } from "./cursor";
import { geminiLocalUIAdapter } from "./gemini-local";
import { grokLocalUIAdapter } from "./grok-local";
import { kimiLocalUIAdapter } from "./kimi-local";
import { hermesGatewayUIAdapter } from "./hermes-gateway";
import { hermesLocalUIAdapter } from "./hermes-local";
import { openCodeLocalUIAdapter } from "./opencode-local";
import { piLocalUIAdapter } from "./pi-local";
import { openClawGatewayUIAdapter } from "./openclaw-gateway";
import { processUIAdapter } from "./process";
import { httpUIAdapter } from "./http";
import { loadDynamicParser, invalidateDynamicParser, setDynamicParserResultNotifier } from "./dynamic-loader";
import { SchemaConfigFields, buildSchemaAdapterConfig } from "./schema-config-fields";
const uiAdapters: UIAdapterModule[] = [];
const adaptersByType = new Map<string, UIAdapterModule>();
// Types registered at module load time — allowed to be overridden by
// external adapters that ship their own ui-parser.js via the server.
const builtinTypes = new Set<string>();
// Original builtin adapters stored for restoration when external overrides
// are deactivated or removed.
const builtinAdaptersByType = new Map<string, UIAdapterModule>();
// Tracks which builtin types currently have an active external override.
const activeExternalOverrides = new Set<string>();
// Generation counter to discard stale dynamic parser loads. When an override
// is deactivated while a load is in-flight, the generation is bumped and the
// stale result is discarded in its .then() handler.
const overrideGeneration = new Map<string, number>();
// Subscriber list — components can register to be notified when adapters change
// (e.g., when a dynamic parser replaces a placeholder).
const adapterChangeListeners = new Set<() => void>();
/** Subscribe to adapter registry changes. Returns unsubscribe function. */
export function onAdapterChange(fn: () => void): () => void {
adapterChangeListeners.add(fn);
return () => adapterChangeListeners.delete(fn);
}
function notifyAdapterChange(): void {
for (const fn of adapterChangeListeners) fn();
}
setDynamicParserResultNotifier(notifyAdapterChange);
function registerBuiltInUIAdapters() {
for (const adapter of [
claudeLocalUIAdapter,
codexLocalUIAdapter,
paperclipRunnerUIAdapter,
cursorCloudUIAdapter,
geminiLocalUIAdapter,
grokLocalUIAdapter,
kimiLocalUIAdapter,
hermesGatewayUIAdapter,
hermesLocalUIAdapter,
openCodeLocalUIAdapter,
piLocalUIAdapter,
cursorLocalUIAdapter,
openClawGatewayUIAdapter,
processUIAdapter,
httpUIAdapter,
]) {
builtinTypes.add(adapter.type);
builtinAdaptersByType.set(adapter.type, adapter);
registerUIAdapter(adapter);
}
}
export function registerUIAdapter(adapter: UIAdapterModule): void {
const existingIndex = uiAdapters.findIndex((entry) => entry.type === adapter.type);
if (existingIndex >= 0) {
uiAdapters.splice(existingIndex, 1, adapter);
} else {
uiAdapters.push(adapter);
}
adaptersByType.set(adapter.type, adapter);
notifyAdapterChange();
}
export function unregisterUIAdapter(type: string): void {
if (type === processUIAdapter.type || type === httpUIAdapter.type) return;
const existingIndex = uiAdapters.findIndex((entry) => entry.type === type);
if (existingIndex >= 0) {
uiAdapters.splice(existingIndex, 1);
}
adaptersByType.delete(type);
}
export function findUIAdapter(type: string): UIAdapterModule | null {
return adaptersByType.get(type) ?? null;
}
registerBuiltInUIAdapters();
export function getUIAdapter(type: string): UIAdapterModule {
const builtIn = adaptersByType.get(type);
if (!builtIn) {
let loadStarted = false;
return {
type,
label: type,
parseStdoutLine: (line: string, ts: string) => {
if (!loadStarted) {
loadStarted = true;
loadDynamicParser(type).then((parserModule) => {
if (parserModule) {
registerUIAdapter({
type,
label: type,
parseStdoutLine: parserModule.parseStdoutLine,
createStdoutParser: parserModule.createStdoutParser,
ConfigFields: SchemaConfigFields,
buildAdapterConfig: buildSchemaAdapterConfig,
});
}
});
}
return processUIAdapter.parseStdoutLine(line, ts);
},
ConfigFields: SchemaConfigFields,
buildAdapterConfig: buildSchemaAdapterConfig,
};
}
return builtIn;
}
/**
* Keep the UI adapter registry in sync with the server's adapter list.
*
* Two concerns:
*
* 1. **Builtin overrides** — when an external adapter ships a ui-parser.js for a
* builtin type, the external parser takes priority. When the external is
* disabled or removed the original builtin parser is restored transparently.
* A generation counter guards against stale loads that resolve after the
* override has been torn down.
*
* 2. **Non-builtin externals** — register a bridge adapter that lazily loads the
* dynamic parser on first stdout line, falling back to the generic process
* adapter. Once the parser resolves the bridge is replaced.
*/
export function syncExternalAdapters(
serverAdapters: {
type: string;
label: string;
disabled?: boolean;
/** When true, the external override for a builtin type is client-side paused. */
overrideDisabled?: boolean;
}[],
): void {
const enabledExternalTypes = new Set(
serverAdapters.filter((a) => !a.disabled && !a.overrideDisabled).map((a) => a.type),
);
const allExternalTypes = new Set(
serverAdapters.map((a) => a.type),
);
// ── Builtin override lifecycle ──────────────────────────────────────────
for (const builtinType of builtinTypes) {
const originalBuiltin = builtinAdaptersByType.get(builtinType);
if (!originalBuiltin) continue;
const hasExternal = allExternalTypes.has(builtinType);
const externalEnabled = enabledExternalTypes.has(builtinType);
const wasOverridden = activeExternalOverrides.has(builtinType);
if (hasExternal && externalEnabled && !wasOverridden) {
// Activate: external just became active → replace builtin with bridge.
activeExternalOverrides.add(builtinType);
const gen = (overrideGeneration.get(builtinType) ?? 0) + 1;
overrideGeneration.set(builtinType, gen);
let loadStarted = false;
const fallbackParser = originalBuiltin.parseStdoutLine;
const externalEntry = serverAdapters.find((a) => a.type === builtinType);
const label = externalEntry?.label ?? builtinType;
registerUIAdapter({
type: builtinType,
label,
parseStdoutLine: (line: string, ts: string) => {
if (!loadStarted) {
loadStarted = true;
loadDynamicParser(builtinType).then((parserModule) => {
// Discard if the override was torn down while the load was in-flight.
if (parserModule && overrideGeneration.get(builtinType) === gen) {
registerUIAdapter({
type: builtinType,
label,
parseStdoutLine: parserModule.parseStdoutLine,
createStdoutParser: parserModule.createStdoutParser,
ConfigFields: originalBuiltin.ConfigFields,
buildAdapterConfig: originalBuiltin.buildAdapterConfig,
});
}
});
}
return fallbackParser(line, ts);
},
ConfigFields: originalBuiltin.ConfigFields,
buildAdapterConfig: originalBuiltin.buildAdapterConfig,
});
} else if ((!hasExternal || !externalEnabled) && wasOverridden) {
// Deactivate: external disabled or removed → restore builtin.
activeExternalOverrides.delete(builtinType);
overrideGeneration.delete(builtinType);
invalidateDynamicParser(builtinType);
registerUIAdapter(originalBuiltin);
}
}
// ── Non-builtin externals ───────────────────────────────────────────────
for (const { type, label } of serverAdapters) {
if (builtinTypes.has(type)) continue; // handled above
const existing = adaptersByType.get(type);
// If this type already has an externally-loaded dynamic parser, skip —
// it was loaded from disk on a previous sync. Only re-trigger loading
// when the server returns a new external adapter that hasn't been loaded yet.
if (existing && existing !== processUIAdapter) continue;
let loadStarted = false;
// Use the existing built-in parser as fallback (if any) so we don't
// regress to the generic process parser while the dynamic one loads.
const fallbackParser = existing?.parseStdoutLine ?? processUIAdapter.parseStdoutLine;
registerUIAdapter({
type,
label,
parseStdoutLine: (line: string, ts: string) => {
if (!loadStarted) {
loadStarted = true;
loadDynamicParser(type).then((parserModule) => {
if (parserModule) {
registerUIAdapter({
type,
label,
parseStdoutLine: parserModule.parseStdoutLine,
createStdoutParser: parserModule.createStdoutParser,
ConfigFields: existing?.ConfigFields ?? SchemaConfigFields,
buildAdapterConfig: existing?.buildAdapterConfig ?? buildSchemaAdapterConfig,
});
}
});
}
return fallbackParser(line, ts);
},
ConfigFields: existing?.ConfigFields ?? SchemaConfigFields,
buildAdapterConfig: existing?.buildAdapterConfig ?? buildSchemaAdapterConfig,
});
}
}
export function listUIAdapters(): UIAdapterModule[] {
return [...uiAdapters];
}