mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 12:37:55 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner now has protocol, provider, tool, package, persistence, and hidden server boundaries. > - The server still cannot select that path for a real agent heartbeat. > - A new runtime must not change any existing direct adapter. > - An experimental runtime must fail closed when its rollout flag is off. > - This pull request adds one guarded Codex vertical slice through runnerd. > - The benefit is a production-built runner path that users cannot start by default. ## Linked Issues or Issue Description Refs #11962 Refs #12111 Refs #12169 Refs #12176 **Subsystem affected** Cross-cutting. The change affects the runner package, server orchestration, shared settings, and adapter configuration UI. **Problem or motivation** The hidden PRP coordinator cannot execute a real heartbeat. The application also needs an explicit rollout boundary before it can expose the experimental runner. Existing direct adapters must keep their current execution and finalization behavior. **Proposed solution** Add `paperclip_runner` as a Codex-only adapter behind the default-off `enableNativeRunner` instance flag. Select the native runtime only for that adapter. Persist the run binding before runnerd starts. Wait for the durable PRP result and terminal event. Resume the real Codex provider thread on later heartbeats. Keep persisted native runs readable and recoverable after the flag changes. **Alternatives considered** The server could route `codex_local` through runnerd. That option would change an existing adapter and weaken rollback safety. The server could expose all providers now. That option would add unreviewed provider behavior. The build could depend on a prebuilt runner binary. That option would make source builds architecture-dependent and difficult to verify. **Roadmap alignment** This work supports the shipped enforced-outcomes, governed-tool, and self-healing-run milestones. It does not add a new roadmap surface. It is the guarded execution step after the merged hidden runner boundaries. **Additional context** This is the next replacement for the closed large runner pull request. Task-thread presentation remains a separate follow-up so this change can preserve the current direct-adapter UI. ## What Changed - Add `paperclip_runner` as an explicit Codex-only adapter. - Add the default-off `enableNativeRunner` instance flag. - Reject fresh create, hire, import, switch, and execution requests while the flag is off. - Allow edits to persisted runner agents while the flag is off. - Recover an already persisted native run even after the flag is disabled. - Keep every built-in direct adapter on its existing runtime path. - Persist an immutable native run binding and revisioned completion contract before runnerd starts. - Execute server to PRP to runnerd to Codex to server through the hidden coordinator. - Validate the durable result against the terminal event and exact completion criteria before finalization. - Preserve the Codex provider thread ID and use `thread/resume` on the next heartbeat. - Strip unsupported Codex configuration fields from the experimental adapter. - Build a target-native release runner binary from source and vendor it into the server distribution. - Install Rust only in the Docker build stage. Do not add a workflow or lockfile change. - Stop the runner process group on completion, cancellation, and forced shutdown. ## Verification - Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All 69 TypeScript tests and 58 Rust tests pass. Protocol, conformance, replay, formatting, and generated-file checks pass. - Run the 12 focused adapter, settings, runtime-selection, coordinator, direct-isolation, and real Codex integration test files. All 186 tests pass. - The real integration test uses PostgreSQL, HTTP, WebSocket, runnerd, and a fake Codex app server. It proves one `thread/start` followed by one `thread/resume`. - Run `pnpm -r typecheck`. - Run `pnpm build`. - Run `pnpm check:token-gates`. - Build the Docker `build` target from a clean context. Confirm that the server distribution contains an executable `paperclip-runnerd` built with Debian Rust 1.85. - Start the server through the source-mode tsx entry point with the package `dist` directory absent. Confirm the vendor shim resolves source exports and the server boots. - Run `pnpm test:run` twice. On this macOS host, 405 files pass and 1 file skips. Eight untouched workspace and loopback tests fail because macOS resolves `/tmp` and `/var` through `/private` and because PID-derived test ports exceed 65535. Linux CI must pass the full suite. - Confirm that the diff contains 52 files. Confirm that it contains no `.github` or `pnpm-lock.yaml` change. ## Risks - The feature flag is off by default. A fresh native start fails with a stable error while the flag is off. - A persisted native run remains recoverable after the flag changes. This prevents rollout changes from corrupting recorded work. - Only local Codex execution is accepted. Other providers and remote work modes fail closed. - Existing direct adapters do not start runnerd, create native rows, use native status arbitration, or enter native finalization. - The runner receives its one-use bootstrap ticket through the child environment. The server does not put the ticket in command arguments or logs. - The server validates the company, task, agent, run, runner, session, completion contract, result, and terminal binding before it accepts completion. - The build compiles a target-native Rust binary. Cross-platform release packaging remains a later concern. Source builds and Docker builds compile for their current target. - Docker needs enough build memory for the existing server TypeScript compile. The Docker build stage sets a 4 GB V8 heap limit. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The exact deployment ID and context-window size are not exposed. The model used agentic reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and applicable tests pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
315 lines
11 KiB
TypeScript
315 lines
11 KiB
TypeScript
import { z } from "zod";
|
|
import { instanceExperimentalSettingsSchema } from "./validators/instance.js";
|
|
|
|
/**
|
|
* Feature catalog for cloud-managed instances.
|
|
*
|
|
* The instance-settings zod schema is the feature manifest; this module adds
|
|
* only metadata about the flags the schema already declares. Keys are derived
|
|
* from the schema type, so adding, removing, or renaming a boolean flag in
|
|
* `instanceExperimentalSettingsSchema` without updating the metadata map is a
|
|
* compile error (and vice versa).
|
|
*
|
|
* Tiers:
|
|
* - `preference`: tenant-controllable taste setting; the cloud harness does
|
|
* not manage it.
|
|
* - `managed`: the cloud harness may set this per fleet/stack via
|
|
* `PAPERCLIP_MANAGED_CONFIG`.
|
|
* - `floor`: pinned by code on managed instances; no flag value may widen it.
|
|
*/
|
|
export const FEATURE_TIERS = ["preference", "managed", "floor"] as const;
|
|
|
|
export type FeatureTier = (typeof FEATURE_TIERS)[number];
|
|
|
|
type ExperimentalSettings = z.infer<typeof instanceExperimentalSettingsSchema>;
|
|
|
|
/**
|
|
* The boolean flag keys of the experimental settings schema. Non-flag keys
|
|
* (activation timestamps, numeric tuning values) are excluded.
|
|
*/
|
|
export type InstanceFeatureKey = {
|
|
[K in keyof ExperimentalSettings]: ExperimentalSettings[K] extends boolean ? K : never;
|
|
}[keyof ExperimentalSettings];
|
|
|
|
export interface FeatureCatalogEntry {
|
|
title: string;
|
|
description: string;
|
|
tier: FeatureTier;
|
|
/** Desired default on cloud-managed instances. */
|
|
cloudDefault: boolean;
|
|
/** Must match the schema default; enforced by test. */
|
|
selfHostedDefault: boolean;
|
|
}
|
|
|
|
export const INSTANCE_FEATURE_CATALOG: Record<InstanceFeatureKey, FeatureCatalogEntry> = {
|
|
enableEnvironments: {
|
|
title: "Environments",
|
|
description:
|
|
"Show environment management in company settings and allow project and agent environment assignment controls.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableNativeRunner: {
|
|
title: "Paperclip Runner",
|
|
description:
|
|
"Allow new Codex agents to use the experimental Rust Paperclip Runner transport.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableManagedSandboxOnly: {
|
|
title: "Managed Environment Only",
|
|
description:
|
|
"Hide the local environment and run all agents in the platform-managed environment.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableIsolatedWorkspaces: {
|
|
title: "Isolated Workspaces",
|
|
description:
|
|
"Show execution workspace controls in project configuration and allow isolated workspace behavior for task runs.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableStreamlinedLeftNavigation: {
|
|
title: "Streamlined Left Navigation",
|
|
description: "Use the streamlined main sidebar navigation layout.",
|
|
tier: "preference",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enableApps: {
|
|
title: "Apps",
|
|
description:
|
|
"Show the Apps navigation and allow access to app connections, gateways, and advanced app tooling.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enablePipelines: {
|
|
title: "Pipelines",
|
|
description: "Enable pipeline definitions and pipeline-driven case production surfaces.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableCases: {
|
|
title: "Cases",
|
|
description:
|
|
"Durable work products that tasks create and iterate on. Adds the Cases tab and the agent case API.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableConferenceRoomChat: {
|
|
title: "Conference Room Chat",
|
|
description:
|
|
"Add the Conference Room team chat, the live activity feed, and the redesigned onboarding; restyles task threads as chat bubbles.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableClassicTaskInterface: {
|
|
title: "Classic Task Interface",
|
|
description:
|
|
"Restore the pre-chat task detail page: the page-level header with inline description editor, the plain comment thread, and the fixed Properties sidebar. Chat-only features (streaming activity folding, inline plan/question cards, the three-mode composer) are unavailable in the classic view.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableTaskWatchdogs: {
|
|
title: "Task Watchdogs",
|
|
description:
|
|
"Show task detail controls for configuring watchdog agents that verify stopped task subtrees and restore live paths when work should continue.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableIssuePlanDecompositions: {
|
|
title: "Task Plan Decomposition Panel",
|
|
description: "Show accepted-plan decomposition history on task detail pages.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableExperimentalFileViewer: {
|
|
title: "Experimental File Viewer",
|
|
description:
|
|
"Show task detail controls for browsing and previewing workspace files relative to a task.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableStatusCards: {
|
|
title: "Status Cards",
|
|
description:
|
|
"Enable the experimental shared status-card board, update engine, and gated API.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableExternalObjects: {
|
|
title: "External Objects",
|
|
description:
|
|
"Detect external URLs in issues and show resolved status for pull requests, tickets, and other referenced work objects.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableSmokeLab: {
|
|
title: "Smoke Lab",
|
|
description:
|
|
"Add the Smoke Lab tab and dashboard card for exercising integration paths against deterministic local fixtures. Private deployments only.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableBuiltInAgents: {
|
|
title: "Built-in Agents",
|
|
description:
|
|
"Show Paperclip-managed built-in agent surfaces, including roster badges, the Built-in agents tab, and setup controls.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableBetaSkills: {
|
|
title: "Beta skills",
|
|
description: "Allow agents to pin beta releases of the Paperclip core skill.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableSummaries: {
|
|
title: "Summaries",
|
|
description:
|
|
"Show Summarizer-generated status slots on project and workspace pages, with on-demand refresh and revision history.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableDecisions: {
|
|
title: "Decisions",
|
|
description:
|
|
"Show the Decisions item in the main sidebar — the attention home that surfaces tasks awaiting input.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableGoalsSidebarLink: {
|
|
title: "Goals Sidebar Link",
|
|
description: "Restore the Goals item in the main sidebar while the goals surface is being evaluated.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableSimplifiedEnglishInteractions: {
|
|
title: "Simplified English Interactions",
|
|
description:
|
|
"Instruct agents to write user interactions (confirmations, questions, suggested tasks) in ASD-STE100 Simplified Technical English with brief decision context.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableServerInfoDebugView: {
|
|
title: "Server Info Debug View",
|
|
description:
|
|
"Show a Server section in the account drawer with the current server restart time and running commit.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
autoRestartDevServerWhenIdle: {
|
|
title: "Auto-Restart Dev Server When Idle",
|
|
description:
|
|
"In local development, wait for queued and running agent runs to finish, then restart the server automatically when backend changes make the current boot stale.",
|
|
tier: "preference",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableIssueGraphLivenessAutoRecovery: {
|
|
title: "Auto-Create Recovery Tasks",
|
|
description:
|
|
"Let the heartbeat scheduler create recovery tasks for task dependency chains found inside the configured lookback window.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableWorkspaceBranchReconcileForward: {
|
|
title: "Workspace Branch Reconcile Forward",
|
|
description:
|
|
"Let execution workspaces reconcile a diverged recorded branch forward instead of failing branch containment.",
|
|
tier: "managed",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enableWorkspaceDirtyQuarantineRepair: {
|
|
title: "Workspace Dirty Quarantine Repair",
|
|
description:
|
|
"Let workspace runtime recovery quarantine and repair dirty execution workspaces before runs.",
|
|
tier: "managed",
|
|
cloudDefault: true,
|
|
selfHostedDefault: true,
|
|
},
|
|
enableOwnerInstanceAdmin: {
|
|
title: "Owner Instance Admin",
|
|
description:
|
|
"On cloud-managed instances, grant the stack owner instance-admin access to their own dedicated instance. Elevation is computed at the trusted-header auth boundary; no instance admin role rows are created. Inert on self-hosted instances.",
|
|
tier: "managed",
|
|
cloudDefault: true,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableSandboxDuplexBridge: {
|
|
title: "Sandbox Duplex Bridge",
|
|
description:
|
|
"Let a run open the sandbox duplex command-stream bridge when the provider grants the capability. The host reads this per run before it selects the transport. Off keeps the file bridge for every run.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
enableWorktreeRunExecution: {
|
|
title: "Worktree Run Execution",
|
|
description:
|
|
"Let the scheduler execute runs inside an isolated git-worktree preview instance for tasks created after activation.",
|
|
tier: "managed",
|
|
cloudDefault: false,
|
|
selfHostedDefault: false,
|
|
},
|
|
};
|
|
|
|
export const INSTANCE_FEATURE_KEYS = Object.keys(INSTANCE_FEATURE_CATALOG).sort() as InstanceFeatureKey[];
|
|
|
|
/**
|
|
* Shape of the `feature-catalog.json` release artifact the cloud harness
|
|
* imports per app release and validates feature writes against.
|
|
*/
|
|
export const featureCatalogArtifactSchema = z
|
|
.object({
|
|
catalogVersion: z.string().min(1),
|
|
features: z.record(
|
|
z.string().min(1),
|
|
z.object({ tier: z.enum(FEATURE_TIERS) }).strict(),
|
|
),
|
|
})
|
|
.strict();
|
|
|
|
export type FeatureCatalogArtifact = z.infer<typeof featureCatalogArtifactSchema>;
|
|
|
|
export function buildFeatureCatalogArtifact(catalogVersion: string): FeatureCatalogArtifact {
|
|
if (catalogVersion.trim().length === 0) {
|
|
throw new Error("catalogVersion must be a non-empty string");
|
|
}
|
|
const features: FeatureCatalogArtifact["features"] = {};
|
|
for (const key of INSTANCE_FEATURE_KEYS) {
|
|
features[key] = { tier: INSTANCE_FEATURE_CATALOG[key].tier };
|
|
}
|
|
return { catalogVersion, features };
|
|
}
|
|
|
|
/** Deterministic serialization (sorted keys, trailing newline) for the artifact file. */
|
|
export function renderFeatureCatalogArtifact(catalogVersion: string): string {
|
|
return `${JSON.stringify(buildFeatureCatalogArtifact(catalogVersion), null, 2)}\n`;
|
|
}
|