Open Codex driver to authorized protocol tools

doc/plans/paperclip-runner-protocol-binding-roster.md

@@ -0,0 +1,45 @@
+# Paperclip Runner protocol binding roster
+
+This roster is the implementation ledger for binding the canonical Paperclip Runner protocol to the real Paperclip control plane. The action definitions under `packages/paperclip-runner/src/protocol-actions/` remain the source of truth for schemas and examples; this document records production integration status.
+
+## Current baseline
+
+- Canonical protocol actions: **41**
+- Actions in the Codex semantic catalog: **28**
+- Actions with a real Paperclip service binding: **0**
+- Actions audited end-to-end over production PRP: **0**
+- Existing semantic conformance bridge: five route-backed test operations (`report_progress`, `write_document`, confirmation-only `request_human_input`, `set_dependencies`, and `finish_task`). It is test-only and is not a production binding.
+
+The generated action metadata currently says `realServiceBinding: unbound` and `prpBindingStatus: audit_pending` for every action. `realBindingStatus: live_codex` means only that a Codex scenario exercised the mock semantic implementation; it must not be interpreted as a production Paperclip binding.
+
+## Binding groups
+
+| Group | Actions | Real authority target | Status |
+| --- | --- | --- | --- |
+| Active task reads | `get_task_context`, `get_task_history`, `search_tasks` | issue, comment, project, and assignment services | unbound |
+| Task mutations | `report_progress`, `answer_status_question`, `finish_task`, `block_task`, `request_review`, `set_dependencies`, `create_task` | issue/comment services and native status arbiter | unbound |
+| Documents and deliverables | `list_documents`, `read_document`, `list_document_revisions`, `write_document`, `register_deliverable` | document and attachment services | unbound |
+| Human interaction | `request_human_input` | interaction service and continuation scheduler | unbound |
+| Agents | `list_agents`, `get_agent` | agent service with company scope | unbound |
+| Approvals | `list_approvals`, `get_approval`, `get_approval_context`, `request_approval`, `decide_approval`, `comment_on_approval` | approval service | unbound |
+| Workspace | `get_workspace_runtime`, `control_workspace_service` | workspace runtime service | unbound |
+| Scheduling | `schedule_wake` | wakeup service | unbound |
+| Discovery | tool search and tool schema lookup | authorized catalog projection | mock implementation only |
+| Administrative surface | `administer_company`, `export_company`, `list_company_skills`, `sync_company_skills`, `list_projects`, `list_goals`, `list_routines`, `manage_routine`, `list_cases`, `upsert_case`, `inspect_operation_result` | existing company-scoped API/services; individually authorization-gated | protocol-only/scenario mock |
+| Secrets | `list_secret_metadata`, `read_secret_value` | secrets service; metadata/value claims separated and output redacted | protocol-only/scenario mock |
+| Escape hatch | `generic_api_request` | no production binding planned until a narrow allowlist is designed | test-only |
+
+## Rules for marking a binding complete
+
+An action moves from `unbound` only when all of the following are true:
+
+1. The runner advertises the canonical input and response schema only when the run identity is authorized for the operation.
+2. The PRP call is bound to company, issue, agent, run, session, turn, call, lease, catalog digest, and protocol version.
+3. The server invokes the same service or HTTP route used by the real application; no mock state reducer is in the production path.
+4. Existing authorization, optimistic concurrency, idempotency, redaction, audit, and continuation behavior is preserved.
+5. Unit tests cover allow/deny/replay behavior and an end-to-end real-server eval records the tool call, state diff, thread projection, and assertion result.
+6. The generated action metadata names the concrete service binding and records the completed PRP audit.
+
+## Cutover order
+
+The first production vertical slice is `get_task_context` + `report_progress` + `finish_task`. Next are documents, interactions, task dependencies/creation, approvals, agent/project reads, workspace/scheduling, then the separately privileged administrative and secret surfaces. `generic_api_request` stays unavailable in production.

packages/paperclip-runner/src/backends/codex-native-backend.ts

@@ -2,6 +2,7 @@ import { createCodexTaskEnvelope } from "../contracts/codex.js";
 import type { NativeExecutionInputV1 } from "../contracts/native-execution.js";
 import type { NativeSessionBackend } from "../contracts/native-session-backend.js";
 import { CodexAppServerDriver } from "../drivers/codex/codex-app-server-driver.js";
+import type { CodexAppServerTransport } from "../drivers/codex/app-server-transport.js";
 import { HarnessDriverBackend } from "./harness-driver-backend.js";
 
 /**
@@ -17,6 +18,15 @@ export function createCodexNativeSessionBackend(
       processGroupId: number | null;
       startedAt: string;
     }) => Promise<void>;
+    transportFactory?: () => CodexAppServerTransport;
+    dynamicTools?: readonly Readonly<Record<string, unknown>>[];
+    dynamicToolHandler?: (call: {
+      tool: string;
+      callId: string;
+      threadId: string;
+      turnId: string;
+      arguments: unknown;
+    }) => Promise<unknown>;
   } = {},
 ): NativeSessionBackend {
   return new HarnessDriverBackend(new CodexAppServerDriver({
@@ -33,5 +43,8 @@ export function createCodexNativeSessionBackend(
     }),
     runnerInstanceId: options.runnerInstanceId ?? `paperclip-native-${input.binding.runId}`,
     onSpawn: options.onSpawn,
+    transportFactory: options.transportFactory,
+    dynamicTools: options.dynamicTools,
+    dynamicToolHandler: options.dynamicToolHandler,
   }));
 }

packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.test.ts

@@ -940,6 +940,42 @@ describe("Codex app-server Codex driver", () => {
     expect(events.filter((event) => event.eventType === "turn.completed")).toHaveLength(1);
   });
 
+  it("advertises and dispatches run-authorized control-plane tools", async () => {
+    const transport = new FakeCodexTransport();
+    const handler = vi.fn(async (call) => ({ task: { id: "issue-1" }, callId: call.callId }));
+    const session = await makeDriver([transport], {
+      dynamicTools: [{
+        name: "get_task_context",
+        description: "Read the assigned task.",
+        inputSchema: { type: "object", additionalProperties: false },
+      }],
+      dynamicToolHandler: handler,
+    }).openSession({ runId: "run-tools", normalizedSessionId: "normalized-tools", workingDirectory: "/workspace" });
+    await session.startTurn({ message: { role: "user", text: "Inspect the task." } });
+
+    expect(transport.calls.find((call) => call.method === "thread/start")?.params.dynamicTools)
+      .toEqual(expect.arrayContaining([expect.objectContaining({ name: "get_task_context" })]));
+    const response = await transport.invoke({
+      id: "rpc-tool",
+      method: "item/tool/call",
+      params: {
+        threadId: "thread-1",
+        turnId: "turn-1",
+        callId: "call-context",
+        tool: "get_task_context",
+        arguments: {},
+      },
+    });
+    expect(response).toMatchObject({ success: true });
+    expect(JSON.parse(String((response.contentItems as Array<Record<string, unknown>>)[0]?.text)))
+      .toEqual({ task: { id: "issue-1" }, callId: "call-context" });
+    expect(handler).toHaveBeenCalledWith(expect.objectContaining({
+      tool: "get_task_context",
+      callId: "call-context",
+      arguments: {},
+    }));
+  });
+
   it("fails closed when an agent message changes a tool-committed result", async () => {
     const transport = new FakeCodexTransport();
     const session = await makeDriver([transport]).openSession({

packages/paperclip-runner/src/drivers/codex/codex-app-server-driver.ts

@@ -92,6 +92,16 @@ export interface CodexAppServerDriverOptions {
   taskEnvelope: CodexTaskEnvelope;
   conversationMode?: "task" | "direct";
   transportFactory?: () => CodexAppServerTransport;
+  /** Additional control-plane tools exposed to the provider for this run. */
+  dynamicTools?: readonly Readonly<Record<string, unknown>>[];
+  /** Executes an admitted additional tool call. Completion tools remain driver-owned. */
+  dynamicToolHandler?: (call: {
+    tool: string;
+    callId: string;
+    threadId: string;
+    turnId: string;
+    arguments: unknown;
+  }) => Promise<unknown>;
   environment?: NodeJS.ProcessEnv;
   now?: () => Date;
   runnerInstanceId?: string;
@@ -218,6 +228,16 @@ function blockToolSpec(): Record<string, unknown> {
   };
 }
 
+function dynamicToolResponse(value: unknown): Record<string, unknown> {
+  return {
+    success: true,
+    contentItems: [{
+      type: "inputText",
+      text: typeof value === "string" ? value : JSON.stringify(value),
+    }],
+  };
+}
+
 const SKILLLESS_BASE_CONFIG = {
   "skills.include_instructions": false,
   include_apps_instructions: false,
@@ -359,7 +379,9 @@ export class CodexAppServerDriver implements HarnessDriver {
         ...(this.#direct() ? {} : { baseInstructions: CODEX_SKILLLESS_BASE_INSTRUCTIONS }),
         dynamicTools: this.#direct()
           ? []
-          : this.#caps.dynamicTools ? [finishToolSpec(), blockToolSpec()] : [],
+          : this.#caps.dynamicTools
+            ? [...(this.#options.dynamicTools ?? []), finishToolSpec(), blockToolSpec()]
+            : [],
         experimentalRawEvents: false,
         persistExtendedHistory: false,
       });
@@ -577,7 +599,9 @@ export class CodexAppServerDriver implements HarnessDriver {
         ).sort(),
         dynamicToolNames: this.#direct()
           ? []
-          : this.#caps.dynamicTools ? [...CODEX_SEMANTIC_TOOL_NAMES] : [],
+          : this.#caps.dynamicTools
+            ? [...(this.#options.dynamicTools ?? []).map((tool) => text(tool.name)), ...CODEX_SEMANTIC_TOOL_NAMES]
+            : [],
         modelInputKinds: ["text"],
         liveConsole: {
           conversationMode: this.#direct() ? "direct" : "task",
@@ -612,6 +636,8 @@ export class CodexAppServerDriver implements HarnessDriver {
       now: this.#options.now ?? (() => new Date()),
       runnerInstanceId: this.#options.runnerInstanceId ?? "runner-codex",
       capabilities: this.#caps,
+      dynamicTools: this.#options.dynamicTools ?? [],
+      dynamicToolHandler: this.#options.dynamicToolHandler,
     });
   }
 }
@@ -626,6 +652,8 @@ class CodexHarnessSession implements HarnessSession {
   readonly #now: () => Date;
   readonly #runnerInstanceId: string;
   readonly #capabilities: CodexCapabilities;
+  readonly #dynamicTools: readonly Readonly<Record<string, unknown>>[];
+  readonly #dynamicToolHandler: CodexAppServerDriverOptions["dynamicToolHandler"];
   readonly #events = new AsyncQueue<PrpEvent>();
   #sourceSequence: number;
   #activeTurnId: string | null;
@@ -666,6 +694,8 @@ class CodexHarnessSession implements HarnessSession {
     now: () => Date;
     runnerInstanceId: string;
     capabilities: CodexCapabilities;
+    dynamicTools: readonly Readonly<Record<string, unknown>>[];
+    dynamicToolHandler?: CodexAppServerDriverOptions["dynamicToolHandler"];
   }) {
     this.#transport = input.transport;
     this.#runId = input.runId;
@@ -678,6 +708,8 @@ class CodexHarnessSession implements HarnessSession {
     this.#now = input.now;
     this.#runnerInstanceId = input.runnerInstanceId;
     this.#capabilities = input.capabilities;
+    this.#dynamicTools = input.dynamicTools;
+    this.#dynamicToolHandler = input.dynamicToolHandler;
     this.#goal = input.goal === undefined ? null : structuredClone(input.goal);
     for (const entry of input.lineage ?? [input.opened.lineage]) {
       this.#lineage.set(entry.threadId, structuredClone(entry));
@@ -1282,6 +1314,23 @@ class CodexHarnessSession implements HarnessSession {
         return rejectedToolCall("Semantic tool call was outside the active thread and turn.");
       }
       if (!isSemanticTool(tool)) {
+        const admitted = this.#dynamicTools.some((candidate) => candidate.name === tool);
+        if (admitted && this.#dynamicToolHandler !== undefined) {
+          try {
+            return dynamicToolResponse(await this.#dynamicToolHandler({
+              tool,
+              callId,
+              threadId,
+              turnId,
+              arguments: request.params.arguments,
+            }));
+          } catch (error) {
+            return {
+              success: false,
+              contentItems: [{ type: "inputText", text: boundedText(error instanceof Error ? error.message : error) }],
+            };
+          }
+        }
         this.#diagnoseUnsupported(`dynamic tool ${tool}`);
         return rejectedToolCall("Unsupported tool.");
       }