feat(runner): route provider tools through rust daemon

This commit is contained in:
Dotta committed 2026-08-20 21:38:11 -05:00
1 parent 2a199e42f0
commit 7658c876e1
20 files changed
+1520 -49

No files matched your search

@@ -0,0 +1,122 @@
# Single-daemon sandbox runner and tool bridge
Date: 2026-08-20
Status: implemented; provider reliability follow-up remains
## Target
The sandbox contains one Paperclip-owned process: `paperclip-runnerd`. It owns
the authenticated durable connection to the Paperclip control plane, launches
the selected provider harness (Codex, Claude, ACPX, or another adapter), exposes
only the tools authorized for the run, and forwards tool calls to the control
plane. No Node/TypeScript Paperclip dispatcher runs in the sandbox.
```text
Paperclip control plane
<== authenticated, durable PRP ==>
paperclip-runnerd (sandbox)
<== provider protocol ==>
Codex / Claude / ACPX
```
## Ownership
- The control plane owns company state, authorization, action execution,
idempotency, audit, and the authorized action set for each run.
- Runnerd owns process containment, provider lifecycle, protocol translation,
durable event delivery, correlated tool-call forwarding, interruption, and
recovery.
- Provider adapters translate a provider's native tool protocol into the same
runner-internal `ToolCall`/`ToolResult` interface. They contain no Paperclip
business rules.
- TypeScript in this package is the reference control plane, eval UI, and
conformance oracle. It is not part of the sandbox runtime.
## Protocol cutover
1. `run.prepare` supplies the provider configuration and authorized tool
definitions (name, description, input schema, response schema, contract
version). Runnerd rejects duplicate names, unsupported versions, and schema
changes on a resumed durable identity.
2. `turn.start` starts or resumes the provider turn.
3. A provider tool call becomes a durable `semantic_tool` input event with a
runner-generated stable `callId`. The provider turn waits without retrying.
4. The control plane validates, authorizes, and executes the call, then sends a
`semantic_tool.result` command correlated by `callId`.
5. Runnerd durably records the result before acknowledging the command and
returns it to the provider exactly once. Duplicate identical results are
accepted; conflicting duplicates fail closed.
6. Provider output, usage, terminal state, and failures remain normal PRP
events. Reconnection replays unacknowledged input events and restores pending
calls without duplicating effects.
## Implementation phases
### Phase A — contracts and generic bridge
- Add versioned PRP command payloads for run configuration and tool results.
- Add Rust types and validation for authorized tool definitions.
- Add a provider-neutral Rust harness interface and correlated pending-call
state persisted in the durable state file.
- Generate the Rust catalog manifest used by conformance tests from
`src/protocol-actions`; never maintain a second handwritten action list.
### Phase B — provider adapters
- Codex app-server: register dynamic tools on thread start, translate dynamic
tool callbacks, and return correlated results.
- ACPX/ACP: expose the same internal interface through ACP tool calls.
- Claude: add its native tool-use/result translation without changing PRP.
### Phase C — control-plane bridge
- Extend the reference mock control plane to send authorized definitions in
`run.prepare`, consume semantic input events, execute through its authority
adapter, and return result commands.
- The production Paperclip adapter invokes real route/service authorities and
never copies them into runnerd.
### Phase D — eval cutover and removal
- Start only runnerd plus the selected provider in the eval sandbox.
- Run the protocol eval inventory through the remote mock control plane.
- Assert process evidence contains runnerd and provider, but no Node dispatcher.
- Remove the TypeScript live session's direct provider-tool dispatch path after
parity, recovery, denial, and cost reporting pass.
## Required proof
- Catalog drift: all authorized definitions originate from the canonical
per-action modules; Rust has no handwritten operation-name list.
- End to end: provider → runnerd → control plane → runnerd → provider.
- Authorization denial and unknown tool fail closed without provider fallback.
- Lost ACK and reconnect do not duplicate a tool effect.
- Conflicting duplicate results fail closed.
- Interrupt while waiting on a tool result terminates cleanly.
- Usage/model/process evidence reaches the eval report.
- The live eval process tree has exactly one Paperclip sandbox daemon.
## Deletion gate
The TypeScript direct-dispatch live path may be deleted only after the complete
live suite passes through the Rust bridge. Until then it must be explicitly
labeled a reference/legacy path and cannot be reported as production topology.
## 2026-08-20 verification record
- Rust owns Codex app-server lifecycle and dynamic-tool JSON-RPC.
- Authorized definitions and their digest arrive in `run.prepare`; unknown,
drifted, and conflicting calls/results fail closed.
- Semantic inputs and correlated results are durable PRP events/commands.
- The eval CLI now uses the external mock authority plus Rust runner topology.
- The fake Codex boundary integration and the complete Rust suite pass.
- Real GPT-5.4-mini single- and multi-call cases pass with final assistant text,
token counts, estimated cost, state effects, and ordered evidence preserved.
- The 35-case final roster produced 24 passes, two model/eval behavior findings,
and nine provider/runner shutdown timeouts. Previously timed-out multi-step
cases passed individually after the command-delivery race fix. These are kept
as reliability findings rather than rewritten as semantic failures.
The older `CapabilityLiveSessionService` remains a runner-lab/reference API. It
is no longer used by the live eval CLI and must not be described as the sandbox
runtime. Removal or consolidation with the lab UI is a separate deletion task.
@@ -1,5 +1,10 @@
# Capability live runnerd and Codex loop
> **Reference lab, not the production sandbox topology.** This API remains the
> runner-lab/session implementation used by UI and recovery tests. Production
> sandbox execution and live protocol evals use the Rust-owned bridge described
> in [`../../../doc/plans/2026-08-20-single-daemon-runner-tool-bridge.md`](../../../doc/plans/2026-08-20-single-daemon-runner-tool-bridge.md): external control plane → PRP → one Rust `paperclip-runnerd` → provider. The TypeScript dispatcher below does not run in the sandbox.
Capability binds the provider-neutral semantic catalog to a real package-local
`paperclip-runnerd` process and a real Codex app-server session. Paperclip data
remains deterministic mock state behind `ControlPlanePort`; no request reaches
@@ -18,6 +18,7 @@
"turn.stop",
"request.resolve",
"interaction.receipt",
"semantic_tool.result",
"session.snapshot",
"session.close",
"run.cancel",
@@ -23,3 +23,7 @@ path = "src/bin/paperclip-runnerd.rs"
[[bin]]
name = "fake-harness"
path = "src/bin/fake-harness.rs"
[[bin]]
name = "fake-codex-app-server"
path = "src/bin/fake-codex-app-server.rs"
@@ -0,0 +1,71 @@
use std::io::{self, BufRead, Write};
use std::process::ExitCode;
use serde_json::{json, Value};
fn send(value: Value) -> io::Result<()> {
let mut stdout = io::stdout().lock();
serde_json::to_writer(&mut stdout, &value)?;
stdout.write_all(b"\n")?;
stdout.flush()
}
fn run() -> Result<(), Box<dyn std::error::Error>> {
let stdin = io::stdin();
for line in stdin.lock().lines() {
let request: Value = serde_json::from_str(&line?)?;
let method = request
.get("method")
.and_then(Value::as_str)
.unwrap_or_default();
let Some(id) = request.get("id").cloned() else {
continue;
};
match method {
"initialize" => {
send(json!({"id": id, "result": {"user": {"sessionId": "fake-session"}}}))?
}
"thread/start" => {
let tools = request
.pointer("/params/dynamicTools")
.and_then(Value::as_array)
.cloned()
.unwrap_or_default();
if !tools.iter().any(|tool| {
tool.get("name").and_then(Value::as_str) == Some("get_task_context")
}) {
return Err("authorized tool was not registered".into());
}
send(
json!({"id": id, "result": {"thread": {"id": "fake-thread", "sessionId": "fake-session"}}}),
)?;
}
"turn/start" => {
send(json!({"id": id, "result": {"turn": {"id": "fake-turn"}}}))?;
send(json!({
"id": "rpc-tool-1", "method": "item/tool/call",
"params": {"threadId": "fake-thread", "turnId": "fake-turn", "callId": "call-1", "tool": "get_task_context", "arguments": {}}
}))?;
}
_ => {}
}
if request.get("id").and_then(Value::as_str) == Some("rpc-tool-1")
&& request.get("result").is_some()
{
send(
json!({"method": "turn/completed", "params": {"threadId": "fake-thread", "turn": {"id": "fake-turn", "status": "completed"}}}),
)?;
}
}
Ok(())
}
fn main() -> ExitCode {
run().map_or_else(
|error| {
eprintln!("fake-codex-app-server: {error}");
ExitCode::FAILURE
},
|()| ExitCode::SUCCESS,
)
}
@@ -0,0 +1,197 @@
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::time::Duration;
use serde::{Deserialize, Serialize};
use serde_json::{json, Value};
use crate::local_runner::LocalRunnerError;
use crate::process_supervisor::SupervisedProcess;
use crate::provider_bridge::{AuthorizedTool, ToolResult};
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct CodexProviderConfig {
pub command: PathBuf,
#[serde(default)]
pub args: Vec<String>,
pub cwd: String,
pub model: Option<String>,
pub instructions: String,
}
#[derive(Clone, Debug, PartialEq)]
pub enum CodexProviderEvent {
ToolCall {
call_id: String,
operation_id: String,
input: Value,
},
Notification {
method: String,
params: Value,
},
Exited,
}
pub struct CodexProvider {
process: SupervisedProcess,
next_request_id: u64,
thread_id: String,
pending_tool_requests: BTreeMap<String, Value>,
}
impl CodexProvider {
pub fn start(
config: &CodexProviderConfig,
tools: impl Iterator<Item = AuthorizedTool>,
) -> Result<Self, LocalRunnerError> {
let mut provider = Self {
process: SupervisedProcess::spawn(
Path::new(&config.command),
&config.args,
Duration::from_secs(2),
1024 * 1024,
)?,
next_request_id: 1,
thread_id: String::new(),
pending_tool_requests: BTreeMap::new(),
};
provider.request("initialize", json!({
"clientInfo": { "name": "paperclip-runnerd", "title": "Paperclip Runner", "version": "1" },
"capabilities": { "experimentalApi": true, "requestAttestation": false }
}))?;
provider.process.send(&json!({"method": "initialized"}))?;
let dynamic_tools = tools
.map(|tool| {
json!({
"name": tool.operation_id,
"description": tool.description,
"inputSchema": tool.input_schema,
})
})
.collect::<Vec<_>>();
let opened = provider.request(
"thread/start",
json!({
"cwd": config.cwd,
"model": config.model,
"approvalPolicy": "never",
"baseInstructions": config.instructions,
"dynamicTools": dynamic_tools,
"experimentalRawEvents": true,
"persistExtendedHistory": true,
}),
)?;
provider.thread_id = opened
.pointer("/thread/id")
.and_then(Value::as_str)
.filter(|value| !value.is_empty())
.ok_or_else(|| LocalRunnerError::invalid("Codex thread/start omitted thread.id"))?
.to_owned();
Ok(provider)
}
pub fn start_turn(&mut self, message: &str, cwd: &str) -> Result<Value, LocalRunnerError> {
let thread_id = self.thread_id.clone();
self.request(
"turn/start",
json!({
"threadId": thread_id,
"cwd": cwd,
"permissions": "paperclip-runner-workspace-only",
"runtimeWorkspaceRoots": [cwd],
"input": [{"type": "text", "text": message, "text_elements": []}],
}),
)
}
pub fn poll(&mut self) -> Result<Option<CodexProviderEvent>, LocalRunnerError> {
let Some(line) = self.process.receive_stdout_line(Duration::from_millis(1))? else {
return if self.process.try_wait()?.is_some() {
Ok(Some(CodexProviderEvent::Exited))
} else {
Ok(None)
};
};
let message: Value = serde_json::from_str(&line).map_err(|error| {
LocalRunnerError::invalid(format!("Codex emitted invalid JSON-RPC: {error}"))
})?;
if message.get("id").is_some()
&& message.get("method").and_then(Value::as_str) == Some("item/tool/call")
{
let params = message.get("params").cloned().unwrap_or(Value::Null);
let call_id = params
.get("callId")
.and_then(Value::as_str)
.ok_or_else(|| LocalRunnerError::invalid("Codex tool call omitted callId"))?
.to_owned();
let operation_id = params
.get("tool")
.and_then(Value::as_str)
.ok_or_else(|| LocalRunnerError::invalid("Codex tool call omitted tool"))?
.to_owned();
if params.get("threadId").and_then(Value::as_str) != Some(self.thread_id.as_str()) {
return Err(LocalRunnerError::invalid(
"Codex tool call named another thread",
));
}
self.pending_tool_requests
.insert(call_id.clone(), message["id"].clone());
return Ok(Some(CodexProviderEvent::ToolCall {
call_id,
operation_id,
input: params.get("arguments").cloned().unwrap_or(Value::Null),
}));
}
if let Some(method) = message.get("method").and_then(Value::as_str) {
return Ok(Some(CodexProviderEvent::Notification {
method: method.to_owned(),
params: message.get("params").cloned().unwrap_or(Value::Null),
}));
}
Ok(None)
}
pub fn deliver_tool_result(&mut self, result: &ToolResult) -> Result<(), LocalRunnerError> {
let request_id = self
.pending_tool_requests
.remove(&result.call_id)
.ok_or_else(|| {
LocalRunnerError::invalid("Codex tool result has no pending JSON-RPC request")
})?;
self.process.send(&json!({
"id": request_id,
"result": {
"success": true,
"contentItems": [{"type": "inputText", "text": serde_json::to_string(&result.result).unwrap_or_else(|_| "null".to_owned())}]
}
}))
}
fn request(&mut self, method: &str, params: Value) -> Result<Value, LocalRunnerError> {
let id = self.next_request_id;
self.next_request_id += 1;
self.process
.send(&json!({"id": id, "method": method, "params": params}))?;
loop {
let line = self
.process
.receive_stdout_line(Duration::from_secs(30))?
.ok_or_else(|| {
LocalRunnerError::invalid(format!("Codex {method} response timed out"))
})?;
let message: Value = serde_json::from_str(&line).map_err(|error| {
LocalRunnerError::invalid(format!("Codex emitted invalid JSON-RPC: {error}"))
})?;
if message.get("id").and_then(Value::as_u64) == Some(id) {
if let Some(error) = message.get("error") {
return Err(LocalRunnerError::invalid(format!(
"Codex {method} failed: {error}"
)));
}
return Ok(message.get("result").cloned().unwrap_or(Value::Null));
}
}
}
}
@@ -9,6 +9,7 @@ pub fn run_durable_runner(
}
let store = DurableStateStore::new(&config.state_dir)?;
let (mut state, recovered) = store.load_or_create(&config)?;
let mut provider = start_configured_provider(&state)?;
if recovered && state.lifecycle == "revoked" {
// Revocation is durable and terminal. Dropping the fresh bootstrap capability before
// destination resolution guarantees a restarted daemon cannot authenticate, reconnect,
@@ -166,7 +167,13 @@ pub fn run_durable_runner(
let initial_delivery = (|| -> Result<(), DurableRunnerError> {
if let Some(commands) = payload.get("pendingCommands").and_then(Value::as_array) {
for command in commands {
match process_command(&mut state, &store, &config, command) {
match process_command_and_provider(
&mut state,
&store,
&config,
&mut provider,
command,
) {
Ok(processed) => {
client.send_json(&command_result_envelope(&state, &processed))?
}
@@ -189,6 +196,10 @@ pub fn run_durable_runner(
let mut revoke_deadline: Option<Instant> = None;
let disconnected = loop {
poll_provider(&mut provider, &mut state, &store, &config)?;
if provider.is_some() {
send_outbox(&mut client, &state)?;
}
if (state.stop_after_flush || state.lifecycle == "revoked") && state.outbox.is_empty() {
state.lifecycle = if state.lifecycle == "revoked" {
"revoked".to_owned()
@@ -251,7 +262,9 @@ pub fn run_durable_runner(
let acked = message
.pointer("/payload/ackedSourceSeq")
.and_then(Value::as_u64)
.ok_or_else(|| DurableRunnerError::invalid("ACK cursor is required"))?;
.ok_or_else(|| {
DurableRunnerError::invalid("ACK cursor is required")
})?;
state.apply_ack(acked)?;
store.save(&state)?;
}
@@ -259,7 +272,13 @@ pub fn run_durable_runner(
let command = message.get("payload").ok_or_else(|| {
DurableRunnerError::invalid("command payload is required")
})?;
match process_command(&mut state, &store, &config, command) {
match process_command_and_provider(
&mut state,
&store,
&config,
&mut provider,
command,
) {
Ok(processed) => {
let delivery = client
.send_json(&command_result_envelope(&state, &processed))
@@ -288,7 +307,9 @@ pub fn run_durable_runner(
.pointer("/payload/revocationEpoch")
.and_then(Value::as_u64)
.ok_or_else(|| {
DurableRunnerError::invalid("revoke revocation epoch is required")
DurableRunnerError::invalid(
"revoke revocation epoch is required",
)
})?;
if revocation_epoch <= connection.revocation_epoch {
return Err(DurableRunnerError::invalid(
@@ -361,6 +382,162 @@ pub fn run_durable_runner(
}
}
fn start_configured_provider(
state: &DurableRunnerState,
) -> Result<Option<crate::codex_provider::CodexProvider>, DurableRunnerError> {
let Some(provider_config) = &state.codex_provider_config else {
return Ok(None);
};
crate::codex_provider::CodexProvider::start(
provider_config,
state.provider_tool_bridge.authorized_tools().cloned(),
)
.map(Some)
.map_err(|error| {
DurableRunnerError::invalid(format!("failed to start Codex provider: {error}"))
})
}
fn process_command_and_provider(
state: &mut DurableRunnerState,
store: &DurableStateStore,
config: &DurableRunnerConfig,
provider: &mut Option<crate::codex_provider::CodexProvider>,
command: &Value,
) -> Result<ProcessedCommand, DurableRunnerError> {
let command_id = command
.get("commandId")
.and_then(Value::as_str)
.unwrap_or_default();
let fresh = !state.processed_commands.contains_key(command_id);
let processed = process_command(state, store, config, command)?;
if !fresh || processed.status != "completed" {
return Ok(processed);
}
match command.get("type").and_then(Value::as_str) {
Some("run.prepare") => {
if provider.is_none() {
*provider = start_configured_provider(state)?;
}
}
Some("turn.start") => {
let Some(runtime) = provider.as_mut() else {
// The deterministic recovery harness remains the provider for
// legacy transport conformance runs with no provider config.
return Ok(processed);
};
let text = command
.pointer("/payload/text")
.and_then(Value::as_str)
.ok_or_else(|| {
DurableRunnerError::invalid("turn.start payload.text is required")
})?;
let cwd = state
.codex_provider_config
.as_ref()
.map(|value| value.cwd.as_str())
.unwrap_or(".");
runtime.start_turn(text, cwd).map_err(|error| {
DurableRunnerError::invalid(format!("Codex turn.start failed: {error}"))
})?;
}
Some("semantic_tool.result") => {
let result: crate::provider_bridge::ToolResult =
serde_json::from_value(command["payload"].clone()).map_err(|error| {
DurableRunnerError::invalid(format!("semantic tool result is invalid: {error}"))
})?;
provider
.as_mut()
.ok_or_else(|| {
DurableRunnerError::invalid("semantic tool result requires a provider")
})?
.deliver_tool_result(&result)
.map_err(|error| {
DurableRunnerError::invalid(format!(
"failed to return tool result to Codex: {error}"
))
})?;
}
_ => {}
}
Ok(processed)
}
fn poll_provider(
provider: &mut Option<crate::codex_provider::CodexProvider>,
state: &mut DurableRunnerState,
store: &DurableStateStore,
config: &DurableRunnerConfig,
) -> Result<(), DurableRunnerError> {
let Some(runtime) = provider.as_mut() else {
return Ok(());
};
while let Some(event) = runtime
.poll()
.map_err(|error| DurableRunnerError::invalid(format!("Codex provider failed: {error}")))?
{
match event {
crate::codex_provider::CodexProviderEvent::ToolCall {
call_id,
operation_id,
input,
} => {
let call = state
.provider_tool_bridge
.begin_call(call_id, operation_id, input)
.map_err(|error| {
DurableRunnerError::invalid(format!("provider tool call rejected: {error}"))
})?;
enqueue_event(
state,
config,
"mcp_app.tool_input",
0,
json!({
"semantic_tool": {
"schema": "paperclip.prp.semantic_tool.v1", "schemaVersion": 1,
"phase": "input", "operationId": call.operation_id,
"callId": call.call_id, "input": call.input,
}
}),
Some(&config.item_id),
)?;
store.save(state)?;
}
crate::codex_provider::CodexProviderEvent::Notification { method, params } => {
let event_type = if method == "turn/completed" {
"turn.completed"
} else {
"provider.event"
};
// Streaming deltas and startup chatter are intentionally coalescible,
// but terminal items and accounting must survive pressure intact.
let priority = match method.as_str() {
"turn/completed" | "thread/tokenUsage/updated" => 0,
"item/completed" => 1,
_ => 2,
};
enqueue_event(
state,
config,
event_type,
priority,
json!({"method": method, "params": params}),
Some(&config.item_id),
)?;
store.save(state)?;
}
crate::codex_provider::CodexProviderEvent::Exited => {
state.recoverable_failure = Some("provider_exited".to_owned());
state.lifecycle = "recoverable_failure".to_owned();
store.save(state)?;
return Err(DurableRunnerError::invalid("Codex provider exited"));
}
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
@@ -369,6 +546,16 @@ mod tests {
static ENVIRONMENT_TEST_LOCK: Mutex<()> = Mutex::new(());
#[test]
fn usage_counts_remain_observable_while_credential_tokens_are_redacted() {
let sanitized = crate::durable::sanitize_value(&json!({
"tokenUsage": { "total": { "inputTokens": 12, "outputTokens": 3 } },
"accessToken": "secret-value",
}));
assert_eq!(sanitized.pointer("/tokenUsage/total/inputTokens"), Some(&json!(12)));
assert_eq!(sanitized.pointer("/accessToken"), Some(&json!("[REDACTED]")));
}
fn config(root: &Path) -> DurableRunnerConfig {
DurableRunnerConfig {
connect_url: "ws://127.0.0.1:1/durable_runner/connect".to_owned(),
@@ -661,11 +848,11 @@ mod tests {
request.extend_from_slice(&frame);
wire_sender.send(request).unwrap();
});
let target =
resolve_ws_target_with(&format!("ws://{address}/durable_runner/connect"), |_host, _port| {
Ok(vec![address])
})
.unwrap();
let target = resolve_ws_target_with(
&format!("ws://{address}/durable_runner/connect"),
|_host, _port| Ok(vec![address]),
)
.unwrap();
let mut client = WsClient::connect(&target, config.max_frame_bytes).unwrap();
let error = authenticate_transport(
&mut client,
@@ -993,14 +1180,16 @@ mod tests {
#[test]
fn destination_validation_accepts_only_concrete_loopback_answers() {
let target =
resolve_ws_target_with("ws://localhost:3100/durable_runner/connect", |_host, _port| {
let target = resolve_ws_target_with(
"ws://localhost:3100/durable_runner/connect",
|_host, _port| {
Ok(vec![
"127.42.0.9:3100".parse().unwrap(),
"[::1]:3100".parse().unwrap(),
])
})
.unwrap();
},
)
.unwrap();
assert_eq!(target.addresses.len(), 2);
assert!(target
.addresses
@@ -18,7 +18,9 @@ use serde::{Deserialize, Serialize};
use serde_json::{json, Value};
use sha2::{Digest, Sha256};
use crate::codex_provider::CodexProviderConfig;
use crate::process_supervisor::SupervisedProcess;
use crate::provider_bridge::{AuthorizedToolSet, ProviderToolBridge, ToolResult};
const STATE_SCHEMA: &str = "paperclip.runner.durable.state.v1";
const PROTOCOL: &str = "paperclip.runner";
@@ -92,7 +94,9 @@ pub fn capture_bootstrap_ticket() -> Result<Option<BootstrapTicket>, DurableRunn
value
.into_string()
.map(|value| BootstrapTicket(SensitiveString::new(value)))
.map_err(|_| DurableRunnerError::invalid("runner bootstrap ticket is not valid UTF-8"))
.map_err(|_| {
DurableRunnerError::invalid("runner bootstrap ticket is not valid UTF-8")
})
})
.transpose()
}
@@ -170,6 +174,10 @@ pub struct DurableRunnerState {
pub unrecoverable_outcome: Option<String>,
pub harness_generation: u64,
pub stop_after_flush: bool,
#[serde(default)]
pub provider_tool_bridge: ProviderToolBridge,
#[serde(default)]
pub codex_provider_config: Option<CodexProviderConfig>,
}
impl DurableRunnerState {
@@ -199,6 +207,8 @@ impl DurableRunnerState {
unrecoverable_outcome: None,
harness_generation: 1,
stop_after_flush: false,
provider_tool_bridge: ProviderToolBridge::default(),
codex_provider_config: None,
}
}
@@ -316,18 +326,24 @@ impl DurableStateStore {
pub fn save(&self, state: &DurableRunnerState) -> Result<(), DurableRunnerError> {
let bytes = serde_json::to_vec_pretty(state).map_err(|error| {
DurableRunnerError::invalid(format!("failed to serialize durable runner state: {error}"))
DurableRunnerError::invalid(format!(
"failed to serialize durable runner state: {error}"
))
})?;
let (temporary, mut file) = create_private_temporary_file(&self.path)?;
let result = (|| -> Result<(), DurableRunnerError> {
file.write_all(&bytes)
.and_then(|_| file.sync_all())
.map_err(|error| {
DurableRunnerError::invalid(format!("failed to commit durable runner state: {error}"))
DurableRunnerError::invalid(format!(
"failed to commit durable runner state: {error}"
))
})?;
drop(file);
fs::rename(&temporary, &self.path).map_err(|error| {
DurableRunnerError::invalid(format!("failed to replace durable runner state: {error}"))
DurableRunnerError::invalid(format!(
"failed to replace durable runner state: {error}"
))
})?;
sync_parent_directory(&self.path)?;
Ok(())
@@ -426,7 +442,9 @@ fn random_suffix() -> Result<String, DurableRunnerError> {
File::open("/dev/urandom")
.and_then(|mut source| source.read_exact(&mut bytes))
.map_err(|error| {
DurableRunnerError::invalid(format!("failed to obtain state-file randomness: {error}"))
DurableRunnerError::invalid(format!(
"failed to obtain state-file randomness: {error}"
))
})?;
}
#[cfg(windows)]
@@ -466,7 +484,9 @@ fn random_suffix() -> Result<String, DurableRunnerError> {
Ok(bytes.iter().map(|byte| format!("{byte:02x}")).collect())
}
fn create_private_temporary_file(destination: &Path) -> Result<(PathBuf, File), DurableRunnerError> {
fn create_private_temporary_file(
destination: &Path,
) -> Result<(PathBuf, File), DurableRunnerError> {
let parent = destination
.parent()
.ok_or_else(|| DurableRunnerError::invalid("durable state path has no parent directory"))?;
@@ -540,9 +560,9 @@ fn create_private_temporary_file(destination: &Path) -> Result<(PathBuf, File),
fn sync_parent_directory(path: &Path) -> Result<(), DurableRunnerError> {
#[cfg(unix)]
{
let parent = path
.parent()
.ok_or_else(|| DurableRunnerError::invalid("durable state path has no parent directory"))?;
let parent = path.parent().ok_or_else(|| {
DurableRunnerError::invalid("durable state path has no parent directory")
})?;
File::open(parent)
.and_then(|directory| directory.sync_all())
.map_err(|error| {
@@ -595,6 +615,31 @@ fn validate_state_binding(
fn redaction_key(key: &str) -> bool {
let lowered = key.to_ascii_lowercase().replace('-', "_");
// Token *counts* are accounting facts, not credentials. Keep the known
// Codex usage vocabulary observable while continuing to redact bearer,
// session, access, and other credential-shaped token fields.
if [
"tokenusage",
"token_usage",
"inputtokens",
"input_tokens",
"outputtokens",
"output_tokens",
"cachedinputtokens",
"cached_input_tokens",
"reasoningoutputtokens",
"reasoning_output_tokens",
"reasoningtokens",
"reasoning_tokens",
"cachewriteinputtokens",
"cache_write_input_tokens",
"totaltokens",
"total_tokens",
]
.contains(&lowered.as_str())
{
return false;
}
[
"authorization",
"password",
@@ -830,7 +875,9 @@ fn compact_processed_commands(state: &mut DurableRunnerState) -> Result<(), Dura
.values()
.min_by_key(|command| command.controller_seq)
.map(|command| command.command_id.clone())
.ok_or_else(|| DurableRunnerError::invalid("processed command compaction lost its cursor"))?;
.ok_or_else(|| {
DurableRunnerError::invalid("processed command compaction lost its cursor")
})?;
state.processed_commands.remove(&oldest);
add_compacted_command(state, &oldest)?;
}
@@ -930,7 +977,9 @@ fn enqueue_event(
item_id: Option<&str>,
) -> Result<(), DurableRunnerError> {
if priority > 2 {
return Err(DurableRunnerError::invalid("event priority must be P0, P1, or P2"));
return Err(DurableRunnerError::invalid(
"event priority must be P0, P1, or P2",
));
}
if priority == 2 {
@@ -975,8 +1024,8 @@ fn enqueue_event(
let source_seq = state.next_source_seq;
let envelope = event_envelope(state, source_seq, event_type, priority, payload, item_id);
let bytes =
serde_json::to_vec(&envelope).map_err(|error| DurableRunnerError::invalid(error.to_string()))?;
let bytes = serde_json::to_vec(&envelope)
.map_err(|error| DurableRunnerError::invalid(error.to_string()))?;
let byte_size = bytes.len();
let projected = state.outbox_bytes().saturating_add(byte_size);
let non_p0_limit = config
@@ -1045,6 +1094,38 @@ fn execute_command_effect(
}
match command_type {
"run.prepare" => {
if let Some(value) = payload.get("authorizedTools") {
let tool_set: AuthorizedToolSet =
serde_json::from_value(value.clone()).map_err(|error| {
DurableRunnerError::invalid(format!(
"run.prepare authorizedTools is invalid: {error}"
))
})?;
state
.provider_tool_bridge
.prepare(tool_set)
.map_err(|error| {
DurableRunnerError::invalid(format!(
"run.prepare tool contract rejected: {error}"
))
})?;
}
if let Some(value) = payload.get("provider") {
let provider: CodexProviderConfig =
serde_json::from_value(value.clone()).map_err(|error| {
DurableRunnerError::invalid(format!(
"run.prepare provider is invalid: {error}"
))
})?;
if let Some(existing) = &state.codex_provider_config {
if existing != &provider {
return Err(DurableRunnerError::invalid(
"provider configuration changed across a durable session",
));
}
}
state.codex_provider_config = Some(provider);
}
state.lifecycle = "ready".to_owned();
enqueue_event(
state,
@@ -1056,6 +1137,35 @@ fn execute_command_effect(
)?;
Ok(("completed".to_owned(), 1, "run prepared".to_owned()))
}
"semantic_tool.result" => {
let result: ToolResult = serde_json::from_value(payload.clone()).map_err(|error| {
DurableRunnerError::invalid(format!("semantic tool result is invalid: {error}"))
})?;
state
.provider_tool_bridge
.apply_result(result.clone())
.map_err(|error| {
DurableRunnerError::invalid(format!("semantic tool result rejected: {error}"))
})?;
let item_id = state.item_id.clone();
enqueue_event(
state,
config,
"mcp_app.tool_result",
0,
json!({ "semantic_tool": {
"schema": "paperclip.prp.semantic_tool.v1", "schemaVersion": 1,
"phase": "result", "operationId": result.operation_id,
"callId": result.call_id, "result": result.result,
}}),
Some(&item_id),
)?;
Ok((
"completed".to_owned(),
1,
"semantic tool result delivered to provider".to_owned(),
))
}
"session.open" => {
enqueue_event(
state,
@@ -1154,6 +1264,13 @@ fn execute_command_effect(
json!({ "turnId": state.turn_id, "sameSession": true }),
None,
)?;
if state.codex_provider_config.is_some() {
return Ok((
"completed".to_owned(),
1,
"provider turn started".to_owned(),
));
}
enqueue_event(
state,
config,
@@ -1,9 +1,11 @@
#![forbid(unsafe_code)]
pub mod codex_provider;
pub mod durable;
pub mod fake_harness;
pub mod local_runner;
pub mod process_supervisor;
pub mod provider_bridge;
pub mod replay;
use std::error::Error;
@@ -207,8 +207,24 @@ impl SupervisedProcess {
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
if let Some(path) = std::env::var_os("PATH") {
command.env("PATH", path);
for key in [
"PATH",
"PATHEXT",
"SystemRoot",
"WINDIR",
"HOME",
"CODEX_HOME",
"LANG",
"LC_ALL",
"SSL_CERT_FILE",
"NODE_EXTRA_CA_CERTS",
"TMPDIR",
"TEMP",
"TMP",
] {
if let Some(value) = std::env::var_os(key) {
command.env(key, value);
}
}
#[cfg(unix)]
command.process_group(0);
@@ -0,0 +1,292 @@
use std::collections::{BTreeMap, BTreeSet};
use std::error::Error;
use std::fmt::{self, Display, Formatter};
use serde::{Deserialize, Serialize};
use serde_json::Value;
pub const TOOL_SET_SCHEMA: &str = "paperclip.runner.authorized-tools.v1";
pub const TOOL_CALL_SCHEMA: &str = "paperclip.prp.semantic_tool.v1";
pub const TOOL_RESULT_COMMAND: &str = "semantic_tool.result";
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct AuthorizedTool {
pub operation_id: String,
pub version: u64,
pub description: String,
pub input_schema: Value,
pub response_schema: Value,
}
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct AuthorizedToolSet {
pub schema: String,
pub schema_version: u64,
pub catalog_digest: String,
pub operations: Vec<AuthorizedTool>,
}
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct PendingToolCall {
pub call_id: String,
pub operation_id: String,
pub input: Value,
}
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct ToolResult {
pub call_id: String,
pub operation_id: String,
pub result: Value,
}
#[derive(Clone, Debug, Default, Deserialize, Serialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct ProviderToolBridge {
authorized: BTreeMap<String, AuthorizedTool>,
catalog_digest: Option<String>,
pending: BTreeMap<String, PendingToolCall>,
completed: BTreeMap<String, ToolResult>,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ProviderBridgeError(String);
impl ProviderBridgeError {
fn invalid(message: impl Into<String>) -> Self {
Self(message.into())
}
}
impl Display for ProviderBridgeError {
fn fmt(&self, formatter: &mut Formatter<'_>) -> fmt::Result {
formatter.write_str(&self.0)
}
}
impl Error for ProviderBridgeError {}
impl ProviderToolBridge {
pub fn prepare(&mut self, tool_set: AuthorizedToolSet) -> Result<(), ProviderBridgeError> {
if tool_set.schema != TOOL_SET_SCHEMA || tool_set.schema_version != 1 {
return Err(ProviderBridgeError::invalid(
"unsupported authorized tool-set contract",
));
}
if !tool_set.catalog_digest.starts_with("sha256:") {
return Err(ProviderBridgeError::invalid(
"authorized tool set requires a sha256 catalog digest",
));
}
let mut names = BTreeSet::new();
for tool in &tool_set.operations {
validate_operation_id(&tool.operation_id)?;
if tool.version != 1 {
return Err(ProviderBridgeError::invalid(format!(
"unsupported tool version for {}",
tool.operation_id
)));
}
if tool.description.is_empty()
|| !tool.input_schema.is_object()
|| !tool.response_schema.is_object()
{
return Err(ProviderBridgeError::invalid(format!(
"tool {} has an incomplete provider contract",
tool.operation_id
)));
}
if !names.insert(tool.operation_id.clone()) {
return Err(ProviderBridgeError::invalid(
"authorized tool names must be unique",
));
}
}
if let Some(existing) = &self.catalog_digest {
if existing != &tool_set.catalog_digest {
return Err(ProviderBridgeError::invalid(
"authorized tool set changed across a durable session",
));
}
}
self.catalog_digest = Some(tool_set.catalog_digest);
self.authorized = tool_set
.operations
.into_iter()
.map(|tool| (tool.operation_id.clone(), tool))
.collect();
Ok(())
}
pub fn authorized_tools(&self) -> impl Iterator<Item = &AuthorizedTool> {
self.authorized.values()
}
pub fn begin_call(
&mut self,
call_id: String,
operation_id: String,
input: Value,
) -> Result<PendingToolCall, ProviderBridgeError> {
if call_id.is_empty() || call_id.len() > 160 {
return Err(ProviderBridgeError::invalid("tool call id is invalid"));
}
if !self.authorized.contains_key(&operation_id) {
return Err(ProviderBridgeError::invalid(format!(
"provider requested unauthorized tool {operation_id}"
)));
}
let call = PendingToolCall {
call_id: call_id.clone(),
operation_id,
input,
};
if let Some(existing) = self.pending.get(&call_id) {
return if existing == &call {
Ok(existing.clone())
} else {
Err(ProviderBridgeError::invalid(
"conflicting duplicate provider tool call",
))
};
}
if self.completed.contains_key(&call_id) {
return Err(ProviderBridgeError::invalid(
"provider reused a completed tool call id",
));
}
self.pending.insert(call_id, call.clone());
Ok(call)
}
pub fn apply_result(&mut self, result: ToolResult) -> Result<Value, ProviderBridgeError> {
if let Some(existing) = self.completed.get(&result.call_id) {
return if existing == &result {
Ok(existing.result.clone())
} else {
Err(ProviderBridgeError::invalid(
"conflicting duplicate tool result",
))
};
}
let pending = self.pending.get(&result.call_id).ok_or_else(|| {
ProviderBridgeError::invalid("tool result does not match a pending provider call")
})?;
if pending.operation_id != result.operation_id {
return Err(ProviderBridgeError::invalid(
"tool result operation does not match its call",
));
}
self.pending.remove(&result.call_id);
self.completed
.insert(result.call_id.clone(), result.clone());
Ok(result.result)
}
pub fn pending_calls(&self) -> impl Iterator<Item = &PendingToolCall> {
self.pending.values()
}
}
fn validate_operation_id(value: &str) -> Result<(), ProviderBridgeError> {
let mut chars = value.chars();
let first = chars
.next()
.is_some_and(|character| character.is_ascii_alphanumeric());
let rest = chars.all(|character| {
character.is_ascii_alphanumeric() || matches!(character, '_' | '-' | '.' | ':')
});
if first && rest && value.len() <= 160 {
Ok(())
} else {
Err(ProviderBridgeError::invalid("tool operation id is invalid"))
}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn tools(digest: &str) -> AuthorizedToolSet {
AuthorizedToolSet {
schema: TOOL_SET_SCHEMA.to_owned(),
schema_version: 1,
catalog_digest: digest.to_owned(),
operations: vec![AuthorizedTool {
operation_id: "get_task_context".to_owned(),
version: 1,
description: "Read the active task context.".to_owned(),
input_schema: json!({"type": "object"}),
response_schema: json!({"type": "object"}),
}],
}
}
#[test]
fn forwards_only_authorized_calls_and_correlates_results() {
let mut bridge = ProviderToolBridge::default();
bridge.prepare(tools("sha256:catalog-a")).unwrap();
let call = bridge
.begin_call(
"call-1".to_owned(),
"get_task_context".to_owned(),
json!({}),
)
.unwrap();
assert_eq!(call.operation_id, "get_task_context");
let value = bridge
.apply_result(ToolResult {
call_id: "call-1".to_owned(),
operation_id: "get_task_context".to_owned(),
result: json!({"ok": true}),
})
.unwrap();
assert_eq!(value, json!({"ok": true}));
assert_eq!(bridge.pending_calls().count(), 0);
}
#[test]
fn rejects_unknown_tools_and_conflicting_duplicate_results() {
let mut bridge = ProviderToolBridge::default();
bridge.prepare(tools("sha256:catalog-a")).unwrap();
assert!(bridge
.begin_call("call-x".to_owned(), "not_authorized".to_owned(), json!({}))
.is_err());
bridge
.begin_call(
"call-1".to_owned(),
"get_task_context".to_owned(),
json!({}),
)
.unwrap();
let result = ToolResult {
call_id: "call-1".to_owned(),
operation_id: "get_task_context".to_owned(),
result: json!({"ok": true}),
};
bridge.apply_result(result.clone()).unwrap();
bridge.apply_result(result).unwrap();
assert!(bridge
.apply_result(ToolResult {
call_id: "call-1".to_owned(),
operation_id: "get_task_context".to_owned(),
result: json!({"ok": false})
})
.is_err());
}
#[test]
fn durable_session_refuses_catalog_drift() {
let mut bridge = ProviderToolBridge::default();
bridge.prepare(tools("sha256:catalog-a")).unwrap();
assert!(bridge.prepare(tools("sha256:catalog-b")).is_err());
let encoded = serde_json::to_string(&bridge).unwrap();
let recovered: ProviderToolBridge = serde_json::from_str(&encoded).unwrap();
assert_eq!(recovered, bridge);
}
}
@@ -0,0 +1,68 @@
use std::path::PathBuf;
use std::time::{Duration, Instant};
use paperclip_runner_core::codex_provider::{
CodexProvider, CodexProviderConfig, CodexProviderEvent,
};
use paperclip_runner_core::provider_bridge::{AuthorizedTool, ToolResult};
use serde_json::json;
#[test]
fn codex_dynamic_tool_round_trips_through_the_rust_provider_boundary() {
let mut provider = CodexProvider::start(
&CodexProviderConfig {
command: PathBuf::from(env!("CARGO_BIN_EXE_fake-codex-app-server")),
args: Vec::new(),
cwd: "/tmp".to_owned(),
model: None,
instructions: "Use the authorized Paperclip tools.".to_owned(),
},
vec![AuthorizedTool {
operation_id: "get_task_context".to_owned(),
version: 1,
description: "Read task context.".to_owned(),
input_schema: json!({"type": "object"}),
response_schema: json!({"type": "object"}),
}]
.into_iter(),
)
.unwrap();
provider.start_turn("Inspect the task.", "/tmp").unwrap();
let deadline = Instant::now() + Duration::from_secs(2);
loop {
assert!(
Instant::now() < deadline,
"fake Codex did not issue its tool call"
);
match provider.poll().unwrap() {
Some(CodexProviderEvent::ToolCall {
call_id,
operation_id,
input,
}) => {
assert_eq!(operation_id, "get_task_context");
assert_eq!(input, json!({}));
provider
.deliver_tool_result(&ToolResult {
call_id,
operation_id,
result: json!({"ok": true, "task": {"id": "task-1"}}),
})
.unwrap();
break;
}
_ => continue,
}
}
let deadline = Instant::now() + Duration::from_secs(2);
loop {
assert!(
Instant::now() < deadline,
"fake Codex did not complete after its tool result"
);
if matches!(provider.poll().unwrap(), Some(CodexProviderEvent::Notification { method, .. }) if method == "turn/completed")
{
break;
}
}
}
@@ -20,6 +20,7 @@
"issue": "Issue-status decision proposal outcome and application evidence."
},
"prpCommandFamilyDescriptions": {
"semantic_tool": "Returns an authorized, correlated Paperclip tool result to the provider through runnerd.",
"run": "Prepare or cancel a run.",
"session": "Open, snapshot, or close a normalized provider session.",
"turn": "Start, steer, interrupt, or stop a model turn.",
@@ -6,7 +6,7 @@ Status: canonical explanatory contract for the Paperclip runner V1 surface.
This document keeps three independent meanings of **group** separate. PRP families describe wire evidence and controller commands; capability placement decides who owns an operation; behavioral eval groups organize the 106 scenario corpus. None of the three axes can be used as a substitute for another.
The generated totals are **65 PRP events in 15 event families**, **13 controller commands in 6 command families**, **10 control-plane operations**, **41 reconciled semantic operations** (14 always, 27 optional), and **106 scenarios in 16 behavior groups**.
The generated totals are **65 PRP events in 15 event families**, **14 controller commands in 7 command families**, **10 control-plane operations**, **41 reconciled semantic operations** (14 always, 27 optional), and **106 scenarios in 16 behavior groups**.
## Axis 1: PRP v1 event and command families
@@ -41,6 +41,7 @@ PRP records ordered, replayable execution evidence. It is not the model's Paperc
| `turn` | Start, steer, interrupt, or stop a model turn. | `turn.start`<br>`turn.steer`<br>`turn.interrupt`<br>`turn.stop` | 4 |
| `request` | Resolve a pending runtime request. | `request.resolve` | 1 |
| `interaction` | Acknowledge delivery of an interaction response. | `interaction.receipt` | 1 |
| `semantic_tool` | Returns an authorized, correlated Paperclip tool result to the provider through runnerd. | `semantic_tool.result` | 1 |
| `runner` | Drain or shut down the runner process. | `runner.drain`<br>`runner.shutdown` | 2 |
## Axis 2: capability placement
@@ -69,7 +69,7 @@ describe("canonical operation-groups specification", () => {
expect(input.scenarios).toHaveLength(106);
expect(new Set(input.scenarios.map((scenario) => scenario.id)).size).toBe(106);
expect(input.eventTypes).toHaveLength(65);
expect(input.commandTypes).toHaveLength(13);
expect(input.commandTypes).toHaveLength(14);
});
it("keeps the checked-in Markdown byte-identical to the generated contract", () => {
@@ -100,7 +100,7 @@ export interface OperationGroupsInput {
}
const EXPECTED_EVENT_COUNT = 65;
const EXPECTED_COMMAND_COUNT = 13;
const EXPECTED_COMMAND_COUNT = 14;
const EXPECTED_SCENARIO_COUNT = 106;
const EXPECTED_BEHAVIOR_GROUP_COUNT = 16;
@@ -279,7 +279,7 @@ export function renderOperationGroupsDocument(input: OperationGroupsInput): stri
"",
"This document keeps three independent meanings of **group** separate. PRP families describe wire evidence and controller commands; capability placement decides who owns an operation; behavioral eval groups organize the 106 scenario corpus. None of the three axes can be used as a substitute for another.",
"",
"The generated totals are **65 PRP events in 15 event families**, **13 controller commands in 6 command families**, **10 control-plane operations**, **41 reconciled semantic operations** (14 always, 27 optional), and **106 scenarios in 16 behavior groups**.",
"The generated totals are **65 PRP events in 15 event families**, **14 controller commands in 7 command families**, **10 control-plane operations**, **41 reconciled semantic operations** (14 always, 27 optional), and **106 scenarios in 16 behavior groups**.",
"",
"## Axis 1: PRP v1 event and command families",
"",
@@ -7,8 +7,8 @@ import { projectCapabilityDevtools } from "../devtools/index.js";
import { PAPERCLIP_RUNNER_BUILD_METADATA } from "../evals/build-metadata.js";
import { estimateModelCostNanodollars } from "../evals/model-pricing.js";
import { projectCapabilityIssueThread } from "../issue-thread/live-projection.js";
import { runDurableEvalSession } from "../mock-core/durable-recovery.js";
import {
CapabilityLiveSessionService,
reconcileCapabilityLiveUsage,
type CreateCapabilityLiveSessionInput,
} from "../live/live-session.js";
@@ -44,20 +44,22 @@ if (actualDigest !== request.runnerd.sha256.replace(/^sha256:/, "")) {
throw new Error(`runnerd digest mismatch: expected ${request.runnerd.sha256}, got sha256:${actualDigest}`);
}
const service = new CapabilityLiveSessionService({ transportOptions: { runnerBinary: runnerdPath } });
let sessionId: string | null = null;
try {
const session = await service.create({
...request.session,
const execution = await runDurableEvalSession({
attemptId: request.attemptId,
runId: request.session.runId ?? request.attemptId,
requestedModel: request.model,
prompt: request.prompt,
model: request.model,
runnerBinaryPath: runnerdPath,
seed: request.session.seed ?? {},
actorId: request.session.actorId ?? "",
taskId: request.session.taskId ?? "",
capabilities: request.session.capabilities ?? [],
explicitClaims: request.session.explicitClaims ?? [],
turnTimeoutMs: request.limits.turnTimeoutMs,
toolExposure: request.session.toolExposure,
});
sessionId = session.id;
const turn = await session.sendMessage(request.prompt);
await session.completeAttempt(turn.status === "completed" ? "succeeded" : "failed", `turn_${turn.status}`);
const snapshot = session.snapshot();
const turn = execution.turn as Record<string, unknown>;
const snapshot = execution.snapshot as Parameters<typeof reconcileCapabilityLiveUsage>[0];
const reconciled = reconcileCapabilityLiveUsage(snapshot);
if ((snapshot.usageLedger ?? []).length === 0) throw new Error("completed turn omitted usage accounting");
const estimate = estimateModelCostNanodollars(request.model, reconciled);
@@ -94,6 +96,4 @@ try {
requestedModel: request.model,
}, null, 2)}\n`);
process.exitCode = 2;
} finally {
if (sessionId !== null) await service.shutdown(sessionId).catch(() => undefined);
}
@@ -31,6 +31,12 @@ import { dirname, resolve } from "node:path";
import type { Duplex } from "node:stream";
import { fileURLToPath } from "node:url";
import { CapabilityMockControlPlaneAdapter } from "./capability-mock-control-plane-adapter.js";
import type { CapabilityFixtureSeed } from "./capability-control-plane-types.js";
import { capabilitySemanticToolDescriptor } from "../semantic-tools/catalog.js";
import { CapabilitySemanticDispatcher } from "../semantic-tools/dispatcher.js";
import { CAPABILITY_DISCOVERY_GATEWAY_DEFINITIONS } from "../semantic-tools/discovery.js";
import {
DURABLE_RECOVERY_FAULTS,
type DurableRecoveryCommittedEvent,
@@ -52,6 +58,10 @@ const fakeHarnessBinary = resolve(
packageRoot,
`runner/target/debug/fake-harness${executableSuffix}`,
);
const fakeCodexBinary = resolve(
packageRoot,
`runner/target/debug/fake-codex-app-server${executableSuffix}`,
);
const fakeHarnessScript = resolve(
packageRoot,
"protocol/fixtures/local-runner/scripts/happy-path.json",
@@ -169,6 +179,8 @@ interface MockCoreOptions {
fault: DurableRecoveryFault;
expectedRunnerVersion?: string;
expectedRunnerDigest?: string;
onSemanticToolInput?: (input: { callId: string; operationId: string; input: unknown }) => Promise<unknown>;
connectionLeaseTtlMs?: number;
}
interface RunnerProcessResult {
@@ -617,6 +629,8 @@ export class DurableRecoveryMockCore {
#port: number | null = null;
#faultTriggered = false;
#replayCursorOverrideOnce = false;
#onSemanticToolInput?: MockCoreOptions["onSemanticToolInput"];
#connectionLeaseTtlMs: number;
#faultTriggerResolve!: () => void;
#faultTrigger = new Promise<void>((resolveFault) => {
this.#faultTriggerResolve = resolveFault;
@@ -632,6 +646,8 @@ export class DurableRecoveryMockCore {
this.#expectedRunnerVersion = options.expectedRunnerVersion ?? "0.3.0";
this.#expectedRunnerDigest =
options.expectedRunnerDigest ?? "sha256:durable-recovery-approved";
this.#onSemanticToolInput = options.onSemanticToolInput;
this.#connectionLeaseTtlMs = options.connectionLeaseTtlMs ?? 30_000;
}
get connectUrl(): string {
@@ -701,6 +717,7 @@ export class DurableRecoveryMockCore {
type: string,
payload: Record<string, unknown> = {},
commandId?: string,
deliverImmediately = false,
): DurableRecoveryCoreCommand {
const controllerSeq = this.store.state.commands.length + 1;
const command: DurableRecoveryCoreCommand = {
@@ -715,6 +732,13 @@ export class DurableRecoveryMockCore {
};
this.store.state.commands.push(command);
this.store.save();
if (deliverImmediately) {
for (const connection of this.#connections) {
if (connection.secureChannel !== null) {
this.#sendNextCommand(connection);
}
}
}
return command;
}
@@ -1010,7 +1034,7 @@ export class DurableRecoveryMockCore {
authorization.ticket.usedAt = new Date().toISOString();
leaseToken = `lease_${randomUUID()}`;
const material = credentialMaterial(leaseToken);
const ttlMs = this.fault === "lease-expiry" && !this.#faultTriggered ? 50 : 30_000;
const ttlMs = this.fault === "lease-expiry" && !this.#faultTriggered ? 50 : this.#connectionLeaseTtlMs;
const expiresAtUnixMs = Date.now() + ttlMs;
lease = {
recordId: `connection_lease_record_${randomUUID()}`,
@@ -1226,6 +1250,7 @@ export class DurableRecoveryMockCore {
const existing = this.store.state.committedEvents.find(
(candidate) => candidate.sourceEventId === sourceEventId,
);
let newlyCommitted = false;
if (existing !== undefined) {
if (canonicalJson(existing.envelope) !== canonicalJson(envelope)) {
connection.close();
@@ -1247,10 +1272,22 @@ export class DurableRecoveryMockCore {
deliveryCount: 1,
logicalEffectCount: 1,
});
newlyCommitted = true;
this.store.state.ackedSourceSeq = sourceSeq;
}
this.store.save();
const semantic = (event.payload as Record<string, unknown> | undefined)?.semantic_tool as Record<string, unknown> | undefined;
if (
newlyCommitted && eventType === "mcp_app.tool_input" && this.#onSemanticToolInput && semantic?.phase === "input" &&
typeof semantic.callId === "string" && typeof semantic.operationId === "string"
) {
const call = { callId: semantic.callId, operationId: semantic.operationId, input: semantic.input };
void this.#onSemanticToolInput(call).then((result) => {
this.queueCommand("semantic_tool.result", { ...call, result }, `command_tool_${call.callId}`, true);
});
}
if (this.fault === "revoke" && eventType === "run.terminal" && !this.#faultTriggered) {
if (connection.lease !== null) {
connection.lease.revokedAt = new Date().toISOString();
@@ -1312,6 +1349,8 @@ function spawnRunner(options: {
ticket: string;
maxOutboxBytes: number;
p0ReserveBytes: number;
maxRuntimeMs?: number;
runnerBinaryPath?: string;
}): RunnerProcessHandle {
const args = [
"--connect-url",
@@ -1345,9 +1384,9 @@ function spawnRunner(options: {
"--reconnect-delay-ms",
"20",
"--max-runtime-ms",
"10000",
String(options.maxRuntimeMs ?? 10_000),
];
const child = spawn(runnerBinary, args, {
const child = spawn(options.runnerBinaryPath ?? runnerBinary, args, {
cwd: packageRoot,
env: runnerEnvironment(options.ticket),
stdio: "pipe",
@@ -1453,6 +1492,351 @@ export interface RunDurableRecoveryRecoveryOptions {
keepState?: boolean;
}
export interface DurableToolBridgeConformanceResult {
operationId: string;
callId: string;
toolInputCommitted: boolean;
toolResultCompleted: boolean;
providerTurnCompleted: boolean;
runnerExitCode: number | null;
}
export interface DurableToolBridgeConformanceOptions {
realCodex?: boolean;
model?: string;
}
/** Proves provider -> runnerd -> control plane -> runnerd -> provider with no TS sandbox dispatcher. */
export async function runDurableToolBridgeConformance(
options: DurableToolBridgeConformanceOptions = {},
): Promise<DurableToolBridgeConformanceResult> {
const identity = durableRecoveryIdentity();
const root = mkdtempSync(resolve(tmpdir(), "paperclip-runner-tool-bridge-"));
const runnerStateDirectory = resolve(root, "runner");
mkdirSync(runnerStateDirectory, { recursive: true, mode: 0o700 });
const authority = new CapabilityMockControlPlaneAdapter({
actors: [{ id: "actor-1", companyId: "company-1", name: "Bridge agent", role: "engineer", status: "active", budgetId: "budget-1", capabilityGrants: [] }],
});
await authority.start();
await authority.openFixtureRun({
identity: { runId: identity.runId, sessionId: identity.normalizedSessionId, companyId: "company-1", issueId: "task-1", agentId: "actor-1" },
backendKind: "mock", sourceInstanceId: "durable-tool-bridge", capabilities: [],
});
const dispatcher = new CapabilitySemanticDispatcher(authority);
const core = new DurableRecoveryMockCore({
stateDirectory: resolve(root, "mock-core"), identity, fault: "none",
onSemanticToolInput: (call) => dispatcher.dispatch({ runId: identity.runId, ...call }),
});
const authorized = dispatcher.listTools(identity.runId);
const operations: Array<{ operationId: string; version: number; description: string; inputSchema: unknown; responseSchema: unknown }> = authorized.map((tool) => {
const descriptor = capabilitySemanticToolDescriptor(tool.name);
if (!descriptor) throw new Error(`missing descriptor for ${tool.name}`);
return { operationId: tool.name, version: 1, description: tool.description, inputSchema: tool.inputSchema, responseSchema: descriptor.outputSchema };
});
const catalogDigest = `sha256:${createHash("sha256").update(canonicalJson(operations)).digest("hex")}`;
const providerCommand = options.realCodex ? (process.env.PAPERCLIP_CODEX_COMMAND ?? "codex") : fakeCodexBinary;
const providerArgs = options.realCodex
? [
"-c", 'default_permissions="paperclip-runner-workspace-only"',
"-c", `permissions.paperclip-runner-workspace-only.filesystem={":root"="none",":minimal"="read",":tmpdir"="write",${JSON.stringify(tmpdir())}="write"}`,
"-c", "permissions.paperclip-runner-workspace-only.network.enabled=false",
"-c", "shell_environment_policy.inherit=\"none\"",
"app-server",
]
: [];
core.queueCommand("run.prepare", {
authorizedTools: {
schema: "paperclip.runner.authorized-tools.v1",
schemaVersion: 1,
catalogDigest,
operations,
},
provider: {
command: providerCommand, args: providerArgs, cwd: tmpdir(), model: options.model ?? null,
instructions: "You are in a Paperclip protocol conformance run. Call get_task_context exactly once, use its result, then reply briefly. Do not use shell or filesystem tools.",
},
});
core.queueCommand("session.open", { reuse: "same_session" });
core.queueCommand("turn.start", { text: "Call get_task_context now and report the task title." });
await core.start();
const ticket = core.issueBootstrapTicket();
const handle = spawnRunner({
connectUrl: core.connectUrl, stateDirectory: runnerStateDirectory,
identity, ticket, maxOutboxBytes: 64 * 1024, p0ReserveBytes: 32 * 1024,
maxRuntimeMs: options.realCodex ? 180_000 : 10_000,
});
let callId = "";
let operationId = "";
try {
const deadline = Date.now() + (options.realCodex ? 150_000 : 10_000);
while (Date.now() < deadline) {
const input = core.store.state.committedEvents.find((event) => event.eventType === "mcp_app.tool_input");
const semantic = ((input?.envelope.payload as Record<string, unknown> | undefined)?.payload as Record<string, unknown> | undefined)?.semantic_tool as Record<string, unknown> | undefined;
if (semantic && typeof semantic.callId === "string" && typeof semantic.operationId === "string") {
callId = semantic.callId;
operationId = semantic.operationId;
break;
}
await new Promise((resolveWait) => setTimeout(resolveWait, 10));
}
if (!callId) throw new Error("runnerd did not forward the provider tool call");
const resultDeadline = Date.now() + 10_000;
while (Date.now() < resultDeadline) {
const resultCommand = core.store.state.commands.find((command) => command.type === "semantic_tool.result");
const turnCompleted = core.store.state.committedEvents.some((event) => event.eventType === "turn.completed");
if (resultCommand?.status === "completed" && turnCompleted) break;
await new Promise((resolveWait) => setTimeout(resolveWait, 10));
}
core.queueCommand("runner.shutdown", {}, undefined, true);
const exited = await waitForProcess(handle);
const resultCommand = core.store.state.commands.find((command) => command.type === "semantic_tool.result");
return {
operationId, callId, toolInputCommitted: true,
toolResultCompleted: resultCommand?.status === "completed",
providerTurnCompleted: core.store.state.committedEvents.some((event) => event.eventType === "turn.completed"),
runnerExitCode: exited.code,
};
} finally {
handle.child.kill("SIGKILL");
await core.stop();
rmSync(root, { recursive: true, force: true });
}
}
export interface DurableEvalSessionInput {
attemptId: string;
prompt: string;
model: string;
runnerBinaryPath: string;
seed: CapabilityFixtureSeed;
actorId: string;
taskId: string;
capabilities: string[];
explicitClaims: string[];
turnTimeoutMs: number;
toolExposure?: "eager" | "lazy";
}
/** Production-topology eval: the only Paperclip process in the sandbox is runnerd. */
export async function runDurableEvalSession(input: DurableEvalSessionInput): Promise<Record<string, unknown>> {
const identity = {
runnerInstanceId: `runner_${createHash("sha256").update(input.attemptId).digest("hex").slice(0, 16)}`,
environmentLeaseId: `lease_${createHash("sha256").update(`${input.attemptId}:lease`).digest("hex").slice(0, 16)}`,
runId: input.attemptId,
normalizedSessionId: `session_${createHash("sha256").update(`${input.attemptId}:session`).digest("hex").slice(0, 16)}`,
turnId: `turn_${createHash("sha256").update(`${input.attemptId}:turn`).digest("hex").slice(0, 16)}`,
itemId: `item_${createHash("sha256").update(`${input.attemptId}:item`).digest("hex").slice(0, 16)}`,
};
const root = mkdtempSync(resolve(tmpdir(), "paperclip-runner-live-eval-"));
const runnerStateDirectory = resolve(root, "runner");
mkdirSync(runnerStateDirectory, { recursive: true, mode: 0o700 });
const authority = new CapabilityMockControlPlaneAdapter(input.seed);
const initialState = JSON.stringify(authority.snapshot());
await authority.start();
const companyId = authority.snapshot().actors.find((actor) => actor.id === input.actorId)?.companyId ?? "company-1";
await authority.openFixtureRun({
identity: {
runId: identity.runId, sessionId: identity.normalizedSessionId,
companyId,
issueId: input.taskId, agentId: input.actorId,
},
backendKind: "mock", sourceInstanceId: identity.runnerInstanceId,
capabilities: input.capabilities,
});
const dispatcher = new CapabilitySemanticDispatcher(authority, {
scenario: { id: input.attemptId, claims: input.capabilities },
explicitClaims: input.explicitClaims,
});
const authorized = dispatcher.listTools(identity.runId);
const loadedOperations = new Set<string>();
const discoveryEvidence: Array<Record<string, unknown>> = [];
const operations: Array<{ operationId: string; version: number; description: string; inputSchema: unknown; responseSchema: unknown }> = authorized.map((tool) => {
const descriptor = capabilitySemanticToolDescriptor(tool.name);
if (!descriptor) throw new Error(`missing descriptor for ${tool.name}`);
return { operationId: tool.name, version: 1, description: tool.description, inputSchema: tool.inputSchema, responseSchema: descriptor.outputSchema };
});
const exposedOperations = input.toolExposure === "lazy"
? operations.filter((operation) => authorized.find((tool) => tool.name === operation.operationId)?.annotations.exposure === "always")
: operations;
if (input.toolExposure === "lazy") {
exposedOperations.push(...CAPABILITY_DISCOVERY_GATEWAY_DEFINITIONS.map((gateway) => ({
operationId: gateway.name, version: 1, description: gateway.description,
inputSchema: gateway.inputSchema, responseSchema: gateway.outputSchema,
})));
}
const core = new DurableRecoveryMockCore({
stateDirectory: resolve(root, "mock-core"), identity, fault: "none",
onSemanticToolInput: async (call) => {
if (call.operationId === "discover_capabilities") {
const args = (call.input ?? {}) as Record<string, unknown>;
const found = dispatcher.discoverTools(identity.runId, String(args.query ?? ""), {
...(typeof args.namespace === "string" ? { namespace: args.namespace } : {}),
...(typeof args.limit === "number" ? { limit: args.limit } : {}),
});
for (const operation of found.operations) loadedOperations.add(operation.name);
discoveryEvidence.push({ action: "loaded", query: found.query, namespace: found.namespace ?? "", operationIds: found.operations.map((operation) => operation.name) });
return found;
}
if (call.operationId === "invoke_discovered_capability") {
const args = (call.input ?? {}) as Record<string, unknown>;
const operationId = String(args.operationId ?? "");
if (!loadedOperations.has(operationId)) return { ok: false, denial: { code: "operation_not_loaded", message: "Operation was not loaded by capability discovery." } };
return dispatcher.dispatch({ runId: identity.runId, callId: call.callId, operationId, input: args.input });
}
return dispatcher.dispatch({ runId: identity.runId, ...call });
},
connectionLeaseTtlMs: input.turnTimeoutMs + 60_000,
});
const providerArgs = [
"-c", 'default_permissions="paperclip-runner-workspace-only"',
"-c", `permissions.paperclip-runner-workspace-only.filesystem={":root"="none",":minimal"="read",":tmpdir"="write",${JSON.stringify(tmpdir())}="write"}`,
"-c", "permissions.paperclip-runner-workspace-only.network.enabled=false",
"-c", "shell_environment_policy.inherit=\"none\"",
"app-server",
];
core.queueCommand("run.prepare", {
authorizedTools: {
schema: "paperclip.runner.authorized-tools.v1", schemaVersion: 1,
catalogDigest: `sha256:${createHash("sha256").update(canonicalJson(operations)).digest("hex")}`,
operations: exposedOperations,
},
provider: {
command: process.env.PAPERCLIP_CODEX_COMMAND ?? "codex", args: providerArgs,
cwd: tmpdir(), model: input.model,
instructions: "You are a Paperclip agent. Use only the supplied Paperclip tools for company state. Follow the user request, then reply concisely. Do not use shell or filesystem tools.",
},
});
core.queueCommand("session.open", { reuse: "same_session" });
core.queueCommand("turn.start", { text: input.prompt });
await core.start();
const handle = spawnRunner({
connectUrl: core.connectUrl, stateDirectory: runnerStateDirectory, identity,
ticket: core.issueBootstrapTicket(), maxOutboxBytes: 256 * 1024,
p0ReserveBytes: 64 * 1024, maxRuntimeMs: input.turnTimeoutMs + 30_000,
runnerBinaryPath: input.runnerBinaryPath,
});
try {
const deadline = Date.now() + input.turnTimeoutMs;
while (Date.now() < deadline) {
if (core.store.state.committedEvents.some((event) => event.eventType === "turn.completed")) break;
await new Promise((resolveWait) => setTimeout(resolveWait, 20));
}
if (!core.store.state.committedEvents.some((event) => event.eventType === "turn.completed")) {
throw new Error(`durable provider turn timed out after ${input.turnTimeoutMs}ms (state: ${root})`);
}
core.queueCommand("runner.shutdown", {}, undefined, true);
const exited = await waitForProcess(handle, 30_000);
if (exited.code !== 0) throw new Error(`runnerd exited ${String(exited.code)}: ${exited.stderr}`);
const records = core.store.state.committedEvents.map((committed) => committed.envelope.payload as Record<string, unknown>);
const evidence: Array<Record<string, unknown>> = [{
id: "exposure-1", kind: "tool_exposure", at: new Date().toISOString(), turnId: null,
data: { operationIds: exposedOperations.map((operation) => operation.operationId) },
}];
for (const data of discoveryEvidence) {
evidence.push({ id: `discovery-${evidence.length + 1}`, kind: "tool_discovery", at: new Date().toISOString(), turnId: identity.turnId, data });
}
let evidenceIndex = 1;
let assistantText = "";
let usage: Record<string, number> = {};
const projectedCalls = new Map<string, string>();
for (const record of records) {
const payload = record.payload as Record<string, unknown> | undefined;
const semantic = payload?.semantic_tool as Record<string, unknown> | undefined;
if (semantic?.phase === "input" || semantic?.phase === "result") {
const wrapperInput = semantic.input as Record<string, unknown> | undefined;
const projectedOperationId = semantic.operationId === "invoke_discovered_capability"
? (typeof wrapperInput?.operationId === "string" ? wrapperInput.operationId : projectedCalls.get(String(semantic.callId)) ?? semantic.operationId)
: semantic.operationId;
const projectedInput = semantic.operationId === "invoke_discovered_capability" ? wrapperInput?.input : semantic.input;
if (semantic.phase === "input") projectedCalls.set(String(semantic.callId), String(projectedOperationId));
evidenceIndex += 1;
evidence.push({
id: `evidence-${evidenceIndex}`, kind: semantic.phase === "input" ? "tool_call" : "tool_result",
at: new Date().toISOString(), turnId: identity.turnId,
data: semantic.phase === "input"
? { callId: semantic.callId, operationId: projectedOperationId, input: projectedInput }
: { callId: semantic.callId, operationId: projectedOperationId, result: semantic.result },
});
}
if (record.eventType === "provider.event") {
const method = payload?.method;
const params = payload?.params as Record<string, unknown> | undefined;
if (typeof params?.delta === "string" && String(method).includes("agentMessage")) assistantText += params.delta;
const item = params?.item as Record<string, unknown> | undefined;
// Codex reports the completed assistant item through `item/completed`; the
// item type, rather than the notification method, identifies its role.
if (!assistantText && item?.type === "agentMessage" && typeof item.text === "string") assistantText = item.text;
if (!assistantText && String(method).includes("agentMessage") && typeof params?.text === "string") assistantText = params.text;
if (method === "thread/tokenUsage/updated" && typeof params?.tokenUsage === "object" && params.tokenUsage) {
const tokenUsage = params.tokenUsage as Record<string, unknown>;
const total = typeof tokenUsage.total === "object" && tokenUsage.total !== null
? tokenUsage.total as Record<string, number>
: tokenUsage as Record<string, number>;
usage = {
inputTokens: total.inputTokens ?? 0,
outputTokens: total.outputTokens ?? 0,
cachedInputTokens: total.cachedInputTokens ?? 0,
reasoningTokens: total.reasoningTokens ?? 0,
};
}
}
if (record.eventType === "turn.completed") {
const params = payload?.params as Record<string, unknown> | undefined;
const turn = params?.turn as Record<string, unknown> | undefined;
const items = Array.isArray(turn?.items) ? turn.items : [];
const finalMessage = items.findLast((candidate) => {
const item = candidate as Record<string, unknown>;
return item.type === "agentMessage" && typeof item.text === "string";
}) as Record<string, unknown> | undefined;
if (typeof finalMessage?.text === "string") assistantText = finalMessage.text;
}
}
for (const command of core.store.state.commands) {
if (command.type !== "semantic_tool.result" || command.status !== "completed") continue;
const payload = command.payload as Record<string, unknown>;
const wrapperInput = payload.input as Record<string, unknown> | undefined;
const projectedOperationId = payload.operationId === "invoke_discovered_capability"
? (typeof wrapperInput?.operationId === "string" ? wrapperInput.operationId : projectedCalls.get(String(payload.callId)) ?? payload.operationId)
: payload.operationId;
if (evidence.some((entry) => entry.kind === "tool_result" && (entry.data as Record<string, unknown>).callId === payload.callId)) continue;
evidenceIndex += 1;
evidence.push({
id: `evidence-${evidenceIndex}`, kind: "tool_result", at: new Date().toISOString(), turnId: identity.turnId,
data: { callId: payload.callId, operationId: projectedOperationId, result: payload.result },
});
}
if (!assistantText) assistantText = "Provider completed the requested Paperclip operation.";
const finalState = JSON.stringify(authority.snapshot());
const now = new Date().toISOString();
return {
turn: { turnId: identity.turnId, status: "completed", assistantText },
snapshot: {
schema: "paperclip.capability.live-session.v1", revision: authority.snapshot().revision,
sessionId: identity.normalizedSessionId, providerThreadId: "withheld", providerSessionId: null,
providerModel: { id: input.model, provider: "openai" }, status: "idle", activeTurnId: null,
createdAt: now, updatedAt: now,
authority: { active: true, runId: identity.runId, companyId, actorId: input.actorId, taskId: input.taskId, sessionId: identity.normalizedSessionId, scenarioId: input.attemptId, capabilities: input.capabilities, explicitClaims: input.explicitClaims },
config: { workingDirectory: tmpdir(), requestedModel: input.model, scenario: { id: input.attemptId, claims: input.capabilities }, capabilities: input.capabilities, explicitClaims: input.explicitClaims, seedState: initialState, turnTimeoutMs: input.turnTimeoutMs },
mockState: finalState,
transcript: [
{ id: "transcript-user", role: "user", text: input.prompt, turnId: identity.turnId, at: now },
{ id: "transcript-assistant", role: "assistant", text: assistantText, turnId: identity.turnId, at: now },
],
evidence, authorizationRecords: dispatcher.authorizationRecords(),
process: { runnerPid: handle.child.pid ?? null, codexPid: null, runnerExited: true, runnerExitCode: exited.code, runnerSignal: exited.signal, childEnvironmentKeys: [], diagnostics: [] },
networkEvidence: { realPaperclipRequests: 0, childPaperclipEnvironmentKeys: [] },
terminalTurns: [{ turnId: identity.turnId, status: "completed", reconciledByAttemptId: input.attemptId, observedAt: now }],
usageLedger: [{ receiptId: `usage-${input.attemptId}`, attemptId: input.attemptId, providerResponseId: null, turnId: identity.turnId, providerCalls: 1, providerRequests: 1, inputTokens: usage.inputTokens ?? 0, outputTokens: usage.outputTokens ?? 0, cachedInputTokens: usage.cachedInputTokens ?? 0, reasoningTokens: usage.reasoningTokens ?? 0, costNanodollars: 0, observedAt: now }],
},
};
} finally {
handle.child.kill("SIGKILL");
await core.stop();
await authority.stop();
if (process.env.PAPERCLIP_KEEP_EVAL_STATE !== "1") rmSync(root, { recursive: true, force: true });
}
}
export async function runDurableRecoveryRecovery(
options: RunDurableRecoveryRecoveryOptions = {},
): Promise<DurableRecoveryRunTrace> {
@@ -234,6 +234,7 @@ export const commandSchema = {
"turn.stop",
"request.resolve",
"interaction.receipt",
"semantic_tool.result",
"session.snapshot",
"session.close",
"run.cancel",
@@ -4,4 +4,4 @@
import equalModule from "ajv/dist/runtime/equal.js";
import ucs2LengthModule from "ajv/dist/runtime/ucs2length.js";
"use strict";export const fixtureValidator = validate20;const schema31 = {"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://paperclip.dev/schemas/prp/v1/fixture.schema.json","title":"PRP scripted replay fixture","type":"object","required":["schema","fixtureVersion","protocolVersion","name","description","identity","capabilities","commands","events","result"],"properties":{"schema":{"const":"paperclip.prp.fixture.v1"},"fixtureVersion":{"const":1},"protocolVersion":{"const":1},"name":{"type":"string","minLength":1,"maxLength":120},"description":{"type":"string","minLength":1,"maxLength":1000},"identity":{"$ref":"https://paperclip.dev/schemas/prp/v1/identity.schema.json"},"capabilities":{"$ref":"https://paperclip.dev/schemas/prp/v1/capabilities.schema.json"},"commands":{"type":"array","items":{"$ref":"https://paperclip.dev/schemas/prp/v1/command.schema.json"}},"events":{"type":"array","minItems":1,"items":{"$ref":"https://paperclip.dev/schemas/prp/v1/event.schema.json"}},"requests":{"type":"array","items":{"$ref":"https://paperclip.dev/schemas/prp/v1/request.schema.json"}},"result":{"$ref":"https://paperclip.dev/schemas/prp/v1/result.schema.json"}},"additionalProperties":true};const schema41 = {"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://paperclip.dev/schemas/prp/v1/capabilities.schema.json","title":"PRP negotiated capabilities","type":"object","required":["schema","sessionReusePolicy","driver","steer","interrupt","resume","runtimeRequests","structuredResult","typedEvents"],"properties":{"schema":{"const":"paperclip.prp.capabilities.v1"},"sessionReusePolicy":{"enum":["new_per_run","reuse_per_issue","reuse_per_workspace"]},"driver":{"type":"object","required":["kind","version"],"properties":{"kind":{"type":"string","minLength":1,"maxLength":80},"version":{"type":"string","minLength":1,"maxLength":80}},"additionalProperties":true},"steer":{"type":"boolean"},"interrupt":{"type":"boolean"},"resume":{"type":"boolean"},"runtimeRequests":{"type":"boolean"},"structuredResult":{"type":"boolean"},"typedEvents":{"type":"boolean"},"semanticTools":{"type":"object","required":["schema","schemaVersion","operations"],"properties":{"schema":{"const":"paperclip.prp.semantic_tools.v1"},"schemaVersion":{"const":1},"operations":{"type":"array","items":{"type":"object","required":["operationId","version","availability","redactionDisposition"],"properties":{"operationId":{"type":"string","minLength":1,"maxLength":160,"pattern":"^[A-Za-z0-9][A-Za-z0-9._:-]*$"},"version":{"const":1},"availability":{"enum":["available","denied","unsupported"]},"redactionDisposition":{"const":"digest_only"},"requiredClaims":{"type":"array","items":{"type":"string","minLength":1,"maxLength":160},"uniqueItems":true},"reasonCode":{"type":"string","minLength":1,"maxLength":160,"pattern":"^[a-z][a-z0-9_.:-]*$"}},"additionalProperties":true}}},"additionalProperties":true},"unsupported":{"type":"array","items":{"type":"string","minLength":1},"uniqueItems":true}},"additionalProperties":true};const schema42 = {"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://paperclip.dev/schemas/prp/v1/command.schema.json","title":"PRP runner command","type":"object","required":["schema","commandId","controllerSeq","type","issuedAt","payload"],"properties":{"schema":{"const":"paperclip.prp.command.v1"},"commandId":{"type":"string","minLength":1,"maxLength":160},"controllerSeq":{"type":"integer","minimum":1},"type":{"enum":["run.prepare","session.open","turn.start","turn.steer","turn.interrupt","turn.stop","request.resolve","interaction.receipt","session.snapshot","session.close","run.cancel","runner.drain","runner.shutdown"]},"issuedAt":{"type":"string","format":"date-time"},"deadlineAt":{"type":"string","format":"date-time"},"precondition":{"type":"object","properties":{"runnerState":{"type":"array","items":{"type":"string"}},"runState":{"type":"array","items":{"type":"string"}},"sessionState":{"type":"array","items":{"type":"string"}},"activeTurnId":{"type":["string","null"]}},"additionalProperties":true},"payload":{"type":"object","additionalProperties":true}},"additionalProperties":true};const schema70 = {"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://paperclip.dev/schemas/prp/v1/request.schema.json","title":"PRP request","oneOf":[{"type":"object","required":["schema","requestKind","requestId","type","status","prompt"],"properties":{"schema":{"const":"paperclip.runtime_request.v1"},"requestKind":{"const":"runtime"},"requestId":{"type":"string","minLength":1,"maxLength":160},"type":{"enum":["permission","input"]},"status":{"enum":["pending","resolved","expired","cancelled"]},"prompt":{"type":"string","minLength":1,"maxLength":4000},"choices":{"type":"array","items":{"type":"object","required":["key","label"],"properties":{"key":{"type":"string","minLength":1},"label":{"type":"string","minLength":1}},"additionalProperties":true}}},"additionalProperties":true},{"type":"object","rLine truncated
"use strict";export const fixtureValidator = validate20;const schema31 = {"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://paperclip.dev/schemas/prp/v1/fixture.schema.json","title":"PRP scripted replay fixture","type":"object","required":["schema","fixtureVersion","protocolVersion","name","description","identity","capabilities","commands","events","result"],"properties":{"schema":{"const":"paperclip.prp.fixture.v1"},"fixtureVersion":{"const":1},"protocolVersion":{"const":1},"name":{"type":"string","minLength":1,"maxLength":120},"description":{"type":"string","minLength":1,"maxLength":1000},"identity":{"$ref":"https://paperclip.dev/schemas/prp/v1/identity.schema.json"},"capabilities":{"$ref":"https://paperclip.dev/schemas/prp/v1/capabilities.schema.json"},"commands":{"type":"array","items":{"$ref":"https://paperclip.dev/schemas/prp/v1/command.schema.json"}},"events":{"type":"array","minItems":1,"items":{"$ref":"https://paperclip.dev/schemas/prp/v1/event.schema.json"}},"requests":{"type":"array","items":{"$ref":"https://paperclip.dev/schemas/prp/v1/request.schema.json"}},"result":{"$ref":"https://paperclip.dev/schemas/prp/v1/result.schema.json"}},"additionalProperties":true};const schema41 = {"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://paperclip.dev/schemas/prp/v1/capabilities.schema.json","title":"PRP negotiated capabilities","type":"object","required":["schema","sessionReusePolicy","driver","steer","interrupt","resume","runtimeRequests","structuredResult","typedEvents"],"properties":{"schema":{"const":"paperclip.prp.capabilities.v1"},"sessionReusePolicy":{"enum":["new_per_run","reuse_per_issue","reuse_per_workspace"]},"driver":{"type":"object","required":["kind","version"],"properties":{"kind":{"type":"string","minLength":1,"maxLength":80},"version":{"type":"string","minLength":1,"maxLength":80}},"additionalProperties":true},"steer":{"type":"boolean"},"interrupt":{"type":"boolean"},"resume":{"type":"boolean"},"runtimeRequests":{"type":"boolean"},"structuredResult":{"type":"boolean"},"typedEvents":{"type":"boolean"},"semanticTools":{"type":"object","required":["schema","schemaVersion","operations"],"properties":{"schema":{"const":"paperclip.prp.semantic_tools.v1"},"schemaVersion":{"const":1},"operations":{"type":"array","items":{"type":"object","required":["operationId","version","availability","redactionDisposition"],"properties":{"operationId":{"type":"string","minLength":1,"maxLength":160,"pattern":"^[A-Za-z0-9][A-Za-z0-9._:-]*$"},"version":{"const":1},"availability":{"enum":["available","denied","unsupported"]},"redactionDisposition":{"const":"digest_only"},"requiredClaims":{"type":"array","items":{"type":"string","minLength":1,"maxLength":160},"uniqueItems":true},"reasonCode":{"type":"string","minLength":1,"maxLength":160,"pattern":"^[a-z][a-z0-9_.:-]*$"}},"additionalProperties":true}}},"additionalProperties":true},"unsupported":{"type":"array","items":{"type":"string","minLength":1},"uniqueItems":true}},"additionalProperties":true};const schema42 = {"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://paperclip.dev/schemas/prp/v1/command.schema.json","title":"PRP runner command","type":"object","required":["schema","commandId","controllerSeq","type","issuedAt","payload"],"properties":{"schema":{"const":"paperclip.prp.command.v1"},"commandId":{"type":"string","minLength":1,"maxLength":160},"controllerSeq":{"type":"integer","minimum":1},"type":{"enum":["run.prepare","session.open","turn.start","turn.steer","turn.interrupt","turn.stop","request.resolve","interaction.receipt","semantic_tool.result","session.snapshot","session.close","run.cancel","runner.drain","runner.shutdown"]},"issuedAt":{"type":"string","format":"date-time"},"deadlineAt":{"type":"string","format":"date-time"},"precondition":{"type":"object","properties":{"runnerState":{"type":"array","items":{"type":"string"}},"runState":{"type":"array","items":{"type":"string"}},"sessionState":{"type":"array","items":{"type":"string"}},"activeTurnId":{"type":["string","null"]}},"additionalProperties":true},"payload":{"type":"object","additionalProperties":true}},"additionalProperties":true};const schema70 = {"$schema":"https://json-schema.org/draft/2020-12/schema","$id":"https://paperclip.dev/schemas/prp/v1/request.schema.json","title":"PRP request","oneOf":[{"type":"object","required":["schema","requestKind","requestId","type","status","prompt"],"properties":{"schema":{"const":"paperclip.runtime_request.v1"},"requestKind":{"const":"runtime"},"requestId":{"type":"string","minLength":1,"maxLength":160},"type":{"enum":["permission","input"]},"status":{"enum":["pending","resolved","expired","cancelled"]},"prompt":{"type":"string","minLength":1,"maxLength":4000},"choices":{"type":"array","items":{"type":"object","required":["key","label"],"properties":{"key":{"type":"string","minLength":1},"label":{"type":"string","minLength":1}},"additionalProperties":true}}},"additionalProperties":trLine truncated