Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-27 19:35:13 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge commit '1c892ff4640370f2a658beab3339080fd4a86de4' into codex/runner-hosted-admission

Browse Source 
* commit '1c892ff4640370f2a658beab3339080fd4a86de4':
  fix(recovery): preserve native workspaces after control deadlines (#17788)
  fix(evals): share bounded request policy with qualification (#17765)
  fix(tasks): hide empty message history disclosure (#17751)
  perf(runner): keep event journal writes off the HTTP thread (#17745)
  Bound native OpenCode assigned-tool catalog projection (#17740)
  fix(ui): keep iPhone task chat stable while typing and streaming (#17232)
  fix(runner): preserve disabled Codex web search (#17744)
  fix(ui): identify accounts in AI connection choices (#17738)
  fix(recovery): wake retained workspace preservation after Retry (#17741)
  fix(chat): admit task Retry after GitHub reconnect (#17733)
  fix(runner): allow bounded controller admission before welcome (#17735)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed 2026-10-26 14:22:48 -05:00
2 parents e77ded0d55 1c892ff464
commit dbce6f0260
119 files changed
+3583 -803
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

DESIGN.md
+6 -2
View File
@@ -60,8 +60,12 @@ title sizes remain larger.
The bottom navigation responds to accumulated scrolling, ignoring small
reversals and Safari's edge bounce. It glides out and eases back in with shared
motion tokens; the task composer follows the same motion. Keep page padding
stable while the navigation moves, and honor reduced-motion preferences.
motion tokens. Keep page padding stable while the navigation moves, and honor
reduced-motion preferences.
Mobile task conversations size their shell to the visible viewport, with one
message scroller and a normal flex footer for the composer. Navigation stays at
the layout viewport bottom, behind the keyboard.
Cold task opens reveal the instruction and recent conversation independently of
plans, related metadata, and historical run activity. Show the latest human
doc/DEVELOPING.md
+8
View File
@@ -1983,6 +1983,14 @@ runs; immutable run descriptors remain readable. The **Paperclip Runner**
experimental setting applies to other native providers. Native Codex, OpenCode,
ACPX Claude, and ACPX Pi do not require it. Company access checks still apply.
Native Codex preserves an explicit `adapterConfig.search: false` when converting
or saving an agent. The disabled hosted-search policy is recorded in the native
execution input and launch requirements, so prepared launches and recovery keep
it. An absent setting retains the provider default; this does not add support
for converting a Legacy agent with `search: true`. Authentication hello probes
always disable hosted web search because they only verify the selected account
and model.
OpenCode requires an explicit `provider/model` before testing or finishing setup.
An inactive setup draft can be saved before choosing a model. Automatic native
selection uses `allow` permissions and a `per_turn` lifecycle. Native `ask` and
doc/architecture/runner-copilot-capabilities.md
+19 -1
View File
@@ -1,6 +1,24 @@
# Copilot 1.0.88 rich ACP capability audit
Current integration (2026-10-17): **Copilot profile v21 remains pending** with
Current source declaration (2026-10-26): **Copilot profile v61** binds controller
journal retirement and the historical profile decoder. Its digest is
`sha256:956276ebee59f880249fa6955adc224af9ca7b6a9da59bcc5da4a8667550c5b3`.
The v60 declaration remains unchanged and readable as history. This source
identity update changes no Copilot executable, arguments, permission policy,
tool receipts, or qualification flags. Offline compatibility checks supply no
new live qualification; prior live outcomes retain their original identities.
Historical source declaration (2026-10-26): **Copilot profile v60** binds the shared
sources that carry Codex's explicit hosted-search restriction. Its digest is
`sha256:ca3c2b38ed5f6e25cf1c96c1f73781a86cc90c1c0add93f64716c66f28b3f3aa`.
The v59 declaration remains unchanged and readable as history. Exact-current
launch admission still applies. The Copilot executable, arguments, permissions,
receipts, and qualification flags are unchanged. Copilot authentication probes
do not carry the Codex-only setting. Offline compatibility checks do not supply
new live qualification; prior live outcomes remain attached to their original
source identities.
Historical integration (2026-10-17): **Copilot profile v21 remained pending** with
digest `sha256:1dad3d26d91e66c6f911dbfa43e7963f36c0bbf0d7709f4bbb568029a8b4b46a`.
It binds the shared provider completion contract, direct driver, production sidecar, and controller
continuation resolver. Installation verification requires this generated v21
doc/connections/AI-CONNECTIONS.md
+6
View File
@@ -59,6 +59,12 @@ the selected connection owns its provider routing.
Changing those separately may make a binding incompatible; saving then requires
a compatible choice. Agent configuration cannot grant access to another account.
Agent account choices and legacy-adoption confirmations show the sign-in method,
personal or company-shared scope, and available account or owner identity beside
the connection name. Accounts with the same provider name can be distinguished
before selection. These labels use existing authorized account metadata, never
credentials or authentication paths.
Personal defaults are unique per company, user, and provider. A Claude bot can use one user’s subscription and another user’s API key without changing its harness or model. Explicit shared selections remain pinned to the selected account and method.
The first successful personal connection sets a default only when none exists.
The additive `ai_provider_defaults` table preserves the legacy per-method preferences. Migration selects each user’s most recently updated provider preference (including unavailable accounts), and rerunning it never overwrites a provider default. New writes maintain the legacy table for older servers. A database trigger propagates older servers’ explicit default updates to the provider default. Inserting an additional method default does not replace an existing provider default.
doc/execution-semantics.md
+52 -6
View File
@@ -730,7 +730,16 @@ recovery path with the specific scan cause and an operator action. Generic
stranded-work recovery must not grant another budget for these errors. This
does not automatically replay historical generic `setup_failed` runs.
### ACP startup handshake bound
### Runner authentication and ACP startup handshake bounds
The native Runner waits up to the ten-second connection setup budget for the
authenticated welcome. The current execution or reconnect deadline can shorten
that wait. Controller admission and durable lease persistence happen before the
welcome, so a short internal response limit must not discard an otherwise valid
bootstrap ticket during that work. A failed welcome still makes the one-use
ticket unsafe to replay. Authentication proofs, credential expiry, and identity
checks remain required. Steady-state reads return to their 250 ms polling bound
after the welcome.
An adapter-backed live path also requires that the ACP startup handshake itself cannot hang forever. The engine bounds the handshake with a fixed startup deadline and a poll of the duplex control-channel disposition. Either condition ends the handshake and reports a closed, typed code, so the issue can reach a settled disposition instead of staying `in_progress` with no observable next action.
@@ -1327,6 +1336,13 @@ does not reset the provider-attempt budget.
Provider execution and control-plane finalization have different clocks. A healthy provider can think or execute a long tool without output. Once execution settles, recovery and finalization control steps have a 60-second deadline, checked on startup and every 15 seconds. With a healthy database and scheduler, an abandoned transition must be repaired or surfaced within 90 seconds. Terminal persistence must not wait on provider cleanup or publication; a late finalizer cannot change a reassigned or closed task or release another run's locks. Historical ambiguous runs are never automatically replayed after an upgrade.
When that deadline fails a native run with a prepared workspace, the same
transaction records the exact source's file-preservation obligation. Existing
stop-and-retain recovery contains the provider and saves the files before
ordinary Retry can continue. Finalized workspaces, mismatched sources and
accepted results keep their existing settlement paths. Reassignment does not
discard the stopped run's files or grant authority over the new assignee.
Every continuation carries the triggering request, ordered user direction, interaction outcomes, completed work, and explicit history coverage. A delivered message remains part of the task's request after its connection or approval resolves. The original title is background; a completed Notion read does not satisfy a later Gmail request. Author and source-trust boundaries survive rendering into both native and legacy prompts. Missing required history must be fetched before dispatch rather than described as complete.
### Interrupted conversation continuation
@@ -1376,14 +1392,25 @@ accepted native result. A successful `workspace_recovery` operation and matching
recovery record prove persistence; adapter output or a user repair note cannot.
The failed run and unknown external effects remain historical facts.
Startup and periodic recovery discover these obligations through the retained
workspace index. Two background exports per server keep unrelated recovery
responsive. Automatic file recovery has three total attempts with a 30-second
delay. An idempotent board request through
Startup and the retained-workspace sweep reconstruct these obligations through
the retained workspace index. Committed Retry, recovery admission, and recovery
progress writes wake a separate preservation worker. It follows the persisted
retry deadline and becomes idle when no eligible recovery remains. Two background
exports per server keep unrelated execution finalization responsive. A resolved
exact-source files-only obligation remains eligible; unrelated resolved or
cancelled recovery history does not keep this worker active. The persisted
attempt budget applies across restarts. An idempotent board request through
`POST /api/issues/:id/recovery-actions/recover-workspace` admits one additional
attempt after a transient failure. It requires task mutation access, accepts no
infrastructure certification, and grants no agent or provider execution authority.
Restarts do not reset the attempt limit. Missing stop or descriptor proof keeps
Each completed export frees its own slot and wakes the next scan. A slow export
does not hold a free slot. Completed cleanup and saved-input obligations receive
one bounded pass through unvisited sources. The pass stops at its tail so paused
input does not keep the worker active; a committed change starts a fresh pass.
Shutdown joins this worker's exports before closing the database. Repeated Retry
during physical preservation binds the canonical intent
without changing the attempt's generation or revision. Progress writes preserve
that newer intent so its fenced completion can still commit. Missing stop or descriptor proof keeps
the source allocated and exposes the specific service blocker.
File recovery alone never requests another agent turn. Existing saved human
@@ -1995,6 +2022,25 @@ not grant execution authority: calls still use the existing gateway policy,
approval, revocation, and audit paths.
Native OpenCode sessions use the server-owned assigned-tool relay for local and
remote execution. The provider receives only bounded search and call schemas,
even when its protocol can fit the full catalog. This bounds provider grammar
compilation independently of the transport's tool-count and byte limits. Search
returns pinned tool aliases and schemas, intersects fresh grants, and respects
planning/ask restrictions. OpenCode discovery pages use a 32 KiB budget; larger
schemas use 16K-character chunks so JSON escaping and the result envelope fit
its HTTP MCP bridge. Other harnesses keep the existing PRP discovery budgets.
Calls retain the original gateway binding and its
argument validation, approvals, revocation, and audit rules; prompt arguments
cannot choose a gateway or credential. Assigned gateway launch credentials stay
on the controller. The MCP bridge preserves exact declared operation names,
including names that start with `paperclip_`; only a catalog-bound transport
prefix may resolve to that declared name. Shortened or unknown names do not
gain authority. OpenCode sessions with the former direct catalog rotate their
provider checkpoint before reuse; other providers retain their existing tool
contract fingerprints.
### Use real connection requests (2026-09-14)
When a user asks to connect a known service, the agent searches for that service
doc/task-recovery.md
+20 -2
View File
@@ -15,8 +15,26 @@ new generation; a repeated HTTP request cannot.
The existing exact-run retry endpoint retains its source-bound contract.
Connected conversations use their existing inbound authorization owner. A board
Retry changes the workspace strategy, not the external principal, source batch,
conversation access or publication authority.
Retry preserves external principal, conversation access and publication checks.
A GitHub Retry selects already-saved, undelivered input before replaying the
failed run's older request. The selected receipts transfer to the retry owner
under the dispatch transaction. They link to its one successor, including after
a restart or a lost dispatch response. Rejected, unadmitted retries release the
saved input for another deliberate Retry.
A deliberate GitHub Retry can admit saved input after the same App reconnects.
It verifies the App binding in the accepted delivery or its historical working
comment receipt, then checks current sender, repository, conversation and task
access. The retry records the current runtime generation and credential-reference
fingerprint. Another reconnect invalidates that admission. Historical deliveries,
stale webhook callbacks and the exact-run retry contract retain their fences.
Task Retry keeps the complete saved-input batch, including batches larger than
the exact-run endpoint's limit. Source rows, receipts and invalidations are read
in batches. Within one transaction, messages with the same principal and automatic
policy context share a current-access check; every message still proves its App,
destination, comment link and immutable source. Different automatic policy contexts
receive separate access checks. Input in another conversation remains queued.
A connected-conversation Retry can save its exact request while the matching
system recovery generation verifies the old executor. Saving that wait grants
maps/agent-access-map/recovery.md
+18
View File
@@ -4,6 +4,17 @@ Product map: [Recovery](../feature-map/recovery.md).
Feature ID: `recovery`.
Native finalization deadline reconciliation records unfinished workspace
preservation through the existing server-owned recovery path. It grants no
additional agent API, provider-stop authority or permission to replay work.
Ordinary board Retry can reauthorize saved GitHub input for the same App after
reconnect. It pins current runtime authority and preserves sender, repository,
conversation and task checks. Pending input takes precedence over the older
failed request and transfers once to the retry owner. This grants no new agent
Retry API or stale webhook authority. Local PostgreSQL and HTTP-route regressions
cover these transitions; hosted execution and delivery remain separate acceptance.
Source revision: `3bbc2104b65caf9f7763938a01260a5673e12f7a`.
Review depth and executed proof are separate. The operation records enumerate the named behavior and access boundaries. Their evidence states describe the reviewed sources; run results and environment limits belong in the PR or verification artifacts. A listed API or source test does not establish a live authorized journey. Follow the [map contract](../README.md).
@@ -225,6 +236,13 @@ Gaps: An acknowledged retry can remain pending or fail; report actual useful res
Capability ID: `recovery/workspace`.
Committed authorized Retry and workspace-recovery requests notify the internal
preservation worker. Startup reconstructs its source-bound obligations and saved
retry deadlines. A duplicate request during physical preservation does not
replace the owner's generation or revision. Resolved files-only recovery retains
its no-replay restriction. The local PostgreSQL regression checks the producer,
worker, and physical source receipt; it is not deployed provider acceptance.
The internal preservation resume path grants no board or agent execution API.
It requires the exact source action, company, run, lease and environment, a
held physical workspace owner, and its current stop-only cleanup claim. The
maps/agent-access-map/runs-adapters.md
+12
View File
@@ -162,6 +162,18 @@ No grant, token, authority cache or provider dispatch bypass is added. The
[executor controls](../../server/src/services/native-runtime/native-session-executor.test.ts)
remain separate from live latency and permitted/denied agent acceptance.
Scoped OpenCode assigned-tool source evidence: local and remote native sessions
use the server-owned authenticated relay. The provider sees bounded
`paperclip_search_assigned_tools` and `paperclip_call_assigned_tool` schemas,
then uses pinned aliases. Search intersects fresh grants and work modes; calls
retain the run's original gateway policy, validation, approvals, revocation and
audit path. Prompt arguments cannot select gateway authority. Assigned gateway
launch credentials stay on the controller. Former direct-catalog OpenCode
checkpoints refresh; other provider fingerprints stay unchanged. The
[projection controls](../../server/src/services/native-runtime/assigned-mcp-tools.test.ts)
and [wiring controls](../../server/src/services/native-runtime/native-session-executor.test.ts)
are source evidence, not live permitted/denied sandbox acceptance.
For a linked Codex subscription with server-resolved managed credential provenance,
setup checks the resolved owner and stages a private probe copy before local or
remote isolation. Without a selected API key, an unbound linked `CODEX_HOME` is
maps/feature-map/agents.md
+8
View File
@@ -128,6 +128,14 @@ from the hiring picker; API fixture creation does not prove this UI journey.
Automated: [instruction service](../../server/src/__tests__/agent-instructions-service.test.ts) and [instruction revisions](../../server/src/__tests__/agent-instruction-revisions.test.ts) cover file/revision contracts.
The [AI account field regression](../../ui/src/components/ai-connections/AiConnectionField.test.tsx)
checks duplicate provider names, account identity in choices and adoption confirmation,
the exact selected grant, and disabled unavailable accounts with authorized metadata fixtures.
Manually compare a personal API key and shared subscription with the same name in
Harness / Runtime. Confirm their method, scope and available identity before saving.
Reload the saved choice and verify the next eligible task uses it; labels alone
do not establish credential validity or change account access.
Manual: Save a harmless instruction edit, reload, and verify its revision and effect in the next eligible run. Pause and resume the disposable agent; confirm pause prevents new work and does not erase previous runs.
For failed setup, use an unavailable model or invalid command on a disposable agent.
maps/feature-map/recovery.md
+28
View File
@@ -33,6 +33,8 @@ Agent access: [companion](../agent-access-map/recovery.md).
workspace link when the destination is disabled. Local PostgreSQL and real
filesystem regressions cover native and legacy recovery and one-time delivery;
hosted provider recovery remains separate acceptance.
Native finalization deadline failures also retain the exact prepared source
for the same stop-and-save workflow before Retry admits another executor.
- `unknown-outcome`: `recovery/unknown-outcome` — uncertain external side effects require inspection and
an explicit decision, not automatic replay.
- `user-continuation`: `recovery/user-continuation` — saved user messages after a stop are reconsidered once
@@ -89,6 +91,14 @@ The connected-chat database regression in
exercises admission and queue continuation. It does not establish hosted
provider recovery or successful delivery.
For GitHub, save a follow-up behind a failed run, then reconnect the same App.
Click ordinary Retry and verify that one successor receives the saved follow-up
instead of replaying the older request. Reload and repeat the same request key;
there must be one successor and no duplicate publication. A second reconnect
after admission, changed App, revoked sender or changed repository must prevent
dispatch. An unadmitted rejection keeps the follow-up available for another
deliberate Retry. Historical deliveries must remain unchanged.
### `post-stop-message`
After a stop, send a new instruction on the same task or inspect one already
@@ -175,6 +185,13 @@ provider restart. For that journey, interrupt a disposable deployment after
workspace preparation and before its first executor claim, then confirm one
provider turn starts on the original run through the normal admission gates.
The [Runner transport regressions](../../packages/paperclip-runner/runner/crates/runner-core/src/durable/transport.rs)
exercise a real WebSocket welcome delayed beyond two seconds. Authentication
uses the ten-second connection setup budget, clipped by the current lifecycle
deadline. The cases also retain failed-welcome ticket fencing, identity checks,
and reconnect replay without repeated provider effects. These transport cases
do not establish hosted provider acceptance.
Active remote process supervision distinguishes a failed health read from a
confirmed exit. The [executor tests](../../server/src/services/native-runtime/native-session-executor.test.ts)
exercise timeout recovery with a real listening process, unchanged ownership,
@@ -242,6 +259,17 @@ gap.
### `operator-retry`
The [workspace recovery regression](../../server/src/services/workspace-recovery.test.ts)
uses the real Retry producer and a quiet delivery coordinator. It checks that a
committed request starts one physical source preservation, duplicate Retry keeps
the active owner's receipt valid, and startup reconstructs a resolved files-only
obligation without provider replay. The [execution-control lifecycle suite](../../server/src/__tests__/execution-control-work.test.ts)
checks independent finalization, stored deadlines, and a bounded pass through
completed obligations followed by quiet paused input.
The physical export regression holds one export while another completes, then
verifies a third Retry can use the free slot and shutdown joins the held work.
These local checks do not prove recovery of a deployed allocation.
Automated: [disposition notices](../../ui/src/components/DispositionRecoveryNotice.test.tsx)
tests pending/error/gated states and waiting for the real retry acknowledgement.
The recovery-card suite covers workspace divergence, reissue, reconcile, and
maps/feature-map/runs-adapters.md
+10
View File
@@ -145,6 +145,16 @@ is released before failure returns. Repeat a warm follow-up and retain its actua
reply time. The [executor controls](../../server/src/services/native-runtime/native-session-executor.test.ts)
cover ordering, failure drain and credential projection; they do not qualify a
live response-time target.
For native OpenCode with assigned connections, inspect the provider tool surface
on both local and sandbox execution. It should contain bounded assigned-tool
search/call schemas. Discover an allowed tool, call its returned alias, revoke
its grant, and confirm discovery and execution enforce that revocation. Repeat
in planning/ask mode with a write tool and confirm denial. The controller must
retain the assigned gateway URL and token. An older direct-catalog OpenCode
checkpoint must refresh without changing another provider's fingerprint. The
[assigned-tool](../../server/src/services/native-runtime/assigned-mcp-tools.test.ts)
and executor controls cover source behavior; live sandbox discovery and a real
call remain separate acceptance evidence.
Missing dependencies, failed authentication, invalid models and
incompatible custom settings must remain actionable errors without silent runner
switching. Saved unchanged execution can still be tested without applying new
maps/feature-map/tasks.md
+1 -1
View File
@@ -39,7 +39,7 @@ Open `/issues` or a project’s Issues view; legacy `/tasks` links redirect.
### `task-detail`
Open `/issues/:issueId` from a row, mention, or direct link. Cold opens show the instruction and recent exchange first. **More messages** loads earlier messages and run activity; incoming messages remain visible while the task stays open.
Open `/issues/:issueId` from a row, mention, or direct link. Cold opens show the instruction and recent exchange first. **More messages** appears only when earlier comments or runs are omitted and loads that history. Fresh tasks with no omitted history have no disclosure. A single legacy run with a truncated log offers **More run output** instead. **More task actions → Show task activity** loads full activity even for a single native run; incoming messages remain visible while the task stays open.
### `task-sharing`
packages/adapters/codex-local/src/ui/build-config.test.ts
+5 -1
View File
@@ -36,6 +36,11 @@ function makeValues(overrides: Partial<CreateConfigValues> = {}): CreateConfigVa
}
describe("buildCodexLocalConfig", () => {
it("preserves explicitly disabled hosted search in native Codex configuration", () => {
expect(buildPaperclipRunnerConfig(makeValues({ search: false }))).toHaveProperty("search", false);
expect(buildPaperclipRunnerConfig(makeValues({ search: true }))).not.toHaveProperty("search");
});
describe("qualified Copilot config", () => {
it("round-trips Copilot with the exact selected model", () => {
const first = buildPaperclipRunnerConfig(makeValues({ model: "gpt-5.6-luna", adapterSchemaValues: { provider: "acpx", acpxAgent: "copilot" } }));
@@ -171,7 +176,6 @@ describe("buildPaperclipRunnerConfig", () => {
"engine",
"agentCommand",
"stateDir",
"search",
"fastMode",
"dangerouslyBypassApprovalsAndSandbox",
"command",
packages/adapters/codex-local/src/ui/build-config.ts
+1
View File
@@ -249,6 +249,7 @@ export function buildPaperclipRunnerConfig(v: CreateConfigValues): Record<string
provider,
...(provider === "codex"
? { model: resolvePaperclipRunnerModel("codex", config.model),
...(v.search === false ? { search: false } : {}),
...(configuredEffort ? { modelReasoningEffort: configuredEffort } : {}) }
: {}),
codexPermissionMode: "never",
packages/paperclip-runner/acpx-profiles.json
+2 -2
View File
@@ -32,12 +32,12 @@
"requiresProviderPolicy": true
},
"copilot": {
"agentProfileVersion": 59,
"agentProfileVersion": 61,
"agentServerPackage": "@github/copilot",
"agentServerVersion": "1.0.88",
"agentRuntimePackage": null,
"agentRuntimeVersion": null,
"commandDigest": "sha256:2f39ffe54a6af832d044b5098c31df9e08d4d27aee0ac675ba4b0c9da7445c40",
"commandDigest": "sha256:956276ebee59f880249fa6955adc224af9ca7b6a9da59bcc5da4a8667550c5b3",
"requiresProviderPolicy": true
},
"claude": {
packages/paperclip-runner/docs/durable-recovery.md
+16
View File
@@ -175,6 +175,22 @@ The ACK means that every source event through sequence 9 is committed or
deduplicated. The runner rejects an ACK that moves backward or beyond its
produced source cursor. It removes only events at or below a valid ACK.
The controller stages `item.delta` output snapshots with asynchronous file writes and file
flushes. A slow volume does not block unrelated HTTP requests during those
operations. The controller still publishes the snapshot atomically, flushes its
directory, and waits for persistence before acknowledging an event.
A newer command or authenticated connection can save the
same event while staging is pending; the older snapshot cannot replace that
newer durable state. A failed write fences further authority until reload.
Lifecycle, control, authentication, and semantic tool events retain synchronous
persistence before transport readers can observe their state. Semantic tool
admission and dispatch cannot yield to a replacement connection between those operations.
Controller replacement must still stop ingress and drain admitted processing
before a successor reuses the journal. External control routes stop their
reconnect loop before the controller joins admitted work. An incomplete drain
cannot certify a reusable checkpoint. Snapshots use compact JSON; existing
indented snapshots remain readable.
For the lost-ACK fault, the mock commits an event, drops the ACK and socket, and
reports the prior cursor once after reconnect. The runner sends the same bytes
again. The mock increments delivery count, keeps one logical event, and sends
packages/paperclip-runner/runner/crates/runner-core/src/durable/transport.rs
+113 -3
View File
@@ -26,7 +26,7 @@ use super::{
const SECURE_FRAME_SCHEMA: &str = "paperclip.runner.secure-frame.v1";
const AUTH_TIMEOUT: Duration = Duration::from_secs(2);
const WELCOME_TIMEOUT: Duration = Duration::from_secs(2);
const WELCOME_TIMEOUT: Duration = CONNECT_TOTAL_TIMEOUT;
const RUNTIME_READ_TIMEOUT: Duration = Duration::from_millis(250);
const CONNECT_ATTEMPT_TIMEOUT: Duration = Duration::from_secs(3);
const CONNECT_TOTAL_TIMEOUT: Duration = Duration::from_secs(10);
@@ -1106,8 +1106,9 @@ impl AuthenticatedTransport {
max_frame_bytes: config.max_frame_bytes,
};
// Ticket validation and lease persistence happen before the server
// emits welcome. Keep that work bounded without assuming a loaded
// control-plane event loop can always respond within 250 ms.
// emits welcome. Give that durable admission the connection setup
// budget; a two-second wait can discard a consumed ticket under
// controller load. The execution/reconnect deadline still wins.
let welcome_deadline = bounded_operation_deadline(connect_deadline, WELCOME_TIMEOUT)?;
let welcome_timeout = welcome_deadline.saturating_duration_since(Instant::now());
if welcome_timeout.is_zero() {
@@ -2358,6 +2359,115 @@ mod tests {
assert!(validate_control_identity(&envelope, &state, Some(&connection)).is_err());
}
#[test]
fn bootstrap_welcome_waits_for_bounded_controller_admission() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
let mut config = config(port);
config.max_runtime = Duration::from_secs(10);
let state = test_state(&config);
let server_config = config.clone();
let server_state = state.clone();
let server = thread::spawn(move || {
let (stream, _) = listener.accept().unwrap();
let mut socket = accept(stream).unwrap();
let expires = current_unix_ms().unwrap() + 60_000;
let mut secure = server_authenticate(
&mut socket,
&server_config,
&server_state,
ServerCredential {
token: "bootstrap-secret",
kind: "bootstrap",
lease_id: None,
expires_at_unix_ms: expires,
revocation_epoch: 0,
},
);
// The authority has the client proof. Durable admission and lease
// persistence may take longer than the old two-second welcome wait.
thread::sleep(Duration::from_secs(3));
send_secure(
&mut socket,
&mut secure,
&server_config,
&welcome(
&server_state,
"connection_1",
Some("lease-secret"),
expires,
0,
vec![],
),
);
});
let endpoint = RunnerTransportEndpoint::new(&config.connect_url, &config.run_id).unwrap();
let ticket = BootstrapTicket::new("bootstrap-secret".to_owned()).unwrap();
let result = AuthenticatedTransport::connect(
&endpoint,
&config,
&state,
Some(&ticket),
None,
Instant::now() + config.max_runtime,
);
server.join().unwrap();
let (transport, greeting) = result.unwrap().unwrap();
assert_eq!(greeting.lease.unwrap().expose().unwrap(), "lease-secret");
assert_eq!(
transport.socket.read_timeout().unwrap(),
Some(RUNTIME_READ_TIMEOUT)
);
}
#[test]
fn bootstrap_welcome_retains_the_shorter_lifecycle_deadline() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
let config = config(port);
let state = test_state(&config);
let server_config = config.clone();
let server_state = state.clone();
let (release, held) = mpsc::channel();
let server = thread::spawn(move || {
let (stream, _) = listener.accept().unwrap();
let mut socket = accept(stream).unwrap();
server_authenticate(
&mut socket,
&server_config,
&server_state,
ServerCredential {
token: "bootstrap-secret",
kind: "bootstrap",
lease_id: None,
expires_at_unix_ms: current_unix_ms().unwrap() + 60_000,
revocation_epoch: 0,
},
);
held.recv_timeout(Duration::from_secs(2)).unwrap();
});
let endpoint = RunnerTransportEndpoint::new(&config.connect_url, &config.run_id).unwrap();
let ticket = BootstrapTicket::new("bootstrap-secret".to_owned()).unwrap();
let started = Instant::now();
let result = AuthenticatedTransport::connect(
&endpoint,
&config,
&state,
Some(&ticket),
None,
started + Duration::from_secs(1),
);
release.send(()).unwrap();
server.join().unwrap();
let failure = match result {
Err(error) => error,
Ok(_) => panic!("welcome authenticated after the lifecycle deadline"),
};
assert!(failure.bootstrap_maybe_consumed);
assert!(started.elapsed() < Duration::from_secs(2));
assert!(failure.error.to_string().contains("deadline elapsed"));
}
#[test]
fn authenticates_bootstrap_and_receives_bound_lease() {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
packages/paperclip-runner/runner/crates/runner-core/src/generated_acpx_profiles.rs
+1 -1
View File
@@ -47,7 +47,7 @@ pub(crate) fn acpx_release_profile(agent: &str) -> Option<AcpxReleaseProfile> {
agent_runtime_package: None,
agent_runtime_version: None,
command_digest:
"sha256:2f39ffe54a6af832d044b5098c31df9e08d4d27aee0ac675ba4b0c9da7445c40",
"sha256:956276ebee59f880249fa6955adc224af9ca7b6a9da59bcc5da4a8667550c5b3",
requires_provider_policy: true,
},
"claude" => AcpxReleaseProfile {
packages/paperclip-runner/scripts/controlled-evals/controller.test.mjs
+9 -4
View File
@@ -8,6 +8,7 @@ import { writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { createRequire } from 'node:module';
const runtime = process.env.PAPERCLIP_CONTROLLER_TEST_RUNTIME
? pathToFileURL(resolve(process.env.PAPERCLIP_CONTROLLER_TEST_RUNTIME) + '/') : import.meta.url;
const { canonical, costRuntime, observabilityBoundConfiguration, requestBindingSha256, sha256, validateManifest } = await import(new URL('./manifest.mjs', runtime));
@@ -18,6 +19,8 @@ const { readRegularFile, replaceEvidenceFile } = await import(new URL('./evidenc
import { aggregateProtocolEvalCampaign, sanitizeProtocolEvalRuns } from '../runner-protocol-eval-campaign.mjs';
const { build } = createRequire(new URL('../../../../package.json', import.meta.url))('esbuild');
const childScript = options => `
import assert from 'node:assert/strict';
import {createHash} from 'node:crypto';
@@ -54,10 +57,7 @@ if(args.has('--observability-config')) {
}
nativeObservability={mode:'controller_otlp',configured:true,verification:'unverified',projectId:config.projectId,traceId:config.traceId,spanId:config.spanId,descriptorSha256:createHash('sha256').update(raw).digest('hex')};
if(${Boolean(options.realObservabilityValidator)}) {
const {createRequire}=await import('node:module');
const require=createRequire(${JSON.stringify(fileURLToPath(new URL('../../../../server/package.json', import.meta.url)))});
const {tsImport}=await import(require.resolve('tsx/esm/api'));
const {prepareEvalObservabilityRuntime}=await tsImport(${JSON.stringify(fileURLToPath(new URL('../../src/cli/eval-observability-runtime.ts', import.meta.url)))},import.meta.url);
const {prepareEvalObservabilityRuntime}=await import(${JSON.stringify(options.observabilityValidatorModule ?? '')});
await mkdir(process.env.HOME+'/codex',{recursive:true});
const prepared=await prepareEvalObservabilityRuntime(args.get('--observability-config'),request,
{evidence:{campaignId:descriptor.campaignId,executionId:descriptor.executionId},transportOptions:{environment:{CODEX_HOME:process.env.HOME+'/codex'}}},process.env);
@@ -90,6 +90,11 @@ async function fixture(t, { count = 1, allocation = '0.01', providerProfile = 'c
const root = await mkdtemp(join(tmpdir(), 'runner-controller-test-'));
t.after(() => rm(root, { recursive: true, force: true }));
const cli = join(root, 'fake-cli.mjs'), binary = join(root, 'runnerd'), archive = join(root, 'runner.tgz');
if (options.realObservabilityValidator) {
options.observabilityValidatorModule = join(root, 'eval-observability-runtime.mjs');
await build({ entryPoints: [fileURLToPath(new URL('../../src/cli/eval-observability-runtime.ts', import.meta.url))],
outfile: options.observabilityValidatorModule, bundle: true, platform: 'node', format: 'esm', target: 'node24', logLevel: 'silent' });
}
await writeFile(cli, childScript(options)); await writeFile(binary, 'synthetic daemon'); await writeFile(archive, 'synthetic package');
const provenance = async path => ({ path, sha256: sha256(await readFile(path)) });
const now = Date.now(), verifiedAt = new Date(now - 1000).toISOString(), expiresAt = new Date(now + 86_400_000).toISOString();
packages/paperclip-runner/scripts/copilot-profile-identity.mjs
+2 -2
View File
@@ -7,9 +7,9 @@ import { fileURLToPath } from "node:url";
import { resolve, dirname } from "node:path";
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const fixture = resolve(root, "test/fixtures/copilot-profile-v59-identity.json");
const fixture = resolve(root, "test/fixtures/copilot-profile-v61-identity.json");
const historical = JSON.parse(readFileSync(resolve(root, "test/fixtures/copilot-profile-v12-identity.json"), "utf8"));
const declaration = { ...historical.declaration, agentProfileVersion: 59 };
const declaration = { ...historical.declaration, agentProfileVersion: 61 };
delete declaration.permissionIdentitySourceSha256;
const sources = {
displayEventSourceSha256: "src/drivers/acpx/copilot-events.ts",
packages/paperclip-runner/src/backends/codex-native-backend.ts
+2
View File
@@ -154,6 +154,8 @@ function createTransportBackedNativeSessionBackend(
return new HarnessDriverBackend(
new CodexAppServerDriver({
...(input.provider.model ? { model: input.provider.model } : {}),
...(input.provider.kind === "codex" && "webSearch" in input.provider && input.provider.webSearch === "disabled"
? { disableWebSearch: true } : {}),
...(input.provider.kind === "codex" && "reasoningEffort" in input.provider && input.provider.reasoningEffort
? { reasoningEffort: input.provider.reasoningEffort }
: {}),
packages/paperclip-runner/src/backends/native-backend-factory.test.ts
+23 -1
View File
@@ -2,7 +2,8 @@ import { QUALIFIED_ACPX_PROFILES, resolveQualifiedAcpxProfile } from "../drivers
import { mkdtempSync, readFileSync, writeFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { afterEach, describe, expect, it, vi } from "vitest";
import * as codexSecurity from "../drivers/codex/codex-security-config.js";
import { buildNativeModelEnvelope, parseNativeExecutionInput, type NativeExecutionInput } from "../contracts/native-execution.js";
import { NATIVE_RUNTIME_ASSET_SCHEMA, PAPERCLIP_EXECUTION_PROMPT, PAPERCLIP_EXECUTION_PROMPT_REVISION, nativeRuntimePromptDigest, canonicalNativeRuntimeContextDigest, composeNativeSystemInstructions } from "../contracts/runtime-context.js";
@@ -210,6 +211,26 @@ function managedExecution(
}
describe("native backend factory", () => {
it.each([undefined, "disabled"] as const)("honors saved webSearch=%s with the default process transport", async webSearch => {
const input = preparedExecution(execution());
if (input.provider.kind !== "codex") throw new Error("Codex fixture required");
if (webSearch) Object.assign(input.provider, { webSearch });
const buildArgs = codexSecurity.createIsolatedCodexAppServerArgs;
let generated: string[] | undefined;
const spy = vi.spyOn(codexSecurity, "createIsolatedCodexAppServerArgs").mockImplementation((...args) => {
generated = buildArgs(...args);
throw new Error("offline-stop-before-process-spawn");
});
try {
const backend = createNativeSessionBackend(input, { environment: {} });
await expect(backend.openSession({ identity: { runId: "run", sessionId: "session", companyId: "company", agentId: "agent" },
workingDirectory: WORKSPACE })).rejects.toThrow("offline-stop-before-process-spawn");
expect(generated).toBeDefined();
if (webSearch) expect(generated?.slice(-3)).toEqual(["-c", 'web_search="disabled"', "app-server"]);
else expect(generated).not.toContain('web_search="disabled"');
} finally { spy.mockRestore(); }
});
it("constructs the Codex backend without starting its transport", async () => {
const backend = createNativeSessionBackend(execution(), {
codexTransportFactory: () => {
@@ -637,6 +658,7 @@ describe("native backend factory", () => {
["cursor", "../../test/fixtures/cursor-acp/profile-v7-identity.json"],
["cursor", "../../test/fixtures/cursor-acp/profile-v10-identity.json"],
["copilot", "../../test/fixtures/copilot-profile-v14-identity.json"],
["copilot", "../../test/fixtures/copilot-profile-v60-identity.json"],
["pi", "../../test-fixtures/pi-acp/profile-v13-identity.json"],
["pi", "../../test-fixtures/pi-acp/profile-v14-identity.json"],
["pi", "../../test-fixtures/pi-acp/profile-v18-identity.json"],
packages/paperclip-runner/src/contracts/native-execution.test.ts
+7
View File
@@ -194,6 +194,13 @@ describe("NativeExecutionInputV1", () => {
});
expect(withEffort.provider).toMatchObject({ kind: "codex", reasoningEffort: "ultra" });
expect(parseNativeExecutionInput(withEffort)).toEqual(withEffort);
const disabledSearch = parseNativeExecutionInput({ ...withEffort, provider: { ...withEffort.provider, webSearch: "disabled" } });
expect(disabledSearch.provider).toHaveProperty("webSearch", "disabled");
expect(parseNativeExecutionInput(JSON.parse(JSON.stringify(disabledSearch)))).toEqual(disabledSearch);
for (const webSearch of [true, false, "live", "cached", null]) {
expect(() => parseNativeExecutionInput({ ...withEffort, provider: { ...withEffort.provider, webSearch } })).toThrow("webSearch");
}
expect(() => parseNativeExecutionInput({
...withEffort,
provider: { kind: "codex", model: "gpt-6-astra", approvalPolicy: "on-request", reasoningEffort: "impossible" },
packages/paperclip-runner/src/contracts/native-execution.ts
+6 -2
View File
@@ -160,7 +160,7 @@ export type NativeProviderConfigV4 =
export type NativeProviderConfigV5 =
| Exclude<NativeProviderConfigV4, { kind: "codex" }>
| (Extract<NativeProviderConfigV4, { kind: "codex" }> & { reasoningEffort?: string });
| (Extract<NativeProviderConfigV4, { kind: "codex" }> & { reasoningEffort?: string; webSearch?: "disabled" });
export interface NativeExecutionInputV1 {
schema: typeof NATIVE_EXECUTION_INPUT_SCHEMA_V1;
@@ -571,7 +571,7 @@ export function parseNativeExecutionInput(value: unknown): NativeExecutionInput
: provider.kind === "acpx"
? ["kind", "agent", "model", isV4 ? "permissionMode" : "permissionPolicy", "profile", "piThinkingLevel", ...(isV4 ? ["mode"] : []), ...(isV9 ? ["claudeEffort"] : [])]
: provider.kind === "codex" && isV4
? ["kind", "model", "approvalPolicy", ...(isV5 ? ["reasoningEffort"] : [])]
? ["kind", "model", "approvalPolicy", ...(isV5 ? ["reasoningEffort", "webSearch"] : [])]
: provider.kind === "opencode" && isV4
? ["kind", "model", "permissionMode"]
: ["kind", "model"],
@@ -780,6 +780,9 @@ export function parseNativeExecutionInput(value: unknown): NativeExecutionInput
if (isV5 && provider.reasoningEffort !== undefined && !["minimal", "low", "medium", "high", "xhigh", "max", "ultra"].includes(String(provider.reasoningEffort))) {
throw new NativeExecutionInputError("input.provider.reasoningEffort is unsupported");
}
if (isV5 && provider.webSearch !== undefined && provider.webSearch !== "disabled") {
throw new NativeExecutionInputError("input.provider.webSearch must be disabled when specified");
}
parsedProvider = {
kind: "codex",
model: providerModel,
@@ -789,6 +792,7 @@ export function parseNativeExecutionInput(value: unknown): NativeExecutionInput
...(isV5 && provider.reasoningEffort !== undefined
? { reasoningEffort: text(provider.reasoningEffort, "input.provider.reasoningEffort") }
: {}),
...(isV5 && provider.webSearch === "disabled" ? { webSearch: "disabled" as const } : {}),
};
}
if (!Array.isArray(contract.criteria) || contract.criteria.length === 0) {
packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts
+113
View File
@@ -13,6 +13,8 @@ import {
} from "node:fs";
import { connect, type Socket } from "node:net";
import nodeFs from "node:fs";
import fsPromises from "node:fs/promises";
import { createServer } from "node:http";
import { syncBuiltinESMExports } from "node:module";
import { tmpdir } from "node:os";
import { resolve } from "node:path";
@@ -47,6 +49,117 @@ const identity: DurableRecoveryIdentity = {
const expectedRunnerVersion = "0.3.0";
const expectedRunnerDigest = `sha256:${"a".repeat(64)}`;
it.each(["event", "newer_command", "replacement", "write_failure", "drain", "semantic_replacement"] as const)(
"keeps HTTP responsive during a durable event write and preserves %s", async (mode) => {
const root = mkdtempSync(resolve(tmpdir(), "runner-event-io-"));
const dispatch = vi.fn(async () => {
expect(JSON.parse(readFileSync(resolve(root, "control-plane-state.json"), "utf8")).ackedSourceSeq).toBe(1);
return { result: { ok: true } };
});
const core = new DurablePrpControlPlane({ stateDirectory: root, identity,
expectedRunnerVersion, expectedRunnerDigest, onCommittedEvent: async () => {}, onSemanticToolInput: dispatch });
const http = createServer((_req, res) => res.end("responsive"));
let release!: () => void;
const barrier = new Promise<void>(resolveWrite => { release = resolveWrite; });
const originalOpen = fsPromises.open;
let staged = false;
let holdFirst = true;
let client: AuthenticatedClient | null = null;
let replacement: AuthenticatedClient | null = null;
const open = vi.spyOn(fsPromises, "open").mockImplementation(async (...args) => {
const file = await originalOpen(...args);
if (holdFirst && String(args[0]).includes(".control-plane-state.json.")) {
holdFirst = false;
const sync = file.sync.bind(file);
file.sync = async () => {
staged = true;
await barrier;
if (mode === "write_failure") throw new Error("controlled disk failure");
await sync();
};
}
return file;
});
try {
await core.start();
await new Promise<void>(resolveListen => http.listen(0, "127.0.0.1", resolveListen));
client = (await authenticate(core, core.issueBootstrapTicket()))!;
const event = semanticInputEvent();
const payload = event.payload as Record<string, unknown>;
if (mode !== "semantic_replacement") {
payload.eventType = "item.delta";
payload.priority = 2;
payload.payload = { delta: "output while the volume is slow" };
}
expect(validatePrpEvent(payload)).toMatchObject({ ok: true });
sendSecure(client, event);
await vi.waitFor(() => expect(staged || dispatch.mock.calls.length > 0).toBe(true));
if (mode === "semantic_replacement") {
replacement = (await authenticate(core, client.leaseToken!))!;
release();
await vi.waitFor(() => expect(dispatch).toHaveBeenCalledTimes(1));
await vi.waitFor(() => expect(core.store.state.commands.some(command => command.type === "semantic_tool.result")).toBe(true));
return;
}
// The actual socket request completes while the real file's fsync is held.
const address = http.address() as { port: number };
expect(await (await fetch(`http://127.0.0.1:${address.port}`)).text()).toBe("responsive");
expect(JSON.parse(readFileSync(core.store.path, "utf8")).ackedSourceSeq).toBe(0);
if (mode === "newer_command") core.queueCommand("session.snapshot", {}, "command_during_event");
let drained = false;
const retirement = mode === "drain" ? (async () => {
await core.stop();
await core.drainPendingConnectionProcessing();
drained = true;
})() : null;
if (retirement) {
await new Promise<void>(resolveTurn => setImmediate(resolveTurn));
expect(drained).toBe(false);
}
if (mode === "replacement") {
replacement = (await authenticate(core, client.leaseToken!))!;
const second = structuredClone(event);
const next = second.payload as Record<string, unknown>;
next.sourceSeq = 2;
next.sourceEventId = "output-event-2";
sendSecure(replacement, second);
expect(await receiveSecure(replacement)).toMatchObject({ kind: "ack", payload: { ackedSourceSeq: 2 } });
}
release();
if (retirement) {
expect(await receiveSecure(client)).toBeNull();
await retirement;
expect(drained).toBe(true);
} else if (mode === "write_failure") {
expect(await receiveSecure(client)).toBeNull();
expect(() => core.queueCommand("session.snapshot")).toThrow("indeterminate");
} else if (mode !== "replacement") {
expect(await receiveSecure(client)).toMatchObject({ kind: "ack", payload: { ackedSourceSeq: 1 } });
}
await core.stop();
await core.drainPendingConnectionProcessing();
const restored = new DurablePrpControlPlane({ stateDirectory: root, identity,
expectedRunnerVersion, expectedRunnerDigest });
const expectedEvents = mode === "replacement" ? 2 : mode === "write_failure" ? 0 : 1;
expect(restored.store.state.ackedSourceSeq).toBe(expectedEvents);
expect(restored.store.state.committedEvents).toHaveLength(expectedEvents);
if (mode === "newer_command") {
expect(restored.store.state.commands.some(command => command.commandId === "command_during_event")).toBe(true);
}
expect(nodeFs.readdirSync(root).filter(name => name.endsWith(".tmp"))).toEqual([]);
} finally {
release();
open.mockRestore();
client?.socket.destroy();
replacement?.socket.destroy();
await core.stop();
await core.drainPendingConnectionProcessing();
await new Promise<void>(resolveClose => http.close(() => resolveClose()));
rmSync(root, { recursive: true, force: true });
}
},
);
it("commits credential updates before ACK and preserves replay identity", async () => {
const root = mkdtempSync(resolve(tmpdir(), "runner-credential-update-"));
let release!: () => void;
packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts
+42 -20
View File
@@ -23,12 +23,13 @@ import {
renameSync,
unlinkSync,
writeFileSync,
type Stats,
} from "node:fs";
import { createServer, type IncomingMessage, type Server } from "node:http";
import { dirname, resolve } from "node:path";
import type { Duplex } from "node:stream";
import { fileURLToPath } from "node:url";
import { withStagedPrivateWrite } from "./staged-private-write.js";
import { verifyPrivateRegularFile } from "./private-state-file.js";
import { NativeSessionProtocolIntegrityError } from "../contracts/native-session-backend.js";
import { ACPX_CREDENTIAL_BINDING_ENV, ACPX_CREDENTIAL_NAMES, CLAUDE_ROUTING_ENV_KEYS, createAcpxSidecarHostEnvironment } from "../drivers/acpx/environment.js";
@@ -1122,22 +1123,6 @@ function verifyPrivateDirectory(path: string): void {
}
}
function verifyPrivateRegularFile(file: Stats, path: string): void {
if (!file.isFile()) {
throw new Error(`Private state path is not a regular file: ${path}`);
}
if (process.platform !== "win32") {
if ((file.mode & 0o777) !== 0o600) {
throw new Error(`Private state file does not use mode 0600: ${path}`);
}
if (process.geteuid !== undefined && file.uid !== process.geteuid()) {
throw new Error(
`Private state file is not owned by the daemon user: ${path}`,
);
}
}
}
export function readPrivateFile(path: string): string | null {
let descriptor: number;
try {
@@ -1218,6 +1203,8 @@ class DurableCoreStore {
readonly path: string;
#state: StoredCoreState;
#writeIndeterminate = false;
#revision = 0;
#savedRevision = 0;
constructor(directory: string, identity: DurableRecoveryIdentity) {
try {
@@ -1255,7 +1242,33 @@ class DurableCoreStore {
save(): void {
this.assertWritable();
atomicPrivateWrite(this.path, `${JSON.stringify(this.#state, null, 2)}\n`);
const revision = ++this.#revision;
try {
atomicPrivateWrite(this.path, `${JSON.stringify(this.#state)}\n`);
this.#savedRevision = revision;
} catch (error) {
this.#writeIndeterminate = true;
throw error;
}
}
async saveEvent(): Promise<void> {
this.assertWritable();
const revision = ++this.#revision;
try {
await withStagedPrivateWrite(this.path, `${JSON.stringify(this.#state)}\n`, (temporary) => {
this.assertWritable();
// A command or replacement connection can persist this event and newer
// authority while its file is staged. Never roll that state back.
if (this.#savedRevision >= revision) return;
renameSync(temporary, this.path);
syncParentDirectory(this.path);
this.#savedRevision = revision;
});
} catch (error) {
this.#writeIndeterminate = true;
throw error;
}
}
assertWritable(): void {
@@ -1268,9 +1281,11 @@ class DurableCoreStore {
/** Persist a complete candidate before publishing any new authority in memory. */
commit(candidate: StoredCoreState): void {
this.assertWritable();
const revision = ++this.#revision;
try {
atomicPrivateWrite(this.path, `${JSON.stringify(candidate, null, 2)}\n`);
atomicPrivateWrite(this.path, `${JSON.stringify(candidate)}\n`);
this.#state = candidate;
this.#savedRevision = revision;
} catch (error) {
// Rename may already have succeeded before directory fsync failed.
// Never overwrite that possibly durable receipt using stale memory.
@@ -3292,7 +3307,14 @@ export class DurablePrpControlPlane {
});
this.#store.state.ackedSourceSeq = sourceSeq;
}
this.#store.save();
// Output deltas have no lifecycle or control effects. Other events must
// become durable before transport readers can observe them in memory.
if (eventType === "item.delta") {
await this.#store.saveEvent();
if (this.#eventOwner !== connection || this.#identity !== identity || this.#protocolIntegrityError !== null) return;
} else {
this.#store.save();
}
if (
isSemanticInput &&
packages/paperclip-runner/src/control-plane/private-state-file.ts
+15
View File
@@ -0,0 +1,15 @@
import type { Stats } from "node:fs";
export function verifyPrivateRegularFile(file: Stats, path: string): void {
if (!file.isFile()) {
throw new Error(`Private state path is not a regular file: ${path}`);
}
if (process.platform !== "win32") {
if ((file.mode & 0o777) !== 0o600) {
throw new Error(`Private state file does not use mode 0600: ${path}`);
}
if (process.geteuid !== undefined && file.uid !== process.geteuid()) {
throw new Error(`Private state file is not owned by the daemon user: ${path}`);
}
}
}
packages/paperclip-runner/src/control-plane/staged-private-write.ts
+33
View File
@@ -0,0 +1,33 @@
import { randomUUID } from "node:crypto";
import { constants } from "node:fs";
import fs from "node:fs/promises";
import { basename, dirname, resolve } from "node:path";
import { verifyPrivateRegularFile } from "./private-state-file.js";
/** Stage durable bytes off the event loop; the owner publishes them without yielding. */
export async function withStagedPrivateWrite(
path: string,
contents: string,
publish: (temporary: string) => void,
): Promise<void> {
const temporary = resolve(dirname(path), `.${basename(path)}.${randomUUID()}.tmp`);
const file = await fs.open(temporary, constants.O_WRONLY | constants.O_CREAT |
constants.O_EXCL | (constants.O_NOFOLLOW ?? 0), 0o600);
try {
try {
if (process.platform !== "win32") await file.chmod(0o600);
const metadata = await file.stat();
verifyPrivateRegularFile(metadata, temporary);
await file.writeFile(contents, "utf8");
await file.sync();
} finally {
await file.close();
}
publish(temporary);
} finally {
// Publication can either rename this file or retain a newer durable write.
await fs.unlink(temporary).catch((error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error;
});
}
}
packages/paperclip-runner/src/drivers/acpx/copilot-installation.test.ts
+32 -2
View File
@@ -87,9 +87,39 @@ vi.mock("./installation-integrity.js", () => ({ verifyNativeAcpxInstallation: vi
describe("Copilot build-owned installation", () => {
it("admits the current profile with environment launch preparation", () => {
expect(QUALIFIED_ACPX_PROFILES.copilot.agentProfileVersion).toBe(59);
expect(QUALIFIED_ACPX_PROFILES.copilot.agentProfileVersion).toBe(61);
expect(QUALIFIED_ACPX_PROFILES.copilot.qualificationStatus).toBeUndefined();
expect(QUALIFIED_ACPX_PROFILES.copilot.commandDigest).toBe("sha256:2f39ffe54a6af832d044b5098c31df9e08d4d27aee0ac675ba4b0c9da7445c40");
expect(QUALIFIED_ACPX_PROFILES.copilot.commandDigest).toBe("sha256:956276ebee59f880249fa6955adc224af9ca7b6a9da59bcc5da4a8667550c5b3");
});
it("preserves v60 while binding controller journal retirement in v61", async () => {
const priorBytes = readFileSync(new URL("../../../test/fixtures/copilot-profile-v60-identity.json", import.meta.url));
expect(createHash("sha256").update(priorBytes).digest("hex")).toBe("e3c45b0311517d05f06ed10f357db713ae157cfa87e4a8becf087c5b6dfa7003");
const prior = JSON.parse(priorBytes.toString("utf8"));
const current = JSON.parse(readFileSync(new URL("../../../test/fixtures/copilot-profile-v61-identity.json", import.meta.url), "utf8"));
expect(Object.keys(current.declaration).filter(key => current.declaration[key] !== prior.declaration[key]).sort()).toEqual([
"agentProfileVersion", "profileCompatibilitySourceSha256", "runtimeRequestBridgeSourceSha256",
]);
expect(Object.keys(current.declaration).sort()).toEqual(Object.keys(prior.declaration).sort());
expect(isSupportedAcpxProfileVersion("copilot", 60)).toBe(true);
expect(current.commandDigest).not.toBe(prior.commandDigest);
await expect(verifyCopilotInstallation({ ...QUALIFIED_ACPX_PROFILES.copilot,
agentProfileVersion: 60, commandDigest: prior.commandDigest })).rejects.toThrow("exact pinned");
});
it("preserves v59 while binding the Codex-only hosted search restriction in v60", async () => {
const priorBytes = readFileSync(new URL("../../../test/fixtures/copilot-profile-v59-identity.json", import.meta.url));
expect(createHash("sha256").update(priorBytes).digest("hex")).toBe("a9116d0981cb0feebec31f61cd23d16b3a263fe6a797b1ba1436c6e41198538c");
const prior = JSON.parse(priorBytes.toString("utf8"));
const current = JSON.parse(readFileSync(new URL("../../../test/fixtures/copilot-profile-v60-identity.json", import.meta.url), "utf8"));
expect(Object.keys(current.declaration).filter(key => current.declaration[key] !== prior.declaration[key]).sort()).toEqual([
"agentProfileVersion", "nativeBackendSourceSha256", "nativeExecutionSourceSha256", "profileCompatibilitySourceSha256", "providerLaunchSourceSha256", "runtimeRequestBridgeSourceSha256",
]);
expect(Object.keys(current.declaration).sort()).toEqual(Object.keys(prior.declaration).sort());
expect(isSupportedAcpxProfileVersion("copilot", 59)).toBe(true);
expect(current.commandDigest).not.toBe(prior.commandDigest);
await expect(verifyCopilotInstallation({ ...QUALIFIED_ACPX_PROFILES.copilot,
agentProfileVersion: 59, commandDigest: prior.commandDigest })).rejects.toThrow("exact pinned");
});
it("preserves v58 while binding the separate remote process reconnect policy in v59", () => {
packages/paperclip-runner/src/drivers/acpx/generated-profiles.ts
+2 -2
View File
@@ -48,12 +48,12 @@ export const QUALIFIED_ACPX_PROFILE_DATA = {
"protocolVersion": 1,
"acpxVersion": "0.13.1",
"agent": "copilot",
"agentProfileVersion": 59,
"agentProfileVersion": 61,
"agentServerPackage": "@github/copilot",
"agentServerVersion": "1.0.88",
"agentRuntimePackage": null,
"agentRuntimeVersion": null,
"commandDigest": "sha256:2f39ffe54a6af832d044b5098c31df9e08d4d27aee0ac675ba4b0c9da7445c40",
"commandDigest": "sha256:956276ebee59f880249fa6955adc224af9ca7b6a9da59bcc5da4a8667550c5b3",
"permissionPolicy": "interactive"
},
"claude": {
packages/paperclip-runner/src/drivers/acpx/profile-compatibility.ts
+2 -2
View File
@@ -2,13 +2,13 @@ import { QUALIFIED_ACPX_PROFILE_DATA } from "./generated-profiles.js";
/** Decodable historical profile revisions, not permission to launch them.
* Exact current profile/package/digest admission is checked separately. */
type HistoricalAcpxProfileVersion = 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58;
type HistoricalAcpxProfileVersion = 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60;
type CurrentAcpxProfileVersion = typeof QUALIFIED_ACPX_PROFILE_DATA[keyof typeof QUALIFIED_ACPX_PROFILE_DATA]["agentProfileVersion"];
export type AcpxProfileVersion = HistoricalAcpxProfileVersion | CurrentAcpxProfileVersion;
const historicalVersions: Readonly<Record<string, readonly HistoricalAcpxProfileVersion[]>> = {
pi: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], claude: [1, 2, 3, 4, 5], codex: [1, 2, 3, 4, 5],
grok: [1, 2, 3, 4, 5], copilot: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58],
grok: [1, 2, 3, 4, 5], copilot: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60],
cursor: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15],
};
packages/paperclip-runner/src/drivers/codex/codex-app-server-driver-impl.ts
+1 -1
View File
@@ -710,7 +710,7 @@ export class CodexAppServerDriver implements HarnessDriver {
this.#options.transportFactory?.(context) ??
new ProcessCodexAppServerTransport({
workingDirectory,
args: createIsolatedCodexAppServerArgs(this.#options.environment, codexExecutableReadOnlyRoots(this.#options.environment ?? process.env), this.#options.instructionWorkingCopyRoot),
args: createIsolatedCodexAppServerArgs(this.#options.environment, codexExecutableReadOnlyRoots(this.#options.environment ?? process.env), this.#options.instructionWorkingCopyRoot, [], this.#options.disableWebSearch),
environment: createSanitizedCodexEnvironment(this.#options.environment),
onDiagnostic: message => this.#options.onDiagnostic?.(redactObservabilityDiagnostic(message, this.#options.environment, this.#options.observability?.otelConfig)),
processGroup: true,
packages/paperclip-runner/src/drivers/codex/codex-driver-types.ts
+2
View File
@@ -19,6 +19,8 @@ export interface CodexAppServerDriverOptions {
model?: string;
/** Per-run reasoning effort sent with each Codex turn, including resumed turns. */
reasoningEffort?: string;
/** Saved hosted-search restriction for the default process transport. */
disableWebSearch?: boolean;
approvalPolicy?: "never" | "on-request" | "untrusted";
/** Paperclip runtime instructions. Codex receives these as additive developer
* instructions; the historical option name remains compatible with callers. */
packages/paperclip-runner/src/drivers/codex/codex-security-config.ts
+2
View File
@@ -254,6 +254,7 @@ export function createIsolatedCodexAppServerArgs(
/** Server-registered run copy, never an environment/config-supplied root. */
instructionWorkingCopyRoot?: string,
protectedPaths: string[] = [],
disableWebSearch = false,
): string[] {
const explicitSource = source;
source ??= process.env;
@@ -356,6 +357,7 @@ export function createIsolatedCodexAppServerArgs(
...(externalRunnerSandbox
? ["--dangerously-bypass-approvals-and-sandbox"]
: []),
...(disableWebSearch ? ["-c", 'web_search="disabled"'] : []),
"app-server",
];
}
packages/paperclip-runner/src/drivers/opencode/mcp-bridge.test.ts
+27
View File
@@ -94,6 +94,33 @@ describe("OpenCode MCP bridge", () => {
expect(await conflict.json()).toMatchObject({ result: { isError: true, content: [{ text: "Duplicate call identity conflict." }] } });
});
it("preserves declared prefixed operation IDs through HTTP MCP and rejects undeclared aliases", async () => {
const names = ["paperclip_search_assigned_tools", "paperclip_call_assigned_tool", "search_assigned_tools"];
const handler = vi.fn(async call => ({ operation: call.tool }));
const bridge = await startOpenCodeMcpBridge({ tools: names.map(name => ({ name, inputSchema: { type: "object" } })), handler });
bridges.push(bridge);
const listed = await (await rpc(bridge, { id: "list", method: "tools/list" })).json() as { result: { tools: Array<{ name: string }> } };
expect(listed.result.tools.map(tool => tool.name)).toEqual([...names, "paperclip_finish", "paperclip_block"]);
for (const [i, name] of names.entries()) {
for (const [j, prefix] of ["", "paperclip_", "paperclip__", "paperclip."].entries()) {
const called = prefix === "paperclip_" && name === "search_assigned_tools" ? name : prefix + name;
const result = await (await rpc(bridge, { id: `call-${i}-${j}`, method: "tools/call", params: { name: called, arguments: {} } })).json();
expect(result).toMatchObject({ result: { content: [{ text: JSON.stringify({ operation: name }) }] } });
}
}
const before = handler.mock.calls.length;
expect(await (await rpc(bridge, { id: "unknown", method: "tools/call", params: { name: "paperclip_unassigned_tool", arguments: {} } })).json()).toMatchObject({ result: { isError: true } });
expect(handler).toHaveBeenCalledTimes(before);
});
it("does not authorize a shortened operation when only its prefixed name is declared", async () => {
const handler = vi.fn(async () => ({ ok: true }));
const bridge = await startOpenCodeMcpBridge({ tools: [{ name: "paperclip_search_assigned_tools", inputSchema: { type: "object" } }], handler });
bridges.push(bridge);
expect(await (await rpc(bridge, { id: "short", method: "tools/call", params: { name: "search_assigned_tools", arguments: {} } })).json()).toMatchObject({ result: { isError: true } });
expect(handler).not.toHaveBeenCalled();
});
it("normalizes all supported OpenCode MCP prefixes", () => {
expect(canonicalOpenCodeMcpToolName("paperclip_documents.read")).toBe("documents.read");
expect(canonicalOpenCodeMcpToolName("paperclip__documents.read")).toBe("documents.read");
packages/paperclip-runner/src/drivers/opencode/mcp-bridge.ts
+7 -2
View File
@@ -108,7 +108,7 @@ function normalizeTools(
const tools: OpenCodeMcpToolDefinition[] = [];
const seen = new Set<string>();
for (const raw of input) {
const name = typeof raw.name === "string" ? canonicalOpenCodeMcpToolName(raw.name) : "";
const name = typeof raw.name === "string" ? raw.name.trim() : "";
if (!name || seen.has(name)) continue;
const inputSchema = isRecord(raw.inputSchema) ? structuredClone(raw.inputSchema) : { type: "object" };
tools.push({
@@ -218,7 +218,12 @@ async function handleRequest(
}
const params = isRecord(message.params) ? message.params : {};
const rawName = typeof params.name === "string" ? params.name : "";
const tool = canonicalOpenCodeMcpToolName(rawName);
// A declared operation may itself start with paperclip_. Preserve that
// authority identity before accepting an OpenCode transport prefix alias.
const exactName = rawName.trim();
const tool = context.admittedTools.some(entry => entry.name === exactName)
? exactName
: canonicalOpenCodeMcpToolName(exactName);
if (!context.admittedTools.some((entry) => entry.name === tool)) {
writeRpc(response, id, { isError: true, content: [{ type: "text", text: "Unsupported tool." }] });
return;
packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.test.ts
+21
View File
@@ -2219,6 +2219,27 @@ describe("OpenCodeServerDriver", () => {
} finally { await session.close({ reason: "monitor default test complete" }); }
});
it("keeps assigned discovery operation IDs at the authenticated MCP and controller boundary", async () => {
await chmod(fixture, 0o755);
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-assigned-"));
const workspace = await mkdtemp(join(tmpdir(), "paperclip-opencode-workspace-"));
roots.push(root, workspace);
const calls: string[] = [];
const operation = "paperclip_search_assigned_tools";
const driver = new OpenCodeServerDriver({ model: "openrouter/deepseek/deepseek-v4.1-flash-1017", runtimeDirectory: root, command: fixture,
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "fixture-key" },
dynamicTools: [{ name: operation, inputSchema: { type: "object" } }],
dynamicToolHandler: async call => { calls.push(call.tool); if (call.tool !== operation) throw new Error("unauthorized operation"); return { tools: [] }; },
});
const session = await driver.openSession({ runId: "assigned-name", normalizedSessionId: "assigned-name", workingDirectory: workspace });
try {
await session.startTurn({ message: { role: "user", text: "Discover assigned tools." } });
const events = await collectTurnEvents(session.events());
expect(calls).toEqual([operation]);
expect(events.some(event => event.eventType === "item.completed" && (event.payload as { item?: { is_error?: boolean } }).item?.is_error)).toBe(false);
} finally { await session.close({ reason: "assigned operation identity test" }); }
});
it("normalizes a structured block result", async () => {
await chmod(fixture, 0o755);
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-driver-"));
packages/paperclip-runner/src/drivers/opencode/opencode-server-driver.ts
+3 -1
View File
@@ -1026,7 +1026,9 @@ class OpenCodeHarnessSession implements HarnessSession {
callId: string;
arguments: unknown;
}): Promise<unknown> {
const tool = canonicalOpenCodeMcpToolName(call.tool);
// The authenticated MCP bridge has resolved this against its closed
// declared catalog. A second prefix strip changes legitimate operation IDs.
const tool = call.tool;
const turnId = this.#activeTurnId;
if (turnId === null)
throw new Error("OpenCode tool call is not bound to an active turn");
packages/paperclip-runner/src/live/runner-provider-launch.test.ts
+48 -1
View File
@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { createRunnerProviderLaunch, preparedRunnerLaunchWorkingDirectory, runnerLaunchRequirementsDigest, validatePreparedRunnerLaunch,
type RunnerProviderLaunchEnvironment, type RunnerProviderLaunchRequirements } from "./runner-provider-launch.js";
runnerProviderLaunchRequirements, type RunnerProviderLaunchEnvironment, type RunnerProviderLaunchRequirements } from "./runner-provider-launch.js";
const requirements: RunnerProviderLaunchRequirements = {
schema: "paperclip.runner.launch-requirements.v1",
@@ -19,6 +19,42 @@ const environment: RunnerProviderLaunchEnvironment = {
};
describe("built-in provider launch preparation", () => {
it("binds disabled hosted search to the prepared launch and overrides execution-host arguments", () => {
const disabled = runnerProviderLaunchRequirements({ provider: "codex", disableCodexWebSearch: true },
requirements.identity, { model: requirements.model }, null, "");
expect(disabled.settings).toHaveProperty("disableCodexWebSearch", true);
expect(runnerLaunchRequirementsDigest(disabled)).not.toBe(runnerLaunchRequirementsDigest({ ...disabled, settings: {} }));
const launch = createRunnerProviderLaunch(disabled, { ...environment,
options: { codexArgs: ["-c", 'web_search="live"', "app-server"] } });
expect(launch.args).toEqual(["-c", 'web_search="live"', "-c", 'web_search="disabled"', "app-server"]);
const prepared = { configurationId: "search-policy", requirementsDigest: runnerLaunchRequirementsDigest(disabled), provider: launch };
expect(() => validatePreparedRunnerLaunch(disabled, prepared)).not.toThrow();
expect(() => validatePreparedRunnerLaunch(disabled, { ...prepared,
provider: { ...launch, args: ["app-server"] } })).toThrow("runner_prepared_launch_requirements_mismatch");
for (const leading of ["--", "--search", "--search=true"]) {
expect(() => createRunnerProviderLaunch(disabled, { ...environment,
options: { codexArgs: [leading, "app-server"] } })).toThrow("runner_codex_web_search_policy_override");
expect(() => validatePreparedRunnerLaunch(disabled, { ...prepared,
provider: { ...launch, args: [leading, ...launch.args as string[]] } })).toThrow("runner_prepared_launch_requirements_mismatch");
}
expect(() => validatePreparedRunnerLaunch(disabled, { ...prepared,
provider: { ...launch, args: ["--", ...launch.args as string[]] } })).toThrow("runner_prepared_launch_requirements_mismatch");
for (const tail of [["-c", 'web_search="live"'], ['-cweb_search="live"'], ["--search"], ['--config=web_search="live"'], ["--"]]) {
expect(() => createRunnerProviderLaunch(disabled, { ...environment,
options: { codexArgs: ["app-server", ...tail] } })).toThrow("runner_codex_web_search_policy_override");
expect(() => validatePreparedRunnerLaunch(disabled, { ...prepared,
provider: { ...launch, args: [...launch.args as string[], ...tail] } })).toThrow("runner_prepared_launch_requirements_mismatch");
}
});
it.each([undefined, false])("preserves default hosted search when disable is %s", disableCodexWebSearch => {
const request = runnerProviderLaunchRequirements({ provider: "codex", disableCodexWebSearch },
requirements.identity, { model: requirements.model }, null, "");
const launch = createRunnerProviderLaunch(request, environment);
expect(launch.args).toEqual(["app-server"]);
});
it("keeps Core's selection and policy when the execution environment changes", () => {
const before = structuredClone(requirements);
const local = createRunnerProviderLaunch(requirements, environment);
@@ -30,6 +66,17 @@ describe("built-in provider launch preparation", () => {
expect(requirements).toEqual(before);
});
it("preserves exact Copilot requirements and launch descriptors when Codex search disabling is absent or false", () => {
const selected = { provider: "acpx" as const, acpxAgent: "copilot" as const, acpxPermissionMode: "deny-all" as const };
const params = { model: "gpt-5.6-luna", approvalPolicy: "never", permissions: "paperclip-runner-workspace-read-only" };
const baseline = runnerProviderLaunchRequirements(selected, requirements.identity, params, null, "Copilot policy");
const scopedProbe = runnerProviderLaunchRequirements({ ...selected, disableCodexWebSearch: false },
requirements.identity, params, null, "Copilot policy");
expect(scopedProbe).toEqual(baseline);
expect(runnerLaunchRequirementsDigest(scopedProbe)).toBe(runnerLaunchRequirementsDigest(baseline));
expect(createRunnerProviderLaunch(scopedProbe, environment)).toEqual(createRunnerProviderLaunch(baseline, environment));
});
it("preserves the ACPX profile, run identity, and permission policy", () => {
const launch = createRunnerProviderLaunch({ ...requirements, provider: "acpx",
model: "claude-selected", settings: { acpxAgent: "claude", acpxPermissionMode: "deny-all" } }, environment);
packages/paperclip-runner/src/live/runner-provider-launch.ts
+30 -6
View File
@@ -23,7 +23,7 @@ export interface RunnerProviderLaunchRequirements {
includeCollaborationModeInstructions: boolean;
settings: Pick<CapabilityRunnerdCodexTransportOptions,
"acpxAgent" | "acpxMode" | "acpxPermissionMode" | "acpxPermissionModePinned" |
"piThinkingLevel" | "claudeEffort" | "managedProfile" | "agentCoreProfile" | "opencodePermissionMode">;
"piThinkingLevel" | "claudeEffort" | "managedProfile" | "agentCoreProfile" | "opencodePermissionMode" | "disableCodexWebSearch">;
instructions: string;
runtimeContext: NativeRuntimeContextSnapshot | null;
}
@@ -59,6 +59,7 @@ export function runnerProviderLaunchRequirements(
piThinkingLevel: options.piThinkingLevel, claudeEffort: options.claudeEffort,
managedProfile: options.managedProfile, agentCoreProfile: options.agentCoreProfile,
opencodePermissionMode: options.opencodePermissionMode,
...(options.disableCodexWebSearch === true ? { disableCodexWebSearch: true } : {}),
},
approvalPolicy: params.approvalPolicy === "on-request" || params.approvalPolicy === "untrusted"
? params.approvalPolicy : "never",
@@ -109,6 +110,14 @@ export function validatePreparedRunnerLaunch(
if (descriptor.kind !== requirements.provider || descriptor.model !== requirements.model) reject();
if (requirements.provider === "codex" && (descriptor.approvalPolicy !== requirements.approvalPolicy
|| Boolean(descriptor.readOnly) !== requirements.readOnly)) reject();
if (requirements.provider === "codex" && requirements.settings.disableCodexWebSearch === true) {
if (!Array.isArray(descriptor.args) || descriptor.args.some(arg => typeof arg !== "string")) reject();
const args = descriptor.args as string[];
const commandIndex = args.indexOf("app-server");
const boundary = commandIndex < 0 ? args.length : commandIndex;
if (args.some(arg => arg === "--" || arg === "--search" || arg.startsWith("--search=")) || (commandIndex >= 0 && commandIndex !== args.length - 1)
|| args[boundary - 2] !== "-c" || args[boundary - 1] !== 'web_search="disabled"') reject();
}
if (requirements.provider === "acpx") {
const policy = descriptor.providerPolicy as Record<string, unknown> | undefined;
if (descriptor.agent !== (requirements.settings.acpxAgent ?? "codex")
@@ -303,8 +312,9 @@ export function createRunnerProviderLaunch(
args:
provider === "opencode"
? [opencodeProxyPath]
: (input.options.codexArgs ??
createRunnerdCodexAppServerArgs({
: createRunnerdCodexAppServerArgs({
args: input.options.codexArgs,
disableWebSearch: settings.disableCodexWebSearch,
environment: input.options.environment,
codexHome,
workingDirectory: input.cwd,
@@ -326,7 +336,7 @@ export function createRunnerProviderLaunch(
]
: []),
],
})),
}),
cwd: input.cwd,
model: requirements.model,
approvalPolicy:
@@ -408,12 +418,14 @@ export function createRunnerdCodexAppServerArgs(input: {
readOnlyRoots?: string[];
instructionWorkingCopyRoot?: string;
disableProjectInstructions?: boolean;
disableWebSearch?: boolean;
args?: string[];
}): string[] {
// The filesystem policy denies HOME and CODEX_HOME to keep credentials and
// runner state outside provider reach. Always bind those names to the actual
// isolated runner home; a stale controller environment must never cause the
// execution workspace itself to become an explicit deny root.
const args = [
const args = input.args ? [...input.args] : [
...(input.disableProjectInstructions ? ["-c", "project_doc_max_bytes=0"] : []),
...createIsolatedCodexAppServerArgs(
{
@@ -426,8 +438,20 @@ export function createRunnerdCodexAppServerArgs(input: {
usesPreconfiguredHome(input.environment) && input.runtimeStateRoot ? [input.runtimeStateRoot] : [],
),
];
if (usesPreconfiguredHome(input.environment) && input.workingDirectory) {
if (!input.args && usesPreconfiguredHome(input.environment) && input.workingDirectory) {
args.unshift("-c", `projects={${JSON.stringify(input.workingDirectory)}={trust_level="trusted"}}`);
}
if (input.disableWebSearch) {
// Apply Core's restriction after host overrides. Keep the override before
// the Codex subcommand and after any earlier provider-home configuration.
const commandIndex = args.indexOf("app-server");
// Restricted launches use the owned app-server argument shape. Refuse
// trailing subcommand flags and option terminators rather than guessing
// whether another CLI spelling could override or neutralize this policy.
if (args.some(arg => arg === "--" || arg === "--search" || arg.startsWith("--search=")) || (commandIndex >= 0 && commandIndex !== args.length - 1)) {
throw new Error("runner_codex_web_search_policy_override");
}
args.splice(commandIndex < 0 ? args.length : commandIndex, 0, "-c", 'web_search="disabled"');
}
return args;
}
packages/paperclip-runner/src/live/runnerd-acpx-permission-forwarding.test.ts
+1
View File
@@ -27,6 +27,7 @@ vi.mock("../control-plane/durable-prp-control-plane.js", async (importOriginal)
}
async start() {}
async stop() {}
async drainPendingConnectionProcessing() {}
disconnectActiveRunner() {}
persistRunAttachTemplate() {}
issueBootstrapTicket() { return {}; }
packages/paperclip-runner/src/live/runnerd-acpx-permission-ordering.test.ts
+1
View File
@@ -25,6 +25,7 @@ vi.mock("../control-plane/durable-prp-control-plane.js", async (importOriginal)
}
async start() {}
async stop() {}
async drainPendingConnectionProcessing() {}
disconnectActiveRunner() {}
persistRunAttachTemplate() {}
issueBootstrapTicket() { return {}; }
packages/paperclip-runner/src/live/runnerd-codex-transport.test.ts
+26 -1
View File
@@ -3343,6 +3343,29 @@ const fakeCodex = resolve(
"../../runner/target/debug/fake-codex-app-server",
);
it("keeps the Copilot authentication launch requirements free of Codex search policy", async () => {
const root = await mkdtemp(join(tmpdir(), "runnerd-copilot-probe-policy-"));
let captured: RunnerProviderLaunchRequirements | undefined;
const launch = vi.fn(() => { throw new Error("provider process must not start"); });
try {
await expect(probeNativeRunnerEnvironment({ runtimeDirectory: root, provider: "acpx", model: "gpt-5.6-luna",
environment: {}, timeoutMs: 1_000,
transportOptions: { acpxAgent: "copilot", acpxCandidateProfile: "copilot", disableCodexWebSearch: true,
runnerArtifact: { version: "offline-fixture", digest: `sha256:${"a".repeat(64)}` },
runnerProcessLauncher: launch,
controlPlaneRegistration: async () => { throw new Error("registration must not start"); },
prepareLaunch: async requirements => {
captured = requirements;
throw new Error("offline-stop-at-Copilot-preparation");
},
},
})).rejects.toThrow("offline-stop-at-Copilot-preparation");
expect(captured).toBeDefined();
expect(captured?.settings).not.toHaveProperty("disableCodexWebSearch");
expect(launch).not.toHaveBeenCalled();
} finally { await rm(root, { recursive: true, force: true }); }
});
it.each(["stuck", "late-release", "late-release-error", "stuck-release", "cooperative-release", "launch-failure", "launch-failure-sandbox", "launch-failure-unsandboxed"] as const)(
"bounds setup registration teardown and retains unconfirmed state (%s)",
async mode => {
@@ -3381,7 +3404,7 @@ it.each(["stuck", "late-release", "late-release-error", "stuck-release", "cooper
...(mode === "launch-failure-sandbox" ? { externallySandboxed: true } : mode === "launch-failure-unsandboxed" ? { externallySandboxed: false } : {}),
runnerBinary: resolve(import.meta.dirname, "../../package.json"),
codexCommand: resolve(import.meta.dirname, "../../package.json"),
sourceCodexHome: "", runnerProcessLauncher: launch,
sourceCodexHome: "", runnerProcessLauncher: launch, disableCodexWebSearch: false,
controlPlaneRegistration: async (_authority, _identity, signal) => {
template = structuredClone(_authority.store.state.runAttachTemplate!);
registrationSignal = signal;
@@ -3405,6 +3428,8 @@ it.each(["stuck", "late-release", "late-release-error", "stuck-release", "cooper
expect(template).toMatchObject({ provider: { externallySandboxed: mode === "launch-failure-sandbox", readOnly: true, approvalPolicy: "never", collaborationMode: "plan" } });
const args = (template!.provider as Record<string, unknown>).args as string[];
expect(args).toContain("project_doc_max_bytes=0");
expect(args).toContain('web_search="disabled"');
expect(args.indexOf('web_search="disabled"')).toBeLessThan(args.indexOf("app-server"));
expect(args[args.indexOf("project_doc_max_bytes=0") - 1]).toBe("-c");
expect(args.indexOf("project_doc_max_bytes=0")).toBeLessThan(args.indexOf("app-server"));
expect(args).not.toContain("--dangerously-bypass-approvals-and-sandbox");
packages/paperclip-runner/src/live/runnerd-codex-transport.ts
+47 -9
View File
@@ -726,6 +726,7 @@ async function releaseRunnerProcessOwnership(input: {
((settlement: "settled" | "unsettled") => Promise<void> | void) | null;
forceKill: () => Promise<void> | void;
release: (() => Promise<void> | void) | null;
retireController?: () => Promise<void>;
}): Promise<void> {
let releaseFailure: unknown;
try {
@@ -735,6 +736,9 @@ async function releaseRunnerProcessOwnership(input: {
}
let checkpointFailure: unknown;
try {
// External ingress must stop reconnecting before admitted journal writes
// can be joined. A failed retirement cannot authorize a new checkpoint.
await input.retireController?.();
if (input.checkpoint !== null) {
// Release only the authenticated control route first. In provider-ingress
// mode this stops its reconnect loop; it does not release the sandbox
@@ -1287,6 +1291,8 @@ export interface CapabilityRunnerdCodexTransportOptions extends RunnerExecutionD
codexCommand?: string;
codexArgs?: string[];
disableCodexProjectInstructions?: boolean;
/** Restrict hosted search without changing the provider default for other runs. */
disableCodexWebSearch?: boolean;
/** Controller-visible Codex home used only to seed the isolated runner home. */
sourceCodexHome?: string | null;
opencodeCommand?: string;
@@ -3567,6 +3573,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
readonly #closedSignal: Promise<void>;
#resolveClosedSignal!: () => void;
#closePromise: Promise<void> | null = null;
#detachPromise: Promise<void> | null = null;
#startupOperation: Promise<unknown> | null = null;
#attachmentOperation: Promise<void> | null = null;
#startupDrainTimedOut = false;
@@ -4464,7 +4471,8 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
close(reason?: string): Promise<void> {
// Detachment relinquishes process ownership. A late execution finalizer
// must not suspend or signal the runner now owned by the next controller.
if (this.#controllerDetachedForRestart) return Promise.resolve();
if (this.#controllerDetachedForRestart)
return this.#detachPromise ?? Promise.resolve();
if (reason) {
this.#diagnostic(
`runner transport close requested: ${reason.replaceAll(/[\r\n]/g, " ").slice(0, 1_000)}`,
@@ -4474,8 +4482,33 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
return this.#closePromise;
}
async detachControllerForRestart(): Promise<void> {
detachControllerForRestart(): Promise<void> {
this.#detachPromise ??= this.#detachControllerOnce();
return this.#detachPromise;
}
async #stopAndDrainAuthority(deadline: number): Promise<void> {
const core = this.#core;
if (!core) return;
await core.stop();
let timer: ReturnType<typeof setTimeout> | undefined;
try {
await Promise.race([
core.drainPendingConnectionProcessing(),
new Promise<never>((_resolve, reject) => {
timer = setTimeout(() => reject(new Error(
"Controller journal drain timed out; preserve the session before replacing its writer.",
)), Math.max(0, deadline - Date.now()));
}),
]);
} finally {
clearTimeout(timer);
}
}
async #detachControllerOnce(): Promise<void> {
if (this.#closed) return;
const deadline = Date.now() + runnerCloseGraceMs(this.options);
this.#controllerDetachedForRestart = true;
this.#closed = true;
this.#resolveClosedSignal();
@@ -4493,13 +4526,15 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
`controller route release failed during restart detach: ${String(error)}`,
);
});
await this.#core?.stop().catch((error: unknown) => {
this.#diagnostic(
`controller authority stop failed during restart detach: ${String(error)}`,
);
});
// A successor must not share this journal with an admitted async write.
// A timeout rejects detachment, including repeated calls, instead of
// reporting that the old writer has retired.
try {
await this.#stopAndDrainAuthority(deadline);
} finally {
this.#queue.close();
}
this.#handle = null;
this.#queue.close();
this.#diagnostic(
"controller authority detached for restart; durable runner left alive",
);
@@ -4723,6 +4758,7 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
this.#evidence.runnerSignal = result.signal as NodeJS.Signals | null;
},
release: this.#controlPlaneRelease,
retireController: () => this.#stopAndDrainAuthority(closeDeadline),
});
} finally {
await this.#core?.stop();
@@ -5529,8 +5565,9 @@ class DurablePrpCodexTransport implements CodexAppServerTransport {
// Keep the durable provider profile and thread identity unchanged.
const recordedCommand = record(runAttachTemplate.provider).command;
runAttachTemplate.runtimeLaunchArgs =
this.options.codexArgs ??
createRunnerdCodexAppServerArgs({
args: this.options.codexArgs,
disableWebSearch: this.options.disableCodexWebSearch,
environment: this.options.environment,
codexHome,
workingDirectory: this.options.resumeWorkingDirectory,
@@ -7287,6 +7324,7 @@ export async function probeNativeRunnerEnvironment(options: {
environment,
externallySandboxed: options.transportOptions?.externallySandboxed === true,
disableCodexProjectInstructions: true,
disableCodexWebSearch: options.provider === "codex",
closeGraceMs: 10_000,
lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null },
});
packages/paperclip-runner/src/live/runnerd-detach-durability.test.ts
+150
View File
@@ -0,0 +1,150 @@
import { spawn, type ChildProcess } from "node:child_process";
import fs from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { expect, it, vi } from "vitest";
import { DurablePrpControlPlane } from "../control-plane/durable-prp-control-plane.js";
import { createCapabilityRunnerdCodexTransport, defaultCapabilityRunnerdBinary, readRunnerdArtifactBinding, runnerdRecoveryInternals } from "./runnerd-codex-transport.js";
it.each(["resolved", "rejected"] as const)("keeps the checkpoint behind controller retirement (%s)", async outcome => {
const steps: string[] = [];
let release!: () => void;
let reject!: (error: Error) => void;
const barrier = new Promise<void>((resolveDrain, rejectDrain) => { release = resolveDrain; reject = rejectDrain; });
const operation = runnerdRecoveryInternals.releaseRunnerProcessOwnership({
runnerSettled: true,
release: () => { steps.push("route"); },
retireController: async () => { steps.push("drain"); await barrier; },
checkpoint: () => { steps.push("checkpoint"); },
forceKill: () => { steps.push("containment"); },
});
await new Promise<void>(resolveTurn => setImmediate(resolveTurn));
expect(steps).toEqual(["route", "drain"]);
if (outcome === "rejected") {
const failed = expect(operation).rejects.toThrow("journal unavailable");
reject(new Error("journal unavailable"));
await failed;
expect(steps).toEqual(["route", "drain", "containment"]);
} else {
release();
await operation;
expect(steps).toEqual(["route", "drain", "checkpoint", "containment"]);
}
});
it.each(["drain", "timeout", "external_close"] as const)("joins an output flush before controller retirement permits a successor writer (%s)", async mode => {
const root = await fs.mkdtemp(join(tmpdir(), "runnerd-detach-write-"));
await fs.mkdir(join(root, "diagnostics"), { mode: 0o700 });
const binary = defaultCapabilityRunnerdBinary();
const artifact = readRunnerdArtifactBinding(binary);
let core!: DurablePrpControlPlane;
const start = DurablePrpControlPlane.prototype.start;
const startSpy = vi.spyOn(DurablePrpControlPlane.prototype, "start").mockImplementation(function (port) {
core = this;
return start.call(this, port);
});
let release!: () => void;
const barrier = new Promise<void>(resolveWrite => { release = resolveWrite; });
let staged = false;
const originalOpen = fs.open;
const openSpy = vi.spyOn(fs, "open").mockImplementation(async (...args) => {
const file = await originalOpen(...args);
if (!staged && String(args[0]).includes(".control-plane-state.json.")) {
const sync = file.sync.bind(file);
file.sync = async () => { staged = true; await barrier; await sync(); };
}
return file;
});
let child: ChildProcess | undefined;
let exit: Promise<void> | undefined;
const closeSteps: string[] = [];
let stopSpy: ReturnType<typeof vi.spyOn> | undefined;
const bundle = createCapabilityRunnerdCodexTransport({
runnerBinary: binary,
codexCommand: resolve(import.meta.dirname, "../../runner/target/debug/fake-codex-app-server"),
codexArgs: ["--state-file", join(root, "provider-state.json"), "--split-event-burst", "--split-event-prefix-count", "1"],
stateDirectory: root,
closeGraceMs: mode === "timeout" ? 200 : 1_500,
...(mode === "external_close" ? { controlPlaneRegistration: async (authority: DurablePrpControlPlane) => {
await authority.start();
const stop = authority.stop.bind(authority);
stopSpy = vi.spyOn(authority, "stop").mockImplementation(async () => {
closeSteps.push("stop");
release();
await stop();
});
return { connectUrl: authority.connectUrl, release: () => { closeSteps.push("release"); } };
} } : {}),
runnerProcessLauncher: spec => {
child = spawn(spec.command, [...spec.args], { cwd: spec.cwd, env: spec.environment, stdio: ["ignore", "pipe", "pipe"] });
const owned = child;
let stderr = "";
owned.stderr?.on("data", chunk => { stderr += String(chunk); });
exit = new Promise(resolveExit => owned.once("exit", () => resolveExit()));
const completion = new Promise<{ code: number | null; signal: string | null; stdout: string; stderr: string }>((resolveExit, reject) => {
owned.once("error", reject);
owned.once("exit", (code, signal) => resolveExit({ code, signal, stdout: "", stderr }));
});
return { child: owned, completion };
},
});
bundle.transport.setServerRequestHandler(async () => ({ success: true, contentItems: [] }));
let detached: Promise<void> | undefined;
try {
await bundle.transport.request("thread/start", { cwd: root, dynamicTools: [{
name: "get_task_context", description: "Read fixture context.",
inputSchema: { type: "object", properties: {}, additionalProperties: false },
}] });
await bundle.transport.request("turn/start", { input: [{ type: "text", text: "Exercise a held output flush." }] });
await vi.waitFor(() => expect(staged).toBe(true), { timeout: 5_000 });
if (mode === "external_close") {
await bundle.transport.close().catch(error => {
expect(error.message).toMatch(/journal drain timed out|did not durably suspend/);
});
expect(closeSteps.indexOf("release")).toBeGreaterThanOrEqual(0);
expect(closeSteps.indexOf("release")).toBeLessThan(closeSteps.indexOf("stop"));
return;
}
let settled = false;
detached = bundle.detachControllerForRestart().then(() => { settled = true; });
await vi.waitFor(() => expect(core.activeRunnerConnectionCount()).toBe(0));
await new Promise<void>(resolveTurn => setImmediate(resolveTurn));
expect(settled).toBe(false);
if (mode === "timeout") {
await expect(detached).rejects.toThrow("journal drain timed out");
await expect(bundle.detachControllerForRestart()).rejects.toThrow("journal drain timed out");
await expect(bundle.transport.close()).rejects.toThrow("journal drain timed out");
release();
await core.drainPendingConnectionProcessing();
} else {
release();
await detached;
}
const successor = new DurablePrpControlPlane({ stateDirectory: join(root, "control-plane"),
identity: core.store.state.identity, expectedRunnerVersion: artifact.version, expectedRunnerDigest: artifact.digest });
successor.queueCommand("session.snapshot", {}, "successor_command");
await new Promise<void>(resolveTurn => setImmediate(resolveTurn));
const persisted = JSON.parse(await fs.readFile(successor.store.path, "utf8"));
expect(persisted.commands.some((command: { commandId: string }) => command.commandId === "successor_command")).toBe(true);
} catch (error) {
const diagnostic = await fs.readFile(join(root, "diagnostics", "runnerd.stderr.log"), "utf8").catch(() => "");
throw new Error(`${String(error)}\n${diagnostic}`, { cause: error });
} finally {
release();
await detached?.catch(() => undefined);
openSpy.mockRestore();
startSpy.mockRestore();
await bundle.detachControllerForRestart().catch(() => undefined);
const providerPid = bundle.evidence().codexPid;
if (providerPid) {
try { process.kill(providerPid, "SIGKILL"); } catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ESRCH") throw error;
}
}
child?.kill("SIGKILL");
await exit;
await core?.drainPendingConnectionProcessing();
stopSpy?.mockRestore();
await fs.rm(root, { recursive: true, force: true });
}
}, 15_000);
packages/paperclip-runner/src/live/runnerd-opencode-assigned-tools.test.ts
+65
View File
@@ -0,0 +1,65 @@
import { createHash } from "node:crypto";
import { existsSync, readFileSync } from "node:fs";
import { chmod, copyFile, mkdir, mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { expect, it } from "vitest";
import { codexSemanticToolSpecs, dynamicToolResponse } from "../drivers/codex/codex-driver-values.js";
import { createRunnerdCodexTransport, defaultCapabilityRunnerdBinary } from "./runnerd-codex-transport.js";
const runner = defaultCapabilityRunnerdBinary();
const proxy = resolve("dist/cli/opencode-app-server-proxy.cjs");
const digest = (path: string) => `sha256:${createHash("sha256").update(readFileSync(path)).digest("hex")}`;
it.skipIf(process.platform === "win32" || !existsSync(runner) || !existsSync(proxy))(
"retains assigned discovery authority across Rust, the proxy, and HTTP MCP",
async () => {
const root = await mkdtemp(join(tmpdir(), "runnerd-opencode-assigned-"));
const workspace = join(root, "workspace");
const executable = join(root, "fake-opencode-server.mjs");
await mkdir(workspace);
await copyFile(resolve("test/fixtures/fake-opencode-server.mjs"), executable);
await chmod(executable, 0o755);
const operation = "paperclip_search_assigned_tools";
const seen: string[] = [];
const { transport } = createRunnerdCodexTransport({
provider: "opencode", runnerBinary: runner, stateDirectory: join(root, "state"), sourceCodexHome: "",
providerNodeCommand: process.execPath, providerNodeCommandSha256: digest(process.execPath),
opencodeCommand: executable, opencodeCommandSha256: digest(executable),
opencodeProxyPath: proxy, opencodeProxySha256: digest(proxy),
environment: { PATH: process.env.PATH, OPENROUTER_API_KEY: "synthetic-fixture-key" },
});
transport.setServerRequestHandler(async request => {
expect(request.method).toBe("item/tool/call");
const tool = request.params.tool;
expect([operation, "paperclip_finish"]).toContain(tool);
seen.push(String(tool));
return dynamicToolResponse(tool === operation ? { tools: [] } : "Accepted completion.");
});
let timer: ReturnType<typeof setTimeout> | undefined;
try {
await transport.request("thread/start", {
cwd: workspace, model: "openrouter/deepseek/deepseek-v4.1-flash-1017",
completionContract: { revision: "assigned-tools-v1", criterionIds: ["objective"] },
dynamicTools: [{ name: operation, description: "Search assigned synthetic tools.", inputSchema: { type: "object" } }, ...codexSemanticToolSpecs()],
});
const completed = (async () => {
for await (const notification of transport.notifications()) {
if (notification.method === "turn/completed") return;
if (notification.method === "turn/failed") throw new Error("Native OpenCode turn failed");
}
throw new Error("Native OpenCode notifications ended before completion");
})();
await transport.request("turn/start", { input: [{ type: "text", text: "Discover assigned synthetic tools." }] });
await Promise.race([completed, new Promise<never>((_, reject) => {
timer = setTimeout(() => reject(new Error("Native assigned-tool smoke timed out")), 20_000);
})]);
expect(seen).toEqual([operation, "paperclip_finish"]);
} finally {
if (timer) clearTimeout(timer);
await transport.close();
await rm(root, { recursive: true, force: true });
}
},
30_000,
);
packages/paperclip-runner/src/live/runnerd-welcome.integration.test.ts
+62
View File
@@ -0,0 +1,62 @@
import { mkdtemp, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { expect, it, vi } from "vitest";
import { createCapabilityRunnerdCodexTransport, defaultCapabilityRunnerdBinary } from "./runnerd-codex-transport.js";
it("starts one provider after delayed bootstrap admission and retains it for a second turn", async () => {
const root = await mkdtemp(join(tmpdir(), "runnerd-delayed-welcome-"));
const calls = join(root, "provider-calls.log");
let admissionDelayed = false;
let admissionTimer: ReturnType<typeof setTimeout> | undefined;
const bundle = createCapabilityRunnerdCodexTransport({
runnerBinary: defaultCapabilityRunnerdBinary(),
codexCommand: resolve(import.meta.dirname, "../../runner/target/debug/fake-codex-app-server"),
codexArgs: ["--state-file", join(root, "provider.json"), "--call-log", calls, "--record-process-start", "--durable-turn-ids"],
stateDirectory: join(root, "state"),
controlPlaneRegistration: async authority => {
const attach = authority.attachWireConnection.bind(authority);
vi.spyOn(authority, "attachWireConnection").mockImplementation(wire => attach({
sendJson: value => wire.sendJson(value),
close: code => wire.close(code),
onClose: listener => wire.onClose(listener),
onJson: listener => wire.onJson(value => {
if (!admissionDelayed && (value as { kind?: string }).kind === "auth_response") {
admissionDelayed = true;
admissionTimer = setTimeout(() => listener(value), 3_000);
} else listener(value);
}),
}));
await authority.start();
return { connectUrl: authority.connectUrl, release: () => authority.stop() };
},
});
const terminalTurns: unknown[] = [];
try {
const opened = await bundle.transport.request("thread/start", { cwd: root, dynamicTools: [] });
expect(admissionDelayed).toBe(true);
for (const text of ["Complete the first turn.", "Continue the same session."]) {
await bundle.transport.request("turn/start", { input: [{ type: "text", text }] });
for await (const event of bundle.transport.notifications()) {
if (event.method === "turn/completed") {
expect(event.params).toMatchObject({ threadId: opened.thread.id });
terminalTurns.push(event.params);
break;
}
}
}
await bundle.transport.close();
const methods = (await readFile(calls, "utf8")).trim().split("\n");
expect(methods.filter(method => method === "process-start")).toHaveLength(1);
expect(methods.filter(method => method === "thread/start")).toHaveLength(1);
expect(methods.filter(method => method === "turn/start")).toHaveLength(2);
expect(terminalTurns).toHaveLength(2);
const runner = JSON.parse(await readFile(join(root, "state/runner/runner-state.json"), "utf8"));
expect(runner.lifecycle).toBe("suspended");
expect(runner.outbox).toEqual([]);
} finally {
if (admissionTimer) clearTimeout(admissionTimer);
await bundle.transport.close();
await rm(root, { recursive: true, force: true });
}
}, 30_000);
packages/paperclip-runner/src/testing.ts
+1
View File
@@ -9,6 +9,7 @@ export * from "./index.js";
export * from "./conformance/control-plane-port.js";
export * from "./conformance/capability-semantic-conformance.js";
export * from "./conformance/harness-driver.js";
export { startOpenCodeMcpBridge } from "./drivers/opencode/mcp-bridge.js";
export * from "./conformance/semantic-conformance.js";
export * from "./mock-core/deterministic-harness-driver.js";
export * from "./mock-core/mock-control-plane-adapter.js";
packages/paperclip-runner/test/fixtures/copilot-profile-v60-identity.json
Vendored
+85
View File
@@ -0,0 +1,85 @@
{
"commandDigest": "sha256:ca3c2b38ed5f6e25cf1c96c1f73781a86cc90c1c0add93f64716c66f28b3f3aa",
"declaration": {
"schema": "paperclip.acpx_profile_declaration.v1",
"agent": "copilot",
"agentProfileVersion": 60,
"acpxVersion": "0.13.1",
"agentServerVersion": "1.0.88",
"protocolPolicyRevision": "copilot-agent-manual-v1",
"acpxPatchSha256": "00d6af17216cb0cad902b48633eb19bf0ec6b05837a984cb24661fca593f09c1",
"policySha256": "a4be69be43db84052560420a3ca546a665c9165573c24d543eb33d144ba138a3",
"distributionSourceSha256": "c0a65e247fc47b263bd369a3d5ab505c4b323d4e6493be8a9ecbeb0e9dbb9107",
"agentFilesBinding": "registered-runtime-context-v1",
"systemInstructionDelivery": "COPILOT_HOME/copilot-instructions.md:replace-under-lifetime-lease-before-launch:v1",
"sharedRuntimeContract": "paperclip.acpx-runtime-contract.v1",
"innerDistributionSourceSha256": "9bb08347eed01f1ab44e065c8b06ead9be3a25973d6e6329dc4c0a099096279a",
"upstreamAppSha256": "7b48282a19b5b0814a0c96ad5e3a125173d792cc55f9525b2effea962f6063c0",
"patchedAppSha256": "61f8eebe0c30cc03b311c6818b2acb158a6d16ca58c8bfac5ebd42db095c2679",
"messageIdentityContract": "copilot-native-message-id-v1",
"ownedDistributionDelivery": "COPILOT_CLI_DIST_DIR:lease-owned-guarded-inner-distribution:v1",
"permissionContextContract": "copilot-edit-permission-context-v1",
"permissionContextSourceSha256": "de2d38d702b3c574161af0c0c04790fb6f6347916eb3916ce7a131a8c229121b",
"permissionAdapterSourceSha256": "ee5929ed2ee647ad0369465dee9d51a00b2dd28e1ff735b43e969ddc9c164668",
"permissionLocationsSourceSha256": "f39a81da395b6e23aa2a1a45572a28846e5e81bfd3d764d9a456f74a79c752e9",
"permissionRedactionSourceSha256": "ad6770c69269087951790e2e87e22ac088e3d432769adf17ebc865ed512e6c4f",
"permissionClassifierSourceSha256": "1161eabd82e26133e3b3db3ab567f65d52808b03f176e93497e02c8330349fb4",
"semanticToolReceiptContract": "paperclip.semantic_tool_receipt.v2",
"semanticReceiptSourceSha256": "b2795726b2234a663fec1a6ca11f0551ae69882fe9e82aa39686e952d036e791",
"semanticBridgeSourceSha256": "8764b8c9032fdc7d1415b01e75a9fc57c550e7a5e11da4bed7d9100082872ae5",
"toolEvidenceSourceSha256": "6eef5039afc75bb851eaf9cd82a6f099e4bd398ca2deb59dd66f5b2a81a7a706",
"semanticNormalizedInputContract": "validated-forwarded-input-commit-v1",
"semanticSidecarSourceSha256": "17361d1782c0dcebb2b7972635f45c890f2a4081f8bc0ef2d2276550e0b91629",
"semanticDirectDriverSourceSha256": "ff572638f2cb5d9f8249ff8e2f412311c1e3af209ef0bf7e5ea9f10a21d91103",
"semanticValidationSourceSha256": "02fbb92681c78c5de5d2f1aae25aabc25a0e5df6b0ff4baccac652192a8da975",
"semanticNormalizationSourceSha256": "8b628b82e8eacc91cdcc6fdca2210ba4cf282f01e9c6f74e95ff271655b5047e",
"semanticCompletionContractSourceSha256": "3343df6362252d44d12b079ff78c8f6539d1544f3ce5ec36ff36db70ab0fac31",
"semanticValidatorsSourceSha256": "f22d9caf2035b600abc1a6625ae23ca97f1409d7e29ed5b7264bb8f9784c7dc9",
"semanticSchemaBundleSourceSha256": "016391a087cc217aafed4ba1f4efafc6e4881a7525952dce824b7f4d24530603",
"semanticSidecarProtocolSourceSha256": "03c0c95b900f9dc18f29f6be27b8c79bc0bcab29780190d46a74e95328081236",
"semanticSidecarReceiptCommitContract": "correlated-tool-resolve-v1",
"displayEventSourceSha256": "d72f9a6035c247bc6a42c067c05753076c54322c4b58984e93922f3325dc6877",
"displayProjectionSourceSha256": "75952606b26dd93bbc582c1f0a4424487377272ea745fb3a27ceae031fd5dc7e",
"configuredEnvironmentSourceSha256": "5c9008d121979a3a8987e6b8a5752172eafdc4a2b5d725a73f3bbb8fb83be8ff",
"configuredEnvironmentRustSourceSha256": "85c87fae62b8a9377e4f0a4c00302c7519fae4397b1c48253f8f2b7ba7faf994",
"profileActivitySourceSha256": "4fdb6b0f9125fd8372ba6f27945fa5c8a3d3b2ec891df544092e3dc7a75d5282",
"profileExtensionsSourceSha256": "a80ddc95eb4505095bfdbbf85c758199949ab20be9c84b34c77f762eb7180571",
"profileInstallationSourceSha256": "3e61c4cdd4b78e38fcb16eaadae4f1fd65ce3c7127184ea7a80948c8e5d77f88",
"attachmentPolicySourceSha256": "4f7638b7cc64e18f6a4ba33fd46de2b3d49225e32aeab41f5c695b268fa4f8e2",
"singleReadEvidenceSourceSha256": "4069393405089bde613cecec04f5123c426594811da4c668bd96f0e7e283550f",
"nativeSessionRuntimeSourceSha256": "1fe6fa9e6b96d84d70432b85a57c473f833606d4a521563d5022b4d3c41576ce",
"nativeRuntimeContextSourceSha256": "71607f80dbb8dc5cc6c123d0b8f3995d6c5407277695a8c82cbedc90fb8f4842",
"nativeRuntimePromptsSourceSha256": "103b83066da72434602e9e461e6278dfb40bb993d2f3b49b2e93df81bbb08b72",
"nativeSessionContractSourceSha256": "22a8c96ce41ba4f03e976fc37f0cc7c5d05cee99fa789941da31ee1dab440b90",
"harnessSessionContractSourceSha256": "03f4cd7f76f34f6d82e009b66aa51f18f6e23dc235b660778060e35b6ea07412",
"harnessBackendSourceSha256": "dc827e281ae6731545b166ee4d709a4f67f594a556dcbe63c70fe8f1aca26249",
"harnessSessionSourceSha256": "81361733e86b56dd5a66415aefc6f72e69be24acc8d3417370154ebb7fef6e94",
"transportContractSourceSha256": "03fca93d36a89b0cc0917335e293ad097fdf04356d21b307827bb0ea4c13f523",
"nativeBackendSourceSha256": "e7b12d4418c7a71f5f2474b046697d3009eae3613b284bec381df5e9965a1c37",
"nativeBackendFactorySourceSha256": "7e431c926b76d53c4c08e3a4105353dcd34bd940f6588a1a68f79631f6feeb44",
"nativeInstructionsSourceSha256": "9173ebccf09cb56a781d2bf5c7eb27fdafb13d3450d99342c6aff86b42d6b43a",
"providerLaunchSourceSha256": "fa8db1c42c378898287c141d8cea9069166981e369c9b9c51422cb6177de5a2c",
"launchPreparationStoreSourceSha256": "ff6025568bbc0aa6ce545772d73dfca2d3da226740bd3a7b386b069e76cff296",
"runtimeRequestBridgeSourceSha256": "8ddd5d26854f4a153a9559ea8105aef95a3ec94f03209e9a797c95932c162e2a",
"runtimeRequestFacadeSourceSha256": "d0ed28e0c5ddad34db8f437e704c8a755eb0b47f27e08e44185f66d1d2205295",
"runtimeRequestNotificationSourceSha256": "6196c6175f11c69295dc00fc773201f4fdeca26c500653870253c43b81e65211",
"guardedInstallationSourceSha256": "56a177d3e702914818dca4ca50e061f0d25f988bfc4c0582c36ce23c1696d82f",
"nativeExecutionSourceSha256": "3a30d52bffad68c7f02349193be84761050eb99039fc0556c2b449891a36e918",
"profileCompatibilitySourceSha256": "8503e7f2f8175e413b42e85a9ff259a00f2278cd966bbeafc9e9e85417cf7dbb",
"installationSourceSha256": "a1c7c6f103b898216973e77ffbac0ef7572e250aa574d1a4065270e9dfa0dc2d",
"assetOwnershipSourceSha256": "1785e3f31e5dad5a13b53e25d8cf9482a9bd89a27b6c127830aed6b81acc605c",
"assetInstallerSourceSha256": "d3bd9514e5920d0cc3567aa50487fb534c57a6f66b8da0d3f22c5f6e499c841f",
"credentialBindingSourceSha256": "7586d44b47c83395498dc8806287cd3538ef58177ba7c5949f5766ed105a0161",
"sidecarErrorSourceSha256": "b09af4577f8bf42eee7036b2ad9842202cfc763e72b53c3287f82c6090cdad80",
"profileSourceSha256": "f5538b3dbefa7634a332b87d5ededcefb56aa7e5790cede1c7a0d948a548a456",
"metadataProbeSourceSha256": "be4c8416edd27061109bc1f33d0c26071b4facbb3f651beaae6b3c0275cce727",
"permissionToolIdentitySourceSha256": "770b47305e571a81f344883057d0e4a482a1cd59ac77d045836593a8e739deb1",
"semanticContinuationSourceSha256": "d248e47a0fe7f2536d1e64cea0b0b410f34104800cd643d995b4473104ecc8e0",
"runtimeHostSourceSha256": "94a73749267f5a80b5df19413a65b4b9263c00a9bf7e98fda00f00b8f9fd32ec",
"runtimeAdapterSourceSha256": "e4c764c2c3c7dd655a965b39472bd1ad8d577d1e7258b25594b171c9429752b8",
"runtimeSandboxSourceSha256": "92dfa53f5a3ed0062913ae417eeccec243b430d444495efde2a8e9c4c02e78ea",
"cancellationSourceSha256": "a07d1a9e982dec0ae130c1ad1efa5bc75c6a39e013b730ba3ee45256f09ceeca",
"agentFilesSourceSha256": "fb32f6badc13dfa22f231479509b631b2f0eba0c99b1ab9f3cd5c0c601aa03ba",
"nativeDistributionSourceSha256": "aed327c389e957483a503cb1bb8b247ee7cd7ff4b528fb4567c39bcb4577690b"
}
}
packages/paperclip-runner/test/fixtures/copilot-profile-v61-identity.json
Vendored
+85
View File
@@ -0,0 +1,85 @@
{
"commandDigest": "sha256:956276ebee59f880249fa6955adc224af9ca7b6a9da59bcc5da4a8667550c5b3",
"declaration": {
"schema": "paperclip.acpx_profile_declaration.v1",
"agent": "copilot",
"agentProfileVersion": 61,
"acpxVersion": "0.13.1",
"agentServerVersion": "1.0.88",
"protocolPolicyRevision": "copilot-agent-manual-v1",
"acpxPatchSha256": "00d6af17216cb0cad902b48633eb19bf0ec6b05837a984cb24661fca593f09c1",
"policySha256": "a4be69be43db84052560420a3ca546a665c9165573c24d543eb33d144ba138a3",
"distributionSourceSha256": "c0a65e247fc47b263bd369a3d5ab505c4b323d4e6493be8a9ecbeb0e9dbb9107",
"agentFilesBinding": "registered-runtime-context-v1",
"systemInstructionDelivery": "COPILOT_HOME/copilot-instructions.md:replace-under-lifetime-lease-before-launch:v1",
"sharedRuntimeContract": "paperclip.acpx-runtime-contract.v1",
"innerDistributionSourceSha256": "9bb08347eed01f1ab44e065c8b06ead9be3a25973d6e6329dc4c0a099096279a",
"upstreamAppSha256": "7b48282a19b5b0814a0c96ad5e3a125173d792cc55f9525b2effea962f6063c0",
"patchedAppSha256": "61f8eebe0c30cc03b311c6818b2acb158a6d16ca58c8bfac5ebd42db095c2679",
"messageIdentityContract": "copilot-native-message-id-v1",
"ownedDistributionDelivery": "COPILOT_CLI_DIST_DIR:lease-owned-guarded-inner-distribution:v1",
"permissionContextContract": "copilot-edit-permission-context-v1",
"permissionContextSourceSha256": "de2d38d702b3c574161af0c0c04790fb6f6347916eb3916ce7a131a8c229121b",
"permissionAdapterSourceSha256": "ee5929ed2ee647ad0369465dee9d51a00b2dd28e1ff735b43e969ddc9c164668",
"permissionLocationsSourceSha256": "f39a81da395b6e23aa2a1a45572a28846e5e81bfd3d764d9a456f74a79c752e9",
"permissionRedactionSourceSha256": "ad6770c69269087951790e2e87e22ac088e3d432769adf17ebc865ed512e6c4f",
"permissionClassifierSourceSha256": "1161eabd82e26133e3b3db3ab567f65d52808b03f176e93497e02c8330349fb4",
"semanticToolReceiptContract": "paperclip.semantic_tool_receipt.v2",
"semanticReceiptSourceSha256": "b2795726b2234a663fec1a6ca11f0551ae69882fe9e82aa39686e952d036e791",
"semanticBridgeSourceSha256": "8764b8c9032fdc7d1415b01e75a9fc57c550e7a5e11da4bed7d9100082872ae5",
"toolEvidenceSourceSha256": "6eef5039afc75bb851eaf9cd82a6f099e4bd398ca2deb59dd66f5b2a81a7a706",
"semanticNormalizedInputContract": "validated-forwarded-input-commit-v1",
"semanticSidecarSourceSha256": "17361d1782c0dcebb2b7972635f45c890f2a4081f8bc0ef2d2276550e0b91629",
"semanticDirectDriverSourceSha256": "ff572638f2cb5d9f8249ff8e2f412311c1e3af209ef0bf7e5ea9f10a21d91103",
"semanticValidationSourceSha256": "02fbb92681c78c5de5d2f1aae25aabc25a0e5df6b0ff4baccac652192a8da975",
"semanticNormalizationSourceSha256": "8b628b82e8eacc91cdcc6fdca2210ba4cf282f01e9c6f74e95ff271655b5047e",
"semanticCompletionContractSourceSha256": "3343df6362252d44d12b079ff78c8f6539d1544f3ce5ec36ff36db70ab0fac31",
"semanticValidatorsSourceSha256": "f22d9caf2035b600abc1a6625ae23ca97f1409d7e29ed5b7264bb8f9784c7dc9",
"semanticSchemaBundleSourceSha256": "016391a087cc217aafed4ba1f4efafc6e4881a7525952dce824b7f4d24530603",
"semanticSidecarProtocolSourceSha256": "03c0c95b900f9dc18f29f6be27b8c79bc0bcab29780190d46a74e95328081236",
"semanticSidecarReceiptCommitContract": "correlated-tool-resolve-v1",
"displayEventSourceSha256": "d72f9a6035c247bc6a42c067c05753076c54322c4b58984e93922f3325dc6877",
"displayProjectionSourceSha256": "75952606b26dd93bbc582c1f0a4424487377272ea745fb3a27ceae031fd5dc7e",
"configuredEnvironmentSourceSha256": "5c9008d121979a3a8987e6b8a5752172eafdc4a2b5d725a73f3bbb8fb83be8ff",
"configuredEnvironmentRustSourceSha256": "85c87fae62b8a9377e4f0a4c00302c7519fae4397b1c48253f8f2b7ba7faf994",
"profileActivitySourceSha256": "4fdb6b0f9125fd8372ba6f27945fa5c8a3d3b2ec891df544092e3dc7a75d5282",
"profileExtensionsSourceSha256": "a80ddc95eb4505095bfdbbf85c758199949ab20be9c84b34c77f762eb7180571",
"profileInstallationSourceSha256": "3e61c4cdd4b78e38fcb16eaadae4f1fd65ce3c7127184ea7a80948c8e5d77f88",
"attachmentPolicySourceSha256": "4f7638b7cc64e18f6a4ba33fd46de2b3d49225e32aeab41f5c695b268fa4f8e2",
"singleReadEvidenceSourceSha256": "4069393405089bde613cecec04f5123c426594811da4c668bd96f0e7e283550f",
"nativeSessionRuntimeSourceSha256": "1fe6fa9e6b96d84d70432b85a57c473f833606d4a521563d5022b4d3c41576ce",
"nativeRuntimeContextSourceSha256": "71607f80dbb8dc5cc6c123d0b8f3995d6c5407277695a8c82cbedc90fb8f4842",
"nativeRuntimePromptsSourceSha256": "103b83066da72434602e9e461e6278dfb40bb993d2f3b49b2e93df81bbb08b72",
"nativeSessionContractSourceSha256": "22a8c96ce41ba4f03e976fc37f0cc7c5d05cee99fa789941da31ee1dab440b90",
"harnessSessionContractSourceSha256": "03f4cd7f76f34f6d82e009b66aa51f18f6e23dc235b660778060e35b6ea07412",
"harnessBackendSourceSha256": "dc827e281ae6731545b166ee4d709a4f67f594a556dcbe63c70fe8f1aca26249",
"harnessSessionSourceSha256": "81361733e86b56dd5a66415aefc6f72e69be24acc8d3417370154ebb7fef6e94",
"transportContractSourceSha256": "03fca93d36a89b0cc0917335e293ad097fdf04356d21b307827bb0ea4c13f523",
"nativeBackendSourceSha256": "e7b12d4418c7a71f5f2474b046697d3009eae3613b284bec381df5e9965a1c37",
"nativeBackendFactorySourceSha256": "7e431c926b76d53c4c08e3a4105353dcd34bd940f6588a1a68f79631f6feeb44",
"nativeInstructionsSourceSha256": "9173ebccf09cb56a781d2bf5c7eb27fdafb13d3450d99342c6aff86b42d6b43a",
"providerLaunchSourceSha256": "fa8db1c42c378898287c141d8cea9069166981e369c9b9c51422cb6177de5a2c",
"launchPreparationStoreSourceSha256": "ff6025568bbc0aa6ce545772d73dfca2d3da226740bd3a7b386b069e76cff296",
"runtimeRequestBridgeSourceSha256": "e6c98128d98a4177c42d68a04c1f749554fd5ec844da1bbbdb3f682670189445",
"runtimeRequestFacadeSourceSha256": "d0ed28e0c5ddad34db8f437e704c8a755eb0b47f27e08e44185f66d1d2205295",
"runtimeRequestNotificationSourceSha256": "6196c6175f11c69295dc00fc773201f4fdeca26c500653870253c43b81e65211",
"guardedInstallationSourceSha256": "56a177d3e702914818dca4ca50e061f0d25f988bfc4c0582c36ce23c1696d82f",
"nativeExecutionSourceSha256": "3a30d52bffad68c7f02349193be84761050eb99039fc0556c2b449891a36e918",
"profileCompatibilitySourceSha256": "f9670b63d68e8b84cb79ab18425028f1ff781adb398cb9955ca80b8e0c9264c7",
"installationSourceSha256": "a1c7c6f103b898216973e77ffbac0ef7572e250aa574d1a4065270e9dfa0dc2d",
"assetOwnershipSourceSha256": "1785e3f31e5dad5a13b53e25d8cf9482a9bd89a27b6c127830aed6b81acc605c",
"assetInstallerSourceSha256": "d3bd9514e5920d0cc3567aa50487fb534c57a6f66b8da0d3f22c5f6e499c841f",
"credentialBindingSourceSha256": "7586d44b47c83395498dc8806287cd3538ef58177ba7c5949f5766ed105a0161",
"sidecarErrorSourceSha256": "b09af4577f8bf42eee7036b2ad9842202cfc763e72b53c3287f82c6090cdad80",
"profileSourceSha256": "f5538b3dbefa7634a332b87d5ededcefb56aa7e5790cede1c7a0d948a548a456",
"metadataProbeSourceSha256": "be4c8416edd27061109bc1f33d0c26071b4facbb3f651beaae6b3c0275cce727",
"permissionToolIdentitySourceSha256": "770b47305e571a81f344883057d0e4a482a1cd59ac77d045836593a8e739deb1",
"semanticContinuationSourceSha256": "d248e47a0fe7f2536d1e64cea0b0b410f34104800cd643d995b4473104ecc8e0",
"runtimeHostSourceSha256": "94a73749267f5a80b5df19413a65b4b9263c00a9bf7e98fda00f00b8f9fd32ec",
"runtimeAdapterSourceSha256": "e4c764c2c3c7dd655a965b39472bd1ad8d577d1e7258b25594b171c9429752b8",
"runtimeSandboxSourceSha256": "92dfa53f5a3ed0062913ae417eeccec243b430d444495efde2a8e9c4c02e78ea",
"cancellationSourceSha256": "a07d1a9e982dec0ae130c1ad1efa5bc75c6a39e013b730ba3ee45256f09ceeca",
"agentFilesSourceSha256": "fb32f6badc13dfa22f231479509b631b2f0eba0c99b1ab9f3cd5c0c601aa03ba",
"nativeDistributionSourceSha256": "aed327c389e957483a503cb1bb8b247ee7cd7ff4b528fb4567c39bcb4577690b"
}
}
packages/plugins/sdk/src/environment-tasks.ts
+1
View File
@@ -39,6 +39,7 @@ export const environmentRunnerLaunchRequirementsSchema = z.object({
acpxPermissionModePinned: z.boolean().optional(), piThinkingLevel: z.enum(["off", "low", "high", "max"]).optional(),
claudeEffort: z.enum(["low", "medium", "high", "xhigh", "max"]).optional(),
opencodePermissionMode: z.enum(["allow", "ask", "deny"]).optional(),
disableCodexWebSearch: z.literal(true).optional(),
managedProfile: z.record(z.string(), z.unknown()).optional(), agentCoreProfile: z.record(z.string(), z.unknown()).optional(),
}).strict(),
approvalPolicy: z.enum(["never", "on-request", "untrusted"]), readOnly: z.boolean(),
packages/plugins/sdk/tests/environment-tasks.test.ts
+34
View File
@@ -11,3 +11,37 @@ describe("task lifecycle policy",()=>{
expect(environmentTaskOperationSchema.safeParse({...submit,launch:{...submit.launch,lifecyclePolicy:{mode:"warm",idleTimeoutMs}}}).success).toBe(false);
});
});
const prepare = {
kind: "prepare", projectIds: [], requirements: {
schema: "paperclip.runner.launch-requirements.v1",
identity: { runnerInstanceId: "runner", environmentLeaseId: "lease", runId: "run",
normalizedSessionId: "session", turnId: "turn", itemId: "item" },
provider: "codex", model: "selected-model", instructions: "Verify the selected account.",
runtimeContext: null, settings: {}, approvalPolicy: "never", readOnly: true,
preparedConversation: false, includeCollaborationModeInstructions: false,
},
};
describe("task hosted search restriction", () => {
it("preserves Core's explicit restriction through plugin launch preparation", () => {
const operation = { ...prepare, requirements: { ...prepare.requirements,
settings: { disableCodexWebSearch: true } } };
expect(environmentTaskOperationSchema.parse(operation)).toEqual(operation);
});
it("keeps the provider default when the restriction is absent", () => {
expect(environmentTaskOperationSchema.parse(prepare)).toEqual(prepare);
});
it.each([false, null, 0, "true", "disabled"])("rejects a non-restricting flag value %s", disableCodexWebSearch => {
expect(environmentTaskOperationSchema.safeParse({ ...prepare, requirements: { ...prepare.requirements,
settings: { disableCodexWebSearch } } }).success).toBe(false);
});
it("continues rejecting unreviewed launch settings", () => {
expect(environmentTaskOperationSchema.safeParse({ ...prepare, requirements: { ...prepare.requirements,
settings: { disableCodexWebSearch: true, hostedSearchBillingMode: "unmetered" } } }).success).toBe(false);
});
});
server/src/__tests__/chat-channels.integration.test.ts
+353
View File
@@ -131,6 +131,7 @@ import { deliverReconciledExecutions } from "../services/execution-recovery-reso
import {
unadmittedChatWakeupCondition,
authorizeFailedChatRunRetryWake,
createDurableChatWakeupRequest,
} from "../services/durable-chat-wakeup.js";
import {
createChatWebhookDiagnostics,
@@ -60661,6 +60662,358 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
return { ...context, issue, action, receipt, runId };
}
async function currentGitHubTaskRetryFixture(withPending = false, pendingCount = 1) {
const f = await failedChatRetryFixture("github", "42", true);
const companyId = f.fixture.companyId, issueId = f.issue.id;
await githubResponseCommentService(db, f.providerFetch).tryAcknowledge(f.action.deliveryId!);
await db.update(heartbeatRuns).set({ runtimeMode: "native", nativeIssueId: issueId,
nativePhase: "terminal_failure", runnerProfileJson: {} }).where(eq(heartbeatRuns.id, f.runId));
await db.insert(nativeRunFinalizations).values({ companyId, issueId, runId: f.runId, phase: "terminal_failure" });
await db.insert(heartbeatRunEvents).values({ companyId, agentId: f.fixture.assignedAgentId, runId: f.runId,
seq: 1, eventType: "native.local_process_stopped", payload: { processPid: 999999999, processGroupId: null } });
let pending: typeof chatActions.$inferSelect | undefined;
if (withPending) {
for (let index = 0; index < pendingCount; index++) {
await deliverMessage({ callbacks: f.runtime.configurations.get(f.endpoint.id)!.callbacks,
endpointId: f.endpoint.id, provider: "github", thread: f.thread.thread,
message: makeMessage({ id: String(990092 + index), text: `@maya continue saved follow-up ${index}`, userId: "42", mentioned: true }),
trigger: "mention" });
const [{ source }] = await db.select({ source: chatActions }).from(chatActions)
.innerJoin(chatDeliveries, eq(chatDeliveries.id, chatActions.deliveryId)).where(and(
eq(chatActions.conversationId, f.conversation.id), eq(chatActions.kind, "inbound_wakeup"),
eq(chatDeliveries.providerEventId, `${f.thread.thread.id}:${990092 + index}`)));
pending ??= source;
await db.update(agentWakeupRequests).set({ status: "deferred_issue_execution", runId: null })
.where(eq(agentWakeupRequests.id, source.id));
await githubResponseCommentService(db, f.providerFetch).tryAcknowledge(source.deliveryId!);
}
}
const [originalDelivery] = await db.select().from(chatDeliveries).where(eq(chatDeliveries.id, f.action.deliveryId!));
await db.update(agents).set({ adapterType: "paperclip_runner", status: "running",
runtimeConfig: { heartbeat: { wakeOnDemand: true, maxConcurrentRuns: 1 } } })
.where(eq(agents.id, f.fixture.assignedAgentId));
await db.insert(heartbeatRuns).values({ companyId, agentId: f.fixture.assignedAgentId,
status: "running", startedAt: new Date() });
f.wakeup.mockImplementation(heartbeatService(db).wakeup);
return { ...f, pending, originalDelivery };
}
it.each(["reconnected", "newer_input", "reconnected_newer_input"] as const)(
"admits current GitHub task Retry without replaying obsolete input: %s", async kind => {
const f = await currentGitHubTaskRetryFixture(kind.includes("newer_input"));
const { pending, originalDelivery } = f;
try {
const companyId = f.fixture.companyId, issueId = f.issue.id;
if (kind.includes("reconnected")) await f.service.configure(f.endpoint.id, { action: "reconnect" }, "owner-user");
const input = { db, companyId, issueId, actorId: "owner-user", actorSource: "cloud_tenant",
observedRunId: f.runId, idempotencyKey: randomUUID(), prepareChatRetry: f.service.prepareFailedChatRunRetry };
const request = await requestTaskRetry(input);
expect(request.accepted).toBe(true);
expect((await requestTaskRetry(input)).requestId).toBe(request.requestId);
expect(await f.service.processFailedChatRunRetry(request.chatActionId!)).toMatchObject({ status: "queued" });
const [wake] = await readAgentWakeupRequestsById(request.chatActionId!);
expect(wake.runId).toBeTruthy();
const expectedComment = pending?.payload.commentId ?? f.action.payload.commentId;
const [run] = await readHeartbeatRunsById(wake.runId!);
expect(run).toMatchObject({ status: "queued", retryOfRunId: f.runId,
contextSnapshot: { wakeCommentIds: [expectedComment], taskRetryRequestId: request.requestId } });
if (pending) expect((await readAgentWakeupRequestsById(pending.id))[0])
.toMatchObject({ status: "coalesced", runId: wake.runId });
await f.service.processFailedChatRunRetry(request.chatActionId!);
await heartbeatService(db).resumeQueuedRuns();
expect((await readHeartbeatRunsByCompanyId(companyId)).filter(r => r.contextSnapshot?.taskRetryRequestId === request.requestId))
.toHaveLength(1);
expect((await db.select().from(chatDeliveries).where(eq(chatDeliveries.id, originalDelivery.id)))[0].normalizedEvent)
.toEqual(originalDelivery.normalizedEvent);
} finally { await f.service.shutdown(); }
},
);
it.each(["legacy_receipt", "concurrent_clicks", "lost_dispatch_reply", "restart", "retry_failed_successor", "http", "handoff_rollback", "board_permission_wait", "peer_conversation", "backlog_51", "consumed_backlog_52"] as const)(
"preserves one current GitHub Retry across %s", async mode => {
const f = await currentGitHubTaskRetryFixture(true, mode.includes("backlog") ? 51 : mode === "peer_conversation" ? 2 : 1);
try {
const companyId = f.fixture.companyId, issueId = f.issue.id;
if (mode === "consumed_backlog_52") {
const sources = await db.select().from(chatActions).where(and(eq(chatActions.conversationId, f.conversation.id),
eq(chatActions.kind, "inbound_wakeup"))).orderBy(asc(chatActions.createdAt), asc(chatActions.id));
expect(sources).toHaveLength(52);
const commentIds = sources.map(source => source.payload.commentId);
await db.update(heartbeatRuns).set({ finishedAt: new Date(), contextSnapshot: {
...(await readHeartbeatRunsById(f.runId))[0].contextSnapshot,
wakeCommentIds: commentIds, wakeCommentId: commentIds.at(-1),
} }).where(eq(heartbeatRuns.id, f.runId));
await db.update(agentWakeupRequests).set({ runId: f.runId, status: "coalesced",
payload: sql`${agentWakeupRequests.payload} || ${JSON.stringify({ coalescedIntoWakeupRequestId: f.action.id })}::jsonb`,
}).where(and(inArray(agentWakeupRequests.id, sources.map(source => source.id)),
sql`${agentWakeupRequests.id} <> ${f.action.id}::uuid`));
}
let peerInput: typeof chatActions.$inferSelect | undefined;
if (mode === "peer_conversation") {
const originId = randomUUID();
await db.insert(chatConversations).values({ ...f.conversation, id: originId, sessionGeneration: 2 });
await db.update(chatActions).set({ conversationId: originId,
payload: sql`jsonb_set(${chatActions.payload}, '{sessionGeneration}', '2'::jsonb)`,
}).where(inArray(chatActions.id, [f.action.id, f.pending!.id]));
await db.update(chatDeliveries).set({ conversationId: originId })
.where(inArray(chatDeliveries.id, [f.action.deliveryId!, f.pending!.deliveryId!]));
await db.update(chatMessageLinks).set({ conversationId: originId })
.where(inArray(chatMessageLinks.deliveryId, [f.action.deliveryId!, f.pending!.deliveryId!]));
[peerInput] = await db.select().from(chatActions).where(and(eq(chatActions.conversationId, f.conversation.id),
eq(chatActions.kind, "inbound_wakeup")));
expect(peerInput).toBeDefined();
}
if (mode === "legacy_receipt") {
const [originalEndpoint] = await readChatEndpointsById(f.endpoint.id);
await db.insert(chatActions).values({ companyId, endpointId: f.endpoint.id,
conversationId: f.conversation.id, deliveryId: f.pending!.deliveryId,
kind: "github_response_comment", providerActionId: `github_response:${f.pending!.deliveryId}`,
payload: { botExternalId: originalEndpoint.botExternalId, agentId: f.fixture.assignedAgentId },
status: "processed" }).onConflictDoNothing();
await db.update(chatDeliveries).set({
normalizedEvent: sql`jsonb_set(${chatDeliveries.normalizedEvent}, '{githubAuthority}',
(${chatDeliveries.normalizedEvent}->'githubAuthority') - 'botExternalId')`,
}).where(eq(chatDeliveries.id, f.pending!.deliveryId!));
}
await f.service.configure(f.endpoint.id, { action: "reconnect" }, "owner-user");
const input = { db, companyId, issueId, actorId: "owner-user", actorSource: "cloud_tenant",
observedRunId: f.runId, idempotencyKey: randomUUID(), prepareChatRetry: f.service.prepareFailedChatRunRetry };
if (mode === "board_permission_wait") {
input.actorId = `retry-operator-${companyId}`;
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: input.actorId,
status: "active", membershipRole: "operator" });
}
const queryEvents: Record<string, unknown>[] = [];
const capture = mode === "backlog_51" ? createDatabaseResponseCapture({ start: Date.now() - 1,
end: Date.now() + 59_000 }, event => queryEvents.push(event)) : undefined;
if (capture) installDatabaseResponseCapture(db, { chatActions, chatDeliveries, chatMessageLinks, issueComments,
agentWakeupRequests, chatEndpoints, chatConversations }, capture);
let one: Awaited<ReturnType<typeof requestTaskRetry>>, two: typeof one;
if (mode === "http") {
const app = express();
app.use(express.json());
app.use((req, _res, next) => { req.actor = boardActor(companyId); next(); });
app.use("/api", issueRoutes(db, createStorageService().storage, { chatRunRetries: f.service }));
app.use(errorHandler);
const send = () => request(app).post(`/api/issues/${issueId}/retry`).send({
idempotencyKey: input.idempotencyKey, observedRunId: input.observedRunId,
}).expect(202);
const replies = await Promise.all([send(), send()]);
[one, two] = replies.map(response => response.body);
} else [one, two] = await Promise.all([requestTaskRetry(input), requestTaskRetry({ ...input,
idempotencyKey: mode === "concurrent_clicks" ? randomUUID() : input.idempotencyKey })]);
if (capture) {
capture.flush();
expect(Number(queryEvents.find(event => event.event === "paperclip.db_response_capture.summary.v1")?.startedQueries))
.toBeLessThan(180);
}
expect(two.requestId).toBe(one.requestId);
expect(one).toMatchObject({ accepted: true, chatActionId: expect.any(String) });
let service = f.service;
if (mode === "restart") {
await service.shutdown();
service = createService(f.runtime, f.providerFetch, { wakeup: heartbeatService(db).wakeup }).service;
}
if (mode === "lost_dispatch_reply") f.wakeup.mockImplementationOnce(async (...args) => {
await heartbeatService(db).wakeup(...args);
throw new Error("Simulated response loss after dispatch commit");
});
if (mode === "handoff_rollback") f.wakeup.mockImplementationOnce(async (agentId, options) => {
const original = options!.durableChatRequest!;
return heartbeatService(db).wakeup(agentId, { ...options, durableChatRequest: createDurableChatWakeupRequest({
...original, onAdmitted: async (tx, receipt) => {
await original.onAdmitted!(tx, receipt);
throw new Error("Simulated interruption before admission commit");
},
}) });
});
if (mode === "board_permission_wait") await db.update(companyMemberships).set({ membershipRole: "viewer" })
.where(and(eq(companyMemberships.companyId, companyId), eq(companyMemberships.principalId, input.actorId)));
await Promise.all([service.processFailedChatRunRetry(one.chatActionId!),
service.processFailedChatRunRetry(one.chatActionId!)]);
if (mode === "handoff_rollback" || mode === "board_permission_wait") {
expect(await readAgentWakeupRequestsById(one.chatActionId!)).toHaveLength(0);
expect((await readAgentWakeupRequestsById(f.pending!.id))[0])
.toMatchObject({ status: "deferred_issue_execution", runId: null });
expect((await readHeartbeatRunsByCompanyId(companyId))
.filter(run => run.contextSnapshot?.taskRetryRequestId === one.requestId)).toHaveLength(0);
}
if (mode === "board_permission_wait") await db.update(companyMemberships).set({ membershipRole: "operator" })
.where(and(eq(companyMemberships.companyId, companyId), eq(companyMemberships.principalId, input.actorId)));
await vi.waitFor(async () => {
await service.processFailedChatRunRetry(one.chatActionId!);
expect((await readAgentWakeupRequestsById(one.chatActionId!))[0]?.runId,
JSON.stringify((await db.select().from(chatActions).where(eq(chatActions.id, one.chatActionId!)))[0]?.result)).toBeTruthy();
}, { timeout: 10_000, interval: 100 });
const [parent] = await readAgentWakeupRequestsById(one.chatActionId!);
expect(parent.runId).toBeTruthy();
expect((await readAgentWakeupRequestsById(f.pending!.id))[0]).toMatchObject({
status: "coalesced", runId: mode === "consumed_backlog_52" ? f.runId : parent.runId,
});
if (peerInput) expect((await readAgentWakeupRequestsById(peerInput.id))[0])
.toMatchObject({ status: "deferred_issue_execution", runId: null });
if (mode === "backlog_51") {
const saved = await db.select({ id: chatActions.id, payload: chatActions.payload,
status: agentWakeupRequests.status, runId: agentWakeupRequests.runId }).from(chatActions)
.innerJoin(agentWakeupRequests, eq(agentWakeupRequests.id, chatActions.id)).where(and(
eq(chatActions.conversationId, f.conversation.id), eq(chatActions.kind, "inbound_wakeup"),
sql`${chatActions.id} <> ${f.action.id}::uuid`,
)).orderBy(asc(chatActions.createdAt), asc(chatActions.id));
expect(saved).toHaveLength(51);
expect(saved.every(source => source.status === "coalesced" && source.runId === parent.runId)).toBe(true);
expect((await readHeartbeatRunsById(parent.runId!))[0].contextSnapshot?.wakeCommentIds)
.toEqual(saved.map(source => source.payload.commentId));
}
await service.processFailedChatRunRetry(one.chatActionId!);
expect((await readHeartbeatRunsByCompanyId(companyId))
.filter(run => run.contextSnapshot?.taskRetryRequestId === one.requestId)).toHaveLength(1);
expect(await db.select().from(chatActions).where(and(eq(chatActions.companyId, companyId),
eq(chatActions.kind, "failed_run_retry")))).toHaveLength(1);
if (mode === "retry_failed_successor" || mode === "backlog_51") {
await db.update(heartbeatRuns).set({ status: "failed", finishedAt: new Date(),
errorCode: "adapter_failed", runtimeMode: "native", nativeIssueId: issueId, nativePhase: "terminal_failure",
}).where(eq(heartbeatRuns.id, parent.runId!));
await db.update(agentWakeupRequests).set({ status: "failed" }).where(eq(agentWakeupRequests.id, one.chatActionId!));
await db.update(issues).set({ executionRunId: null, status: "blocked" }).where(eq(issues.id, issueId));
await db.insert(nativeRunFinalizations).values({ companyId, issueId, runId: parent.runId!, phase: "terminal_failure" });
await db.insert(heartbeatRunEvents).values({ companyId, agentId: f.fixture.assignedAgentId, runId: parent.runId!,
seq: 1, eventType: "native.local_process_stopped", payload: { processPid: 999999998, processGroupId: null } });
await service.configure(f.endpoint.id, { action: "reconnect" }, "owner-user");
const next = await requestTaskRetry({ ...input, observedRunId: parent.runId!, idempotencyKey: randomUUID(),
prepareChatRetry: service.prepareFailedChatRunRetry });
expect(next.requestId).not.toBe(one.requestId);
expect(await service.processFailedChatRunRetry(next.chatActionId!)).toMatchObject({ status: "queued" });
const [second] = await readAgentWakeupRequestsById(next.chatActionId!);
expect((await readHeartbeatRunsById(second.runId!))[0]).toMatchObject({
retryOfRunId: parent.runId, contextSnapshot: {
wakeCommentIds: (await readHeartbeatRunsById(parent.runId!))[0].contextSnapshot!.wakeCommentIds,
},
});
}
} finally { await f.service.shutdown(); }
},
);
it("leaves saved GitHub input pending when a persisted Retry wait loses board authority", async () => {
const f = await currentGitHubTaskRetryFixture(true);
try {
const companyId = f.fixture.companyId, issueId = f.issue.id, actorId = `retry-operator-${companyId}`;
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: actorId,
status: "active", membershipRole: "operator" });
const intent = await requestTaskRetry({ db, companyId, issueId, actorId, actorSource: "cloud_tenant",
observedRunId: f.runId, idempotencyKey: randomUUID(), prepareChatRetry: f.service.prepareFailedChatRunRetry });
let savedOptions: Parameters<typeof f.wakeup>[1];
f.wakeup.mockImplementationOnce(async (_agentId, options) => {
savedOptions = options;
const request = options!.durableChatRequest!;
// Persist the pre-handoff wait as a server interrupted before queue adoption would.
await db.insert(agentWakeupRequests).values({ id: request.id, companyId, agentId: request.agentId,
source: options!.source, triggerDetail: options!.triggerDetail, reason: options!.reason,
requestedByActorType: request.requestedByActorType, requestedByActorId: request.requestedByActorId,
requestedAt: request.requestedAt, idempotencyKey: request.idempotencyKey,
status: "deferred_issue_execution", payload: { ...options!.payload,
taskRetryRequestId: intent.requestId, _paperclipWakeContext: options!.contextSnapshot } });
return null;
});
expect(await f.service.processFailedChatRunRetry(intent.chatActionId!)).toMatchObject({ status: "deferred" });
await db.update(companyMemberships).set({ membershipRole: "viewer" }).where(and(
eq(companyMemberships.companyId, companyId), eq(companyMemberships.principalId, actorId)));
expect(await heartbeatService(db).wakeup(f.fixture.assignedAgentId, savedOptions!)).toBeNull();
expect((await readAgentWakeupRequestsById(f.pending!.id))[0])
.toMatchObject({ status: "deferred_issue_execution", runId: null });
expect((await readHeartbeatRunsByCompanyId(companyId))
.filter(run => run.contextSnapshot?.taskRetryRequestId === intent.requestId)).toHaveLength(0);
await db.update(companyMemberships).set({ membershipRole: "operator" }).where(and(
eq(companyMemberships.companyId, companyId), eq(companyMemberships.principalId, actorId)));
await heartbeatService(db).wakeup(f.fixture.assignedAgentId, savedOptions!);
const [parent] = await readAgentWakeupRequestsById(intent.chatActionId!);
expect(parent.runId).toBeTruthy();
expect((await readAgentWakeupRequestsById(f.pending!.id))[0]).toMatchObject({ status: "coalesced", runId: parent.runId });
} finally { await f.service.shutdown(); }
});
it.each(["changed_bot", "disabled_repository", "retargeted_repository", "revoked_sender", "paused_endpoint",
"edited_source", "deleted_source", "reassigned_task", "second_reconnect", "later_message_wrong_bot", "later_message_deleted", "later_message_revoked_policy"] as const)(
"keeps current GitHub task Retry authority gated: %s", async gate => {
const f = await currentGitHubTaskRetryFixture(true, gate.startsWith("later_message") ? 2 : 1);
try {
const companyId = f.fixture.companyId, issueId = f.issue.id;
await f.service.configure(f.endpoint.id, { action: "reconnect" }, "owner-user");
const input = { db, companyId, issueId, actorId: "owner-user", actorSource: "cloud_tenant",
observedRunId: f.runId, idempotencyKey: randomUUID(), prepareChatRetry: f.service.prepareFailedChatRunRetry };
if (gate === "later_message_revoked_policy") {
const [resource] = await db.select().from(chatEndpointResources).where(eq(chatEndpointResources.id, f.conversation.resourceId!));
await db.update(chatEndpointResources).set({ availability: "available",
metadata: { ...resource.metadata, providerRepositoryId: "418" },
}).where(eq(chatEndpointResources.id, resource.id));
await db.insert(chatGitHubConfigurations).values({ companyId, endpointId: f.endpoint.id,
updatedByUserId: "owner-user", configuration: { version: 1, toolsEnabled: true, responsibleUserId: "owner-user",
memberAccess: "all_linked", people: [], repositories: {},
defaults: { ...defaultGitHubReviewPolicy(), invocation: "linked_authors", events: ["opened", "synchronize"] },
} });
const sources = await db.select().from(chatActions).where(and(eq(chatActions.conversationId, f.conversation.id),
eq(chatActions.kind, "inbound_wakeup"), sql`${chatActions.id} <> ${f.action.id}::uuid`))
.orderBy(asc(chatActions.createdAt), asc(chatActions.id));
for (const [index, source] of sources.entries()) {
const context = { event: index === 0 ? "opened" : "synchronize", deliveryId: source.deliveryId,
repositoryId: "418", repository: resource.providerResourceId,
pullNumber: 7, title: "Saved review request", body: "", baseSha: "a".repeat(40), headSha: "b".repeat(40),
baseBranch: "master", author: { id: "42", login: "octocat", isBot: false },
sender: { id: "42", login: "octocat" }, draft: false, labels: [] };
await db.update(chatDeliveries).set({
normalizedEvent: sql`${chatDeliveries.normalizedEvent} || ${JSON.stringify({ githubAutomatic: { context } })}::jsonb`,
}).where(eq(chatDeliveries.id, source.deliveryId!));
}
}
const accepted = await requestTaskRetry(input);
if (gate.startsWith("later_message")) {
const [later] = await db.select().from(chatActions).where(and(eq(chatActions.conversationId, f.conversation.id),
eq(chatActions.kind, "inbound_wakeup"), sql`${chatActions.id} not in (${f.action.id}::uuid, ${f.pending!.id}::uuid)`));
if (gate === "later_message_wrong_bot") await db.update(chatDeliveries).set({
normalizedEvent: sql`jsonb_set(${chatDeliveries.normalizedEvent}, '{githubAuthority,botExternalId}', ${JSON.stringify(randomUUID())}::jsonb)`,
}).where(eq(chatDeliveries.id, later.deliveryId!));
else if (gate === "later_message_deleted") await db.update(issueComments).set({ deletedAt: new Date() })
.where(eq(issueComments.id, String(later.payload.commentId)));
else await db.update(chatGitHubConfigurations).set({ revision: sql`${chatGitHubConfigurations.revision} + 1`,
configuration: sql`jsonb_set(${chatGitHubConfigurations.configuration}, '{defaults,events}', '["opened"]'::jsonb)`,
}).where(eq(chatGitHubConfigurations.endpointId, f.endpoint.id));
}
if (gate === "changed_bot") await db.update(chatEndpoints).set({ botExternalId: randomUUID() })
.where(eq(chatEndpoints.id, f.endpoint.id));
if (gate === "disabled_repository") await db.update(chatEndpointResources).set({ enabled: false })
.where(eq(chatEndpointResources.id, f.conversation.resourceId!));
if (gate === "retargeted_repository") await db.update(chatEndpointResources).set({ providerResourceId: "another/repository" })
.where(eq(chatEndpointResources.id, f.conversation.resourceId!));
if (gate === "revoked_sender") await db.update(chatIdentityLinks).set({ status: "revoked" })
.where(and(eq(chatIdentityLinks.endpointId, f.endpoint.id), eq(chatIdentityLinks.principalId, f.pending!.principalId!)));
if (gate === "paused_endpoint") await db.update(chatEndpoints).set({ status: "paused" })
.where(eq(chatEndpoints.id, f.endpoint.id));
if (gate === "edited_source") await db.update(issueComments).set({ body: "Changed after Retry", updatedAt: new Date() })
.where(eq(issueComments.id, String(f.pending!.payload.commentId)));
if (gate === "deleted_source") await db.update(issueComments).set({ deletedAt: new Date() })
.where(eq(issueComments.id, String(f.pending!.payload.commentId)));
if (gate === "reassigned_task") await db.update(issues).set({ assigneeAgentId: f.fixture.replacementAgentId })
.where(eq(issues.id, issueId));
if (gate === "second_reconnect") await f.service.configure(f.endpoint.id, { action: "reconnect" }, "owner-user");
expect(await f.service.processFailedChatRunRetry(accepted.chatActionId!)).toMatchObject({ status: "failed" });
expect((await readAgentWakeupRequestsById(f.pending!.id))[0]).toMatchObject({ status: "deferred_issue_execution", runId: null });
expect((await readHeartbeatRunsByCompanyId(companyId))
.filter(run => run.contextSnapshot?.taskRetryRequestId === accepted.requestId)).toHaveLength(0);
const [settled] = await db.select().from(chatActions).where(eq(chatActions.id, accepted.chatActionId!));
expect(settled.result?.taskRetryInputReconciled).toBe(true);
const [intentReceipt] = await readAgentWakeupRequestsById(accepted.requestId!);
await f.service.processFailedChatRunRetry(accepted.chatActionId!);
expect((await readAgentWakeupRequestsById(accepted.requestId!))[0].updatedAt).toEqual(intentReceipt.updatedAt);
if (gate === "second_reconnect") {
const next = await requestTaskRetry({ ...input, idempotencyKey: randomUUID() });
expect(next.requestId).not.toBe(accepted.requestId);
expect(await f.service.processFailedChatRunRetry(next.chatActionId!)).toMatchObject({ status: "queued" });
}
} finally { await f.service.shutdown(); }
},
);
it.each([
"settled",
"distinct_account",
server/src/__tests__/execution-control-work.test.ts
+134 -9
View File
@@ -4,7 +4,7 @@ import { localAiLoginService } from "../services/local-ai-login.js";
import { randomUUID } from "node:crypto";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { eq, sql } from "drizzle-orm";
import { agents, companies, createDb, heartbeatRuns, issues, nativeRunFinalizations, applyPendingMigrations } from "@paperclipai/db";
import { agents, agentWakeupRequests, companies, createDb, heartbeatRuns, issues, issueRecoveryActions, nativeRunFinalizations, applyPendingMigrations } from "@paperclipai/db";
import { installDatabaseWorkSignals } from "../../../packages/db/src/work-signals.js";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js";
@@ -16,7 +16,13 @@ import { terminalizeLegacyExecution } from "../services/legacy-execution-recover
import { reconcileAbandonedExecutionControl } from "../services/execution-control-reconciliation.js";
import { issueRecoveryActionService } from "../services/issue-recovery-actions.js";
import { markExecutionReconciliation, deliverReconciledExecutions, settleUnrecoverableExecutions } from "../services/execution-recovery-resolution.js";
import { nextWorkspaceRecoveryAt } from "../services/system-recovery.js";
import { createSystemRecovery } from "../services/system-recovery-state.js";
import { retainedWorkspaceCandidates } from "../services/workspace-recovery.js";
import { registerWorkspaceRecoveryWork } from "../services/workspace-recovery-work.js";
import { idleWorkSnapshot } from "../services/task-admission.js";
import { heartbeatService } from "../services/heartbeat.js";
import { requestTaskRetry } from "../services/task-retry.js";
const coordinators: ReturnType<typeof createDeliveryWorkCoordinator>[] = [];
afterEach(async () => { for (const worker of coordinators.splice(0)) await worker.stop(); vi.useRealTimers(); });
@@ -94,13 +100,13 @@ describe("execution-control lifecycle", () => {
expect(run).toHaveBeenCalledTimes(1); expect(next).toHaveBeenCalledTimes(1);
await c.stop(); expect(c.nextWakeAt()).toBeNull();
});
it("keeps other obligations responsive while one is waiting, and awaits shutdown", async () => {
it.each([DELIVERY_QUEUES.executionReplacement, DELIVERY_QUEUES.workspaceRecovery])("keeps finalization responsive while %s is waiting, and awaits shutdown", async heldQueue => {
vi.useFakeTimers(); const owner = fakeOwner(); const c = coordinator(owner);
let release!: () => void; const gate = new Promise<void>(resolve => { release = resolve; });
const status = vi.fn(async () => {});
const started = registerExecutionControlWork(c, [
{ queue: DELIVERY_QUEUES.executionReplacement, run: () => gate, next: async () => null },
{ queue: DELIVERY_QUEUES.executionStatus, run: status, next: async () => null },
{ queue: heldQueue, run: () => gate, next: async () => null },
{ queue: DELIVERY_QUEUES.executionFinalization, run: status, next: async () => null },
]);
await vi.advanceTimersByTimeAsync(1); expect(status).toHaveBeenCalledTimes(1);
let stopped = false; const stopping = c.stop().then(() => { stopped = true; });
@@ -120,6 +126,7 @@ describe.skipIf(!support.supported)("execution-control durable work", () => {
if (external) await applyPendingMigrations(external);
else temporary = await startEmbeddedPostgresTestDatabase("execution-sleep-");
db = createDb(external ?? temporary!.connectionString, { maxConnections: 6 });
await Promise.all(Array.from({ length: 6 }, () => db.execute(sql`select pg_sleep(0.05)`)));
}, 90000);
afterAll(async () => { await db?.$client.end({ timeout: 1 }); await temporary?.cleanup(); });
beforeEach(async () => {
@@ -145,22 +152,140 @@ describe.skipIf(!support.supported)("execution-control durable work", () => {
await settle(); expect(c.nextWakeAt()).toBeNull(); const before = calls;
await vi.advanceTimersByTimeAsync(90_000); expect(calls).toBe(before);
}
async function runRow(status = "running") {
const [run] = await db.insert(heartbeatRuns).values({ companyId, agentId, status, runtimeMode: "legacy", contextSnapshot: { issueId } }).returning();
async function runRow(status = "running", sourceIssueId = issueId) {
const [run] = await db.insert(heartbeatRuns).values({ companyId, agentId, status, runtimeMode: "legacy", contextSnapshot: { issueId: sourceIssueId } }).returning();
return run!;
}
it("stops all six pending/deadline scans with empty queues and completed history", async () => {
async function pausedRecoveredSource(actionId = randomUUID()) {
const sourceIssueId = randomUUID();
await db.insert(issues).values({ id: sourceIssueId, companyId, title: "Recovered execution",
status: "in_progress", assigneeAgentId: agentId });
const run = await runRow("failed", sourceIssueId);
await db.update(heartbeatRuns).set({ resultJson: { startupCancellation: { beforeNativeSelection: true } } })
.where(eq(heartbeatRuns.id, run.id));
await db.update(agents).set({ status: "paused" }).where(eq(agents.id, agentId));
await db.insert(issueRecoveryActions).values({ id: actionId, companyId, sourceIssueId,
kind: "active_run_watchdog", ownerType: "system", cause: "native_workspace_sync_out_retry_exhausted",
fingerprint: run.id, nextAction: "Saved work recovered", evidence: { runId: run.id,
systemRecovery: { ...createSystemRecovery(run.id), state: "complete" },
workspaceRecovery: { schema: "paperclip.workspace-recovery.v1", runId: run.id,
status: "recovered", stage: "saving_receipts", attempt: 1, attemptLimit: 6, requestIds: [] } } });
const retry = await requestTaskRetry({ db, companyId, issueId: sourceIssueId, actorId: "board",
observedRunId: run.id, idempotencyKey: randomUUID() });
expect(retry.accepted).toBe(true);
return { run, retry };
}
it("stops all pending/deadline scans with empty queues and completed history", async () => {
await runRow("succeeded");
const schedule = executionControlSchedule(observed);
const c = coordinator(db, undefined, error => { errors.push(error); });
const topics = [DELIVERY_QUEUES.executionFinalization, DELIVERY_QUEUES.executionReplacement, DELIVERY_QUEUES.executionContinuation,
const topics = [DELIVERY_QUEUES.executionFinalization, DELIVERY_QUEUES.workspaceRecovery, DELIVERY_QUEUES.executionReplacement, DELIVERY_QUEUES.executionContinuation,
DELIVERY_QUEUES.executionStatus, DELIVERY_QUEUES.executionDisposition, DELIVERY_QUEUES.localLoginCleanup];
const runs = [() => reconcileAbandonedExecutionControl(observed), () => reconcileSafeNativeReplacements(observed),
const runs = [() => reconcileAbandonedExecutionControl(observed), async () => {}, () => reconcileSafeNativeReplacements(observed),
() => deliverReconciledExecutions(observed, vi.fn(async () => null)), () => deliverExecutionStatuses(observed),
() => settleUnrecoverableExecutions(observed), () => localAiLoginService(observed).reapExpired()];
await registerExecutionControlWork(c, Object.values(schedule).map((next, i) => ({ queue: topics[i]!, run: runs[i]!, next }))).ready;
await quiet(c);
});
it("schedules only a current unfinished workspace recovery and becomes quiet with retained history", async () => {
const run = await runRow("failed");
const state = createSystemRecovery(run.id);
const evidence = { runId: run.id, systemRecovery: state,
workspaceRecovery: { schema: "paperclip.workspace-recovery.v1", runId: run.id,
status: "queued", stage: "verifying_stop", attempt: 0, attemptLimit: 6, requestIds: [] } };
const [action] = await db.insert(issueRecoveryActions).values({ companyId, sourceIssueId: issueId,
kind: "active_run_watchdog", ownerType: "system", cause: "native_workspace_sync_out_retry_exhausted",
fingerprint: run.id, nextAction: "Save work", evidence }).returning();
expect(await nextWorkspaceRecoveryAt(db)).toBe(0);
const due = Date.now() + 60_000;
await db.update(issueRecoveryActions).set({ evidence: { ...evidence, systemRecovery: { ...state, nextAttemptAt: due } } })
.where(eq(issueRecoveryActions.id, action!.id));
expect(await executionControlSchedule(db).workspaceRecovery()).toBe(due);
for (const status of ["cancelled", "resolved"] as const) {
await db.update(issueRecoveryActions).set({ status }).where(eq(issueRecoveryActions.id, action!.id));
expect(await nextWorkspaceRecoveryAt(db)).toBeNull();
expect(await retainedWorkspaceCandidates(db)).toEqual([]);
}
for (const system of [{ ...state, sourceRunId: randomUUID() }, { ...state, state: "complete" }, { ...state, state: "exhausted" }]) {
await db.update(issueRecoveryActions).set({ status: "active", evidence: { ...evidence, systemRecovery: system } })
.where(eq(issueRecoveryActions.id, action!.id));
expect(await nextWorkspaceRecoveryAt(db)).toBeNull();
}
await db.update(issueRecoveryActions).set({ evidence, cause: "agent_unreachable" })
.where(eq(issueRecoveryActions.id, action!.id));
expect(await nextWorkspaceRecoveryAt(db)).toBeNull();
await db.update(issueRecoveryActions).set({ cause: "native_workspace_sync_out_retry_exhausted",
evidence: { ...evidence, workspaceRecovery: { ...evidence.workspaceRecovery, runId: randomUUID() } } })
.where(eq(issueRecoveryActions.id, action!.id));
expect(await nextWorkspaceRecoveryAt(db)).toBeNull();
expect(await retainedWorkspaceCandidates(db)).toEqual([]);
await db.update(issueRecoveryActions).set({ status: "cancelled", evidence: { ...evidence,
systemRecovery: { ...state, state: "running" } } }).where(eq(issueRecoveryActions.id, action!.id));
const c = coordinator(db, undefined, error => { errors.push(error); });
const worker = registerWorkspaceRecoveryWork(observed, c,
{} as Parameters<typeof registerWorkspaceRecoveryWork>[2], async () => {});
try { await worker.ready; await quiet(c); } finally { await worker.stop(); }
});
it.each([1, 3, 7])("visits all %i paused Retry sources once and becomes quiet after file recovery", async count => {
const sources = [];
for (let index = 0; index < count; index++) sources.push(await pausedRecoveredSource());
const resume = vi.fn(heartbeatService(observed).resumeRemoteStopComments);
const c = coordinator(db, undefined, error => { errors.push(error); });
const worker = registerWorkspaceRecoveryWork(observed, c,
{} as Parameters<typeof registerWorkspaceRecoveryWork>[2], resume);
try {
await worker.ready;
for (let pass = 0; pass < 10 && resume.mock.calls.length < count; pass++) await settle();
expect(resume).toHaveBeenCalledTimes(count);
await quiet(c);
for (const result of resume.mock.results) {
await expect(result.value).rejects.toThrow("Agent is not invokable in its current state");
}
for (const { retry } of sources) {
const [saved] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, retry.requestId!));
expect(saved).toMatchObject({ status: "deferred_issue_execution", runId: null });
}
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, companyId))).toHaveLength(count);
expect((await db.select().from(agents).where(eq(agents.id, agentId)))[0]?.status).toBe("paused");
} finally { await c.stop(); await worker.stop(); }
});
it("visits a newly committed earlier Retry during a busy recovered-source pass, then becomes quiet", async () => {
const sources = [];
for (let index = 0; index < 3; index++) {
sources.push(await pausedRecoveredSource(`ffffffff-000${index}-4000-8000-${randomUUID().slice(-12)}`));
}
let release!: () => void;
const gate = new Promise<void>(resolve => { release = resolve; });
const heartbeat = heartbeatService(observed);
const resume = vi.fn(async (run: Parameters<typeof heartbeat.resumeRemoteStopComments>[0]) => {
await gate;
return heartbeat.resumeRemoteStopComments(run);
});
const c = coordinator(db, undefined, error => { errors.push(error); });
const worker = registerWorkspaceRecoveryWork(observed, c,
{} as Parameters<typeof registerWorkspaceRecoveryWork>[2], resume);
try {
await worker.ready;
await vi.waitFor(() => expect(resume).toHaveBeenCalledTimes(2));
const earlier = await pausedRecoveredSource(`00000000-0000-4000-8000-${randomUUID().slice(-12)}`);
sources.push(earlier);
release();
for (let pass = 0; pass < 10 && sources.some(({ run }) =>
!resume.mock.calls.some(([visited]) => visited.id === run.id)); pass++) await settle();
expect(resume.mock.calls.map(([run]) => run.id)).toContain(earlier.run.id);
await quiet(c);
for (const { run, retry } of sources) {
expect(resume.mock.calls.map(([visited]) => visited.id)).toContain(run.id);
const [saved] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, retry.requestId!));
expect(saved).toMatchObject({ status: "deferred_issue_execution", runId: null });
}
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, companyId))).toHaveLength(4);
expect((await db.select().from(agents).where(eq(agents.id, agentId)))[0]?.status).toBe("paused");
} finally { release(); await c.stop(); await worker.stop(); }
});
it("bounds real producer bursts with another company's completed history present", async () => {
const otherCompany = randomUUID(), otherAgent = randomUUID();
await db.insert(companies).values({ id: otherCompany, name: "Other tenant", issuePrefix: "OTH" });
server/src/index.ts
+5
View File
@@ -1,3 +1,4 @@
import { registerWorkspaceRecoveryWork } from "./services/workspace-recovery-work.js";
import { registerEnvironmentCleanupWork } from "./services/environment-cleanup-work.js";
import { loginCleanupSchedule } from "./services/login-cleanup-state.js";
import { registerLoginCleanupWork } from "./services/login-cleanup-work.js";
@@ -1342,6 +1343,9 @@ async function startServerWithDatabaseTeardown(
});
const deliveryWork = app.locals.deliveryWork as ReturnType<typeof createDeliveryWorkCoordinator>;
const executionSchedule = executionControlSchedule(db);
const workspaceRecoveryRuntime = environmentRuntimeService(db, { pluginWorkerManager });
const workspaceRecoveryWork = registerWorkspaceRecoveryWork(db, deliveryWork, workspaceRecoveryRuntime,
accountingHeartbeat.resumeRemoteStopComments, () => !heartbeatSchedulerStopped);
registerExecutionControlWork(deliveryWork, [
{ queue: DELIVERY_QUEUES.executionFinalization, run: () => reconcileAbandonedExecutionControl(db), next: executionSchedule.finalization },
{ queue: DELIVERY_QUEUES.executionStatus, run: () => deliverExecutionStatuses(db), next: executionSchedule.status },
@@ -2103,6 +2107,7 @@ async function startServerWithDatabaseTeardown(
await skillSourceSync.stop();
unsubscribeChatCompletions();
await deliveryWork.stop();
await workspaceRecoveryWork.stop();
unregisterTerminalWorkspaceProbe?.();
unsubscribeNativeWorkspaceSyncRetention?.();
if (heartbeatSchedulerInterval) {
server/src/services/agent-runner-selection.test.ts
+1 -1
View File
@@ -74,7 +74,7 @@ describe("server-owned Codex runner selection", () => {
const input = { adapterType: "codex_local", target, adapterConfig: { model: "gpt-5.6-sol", modelReasoningEffort: "high", env: { OPENAI_API_KEY: { type: "secret_ref", secretId: "secret" } }, cwd: "/workspace", instructionsFilePath: "/workspace/AGENTS.md", workspaceStrategy: { type: "git_worktree" }, timeoutSec: 60, search: false, fastMode: false, dangerouslyBypassApprovalsAndSandbox: true } };
const resolved = resolveNewAgentRunner(input);
expect(resolved).toMatchObject({ adapterType: "paperclip_runner", adapterConfig: { provider: "codex", codexPermissionMode: "never", modelReasoningEffort: "high", env: input.adapterConfig.env, workspaceStrategy: input.adapterConfig.workspaceStrategy, timeoutSec: 60 } });
expect(resolved.adapterConfig).not.toHaveProperty("search");
expect(resolved.adapterConfig).toHaveProperty("search", false);
expect(resolveNewAgentRunner(resolved)).toEqual(resolved);
expect(input.adapterConfig).toHaveProperty("search", false);
});
server/src/services/agent-runner-selection.ts
+1 -1
View File
@@ -270,7 +270,7 @@ function resolveAgentRunner(input: AgentRunnerSelection): { adapterType: string;
incompatible(["env.PAPERCLIP_CODEX_ACP_NETWORK_ACCESS"]);
}
const effort = config.modelReasoningEffort ?? config.reasoningEffort ?? config.effort;
for (const key of ["engine", "command", "extraArgs", "args", "agentCommand", "mode", "nonInteractivePermissions", "stateDir", "warmHandleIdleMs", "dangerouslyBypassApprovalsAndSandbox", "dangerouslyBypassSandbox", "search", "fastMode", "reasoningEffort", "effort"]) delete config[key];
for (const key of ["engine", "command", "extraArgs", "args", "agentCommand", "mode", "nonInteractivePermissions", "stateDir", "warmHandleIdleMs", "dangerouslyBypassApprovalsAndSandbox", "dangerouslyBypassSandbox", "fastMode", "reasoningEffort", "effort"]) delete config[key];
if (config.model !== undefined && typeof config.model !== "string") incompatible(["model"]);
config = { ...config, provider: "codex", model: typeof config.model === "string" && config.model.trim() ? config.model.trim() : PAPERCLIP_RUNNER_DEFAULT_MODELS.codex, codexPermissionMode: config.codexPermissionMode ?? "never", lifecycleMode: config.lifecycleMode ?? "per_turn", ...(effort === undefined ? {} : { modelReasoningEffort: effort }) };
try { resolvePaperclipRunnerNativeProviderInput({ backend: "codex_app_server", adapterConfig: config }); }
server/src/services/chat-channels.ts
+3 -15
View File
@@ -1,3 +1,4 @@
import { credentialFingerprint } from "./chat-channels/credential-fingerprint.js";
import { createChatSdkMessageReceiver } from "./chat-channels/runtime/sdk-messages.js";
import { admitDiscordSdkMessage } from "./chat-channels/discord/messages.js";
import { resolveInboundConversationBinding } from "./chat-channels/inbound/conversation-binding.js";
@@ -312,20 +313,6 @@ function runtimeGeneration(setup: ChatEndpointSetupState): number {
return Number.isSafeInteger(value) && value >= 0 ? value : 0;
}
function credentialFingerprint(refs: ToolCredentialSecretRef[]): string {
const stable = refs
.map((ref) => ({
configPath: ref.configPath,
secretId: ref.secretId,
versionSelector: ref.versionSelector ?? "latest",
}))
.sort((left, right) =>
`${left.configPath}:${left.secretId}:${left.versionSelector}`.localeCompare(
`${right.configPath}:${right.secretId}:${right.versionSelector}`,
),
);
return createHash("sha256").update(JSON.stringify(stable)).digest("hex");
}
type ConversationRow = typeof chatConversations.$inferSelect;
type DeliveryRow = typeof chatDeliveries.$inferSelect;
@@ -847,7 +834,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) {
const { lockTelegramCallbackNoticeAuthority } = createTelegramCallbackNoticeAuthority({ nonDirectDestinationAllowed, lockCurrentPrincipalAuthorization });
const { stageProviderEffect, processProviderEffect, processPendingProviderEffects } = createProviderEffects(db, { endpointRecord, runtimeCallbackEndpoint, withCredentialMutationLease, runtimeFor, invalidateRuntime, lockCurrentPrincipalAuthorization, redactError, MAX_ERROR_TEXT, nonDirectDestinationAllowed, lockTelegramCallbackNoticeAuthority });
const { publicationFiles } = createOutboundAttachments(db, { get storage() { return options.storage; } });
const { authorizeInboundWakeup, authorizeInboundWorkStart, authorizeFastResponse } = createInboundWakeAuthority(db, { runtimeCallbackEndpoint, lockCurrentPrincipalAuthorization });
const { authorizeInboundWakeup, authorizeGitHubTaskRetryBatch, authorizeInboundWorkStart, authorizeFastResponse } = createInboundWakeAuthority(db, { runtimeCallbackEndpoint, lockCurrentPrincipalAuthorization });
const { ingestAttachments, attachmentOmissionDetail } = createInboundAttachments(db, { authorizeInboundWakeup, lifecycleRuntimeFence, sanitizeFilename, issuesSvc, isExternalActionAuthorizationChange, redactError, get storage() { return options.storage; } });
const { stageInboundWakeup, acceptInboundWakeup, settleRejectedInboundWakeups, processInboundWakeup } = createInboundWakeups(db, { authorizeInboundWakeup, authorizeInboundWorkStart, attachmentOmissionDetail, isExternalActionAuthorizationChange, redactError, fetchImpl, get heartbeat() { return options.heartbeat; } });
const { settleTelegramAttachmentOnlyFailure } = createTelegramAttachmentFailure(db, {
@@ -1203,6 +1190,7 @@ export function chatChannelService(db: Db, options: ChatChannelServiceOptions) {
pendingFailedRetries,
} = createChatFailedRunRetry(db, {
authorizeInboundWakeup,
authorizeGitHubTaskRetryBatch,
isExternalActionAuthorizationChange,
inboundWakeScope,
lifecycleRuntimeFence,
server/src/services/chat-channels/credential-fingerprint.ts
+10
View File
@@ -0,0 +1,10 @@
import { createHash } from "node:crypto";
import type { ToolCredentialSecretRef } from "@paperclipai/shared";
export function credentialFingerprint(refs: ToolCredentialSecretRef[]): string {
const stable = refs.map(ref => ({ configPath: ref.configPath, secretId: ref.secretId,
versionSelector: ref.versionSelector ?? "latest" })).sort((left, right) =>
`${left.configPath}:${left.secretId}:${left.versionSelector}`.localeCompare(
`${right.configPath}:${right.secretId}:${right.versionSelector}`));
return createHash("sha256").update(JSON.stringify(stable)).digest("hex");
}
server/src/services/chat-channels/failed-run-retry.ts
+125 -81
View File
@@ -3,6 +3,8 @@ import { readTaskChatRetryIntent, taskChatRetryCanAwaitRecovery, taskChatRetryEx
import { notifyChatActionWork, notifyChatPublicationWork } from "../chat-work-notifications.js";
import { FAILED_RUN_RETRY_STALE_MS, dueChatAction } from "../chat-action-work.js";
import { readChatAuthorizationRun } from "./authorization-run-read.js";
import { selectTaskRetryInput, claimTaskRetryInput, settleTaskRetryInput, releaseUnadmittedTaskRetryInput, taskRetryInputSchema, type TaskRetryInput } from "./task-retry-input.js";
import type { TaskRetryWakeAuthority } from "./github/task-retry-authorization.js";
import {
createHash,
@@ -93,14 +95,16 @@ type RetryRuntimeFence = { credentialFingerprint: string; generation: number };
export type ChatFailedRetrySource = Awaited<ReturnType<ReturnType<typeof createChatFailedRunRetry>["failedChatRetrySource"]>>;
type ChatFailedRunRetryOptions = {
authorizeGitHubTaskRetryBatch: ReturnType<typeof import("./inbound/wakeup-authorization.js").createInboundWakeAuthority>["authorizeGitHubTaskRetryBatch"];
authorizeInboundWakeup: (
database: DbOrTransaction,
action: typeof chatActions.$inferSelect,
notice?: { nonblocking: true; terminal: boolean },
notice?: { nonblocking: true; terminal: boolean; taskRetry?: TaskRetryWakeAuthority },
) => Promise<{
conversation: Pick<typeof chatConversations.$inferSelect, "sessionGeneration">;
endpoint: Pick<typeof chatEndpoints.$inferSelect, "provider">;
delivery: DeliveryRow;
retryRuntimeFence?: RetryRuntimeFence;
}>;
isExternalActionAuthorizationChange: (error: unknown) => boolean;
inboundWakeScope: (action: typeof chatActions.$inferSelect) => string;
@@ -112,6 +116,7 @@ type ChatFailedRunRetryOptions = {
export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOptions) {
const {
authorizeInboundWakeup,
authorizeGitHubTaskRetryBatch,
isExternalActionAuthorizationChange,
inboundWakeScope,
lifecycleRuntimeFence,
@@ -131,14 +136,14 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
}
}
function retryCommentIds(context: Record<string, unknown>): string[] {
function retryCommentIds(context: Record<string, unknown>, taskRetryInput?: TaskRetryInput): string[] {
const ids =
context.wakeCommentIds ??
(context.wakeCommentId ? [context.wakeCommentId] : []);
if (
!Array.isArray(ids) ||
ids.length === 0 ||
ids.length > 50 ||
(!taskRetryInput && ids.length > 50) ||
ids.some((id) => typeof id !== "string" || !isUuidLike(id)) ||
new Set(ids).size !== ids.length ||
(context.wakeCommentId !== undefined &&
@@ -168,6 +173,7 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
failedRunId: string;
sourceScopeSha256: string;
}>;
taskRetryInput?: TaskRetryInput;
sourceScopeSha256: string;
};
function hashFailedRetrySource(
@@ -651,6 +657,8 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
runId?: string;
publication?: boolean;
taskRetryIntentId?: string;
taskRetryInput?: TaskRetryInput;
taskRetryReauthorization?: TaskRetryWakeAuthority;
/** Internal selector for already committed output, never retry admission. */
committedResponse?: {
resultId: string;
@@ -734,7 +742,22 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
(context.taskKey !== issue.identifier && context.taskKey !== issue.id)
)
throw failedChatRetryDenied();
const commentIds = retryCommentIds(context);
const selection = input.taskRetryInput && taskRetryInputSchema.safeParse(input.taskRetryInput);
if (selection && !selection.success) throw failedChatRetryDenied();
const selectedInput = selection?.success ? selection.data : undefined;
if (selectedInput && (!input.taskRetryIntentId || provider !== "github")) throw failedChatRetryDenied();
const reauthorization = input.taskRetryReauthorization ?? (selectedInput ? {
intentId: input.taskRetryIntentId!, sourceRunId: failedRun.id, runtimeFence: selectedInput.runtimeFence,
} : undefined);
const pendingSources = selectedInput?.sourceActionIds.length ? await tx.select().from(chatActions).where(and(
eq(chatActions.companyId, input.companyId), eq(chatActions.kind, "inbound_wakeup"),
sql`${chatActions.payload}->>'issueId' = ${issue.id}`,
inArray(chatActions.id, selectedInput.sourceActionIds),
)) : null;
if (pendingSources && pendingSources.length !== selectedInput!.sourceActionIds.length) throw failedChatRetryDenied();
const pendingById = new Map(pendingSources?.map(source => [source.id, source]));
const commentIds = pendingSources ? selectedInput!.sourceActionIds.map(id =>
String(pendingById.get(id)!.payload.commentId)) : retryCommentIds(context, selectedInput);
const [accepted] = await tx
.select()
.from(nativeRunResults)
@@ -868,7 +891,7 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
answerDelivery
)
throw failedChatRetryDenied();
if (context.chatFailedRunRetry) {
if (context.chatFailedRunRetry && !pendingSources) {
const hint = context.chatFailedRunRetry as Record<string, unknown>;
const [priorIntent] = await tx
.select()
@@ -914,6 +937,9 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
committedResponse: undefined,
taskRetryIntentId: typeof priorIntent.payload.taskRetryIntentId === "string"
? priorIntent.payload.taskRetryIntentId : undefined,
taskRetryInput: priorIntent.payload.taskRetryInput as TaskRetryInput | undefined,
retryActionId: priorIntent.id,
taskRetryReauthorization: reauthorization,
},
ancestors,
);
@@ -945,6 +971,7 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
return hashFailedRetrySource({
...parentScope,
failedRunId: failedRun.id,
...(selectedInput ? { taskRetryInput: { ...selectedInput, runtimeFence: reauthorization?.runtimeFence } } : {}),
retryAncestors: [
...parent.retryAncestors,
{
@@ -958,20 +985,21 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
// Include every receipt that fed this run, including a deferred owner's
// coalesced children. A mutable latest-comment hint cannot drop a sibling.
const admittedBatch =
await tx.execute(sql`select source.id, source.payload->>'commentId' as comment_id
pendingSources ? [] : await tx.execute(sql`select source.id, source.payload->>'commentId' as comment_id
from agent_wakeup_requests receipt join chat_actions source on source.id = receipt.id
where receipt.company_id = ${input.companyId}::uuid and receipt.agent_id = ${input.agentId}::uuid
and (receipt.run_id = ${failedRun.id}::uuid or receipt.id = ${failedRun.wakeupRequestId}::uuid
or receipt.payload->>'coalescedIntoWakeupRequestId' = ${failedRun.wakeupRequestId})
and source.company_id = ${input.companyId}::uuid and source.kind = 'inbound_wakeup' limit 51`);
and source.company_id = ${input.companyId}::uuid and source.kind = 'inbound_wakeup'
${selectedInput ? sql`` : sql`limit 51`}`);
if (
admittedBatch.length !== commentIds.length ||
!pendingSources && (admittedBatch.length !== commentIds.length ||
admittedBatch.some(
(entry) => !commentIds.includes(String(entry.comment_id)),
)
))
)
throw failedChatRetryDenied();
const sources = await tx
const sources = pendingSources ?? await tx
.select()
.from(chatActions)
.where(
@@ -986,8 +1014,9 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
),
);
if (sources.length !== commentIds.length) throw failedChatRetryDenied();
const sourcesByCommentId = new Map(sources.map(source => [source.payload.commentId, source]));
const ordered = commentIds.map(
(id) => sources.find((action) => action.payload.commentId === id)!,
(id) => sourcesByCommentId.get(id)!,
);
const first = ordered[0]!;
if (
@@ -1011,39 +1040,53 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
);
if (!identity[0]?.locked)
throw new NativeChatReviewPresentationContentionError();
const actionIds = ordered.map(action => action.id);
const currentActions = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, input.companyId),
inArray(chatActions.id, actionIds))).orderBy(asc(chatActions.id)).for("update", { noWait: true });
const deliveries = await tx.select().from(chatDeliveries).where(and(eq(chatDeliveries.companyId, input.companyId),
inArray(chatDeliveries.id, ordered.map(action => action.deliveryId!))))
.orderBy(asc(chatDeliveries.id)).for("update", { noWait: true });
const comments = await tx.select().from(issueComments).where(and(eq(issueComments.companyId, input.companyId),
eq(issueComments.issueId, issue.id), inArray(issueComments.id, commentIds)))
.orderBy(asc(issueComments.id)).for("share", { noWait: true });
const receipts = await tx.select().from(agentWakeupRequests).where(and(eq(agentWakeupRequests.companyId, input.companyId),
inArray(agentWakeupRequests.id, [...actionIds, failedRun.wakeupRequestId!, ...(input.retryActionId ? [input.retryActionId] : [])])))
.orderBy(asc(agentWakeupRequests.id)).for("share", { noWait: true });
const actionsById = new Map(currentActions.map(action => [action.id, action]));
const deliveriesById = new Map(deliveries.map(delivery => [delivery.id, delivery]));
const commentsById = new Map(comments.map(comment => [comment.id, comment]));
const receiptsById = new Map(receipts.map(receipt => [receipt.id, receipt]));
const lifecycleRows = await tx.select({
target: sql<string>`${chatDeliveries.normalizedEvent}->'message'->>'targetProviderEventId'`,
context: sql<Record<string, unknown>>`${chatDeliveries.normalizedEvent}->'runtimeContext'`,
}).from(chatDeliveries).where(and(eq(chatDeliveries.companyId, input.companyId),
eq(chatDeliveries.endpointId, first.endpointId),
or(isNull(chatDeliveries.conversationId), eq(chatDeliveries.conversationId, first.conversationId)),
ne(chatDeliveries.state, "filtered"), inArray(chatDeliveries.eventKind, ["message_updated", "message_deleted", "message_restored"]),
inArray(sql<string>`${chatDeliveries.normalizedEvent}->'message'->>'targetProviderEventId'`, deliveries.map(delivery => delivery.providerEventId))));
const lifecycleKey = (target: string, context: Record<string, unknown> | null) =>
JSON.stringify([target, Object.entries(context ?? {}).sort(([left], [right]) => left.localeCompare(right))]);
const invalidated = new Set(lifecycleRows.map(row => lifecycleKey(row.target, row.context)));
const batch = reauthorization ? await authorizeGitHubTaskRetryBatch(tx, ordered, {
nonblocking: true, terminal: input.publication === true, taskRetry: reauthorization,
}).catch(error => { if (isExternalActionAuthorizationChange(error)) throw failedChatRetryDenied(); throw error; }) : null;
const canonicalSources: Array<Record<string, unknown>> = [];
let destination: Awaited<ReturnType<typeof authorizeInboundWakeup>> | null =
null;
for (const action of ordered) {
destination = await authorizeInboundWakeup(tx, action, {
destination = batch?.get(action.id) ?? await authorizeInboundWakeup(tx, action, {
nonblocking: true,
terminal: input.publication === true,
...(reauthorization ? { taskRetry: reauthorization } : {}),
}).catch((error) => {
if (isExternalActionAuthorizationChange(error))
throw failedChatRetryDenied();
throw error;
});
const [delivery] = await tx
.select()
.from(chatDeliveries)
.where(eq(chatDeliveries.id, action.deliveryId!))
.for("update", { noWait: true });
const [current] = await tx
.select()
.from(chatActions)
.where(eq(chatActions.id, action.id))
.for("update", { noWait: true });
const [comment] = await tx
.select()
.from(issueComments)
.where(
and(
eq(issueComments.id, String(action.payload.commentId)),
eq(issueComments.companyId, input.companyId),
eq(issueComments.issueId, issue.id),
),
)
.for("share", { noWait: true });
if (reauthorization && !reauthorization.runtimeFence) reauthorization.runtimeFence = destination.retryRuntimeFence;
const delivery = deliveriesById.get(action.deliveryId!);
const current = actionsById.get(action.id);
const comment = commentsById.get(String(action.payload.commentId));
if (
!current ||
current.status !== "processed" ||
@@ -1053,43 +1096,17 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
!comment ||
comment.deletedAt ||
comment.updatedAt.getTime() !== comment.createdAt.getTime() ||
comment.updatedAt >
(!pendingSources && comment.updatedAt >
(input.committedResponse
? accepted!.createdAt
: failedRun.finishedAt!) ||
: failedRun.finishedAt!)) ||
destination.endpoint.provider !== provider ||
JSON.stringify(lifecycleRuntimeFence(delivery)) !==
JSON.stringify(lifecycleRuntimeFence(destination.delivery))
)
throw failedChatRetryDenied();
const [lifecycle] = await tx
.select({ id: chatDeliveries.id })
.from(chatDeliveries)
.where(
and(
eq(chatDeliveries.companyId, input.companyId),
eq(chatDeliveries.endpointId, first.endpointId),
or(
isNull(chatDeliveries.conversationId),
eq(chatDeliveries.conversationId, first.conversationId),
),
ne(chatDeliveries.state, "filtered"),
sql`${chatDeliveries.normalizedEvent}->'runtimeContext' = ${JSON.stringify(lifecycleRuntimeFence(delivery))}::jsonb`,
inArray(chatDeliveries.eventKind, [
"message_updated",
"message_deleted",
"message_restored",
]),
sql`${chatDeliveries.normalizedEvent}->'message'->>'targetProviderEventId' = ${delivery.providerEventId}`,
),
)
.limit(1);
if (lifecycle) throw failedChatRetryDenied();
const [receipt] = await tx
.select()
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, action.id))
.for("share", { noWait: true });
if (invalidated.has(lifecycleKey(delivery.providerEventId, lifecycleRuntimeFence(delivery)))) throw failedChatRetryDenied();
const receipt = receiptsById.get(action.id);
if (!receipt) throw failedChatRetryDenied();
assertFailedRetryReceipt(
createDurableChatWakeupRequest({
@@ -1107,6 +1124,17 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
}),
receipt,
);
if (pendingSources) {
const direct = receipt.status === "deferred_issue_execution" && !receipt.runId
&& receipt.payload?.coalescedIntoWakeupRequestId === undefined;
const owned = input.retryActionId && receipt.status === "coalesced"
&& receipt.payload?.coalescedIntoWakeupRequestId === input.retryActionId;
if (!direct && !owned) throw failedChatRetryDenied();
if (owned && receipt.runId) {
const owner = receiptsById.get(input.retryActionId!);
if (owner?.runId !== receipt.runId) throw failedChatRetryDenied();
}
}
let owner = receipt;
const seen = new Set<string>();
for (
@@ -1126,25 +1154,16 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
)
throw failedChatRetryDenied();
seen.add(ownerId);
const [next] = await tx
.select()
.from(agentWakeupRequests)
.where(
and(
eq(agentWakeupRequests.id, ownerId),
eq(agentWakeupRequests.companyId, input.companyId),
),
)
.for("share", { noWait: true });
const next = receiptsById.get(ownerId);
if (!next) throw failedChatRetryDenied();
owner = next;
}
if (
owner.id !== failedRun.wakeupRequestId ||
(!pendingSources && (owner.id !== failedRun.wakeupRequestId ||
owner.runId !== failedRun.id ||
owner.agentId !== input.agentId ||
owner.requestedByActorType !== receipt.requestedByActorType ||
owner.requestedByActorId !== receipt.requestedByActorId ||
owner.requestedByActorId !== receipt.requestedByActorId)) ||
["skipped", "cancelled"].includes(receipt.status)
)
throw failedChatRetryDenied();
@@ -1158,7 +1177,7 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
bodySha256: createHash("sha256").update(comment.body).digest("hex"),
omissionReasons: action.payload.attachmentOmissionReasons ?? {},
receiptId: receipt.id,
ownerId: owner.id,
ownerId: pendingSources ? action.id : owner.id,
});
}
if (!destination) throw failedChatRetryDenied();
@@ -1204,6 +1223,8 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
| "system",
requestedByActorId: String(first.payload.requestedByActorId),
retryAncestors: [],
...(selectedInput ? { taskRetryInput: { ...selectedInput,
runtimeFence: selectedInput.runtimeFence ?? destination.retryRuntimeFence } } : {}),
};
return hashFailedRetrySource(scope);
}
@@ -1240,7 +1261,8 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
retryIntentId?: string;
},
): Promise<{ actionId: string; issueId: string }> {
const source = await failedChatRetrySource(tx, { ...input, taskRetryIntentId: input.retryIntentId });
const taskRetryInput = input.retryIntentId ? await selectTaskRetryInput(tx, input) : undefined;
const source = await failedChatRetrySource(tx, { ...input, taskRetryIntentId: input.retryIntentId, taskRetryInput });
const providerActionId = input.retryIntentId
? `task-retry:${input.retryIntentId}` : `failed-run-retry:${source.failedRunId}`;
const [existing] = await tx
@@ -1351,6 +1373,7 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
failedRunId: snapshot.payload.failedRunId,
taskRetryIntentId: typeof snapshot.payload.taskRetryIntentId === "string"
? snapshot.payload.taskRetryIntentId : undefined,
taskRetryInput: snapshot.payload.taskRetryInput as TaskRetryInput | undefined,
});
if (typeof snapshot.payload.taskRetryIntentId === "string" && !await readTaskChatRetryIntent(tx as unknown as Db, {
companyId: input.companyId, issueId: input.issueId, agentId: input.agentId,
@@ -1444,7 +1467,7 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
context.source !== `chat:${source.provider}` ||
context.taskKey !== source.taskKey ||
context.retryOfRunId !== source.failedRunId ||
JSON.stringify(retryCommentIds(context)) !==
JSON.stringify(retryCommentIds(context, source.taskRetryInput)) !==
JSON.stringify(source.commentIds) ||
hint?.version !== 1 ||
hint.actionId !== source.intent.id ||
@@ -1518,8 +1541,19 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
};
}
async function reconcileFailedTaskRetry(action: typeof chatActions.$inferSelect) {
if (!["failed", "cancelled"].includes(action.status) || !action.payload.taskRetryIntentId ||
action.result?.taskRetryInputReconciled === true) return action;
await releaseUnadmittedTaskRetryInput(db, { companyId: action.companyId, retryActionId: action.id });
const [settled] = await db.update(chatActions).set({
result: sql`coalesce(${chatActions.result}, '{}'::jsonb) || '{"taskRetryInputReconciled":true}'::jsonb`,
}).where(and(eq(chatActions.companyId, action.companyId), eq(chatActions.id, action.id),
inArray(chatActions.status, ["failed", "cancelled"]))).returning();
return settled ?? action;
}
async function claimFailedChatRunRetry(actionId: string) {
const [candidate] = await db
const [snapshot] = await db
.select()
.from(chatActions)
.where(
@@ -1528,7 +1562,10 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
eq(chatActions.kind, "failed_run_retry"),
),
);
if (!candidate) throw notFound("Chat retry request not found");
if (!snapshot) throw notFound("Chat retry request not found");
// Reconcile once at settlement; a crash before the marker is safely
// retried here. Status projection itself performs no writes or row locks.
const candidate = await reconcileFailedTaskRetry(snapshot);
const now = new Date();
if (
isShuttingDown() ||
@@ -1571,7 +1608,7 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
)
.returning();
});
return failedChatRetryReceipt(current ?? claimed);
return failedChatRetryReceipt(await reconcileFailedTaskRetry(current ?? claimed));
};
try {
const scope = {
@@ -1644,6 +1681,12 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
true, // The durable wake callback validates admission; later phases recheck containment.
);
},
onAdmitted: async (tx, receipt) => {
if (!source.taskRetryInput) return;
await claimTaskRetryInput(tx, { companyId: claimed.companyId,
retryActionId: claimed.id, sourceActionIds: source.taskRetryInput.sourceActionIds });
await settleTaskRetryInput(tx, { companyId: claimed.companyId, retryActionId: claimed.id, runId: receipt.runId });
},
}),
});
}
@@ -1652,6 +1695,7 @@ export function createChatFailedRunRetry(db: Db, options: ChatFailedRunRetryOpti
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, claimed.id));
if (!receipt) throw new Error("chat_retry_receipt_not_committed");
await settleTaskRetryInput(db, { companyId: claimed.companyId, retryActionId: claimed.id, runId: receipt.runId });
return settle(
["skipped", "failed", "cancelled"].includes(receipt.status)
? "failed"
server/src/services/chat-channels/github/task-preparation.ts
+1 -1
View File
@@ -37,7 +37,7 @@ export async function retainGitHubDeliveryAuthority(
linkedUserId: string | null, taskUserId: string | null,
): Promise<void> {
const guest = !linkedUserId;
await taskTx.update(chatDeliveries).set({ normalizedEvent: sql`${chatDeliveries.normalizedEvent} || ${JSON.stringify({ githubAuthority: { guest, responsibleUserId: taskUserId, sponsorUserId: guest ? taskEndpoint.sponsorUserId : null } })}::jsonb` }).where(eq(chatDeliveries.id, deliveryId));
await taskTx.update(chatDeliveries).set({ normalizedEvent: sql`${chatDeliveries.normalizedEvent} || ${JSON.stringify({ githubAuthority: { guest, botExternalId: taskEndpoint.botExternalId, responsibleUserId: taskUserId, sponsorUserId: guest ? taskEndpoint.sponsorUserId : null } })}::jsonb` }).where(eq(chatDeliveries.id, deliveryId));
}
export async function retainGitHubReviewReceipt(taskTx: DbOrTransaction, endpoint: EndpointRow, issueId: string, deliveryId: string, githubAutomatic: AutomaticRequest): Promise<void> {
server/src/services/chat-channels/github/task-retry-authorization.ts
+126
View File
@@ -0,0 +1,126 @@
import { and, eq, inArray } from "drizzle-orm";
import { chatActions, chatConversations, chatDeliveries, chatEndpointResources, chatMessageLinks, issueComments, toolConnections } from "@paperclipai/db";
import { forbidden } from "../../../errors.js";
import type { DbOrTransaction, EndpointRow, LifecycleRuntimeFence } from "../runtime-types.js";
import { credentialFingerprint } from "../credential-fingerprint.js";
import { readTaskChatRetryIntent } from "../task-retry-authority.js";
export type TaskRetryWakeAuthority = {
intentId: string;
sourceRunId: string;
runtimeFence?: LifecycleRuntimeFence;
};
export function githubTaskRetryAppMatches(endpoint: EndpointRow, delivery: typeof chatDeliveries.$inferSelect,
response: { payload: Record<string, unknown> } | undefined) {
const original = delivery.normalizedEvent.githubAuthority as { botExternalId?: unknown } | undefined;
return original?.botExternalId !== undefined ? original.botExternalId === endpoint.botExternalId
: response?.payload.botExternalId === endpoint.botExternalId && response.payload.agentId === endpoint.assignedAgentId;
}
export function githubTaskRetryDestinationMatches(delivery: typeof chatDeliveries.$inferSelect,
conversation: typeof chatConversations.$inferSelect, resource: typeof chatEndpointResources.$inferSelect | null) {
const originalResource = delivery.normalizedEvent.resource as { providerResourceId?: unknown } | undefined;
const originalThread = delivery.normalizedEvent.conversation as { externalThreadId?: unknown } | undefined;
const automatic = (delivery.normalizedEvent.githubAutomatic ?? delivery.normalizedEvent.githubIssue) as
{ context?: { repositoryId?: unknown } } | undefined;
return resource && originalResource?.providerResourceId === resource.providerResourceId
&& originalThread?.externalThreadId === conversation.externalThreadId
&& (automatic?.context?.repositoryId === undefined
|| automatic.context.repositoryId === resource.metadata.providerRepositoryId);
}
export async function githubTaskRetryRuntimeFence(tx: DbOrTransaction, endpoint: EndpointRow,
action: typeof chatActions.$inferSelect, delivery: typeof chatDeliveries.$inferSelect,
authority: TaskRetryWakeAuthority): Promise<LifecycleRuntimeFence> {
const deny = () => forbidden("This accepted chat message is no longer authorized to start work",
{ code: "chat_action_authorization_changed" });
const retry = await readTaskChatRetryIntent(tx as Parameters<typeof readTaskChatRetryIntent>[0], {
companyId: action.companyId, issueId: String(action.payload.issueId),
agentId: String(action.payload.agentId), sourceRunId: authority.sourceRunId, intentId: authority.intentId,
});
if (!retry || endpoint.provider !== "github" || !endpoint.botExternalId || action.kind !== "inbound_wakeup"
|| action.status !== "processed" || delivery.state !== "processed") throw deny();
const [response] = await tx.select({ payload: chatActions.payload }).from(chatActions).where(and(
eq(chatActions.companyId, action.companyId), eq(chatActions.endpointId, endpoint.id),
eq(chatActions.conversationId, action.conversationId!), eq(chatActions.deliveryId, delivery.id),
eq(chatActions.kind, "github_response_comment"),
));
if (!githubTaskRetryAppMatches(endpoint, delivery, response)) throw deny();
const [connection] = await tx.select({ refs: toolConnections.credentialSecretRefs,
enabled: toolConnections.enabled, status: toolConnections.status }).from(toolConnections).where(and(
eq(toolConnections.companyId, endpoint.companyId), eq(toolConnections.id, endpoint.connectionId),
));
if (!connection?.enabled || connection.status !== "active") throw deny();
const generation = Number((endpoint.setup as { runtimeGeneration?: unknown }).runtimeGeneration ?? 0);
if (!Number.isSafeInteger(generation) || generation < 0) throw deny();
return authority.runtimeFence ?? { generation, credentialFingerprint: credentialFingerprint(connection.refs ?? []) };
}
type GitHubRetryBinding = {
endpoint: EndpointRow;
conversation: typeof chatConversations.$inferSelect;
resource: typeof chatEndpointResources.$inferSelect | null;
delivery: typeof chatDeliveries.$inferSelect;
retryRuntimeFence?: LifecycleRuntimeFence;
};
export function createGitHubTaskRetryBatchAuthority<T extends GitHubRetryBinding>(authorize: (
tx: DbOrTransaction, action: typeof chatActions.$inferSelect,
notice: { nonblocking: true; terminal: boolean; taskRetry: TaskRetryWakeAuthority },
) => Promise<T>) {
async function authorizeGitHubTaskRetryBatch(tx: DbOrTransaction, actions: Array<typeof chatActions.$inferSelect>,
notice: { nonblocking: true; terminal: boolean; taskRetry: TaskRetryWakeAuthority }) {
const deny = () => forbidden("This accepted chat message is no longer authorized to start work",
{ code: "chat_action_authorization_changed" });
const first = actions[0];
if (!first) throw deny();
const commonScope = (action: typeof chatActions.$inferSelect) => JSON.stringify([
action.companyId, action.endpointId, action.conversationId, action.principalId, action.kind, action.status,
...["version", "issueId", "agentId", "sessionGeneration", "requestedByActorType", "requestedByActorId"]
.map(key => action.payload[key]),
]);
if (actions.some(action => commonScope(action) !== commonScope(first))) throw deny();
const deliveries = await tx.select().from(chatDeliveries).where(and(eq(chatDeliveries.companyId, first.companyId),
inArray(chatDeliveries.id, actions.map(action => action.deliveryId!))));
const byDelivery = new Map(deliveries.map(delivery => [delivery.id, delivery]));
const legacy = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, first.companyId),
eq(chatActions.endpointId, first.endpointId), eq(chatActions.conversationId, first.conversationId!),
eq(chatActions.kind, "github_response_comment"), inArray(chatActions.deliveryId, deliveries.map(delivery => delivery.id))));
const legacyByDelivery = new Map(legacy.map(action => [action.deliveryId, action]));
const links = await tx.select({ deliveryId: chatMessageLinks.deliveryId, commentId: chatMessageLinks.commentId,
authorUserId: issueComments.authorUserId, deletedAt: issueComments.deletedAt }).from(chatMessageLinks)
.innerJoin(issueComments, and(eq(issueComments.id, chatMessageLinks.commentId),
eq(issueComments.companyId, first.companyId), eq(issueComments.issueId, String(first.payload.issueId))))
.where(and(eq(chatMessageLinks.companyId, first.companyId), eq(chatMessageLinks.endpointId, first.endpointId),
eq(chatMessageLinks.conversationId, first.conversationId!), eq(chatMessageLinks.direction, "inbound"),
inArray(chatMessageLinks.deliveryId, deliveries.map(delivery => delivery.id))));
const expectedUserId = first.payload.requestedByActorType === "user" ? first.payload.requestedByActorId : null;
const linked = new Set(links.filter(link => (!link.deletedAt || notice.terminal) && link.authorUserId === expectedUserId)
.map(link => `${link.deliveryId}:${link.commentId}`));
const groups = new Map<string, T>();
const authorized = new Map<string, T>();
for (const action of actions) {
const delivery = byDelivery.get(action.deliveryId!);
if (!delivery || !linked.has(`${delivery.id}:${action.payload.commentId}`)) throw deny();
// Reuse only the same live grant within this transaction. Automatic review
// policies with different contexts still receive their own full check.
const grant = JSON.stringify(delivery.normalizedEvent.githubAutomatic ?? delivery.normalizedEvent.githubIssue ?? null);
let current = groups.get(grant);
if (!current) {
current = await authorize(tx, action, notice);
if (current.endpoint.provider !== "github") throw deny();
notice.taskRetry.runtimeFence ??= current.retryRuntimeFence;
groups.set(grant, current);
}
if (delivery.state !== "processed" || delivery.endpointId !== current.endpoint.id ||
delivery.conversationId !== current.conversation.id || delivery.principalId !== action.principalId ||
!githubTaskRetryAppMatches(current.endpoint, delivery, legacyByDelivery.get(delivery.id)) ||
!githubTaskRetryDestinationMatches(delivery, current.conversation, current.resource)) throw deny();
authorized.set(action.id, { ...current, delivery });
}
return authorized;
}
return authorizeGitHubTaskRetryBatch;
}
server/src/services/chat-channels/inbound/wakeup-authorization.ts
+12 -5
View File
@@ -11,6 +11,7 @@ import { agentWakeupRequests, agents, chatActions, chatConversations, chatDelive
import { forbidden } from "../../../errors.js";
import { resolveGitHubWakeAuthority, isGitHubAutomaticDelivery } from "../github/wake-authority.js";
import { assertSlackWakeMention } from "../slack/wake-authority.js";
import { githubTaskRetryRuntimeFence, githubTaskRetryDestinationMatches, createGitHubTaskRetryBatchAuthority, type TaskRetryWakeAuthority } from "../github/task-retry-authorization.js";
import type { createChatInboundAuthority } from "./authorization.js";
import type { createChatMessageLifecycle } from "./lifecycle.js";
type WakeAuthorityOptions = {
@@ -41,7 +42,7 @@ export function createInboundWakeAuthority(db: Db, options: WakeAuthorityOptions
async function authorizeInboundWakeup(
tx: DbOrTransaction,
action: typeof chatActions.$inferSelect,
notice?: { nonblocking: true; terminal: boolean },
notice?: { nonblocking: true; terminal: boolean; taskRetry?: TaskRetryWakeAuthority },
) {
const payload = action.payload;
const deny = () =>
@@ -106,6 +107,8 @@ export function createInboundWakeAuthority(db: Db, options: WakeAuthorityOptions
.limit(1)
.then((rows) => rows[0] ?? null);
const fence = delivery ? lifecycleRuntimeFence(delivery) : null;
const currentFence = endpoint && delivery && notice?.taskRetry
? await githubTaskRetryRuntimeFence(tx, endpoint, action, delivery, notice.taskRetry) : fence;
if (
!endpoint ||
endpoint.assignedAgentId !== payload.agentId ||
@@ -114,8 +117,8 @@ export function createInboundWakeAuthority(db: Db, options: WakeAuthorityOptions
delivery.endpointId !== endpoint.id ||
delivery.conversationId !== action.conversationId ||
delivery.principalId !== action.principalId ||
!fence ||
!(await runtimeCallbackEndpoint(tx as unknown as DbTransaction, endpoint.id, fence, [
!currentFence ||
!(await runtimeCallbackEndpoint(tx as unknown as DbTransaction, endpoint.id, currentFence, [
"verifying",
"active",
]))
@@ -160,6 +163,7 @@ export function createInboundWakeAuthority(db: Db, options: WakeAuthorityOptions
endpoint,
action.principalId,
);
if (notice?.taskRetry && !githubTaskRetryDestinationMatches(delivery, conversation, resource)) throw deny();
if (endpoint.provider === "github") {
const automatic = await resolveGitHubWakeAuthority(tx, endpoint, delivery);
if (automatic.kind === "denied") throw deny();
@@ -214,9 +218,12 @@ export function createInboundWakeAuthority(db: Db, options: WakeAuthorityOptions
source.authorUserId !== expectedUserId
)
throw deny();
return { issue, delivery, endpoint, conversation };
return { issue, delivery, endpoint, conversation, resource,
...(notice?.taskRetry && currentFence ? { retryRuntimeFence: currentFence } : {}) };
}
const authorizeGitHubTaskRetryBatch = createGitHubTaskRetryBatchAuthority(authorizeInboundWakeup);
async function authorizeInboundWorkStart(
tx: DbOrTransaction,
action: typeof chatActions.$inferSelect,
@@ -241,5 +248,5 @@ export function createInboundWakeAuthority(db: Db, options: WakeAuthorityOptions
attachments: normalized.message?.attachments?.map(a => `${a.name ?? "Attachment"} (${a.mimeType ?? "unknown type"})`)
};
}
return { authorizeInboundWakeup, authorizeInboundWorkStart, authorizeFastResponse };
return { authorizeInboundWakeup, authorizeGitHubTaskRetryBatch, authorizeInboundWorkStart, authorizeFastResponse };
}
server/src/services/chat-channels/task-retry-input.ts
+95
View File
@@ -0,0 +1,95 @@
import { z } from "zod";
import { and, asc, eq, inArray, isNull, sql } from "drizzle-orm";
import { agentWakeupRequests, chatActions, chatConversations, chatEndpoints, heartbeatRuns, issues, type Db } from "@paperclipai/db";
import type { DbOrTransaction } from "./runtime-types.js";
export const taskRetryInputSchema = z.object({
// This is a server-selected snapshot of the durable queue, not client input.
// Truncating it strands older requests or makes the newer-input guard reject
// every Retry. Dispatch already carries the full queue with bounded inline text.
sourceActionIds: z.array(z.string().uuid()).refine(ids => new Set(ids).size === ids.length),
runtimeFence: z.object({ generation: z.number().int().nonnegative(),
credentialFingerprint: z.string().regex(/^[a-f0-9]{64}$/) }).optional(),
});
export type TaskRetryInput = z.infer<typeof taskRetryInputSchema>;
export async function selectTaskRetryInput(tx: DbOrTransaction, input: {
companyId: string; issueId: string; agentId: string; failedRunId: string;
}) {
const [conversation] = await tx.select({ id: chatConversations.id }).from(chatConversations)
.innerJoin(chatEndpoints, eq(chatEndpoints.id, chatConversations.endpointId))
.innerJoin(chatActions, and(eq(chatActions.companyId, chatConversations.companyId),
eq(chatActions.conversationId, chatConversations.id)))
.innerJoin(heartbeatRuns, and(eq(heartbeatRuns.companyId, chatConversations.companyId),
eq(heartbeatRuns.wakeupRequestId, chatActions.id))).where(and(
eq(chatConversations.companyId, input.companyId), eq(chatConversations.issueId, input.issueId),
eq(heartbeatRuns.id, input.failedRunId), eq(heartbeatRuns.agentId, input.agentId),
eq(chatEndpoints.provider, "github"), eq(chatEndpoints.assignedAgentId, input.agentId),
eq(chatEndpoints.externalExecutionPolicy, "restricted"),
)).limit(1);
if (!conversation) return undefined;
const pending = await tx.select({ id: chatActions.id }).from(chatActions)
.innerJoin(agentWakeupRequests, and(eq(agentWakeupRequests.companyId, chatActions.companyId),
eq(agentWakeupRequests.id, chatActions.id))).where(and(
eq(chatActions.companyId, input.companyId), eq(chatActions.conversationId, conversation.id),
eq(chatActions.kind, "inbound_wakeup"), eq(chatActions.status, "processed"),
eq(agentWakeupRequests.agentId, input.agentId), eq(agentWakeupRequests.status, "deferred_issue_execution"),
isNull(agentWakeupRequests.runId),
)).orderBy(asc(chatActions.createdAt), asc(chatActions.id)).for("update", { of: agentWakeupRequests });
return taskRetryInputSchema.parse({ sourceActionIds: pending.map(row => row.id) });
}
export async function claimTaskRetryInput(tx: DbOrTransaction, input: {
companyId: string; retryActionId: string; sourceActionIds: string[];
}) {
if (!input.sourceActionIds.length) return;
await tx.update(agentWakeupRequests).set({ status: "coalesced", updatedAt: new Date(),
payload: sql`coalesce(${agentWakeupRequests.payload}, '{}'::jsonb) || ${JSON.stringify({
coalescedIntoWakeupRequestId: input.retryActionId })}::jsonb`,
}).where(and(eq(agentWakeupRequests.companyId, input.companyId),
inArray(agentWakeupRequests.id, input.sourceActionIds), eq(agentWakeupRequests.status, "deferred_issue_execution"),
isNull(agentWakeupRequests.runId)));
}
export async function settleTaskRetryInput(tx: DbOrTransaction, input: { companyId: string; retryActionId: string; runId: string | null }) {
if (!input.runId) return;
await tx.update(agentWakeupRequests).set({ runId: input.runId, updatedAt: new Date() }).where(and(
eq(agentWakeupRequests.companyId, input.companyId), eq(agentWakeupRequests.status, "coalesced"),
isNull(agentWakeupRequests.runId),
sql`${agentWakeupRequests.payload}->>'coalescedIntoWakeupRequestId' = ${input.retryActionId}`,
));
}
export async function releaseUnadmittedTaskRetryInput(db: Db, input: { companyId: string; retryActionId: string }) {
await db.transaction(async tx => {
const [snapshot] = await tx.select({ payload: chatActions.payload }).from(chatActions).where(and(
eq(chatActions.companyId, input.companyId), eq(chatActions.id, input.retryActionId),
));
const issueId = z.string().uuid().safeParse(snapshot?.payload.issueId);
if (!issueId.success) return;
const [issue] = await tx.select({ id: issues.id }).from(issues).where(and(
eq(issues.companyId, input.companyId), eq(issues.id, issueId.data),
)).for("update");
if (!issue) return;
const [action] = await tx.select().from(chatActions).where(and(eq(chatActions.companyId, input.companyId),
eq(chatActions.id, input.retryActionId), eq(chatActions.kind, "failed_run_retry"),
sql`${chatActions.payload}->>'issueId' = ${issue.id}`,
inArray(chatActions.status, ["failed", "cancelled"]))).for("update");
if (!action || typeof action.payload.taskRetryIntentId !== "string") return;
const [receipt] = await tx.select({ runId: agentWakeupRequests.runId }).from(agentWakeupRequests).where(and(
eq(agentWakeupRequests.companyId, input.companyId), eq(agentWakeupRequests.id, input.retryActionId),
)).for("update");
if (receipt?.runId) return;
await tx.update(agentWakeupRequests).set({ status: "deferred_issue_execution", finishedAt: null,
payload: sql`${agentWakeupRequests.payload} - 'coalescedIntoWakeupRequestId'`, updatedAt: new Date(),
}).where(and(eq(agentWakeupRequests.companyId, input.companyId), eq(agentWakeupRequests.status, "coalesced"),
isNull(agentWakeupRequests.runId),
sql`${agentWakeupRequests.payload}->>'coalescedIntoWakeupRequestId' = ${input.retryActionId}`));
await tx.update(agentWakeupRequests).set({ status: "failed", finishedAt: new Date(), updatedAt: new Date() }).where(and(
eq(agentWakeupRequests.companyId, input.companyId), eq(agentWakeupRequests.id, action.payload.taskRetryIntentId),
eq(agentWakeupRequests.reason, "task_retry"), isNull(agentWakeupRequests.runId),
inArray(agentWakeupRequests.status, ["deferred_issue_execution", "queued", "claimed"]),
sql`${agentWakeupRequests.payload}#>>'{taskRetry,chatActionId}' = ${action.id}`,
));
});
}
server/src/services/delivery-work-notifications.ts
+1
View File
@@ -3,6 +3,7 @@ import { signalDatabaseWork, subscribeDatabaseWork } from "@paperclipai/db";
/** Topics for existing durable queues; register intent before their writes. */
export const DELIVERY_QUEUES = {
executionFinalization: "execution-finalization",
workspaceRecovery: "workspace-recovery",
executionReplacement: "execution-replacement",
executionContinuation: "execution-continuation",
executionStatus: "execution-status",
server/src/services/durable-chat-wakeup.ts
+22 -1
View File
@@ -1,6 +1,6 @@
import { createHash } from "node:crypto";
import { agentWakeupRequests, chatActions, chatDeliveries, chatEndpoints, type Db } from "@paperclipai/db";
import { and, eq, or, sql, type SQLWrapper } from "drizzle-orm";
import { and, eq, inArray, or, sql, type SQLWrapper } from "drizzle-orm";
import { HttpError } from "../errors.js";
export interface CommittedChatResponseAuthorizationInput {
@@ -211,6 +211,7 @@ export interface DurableChatWakeupRequest {
readonly idempotencyKey: string;
readonly failedRunRetry?: { readonly failedRunId: string };
readonly authorize: (tx: Db) => Promise<void>;
readonly onAdmitted?: (tx: Db, receipt: { runId: string | null }) => Promise<void>;
}
const trustedRequests = new WeakSet<object>();
@@ -290,6 +291,26 @@ export function assertDurableChatWakeupReceipt(
}
}
export function withDurableChatAdmission<T extends { kind: string; admitted?: boolean }>(request: DurableChatWakeupRequest | undefined,
admit: (tx: Parameters<Parameters<Db["transaction"]>[0]>[0]) => Promise<T>) {
return async (tx: Parameters<Parameters<Db["transaction"]>[0]>[0]) => {
const outcome = await admit(tx);
// An existing deferred receipt is not proof this transaction passed its
// gates. Only dispatch or an explicitly persisted wait admits the input.
if (request?.onAdmitted && (outcome.admitted === true || ["queued", "durable", "coalesced"].includes(outcome.kind))) {
const [receipt] = await tx.select().from(agentWakeupRequests).where(and(
eq(agentWakeupRequests.companyId, request.companyId), eq(agentWakeupRequests.id, request.id),
inArray(agentWakeupRequests.status, ["queued", "claimed", "deferred_issue_execution", "coalesced"]),
));
if (receipt) {
assertDurableChatWakeupReceipt(request, receipt);
await request.onAdmitted(tx as unknown as Db, { runId: receipt.runId });
}
}
return outcome;
};
}
/** Work-start policy only: ongoing tool and publication authorization must not
* revoke an already-started run when the operator enables mention gating. */
export async function slackMentionAllowsRunStart(
server/src/services/execution-control-reconciliation.test.ts
+60 -1
View File
@@ -2,7 +2,7 @@ import { enableRunFailureReporting } from "../__tests__/helpers/run-failure-repo
import { randomUUID } from "node:crypto";
import { eq } from "drizzle-orm";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { agents, companies, completionContracts, createDb, heartbeatRuns, issues, issueRecoveryActions, issueThreadInteractions, nativeRunResults, statusDecisions, workAssessments } from "@paperclipai/db";
import { agents, companies, completionContracts, createDb, environmentLeases, environments, heartbeatRuns, issues, issueRecoveryActions, issueThreadInteractions, nativeRunFinalizations, nativeRunResults, statusDecisions, workAssessments } from "@paperclipai/db";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
@@ -153,6 +153,65 @@ describeEmbeddedPostgres("reconcileAbandonedExecutionControl reports a genuine f
expect(result.surfaced).toBe(1);
expect(mockCaptureRunFailure.mock.calls.slice(captureCallsBefore)).toHaveLength(0);
});
it.each(["prepared", "finalized", "mismatched", "reassigned", "accepted"] as const)("preserves only an exact unfinished native workspace after its deadline (%s)", async state => {
const f = await seedAbandonedRunFixture();
const [environment] = await db.insert(environments).values({ name: `Deadline ${f.runId}`, driver: "sandbox" }).returning();
const [lease] = await db.insert(environmentLeases).values({ companyId: f.companyId, issueId: f.issueId,
heartbeatRunId: f.runId, environmentId: environment.id, provider: "daytona", providerLeaseId: randomUUID(),
status: "active", leasePolicy: "reuse_by_environment", metadata: { remoteCwd: "/work" },
}).returning();
await db.update(heartbeatRuns).set({ runtimeMode: "native", nativeIssueId: f.issueId, runnerProfileJson: {
nativeWorkspaceSync: { schema: "paperclip.native-workspace-sync/v3", state: state === "finalized" ? state : "prepared",
descriptorSha256: "a".repeat(64), baselineSha256: "b".repeat(64), finalHostSha256: null,
workspaceId: f.runId, leaseId: lease.id, providerLeaseId: state === "mismatched" ? randomUUID() : lease.providerLeaseId,
remoteCwd: "/work", resourceDisposition: "keep_running" },
} }).where(eq(heartbeatRuns.id, f.runId));
await db.insert(nativeRunFinalizations).values({ runId: f.runId, companyId: f.companyId, issueId: f.issueId,
phase: "observed", leaseOwner: "abandoned-controller", controlDeadlineAt: new Date(Date.now() - 60_000) });
if (state === "reassigned") {
const nextAgentId = randomUUID();
await db.insert(agents).values({ id: nextAgentId, companyId: f.companyId, name: "New owner", adapterType: "codex_local" });
await db.update(issues).set({ assigneeAgentId: nextAgentId, executionRunId: null, checkoutRunId: null })
.where(eq(issues.id, f.issueId));
}
if (state === "accepted") {
const contractId = randomUUID(), resultId = randomUUID();
await db.insert(completionContracts).values({ id: contractId, companyId: f.companyId, issueId: f.issueId,
revision: 1, schemaVersion: "1", policyVersion: "test", risk: "low", completionAuthority: "agent_claim_policy",
incompleteCriteriaPolicy: "block", contractJson: {}, canonicalSha256: randomUUID(), createdByActorType: "agent", createdByActorId: f.agentId });
await db.update(heartbeatRuns).set({ completionContractId: contractId }).where(eq(heartbeatRuns.id, f.runId));
await db.insert(nativeRunResults).values({ id: resultId, companyId: f.companyId, issueId: f.issueId, runId: f.runId,
completionContractId: contractId, serverFingerprint: randomUUID(), schemaStatus: "valid", resultJson: {}, canonicalSha256: randomUUID() });
await db.update(nativeRunFinalizations).set({ resultId }).where(eq(nativeRunFinalizations.runId, f.runId));
}
await reconcileAbandonedExecutionControl(db);
await reconcileAbandonedExecutionControl(db);
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId));
const [retained] = await db.select().from(environmentLeases).where(eq(environmentLeases.id, lease.id));
expect(run).toMatchObject(state === "accepted" ? { status: "running", errorCode: null }
: { status: "failed", errorCode: "execution_finalization_deadline_exceeded" });
if (state === "prepared" || state === "reassigned") {
expect(run.resultJson).toMatchObject({ workspaceRestoreRecovery: { leaseIds: [lease.id] } });
expect(retained).toMatchObject({ status: "pending_cleanup", leasePolicy: "retain_on_failure",
metadata: { sandboxStopAndRetain: { runId: f.runId, providerLeaseId: lease.providerLeaseId } } });
const recovery = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, f.issueId));
expect(recovery.filter(action => action.evidence?.workspaceRestoreRecovery)).toMatchObject([
{ evidence: { runId: f.runId, workspaceRestoreRecovery: { leaseIds: [lease.id] } } },
]);
} else {
expect(run.resultJson ?? {}).not.toHaveProperty("workspaceRestoreRecovery");
expect(retained).toMatchObject({ status: "active", leasePolicy: "reuse_by_environment" });
}
const [task] = await db.select().from(issues).where(eq(issues.id, f.issueId));
if (state === "reassigned") {
expect(task.assigneeAgentId).not.toBe(f.agentId);
expect(task).toMatchObject({ status: "in_progress", executionRunId: null, checkoutRunId: null });
}
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, f.companyId))).toHaveLength(1);
});
});
describeEmbeddedPostgres("native review execution reconciliation admission", () => {
server/src/services/execution-control-reconciliation.ts
+5 -1
View File
@@ -14,6 +14,7 @@ import { issueRecoveryActionService } from "./issue-recovery-actions.js";
import { hasLiveLegacyController } from "./legacy-controller-lease.js";
import { reportRunFailure } from "./run-failure-report.js";
import { getNativeReviewAssignment, readNativeReviewAssignmentContext } from "./native-runtime/native-review-participant.js";
import { preserveNativeFailedWorkspace } from "./native-runtime/native-failed-workspace-recovery.js";
/** Only newly recorded control deadlines are eligible. Upgrades never replay ambiguous historical runs. */
export async function reconcileAbandonedExecutionControl(
@@ -233,8 +234,10 @@ export async function reconcileAbandonedExecutionControl(
["done", "cancelled"].includes(task.status) ||
(task.executionRunId && task.executionRunId !== run.id) ||
(task.checkoutRunId && task.checkoutRunId !== run.id)
)
) {
if (updatedRun) await preserveNativeFailedWorkspace(tx as unknown as Db, updatedRun);
return true;
}
await tx
.update(issues)
.set({ executionRunId: null, checkoutRunId: null, updatedAt: now })
@@ -261,6 +264,7 @@ export async function reconcileAbandonedExecutionControl(
maxAttempts: 3,
supersedeOnIdentityChange: true,
});
if (updatedRun) await preserveNativeFailedWorkspace(tx as unknown as Db, updatedRun);
return true;
});
if (repaired) surfaced += 1;
server/src/services/execution-control-work.ts
+10 -3
View File
@@ -1,3 +1,4 @@
import { nextWorkspaceRecoveryAt } from "./system-recovery.js";
import { and, asc, eq, isNotNull, sql } from "drizzle-orm";
import { adapterAuthSessions, heartbeatRuns, issueRecoveryActions, nativeRunFinalizations, type Db } from "@paperclipai/db";
import type { createDeliveryWorkCoordinator } from "./delivery-work-coordinator.js";
@@ -12,6 +13,7 @@ export function registerExecutionControlWork(
queue: DeliveryQueue;
run: () => Promise<unknown>;
next: () => Promise<number | null>;
onSettled?: () => void;
}>,
canRun: () => boolean = () => true,
) {
@@ -20,10 +22,10 @@ export function registerExecutionControlWork(
let reconciled = 0;
let next: number | null = null;
let earliestRunAt = 0;
return coordinator.register(duty.queue, {
const worker = coordinator.register(duty.queue, {
retryMs: EXECUTION_RECONCILIATION_INTERVAL_MS,
pausePassiveMaintenance: true,
onSettled: () => { requested += 1; },
onSettled: () => { requested += 1; duty.onSettled?.(); },
run: async () => {
if (!canRun()) { requested += 1; next = null; return; }
const now = Date.now();
@@ -41,8 +43,9 @@ export function registerExecutionControlWork(
hasPending: async () => reconciled !== requested,
nextRunAt: () => next,
});
return { ...worker, wake: () => { requested += 1; worker.wake(); } };
});
return { ready: Promise.all(workers.map(worker => worker.ready)) };
return { ready: Promise.all(workers.map(worker => worker.ready)), wake: () => { for (const worker of workers) worker.wake(); } };
}
export function executionControlSchedule(db: Db) {
@@ -63,6 +66,10 @@ export function executionControlSchedule(db: Db) {
]);
return deadline([native[0]?.at, legacy[0]?.at]);
},
async workspaceRecovery() {
const at = await nextWorkspaceRecoveryAt(db);
return at === null ? null : deadline([new Date(at)]);
},
async replacement() {
const rows = await db.select({ id: heartbeatRuns.id }).from(heartbeatRuns)
.innerJoin(nativeRunFinalizations, eq(nativeRunFinalizations.runId, heartbeatRuns.id))
server/src/services/heartbeat-runner-provider-config.test.ts
+8
View File
@@ -7,6 +7,14 @@ import {
resolvePaperclipRunnerNativeProviderInput,
} from "./native-runtime/provider-profile.js";
describe("native Codex hosted search policy", () => {
it.each([undefined, true, false])("preserves the saved search=%s choice", search => {
const input = resolvePaperclipRunnerNativeProviderInput({ backend: "codex_app_server", adapterConfig: { provider: "codex", model: "gpt-5.4-mini", ...(search === undefined ? {} : { search }) } });
if (search === false) expect(input).toHaveProperty("codexWebSearch", "disabled");
else expect(input).not.toHaveProperty("codexWebSearch");
});
});
describe("Paperclip Runner native provider configuration", () => {
it.each([
["gpt-6.1-sol", "0.158.0", "gpt-6-sol"],
server/src/services/heartbeat/queue.ts
+11 -17
View File
@@ -1,5 +1,4 @@
import { authorizationService } from "../authorization.js";
import { assertTaskRetryConversationAllowed, readTaskRetryIntent, retireSupersededTaskRetry } from "../task-retry.js";
import { assertTaskRetryConversationAllowed, readTaskRetryIntent, retireSupersededTaskRetry, taskRetryMutationAllowed } from "../task-retry.js";
import { insertIssueComment } from "../issue-comment-writes.js";
import { withExternalAdmissionGuard, assertNoExternalOverlap } from "../../modules/external-agents/index.js";
import { isAgentAwaitingSetup } from "../../modules/agent-lifecycle/index.js";
@@ -50,6 +49,7 @@ import {
slackMentionAllowsRunStart,
assertDurableChatWakeupReceipt,
assertDurableChatWakeupRequest,
withDurableChatAdmission,
authorizeFailedChatRunRetryWake,
FailedChatRunRetryAuthorizationError,
unadmittedChatWakeupCondition,
@@ -2809,7 +2809,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende
const cancelledRunsToEmit: (typeof heartbeatRuns.$inferSelect)[] = [];
const outcome = await db
.transaction(async (tx) => {
.transaction(withDurableChatAdmission(durableRequest, async (tx) => {
await tx.execute(
sql`select id from issues where id = ${issueId} and company_id = ${agent.companyId} for update`,
);
@@ -2847,11 +2847,8 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende
if (pending.id !== opts.taskRetryRequestId || retry?.generationId !== pending.id || retry.chatActionId ||
retry.actorId !== opts.requestedByActorId || retry.sourceRunId !== enrichedContextSnapshot.previousRunId)
return { kind: "deferred" as const };
const permission = await authorizationService(tx as unknown as Db).decide({
actor: { type: "board", userId: retry.actorId, source: retry.actorSource }, action: "issue:mutate",
resource: { type: "issue", companyId: agent.companyId, issueId },
});
if (!permission.allowed) return { kind: "deferred" as const };
if (!await taskRetryMutationAllowed(tx as unknown as Db, { companyId: agent.companyId, issueId,
actorId: retry.actorId, actorSource: retry.actorSource })) return { kind: "deferred" as const };
}
if (opts.queuedCommentRequestId) {
const ids = await undeliveredLegacyUserCommentIds(tx as unknown as Db,
@@ -3004,11 +3001,8 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende
if (!failedChatRetry || retry?.chatActionId !== durableRequest.id ||
retry.actorId !== taskRetryActorId || retry.sourceRunId !== durableRequest.failedRunRetry?.failedRunId)
throw new FailedChatRunRetryAuthorizationError();
const permission = await authorizationService(tx as unknown as Db).decide({
actor: { type: "board", userId: retry.actorId, source: retry.actorSource }, action: "issue:mutate",
resource: { type: "issue", companyId: agent.companyId, issueId },
});
if (!permission.allowed) return { kind: "deferred" as const };
if (!await taskRetryMutationAllowed(tx as unknown as Db, { companyId: agent.companyId, issueId,
actorId: retry.actorId, actorSource: retry.actorSource })) return { kind: "deferred" as const };
}
} else {
if (
@@ -3311,7 +3305,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende
payload: sql`jsonb_set(coalesce(${agentWakeupRequests.payload}, '{}'::jsonb), '{executionWait}', ${JSON.stringify(condition)}::jsonb)`,
updatedAt: new Date(),
}).where(eq(agentWakeupRequests.id, executionWaitRequestId));
return { kind: "deferred" as const };
return { kind: "deferred" as const, admitted: true };
}
if (durableRequest || wakeCommentId ||
hasInteractionContinuationWakeContext(enrichedContextSnapshot) ||
@@ -3348,7 +3342,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende
},
});
}
return { kind: "deferred" as const };
return { kind: "deferred" as const, admitted: true };
};
const explicitContinuationRunId = randomUUID();
const executionBlocker = await getExecutionBlocker(
@@ -3974,7 +3968,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende
};
}
if (admission.kind === "deferred") {
return { kind: "deferred" as const };
return { kind: "deferred" as const, admitted: true };
}
// admission.kind === "proceed": no active run absorbed this wake,
// so fall through to the ordinary queue path below.
@@ -4357,7 +4351,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende
// transitions to "running" — Fix A (lazy locking).
return { kind: "queued" as const, run: newRun };
})
}))
.catch((error) => {
if (isExternalChatWaitAuthorizationContention(error))
return { kind: "deferred" as const };
server/src/services/heartbeat/runtime-selection.ts
+1
View File
@@ -971,6 +971,7 @@ export async function selectHeartbeatRuntime(db: Db, input: HeartbeatRuntimeSele
nativeToolContractFingerprint:
nativeToolContractFingerprintForTarget(
executionTarget?.kind ?? "local",
nativeExecution!.provider.kind,
),
...(lockedProfile.sessionCheckpoint != null
? { sessionCheckpoint: lockedProfile.sessionCheckpoint }
server/src/services/native-runtime/assigned-mcp-tools.test.ts
+88
View File
@@ -2,6 +2,7 @@ import { describe, expect, it, vi } from "vitest";
import type { Db } from "@paperclipai/db";
import { ToolGatewayHttpError, type ToolGatewayDescriptor, type ToolGatewayService } from "../tool-gateway.js";
import { createAssignedMcpTools, getAssignedMcpGateway, registerAssignedMcpGateway } from "./assigned-mcp-tools.js";
import { startOpenCodeMcpBridge } from "../../vendor/paperclip-runner/testing.js";
function descriptor(name: string, risk: ToolGatewayDescriptor["risk"] = "read"): ToolGatewayDescriptor {
return { name, displayName: name, description: `Use ${name}`, parametersSchema: { type: "object", properties: { query: { type: "string" } } }, pluginId: "fixture", providerType: "mcp_remote_http", risk };
@@ -18,6 +19,93 @@ describe("assigned MCP runner tools", () => {
const searchName = "paperclip_search_assigned_tools";
const callName = "paperclip_call_assigned_tool";
it.each(["page", "escaped schema"])("keeps on-demand %s results valid through the OpenCode MCP bridge", async variant => {
const schema = { type: "object", description: variant === "page" ? "x".repeat(20_000) : "\"\\\u0000🙂".repeat(30_000) };
const f = fixture(Array.from({ length: variant === "page" ? 5 : 1 }, (_, i) => ({
...descriptor(`large_${i}`), parametersSchema: schema,
})));
const assigned = await createAssignedMcpTools({ ...f, delivery: "on_demand" });
const bridge = await startOpenCodeMcpBridge({ tools: assigned.definitions(), handler: call => assigned.execute(call) });
let id = 0;
const search = async (args: Record<string, unknown>) => {
const response = await fetch(bridge.url, {
method: "POST", headers: { Authorization: `Bearer ${bridge.secret}`, "Content-Type": "application/json" },
body: JSON.stringify({ jsonrpc: "2.0", id: ++id, method: "tools/call", params: { name: searchName, arguments: args } }),
});
const rpc = await response.json() as { result: { isError?: boolean; content: Array<{ text: string }> } };
expect(rpc.result.isError).not.toBe(true);
return JSON.parse(rpc.result.content[0]!.text);
};
try {
let page = await search({ query: "", limit: 20 });
if (variant === "page") {
const seen = [...page.tools];
while (page.nextOffset !== null) {
page = await search({ query: "", offset: page.nextOffset, limit: 20 });
seen.push(...page.tools);
}
expect(seen).toHaveLength(5);
expect(seen.every(tool => JSON.stringify(tool.inputSchema) === JSON.stringify(schema))).toBe(true);
} else {
const name = page.tools[0]!.inputSchemaRef;
expect(name).toBeTypeOf("string");
let offset: number | null = 0;
let reconstructed = "";
do {
const chunk = await search({ query: "", schemaTool: name, schemaOffset: offset });
reconstructed += chunk.schemaJson;
offset = chunk.nextSchemaOffset;
} while (offset !== null);
expect(JSON.parse(reconstructed)).toEqual(schema);
}
} finally {
await bridge.close();
}
});
it.each([1, 741])("uses bounded on-demand delivery for %i tools even when the direct catalog fits", async count => {
const f = fixture(Array.from({ length: count }, (_, i) => descriptor(`app.operation_${i}`)));
const direct = await createAssignedMcpTools(f);
const assigned = await createAssignedMcpTools({ ...f, delivery: "on_demand" });
const fits = vi.fn(() => true);
expect(assigned.definitions(fits).map(tool => tool.name)).toEqual([searchName, callName]);
expect(fits).toHaveBeenCalledExactlyOnceWith(assigned.definitions());
expect(() => assigned.definitions(() => false)).toThrow("assigned_mcp_tool_catalog_capacity_exceeded");
const page = await assigned.execute({ tool: searchName, arguments: { query: "", limit: 1 } }) as { tools: Array<{ name: string }> };
expect(page.tools[0]!.name).toBe(direct.definitions().map(tool => String(tool.name)).sort()[0]);
await assigned.execute({ tool: callName, arguments: { name: page.tools[0]!.name, arguments: { query: "fixture" }, gatewayPublicId: "untrusted", bearerToken: "untrusted" } });
expect(f.executeTool).toHaveBeenCalledExactlyOnceWith(expect.objectContaining({ gatewayPublicId: f.gatewayPublicId, sessionToken: f.bearerToken }));
f.listToolsForNamedGateway.mockResolvedValue([]);
await expect(assigned.execute({ tool: searchName, arguments: { query: "" } })).resolves.toEqual({ tools: [], nextOffset: null });
});
it("keeps on-demand aliases pinned, mode restricted, revocable and approval gated", async () => {
const read = descriptor("mcp.first:search");
const write = descriptor("mcp.second:search", "write");
const f = fixture([read, write]);
const assigned = await createAssignedMcpTools({ ...f, delivery: "on_demand" });
const page = await assigned.execute({ tool: searchName, arguments: { query: "" } }) as { tools: Array<{ name: string; description: string }> };
expect(new Set(page.tools.map(tool => tool.name)).size).toBe(2);
const writeAlias = page.tools.find(tool => tool.description.includes(write.name))!.name;
const call = { tool: callName, arguments: { name: writeAlias, arguments: {} } };
await expect(assigned.execute(call, "planning")).rejects.toThrow("paperclip_runner_tool_mode_denied");
await expect(assigned.execute(call, "ask")).rejects.toThrow("paperclip_runner_tool_mode_denied");
expect(f.executeTool).not.toHaveBeenCalled();
const denied = new ToolGatewayHttpError(403, "Approval required", "approval_required");
f.executeTool.mockRejectedValueOnce(denied);
await expect(assigned.execute(call)).rejects.toBe(denied);
f.listToolsForNamedGateway.mockResolvedValue([read, descriptor("newly.granted")]);
const refreshed = await assigned.execute({ tool: searchName, arguments: { query: "" } }) as { tools: Array<{ name: string }> };
expect(refreshed.tools).toHaveLength(1);
expect(refreshed.tools[0]!.name).not.toBe(writeAlias);
// Execution still goes through the original gateway, which reauthorizes
// each call; discovery never grants an added tool to this pinned session.
const revoked = new ToolGatewayHttpError(403, "Revoked", "connection_revoked");
f.executeTool.mockRejectedValueOnce(revoked);
await expect(assigned.execute(call)).rejects.toBe(revoked);
await expect(assigned.execute({ tool: callName, arguments: { name: "newly.granted", arguments: {} } })).rejects.toThrow("assigned_mcp_tool_unknown");
});
it("deduplicates repeated app tools in direct and on-demand catalogs while retaining revocation", async () => {
const read = descriptor("memory.search");
const write = descriptor("memory.store", "write");
server/src/services/native-runtime/assigned-mcp-tools.ts
+12 -5
View File
@@ -86,6 +86,8 @@ export async function createAssignedMcpTools(input: {
gatewayPublicId: string;
bearerToken: string;
workMode?: WorkMode;
/** Bound provider grammar complexity independently of protocol byte capacity. */
delivery?: "on_demand";
}) {
const listed = await input.gateway.listToolsForNamedGateway({
gatewayPublicId: input.gatewayPublicId,
@@ -103,6 +105,11 @@ export async function createAssignedMcpTools(input: {
tools.set(name, descriptor);
}
const permits = (tool: ToolGatewayDescriptor, mode: WorkMode = input.workMode ?? "standard") => mode === "standard" || tool.risk === "read";
// OpenCode's HTTP bridge returns at most 64K JSON characters. Leave room
// for the envelope and for JSON escaping when returning a schema chunk.
// Other harnesses retain the larger PRP page and chunk budgets.
const searchPageBytes = input.delivery === "on_demand" ? 32 * 1024 : SEARCH_PAGE_BYTES;
const schemaChunkCharacters = input.delivery === "on_demand" ? 16 * 1024 : SCHEMA_CHUNK_CHARACTERS;
async function search(argumentsValue: unknown, currentWorkMode?: WorkMode) {
const args = object(argumentsValue);
@@ -132,7 +139,7 @@ export async function createAssignedMcpTools(input: {
const selected = authorized.find(([name]) => name === args.schemaTool);
if (!selected) throw new Error("assigned_mcp_tool_unknown");
const schema = JSON.stringify(selected[1].parametersSchema);
const end = schemaOffset + SCHEMA_CHUNK_CHARACTERS;
const end = schemaOffset + schemaChunkCharacters;
return {
name: selected[0], schemaJson: schema.slice(schemaOffset, end),
nextSchemaOffset: end < schema.length ? end : null,
@@ -144,13 +151,13 @@ export async function createAssignedMcpTools(input: {
const page: ToolDefinition[] = [];
for (const [name, tool] of matches.slice(offset, offset + limit)) {
let next = definition(name, tool);
if (Buffer.byteLength(JSON.stringify([next]), "utf8") > SEARCH_PAGE_BYTES) {
if (Buffer.byteLength(JSON.stringify([next]), "utf8") > searchPageBytes) {
// One large schema must not block browsing the tools after it. Expose
// a reference whose schema can be fetched in bounded, reauthorized
// chunks, without weakening the gateway's argument validation.
next = { name, description: String(next.description).slice(0, 2000), inputSchemaRef: name };
}
if (Buffer.byteLength(JSON.stringify([...page, next]), "utf8") > SEARCH_PAGE_BYTES) {
if (Buffer.byteLength(JSON.stringify([...page, next]), "utf8") > searchPageBytes) {
break;
}
page.push(next);
@@ -161,9 +168,9 @@ export async function createAssignedMcpTools(input: {
return {
definitions(fits?: (definitions: ToolDefinition[]) => boolean): ToolDefinition[] {
const direct = [...tools].filter(([, tool]) => permits(tool)).map(([name, tool]) => definition(name, tool));
if (!fits || fits(direct)) return direct;
if (input.delivery !== "on_demand" && (!fits || fits(direct))) return direct;
const compact = structuredClone(ON_DEMAND_TOOLS);
if (!fits(compact)) throw new Error("assigned_mcp_tool_catalog_capacity_exceeded");
if (fits && !fits(compact)) throw new Error("assigned_mcp_tool_catalog_capacity_exceeded");
return compact;
},
has(name: string): boolean {
server/src/services/native-runtime/native-execution-input.test.ts
+2 -1
View File
@@ -21,12 +21,13 @@ describe("LCA-05 explicit native work mode", () => {
normalizedSessionId: null,
provider: "codex",
codexReasoningEffort: "high",
codexWebSearch: "disabled",
planningContext: workMode === "planning" ? { documentId: null, baseRevisionId: null, baseRevisionNumber: 0, markdown: "", sha256: "a".repeat(64), reviewContext: {} } : null,
completionContract: { id: "70000000-0000-4000-8000-000000000007", sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1", contract: { revision: "1", objective: "Deliver the requested work", criteria: [{ id: "output", requirement: "Deliver the requested work" }] } },
runtimeContext: nativeRuntimeContextFixture(),
});
expect(input.task.workMode).toBe(workMode);
expect(input.provider).toMatchObject({ kind: "codex", reasoningEffort: "high" });
expect(input.provider).toMatchObject({ kind: "codex", reasoningEffort: "high", webSearch: "disabled" });
expect(input.executionMode).toBe(workMode === "planning" ? "plan" : "default");
expect(input.task.title).toBe(text);
}
server/src/services/native-runtime/native-execution-input.ts
+2
View File
@@ -102,6 +102,7 @@ export interface BuildNativeExecutionInput {
homeMode?: "isolated" | "preconfigured";
codexApprovalPolicy?: NativeCodexApprovalPolicy;
codexReasoningEffort?: string;
codexWebSearch?: "disabled";
opencodePermissionMode?: NativeOpenCodePermissionMode;
acpxPermissionMode?: NativeAcpxPermissionMode;
acpxSessionMode?: "agent" | "plan" | "ask";
@@ -364,6 +365,7 @@ export function buildNativeExecutionInput(input: BuildNativeExecutionInput): Nat
model: input.model ?? null,
approvalPolicy: input.codexApprovalPolicy ?? "never",
...(input.codexReasoningEffort ? { reasoningEffort: input.codexReasoningEffort } : {}),
...(input.codexWebSearch ? { webSearch: input.codexWebSearch } : {}),
},
completionContract: {
id: input.completionContract.id,
server/src/services/native-runtime/native-session-executor.test.ts
+22 -5
View File
@@ -1471,6 +1471,7 @@ describe("remote provider pack manifest", () => {
try {
await createRunnerdBackend({ db: leaseDb(remoteExecution), execution: remoteExecution,
runnerInstanceId: "bundled-image-runner", runnerIngressAuthorized: true,
...(provider === "opencode" ? { runnerEnvironment: { PAPERCLIP_NATIVE_MCP_NAME: "assigned-fixture", PAPERCLIP_NATIVE_MCP_URL: "http://127.0.0.1/mcp/gateways/assigned-fixture", PAPERCLIP_NATIVE_MCP_TOKEN: "private-fixture" } } : {}),
runnerExecutionTarget: { kind: "remote", transport: "sandbox", remoteCwd: "/workspace",
environmentId: "environment", leaseId: "lease", providerKey: "daytona",
effectiveCapabilities: { runnerWebSocketIngress: true }, runner: { execute, syncIn } } as never });
@@ -1478,8 +1479,13 @@ describe("remote provider pack manifest", () => {
state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
const transport = state.createTransport.mock.calls.at(-1)![0] as RunnerTransportOptions & {
controlPlaneRegistration: (authority: unknown) => Promise<unknown>;
environment?: NodeJS.ProcessEnv;
};
expect(transport.runnerBinary).toBe(state.bundledRunnerBinary);
if (provider === "opencode") {
expect(state.createAssignedMcpTools).toHaveBeenCalledWith(expect.objectContaining({ gatewayPublicId: "assigned-fixture", bearerToken: "private-fixture", delivery: "on_demand" }));
for (const name of ["PAPERCLIP_NATIVE_MCP_NAME", "PAPERCLIP_NATIVE_MCP_URL", "PAPERCLIP_NATIVE_MCP_TOKEN"]) expect(transport.environment?.[name]).toBeUndefined();
}
if (claude) {
expect(state.resolveRemoteRunnerBinary).toHaveBeenCalledWith(expect.objectContaining({ remoteCwd: "/workspace", configuredPath: undefined }));
const { bundledRemoteProviderPackManifestPath } = await import("../../vendor/paperclip-runner/index.js");
@@ -7800,7 +7806,7 @@ describe("bounded stopped instruction collection", () => {
describe("native warm session supervision", () => {
it.each(["collect", "close-failure", "stopped", "destroyed", "missing-lease", "foreign-run", "foreign-environment", "deleted-environment", "claim-edit", "claim-add", "claim-remove", "claim-remove-entry", "claim-canonical", "claim-config", "claim-uncertain"])("composes remote warm ownership with real DB successor leases and current-run collection: %s", async ending => {
const tables = await import("@paperclipai/db");
const { eq } = await import("drizzle-orm");
const { eq, sql } = await import("drizzle-orm");
const { randomUUID } = await import("node:crypto");
const { promisify } = await import("node:util");
const { execFile } = await import("node:child_process");
@@ -7813,7 +7819,11 @@ describe("native warm session supervision", () => {
const home = await realpath(await mkdtemp(join(tmpdir(), "warm-remote-db-")));
const previousHome = process.env.PAPERCLIP_HOME;
process.env.PAPERCLIP_HOME = home;
const database = await startEmbeddedPostgresTestDatabase("warm-remote-db-");
const { resolveExternalTestDatabaseUrl } = await import("../../__tests__/helpers/external-test-database.js");
const externalUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_TEST_DATABASE_URL");
const database = externalUrl
? { connectionString: externalUrl, cleanup: () => tables.closeRegisteredClients(externalUrl) }
: await startEmbeddedPostgresTestDatabase("warm-remote-db-");
const db = tables.createDb(database.connectionString);
const companyId = randomUUID(), agentId = randomUUID(), userId = randomUUID(), environmentId = randomUUID(), issueId = randomUUID();
const { resolveManagedInstructionsRoot } = await import("../agent-instructions.js");
@@ -8049,6 +8059,9 @@ describe("native warm session supervision", () => {
finally {
const cleanupFailures: unknown[] = [];
try { await closeWarmNativeSessionsForEnvironment({ environmentId, reason: "test cleanup" }); } catch (error) { cleanupFailures.push(error); }
if (externalUrl) {
try { await db.execute(sql`TRUNCATE TABLE ${tables.companies}, ${tables.environments}, ${tables.authUsers} CASCADE`); } catch (error) { cleanupFailures.push(error); }
}
try { await database.cleanup(); } catch (error) { cleanupFailures.push(error); }
if (previousHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = previousHome;
try {
@@ -11828,7 +11841,10 @@ describe("runnerd provider runtime wiring", () => {
} finally {owner.mockRestore();await rm(home,{recursive:true,force:true});}
});
it("keeps assigned MCP credentials on the control plane for remote Codex", async () => {
it.each(["codex", "opencode"] as const)("keeps assigned MCP credentials on the control plane for %s", async (providerKind) => {
const selectedExecution: NativeExecutionInput = providerKind === "opencode"
? { ...execution, provider: { kind: "opencode", model: "openrouter/deepseek/deepseek-v4.1-flash-1017", permissionMode: "allow" } }
: execution;
const endpoint = process.env.OTEL_EXPORTER_OTLP_ENDPOINT;
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://collector.test");
const exporter = new InMemorySpanExporter();
@@ -11841,13 +11857,13 @@ describe("runnerd provider runtime wiring", () => {
state.createTransport.mockClear();
state.toolAuthorityDefinitions.mockClear();
await createRunnerdBackend({
db: leaseDb(execution), execution, runnerInstanceId: "runner-assigned-mcp",
db: leaseDb(selectedExecution), execution: selectedExecution, runnerInstanceId: "runner-assigned-mcp",
runnerEnvironment: {
PAPERCLIP_NATIVE_MCP_NAME: "paperclip-assigned",
PAPERCLIP_NATIVE_MCP_URL: "http://127.0.0.1:3217/mcp/gateways/assigned-test",
PAPERCLIP_NATIVE_MCP_TOKEN: "private-run-token",
},
runnerExecutionTarget: {
runnerExecutionTarget: providerKind === "opencode" ? undefined : {
kind: "remote", transport: "sandbox", providerKey: "daytona",
leaseId: "lease-assigned-mcp", remoteCwd: "/home/daytona/paperclip-workspace",
runner: { execute: vi.fn() },
@@ -11855,6 +11871,7 @@ describe("runnerd provider runtime wiring", () => {
});
expect(state.createAssignedMcpTools).toHaveBeenCalledWith(expect.objectContaining({
gatewayPublicId: "assigned-test", bearerToken: "private-run-token",
...(providerKind === "opencode" ? { delivery: "on_demand" } : {}),
}));
expect(state.toolAuthorityDefinitions).toHaveBeenCalledWith(expect.objectContaining({ assignedMcpTools }));
state.createBackend.mock.calls[0]![1].codexTransportFactory!();
server/src/services/native-runtime/native-session-executor.ts
+6 -3
View File
@@ -5563,7 +5563,7 @@ function nativeSessionConfigDigest(
// changes so an older thread is rotated instead of falsely resuming
// without newly required tools.
nativeToolContractFingerprint:
nativeToolContractFingerprintForTarget(executionTargetKind),
nativeToolContractFingerprintForTarget(executionTargetKind, execution.provider.kind),
runtimeContract: nativeRuntimeContractForProvider(execution.provider),
}),
)
@@ -11828,10 +11828,10 @@ async function createRunnerdBackendWithinSessionClaim(
if (nativeReview && !await getNativeReviewAssignment(input.db, {
...input.execution.binding, contextSnapshot: nativeReview,
})) throw new Error("native_review_assignment_no_longer_available");
// Remote Codex already sends dynamic tool calls over authenticated PRP. Keep
// Codex and OpenCode send dynamic tool calls over authenticated PRP. Keep
// the assigned gateway on the control plane instead of asking the sandbox to
// reach the host's HTTP origin (which may be private or loopback-only).
const relayAssignedMcp = Boolean(pluginTask) || (remoteTarget !== null && input.execution.provider.kind === "codex") || input.execution.schema === "paperclip.native-execution-input.v6";
const relayAssignedMcp = Boolean(pluginTask) || input.execution.provider.kind === "opencode" || (remoteTarget !== null && input.execution.provider.kind === "codex") || input.execution.schema === "paperclip.native-execution-input.v6";
const assignedMcpUrl = input.runnerEnvironment?.PAPERCLIP_NATIVE_MCP_URL;
const assignedMcpToken = input.runnerEnvironment?.PAPERCLIP_NATIVE_MCP_TOKEN;
const assignedMcpName = input.runnerEnvironment?.PAPERCLIP_NATIVE_MCP_NAME;
@@ -11853,6 +11853,7 @@ async function createRunnerdBackendWithinSessionClaim(
gatewayPublicId: assignedGatewayPublicId!,
bearerToken: assignedMcpToken,
workMode: input.execution.task.workMode,
...(input.execution.provider.kind === "opencode" ? { delivery: "on_demand" as const } : {}),
}))
: undefined),
]);
@@ -12012,6 +12013,7 @@ async function createRunnerdBackendWithinSessionClaim(
},
codexTransportFactory: recovery => createRunnerdCodexTransport({ provider: provider.kind, stateDirectory: root,
...owner.driver, runnerArtifact: description.runner,
...(provider.kind === "codex" ? { disableCodexWebSearch: "webSearch" in provider && provider.webSearch === "disabled" } : {}),
...(provider.kind === "acpx" ? { acpxAgent: provider.agent,
acpxCandidateProfile: provider.agent === "pi" ? undefined : resolveAcpxQualification(provider, process.env),
acpxPermissionMode: provider.permissionMode, acpxMode: provider.mode, piThinkingLevel: provider.piThinkingLevel,
@@ -13396,6 +13398,7 @@ async function createRunnerdBackendWithinSessionClaim(
codexTransportFactory: (recoveryContext) =>
createRunnerdCodexTransport({
onSpawn: input.onSpawn,
...(input.execution.provider.kind === "codex" ? { disableCodexWebSearch: "webSearch" in input.execution.provider && input.execution.provider.webSearch === "disabled" } : {}),
homeMode: preconfiguredHome ? "preconfigured" : "isolated",
onCredentialsUpdated: (event) => appendNativeCredentialUpdate(input.db, {
companyId: input.execution.binding.companyId,
server/src/services/native-runtime/native-session-resume.test.ts
+36 -4
View File
@@ -32,7 +32,7 @@ import {
} from "@paperclipai/paperclip-runner";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
startEmbeddedPostgresTestDatabase as startEmbeddedDatabase,
} from "../../__tests__/helpers/embedded-postgres.js";
import {
canonicalNativeRuntimeContextDigest,
@@ -72,6 +72,20 @@ import {
selectNativeSessionResumeRun,
} from "./native-session-resume.js";
import { nativeRuntimeContextFixture } from "./runtime-context.test-fixture.js";
import { resolveExternalTestDatabaseUrl } from "../../__tests__/helpers/external-test-database.js";
const externalTestDatabaseUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_TEST_DATABASE_URL");
async function startEmbeddedPostgresTestDatabase(prefix: string) {
const externalUrl = externalTestDatabaseUrl;
if (!externalUrl) return startEmbeddedDatabase(prefix);
const { closeRegisteredClients } = await import("@paperclipai/db");
return { connectionString: externalUrl, cleanup: async () => {
const db = createDb(externalUrl);
try { await db.execute(sql`TRUNCATE TABLE companies CASCADE`); }
finally { await closeRegisteredClients(externalUrl); }
} };
}
const companyId = "10000000-0000-4000-8000-000000000001";
const issueId = "20000000-0000-4000-8000-000000000002";
@@ -283,12 +297,13 @@ it("wires exact-session recovery and guarded selected identity into heartbeat pe
});
const embeddedSupport = await getEmbeddedPostgresTestSupport();
const databaseSupported = Boolean(externalTestDatabaseUrl) || embeddedSupport.supported;
const recoveryFakeCodex = resolve(
import.meta.dirname,
"../../../../packages/paperclip-runner/test/fixtures/fake-final-burst-codex-app-server.mjs",
);
(embeddedSupport.supported && existsSync(defaultCapabilityRunnerdBinary())
(databaseSupported && existsSync(defaultCapabilityRunnerdBinary())
? it
: it.skip)(
"keeps actual normalized driver events separate from raw maintenance receipts",
@@ -520,7 +535,7 @@ const recoveryFakeCodex = resolve(
30_000,
);
(embeddedSupport.supported &&
(databaseSupported &&
existsSync(defaultCapabilityRunnerdBinary()) &&
existsSync(recoveryFakeCodex)
? it
@@ -1062,7 +1077,7 @@ const recoveryFakeCodex = resolve(
60_000,
);
(embeddedSupport.supported ? describe : describe.skip)(
(databaseSupported ? describe : describe.skip)(
"native session recovery database orchestration",
() => {
it("uses scoped latest checkpoints, preserves progress barriers, and guards bootstrap persistence after the row lock", async () => {
@@ -1825,6 +1840,23 @@ describe("rebindNativeSessionCheckpoint", () => {
it.each(["local", "remote"] as const)("refreshes only OpenCode's former direct MCP catalog on %s", target => {
const legacy = nativeToolContractFingerprintForTarget(target);
expect(nativeToolContractFingerprintForTarget(target, "codex")).toBe(legacy);
expect(nativeToolContractFingerprintForTarget(target, "acpx")).toBe(legacy);
const current = nativeToolContractFingerprintForTarget(target, "opencode");
expect(current).not.toBe(legacy);
const shortenedCatalog = target === "local"
? "sha256:2ae35faccbb63cfe009d94170ef1ef13f0d57e120af6160540598a3ec72001d7"
: "sha256:ee5285ca5118b8468c0afd1df4ef9c05d85b0934e882bccd0b193dc8722dbb7a";
expect(current).not.toBe(shortenedCatalog);
const currentExecution = { ...execution(currentRunId), provider: { kind: "opencode", model: "openrouter/deepseek/deepseek-v4.1-flash-1017", permissionMode: "allow" }, session: { ...execution(currentRunId).session, driverKind: "opencode_server" } } as NativeExecutionInput;
const previousExecution = { ...execution(previousRunId), provider: currentExecution.provider, session: currentExecution.session };
expect(rebindNativeSessionCheckpoint({ previousRun: previousRun({ nativeToolContractFingerprint: shortenedCatalog, nativeExecutionInput: previousExecution }), currentExecution, executionTargetKind: target })).toBeNull();
expect(rebindNativeSessionCheckpoint({ previousRun: previousRun({ nativeToolContractFingerprint: legacy, nativeExecutionInput: previousExecution }), currentExecution, executionTargetKind: target })).toBeNull();
expect(rebindNativeSessionCheckpoint({ previousRun: previousRun({ nativeToolContractFingerprint: current, nativeExecutionInput: previousExecution }), currentExecution, executionTargetKind: target })).not.toBeNull();
});
it("binds the provider tool catalog to the local or remote execution target", () => {
const remoteFingerprint = nativeToolContractFingerprintForTarget("remote");
expect(remoteFingerprint).not.toBe(NATIVE_TOOL_CONTRACT_FINGERPRINT);
server/src/services/native-runtime/native-session-resume.ts
+5 -1
View File
@@ -29,6 +29,7 @@ export function nativeRuntimeContractForProvider(
*/
export function nativeToolContractFingerprintForTarget(
executionTargetKind: NativeToolExecutionTargetKind,
providerKind?: NativeExecutionInput["provider"]["kind"],
): string {
return `sha256:${createHash("sha256")
.update(
@@ -47,6 +48,9 @@ export function nativeToolContractFingerprintForTarget(
structuredHumanInput:
"always_advertised_run_issue_agent_binding_gated_current_task_description.v2",
semanticCompletion: "finish_accessible_deliverable_evidence.v4",
...(providerKind === "opencode"
? { opencodeAssignedMcpTools: "server_gateway_prp_on_demand_exact_names.v2" }
: {}),
connectorTools: "assigned_resources_and_pinned_skill_bundle.v1",
chatTaskTools: "durable_task_binding_and_accepted_origin.v1",
githubResponses: "bound_working_comment_progress_and_final.v1",
@@ -482,7 +486,7 @@ export function rebindNativeSessionCheckpoint(input: {
const previousProfile = record(input.previousRun.runnerProfileJson);
if (
previousProfile.nativeToolContractFingerprint !==
nativeToolContractFingerprintForTarget(input.executionTargetKind ?? "local")
nativeToolContractFingerprintForTarget(input.executionTargetKind ?? "local", input.currentExecution.provider.kind)
) {
return null;
}
server/src/services/native-runtime/provider-profile.ts
+2
View File
@@ -84,6 +84,7 @@ export type PaperclipRunnerNativeProviderInput =
homeMode?: "isolated" | "preconfigured";
codexApprovalPolicy: "never" | "on-request" | "untrusted";
codexReasoningEffort?: string;
codexWebSearch?: "disabled";
}
| {
provider: "opencode";
@@ -843,5 +844,6 @@ export function resolvePaperclipRunnerNativeProviderInput(input: {
config.codexPermissionMode,
) as "never" | "on-request" | "untrusted",
...(typeof effort === "string" && effort ? { codexReasoningEffort: effort } : {}),
...(config.search === false ? { codexWebSearch: "disabled" as const } : {}),
};
}
server/src/services/system-recovery.ts
+41 -13
View File
@@ -1,10 +1,17 @@
import { and, eq, sql } from "drizzle-orm";
import { issueComments, issueRecoveryActions, type Db } from "@paperclipai/db";
import { heartbeatRuns, issueComments, issueRecoveryActions, type Db } from "@paperclipai/db";
import { writeExecutionControl } from "./execution-control-notifications.js";
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
import { createSystemRecovery, readSystemRecovery, type SystemRecoveryState } from "./system-recovery-state.js";
import type { WorkspaceRecoverySourceIdentity } from "./workspace-recovery-source.js";
import { workspaceRecoverySourceEligible, type WorkspaceRecoverySourceIdentity } from "./workspace-recovery-source.js";
export async function adoptSystemRecovery(db: Db, source: WorkspaceRecoverySourceIdentity) {
return db.transaction(async tx => {
const [existing] = await tx.select({ evidence: issueRecoveryActions.evidence }).from(issueRecoveryActions)
.where(and(eq(issueRecoveryActions.companyId, source.companyId), eq(issueRecoveryActions.id, source.actionId),
eq(issueRecoveryActions.sourceIssueId, source.issueId))).for("update");
if (!existing || existing.evidence.systemRecovery) return;
await notifyDeliveryWork(tx, DELIVERY_QUEUES.workspaceRecovery);
const state = createSystemRecovery(source.runId);
// Atomic, once per source. An exhausted previous execution budget is history;
// adoption grants only the explicitly versioned maintenance capability.
@@ -49,17 +56,22 @@ export async function updateSystemRecovery(db: Db, source: WorkspaceRecoverySour
stage: next.phase === "verify_execution" ? "verifying_stop" : next.phase === "preserve_files" ? "restoring_files" : "saving_receipts",
attempt: next.attempts, attemptLimit: 6, retryable: true, retryAt: next.nextAttemptAt ?? undefined,
};
const [updated] = await db.update(issueRecoveryActions).set({
ownerType: "system", ownerAgentId: null, ownerUserId: null,
evidence: sql`${issueRecoveryActions.evidence} || jsonb_build_object('systemRecovery', ${JSON.stringify(next)}::jsonb,
'workspaceRecovery', ${JSON.stringify(compatibility)}::jsonb || jsonb_build_object('requestIds',
coalesce(${issueRecoveryActions.evidence}#>'{workspaceRecovery,requestIds}', '[]'::jsonb)))`,
updatedAt: new Date(),
}).where(and(eq(issueRecoveryActions.companyId, source.companyId), eq(issueRecoveryActions.id, source.actionId),
sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,generationId}' = ${previous.generationId}`,
sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,revision}' = ${String(previous.revision)}`))
.returning({ id: issueRecoveryActions.id });
return Boolean(updated);
return writeExecutionControl(db, DELIVERY_QUEUES.workspaceRecovery, async tx => {
const [updated] = await tx.update(issueRecoveryActions).set({
ownerType: "system", ownerAgentId: null, ownerUserId: null,
evidence: sql`${issueRecoveryActions.evidence} || jsonb_build_object('systemRecovery', ${JSON.stringify(next)}::jsonb ||
${next.generationId === previous.generationId && next.intentId === previous.intentId
? sql`jsonb_build_object('intentId', ${issueRecoveryActions.evidence}#>'{systemRecovery,intentId}')`
: sql`'{}'::jsonb`},
'workspaceRecovery', ${JSON.stringify(compatibility)}::jsonb || jsonb_build_object('requestIds',
coalesce(${issueRecoveryActions.evidence}#>'{workspaceRecovery,requestIds}', '[]'::jsonb)))`,
updatedAt: new Date(),
}).where(and(eq(issueRecoveryActions.companyId, source.companyId), eq(issueRecoveryActions.id, source.actionId),
sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,generationId}' = ${previous.generationId}`,
sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,revision}' = ${String(previous.revision)}`))
.returning({ id: issueRecoveryActions.id });
return Boolean(updated);
});
}
export async function loadSystemRecovery(db: Db, source: WorkspaceRecoverySourceIdentity) {
@@ -68,3 +80,19 @@ export async function loadSystemRecovery(db: Db, source: WorkspaceRecoverySource
eq(issueRecoveryActions.id, source.actionId)));
return readSystemRecovery(row?.evidence);
}
export async function nextWorkspaceRecoveryAt(db: Db): Promise<number | null> {
const rows = await db.select({ at: sql<number>`case
when jsonb_typeof(${issueRecoveryActions.evidence}#>'{systemRecovery,nextAttemptAt}') = 'number'
then (${issueRecoveryActions.evidence}#>>'{systemRecovery,nextAttemptAt}')::bigint else 0 end` })
.from(issueRecoveryActions).innerJoin(heartbeatRuns, and(
eq(heartbeatRuns.companyId, issueRecoveryActions.companyId),
eq(heartbeatRuns.id, sql`case when ${issueRecoveryActions.evidence}->>'runId' ~* '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
then (${issueRecoveryActions.evidence}->>'runId')::uuid end`),
sql`${heartbeatRuns.status} in ('failed', 'interrupted', 'timed_out', 'cancelled', 'succeeded')`,
)).where(and(
workspaceRecoverySourceEligible(),
sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,state}' in ('scheduled', 'running', 'awaiting_evidence')`,
)).orderBy(sql`1`).limit(1);
return rows.length ? Number(rows[0].at) : null;
}
server/src/services/task-retry.ts
+24 -3
View File
@@ -1,3 +1,5 @@
import { writeExecutionControl } from "./execution-control-notifications.js";
import { DELIVERY_QUEUES } from "./delivery-work-notifications.js";
import { z } from "zod";
import { randomUUID } from "node:crypto";
import { and, desc, eq, inArray, or, sql } from "drizzle-orm";
@@ -8,6 +10,20 @@ import { createSystemRecovery, readSystemRecovery } from "./system-recovery-stat
import { updateSystemRecovery } from "./system-recovery.js";
import { instanceSettingsService } from "./instance-settings.js";
import { notifyChatActionWork } from "./chat-work-notifications.js";
import { authorizationService } from "./authorization.js";
export async function taskRetryMutationAllowed(db: Db, input: {
companyId: string; issueId: string; actorId: string; actorSource: TaskRetryIntent["actorSource"];
}) {
const [issue] = await db.select({ status: issues.status, assigneeAgentId: issues.assigneeAgentId,
assigneeUserId: issues.assigneeUserId }).from(issues).where(and(eq(issues.companyId, input.companyId), eq(issues.id, input.issueId)));
if (!issue) return false;
const permission = await authorizationService(db).decide({
actor: { type: "board", userId: input.actorId, source: input.actorSource }, action: "issue:mutate",
resource: { type: "issue", companyId: input.companyId, issueId: input.issueId, ...issue },
});
return permission.allowed;
}
export async function assertTaskRetryConversationAllowed(
db: Db,
@@ -59,7 +75,7 @@ export async function requestTaskRetry(input: {
prepareChatRetry?: (tx: Db, scope: { companyId: string; issueId: string; agentId: string;
failedRunId: string; initiatedByUserId: string; retryIntentId?: string }) => Promise<{ actionId: string }>;
}) {
return input.db.transaction(async tx => {
return writeExecutionControl(input.db, DELIVERY_QUEUES.workspaceRecovery, async tx => {
const db = tx as unknown as Db;
const [issue] = await tx.select().from(issues).where(and(eq(issues.companyId, input.companyId),
eq(issues.id, input.issueId))).for("update");
@@ -115,6 +131,7 @@ export async function requestTaskRetry(input: {
const intent: TaskRetryIntent = { schema: "paperclip.task-retry.v1", generationId: active?.id ?? id,
sourceRunId: latest.id, actorId: input.actorId,
actorSource: taskRetryIntentSchema.shape.actorSource.parse(input.actorSource ?? "session"), strategy: "evaluate",
...(active && activeIntent?.chatActionId ? { chatActionId: activeIntent.chatActionId } : {}),
...(active ? { canonicalIntentId: active.id } : {}) };
// Preserve external chat authority through its existing admission owner.
// Generic board Retry must never manufacture a chat-inbound request.
@@ -139,6 +156,11 @@ export async function requestTaskRetry(input: {
for (const action of records) {
const previous = readSystemRecovery(action.evidence);
if (!previous || previous.state === "complete") continue;
if (previous.state === "running") {
await updateSystemRecovery(db, { companyId: issue.companyId, issueId: issue.id, runId: latest.id,
actionId: action.id }, previous, { ...previous, intentId: active?.id ?? id });
continue;
}
// Active repair keeps its attempt and cooldown. An exhausted generation
// gets a new budget only because this click is a new durable intent.
const next = previous.state === "exhausted"
@@ -148,8 +170,7 @@ export async function requestTaskRetry(input: {
: { ...previous, revision: previous.revision + 1, intentId: active?.id ?? id,
// Retry may advance our backoff, never a provider's cooldown and
// never an operation whose physical owner is still running.
nextAttemptAt: previous.state === "running" ? previous.nextAttemptAt
: Math.max(Date.now(), previous.providerCooldownAt ?? 0) };
nextAttemptAt: Math.max(Date.now(), previous.providerCooldownAt ?? 0) };
if (next.generationId !== previous.generationId) await persistActivity(db, {
companyId: issue.companyId, actorType: "system", actorId: "task-retry", entityType: "issue", entityId: issue.id,
action: "issue.recovery_generation_replaced", runId: latest.id,
server/src/services/workspace-recovery-source.ts
+23
View File
@@ -10,6 +10,29 @@ export const workspaceExportCauses = ["native_workspace_sync_out_unrecoverable",
export const terminalWorkspaceRunStates = ["failed", "interrupted", "timed_out", "cancelled", "succeeded"];
export type WorkspaceRecoverySourceIdentity = { companyId: string; issueId: string; runId: string; actionId: string };
export function workspaceRecoverySourceEligible(includeCompleted = false) {
const actionEligible = sql`(${issueRecoveryActions.status} in ('active', 'escalated') or
(${issueRecoveryActions.status} = 'resolved' and (
(${issueRecoveryActions.evidence}#>>'{automaticRecovery,policy}' = 'preserve_without_replay_v1'
and ${issueRecoveryActions.evidence}#>>'{automaticRecovery,replay}' = 'blocked'
and ${issueRecoveryActions.evidence}#>>'{automaticRecovery,runId}' = ${heartbeatRuns.id}::text) or
(${issueRecoveryActions.evidence}#>>'{historicalWorkspaceSource,schema}' = 'paperclip.historical-workspace-source.v1'
and ${issueRecoveryActions.evidence}#>>'{historicalWorkspaceSource,runId}' = ${heartbeatRuns.id}::text))))`;
return and(
sql`${issueRecoveryActions.evidence} ? 'systemRecovery'`,
includeCompleted ? sql`(${actionEligible} or ${issueRecoveryActions.evidence}#>>'{systemRecovery,state}' = 'complete')` : actionEligible,
sql`coalesce(${heartbeatRuns.nativeIssueId}::text, ${heartbeatRuns.issueId}::text, ${heartbeatRuns.contextSnapshot}->>'issueId') = ${issueRecoveryActions.sourceIssueId}::text`,
sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,sourceRunId}' = ${heartbeatRuns.id}::text`,
sql`${issueRecoveryActions.evidence}#>>'{workspaceRecovery,runId}' = ${heartbeatRuns.id}::text`,
sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,policyVersion}' = '1'`,
sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,schema}' = 'paperclip.system-recovery.v1'`,
sql`${issueRecoveryActions.evidence}#>>'{workspaceRecovery,schema}' = 'paperclip.workspace-recovery.v1'`,
sql`(${issueRecoveryActions.evidence}->'workspaceRestoreRecovery'->>'schema' = 'paperclip.workspace-restore-recovery.v1'
or ${issueRecoveryActions.evidence}->'historicalWorkspaceSource'->>'schema' = 'paperclip.historical-workspace-source.v1'
or ${issueRecoveryActions.cause} in ('native_workspace_sync_out_unrecoverable', 'native_workspace_sync_out_retry_exhausted'))`,
);
}
/** The preservation owner may reopen only its exact stop-only allocation.
* This does not release the source's ordinary execution or destruction fence. */
export async function ownedWorkspaceRecoveryLease(db: Db, input: WorkspaceRecoverySourceIdentity & {
server/src/services/workspace-recovery-work.ts
+37
View File
@@ -0,0 +1,37 @@
import type { Db } from "@paperclipai/db";
import type { createDeliveryWorkCoordinator } from "./delivery-work-coordinator.js";
import { DELIVERY_QUEUES } from "./delivery-work-notifications.js";
import { executionControlSchedule, registerExecutionControlWork } from "./execution-control-work.js";
import type { EnvironmentRuntimeService } from "./environment-runtime.js";
import { reconcileRetainedWorkspaces } from "./workspace-recovery.js";
export function registerWorkspaceRecoveryWork(db: Db, coordinator: ReturnType<typeof createDeliveryWorkCoordinator>,
runtime: EnvironmentRuntimeService, onRecovered: Parameters<typeof reconcileRetainedWorkspaces>[2],
canRun: () => boolean = () => true) {
const pending = new Set<Promise<void>>();
let stopped = false;
let scan: Promise<void> | undefined;
let wakeWhileBusy = false;
let passiveSweep: { cursor?: string } = {};
const schedule = executionControlSchedule(db);
const worker = registerExecutionControlWork(coordinator, [{
queue: DELIVERY_QUEUES.workspaceRecovery,
onSettled: () => { passiveSweep = {}; if (pending.size >= 2) wakeWhileBusy = true; },
run: () => scan = (async () => {
const work = await reconcileRetainedWorkspaces(db, runtime, onRecovered, true, continueSweep => {
if (!stopped && (continueSweep || wakeWhileBusy)) { wakeWhileBusy = false; worker.wake(); }
}, passiveSweep);
for (const job of work ?? []) {
pending.add(job);
const settled = () => { pending.delete(job); };
void job.then(settled, settled);
}
})(),
next: () => pending.size >= 2 ? Promise.resolve(null) : schedule.workspaceRecovery(),
}], () => !stopped && canRun());
return { ready: worker.ready, async stop() {
stopped = true;
await scan?.catch(() => {});
await Promise.all(pending);
} };
}
server/src/services/workspace-recovery.test.ts
+192 -6
View File
@@ -20,21 +20,29 @@ import { generateKeyPairSync, randomUUID, sign } from "node:crypto";
import { cp, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import express from "express";
import request from "supertest";
import { and, eq, inArray, sql } from "drizzle-orm";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { agents, approvals, budgetPolicies, costEvents, completionContracts, issueApprovals, issueTreeHolds, issueTreeHoldMembers, companies, createDb, environmentLeases, environments, heartbeatRuns, issueComments, issueRecoveryActions, issues, nativeRunFinalizations, nativeRunResults, agentWakeupRequests, workspaceOperations } from "@paperclipai/db";
import { agents, approvals, budgetPolicies, costEvents, completionContracts, issueApprovals, issueTreeHolds, issueTreeHoldMembers, companies, createDb, applyPendingMigrations, environmentLeases, environments, heartbeatRuns, issueComments, issueRecoveryActions, issues, nativeRunFinalizations, nativeRunResults, agentWakeupRequests, workspaceOperations } from "@paperclipai/db";
import { resumePreservedResultFinalization } from "./preserved-result-finalization.js";
import type { AdapterSandboxExecutionTarget } from "@paperclipai/adapter-utils/execution-target";
import type { EnvironmentLease } from "@paperclipai/shared";
import { startEmbeddedPostgresTestDatabase } from "../__tests__/helpers/embedded-postgres.js";
import { resolveExternalTestDatabaseUrl } from "../__tests__/helpers/external-test-database.js";
import { prepareNativeWorkspaceSync } from "./native-runtime/native-workspace-sync.js";
import { nativeWorkspaceFinalizationLockKey } from "./native-runtime/native-workspace-finalization-ownership.js";
import { remoteTerminationReceipt } from "./remote-execution-termination.js";
import { prepareStoppedSandboxForRecovery } from "./sandbox-stop-and-retain.js";
import { recoverRetainedWorkspace, requestWorkspaceRecovery, scheduleRecoveredWorkspaceCleanup, reconcileRetainedWorkspaces, retainedWorkspaceCandidates } from "./workspace-recovery.js";
import * as activity from "./activity-log.js";
import { createDeliveryWorkCoordinator } from "./delivery-work-coordinator.js";
import { registerWorkspaceRecoveryWork } from "./workspace-recovery-work.js";
import { heartbeatService } from "./heartbeat.js";
import { hasVerifiedWorkspaceRepair, readWorkspaceRecovery } from "./workspace-recovery-receipt.js";
import { getExecutionBlocker } from "./execution-blocker.js";
import { issueRoutes } from "../routes/issues.js";
import { errorHandler } from "../middleware/index.js";
import * as workerProof from "./native-runtime/native-workspace-platform-proof.js";
const targets = vi.hoisted(() => new Map<string, AdapterSandboxExecutionTarget>());
vi.mock("./environment-execution-target.js", async importOriginal => {
@@ -51,7 +59,13 @@ describe("automatic retained workspace recovery", () => {
beforeAll(async () => {
root = await mkdtemp(path.join(os.tmpdir(), "workspace-recovery-"));
process.env.PAPERCLIP_HOME = path.join(root, "home"); process.env.PAPERCLIP_INSTANCE_ID = "recovery";
temporary = await startEmbeddedPostgresTestDatabase("workspace-recovery-"); db = createDb(temporary.connectionString);
const external = resolveExternalTestDatabaseUrl("PAPERCLIP_WORKSPACE_RECOVERY_TEST_DATABASE_URL");
if (external) {
await applyPendingMigrations(external);
temporary = { connectionString: external, cleanup: async () => {} };
} else temporary = await startEmbeddedPostgresTestDatabase("workspace-recovery-");
db = createDb(temporary.connectionString);
if (external) await db.execute(sql`TRUNCATE TABLE companies CASCADE`);
}, 30_000);
afterAll(async () => {
await db?.$client.end(); await temporary?.cleanup(); await rm(root, { recursive: true, force: true });
@@ -123,6 +137,178 @@ describe("automatic retained workspace recovery", () => {
const action = async (id: string) => (await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, id)))[0]!;
const run = async (id: string) => (await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, id)))[0]!;
it("wakes physical preservation from HTTP Retry before passive history without replaying the provider", async () => {
for (let index = 0; index < 5; index++) {
const old = await seed();
await recoverRetainedWorkspace(old.input);
await db.update(issueRecoveryActions).set({
id: `00000000-000${index}-4000-8000-${randomUUID().slice(-12)}`,
}).where(eq(issueRecoveryActions.id, old.actionId));
}
const f = await seed();
await adoptSystemRecovery(db, f);
const before = await action(f.actionId);
const exhausted = failSystemRecovery(readSystemRecovery(before.evidence)!,
new SystemRecoveryFailure("source_binding_invalid", "service_required"));
await db.update(issueRecoveryActions).set({ cause: "native_workspace_sync_out_retry_exhausted",
evidence: { ...before.evidence, systemRecovery: exhausted } })
.where(eq(issueRecoveryActions.id, f.actionId));
const errors = vi.fn();
const competingCleanup = await retainedWorkspaceCandidates(db, undefined, false, true);
expect(competingCleanup).toHaveLength(5);
expect(competingCleanup.every(row => row.action.id.startsWith("00000000"))).toBe(true);
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, canReconcile: () => true, onError: errors });
const recovered = vi.fn(async () => {});
const worker = registerWorkspaceRecoveryWork(db, coordinator, f.environmentRuntime, recovered);
try {
await worker.ready;
await db.update(agents).set({ status: "paused" }).where(eq(agents.companyId, f.companyId));
const app = express();
app.use(express.json());
app.use((req, _res, next) => { req.actor = { type: "board", source: "local_implicit", userId: "board" }; next(); });
app.use("/api", issueRoutes(db, {} as Parameters<typeof issueRoutes>[1]));
app.use(errorHandler);
const body = { idempotencyKey: randomUUID(), observedRunId: f.runId };
const retry = await request(app).post(`/api/issues/${f.issueId}/retry`).send(body).expect(202);
const repeated = await request(app).post(`/api/issues/${f.issueId}/retry`).send(body).expect(202);
const coalesced = await request(app).post(`/api/issues/${f.issueId}/retry`)
.send({ ...body, idempotencyKey: randomUUID() }).expect(202);
expect(repeated.body.requestId).toBe(retry.body.requestId);
expect(coalesced.body.requestId).toBe(retry.body.requestId);
await expect.poll(async () => readSystemRecovery((await action(f.actionId)).evidence)?.state,
{ timeout: 20_000 }).toBe("complete");
expect(await readFile(path.join(f.host, "saved.txt"), "utf8")).toBe("finished work\n");
expect(f.runtime.resumeRunLease.mock.calls.filter(([input]) => input.lease.id === f.lease.id)).toHaveLength(1);
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, f.companyId))).toHaveLength(1);
expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.companyId, f.companyId))).toHaveLength(0);
const reloaded = await request(app).get(`/api/issues/${f.issueId}/recovery-actions`).expect(200);
expect(JSON.stringify(reloaded.body)).toContain('"state":"complete"');
expect(errors).not.toHaveBeenCalled();
} finally { await coordinator.stop(); await worker.stop(); }
}, 30_000);
it("reconstructs a resolved files-only preservation obligation at startup without replay", async () => {
const f = await seed("legacy");
await adoptSystemRecovery(db, f);
const before = await action(f.actionId);
const automaticRecovery = { policy: "preserve_without_replay_v1", runId: f.runId, replay: "blocked" };
await db.update(issueRecoveryActions).set({ status: "resolved", outcome: "blocked",
evidence: { ...before.evidence, automaticRecovery } }).where(eq(issueRecoveryActions.id, f.actionId));
const errors = vi.fn();
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, canReconcile: () => true, onError: errors });
const worker = registerWorkspaceRecoveryWork(db, coordinator, f.environmentRuntime, async () => {});
try {
await worker.ready;
await expect.poll(async () => readSystemRecovery((await action(f.actionId)).evidence)?.state,
{ timeout: 20_000 }).toBe("complete");
expect((await action(f.actionId)).evidence.automaticRecovery).toEqual(automaticRecovery);
expect(f.runtime.resumeRunLease.mock.calls.filter(([input]) => input.lease.id === f.lease.id)).toHaveLength(1);
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, f.companyId))).toHaveLength(1);
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, f.companyId))).toHaveLength(0);
expect(errors).not.toHaveBeenCalled();
} finally { await coordinator.stop(); await worker.stop(); }
}, 30_000);
it("fills a free export slot while another export is held and joins both at shutdown", async () => {
const held = await seed(), fast = await seed();
const fixtures = [held, fast];
const runtimeFor = (runId: string) => fixtures.find(fixture => fixture.runId === runId)!.environmentRuntime;
const runtime = { ...held.environmentRuntime,
resumeWorkspaceRecoveryLease: (input: Parameters<typeof held.environmentRuntime.resumeWorkspaceRecoveryLease>[0]) =>
runtimeFor(input.source.runId).resumeWorkspaceRecoveryLease(input),
retryPendingSandboxTeardown: (input: Parameters<typeof held.environmentRuntime.retryPendingSandboxTeardown>[0]) =>
runtimeFor(input.lease.heartbeatRunId!).retryPendingSandboxTeardown(input),
};
let active = 0, maximum = 0, release!: () => void;
const gate = new Promise<void>(resolve => { release = resolve; });
const countExport = (fixture: typeof held, pause = false) => {
const resume = fixture.runtime.resumeRunLease.getMockImplementation()!;
fixture.runtime.resumeRunLease.mockImplementation(async input => {
active += 1; maximum = Math.max(maximum, active);
try { if (pause) await gate; return await resume(input); } finally { active -= 1; }
});
};
countExport(held, true); countExport(fast);
const errors = vi.fn();
const coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, onError: errors });
const worker = registerWorkspaceRecoveryWork(db, coordinator, runtime, async () => {});
try {
await expect.poll(() => held.runtime.resumeRunLease.mock.calls.length, { timeout: 20_000 }).toBe(1);
await expect.poll(async () => readSystemRecovery((await action(fast.actionId)).evidence)?.state,
{ timeout: 20_000 }).toBe("complete");
const third = await seed(); fixtures.push(third); countExport(third);
await requestTaskRetry({ db, companyId: third.companyId, issueId: third.issueId, actorId: "board",
observedRunId: third.runId, idempotencyKey: randomUUID() });
await expect.poll(async () => readSystemRecovery((await action(third.actionId)).evidence)?.state,
{ timeout: 20_000 }).toBe("complete");
expect(readSystemRecovery((await action(held.actionId)).evidence)?.state).toBe("running");
expect(maximum).toBeLessThanOrEqual(2);
expect(await readFile(path.join(third.host, "saved.txt"), "utf8")).toBe("finished work\n");
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, third.companyId))).toHaveLength(1);
let stopped = false;
const stopping = Promise.all([coordinator.stop(), worker.stop()]).then(() => { stopped = true; });
await Promise.resolve(); expect(stopped).toBe(false);
release(); await stopping;
expect(readSystemRecovery((await action(held.actionId)).evidence)?.state).toBe("complete");
expect(errors).not.toHaveBeenCalled();
} finally { release(); await coordinator.stop(); await worker.stop(); }
}, 60_000);
it("rotates past due sources whose physical locks belong to other workers", async () => {
const blocked = [await seed(), await seed()], available = await seed();
for (const [index, fixture] of [...blocked, available].entries()) {
await adoptSystemRecovery(db, fixture);
const id = `00000001-000${index}-4000-8000-${randomUUID().slice(-12)}` as const;
await db.update(issueRecoveryActions).set({ id }).where(eq(issueRecoveryActions.id, fixture.actionId));
fixture.actionId = id; fixture.input.actionId = id;
}
let unlock!: () => void;
const gate = new Promise<void>(resolve => { unlock = resolve; });
const held = blocked.map(fixture => {
let ready!: () => void;
const acquired = new Promise<void>(resolve => { ready = resolve; });
const lock = db.transaction(async tx => {
await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${nativeWorkspaceFinalizationLockKey(fixture.companyId, fixture.runId)}, 0))`);
ready(); await gate;
});
return { acquired, lock };
});
try {
await Promise.all(held.map(item => item.acquired));
await Promise.all(await reconcileRetainedWorkspaces(db, available.environmentRuntime, async () => {}, true) ?? []);
expect(blocked.every(fixture => fixture.runtime.resumeRunLease.mock.calls.length === 0)).toBe(true);
await Promise.all(await reconcileRetainedWorkspaces(db, available.environmentRuntime, async () => {}, true) ?? []);
expect(readSystemRecovery((await action(available.actionId)).evidence)?.state).toBe("complete");
expect(await readFile(path.join(available.host, "saved.txt"), "utf8")).toBe("finished work\n");
expect(blocked.every(fixture => fixture.runtime.resumeRunLease.mock.calls.length === 0)).toBe(true);
} finally { unlock(); await Promise.all(held.map(item => item.lock)); }
}, 30_000);
it("coalesces Retry while preservation owns its physical operation", async () => {
const f = await seed();
await adoptSystemRecovery(db, f);
const resume = f.runtime.resumeRunLease.getMockImplementation()!;
let release!: () => void;
const gate = new Promise<void>(resolve => { release = resolve; });
f.runtime.resumeRunLease.mockImplementation(async input => { await gate; return resume(input); });
const preserving = recoverRetainedWorkspace(f.input);
let requestId: string | null | undefined;
try {
await expect.poll(() => f.runtime.resumeRunLease.mock.calls.length).toBe(1);
const request = { db, companyId: f.companyId, issueId: f.issueId, actorId: "board",
idempotencyKey: randomUUID(), observedRunId: f.runId };
const first = await requestTaskRetry(request);
requestId = first.requestId;
const second = await requestTaskRetry({ ...request, idempotencyKey: randomUUID() });
expect(second.requestId).toBe(first.requestId);
} finally { release(); await preserving; }
expect(readSystemRecovery((await action(f.actionId)).evidence)?.state).toBe("complete");
expect(readSystemRecovery((await action(f.actionId)).evidence)?.intentId).toBe(requestId);
expect(f.runtime.resumeRunLease.mock.calls.filter(([input]) => input.lease.id === f.lease.id)).toHaveLength(1);
expect(await readFile(path.join(f.host, "saved.txt"), "utf8")).toBe("finished work\n");
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, f.companyId))).toHaveLength(1);
}, 30_000);
it("adopts only the exact historical infrastructure notice, preserving prose and user-action notices", async () => {
const f = await seed("legacy", false);
const metadata = { version: 1 as const, sourceRunId: f.runId,
@@ -285,7 +471,7 @@ describe("automatic retained workspace recovery", () => {
expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.companyId, f.companyId))).toHaveLength(0);
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, f.companyId))).toHaveLength(0);
await recoverRetainedWorkspace(f.input);
expect(f.runtime.resumeRunLease).toHaveBeenCalledOnce();
expect(f.runtime.resumeRunLease.mock.calls.filter(([input]) => input.lease.id === f.lease.id)).toHaveLength(1);
expect(await db.select().from(workspaceOperations).where(and(eq(workspaceOperations.companyId, f.companyId), eq(workspaceOperations.phase, "workspace_recovery")))).toHaveLength(1);
}, 30_000);
@@ -304,9 +490,9 @@ describe("automatic retained workspace recovery", () => {
expect(readWorkspaceRecovery((await action(f.actionId)).evidence)?.status).toBe("recovered");
expect(await readFile(path.join(f.host, "saved.txt"), "utf8")).toBe("finished work\n");
expect(await readFile(path.join(f.host, "newer.txt"), "utf8")).toBe("newer local work\n");
expect(f.runtime.resumeRunLease).toHaveBeenCalledOnce();
expect(f.runtime.resumeRunLease.mock.calls.filter(([input]) => input.lease.id === f.lease.id)).toHaveLength(1);
await recoverRetainedWorkspace({ ...f.input, environmentRuntime: runtime });
expect(f.runtime.resumeRunLease).toHaveBeenCalledOnce();
expect(f.runtime.resumeRunLease.mock.calls.filter(([input]) => input.lease.id === f.lease.id)).toHaveLength(1);
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, f.companyId))).toHaveLength(1);
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, f.companyId))).toHaveLength(0);
}, 30_000);
@@ -405,7 +591,7 @@ describe("automatic retained workspace recovery", () => {
f.runtime.resumeRunLease.mockImplementationOnce(async () => { ready(); await gate; return { providerLeaseId: f.providerLeaseId, metadata: { remoteCwd: f.remote } }; });
const first = recoverRetainedWorkspace(f.input);
await started;
try { await recoverRetainedWorkspace(f.input); expect(f.runtime.resumeRunLease).toHaveBeenCalledOnce(); }
try { await recoverRetainedWorkspace(f.input); expect(f.runtime.resumeRunLease.mock.calls.filter(([input]) => input.lease.id === f.lease.id)).toHaveLength(1); }
finally { release(); await first; }
expect(readWorkspaceRecovery((await action(f.actionId)).evidence)?.status).toBe("recovered");
}, 30_000);
server/src/services/workspace-recovery.ts
+41 -24
View File
@@ -1,3 +1,5 @@
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
import { beginIdleTrackedWork } from "./task-admission.js";
import { writeEnvironmentCleanup } from "./environment-cleanup-notifications.js";
import { resumePreservedResultFinalization } from "./preserved-result-finalization.js";
import { and, asc, eq, gt, inArray, ne, or, sql } from "drizzle-orm";
@@ -19,7 +21,7 @@ import { classifyNativeWorkspaceFailure } from "./native-runtime/native-workspac
import { adoptSystemRecovery, loadSystemRecovery, updateSystemRecovery } from "./system-recovery.js";
import { createSystemRecovery, failSystemRecovery, readSystemRecovery, reserveSystemRecoveryAttempt, SystemRecoveryFailure, type SystemRecoveryState } from "./system-recovery-state.js";
import { inspectWorkspaceRecoverySource } from "./workspace-recovery-source.js";
import { inspectWorkspaceRecoverySource, workspaceRecoverySourceEligible } from "./workspace-recovery-source.js";
import { preserveWorkspaceSnapshot } from "./workspace-recovery-snapshot.js";
import { adoptHistoricalWorkspaceRecoveries } from "./workspace-recovery-backfill.js";
@@ -48,12 +50,13 @@ export async function requestWorkspaceRecovery(input: Source & { db: Db; idempot
requestIds: [...(prior?.requestIds ?? []), input.idempotencyKey] };
const previous = readSystemRecovery(action.evidence);
const active = previous && ["scheduled", "running", "awaiting_evidence"].includes(previous.state);
const system = active ? { ...previous, revision: previous.revision + 1,
nextAttemptAt: previous.state === "running" ? previous.nextAttemptAt : Math.max(Date.now(), previous.providerCooldownAt ?? 0) } : { ...createSystemRecovery(run.id),
const system = previous?.state === "running" ? previous : active ? { ...previous, revision: previous.revision + 1,
nextAttemptAt: Math.max(Date.now(), previous.providerCooldownAt ?? 0) } : { ...createSystemRecovery(run.id),
revision: (previous?.revision ?? -1) + 1,
preservationOperationIds: previous?.preservationOperationIds ?? [],
providerCooldownAt: previous?.providerCooldownAt ?? null,
nextAttemptAt: previous?.providerCooldownAt ?? null };
await notifyDeliveryWork(tx, DELIVERY_QUEUES.workspaceRecovery);
await tx.update(issueRecoveryActions).set({ ownerType: "system", ownerAgentId: null, ownerUserId: null,
evidence: { ...action.evidence, workspaceRecovery: state, systemRecovery: system }, updatedAt: new Date() })
.where(eq(issueRecoveryActions.id, action.id));
@@ -286,6 +289,7 @@ export async function recoverRetainedWorkspace(input: Source & { db: Db; environ
const cursors = new WeakMap<Db, string>();
const workInFlight = new WeakMap<Db, Set<string>>();
const unfinishedCursors = new WeakMap<Db, string>();
/** Destruction is a later cleanup obligation, authorized only by a matching
* saved-file receipt and the original recorded disposition. */
@@ -309,29 +313,31 @@ export async function scheduleRecoveredWorkspaceCleanup(db: Db, run: typeof hear
sql`${environmentLeases.metadata} = ${JSON.stringify(lease.metadata)}::jsonb`,
sql`${environmentLeases.metadata}->>'workspaceRecoveryCleanup' is null`)));
}
export function retainedWorkspaceCandidates(db: Db, cursor?: string) {
export function retainedWorkspaceCandidates(db: Db, cursor?: string, unfinishedOnly = false, obligationsOnly = false) {
const cleanup = sql`(${heartbeatRuns.runnerProfileJson}#>>'{nativeWorkspaceSync,resourceDisposition}' = 'destroy' and exists (
select 1 from ${environmentLeases} where ${environmentLeases.companyId} = ${heartbeatRuns.companyId}
and ${environmentLeases.heartbeatRunId} = ${heartbeatRuns.id}
and ${environmentLeases.id}::text = ${heartbeatRuns.runnerProfileJson}#>>'{nativeWorkspaceSync,leaseId}'
and ${environmentLeases.status} = 'released' and ${environmentLeases.cleanupStatus} = 'success'
and ${environmentLeases.metadata}->'workspaceRecoveryCleanup' is null))`;
const waitingInput = sql`exists (select 1 from ${agentWakeupRequests}
where ${agentWakeupRequests.companyId} = ${issueRecoveryActions.companyId}
and ${agentWakeupRequests.status} = 'deferred_issue_execution' and ${agentWakeupRequests.requestedByActorType} = 'user'
and ${agentWakeupRequests.payload}->>'issueId' = ${issueRecoveryActions.sourceIssueId}::text)`;
return db.select({ action: issueRecoveryActions, run: heartbeatRuns }).from(issueRecoveryActions)
.innerJoin(heartbeatRuns, and(eq(heartbeatRuns.companyId, issueRecoveryActions.companyId), eq(heartbeatRuns.id,
sql`case when ${issueRecoveryActions.evidence}->>'runId' ~* '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
then (${issueRecoveryActions.evidence}->>'runId')::uuid end`)))
.where(and(inArray(heartbeatRuns.status, terminal),
sql`(${issueRecoveryActions.evidence}->'workspaceRestoreRecovery'->>'schema' = 'paperclip.workspace-restore-recovery.v1'
or ${issueRecoveryActions.evidence}->'historicalWorkspaceSource'->>'schema' = 'paperclip.historical-workspace-source.v1'
or ${issueRecoveryActions.cause} in ('native_workspace_sync_out_unrecoverable', 'native_workspace_sync_out_retry_exhausted'))`,
sql`${issueRecoveryActions.evidence} ? 'systemRecovery'`,
workspaceRecoverySourceEligible(true),
unfinishedOnly ? sql`${issueRecoveryActions.evidence}#>>'{systemRecovery,state}' in ('scheduled', 'running', 'awaiting_evidence')` : undefined,
obligationsOnly ? sql`(${issueRecoveryActions.evidence}#>>'{systemRecovery,state}' in ('scheduled', 'running', 'awaiting_evidence')
or (${issueRecoveryActions.evidence}#>>'{systemRecovery,state}' = 'complete' and ${cleanup}) or ${waitingInput})` : undefined,
sql`(${issueRecoveryActions.evidence}#>>'{systemRecovery,state}' in ('scheduled', 'running', 'awaiting_evidence', 'complete', 'exhausted')
and (${issueRecoveryActions.evidence}#>'{systemRecovery,nextAttemptAt}' = 'null'::jsonb
or ${issueRecoveryActions.evidence}#>'{systemRecovery,nextAttemptAt}' <= to_jsonb(${Date.now()}::bigint)))`,
sql`(${issueRecoveryActions.evidence}#>>'{workspaceRecovery,status}' is distinct from 'recovered' or
(${heartbeatRuns.runnerProfileJson}#>>'{nativeWorkspaceSync,resourceDisposition}' = 'destroy' and exists (
select 1 from ${environmentLeases} where ${environmentLeases.companyId} = ${heartbeatRuns.companyId}
and ${environmentLeases.heartbeatRunId} = ${heartbeatRuns.id}
and ${environmentLeases.id}::text = ${heartbeatRuns.runnerProfileJson}#>>'{nativeWorkspaceSync,leaseId}'
and ${environmentLeases.status} = 'released' and ${environmentLeases.cleanupStatus} = 'success'
and ${environmentLeases.metadata}->'workspaceRecoveryCleanup' is null)) or exists (
select 1 from ${agentWakeupRequests} where ${agentWakeupRequests.companyId} = ${issueRecoveryActions.companyId}
and ${agentWakeupRequests.status} = 'deferred_issue_execution' and ${agentWakeupRequests.requestedByActorType} = 'user'
and ${agentWakeupRequests.payload}->>'issueId' = ${issueRecoveryActions.sourceIssueId}::text))`,
${cleanup} or ${waitingInput})`,
cursor ? gt(issueRecoveryActions.id, cursor) : undefined,
)).orderBy(asc(issueRecoveryActions.id)).limit(5);
}
@@ -339,19 +345,29 @@ export function retainedWorkspaceCandidates(db: Db, cursor?: string) {
/** Bounded discovery shares the periodic recovery lifecycle. A repaired source
* still owes cleanup even without saved input or after task reconciliation. */
export async function reconcileRetainedWorkspaces(db: Db, runtime: EnvironmentRuntimeService,
onRecovered: (run: typeof heartbeatRuns.$inferSelect) => Promise<void>) {
onRecovered: (run: typeof heartbeatRuns.$inferSelect) => Promise<void>, obligationsOnly = false,
onFinished?: (continueSweep: boolean) => void, passiveSweep?: { cursor?: string }) {
await adoptHistoricalWorkspaceRecoveries(db);
let pending = workInFlight.get(db);
if (!pending) { pending = new Set(); workInFlight.set(db, pending); }
if (pending.size >= 2) return;
let rows = await retainedWorkspaceCandidates(db, cursors.get(db));
if (!rows.length) rows = await retainedWorkspaceCandidates(db);
let rows = await retainedWorkspaceCandidates(db, unfinishedCursors.get(db), true);
if (!rows.length) rows = await retainedWorkspaceCandidates(db, undefined, true);
const unfinished = rows.length > 0;
if (!rows.length) rows = await retainedWorkspaceCandidates(db, passiveSweep ? passiveSweep.cursor : cursors.get(db), false, obligationsOnly);
if (!rows.length && !passiveSweep) rows = await retainedWorkspaceCandidates(db, undefined, false, obligationsOnly);
const work: Promise<void>[] = [];
let hasUnvisited = false;
for (const { action, run } of rows) {
if (pending.size >= 2) break;
cursors.set(db, action.id);
if (pending.size >= 2) { hasUnvisited = true; break; }
if (unfinished) unfinishedCursors.set(db, action.id);
else if (passiveSweep) passiveSweep.cursor = action.id;
else cursors.set(db, action.id);
if (pending.has(run.id)) continue;
pending.add(run.id);
void (async () => { try {
const finish = beginIdleTrackedWork();
const physical = ["scheduled", "running", "awaiting_evidence"].includes(readSystemRecovery(action.evidence)?.state ?? "");
work.push((async () => { try {
await recoverRetainedWorkspace({ db, environmentRuntime: runtime, companyId: action.companyId, issueId: action.sourceIssueId, runId: run.id, actionId: action.id });
const [updated] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, action.id));
const recovered = readWorkspaceRecovery(updated?.evidence ?? null)?.status === "recovered";
@@ -370,6 +386,7 @@ export async function reconcileRetainedWorkspaces(db: Db, runtime: EnvironmentRu
}
}
} catch { logger.warn({ runId: run.id }, "Retained workspace remains pending"); }
finally { pending.delete(run.id); } })();
finally { pending.delete(run.id); finish(); onFinished?.(physical || hasUnvisited); } })());
}
return work;
}
server/src/vendor/paperclip-runner/testing.ts
Vendored
+4
View File
@@ -26,6 +26,10 @@ const sourceUrl = new URL(
);
const runnerTesting = await import(sourceUrl.href) as RunnerTestingModule;
export const startOpenCodeMcpBridge:
RunnerTestingModule["startOpenCodeMcpBridge"] =
runnerTesting.startOpenCodeMcpBridge;
export const CAPABILITY_HIGH_RISK_SEMANTIC_VECTORS:
RunnerTestingModule["CAPABILITY_HIGH_RISK_SEMANTIC_VECTORS"] =
runnerTesting.CAPABILITY_HIGH_RISK_SEMANTIC_VECTORS;
tests/runner-e2e/SECURITY.md
+11
View File
@@ -506,6 +506,17 @@ select arbitrary destinations. Gateway response caching and fallback are refused
Request journal and usage receipts contain no prompt/response bodies or keys.
The manifest, journal, and receipts are not public report projections.
`boundedEvalPayload` is the shared, credential-free request validator and
normalizer. Qualification callers must apply it before recording budget-denial
proof. The forwarding path applies the same policy before a ledger reservation.
OpenAI tools may use one namespace level around local function or custom tools.
Function and custom leaves require nonblank string names.
Namespace wrappers contain only `type`, `name`, optional `description`, and
`tools`. Nested namespaces, hosted tools, and tool lists attached to leaves fail
closed. The request may contain at most 1,024 tool definitions, counting namespace
wrappers and their children. Anthropic tool policy stays separate. Output limits,
billing-mode restrictions, and multimodal exclusions apply in both paths.
## Configured repository and conversion proof
The repository proof flow seeds only the exact `workspace` child of its resolved
tests/runner-e2e/eval-gateway.test.ts
+66 -1
View File
@@ -6,7 +6,7 @@ import { connect } from "node:net";
import { afterEach, expect, it, vi } from "vitest";
import { openEvalCostLedger, readEvalCostLedger } from "./cost-ledger.js";
import { beginEvalAgentPhase, createEvalAttemptPhase, readEvalAttemptPhase } from "./attempt-phase.js";
import { evalRequestReservation, EvalUsageStream, forwardEvalRequest, startEvalGateway, type EvalGatewayRoute, type EvalGatewayReceipt } from "./eval-gateway.js";
import { boundedEvalPayload, evalRequestReservation, EvalUsageStream, forwardEvalRequest, startEvalGateway, type EvalGatewayRoute, type EvalGatewayReceipt } from "./eval-gateway.js";
const directories: string[] = [], closes: Array<() => Promise<void>> = [];
afterEach(async () => { for (const close of closes.splice(0).reverse()) await close(); for (const dir of directories.splice(0)) await rm(dir, { recursive: true, force: true }); });
const identity = { executionId: "hello", attemptId: "attempt", phase: "agent" as const };
@@ -135,6 +135,71 @@ it("admits local function and custom tools inside a Codex namespace", async () =
expect(result.outcome).toBe("succeeded");
expect(transport).toHaveBeenCalledOnce();
});
it("validates the native daemon's namespace envelope before reserving and forwarding", async () => {
const f = await fixture();
const localTool = (name: string) => ({ type: "function", name, description: "Synthetic local tool",
parameters: { type: "object", properties: {}, additionalProperties: false }, strict: true });
const tools = [...Array.from({ length: 7 }, (_, index) => localTool(`local_${index}`)),
{ type: "namespace", name: "local_tools", description: "Synthetic local namespace",
tools: Array.from({ length: 5 }, (_, index) => localTool(`operation_${index}`)) }];
const payload = Object.freeze({ ...f.payload, tools });
const normalized = boundedEvalPayload(payload, { provider: route.provider, price: route.price, maximumOutputTokens: 100 });
expect(normalized).toEqual({ ...payload, max_output_tokens: 100, service_tier: "default" });
expect(payload).not.toHaveProperty("service_tier");
expect(payload.max_output_tokens).toBe(500);
expect(f.ledger.snapshot().requests).toHaveLength(0);
const transport = vi.fn<typeof fetch>(async (_url, init) => {
expect(JSON.parse(init!.body as string)).toEqual(normalized);
expect((await readEvalCostLedger(f.file)).state.snapshot().inflight).toBe(1);
return response();
});
const receipt = await forwardEvalRequest({ ...f, payload, transport });
expect(receipt).toMatchObject({ outcome: "succeeded", costUsd: "0.000068000", final: true });
expect(transport).toHaveBeenCalledOnce();
});
it.each([
{ type: "namespace", name: " ", tools: [{ type: "function", name: "local" }] },
{ type: "namespace", name: "local", description: {}, tools: [{ type: "function", name: "local" }] },
{ type: "namespace", name: "local", tools: [] },
{ type: "namespace", name: "local", tools: null },
{ type: "namespace", name: "local", tools: [null] },
{ type: ["function"], name: "local" },
{ type: ["custom"], name: "local" },
{ type: "namespace", name: "local", tools: [{ type: ["function"], name: "local" }] },
{ type: "namespace", name: "local", tools: [{ type: ["custom"], name: "local" }] },
...[undefined, null, 7, "", " "].flatMap(name => ["function", "custom"].flatMap(type =>
[{ type, name }, { type: "namespace", name: "local", tools: [{ type, name }] }])),
{ type: "namespace", name: "local", tools: [{ type: "function", name: "local" }], future_hosted_tool: true },
{ type: "namespace", name: "local", tools: [{ type: "namespace", name: "nested", tools: [{ type: "function", name: "local" }] }] },
...["web_search", "file_search", "image_generation", "code_interpreter", "computer", "mcp", "unknown"].map(type =>
({ type: "namespace", name: "local", tools: [{ type }] })),
])("rejects malformed, nested, or hosted namespace tools before any side effects", async tool => {
const f = await fixture(), transport = vi.fn<typeof fetch>(), persist = vi.fn();
const payload = { ...f.payload, tools: [tool] };
expect(() => boundedEvalPayload(payload, f.route)).toThrow("unqualified_hosted_tool");
await expect(forwardEvalRequest({ ...f, payload, transport, persist })).rejects.toThrow("unqualified_hosted_tool");
expect(transport).not.toHaveBeenCalled(); expect(persist).not.toHaveBeenCalled();
expect(f.ledger.snapshot().requests).toHaveLength(0);
});
it.each(["flat", "namespace"] as const)("bounds the total %s tool definitions including namespace wrappers", async shape => {
const f = await fixture(), transport = vi.fn<typeof fetch>(), persist = vi.fn();
const tool = { type: "function", name: "local_tool" };
const tools = shape === "flat" ? Array.from({ length: 1024 }, () => tool)
: [{ type: "namespace", name: "local", tools: Array.from({ length: 1023 }, () => tool) }];
expect(boundedEvalPayload({ ...f.payload, tools }, f.route).tools).toEqual(tools);
const payload = { ...f.payload, tools: [...tools, tool] };
await expect(forwardEvalRequest({ ...f, payload, transport, persist })).rejects.toThrow("unqualified_hosted_tool");
expect(transport).not.toHaveBeenCalled(); expect(persist).not.toHaveBeenCalled();
expect(f.ledger.snapshot().requests).toHaveLength(0);
});
it("keeps Anthropic custom tools and output limits separate from OpenAI namespaces", () => {
const anthropic = { provider: "anthropic" as const, price: { ...route.price, provider: "anthropic" as const }, maximumOutputTokens: 100 };
const tools = [{ name: "local_tool", input_schema: { type: "object" } }, { type: "custom", name: "other_tool" }];
expect(boundedEvalPayload({ model: "fixture", tools, max_tokens: 500 }, anthropic))
.toEqual({ model: "fixture", tools, max_tokens: 100, service_tier: "standard_only" });
expect(() => boundedEvalPayload({ model: "fixture", tools: [{ type: "namespace", name: "local", tools }] }, anthropic))
.toThrow("unqualified_hosted_tool");
});
it.each([
{ type: "namespace", name: "local", tools: [{ type: "web_search" }] },
{ type: "namespace", name: "local", tools: [{ type: "namespace", name: "nested", tools: [{ type: "web_search" }] }] },
tests/runner-e2e/eval-gateway.ts
+26 -11
View File
@@ -65,19 +65,34 @@ function modelReservationBound(route: EvalGatewayRoute, reservation: string) {
}) : undefined;
}
function boundedPayload(value: unknown, route: EvalGatewayRoute) {
function qualifiedOpenAiTools(tools: unknown): boolean {
if (!Array.isArray(tools)) return false;
let remaining = 1024;
const leaf = (tool: unknown): boolean => {
const definition = row(tool);
return --remaining >= 0 && (definition.type === "function" || definition.type === "custom")
&& typeof definition.name === "string" && Boolean(definition.name.trim()) && !Object.hasOwn(definition, "tools");
};
return tools.every(tool => {
const definition = row(tool);
if (definition.type !== "namespace") return leaf(tool);
return --remaining >= 0 && typeof definition.name === "string" && Boolean(definition.name.trim())
&& (definition.description === undefined || typeof definition.description === "string")
&& Object.keys(definition).every(key => ["type", "name", "description", "tools"].includes(key))
&& Array.isArray(definition.tools) && definition.tools.length > 0 && definition.tools.every(leaf);
});
}
export function boundedEvalPayload(value: unknown, route: Pick<EvalGatewayRoute, "provider" | "price" | "maximumOutputTokens">): Record<string, unknown> {
const body = row(value);
if (body.model !== route.price.model) throw new EvalRequestError("unqualified_model");
if (body.background === true || body.mcp_servers !== undefined || body.service_tier !== undefined && !["auto", "default", "standard_only"].includes(String(body.service_tier))) throw new EvalRequestError("unqualified_billing_mode");
if (body.speed !== undefined && body.speed !== "standard" || body.inference_geo !== undefined) throw new EvalRequestError("unqualified_billing_mode");
if (body.tools !== undefined && (!Array.isArray(body.tools) || body.tools.some(tool => {
const definition = row(tool), type = definition.type;
if (route.provider !== "openai") return type !== undefined && type !== "custom" || Object.hasOwn(definition, "tools");
if (type === "namespace") return typeof definition.name !== "string" || !definition.name
|| !Array.isArray(definition.tools) || !definition.tools.length
|| definition.tools.some(nested => !["function", "custom"].includes(String(row(nested).type)) || Object.hasOwn(row(nested), "tools"));
return !["function", "custom"].includes(String(type)) || Object.hasOwn(definition, "tools");
}))) throw new EvalRequestError("unqualified_hosted_tool");
if (body.tools !== undefined && (route.provider === "openai" ? !qualifiedOpenAiTools(body.tools)
: !Array.isArray(body.tools) || body.tools.some(tool => {
const definition = row(tool), type = definition.type;
return type !== undefined && type !== "custom" || Object.hasOwn(definition, "tools");
}))) throw new EvalRequestError("unqualified_hosted_tool");
const inspect = (item: unknown): void => {
if (Array.isArray(item)) { item.forEach(inspect); return; }
if (!item || typeof item !== "object") return;
@@ -119,8 +134,8 @@ export async function forwardEvalRequest(input: {
transport?: typeof fetch;
}): Promise<EvalGatewayReceipt> {
const { route, ledger } = input;
let body: ReturnType<typeof boundedPayload>;
try { body = boundedPayload(input.payload, route); }
let body: ReturnType<typeof boundedEvalPayload>;
try { body = boundedEvalPayload(input.payload, route); }
catch (error) { throw error instanceof EvalRequestError ? error : new EvalRequestError("invalid_request_payload"); }
const reservation = evalRequestReservation(route);
if (!route.apiKey || /[\r\n]/.test(route.apiKey)) throw new Error("missing_gateway_credential");
ui/src/components/Layout.production.tsx
+9 -4
View File
@@ -45,6 +45,7 @@ import { useSidebar } from "../context/SidebarContext";
import { useKeyboardShortcuts } from "../hooks/useKeyboardShortcuts";
import { useCompanyPageMemory } from "../hooks/useCompanyPageMemory";
import { useMobileNavVisibility } from "../hooks/useMobileNavVisibility";
import { useMobileVisualViewport } from "../hooks/useMobileVisualViewport";
import { healthApi } from "../api/health";
import {
resolveArchivedCompanyBounce,
@@ -178,6 +179,7 @@ export function Layout() {
const scrollMemory = useRef(new NavigationScrollMemory());
const activeScrollKey = useRef<string>(location.key);
const mobileNavVisible = useMobileNavVisibility(isMobile, location.pathname);
const mobileViewportRef = useMobileVisualViewport(isMobile);
const [shortcutsOpen, setShortcutsOpen] = useState(false);
const matchedCompany = useMemo(() => {
if (!companyPrefix) return null;
@@ -588,8 +590,10 @@ export function Layout() {
return (
<ChatSetupSidebarProvider>
<div
ref={mobileViewportRef}
data-mobile={isMobile ? "true" : undefined}
className={cn(
"bg-background text-foreground pt-(--sz-safe-top)",
"paperclip-app-shell bg-background text-foreground pt-(--sz-safe-top)",
// overflow-x-clip on mobile keeps a stray wide descendant from making the
// whole viewport scroll horizontally. clip (not hidden) leaves overflow-y
// computed as visible, so native body scroll + the sticky breadcrumb keep
@@ -609,7 +613,7 @@ export function Layout() {
<DevRestartBanner devServer={health?.devServer} />
<div
className={cn(
"min-h-0 flex-1",
"app-main-row min-h-0 flex-1",
isMobile ? "w-full" : "flex overflow-clip",
)}
>
@@ -663,12 +667,13 @@ export function Layout() {
<div
className={cn(
"flex min-w-0 flex-col",
"app-content-column flex min-w-0 flex-col",
isMobile ? "w-full" : "h-full flex-1",
)}
>
<div
className={cn(
"app-breadcrumb",
isMobile &&
"sticky top-0 z-20 bg-background/95 backdrop-blur supports-[backdrop-filter]:bg-background/85",
)}
@@ -681,7 +686,7 @@ export function Layout() {
</div>
) : null}
</div>
<div className={cn(isMobile ? "block" : "flex flex-1 min-h-0")}>
<div className={cn("app-content-row", isMobile ? "block" : "flex flex-1 min-h-0")}>
<main
id="main-content"
ref={mainContentRef}
ui/src/components/Layout.tsx
+9 -3
View File
@@ -40,6 +40,7 @@ import { useKeyboardShortcuts } from "../hooks/useKeyboardShortcuts";
import { useStreamlinedUiEnabled } from "../hooks/useStreamlinedUiEnabled";
import { useCompanyPageMemory } from "../hooks/useCompanyPageMemory";
import { useMobileNavVisibility } from "../hooks/useMobileNavVisibility";
import { useMobileVisualViewport } from "../hooks/useMobileVisualViewport";
import { healthApi } from "../api/health";
import { resolveArchivedCompanyBounce, shouldSyncCompanySelectionFromRoute } from "../lib/company-selection";
import { useOptionalToastActions } from "../context/ToastContext";
@@ -143,6 +144,7 @@ export function Layout({ sidebarSections }: { sidebarSections?: ReactNode }) {
const scrollMemory = useRef(new NavigationScrollMemory());
const activeScrollKey = useRef<string>(location.key);
const mobileNavVisible = useMobileNavVisibility(isMobile, location.pathname);
const mobileViewportRef = useMobileVisualViewport(isMobile);
const [shortcutsOpen, setShortcutsOpen] = useState(false);
const matchedCompany = useMemo(() => {
if (!companyPrefix) return null;
@@ -574,8 +576,10 @@ export function Layout({ sidebarSections }: { sidebarSections?: ReactNode }) {
return (
<ChatSetupSidebarProvider>
<div
ref={mobileViewportRef}
data-mobile={isMobile ? "true" : undefined}
className={cn(
"bg-background text-foreground pt-(--sz-safe-top)",
"paperclip-app-shell bg-background text-foreground pt-(--sz-safe-top)",
// overflow-x-clip on mobile keeps a stray wide descendant from making the
// whole viewport scroll horizontally. clip (not hidden) leaves overflow-y
// computed as visible, so native body scroll + the sticky breadcrumb keep
@@ -591,7 +595,7 @@ export function Layout({ sidebarSections }: { sidebarSections?: ReactNode }) {
</a>
<WorktreeBanner />
<DevRestartBanner devServer={health?.devServer} />
<div className={cn("min-h-0 flex-1", isMobile ? "w-full" : "flex overflow-clip")}>
<div className={cn("app-main-row min-h-0 flex-1", isMobile ? "w-full" : "flex overflow-clip")}>
{isMobile && sidebarOpen && (
<button
type="button"
@@ -656,9 +660,10 @@ export function Layout({ sidebarSections }: { sidebarSections?: ReactNode }) {
</SecondarySidebar>
) : null}
<div className={cn("flex min-w-0 flex-col", isMobile ? "w-full" : "h-full flex-1")}>
<div className={cn("app-content-column flex min-w-0 flex-col", isMobile ? "w-full" : "h-full flex-1")}>
<div
className={cn(
"app-breadcrumb",
!isMobile && useStreamlinedTaskDetailShell && "hidden",
isMobile && "sticky top-0 z-20 bg-background/95 backdrop-blur supports-[backdrop-filter]:bg-background/85",
)}
@@ -672,6 +677,7 @@ export function Layout({ sidebarSections }: { sidebarSections?: ReactNode }) {
) : null}
</div>
<div className={cn(
"app-content-row",
isMobile ? "block" : "flex flex-1 min-h-0",
!isMobile && useStreamlinedTaskDetailShell && "streamlined-task-detail-surface",
)}>
ui/src/components/TaskChatThread.test.tsx
+37 -14
View File
@@ -26,6 +26,7 @@ const transcriptState = vi.hoisted(() => ({
transcriptByRun: new Map(),
isInitialHydrating: false,
hydratedRunIds: undefined as Set<string> | undefined,
truncatedRunIds: new Set<string>(),
}));
const nativeTranscriptState = vi.hoisted(() => ({
transcriptByRun: new Map(),
@@ -35,6 +36,7 @@ const nativeTranscriptState = vi.hoisted(() => ({
}));
const transcriptHookRuns = vi.hoisted(() => ({
legacy: [] as unknown[][],
legacyReadLimits: [] as (number | undefined)[],
native: [] as unknown[][],
}));
const sidebarState = vi.hoisted(() => ({ isMobile: false }));
@@ -51,12 +53,14 @@ const editorCallbacks = vi.hoisted(() => ({ change: (_value: string) => {} }));
const streamlinedState = vi.hoisted(() => ({ enabled: true }));
vi.mock("@/components/transcript/useLiveRunTranscripts", () => ({
useLiveRunTranscripts: ({ runs }: { runs: unknown[] }) => {
useLiveRunTranscripts: ({ runs, logReadLimitBytes }: { runs: unknown[]; logReadLimitBytes?: number }) => {
transcriptHookRuns.legacy.push(runs);
transcriptHookRuns.legacyReadLimits.push(logReadLimitBytes);
return {
transcriptByRun: new Map(transcriptState.transcriptByRun),
isInitialHydrating: transcriptState.isInitialHydrating,
hydratedRunIds: transcriptState.hydratedRunIds,
truncatedRunIds: transcriptState.truncatedRunIds,
};
},
}));
@@ -122,11 +126,13 @@ beforeEach(() => {
transcriptState.transcriptByRun.clear();
transcriptState.isInitialHydrating = false;
transcriptState.hydratedRunIds = undefined;
transcriptState.truncatedRunIds.clear();
nativeTranscriptState.transcriptByRun.clear();
nativeTranscriptState.errorsByRun.clear();
nativeTranscriptState.hydratedRunIds = undefined;
nativeTranscriptState.terminalHydratedRunIds = undefined;
transcriptHookRuns.legacy.length = 0;
transcriptHookRuns.legacyReadLimits.length = 0;
transcriptHookRuns.native.length = 0;
sidebarState.isMobile = false;
planState.data = null;
@@ -371,6 +377,27 @@ it.each([true, false])("keeps cold-open messages readable while history is pendi
expect(container.querySelector('[data-testid="mock-editor"]')).not.toBeNull();
});
it.each([0, 32_000, 100_000])("offers earlier output only when a single legacy run exceeds the compact window (%s bytes)", (logBytes) => {
const onExpandHistory = vi.fn();
if (logBytes > 32_000) transcriptState.truncatedRunIds.add("legacy-run");
const props = {
issueId: "single-legacy-run", comments: [], onAdd: async () => {}, onExpandHistory,
linkedRuns: [{ runId: "legacy-run", agentId: "agent", status: "succeeded", adapterType: "claude_local", runtimeMode: "legacy" as const, createdAt: "2025-01-01T10:00:00Z", startedAt: "2025-01-01T10:00:00Z", logBytes }],
};
render(<TaskChatThread {...props} compactHistory />);
expect(container.textContent).not.toContain("More messages");
const outputControl = [...container.querySelectorAll("button")].find((button) => button.textContent === "More run output");
expect(Boolean(outputControl)).toBe(logBytes > 32_000);
expect(transcriptHookRuns.legacyReadLimits.at(-1)).toBe(32_000);
if (outputControl) {
act(() => outputControl.click());
expect(onExpandHistory).toHaveBeenCalledOnce();
render(<TaskChatThread {...props} compactHistory={false} />);
expect(container.textContent).not.toContain("More run output");
expect(transcriptHookRuns.legacyReadLimits.at(-1)).toBeUndefined();
}
});
it("defers initial history for a saved conversation with its own draft key", () => {
render(<TaskChatThread comments={[]} onAdd={async () => {}} issueId="saved-chat"
conversationMode draftKey="agent-chat:company:saved-agent" initialHistoryPending />);
@@ -3535,7 +3562,7 @@ describe("TaskChatThread composer alignment", () => {
const dock = container
.querySelector('[data-testid="mock-editor"]')
?.closest("div.sticky") as HTMLElement | null;
?.closest('[data-testid="task-chat-composer-dock"]') as HTMLElement | null;
expect(dock?.className).toContain("w-full");
expect(dock?.className).toContain("max-w-(--tc-shell-max-w)");
@@ -4319,26 +4346,22 @@ describe("TaskChatThread Paperclip Runner queue", () => {
});
});
describe("TaskChatThread mobile composer dock (PAP-495)", () => {
it("pins the composer to the nav-aware bottom offset so its action row clears the auto-hiding bottom nav", () => {
describe("TaskChatThread mobile composer dock", () => {
it("scrolls messages independently of the composer on mobile", () => {
sidebarState.isMobile = true;
renderThread({
comments: [],
comments: createLongThreadComments(),
onAdd: async () => {},
draftKey: "task-chat-draft:issue-mobile",
});
const dock = container
.querySelector('[data-testid="mock-editor"]')
?.closest("div.sticky") as HTMLElement | null;
const dock = container.querySelector('[data-testid="task-chat-composer-dock"]');
const scroller = container.querySelector('[data-testid="task-chat-scroller"]');
expect(dock).not.toBeNull();
// Bottom offset comes from --tc-composer-bottom (Layout raises it to the nav
// height while the nav is on screen) — NOT the raw safe-area dock, which is
// what let the nav occlude the action row before PAP-495.
expect(dock?.className).toContain("bottom-(--tc-composer-bottom)");
expect(dock?.className).not.toContain("bottom-(--sz-calc-8)");
expect(scroller).not.toBeNull();
expect(scroller?.textContent).toContain("Thread message 1");
expect(scroller?.contains(dock)).toBe(false);
});
});
ui/src/components/TaskChatThread.tsx
+17 -19
View File
@@ -110,7 +110,6 @@ import {
useRunnerGoalControl,
} from "@/components/task-chat/RunnerGoalWidget";
import { TaskChatQueuedMessages } from "@/components/task-chat/TaskChatQueuedMessages";
import { TaskChatWindowScroll } from "@/components/task-chat/useWindowAutoFollow";
import { useSidebar } from "@/context/SidebarContext";
import { useStreamlinedUiEnabled } from "@/hooks/useStreamlinedUiEnabled";
import { useDismissedTaskQuestions } from "@/hooks/useDismissedTaskQuestions";
@@ -139,6 +138,7 @@ import { useQueryClient } from "@tanstack/react-query";
import { TaskChatPresentationProvider } from "@/components/task-chat/presentation-mode";
const EMPTY_BROWSERS: TaskBrowser[] = [];
const COMPACT_RUN_LOG_READ_LIMIT_BYTES = 32_000;
function toMs(value: Date | string | null | undefined): number {
if (!value) return 0;
@@ -425,6 +425,7 @@ export type TaskChatThreadProps = ComponentProps<typeof IssueChatThread> & {
creationActivity?: ActivityEvent[];
compactHistory?: boolean;
historyControl?: ReactNode;
onExpandHistory?: () => void;
initialHistoryPending?: boolean;
initialHistoryError?: boolean;
onRetryInitialHistory?: () => void;
@@ -967,9 +968,10 @@ export function TaskChatThread(props: TaskChatThreadProps) {
transcriptByRun: logTranscriptByRun,
isInitialHydrating: logsAreInitiallyHydrating,
hydratedRunIds: hydratedLogRunIds,
truncatedRunIds,
errorsByRun: logErrorsByRun,
retry: retryLogs,
} = useLiveRunTranscripts({ runs: logRuns, companyId, ...(props.compactHistory ? { logReadLimitBytes: 32_000 } : {}) });
} = useLiveRunTranscripts({ runs: logRuns, companyId, ...(props.compactHistory ? { logReadLimitBytes: COMPACT_RUN_LOG_READ_LIMIT_BYTES } : {}) });
const fallbackByRunRef = useRef(
new Map<string, NonNullable<ReturnType<typeof logTranscriptByRun.get>>>(),
);
@@ -2825,9 +2827,15 @@ export function TaskChatThread(props: TaskChatThreadProps) {
],
);
const historyControl = props.historyControl ?? (
props.compactHistory && props.onExpandHistory
&& logRuns.some((run) => truncatedRunIds?.has(run.id))
? <Button type="button" variant="ghost" size="sm" onClick={props.onExpandHistory}>More run output</Button>
: null
);
const renderBrief = useCallback(
() => issueBrief ? <><TaskChatDescriptionBubble brief={issueBrief} />{props.historyControl}</> : null,
[issueBrief, props.historyControl],
() => issueBrief ? <><TaskChatDescriptionBubble brief={issueBrief} />{historyControl}</> : null,
[issueBrief, historyControl],
);
const assignedAgentForNotice = useMemo(() => {
@@ -3040,25 +3048,19 @@ export function TaskChatThread(props: TaskChatThreadProps) {
<TaskChatRequestHistoryContext.Provider value={issueId ?? "unknown"}>
<TaskChatExpansionState.Provider value={expansionState.current}>
<TaskChatScrollReady.Provider value={!historyPending || historyWaitExpired}>
<TaskChatWindowScroll
contentKey={isMobile ? autoFollowContentKey : 0}
enabled={isMobile && historyRevealed}
/>
<TaskChatPresentationProvider
mode={streamlinedUiEnabled ? "streamlined" : "production"}
>
<div
className={cn(
"flex flex-col",
!isMobile && "min-h-0 flex-1",
"flex min-h-0 flex-1 flex-col",
isMobile && !historyRevealed && "task-chat-history-pending",
)}
data-testid="task-chat-thread"
>
<div
className={cn(
"relative flex flex-col",
(!isMobile || !historyRevealed) && "min-h-0 flex-1",
"relative flex min-h-0 flex-1 flex-col",
)}
aria-busy={!historyRevealed}
>
@@ -3093,8 +3095,7 @@ export function TaskChatThread(props: TaskChatThreadProps) {
<div
data-testid="task-chat-history-content"
className={cn(
"flex flex-col",
!isMobile && "min-h-0 flex-1",
"flex min-h-0 flex-1 flex-col",
!historyRevealed && "invisible",
)}
inert={!historyRevealed}
@@ -3103,9 +3104,7 @@ export function TaskChatThread(props: TaskChatThreadProps) {
<>
{items.length === 0 && !tailRunId && !optimisticRunnerStartup ? (
<div
className={
isMobile ? undefined : "min-h-0 flex-1 overflow-y-auto"
}
className="min-h-0 flex-1 overflow-y-auto"
>
{threadHeaderWithBlockers ? (
<div
@@ -3137,7 +3136,7 @@ export function TaskChatThread(props: TaskChatThreadProps) {
<TaskChatThreadView
items={items}
attachments={attachments}
header={<>{threadHeaderWithBlockers}{!issueBrief ? props.historyControl : null}</>}
header={<>{threadHeaderWithBlockers}{!issueBrief ? historyControl : null}</>}
renderInteraction={renderInteraction}
renderBrief={renderBrief}
renderMessageActions={renderMessageActions}
@@ -3241,7 +3240,6 @@ export function TaskChatThread(props: TaskChatThreadProps) {
}
contentKey={`${autoFollowContentKey}:${historyRevealed}`}
className={isMobile ? undefined : "pt-3"}
scroll={!isMobile}
/>
)}
</>
ui/src/components/ai-connections/AiConnectionField.test.tsx
+29 -2
View File
@@ -17,7 +17,6 @@ vi.mock("./AiConnectionCredentialStep", () => ({
return <div>Provider sign-in</div>;
},
}));
vi.mock("./AiConnectionManagement", () => ({ AiConnectionLegacyNotice: () => null }));
vi.mock("./AiProviderSetup", () => ({ AiProviderSetup: () => <div>Advanced provider setup</div> }));
vi.mock("@/pages/apps/AppLogo", () => ({ AppLogo: () => null }));
@@ -82,6 +81,33 @@ it("reconnects the unavailable personal default in place", async () => {
expect(onChange).toHaveBeenCalledWith({ provider: "anthropic", method: "subscription", mode: "responsible_user" });
});
it("identifies duplicate-named accounts before adopting the exact personal grant", async () => {
const personal = account({ id: "00000000-0000-4000-8000-000000000001", grantId: "00000000-0000-4000-8000-000000000002", provider: "openai", method: "api_key", name: "OpenAI", ownerName: "Avery", status: "connected", isDefault: false });
const shared = account({ id: "00000000-0000-4000-8000-000000000003", grantId: "00000000-0000-4000-8000-000000000004", provider: "openai", method: "subscription", name: "OpenAI", accountLabel: "engineering@example.test", ownership: "shared", ownerUserId: undefined, status: "connected", isDefault: false });
const sharedKey = account({ id: "00000000-0000-4000-8000-000000000005", grantId: "00000000-0000-4000-8000-000000000006", provider: "openai", method: "api_key", name: "OpenAI", ownerName: "Engineering", ownership: "shared", ownerUserId: undefined, status: "connected", isDefault: false });
const unavailable = account({ id: "expired-account", grantId: "expired-grant", provider: "openai", method: "api_key", name: "Expired account", ownerName: "Avery" });
mocks.list.mockResolvedValue({ currentUserId: "owner", connections: [shared, personal, sharedKey, unavailable], canManageConnections: true });
flushSync(() => root.render(<QueryClientProvider client={client}>
<AiConnectionField companyId="company" agentId="agent" agentName="Nova" adapterType="codex_local" legacy onChange={onChange} />
</QueryClientProvider>));
await settle();
await click("Choose a managed connection");
const trigger = container.querySelector('[role="combobox"][aria-label="Connection"]')!;
flushSync(() => trigger.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowDown", bubbles: true })));
await settle();
const choices = [...document.querySelectorAll<HTMLElement>('[role="option"]')];
expect(choices.map(choice => choice.textContent)).toContain("OpenAI · API key · Personal · Avery");
expect(choices.map(choice => choice.textContent)).toContain("OpenAI · ChatGPT subscription · Company shared · engineering@example.test");
expect(choices.map(choice => choice.textContent)).toContain("OpenAI · API key · Company shared · Engineering");
expect(choices.find(choice => choice.textContent?.includes("Expired account"))?.getAttribute("aria-disabled")).toBe("true");
flushSync(() => choices.find(choice => choice.textContent === "OpenAI · API key · Personal · Avery")!.click());
await settle();
expect(document.querySelector('[role="dialog"]')?.textContent).toContain("OpenAI · API key · Personal · Avery");
expect(onChange).not.toHaveBeenCalled();
await click("Use this binding when saved");
expect(onChange).toHaveBeenCalledExactlyOnceWith({ provider: "openai", method: "api_key", mode: "delegated", connectionId: personal.id, grantId: personal.grantId });
});
it("opens provider setup directly when connecting from the advanced mode", async () => {
await mount([], true, true);
await click("Connect another account");
@@ -163,7 +189,7 @@ it("offers Pi account selection before choosing its required model", async () =>
});
it.each(["openai", "anthropic", "google", "openrouter"] as const)("keeps a matching saved Pi %s API-key account selectable", async provider => {
const saved = account({ provider, method: "api_key", name: "Pi provider account", status: "connected" });
const saved = account({ provider, method: "api_key", name: "Pi provider account", accountLabel: "avery@example.test", status: "connected" });
mocks.list.mockResolvedValue({ currentUserId: "owner", connections: [saved], canManageConnections: true });
flushSync(() => root.render(<QueryClientProvider client={client}>
<AiConnectionField companyId="company" agentId="agent" agentName="Pi" adapterType="pi_local" model={`${provider}/provider-model`}
@@ -174,6 +200,7 @@ it.each(["openai", "anthropic", "google", "openrouter"] as const)("keeps a match
expect(container.textContent).not.toContain("incompatible");
expect(container.textContent).not.toContain("does not support");
const trigger = container.querySelector('[role="combobox"][aria-label="Connection"]')!;
expect(trigger.getAttribute("title")).toBe("Pi provider account · API key · Personal · avery@example.test");
flushSync(() => trigger.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowDown", bubbles: true })));
await settle();
const choice = [...document.querySelectorAll<HTMLElement>('[role="option"]')].find(node => node.textContent?.includes("Pi provider account"));
ui/src/components/ai-connections/AiConnectionField.tsx
+8 -4
View File
@@ -14,6 +14,7 @@ import { AiConnectionSelect } from "./AiConnectionSelect";
import { AiProviderSetup } from "./AiProviderSetup";
import { AiConnectionCredentialStep } from "./AiConnectionCredentialStep";
import { AiConnectionLegacyNotice } from "./AiConnectionManagement";
import { aiConnectionDisplayLabel } from "./model";
import { Button } from "@/components/ui/button";
import { Checkbox } from "@/components/ui/checkbox";
import {
@@ -89,6 +90,11 @@ export function AiConnectionField({
enabled: Boolean(provider),
});
const personalDefault = accounts.data?.connections.find((account) => account.provider === provider && account.isDefault && account.ownership === "personal" && account.ownerUserId === accounts.data.currentUserId);
const pendingAccount = pendingAdoption?.mode === "responsible_user"
? personalDefault
: pendingAdoption && pendingAdoption.mode !== "router"
? accounts.data?.connections.find(account => account.id === pendingAdoption.connectionId && account.grantId === pendingAdoption.grantId)
: undefined;
const selectDefault = useMutation({
mutationFn: async (result: NonNullable<typeof savedAccount>) => {
// Reconnect retains the existing default and its access. A new account
@@ -177,10 +183,8 @@ export function AiConnectionField({
</DialogHeader>
{pendingAdoption?.mode !== "router" && <p className="text-sm">
{pendingAdoption?.mode === "responsible_user"
? `Responsible user’s default. For you: ${accounts.data?.connections.find((account) => account.isDefault && account.provider === provider)?.name ?? "Not connected"}. Other users use their own default.`
: accounts.data?.connections.find(
(account) => account.id === pendingAdoption?.connectionId,
)?.name}
? `Responsible user’s default. For you: ${pendingAccount ? aiConnectionDisplayLabel(pendingAccount) : "Not connected"}. Other users use their own default.`
: pendingAccount && aiConnectionDisplayLabel(pendingAccount)}
</p>}
{pendingAdoption?.mode !== "router" && <p className="text-xs text-muted-foreground">
After adoption, missing credentials block execution. Previous
ui/src/components/ai-connections/AiConnectionSelect.tsx
+6 -6
View File
@@ -14,15 +14,15 @@ import {
} from "@/components/ui/select";
import { AppLogo } from "@/pages/apps/AppLogo";
import { Button } from "@/components/ui/button";
import { aiConnectionProblem, personalAiDefault } from "./model";
import { aiConnectionDisplayLabel, aiConnectionProblem, personalAiDefault } from "./model";
import type { AiConnectionPickerProps } from "./AiConnectionPicker";
import type { AiConnectionSummary } from "./model";
function ConnectionLabel({ connection, label }: { connection: AiConnectionSummary; label?: string }) {
function ConnectionLabel({ connection, label, wrap = false }: { connection: AiConnectionSummary; label?: string; wrap?: boolean }) {
return (
<span className="flex min-w-0 items-center gap-2">
<AppLogo name={connection.name} brandKey={aiConnectionCatalogSlug(connection.provider, connection.routing)} size={20} compact />
<span className="truncate">{label ?? connection.name}</span>
<span className={wrap ? "min-w-0 break-words whitespace-normal" : "truncate"}>{label ?? aiConnectionDisplayLabel(connection)}</span>
</span>
);
}
@@ -119,7 +119,7 @@ export function AiConnectionSelect({
onValueChange={change}
disabled={readOnly || loading || Boolean(error)}
>
<SelectTrigger id={selectId} aria-label="Connection" className="w-full">
<SelectTrigger id={selectId} aria-label="Connection" title={selected ? aiConnectionDisplayLabel(selected) : undefined} className="w-full">
<SelectValue
placeholder={
loading ? "Loading connections…" : "Choose a connection"
@@ -135,7 +135,7 @@ export function AiConnectionSelect({
<SelectContent>
{incompatible && selected && (
<SelectItem value={selected.id} disabled>
<ConnectionLabel connection={selected} label={`${selected.name} · Incompatible`} />
<ConnectionLabel connection={selected} label={`${aiConnectionDisplayLabel(selected)} · Incompatible`} wrap />
</SelectItem>
)}
{compatible.map((c) => (
@@ -144,7 +144,7 @@ export function AiConnectionSelect({
value={c.id}
disabled={Boolean(aiConnectionProblem(c))}
>
<ConnectionLabel connection={c} label={`${c.name}${c.status !== "connected" ? " · Reconnect" : ""}`} />
<ConnectionLabel connection={c} label={`${aiConnectionDisplayLabel(c)}${c.status !== "connected" ? " · Reconnect" : ""}`} wrap />
</SelectItem>
))}
{(canUseDefault || value?.mode === "responsible_user") && (
ui/src/components/ai-connections/model.ts
+10
View File
@@ -57,6 +57,16 @@ export function aiMethodLabel(provider: AiProvider, method: AiAuthMethod) {
: provider === "github" ? "Personal access token" : "API key";
}
export function aiConnectionDisplayLabel(connection: AiConnectionSummary) {
const personal = connection.ownership === "personal";
return [
connection.name,
aiMethodLabel(connection.provider, connection.method),
personal ? "Personal" : "Company shared",
connection.accountLabel || connection.ownerName,
].filter(Boolean).join(" · ");
}
export function matchesAiRequirement(
connection: AiConnectionSummary,
requirement: AiConnectionRequirement,
ui/src/components/task-chat/TaskChatComposer.test.tsx
+1 -1
View File
@@ -856,7 +856,7 @@ describe("TaskChatComposer", () => {
render(<TaskChatComposer onAdd={vi.fn()} workMode="standard" mobile />);
expect(editable().dataset.contentClassName).toContain("min-h-(--sz-48px)");
expect(editable().dataset.contentClassName).toContain("max-h-(--sz-28dvh)");
expect(editable().dataset.contentClassName).toContain("max-h-(--tc-composer-editor-max-height)");
});
it("submits the trimmed body on Cmd+Enter and clears the draft", async () => {
Loaded 100 of 119 files, more files were not shown because too many files have changed in this diff. Show more