mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
fix(runner): keep settled OpenCode sessions across a controller proxy rebuild (#19641)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The native Runner keeps an OpenCode conversation in durable state,
so a task can continue after a control-plane restart.
> - Saved OpenCode state binds one digest of the launch profile: the
Node command, the proxy bundle and the OpenCode executable, each by path
and SHA-256.
> - The Node command and the proxy bundle are controller runtime. A
rebuild or deploy between a hard kill and recovery replaces their bytes.
> - The recovery probe then refused a settled conversation with
`provider_state_incompatible`, although the OpenCode release and the
conversation were unchanged.
> - ACPX already re-binds a settled session after a Runner upgrade
(#18879). OpenCode did not.
> - This pull request records the OpenCode release digest in saved state
and re-binds a settled session when that release is unchanged.
> - The benefit is that a task continues automatically after a restart
that also changed the controller build. A real release change still
fails closed.
## Linked Issues or Issue Description
Refs #19558, #19501, #18879.
**What happened?**
A restart-chaos wave killed the control plane with SIGKILL while an
OpenCode run was settled between turns. The first recovery attempt
restored the session. The controller's runner `dist` was then rebuilt in
the same checkout. The next recovery attempt failed in the
`runner.drain` resume probe with `OpenCode runner launch profile changed
across durable recovery` (`native_provider_state_incompatible`). The
saved state matched the original proxy bytes exactly. Only the proxy
bundle bytes had changed. The OpenCode executable, the paths and the
model were the same.
**Expected behavior**
A settled OpenCode session continues after a restart when only
controller runtime bytes changed. A changed OpenCode release, or a
session with work in flight, fails closed. The existing disposition then
offers one Retry that continues in a fresh provider session.
**Steps to reproduce**
1. Start an OpenCode run and let it settle between turns.
2. Kill the runner and the controller.
3. Replace the qualified proxy bundle with valid new bytes (rebuild or
deploy).
4. Recover the run. The resume probe fails with
`provider_state_incompatible`.
**Paperclip version or commit**
`master` at a08f9494ea.
## What Changed
- `CodexProviderState` records `opencodeReleaseDigest`: the SHA-256 of
the qualified OpenCode executable, which owns the saved conversation.
- Restore re-binds a saved OpenCode session to the current launch
profile only when all of these are true:
- The saved release digest equals the current executable digest.
- The saved configuration names the current command and proxy paths (the
existing binding check).
- The session is settled: no active or ambiguous turn, startup attempt,
receipt-limit interrupt, pending tool call, undelivered or queued
provider event, or working goal. The lifecycle is `prepared`,
`session_open`, or `provider_exited` (an observed idle exit that restore
already restarts).
- Every other mismatch keeps the exact binding and fails closed with
`provider_state_incompatible`. State written before the new field has no
release digest, so it stays exact.
- Session start still verifies every launch artifact and the restored
provider identity.
- `docs/durable-recovery.md` documents the OpenCode re-bind rule next to
the ACPX rule.
- The #19558 regression that required refusal of a new proxy for a
prepared run now asserts the re-bind. The remote pack retention from
#19558 is unchanged.
## Verification
- Red first:
`settled_opencode_session_continues_after_a_controller_proxy_rebuild`
failed on unchanged production code with the production message, from
the same `runner.drain` probe. It passes after the change. It also
checks that the conversation is resumed (`thread/resume`), that a new
turn completes, and that the re-bound digest persists.
- Counter-tests that still refuse: changed OpenCode release on a settled
session (state bytes unchanged), proxy rebuild with an active turn
(state bytes unchanged), and legacy state without a release digest. The
existing executable-drift test still refuses.
`exited_opencode_session_continues_after_a_controller_proxy_rebuild`
covers a kill after an observed idle provider exit (red before the
follow-up commit, green after).
- `cargo test -p paperclip-runner-core --test native_provider_backend`:
45 passed (one unrelated Pi lifetime-fence test flaked once under load
35 and passed 3/3 alone). `cargo test -p paperclip-runner-core --lib
provider_backend`: 145 passed. `cargo fmt` and `cargo clippy` are clean
for the changed code.
- Live restart-chaos waves (local env, OpenCode only, concurrency 2,
`--rc06-masked`). A drift injector appended a valid comment to the local
OpenCode proxy bundle after every fault took the controller down, so
each recovery saw a changed proxy digest.
- Wave 1, seed 3201, 4 tasks, 4 × K1: 3 tasks finished automatically.
The fourth was killed with a turn in flight. Recovery failed closed with
`native_provider_state_incompatible`, as designed. One Retry worked on
the first click and completed the task (R3 pass). That task's grade is
`R5:silent_dead_run`: the Retry run was quiet for 180 s during the B5
benchmark. This is not a launch-profile failure.
- Wave 2, seed 3207, 3 tasks, K1 × 2 and K6: 3/3 pass. The K6 task also
had a turn in flight. It failed closed and one Retry completed it.
- In both waves every kill landed with a turn in flight or before a
durable session. The settled re-bind path was therefore not reached
live. The Rust executor regression above covers that path.
- Environment: macOS arm64, Node 26.4.0, local env, load average 17–39.
## Risks
- A settled OpenCode session now runs on the current proxy bundle after
a controller upgrade. The proxy keeps no state of its own; OpenCode owns
the conversation store. ACPX already accepts the same trade-off for its
sidecar.
- Sessions with work in flight are still refused. This is conservative
and matches ACPX.
- Runs saved by an older Runner have no release digest, so they keep the
old exact behavior once.
- Cursor Cloud keeps its exact binding. It is out of scope.
## Model Used
Anthropic Claude, model ID `claude-opus-5-5`, used through Claude Code
with tool use (shell, file edits) and extended reasoning.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
254b44f72b
commit
20fc37778f
3 files changed
+394
-25
No files matched your search
@@ -114,11 +114,27 @@ Retry then continues the task in a fresh provider session in the same workspace.
|
||||
|
||||
Saved OpenCode and Cursor Cloud provider state binds the digest of its own
|
||||
qualified launch profile (command, proxy script and, for OpenCode, the
|
||||
executable, each by path and SHA-256). These providers do not re-bind. A changed
|
||||
executable, each by path and SHA-256). Cursor Cloud does not re-bind. A changed
|
||||
profile fails restore with the same `provider_state_incompatible`
|
||||
classification and follows the same disposition, so the controller does not
|
||||
retry the restore or try to reattach a runner that already rejected it.
|
||||
|
||||
Saved OpenCode state also records the SHA-256 of the OpenCode executable, the
|
||||
upstream release that owns the saved conversation. The Node command and the
|
||||
proxy bundle are controller runtime: a rebuild or deploy between a hard kill and
|
||||
recovery replaces their bytes, and the proxy keeps no state of its own. Restore
|
||||
re-binds a saved OpenCode session to the current launch profile only when all of
|
||||
these are true:
|
||||
|
||||
- The saved release digest equals the current executable's digest.
|
||||
- The saved configuration names the current command and proxy paths.
|
||||
- The session is settled. It has no active or ambiguous turn, startup attempt,
|
||||
receipt-limit interrupt, pending tool call, undelivered provider event, or
|
||||
working goal. An observed provider exit with nothing in flight is settled.
|
||||
|
||||
A changed OpenCode release, an unsettled session, and state written before the
|
||||
release digest existed keep the exact binding and fail closed as above.
|
||||
|
||||
## Profile identity
|
||||
|
||||
Every ACPX provider's `commandDigest` attests only its own upstream release.
|
||||
|
||||
@@ -1218,6 +1218,12 @@ struct CodexProviderState {
|
||||
config: CodexProviderConfig,
|
||||
#[serde(default)]
|
||||
opencode_launch_profile_digest: Option<String>,
|
||||
/// SHA-256 of the qualified OpenCode executable: the upstream release that
|
||||
/// owns the saved conversation. The launch-profile digest also covers the
|
||||
/// controller's Node command and proxy bundle, which a rebuild or deploy
|
||||
/// replaces; this field lets a settled session tell those apart.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
opencode_release_digest: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
cursor_cloud_launch_profile_digest: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
@@ -1341,6 +1347,7 @@ impl CodexProviderState {
|
||||
lifecycle: "prepared".to_owned(),
|
||||
config,
|
||||
opencode_launch_profile_digest: None,
|
||||
opencode_release_digest: None,
|
||||
cursor_cloud_launch_profile_digest: None,
|
||||
provider_termination_receipt: None,
|
||||
cursor_cloud_terminal_receipt: None,
|
||||
@@ -2007,6 +2014,51 @@ impl CodexCommandExecutor {
|
||||
Ok(Some(format!("sha256:{:x}", digest.finalize())))
|
||||
}
|
||||
|
||||
fn opencode_release_digest(&self, config: &CodexProviderConfig) -> Option<String> {
|
||||
if config.provider != "opencode" {
|
||||
return None;
|
||||
}
|
||||
self.opencode_launch_profile
|
||||
.as_ref()
|
||||
.map(|profile| profile.executable.sha256.clone())
|
||||
}
|
||||
|
||||
// The launch-profile digest binds every artifact by path and bytes. Only
|
||||
// the OpenCode executable owns the saved conversation; the Node command
|
||||
// and proxy bundle are controller runtime that a rebuild or deploy
|
||||
// replaces between a hard kill and recovery. The proxy keeps no state of
|
||||
// its own. Re-bind a saved session to the current profile only when:
|
||||
// - The saved release digest equals this startup's executable digest.
|
||||
// State written before that field existed stays exact.
|
||||
// - The saved config names this startup's command and proxy paths
|
||||
// (bind_opencode_launch_profile has already checked this).
|
||||
// - No provider work is in flight: no active or ambiguous turn, startup
|
||||
// attempt, receipt-limit interrupt, pending tool call, undelivered event
|
||||
// or working goal. The next launch only loads the conversation. An
|
||||
// observed provider exit with nothing in flight qualifies, because
|
||||
// restore restarts that provider the same way.
|
||||
// Session start still verifies every artifact and the restored provider
|
||||
// identity.
|
||||
fn rebinds_settled_opencode_session(&self, state: &CodexProviderState) -> bool {
|
||||
state.opencode_release_digest.is_some()
|
||||
&& state.opencode_release_digest == self.opencode_release_digest(&state.config)
|
||||
&& matches!(
|
||||
state.lifecycle.as_str(),
|
||||
"prepared" | "session_open" | "provider_exited"
|
||||
)
|
||||
&& state.active_provider_turn_id.is_none()
|
||||
&& !state.ambiguous_turn_start_pending
|
||||
&& state.startup_attempt.is_none()
|
||||
&& !state.receipt_limit_interrupt_pending
|
||||
&& state.tool_bridge.pending_calls().next().is_none()
|
||||
&& state.pending_events.is_empty()
|
||||
&& state.queued_events.is_empty()
|
||||
&& state
|
||||
.goal
|
||||
.as_ref()
|
||||
.is_none_or(|goal| goal.status != "active" && !goal.working_now)
|
||||
}
|
||||
|
||||
fn bind_cursor_cloud_launch_profile(
|
||||
&self,
|
||||
config: &CodexProviderConfig,
|
||||
@@ -2286,6 +2338,11 @@ impl CodexCommandExecutor {
|
||||
})?;
|
||||
state.validate()?;
|
||||
let expected_launch_profile_digest = self.bind_opencode_launch_profile(&state.config)?;
|
||||
if state.opencode_launch_profile_digest != expected_launch_profile_digest
|
||||
&& self.rebinds_settled_opencode_session(&state)
|
||||
{
|
||||
state.opencode_launch_profile_digest = expected_launch_profile_digest.clone();
|
||||
}
|
||||
if state.opencode_launch_profile_digest != expected_launch_profile_digest {
|
||||
return Err(DurableRunnerError::invalid(
|
||||
"OpenCode runner launch profile changed across durable recovery",
|
||||
@@ -2888,6 +2945,7 @@ impl CodexCommandExecutor {
|
||||
})?;
|
||||
let mut state = CodexProviderState::new(config, completion_contract, tool_bridge);
|
||||
state.opencode_launch_profile_digest = opencode_launch_profile_digest;
|
||||
state.opencode_release_digest = self.opencode_release_digest(&state.config);
|
||||
state.cursor_cloud_launch_profile_digest = cursor_cloud_launch_profile_digest;
|
||||
self.state = Some(state);
|
||||
self.save_state()?;
|
||||
@@ -7414,6 +7472,7 @@ done
|
||||
conversation_mode: None,
|
||||
},
|
||||
opencode_launch_profile_digest: None,
|
||||
opencode_release_digest: None,
|
||||
cursor_cloud_launch_profile_digest: None,
|
||||
provider_termination_receipt: None,
|
||||
cursor_cloud_terminal_receipt: None,
|
||||
|
||||
+318
-24
@@ -2439,7 +2439,9 @@ fn rejects_opencode_launch_profile_drift_across_fresh_recovery() {
|
||||
|
||||
#[test]
|
||||
#[cfg_attr(windows, ignore = "verified launch is not supported on Windows yet")]
|
||||
fn retains_opencode_state_when_a_valid_new_proxy_replaces_the_admitted_bytes() {
|
||||
fn prepared_opencode_state_follows_a_valid_new_proxy() {
|
||||
// A prepared run has no provider work in flight, so a controller proxy
|
||||
// upgrade re-binds it. The OpenCode release stays the admitted one.
|
||||
let directory = temporary_directory("opencode-proxy-upgrade-recovery");
|
||||
let config = opencode_config(&directory);
|
||||
let mut first = NativeProviderCommandExecutor::with_runner_config(&directory, &config);
|
||||
@@ -2452,44 +2454,336 @@ fn retains_opencode_state_when_a_valid_new_proxy_replaces_the_admitted_bytes() {
|
||||
.unwrap();
|
||||
first.shutdown().unwrap();
|
||||
drop(first);
|
||||
let state_path = directory.join("codex-provider-state.json");
|
||||
let saved_state = fs::read(&state_path).unwrap();
|
||||
let proxy = directory.join("qualified-opencode-proxy-script");
|
||||
let original = fs::read(&proxy).unwrap();
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&proxy, fs::Permissions::from_mode(0o700)).unwrap();
|
||||
let mut upgraded = original.clone();
|
||||
upgraded.extend_from_slice(b"\n# qualified next proxy release\n");
|
||||
fs::write(&proxy, &upgraded).unwrap();
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&proxy, fs::Permissions::from_mode(0o500)).unwrap();
|
||||
let mut next = config.clone();
|
||||
next.opencode_launch_profile.as_mut().unwrap().proxy_script = qualified_artifact(proxy.clone());
|
||||
let next = rebuild_opencode_proxy(&directory, &config);
|
||||
let mut recovered = NativeProviderCommandExecutor::with_runner_config(&directory, &next);
|
||||
let error = recovered
|
||||
recovered
|
||||
.execute(&command(
|
||||
2,
|
||||
"run.prepare",
|
||||
opencode_prepare_payload(&directory),
|
||||
))
|
||||
.unwrap_err();
|
||||
assert_eq!(error.classification(), Some(PROVIDER_STATE_INCOMPATIBLE));
|
||||
assert_eq!(fs::read(&state_path).unwrap(), saved_state);
|
||||
drop(recovered);
|
||||
.unwrap();
|
||||
recovered
|
||||
.execute(&command(3, "session.open", json!({})))
|
||||
.unwrap();
|
||||
recovered.shutdown().unwrap();
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
/// Replace the qualified proxy with valid new bytes, as a controller rebuild or
|
||||
/// deploy does, and return the launch profile that names those bytes.
|
||||
fn rebuild_opencode_proxy(directory: &Path, config: &DurableRunnerConfig) -> DurableRunnerConfig {
|
||||
let proxy = directory.join("qualified-opencode-proxy-script");
|
||||
let mut rebuilt = fs::read(&proxy).unwrap();
|
||||
rebuilt.extend_from_slice(b"\n# rebuilt controller proxy\n");
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&proxy, fs::Permissions::from_mode(0o700)).unwrap();
|
||||
fs::write(&proxy, original).unwrap();
|
||||
fs::write(&proxy, &rebuilt).unwrap();
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&proxy, fs::Permissions::from_mode(0o500)).unwrap();
|
||||
let mut pinned = NativeProviderCommandExecutor::with_runner_config(&directory, &config);
|
||||
pinned
|
||||
let mut next = config.clone();
|
||||
next.opencode_launch_profile.as_mut().unwrap().proxy_script = qualified_artifact(proxy);
|
||||
next
|
||||
}
|
||||
|
||||
fn acknowledge_delivered_events(executor: &mut NativeProviderCommandExecutor) {
|
||||
let events = executor.poll_events().unwrap();
|
||||
executor.acknowledge_events(events.len()).unwrap();
|
||||
}
|
||||
|
||||
fn poll_opencode_until(
|
||||
executor: &mut NativeProviderCommandExecutor,
|
||||
event_type: &str,
|
||||
) -> Vec<PolledEvent> {
|
||||
let deadline = std::time::Instant::now() + Duration::from_secs(10);
|
||||
let mut observed = Vec::new();
|
||||
while std::time::Instant::now() < deadline {
|
||||
let events = executor.poll_events().unwrap();
|
||||
let count = events.len();
|
||||
observed.extend(events);
|
||||
executor.acknowledge_events(count).unwrap();
|
||||
if observed.iter().any(|event| event.event_type == event_type) {
|
||||
return observed;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(1));
|
||||
}
|
||||
panic!("OpenCode did not emit {event_type}: {observed:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg_attr(windows, ignore = "verified launch is not supported on Windows yet")]
|
||||
fn settled_opencode_session_continues_after_a_controller_proxy_rebuild() {
|
||||
// A hard kill leaves a settled OpenCode conversation. The controller then
|
||||
// restarts on a rebuilt proxy bundle; the OpenCode release is unchanged.
|
||||
let directory = temporary_directory("opencode-settled-proxy-rebuild");
|
||||
let config = opencode_config(&directory);
|
||||
let mut first = NativeProviderCommandExecutor::with_runner_config(&directory, &config);
|
||||
first
|
||||
.execute(&command(
|
||||
3,
|
||||
1,
|
||||
"run.prepare",
|
||||
opencode_prepare_payload(&directory),
|
||||
))
|
||||
.unwrap();
|
||||
pinned.shutdown().unwrap();
|
||||
first
|
||||
.execute(&command(2, "session.open", json!({})))
|
||||
.unwrap();
|
||||
acknowledge_delivered_events(&mut first);
|
||||
drop(first);
|
||||
let state_path = directory.join("codex-provider-state.json");
|
||||
let saved_binding = |field: &str| -> Value {
|
||||
serde_json::from_slice::<Value>(&fs::read(&state_path).unwrap()).unwrap()[field].clone()
|
||||
};
|
||||
let admitted_digest = saved_binding("opencodeLaunchProfileDigest");
|
||||
let admitted_release = saved_binding("opencodeReleaseDigest");
|
||||
assert!(admitted_digest.is_string() && admitted_release.is_string());
|
||||
|
||||
let rebuilt = rebuild_opencode_proxy(&directory, &config);
|
||||
let resumes_before = opencode_call_count(&directory, "thread/resume");
|
||||
let mut recovered = NativeProviderCommandExecutor::with_runner_config(&directory, &rebuilt);
|
||||
recovered
|
||||
.execute(&command(3, "runner.drain", json!({})))
|
||||
.expect("a settled session re-binds to the rebuilt proxy");
|
||||
assert_eq!(
|
||||
opencode_call_count(&directory, "thread/resume"),
|
||||
resumes_before + 1,
|
||||
"recovery restores the same OpenCode conversation",
|
||||
);
|
||||
recovered
|
||||
.execute(&command(
|
||||
4,
|
||||
"turn.start",
|
||||
json!({"text": "Continue the task."}),
|
||||
))
|
||||
.unwrap();
|
||||
let observed = poll_opencode_until(&mut recovered, "turn.completed");
|
||||
assert!(observed
|
||||
.iter()
|
||||
.any(|event| event.event_type == "turn.completed"));
|
||||
recovered.shutdown().unwrap();
|
||||
drop(recovered);
|
||||
|
||||
// The re-bound digest is durable and the release binding is unchanged.
|
||||
let rebound_digest = saved_binding("opencodeLaunchProfileDigest");
|
||||
assert!(rebound_digest.is_string());
|
||||
assert_ne!(rebound_digest, admitted_digest);
|
||||
assert_eq!(saved_binding("opencodeReleaseDigest"), admitted_release);
|
||||
// The next recovery on the rebuilt profile matches the saved digest.
|
||||
let mut again = NativeProviderCommandExecutor::with_runner_config(&directory, &rebuilt);
|
||||
again
|
||||
.execute(&command(5, "session.snapshot", json!({})))
|
||||
.unwrap();
|
||||
again.shutdown().unwrap();
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg_attr(windows, ignore = "verified launch is not supported on Windows yet")]
|
||||
fn opencode_release_change_still_refuses_a_settled_session() {
|
||||
let directory = temporary_directory("opencode-settled-release-change");
|
||||
let config = opencode_config(&directory);
|
||||
let mut first = NativeProviderCommandExecutor::with_runner_config(&directory, &config);
|
||||
first
|
||||
.execute(&command(
|
||||
1,
|
||||
"run.prepare",
|
||||
opencode_prepare_payload(&directory),
|
||||
))
|
||||
.unwrap();
|
||||
first
|
||||
.execute(&command(2, "session.open", json!({})))
|
||||
.unwrap();
|
||||
acknowledge_delivered_events(&mut first);
|
||||
drop(first);
|
||||
let state_path = directory.join("codex-provider-state.json");
|
||||
let saved_state = fs::read(&state_path).unwrap();
|
||||
|
||||
let executable = directory.join("qualified-opencode-executable");
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&executable, fs::Permissions::from_mode(0o700)).unwrap();
|
||||
fs::write(&executable, "next OpenCode release\n").unwrap();
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&executable, fs::Permissions::from_mode(0o500)).unwrap();
|
||||
let mut released = rebuild_opencode_proxy(&directory, &config);
|
||||
released
|
||||
.opencode_launch_profile
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.executable = qualified_artifact(executable);
|
||||
let mut recovered = NativeProviderCommandExecutor::with_runner_config(&directory, &released);
|
||||
let error = recovered
|
||||
.execute(&command(3, "runner.drain", json!({})))
|
||||
.unwrap_err();
|
||||
assert!(error
|
||||
.to_string()
|
||||
.contains("launch profile changed across durable recovery"));
|
||||
assert_eq!(error.classification(), Some(PROVIDER_STATE_INCOMPATIBLE));
|
||||
assert_eq!(fs::read(&state_path).unwrap(), saved_state);
|
||||
drop(recovered);
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg_attr(windows, ignore = "verified launch is not supported on Windows yet")]
|
||||
fn exited_opencode_session_continues_after_a_controller_proxy_rebuild() {
|
||||
// The turn completes, then the idle provider exits nonzero. The observed
|
||||
// exit leaves no provider work in flight, so a rebuilt proxy re-binds it.
|
||||
let directory = temporary_directory("opencode-exited-proxy-rebuild");
|
||||
let mut config = opencode_config(&directory);
|
||||
let proxy_script = directory.join("qualified-opencode-proxy-script");
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&proxy_script, fs::Permissions::from_mode(0o700)).unwrap();
|
||||
fs::write(
|
||||
&proxy_script,
|
||||
format!(
|
||||
"#!/bin/sh\nexec '{}' --state-file '{}' --call-log '{}' --require-completion-contract --fail-after-turn-completion --fail-after-turn-completion-delay-ms 250\n",
|
||||
env!("CARGO_BIN_EXE_fake-codex-app-server"),
|
||||
directory.join("fake-opencode-state.json").display(),
|
||||
directory.join("fake-opencode-calls.log").display(),
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&proxy_script, fs::Permissions::from_mode(0o500)).unwrap();
|
||||
config
|
||||
.opencode_launch_profile
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.proxy_script = qualified_artifact(proxy_script);
|
||||
let mut first = NativeProviderCommandExecutor::with_runner_config(&directory, &config);
|
||||
first
|
||||
.execute(&command(
|
||||
1,
|
||||
"run.prepare",
|
||||
opencode_prepare_payload(&directory),
|
||||
))
|
||||
.unwrap();
|
||||
first
|
||||
.execute(&command(2, "session.open", json!({})))
|
||||
.unwrap();
|
||||
first
|
||||
.execute(&command(
|
||||
3,
|
||||
"turn.start",
|
||||
json!({"text": "Finish the task."}),
|
||||
))
|
||||
.unwrap();
|
||||
poll_opencode_until(&mut first, "session.failed");
|
||||
acknowledge_delivered_events(&mut first);
|
||||
drop(first);
|
||||
// The runner restarts an exited idle provider on its next poll. A hard
|
||||
// kill can land first and leave the observed exit as the saved lifecycle.
|
||||
let state_path = directory.join("codex-provider-state.json");
|
||||
let mut saved: Value = serde_json::from_slice(&fs::read(&state_path).unwrap()).unwrap();
|
||||
assert!(saved["activeProviderTurnId"].is_null());
|
||||
assert_eq!(saved["pendingEvents"], json!([]));
|
||||
saved["lifecycle"] = json!("provider_exited");
|
||||
fs::write(&state_path, serde_json::to_vec(&saved).unwrap()).unwrap();
|
||||
|
||||
let rebuilt = rebuild_opencode_proxy(&directory, &config);
|
||||
let resumes_before = opencode_call_count(&directory, "thread/resume");
|
||||
let mut recovered = NativeProviderCommandExecutor::with_runner_config(&directory, &rebuilt);
|
||||
recovered
|
||||
.execute(&command(4, "runner.drain", json!({})))
|
||||
.expect("an exited, settled session re-binds to the rebuilt proxy");
|
||||
assert_eq!(
|
||||
opencode_call_count(&directory, "thread/resume"),
|
||||
resumes_before + 1,
|
||||
"recovery restarts the same OpenCode conversation",
|
||||
);
|
||||
drop(recovered);
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg_attr(windows, ignore = "verified launch is not supported on Windows yet")]
|
||||
fn opencode_proxy_rebuild_refuses_an_active_turn() {
|
||||
let directory = temporary_directory("opencode-active-proxy-rebuild");
|
||||
let mut config = opencode_config(&directory);
|
||||
let proxy_script = directory.join("qualified-opencode-proxy-script");
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&proxy_script, fs::Permissions::from_mode(0o700)).unwrap();
|
||||
fs::write(
|
||||
&proxy_script,
|
||||
format!(
|
||||
"#!/bin/sh\nexec '{}' --state-file '{}' --call-log '{}' --require-completion-contract --durable-turn-ids --linger-after-turn-start\n",
|
||||
env!("CARGO_BIN_EXE_fake-codex-app-server"),
|
||||
directory.join("fake-opencode-state.json").display(),
|
||||
directory.join("fake-opencode-calls.log").display(),
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
#[cfg(unix)]
|
||||
fs::set_permissions(&proxy_script, fs::Permissions::from_mode(0o500)).unwrap();
|
||||
config
|
||||
.opencode_launch_profile
|
||||
.as_mut()
|
||||
.unwrap()
|
||||
.proxy_script = qualified_artifact(proxy_script);
|
||||
let mut first = NativeProviderCommandExecutor::with_runner_config(&directory, &config);
|
||||
first
|
||||
.execute(&command(
|
||||
1,
|
||||
"run.prepare",
|
||||
opencode_prepare_payload(&directory),
|
||||
))
|
||||
.unwrap();
|
||||
first
|
||||
.execute(&command(2, "session.open", json!({})))
|
||||
.unwrap();
|
||||
first
|
||||
.execute(&command(3, "turn.start", json!({"text": "Edit the file."})))
|
||||
.unwrap();
|
||||
drop(first);
|
||||
let state_path = directory.join("codex-provider-state.json");
|
||||
let saved_state = fs::read(&state_path).unwrap();
|
||||
|
||||
let rebuilt = rebuild_opencode_proxy(&directory, &config);
|
||||
let mut recovered = NativeProviderCommandExecutor::with_runner_config(&directory, &rebuilt);
|
||||
let error = recovered
|
||||
.execute(&command(4, "runner.drain", json!({})))
|
||||
.unwrap_err();
|
||||
assert_eq!(error.classification(), Some(PROVIDER_STATE_INCOMPATIBLE));
|
||||
assert_eq!(fs::read(&state_path).unwrap(), saved_state);
|
||||
drop(recovered);
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg_attr(windows, ignore = "verified launch is not supported on Windows yet")]
|
||||
fn opencode_state_without_a_release_binding_stays_exact() {
|
||||
// State written before the release binding cannot prove which artifact
|
||||
// changed, so a changed profile keeps failing closed.
|
||||
let directory = temporary_directory("opencode-legacy-proxy-rebuild");
|
||||
let config = opencode_config(&directory);
|
||||
let mut first = NativeProviderCommandExecutor::with_runner_config(&directory, &config);
|
||||
first
|
||||
.execute(&command(
|
||||
1,
|
||||
"run.prepare",
|
||||
opencode_prepare_payload(&directory),
|
||||
))
|
||||
.unwrap();
|
||||
first
|
||||
.execute(&command(2, "session.open", json!({})))
|
||||
.unwrap();
|
||||
acknowledge_delivered_events(&mut first);
|
||||
drop(first);
|
||||
let state_path = directory.join("codex-provider-state.json");
|
||||
let mut legacy: Value = serde_json::from_slice(&fs::read(&state_path).unwrap()).unwrap();
|
||||
legacy
|
||||
.as_object_mut()
|
||||
.unwrap()
|
||||
.remove("opencodeReleaseDigest");
|
||||
fs::write(&state_path, serde_json::to_vec(&legacy).unwrap()).unwrap();
|
||||
|
||||
let rebuilt = rebuild_opencode_proxy(&directory, &config);
|
||||
let mut recovered = NativeProviderCommandExecutor::with_runner_config(&directory, &rebuilt);
|
||||
let error = recovered
|
||||
.execute(&command(3, "runner.drain", json!({})))
|
||||
.unwrap_err();
|
||||
assert_eq!(error.classification(), Some(PROVIDER_STATE_INCOMPATIBLE));
|
||||
drop(recovered);
|
||||
fs::remove_dir_all(directory).unwrap();
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user