mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-28 04:02:18 +01:00
fix(runner): tolerate Codex account notifications during active turns (#13902)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Rust runner connects task runs to Codex app-server sessions. > - Codex sends account notifications on the connection during authentication updates. > - These notifications have no task, thread, or turn identity. > - The runner treated a valid account update as an invalid task event and stopped the run. > - This pull request classifies account updates as connection information while preserving task identity checks. > - Sandbox images can also contain an older runner with compatible metadata. The server must compare its bytes before reuse. > - Slack conversations use the deployed runner and can complete when Codex refreshes account state. ## Linked Issues or Issue Description Related: #13853. That change fixes the supported Codex version range. This fixes a separate notification failure after the version check succeeds. **What happened?** An active Codex task stopped with `thread_binding_mismatch` when the provider emitted `account/updated` without a thread ID. Slack showed that the agent stopped before completing its turn. **Expected behavior** Connection-level account updates must not stop a task or acquire task authority. Account details must not appear in task output. **Steps to reproduce** 1. Start a task with the native Codex runner. 2. Emit `account/updated` during the turn, with `authMode` and `planType` but no thread ID. 3. Observe that the old runner rejects the notification and fails the turn. **Paperclip version or commit** Reproduced after #13853. The fix is based on `b648d8cdd`. **Deployment mode** Cloud staging with a remote Codex runner and a Slack chat connection. ## What Changed - Classify `account/updated` and `account/login/completed` as connection information. - Reject account notifications that contain execution identity fields. - Reuse the existing bounded diagnostic path without publishing account payloads. - Test both notification types during two consecutive turns. Preserve existing identity rejection tests. - Compare preinstalled sandbox runner bytes with the controller artifact. Stage the deployed binary after a mismatch, failed checksum, or timeout. Keep exact retained artifact reuse. ## Verification - `cargo test --release --locked -p paperclip-runner-core`: passed, 582 test executions; two existing ignored cases. - `cargo test --release --locked -p paperclip-runner-core --test codex_provider`: 89 passed; two existing ignored cases. - `cargo fmt --all` and `git diff --check`: passed. - Repository-wide `pnpm -r typecheck`: passed. Server typecheck also passed after the artifact-selection change. - Server executor suite: 456 tests passed, including preinstalled digest match, mismatch, checksum failure, and timeout. - Repository-wide Vitest and build are running. - Deployed `5648d90d5dd762c5a8b697face269f6eea9cd59d` to one staging stack and verified its serving SHA. - Retried the failed conversation through the Slack UI. The agent replied and the task completed. - Sent a fresh Slack mention asking which bots belong to the channel. Verified successful `slack_members` and `slack_user` calls, a successful run, and the delivered Slack answer. - Sent a follow-up in the same Slack thread without another mention. The agent returned the requested names from context, with one final reply. ## Risks - Only two known connection-level notification methods change behavior. Unknown authoritative methods and malformed or conflicting identities still fail closed. - An older sandbox runner now requires one upload when its bytes differ from the controller artifact. Remote OS and architecture checks still apply. - No schema, credential, permission, Codex version-range, or UI changes. The Codex minimum remains 0.149.0. ## Model Used OpenAI GPT-6 (`gpt-6-astra`) through Codex, with reasoning, code editing, terminal tools, and browser verification. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
30182c704c
commit
a6448cd060
5 files changed
+138
-37
No files matched your search
@@ -10396,7 +10396,7 @@ describe("runnerd provider runtime wiring", () => {
|
||||
});
|
||||
|
||||
it.each([
|
||||
...["current", "stale", "missing", "retained", "retained-mismatch", "retained-error", "retained-timeout", "retained-explicit"]
|
||||
...["current", "preinstalled-exact", "preinstalled-mismatch", "preinstalled-error", "preinstalled-timeout", "stale", "missing", "retained", "retained-mismatch", "retained-error", "retained-timeout", "retained-explicit"]
|
||||
.map((image) => ({ image, version: "0.156.0", compatible: true })),
|
||||
...["0.149.0", "0.149.1", "0.153.4", "0.156.1"]
|
||||
.map((version) => ({ image: "current", version, compatible: true })),
|
||||
@@ -10405,11 +10405,11 @@ describe("runnerd provider runtime wiring", () => {
|
||||
])("uses shared Codex and the server-owned replacement artifact (image=$image, Codex=$version)", async ({ image, version, compatible }) => {
|
||||
const retained = image.startsWith("retained");
|
||||
const exactRetained = image === "retained" || image === "retained-explicit";
|
||||
const needsReplacement = image !== "current" && !exactRetained;
|
||||
const needsReplacement = image !== "current" && image !== "preinstalled-exact" && !exactRetained;
|
||||
// The mocked remote executes metadata probes; artifact staging only needs bytes.
|
||||
// Keep this regression independent of a locally compiled Rust runner binary.
|
||||
const controllerArtifact = join(isolatedStateDirectory, "paperclip-runnerd");
|
||||
if (needsReplacement || retained) {
|
||||
if (needsReplacement || retained || image === "preinstalled-exact") {
|
||||
await writeFile(controllerArtifact, "fixture runner artifact");
|
||||
state.resolveRunnerBinary.mockReturnValueOnce(controllerArtifact);
|
||||
}
|
||||
@@ -10420,10 +10420,10 @@ describe("runnerd provider runtime wiring", () => {
|
||||
let stdout = "";
|
||||
const script = command.args?.[1] ?? "";
|
||||
if (script.includes('sha256sum "$1"')) {
|
||||
if (image === "retained-error") throw new Error("checksum unavailable");
|
||||
if (image.endsWith("-error")) throw new Error("checksum unavailable");
|
||||
return {
|
||||
exitCode: 0, signal: null, timedOut: image === "retained-timeout", stderr: "",
|
||||
stdout: `${createHash("sha256").update(image === "retained-mismatch" ? "stale artifact" : "fixture runner artifact").digest("hex")} ${command.args?.[3]}\n`,
|
||||
exitCode: 0, signal: null, timedOut: image.endsWith("-timeout"), stderr: "",
|
||||
stdout: `${createHash("sha256").update(image.endsWith("-mismatch") ? "stale artifact" : "fixture runner artifact").digest("hex")} ${command.args?.[3]}\n`,
|
||||
};
|
||||
} else if (command.args?.[0] === "--build-metadata") {
|
||||
stdout = JSON.stringify({
|
||||
|
||||
@@ -10852,9 +10852,33 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
remotePrepared = true;
|
||||
return;
|
||||
}
|
||||
// A resumed sandbox may already contain the exact controller-owned binary.
|
||||
// Do not upload it every turn, but never treat compatible metadata alone as
|
||||
// proof of artifact identity (especially for an explicit operator override).
|
||||
// Compatibility metadata does not prove artifact identity. Reuse only the
|
||||
// exact controller-owned bytes when the controller artifact is available.
|
||||
const matchesControllerRunnerArtifact = async (executable: string): Promise<boolean> => {
|
||||
const expected = createHash("sha256")
|
||||
.update(readFileSync(controllerRunnerBinary))
|
||||
.digest("hex");
|
||||
try {
|
||||
const probe = await remoteCommandRunner.execute({
|
||||
command: "sh",
|
||||
args: [
|
||||
"-c",
|
||||
'test -x "$1" || exit 1; if command -v sha256sum >/dev/null 2>&1; then sha256sum "$1"; else shasum -a 256 "$1"; fi',
|
||||
"paperclip-runner-artifact",
|
||||
executable,
|
||||
],
|
||||
cwd: remoteTarget.remoteCwd,
|
||||
bypassSession: true,
|
||||
timeoutMs: 10_000,
|
||||
});
|
||||
return probe.exitCode === 0 && !probe.timedOut &&
|
||||
/^[a-f0-9]{64}\s/.test(probe.stdout) &&
|
||||
probe.stdout.trim().split(/\s+/)[0] === expected;
|
||||
} catch {
|
||||
// An unavailable checksum uses the verified staging path.
|
||||
return false;
|
||||
}
|
||||
};
|
||||
let runnerArtifactPrepared = false;
|
||||
if (
|
||||
sandboxLeaseAcquisition?.outcome === "resumed" &&
|
||||
@@ -10863,32 +10887,7 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
runnerArtifactPrepared = await measureNativeRunnerSpan(
|
||||
input.trace,
|
||||
"runner.artifact.verify_retained",
|
||||
async () => {
|
||||
const expected = createHash("sha256")
|
||||
.update(readFileSync(controllerRunnerBinary))
|
||||
.digest("hex");
|
||||
try {
|
||||
const probe = await remoteCommandRunner.execute({
|
||||
command: "sh",
|
||||
args: [
|
||||
"-c",
|
||||
'test -x "$1" || exit 1; if command -v sha256sum >/dev/null 2>&1; then sha256sum "$1"; else shasum -a 256 "$1"; fi',
|
||||
"paperclip-runner-artifact",
|
||||
remoteBinary,
|
||||
],
|
||||
cwd: remoteTarget.remoteCwd,
|
||||
bypassSession: true,
|
||||
timeoutMs: 10_000,
|
||||
});
|
||||
return probe.exitCode === 0 && !probe.timedOut &&
|
||||
/^[a-f0-9]{64}\s/.test(probe.stdout) &&
|
||||
probe.stdout.trim().split(/\s+/)[0] === expected;
|
||||
} catch {
|
||||
// Missing binaries, checksum tools, or a failed probe use the
|
||||
// ordinary verified staging path; none authorizes cached execution.
|
||||
return false;
|
||||
}
|
||||
},
|
||||
() => matchesControllerRunnerArtifact(remoteBinary),
|
||||
);
|
||||
}
|
||||
const explicitRemoteBinary = input.runnerRemoteBinaryPath?.trim() || null;
|
||||
@@ -10905,6 +10904,10 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
"runner.artifact.verify_preinstalled",
|
||||
() => verifyRemoteRunner(requiredMode, preinstalledRunner),
|
||||
);
|
||||
if (existsSync(controllerRunnerBinary) &&
|
||||
!await matchesControllerRunnerArtifact(preinstalledRunner)) {
|
||||
throw new Error("runner_remote_preinstalled_artifact_mismatch");
|
||||
}
|
||||
await measureNativeRunnerSpan(
|
||||
input.trace,
|
||||
"runner.artifact.link",
|
||||
|
||||
Reference in new issue
Block a user