From f6a15200c20ded0b68ec9ee344fdb3ecd0c004f9 Mon Sep 17 00:00:00 2001 From: Dotta Date: Fri, 9 Oct 2026 11:43:00 -0500 Subject: [PATCH] fix: preserve native Codex sign-in and settle retry fixture Co-Authored-By: Paperclip --- doc/plans/2026-10-06-default-agent-runner.md | 61 ++++++++++++++++--- .../native-runtime/setup-readiness.test.ts | 19 +++++- .../native-runtime/setup-readiness.ts | 6 +- skills/paperclip-create-agent/SKILL.md | 3 + .../references/api-reference.md | 12 ---- tests/e2e/chat-adapters-ui-messaging.spec.ts | 24 +++++++- .../AgentConfigForm.render.test.tsx | 23 +++++++ 7 files changed, 124 insertions(+), 24 deletions(-) diff --git a/doc/plans/2026-10-06-default-agent-runner.md b/doc/plans/2026-10-06-default-agent-runner.md index 57b61c8827..bcc98ce087 100644 --- a/doc/plans/2026-10-06-default-agent-runner.md +++ b/doc/plans/2026-10-06-default-agent-runner.md @@ -38,10 +38,12 @@ Docker materialization, Git installer, or extra login isolation wholesale. Authorized 2026-10-09. Canonical branch: `codex/codex-runner-default`. Integration base: master `2dd9811e8be1dab59cccd86cdb25e8953b02fe07`. Canonical PR: [draft #15683](https://github.com/paperclipai/paperclip/pull/15683). -Latest pushed candidate before integration: `4275309f9d619055f48a9b21cb0ad0a1ff7c2c63`. -The follow-up fixes configure-on-first-use for an existing built-in that still -needs setup. The PR remains under 100 files, with no workflow or lockfile -changes. Fresh final-head CI and review are required after that fix. +Last pushed candidate: `ad5d8826ad18f72d672fcafbdd9fec24ef8c9fef`. +It integrates the current agent lifecycle boundary. The next narrow follow-up +restores the native Codex sign-in action and settles an existing chat test's +canonical lookup before interaction. The PR remains under 100 files, with no +workflow, lockfile, or database migration changes. Fresh final-head CI and +review are required after that follow-up. Preview: [isolated Codex QA](http://127.0.0.1:3125), normal dev supervisor, separate database. Hosted Linux live onboarding passed at `38199ea8`; it remains historical proof after the lifecycle integration. UI has separate revision-bound proof. @@ -82,10 +84,10 @@ Claude's explicit native fixture is unchanged until its later default PR. | Production create/edit/onboarding UI and stories | UI worker | 171 fresh focused cases, 30 desktop/10 mobile captures and six keyboard checks; built-in follow-up passes 15 cases, UI typecheck and token gates | | Built-ins, plugins, approvals, imports/exports, CLI | Creation-path worker | Integrated lifecycle: 105 portability, eight approval and three asset cases pass; 73 DB cases skipped on macOS; final Linux checks required | | Execution/dev gate, E2E gaps, packaging, live journeys | Lead | Selected Linux onboarding case passes all 13 assertions, actual task/artifact/follow-up and cleanup; 149 public-projection/support cases pass | -| Full CI and final review | Lead | All seven known findings fixed/resolved; master lifecycle merge conflicts resolved locally; new combined head needs full CI and fresh review | -| Local subscription task + follow-up | Lead | Embedded-browser onboarding prepared; attended sign-in unfinished; macOS automatic legacy selection is expected | +| Full CI and final review | Lead | Integrated head: 49 checks pass, three fail, four skip; native auth action finding and existing chat readiness fixture fixed locally; fresh combined-head checks/review required | +| Local subscription task + follow-up | Lead | Embedded-browser Codex sign-in and first-agent creation completed; saved codex_local is expected on macOS; useful task/follow-up still pending | | Managed staging API/subscription onboarding | Lead | Fleet Admin sign-in completed; dedicated tenant exists, active/awake and pinned to old release; scoped one-hour QA owner access approved; final serving revision/snapshot/runtime/cleanup qualification pending | -| Shipped npm/Docker/cloud artifacts | Lead | Clean Linux npm consumer passes at the prior candidate: 18 packages, no Codex payloads, host dependency from official npm; final-head replay pending; actual cloud composition is post-merge | +| Shipped npm/Docker/cloud artifacts | Lead | Integrated-head clean Linux npm consumer passes: 18 packages, no Codex payloads, host dependency from official npm; immutable staging preparation reports missing matching DB package; actual cloud composition is post-merge | Keep CPU-heavy Rust/Docker builds in hosted CI. Use Node 24, bounded workers, conservative provider turns/timeouts and existing QA accounts. Login interactions @@ -289,3 +291,48 @@ Next action: freeze/push the integrated candidate, require exact-head CI and fresh 5/5 review, restart the matched local preview and qualify the dedicated managed target against that candidate. The previous Linux run is retained evidence; it must not be relabeled as the new live measurement. + +### 2026-10-09 Integrated-head qualification and final corrections + +At `ad5d8826`, all checks completed: 49 pass, three fail and four skip. Required +verification, workspace typecheck, build, Linux server/DB suites, native +compilation and the real clean 18-package consumer pass. The consumer contains +no Codex payloads and selects its owned official-npm Codex dependency despite a +different outer version. CI merge and generated-lock identities remain separate +from the PR source. Applicable code-owner approval is still unproven. + +The only browser failure is the existing selected-run denial case, whose fixture +uses the process adapter. Its URL becomes canonical before its agent query +settles. The test now waits for the successful company-scoped canonical response +and verifies its complete agent identity before clicking. All nine existing +retry cases pass against the real isolated preview with their exact request and +denial assertions unchanged. A supporting query lifecycle control reproduces +lost mutation feedback during a remount; the CI cause remains an inference, +because the controlled early-click browser reproduction did not activate. + +Fresh review scored 4/5 and found one native Codex onboarding defect: an +authentication error lacked the shared check that displays the sign-in action. +The missing check reproduced before the fix. The correction retains the native +diagnostic, failed result and explicit runner choice. All 174 readiness, registry +and production-form render cases pass, including selected-environment login and +negative model/dependency controls; server/UI typechecks pass. The agent creation +skill retains the full API contract and CLI equivalents in one place, avoiding a +redundant reference edit and keeping the PR at 99 files. + +The attended embedded-browser Codex sign-in succeeded, and first-agent onboarding +created the isolated Mac agent as `codex_local`, as expected on this platform. +The ordinary first-task interaction is open; task, artifact and follow-up are +not yet proven. No task provider call was made during these corrections. + +The dedicated staging tenant's old release remains unchanged. Read-only +immutable preparation [37959487896](https://github.com/paperclipai/paperclip-cloud/actions/runs/37959487896) +matched the intended QA target and reported a matching DB package/migrations +artifact gap. Rollout and serving verification were skipped. Build the frozen +candidate's isolated preview image and packages through the existing trusted +remote workflow; never substitute an older image or claim private Cloud +composition from the Core-only preview. Keep the pin and provisioning default. + +Next action: push these narrow corrections, run fresh full CI and review, build +matching immutable preview artifacts in hosted CI, then qualify the one QA tenant +after those gates pass. Retain all historical cost and cleanup holds. The draft +remains neither merge-ready nor production-ready. diff --git a/server/src/services/native-runtime/setup-readiness.test.ts b/server/src/services/native-runtime/setup-readiness.test.ts index 8ef4e4076e..beb09c64d3 100644 --- a/server/src/services/native-runtime/setup-readiness.test.ts +++ b/server/src/services/native-runtime/setup-readiness.test.ts @@ -2,6 +2,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, posix } from "node:path"; +import { ADAPTER_AUTH_MISSING_CHECK_CODE } from "@paperclipai/shared"; const { execute, nativeProbe, selectedRunner, sshExecute, sshRunner, nativeArtifacts, remoteLauncher, registerPrp, ingressConnect } = vi.hoisted(() => ({ execute: vi.fn(), nativeProbe: vi.fn(), selectedRunner: vi.fn(), sshExecute: vi.fn(), sshRunner: vi.fn(), nativeArtifacts: vi.fn(), remoteLauncher: vi.fn(), registerPrp: vi.fn(), ingressConnect: vi.fn() })); vi.mock("@paperclipai/adapter-utils/execution-target", () => ({ runAdapterExecutionTargetShellCommand: execute })); vi.mock("./native-ssh-command-runner.js", () => ({ createNativeSshCommandRunner: sshRunner })); @@ -179,11 +180,27 @@ describe("Codex selected native account verification", () => { ])("fails closed and redacts native failures (%s)", async (message, code) => { nativeProbe.mockRejectedValue(new Error(message)); const result = await testNativeRunnerAuthentication({ companyId: "company", adapterType: "paperclip_runner", config: { env: { OPENAI_API_KEY: "bound-account" } } }, "codex", "gpt-6.1-sol"); - expect(result).toMatchObject({ status: "fail", checks: [{ code: `codex_hello_probe_${code}`, level: "error" }] }); + expect(result.status).toBe("fail"); + expect(result.checks[0]).toMatchObject({ code: `codex_hello_probe_${code}`, level: "error" }); + expect(result.checks.some(check => check.code === ADAPTER_AUTH_MISSING_CHECK_CODE)).toBe(code === "auth_required"); expect(JSON.stringify(result)).not.toContain("bound-account"); expect(nativeProbe).toHaveBeenCalledOnce(); expect(execute).not.toHaveBeenCalled(); }); + it.each([ + ["authentication required: selected-account", true], + ["Model is unavailable", false], + ["native executable is missing", false], + ])("projects the canonical sandbox login check only for authentication failures (%s)", async (message, authentication) => { + nativeProbe.mockRejectedValue(new Error(message)); + const result = await testNativeRunnerAuthentication(context, "codex", "gpt-6.1-sol"); + expect(result.status).toBe("fail"); + expect(result.checks[0].code).toBe(`codex_hello_probe_${authentication ? "auth_required" : "failed"}`); + expect(result.checks.some(check => check.code === ADAPTER_AUTH_MISSING_CHECK_CODE)).toBe(authentication); + if (authentication) expect(result.checks).toContainEqual(expect.objectContaining({ code: ADAPTER_AUTH_MISSING_CHECK_CODE, level: "error" })); + expect(nativeProbe).toHaveBeenCalledOnce(); + expect(execute).not.toHaveBeenCalled(); + }); it.each([ { helloProbePassed: undefined }, { effectiveModel: "another-model" }, { effectiveModel: null }, { effectiveModel: "" }, { provider: "opencode" }, { providerDriver: "codex_cli" }, ])("rejects installation-only, legacy, or mismatched receipts", async override => { diff --git a/server/src/services/native-runtime/setup-readiness.ts b/server/src/services/native-runtime/setup-readiness.ts index 78fb15ddf3..c03f325099 100644 --- a/server/src/services/native-runtime/setup-readiness.ts +++ b/server/src/services/native-runtime/setup-readiness.ts @@ -3,6 +3,7 @@ import { tmpdir } from "node:os"; import { join, posix } from "node:path"; import { copyBackCodexAuth } from "@paperclipai/adapter-codex-local/server"; import type { AdapterEnvironmentTestContext, AdapterEnvironmentTestResult } from "@paperclipai/adapter-utils"; +import { ADAPTER_AUTH_MISSING_CHECK_CODE } from "@paperclipai/shared"; import { probeNativeRunnerEnvironment, resolveSourceCodexHome } from "../../vendor/paperclip-runner/index.js"; import { resolvePaperclipRunnerBinary } from "./native-codex-runner.js"; import { readLocalAiCredentialFile } from "../local-ai-credential-file.js"; @@ -207,6 +208,9 @@ export async function testNativeRunnerAuthentication(context: AdapterEnvironment const message = redactNativeProbeMessage(error instanceof Error ? error.message : "The selected native runtime could not verify this account.", environment); const authentication = /(?:invalid (?:api[- ]?key|auth(?:entication)? token)|authentication (?:failed|required)|unauthenticated|unauthorized|not authenticated|please (?:log|sign) in|not logged in|\b(?:401|403)\b)/i.test(message); return { adapterType: "paperclip_runner", status: "fail", testedAt: new Date().toISOString(), checks: [{ code: `${provider}_hello_probe_${authentication ? "auth_required" : /timed out/i.test(message) ? "timeout" : "failed"}`, level: "error", message, - hint: "Check the selected account, model access, and native runtime prerequisites, then retry. Legacy runner is available explicitly in Advanced." }] }; + hint: "Check the selected account, model access, and native runtime prerequisites, then retry. Legacy runner is available explicitly in Advanced." }, + ...(authentication && provider === "codex" ? [{ code: ADAPTER_AUTH_MISSING_CHECK_CODE, level: "error" as const, + message: "The selected Codex account needs authentication.", + hint: "Sign in to Codex or choose an authenticated account, then test again." }] : [])] }; } } diff --git a/skills/paperclip-create-agent/SKILL.md b/skills/paperclip-create-agent/SKILL.md index 0787e1383a..800c58fb6e 100644 --- a/skills/paperclip-create-agent/SKILL.md +++ b/skills/paperclip-create-agent/SKILL.md @@ -201,3 +201,6 @@ cannot honor. Setup and authentication errors never silently change runners. Existing agents keep their saved execution after unrelated edits. An approved hire uses the configuration reviewed by the board. `inheritRuntimeFrom: caller` keeps its existing runtime inheritance and credential restrictions. + +CLI equivalents: `paperclipai agent create --runner legacy --payload-json ...` +and `paperclipai agent hire --runner legacy --payload-json ...`. diff --git a/skills/paperclip-create-agent/references/api-reference.md b/skills/paperclip-create-agent/references/api-reference.md index e77ec696df..422e0be98c 100644 --- a/skills/paperclip-create-agent/references/api-reference.md +++ b/skills/paperclip-create-agent/references/api-reference.md @@ -108,15 +108,3 @@ For hire approvals: - All actions are logged in activity for auditability. - Use markdown in issue/approval comments and include links to approval, agent, and source issue. - After approval resolution, requester may be woken with `PAPERCLIP_APPROVAL_ID` and should reconcile linked issues. - -## Codex execution choice - -Hire requests and direct creation can include `runner: "auto" | "paperclip" | -"legacy"`. Omitted is automatic for new agents: `adapterType: "codex_local"` -selects Paperclip Runner on supported targets; other harnesses keep their -existing execution. Explicit `legacy` preserves Codex CLI execution. The server -returns the resolved adapter and configuration, which are frozen for approval. -Use those returned values for later edits; unrelated updates do not upgrade bots. - -CLI equivalents: `paperclipai agent create --runner legacy --payload-json ...` -and `paperclipai agent hire --runner legacy --payload-json ...`. diff --git a/tests/e2e/chat-adapters-ui-messaging.spec.ts b/tests/e2e/chat-adapters-ui-messaging.spec.ts index cbd94d7bdf..a0fcdda280 100644 --- a/tests/e2e/chat-adapters-ui-messaging.spec.ts +++ b/tests/e2e/chat-adapters-ui-messaging.spec.ts @@ -1437,10 +1437,28 @@ test.describe("Exact failed chat run retry", () => { surface === "agent run" ? `/${seed.prefix}/agents/${seed.agentId}/runs/${failedRunId}` : `/${seed.prefix}/inbox/all`; + const canonicalAgentResponse = + surface === "agent run" + ? page.waitForResponse((response) => { + const url = new URL(response.url()); + return ( + response.request().method() === "GET" && + url.pathname === "/api/agents/maya" && + url.searchParams.get("companyId") === seed.companyId + ); + }) + : null; await page.goto(startPath); - if (surface === "agent run") { - // Canonicalization reloads the agent query. Interact after that - // navigation so a remount cannot discard the retry mutation result. + if (canonicalAgentResponse) { + // The URL changes before the canonical query finishes. Wait for its + // body so the loading boundary cannot discard the retry mutation. + const response = await canonicalAgentResponse; + expect(response.ok()).toBe(true); + expect(await response.json()).toMatchObject({ + id: seed.agentId, + companyId: seed.companyId, + urlKey: "maya", + }); await expect(page).toHaveURL( new RegExp(`/${seed.prefix}/agents/maya/runs/${failedRunId}$`), ); diff --git a/ui/src/components/AgentConfigForm.render.test.tsx b/ui/src/components/AgentConfigForm.render.test.tsx index 5d8d41bf2c..433fe70b10 100644 --- a/ui/src/components/AgentConfigForm.render.test.tsx +++ b/ui/src/components/AgentConfigForm.render.test.tsx @@ -1313,6 +1313,29 @@ describe("AgentConfigForm environment selector", () => { expect(findButton(result.container, "Sign in")).toBeTruthy(); }); + it.each([ + ["auth_required", false], ["auth_required", true], ["failed", false], + ] as const)("gates native Codex sandbox login on the shared auth check (%s, %s)", async (diagnostic, authentication) => { + mockAgentsApi.testEnvironment.mockResolvedValue({ + adapterType: "paperclip_runner", status: "fail", testedAt: new Date(0).toISOString(), + checks: [ + { code: `codex_hello_probe_${diagnostic}`, level: "error", message: "Native Codex setup failed." }, + ...(authentication ? [{ code: "adapter_auth_missing", level: "error", message: "The selected Codex account needs authentication." }] : []), + ], + }); + const result = await renderCodexSandbox({ adapterType: "paperclip_runner", adapterConfig: { provider: "codex" } }); + roots.push(result.root); + expect(findButton(result.container, "Sign in")).toBeFalsy(); + await runTest(result.container); + expect(Boolean(findButton(result.container, "Sign in"))).toBe(authentication); + if (authentication) { + await startLogin(result.container); + expect(mockAgentsApi.startAdapterAuthLogin).toHaveBeenCalledWith("company-1", "codex_local", { environmentId: "sandbox-1" }); + } else { + expect(mockAgentsApi.startAdapterAuthLogin).not.toHaveBeenCalled(); + } + }); + it("hides the Codex login for a provider without the login pseudo-terminal capability", async () => { // The Codex device login runs on a real pseudo-terminal, so it needs a // provider that advertises the login pseudo-terminal capability. E2B reports