mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
fix(agents): repair queued setup through task connection cards (#20361)
## Thinking Path > - Paperclip manages AI agents doing work for people. > - Agent verification must finish before an accepted task can start. > - A missing AI account default can prevent verification before a provider starts. > - Lifecycle policy treated that owned configuration refusal as a temporary failure and retried each minute. > - The accepted turn remained Queued without explaining the prerequisite. > - Park account configuration failures and reuse the task's existing inline connection card. > - Successful repair rechecks verification and releases the original queued turn under its original authority. ## Linked Issues or Issue Description **What happened?** An OpenCode agent selected the responsible user's OpenRouter default, but that default did not exist. Verification retried unchanged configuration indefinitely and its accepted task stayed Queued. **Expected behavior:** Preserve accepted work, explain the setup prerequisite, and let the appropriate person repair it using the existing task connection UI. Completion must verify the agent before starting the same queued turn. Genuine pause, approval and access gates remain effective. **Reproduction:** Create an OpenCode Runner agent with a responsible-user OpenRouter binding and no personal default; assign a task while verification is pending. The base retries verification and shows an unexplained queue. Related: #18702 handles pending task connection recovery after account repair. This PR handles the earlier agent-verification boundary. #20330 addresses a separate Codex governed-wait timeout and is already merged. ## What Changed - Share owned account-selection failure evidence through a pure library contract. Lifecycle policy parks those configuration failures with fixed, redacted guidance; temporary infrastructure errors retain their retries. - Batch setup-state reads for unstarted queued turns. Distinguish manual repair from automatic setup retries and preserve Stop, started/history state and lifecycle holds. - Reuse the existing `connection_intent` card, account selection and completion APIs. Its controller-only `agentSetup` marker accepts only never-started legacy turns; runtime tokens still require a running run. - Address the card to the lifecycle verifier's account owner, preserving the task's execution principal and private-task visibility. Attach missing cards on admission, verification, startup and full recovery, including retained retries and continuations. Repeated delivery and publication remain bounded. A new queued turn supersedes the setup card of a stopped turn; generic connection cards retain cross-run reuse. - Recheck lifecycle verification after usable-account completion, then release the existing turn through the normal readiness path. Preserve decline, Stop, reassignment and changed setup-owner gates. Fence an in-flight verification failure from overwriting a repair; add no provider wake or recurring worker. - Park owned account refusals during resume as well as initial verification. Bind setup cards to the lifecycle operation, settle obsolete deliveries, and keep explicit Retry guidance accurate. - Reuse the router’s existing lifecycle pin for setup repair without rerouting the task, advancing the pool cursor, or replacing its saved binding. Legacy direct-account cards remain readable. - Hide the incorrect Resume banner during setup, retaining genuine manual/budget pause controls. Document the execution and retry contract. ## Verification - Red: eight original behavior assertions failed on the base, including the real selector's retry decision and the real batched queue projection. Six owned selection cases failed classification. Temporary-error presentation, different task/verifier users, retained retry/continuation recovery and the setup Resume banner also have failing-before/passing-after regressions. - Green: 430 affected backend, lifecycle, shared-contract and UI tests passed before the final review corrections. Subsequent focused runs cover one inline card, real account selection/completion, duplicate delivery/publication, verification before readiness, unchanged run authority, private tasks, no extra provider wake, decline/Stop/reassignment, company isolation, invalid runtime tokens, transient retries and redacted diagnostics. - Final setup/card and generic connection-service suites: 49 tests passed. The stopped-turn case failed before the correction, then passed with real account completion, the stopped source remaining cancelled, and the replacement source remaining queued. - After rebase, all 129 focused assertions passed. The local queue suite’s 88 execution assertions passed repeatedly, but its database-fixture teardown exceeded both 30-second and 120-second hooks. An isolated queue case completed, and the setup/connection suites fully passed. The original queue test and cleanup remain unchanged; broad CI supplies the full-suite result. - Server/UI typechecks, module boundaries, token gates and diff checks passed. `pnpm verify --explain` requested broad CI because the execution-contract document changed; no redundant full local suite was substituted. - Real source acceptance: macOS arm64, Node 26.4.0, isolated embedded PostgreSQL and local board identity. Created an OpenCode/OpenRouter agent and task through the API with no default. Admission retained one unstarted turn and produced the existing inline Connect OpenRouter card. The browser opened its credential form and closed it without entering credentials. The same card/run persisted after server restart and reload, with the setup label and no incorrect Resume. The browser also reproduced Stop followed by a new queued turn retaining the stale card on the prior head. On `12f609c98372fd1e50644028db75cfcc8deb210d`, startup superseded the stale card and retained the same replacement turn, with the original turn still cancelled and no provider execution. - The earlier CI case-search failure also reproduces on exact base `0e3fdebea8257bdff11573081c76b306f12db230` without this PR. Its repair is now in master via #20371; this branch is rebased onto it and contains no API-search changes. - Staging's actual Runtime test returned “Connect an account and choose your personal default” before provider startup. The original turn later started independently on deployed code, then hit a separate workspace-export failure. Automatic file recovery subsequently marked that run successful; its task remains Blocked and the trial’s useful output is unverified. This PR is not deployed; provider authentication through its card and downstream publication remain unverified. No account, credential, default, permission or binding was changed during diagnosis. - Previous revision review: Greptile 5/5 on `12f609c98372fd1e50644028db75cfcc8deb210d`, with all four inline findings addressed and resolved. [Full CI passed](https://github.com/paperclipai/paperclip/actions/runs/42713494508) on that revision: 59 successful checks, one neutral check and three intentional skips, with no pending or failed checks. - Latest revision `976bcae993805a8a2222395bbff9394f6791b424`: resume, router and Retry-copy regressions failed before repair. Green: 92 lifecycle/setup/shared tests, 83 setup/generic-connection/router tests, and 137 UI/shared tests. Server/UI typechecks, module boundaries and token gates passed. [Full CI passed](https://github.com/paperclipai/paperclip/actions/runs/42720731137) on this exact revision: 59 successful checks and three intentional skips, with no pending or failed checks. Greptile reviewed this head at 5/5; all six review findings are addressed and resolved. Real Stop/retry acceptance retained the same replacement turn and current-operation card through two actual server restarts, with no provider execution. ## Risks - Known account configuration failures now require repair or explicit Retry. Temporary failures keep automatic retries; accepted work and authority remain intact. - The projection adds one company-scoped batch read only for unstarted queued turns. Card attachment examines at most 100 active candidates and adds no timer, worker, schema or migration. - The optional stored setup-card marker requires an aware reader during rollback. No run/session/checkpoint identity format changes. Setup presentation grants no access and cannot bypass verification or other gates. ## Model Used - OpenAI GPT-6 (Codex), with reasoning, tool use, code editing and test execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
8004ad069e
commit
b09f17fbfd
24 files changed
+785
-66
No files matched your search
@@ -1,3 +1,5 @@
|
||||
import { aiConnectionConfigurationFailure, aiConnectionCredentialNotSharedFailure } from "../../../services/ai-connection-configuration-failure.js";
|
||||
import { unprocessable } from "../../../errors.js";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { createLifecycleWorker } from "../application/worker.js";
|
||||
@@ -85,6 +87,23 @@ describe("lifecycle failure diagnostics", () => {
|
||||
expect(lifecycleVerificationNeedsRetry({ stage: "host", phase: "cleaning_up" }, error)).toBe(false);
|
||||
});
|
||||
|
||||
it.each([
|
||||
"ai_connection_responsible_user_missing", "ai_connection_default_missing", "ai_connection_missing",
|
||||
"ai_connection_incompatible", "ai_connection_unavailable", "ai_connection_credential_not_shared",
|
||||
] as const)("holds owned AI selection failures with safe guidance: %s", reason => {
|
||||
const error = reason === "ai_connection_credential_not_shared" ? aiConnectionCredentialNotSharedFailure()
|
||||
: aiConnectionConfigurationFailure(reason, "private provider detail", { connectionId: "private-id" });
|
||||
const context = { stage: "host" as const, phase: "verifying" as const };
|
||||
expect(lifecycleVerificationNeedsRetry(context, error)).toBe(true);
|
||||
const message = lifecycleFailureMessage(context, error);
|
||||
expect(message).toContain("Automatic verification is paused");
|
||||
expect(message).not.toMatch(/private|retry automatically/);
|
||||
expect(lifecycleVerificationNeedsRetry({ stage: "plugin", phase: "verifying" }, error)).toBe(false);
|
||||
// Only the selector can establish this evidence; unrelated HTTP errors still retry.
|
||||
expect(lifecycleVerificationNeedsRetry(context, unprocessable("same text", { code: reason }))).toBe(false);
|
||||
expect(lifecycleVerificationNeedsRetry(context, { code: reason })).toBe(false);
|
||||
});
|
||||
|
||||
it("allows two automatic retries of a generic failed test before waiting for repair", () => {
|
||||
for (const verificationFailures of [1, 2, 3, 8_000]) {
|
||||
const context = { stage: "host" as const, phase: "verifying" as const, attempts: 8_000, verificationFailures };
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { agents, companies, createDb, applyPendingMigrations, closeRegisteredClients, type Db } from "@paperclipai/db";
|
||||
import { agents, companies, companyMemberships, createDb, applyPendingMigrations, closeRegisteredClients, type Db } from "@paperclipai/db";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../../__tests__/helpers/embedded-postgres.js";
|
||||
@@ -12,6 +12,8 @@ import type { LifecycleAgent } from "../application/ports.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../../../services/delivery-work-notifications.js";
|
||||
import { resolveExternalTestDatabaseUrl } from "../../../__tests__/helpers/external-test-database.js";
|
||||
|
||||
import { aiConnectionService } from "../../../services/ai-connections.js";
|
||||
|
||||
const external = resolveExternalTestDatabaseUrl("PAPERCLIP_LIFECYCLE_VERIFICATION_RETRY_TEST_DATABASE_URL");
|
||||
const support = external ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
(support.supported ? describe : describe.skip)("failed lifecycle verification retries", () => {
|
||||
@@ -53,6 +55,80 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu
|
||||
return { agent: await current(agent.id), work, driver, runHost };
|
||||
}
|
||||
|
||||
it("parks a real missing-default selection across restart and permits repair through the normal ready handoff", async () => {
|
||||
const userId = randomUUID();
|
||||
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: userId, status: "active" });
|
||||
let repaired = false;
|
||||
const runHost = vi.fn(async (agent: LifecycleAgent) => {
|
||||
if (agent.lifecycleState === "verifying" && !repaired) await aiConnectionService(db).select({
|
||||
companyId, userId, agentId: agent.id, adapterType: "paperclip_runner", runnerProvider: "opencode",
|
||||
model: "openrouter/deepseek/deepseek-v4.1-flash-1017",
|
||||
binding: { mode: "responsible_user", provider: "openrouter", method: "api_key" },
|
||||
});
|
||||
return "complete" as const;
|
||||
});
|
||||
const onReady = vi.fn();
|
||||
const driver = { requiredPluginIds: async () => [], runHost, runPlugin: async () => "complete" as const, onReady };
|
||||
const work = configureAgentLifecycle(db, driver); workers.push(work);
|
||||
const hired = await createAgentLifecycle(db).requestHire(companyId, { name: "Missing default", adapterType: "process" });
|
||||
await work.process(hired.id);
|
||||
const held = await current(hired.id);
|
||||
expect(held).toMatchObject({ lifecycleState: "verifying", status: "paused", lifecycleOperation: { retryRequired: true } });
|
||||
expect(held.lifecycleError).toContain("personal default");
|
||||
expect(held.lifecycleError).not.toContain("retry automatically");
|
||||
expect(held.lifecycleOperation).not.toHaveProperty("retryAt");
|
||||
expect(onReady).not.toHaveBeenCalled();
|
||||
await work.stop();
|
||||
const restarted = startAgentLifecycle(db, createAgentLifecycleEffects(), driver, () => true); workers.push(restarted);
|
||||
const calls = runHost.mock.calls.length;
|
||||
expect(await restarted.hasPending()).toBe(false);
|
||||
expect(restarted.nextRunAt()).toBeNull();
|
||||
await restarted.sweep();
|
||||
expect(runHost).toHaveBeenCalledTimes(calls);
|
||||
// Repair and Retry must still pass verification before the normal ready handoff.
|
||||
repaired = true;
|
||||
await createAgentLifecycle(db).retry(hired.id);
|
||||
await restarted.sweep();
|
||||
expect(await current(hired.id)).toMatchObject({ lifecycleState: "ready", status: "idle", lifecycleError: null });
|
||||
expect(onReady).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("parks owned account failures after pause/resume and accepts a durable connection repair recheck", async () => {
|
||||
const userId = randomUUID();
|
||||
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: userId, status: "active" });
|
||||
let repaired = false;
|
||||
const runHost = vi.fn(async (agent: LifecycleAgent) => {
|
||||
if (agent.lifecycleState === "resuming" && !repaired) await aiConnectionService(db).select({ companyId, userId,
|
||||
agentId: agent.id, adapterType: "paperclip_runner", runnerProvider: "opencode",
|
||||
model: "openrouter/deepseek/deepseek-v4.1-flash-1017", binding: { mode: "responsible_user", provider: "openrouter", method: "api_key" } });
|
||||
return "complete" as const;
|
||||
});
|
||||
const onReady = vi.fn();
|
||||
const driver = { requiredPluginIds: async () => [], runHost, runPlugin: async () => "complete" as const, onReady };
|
||||
const lifecycle = createAgentLifecycle(db), work = configureAgentLifecycle(db, driver); workers.push(work);
|
||||
const hired = await lifecycle.requestHire(companyId, { name: "Resume account repair", adapterType: "process" });
|
||||
await work.process(hired.id);
|
||||
await lifecycle.pauseAgent(hired.id); await work.process(hired.id);
|
||||
expect((await current(hired.id)).lifecycleState).toBe("paused");
|
||||
onReady.mockClear();
|
||||
await lifecycle.resumeAgent(hired.id); await work.process(hired.id);
|
||||
const held = await current(hired.id);
|
||||
expect(held).toMatchObject({ lifecycleState: "resuming", lifecycleOperation: { retryRequired: true } });
|
||||
expect(held.lifecycleOperation).not.toHaveProperty("retryAt");
|
||||
await work.stop();
|
||||
const restarted = startAgentLifecycle(db, createAgentLifecycleEffects(), driver, () => true); workers.push(restarted);
|
||||
const calls = runHost.mock.calls.length;
|
||||
expect(await restarted.hasPending()).toBe(false);
|
||||
expect(restarted.nextRunAt()).toBeNull();
|
||||
await restarted.sweep(); expect(runHost).toHaveBeenCalledTimes(calls);
|
||||
repaired = true;
|
||||
expect(await db.transaction(tx => requestAgentLifecycleRecheckInTransaction(tx, { companyId, agentId: hired.id,
|
||||
configurationRepaired: true }, () => notifyDeliveryWork(tx, DELIVERY_QUEUES.agentLifecycle)))).toBe(true);
|
||||
await restarted.sweep();
|
||||
expect((await current(hired.id)).lifecycleState).toBe("ready");
|
||||
expect(onReady).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("persists a failed test across worker restarts without scanning, claiming, or probing it again", async () => {
|
||||
const { agent, work, driver, runHost } = await failed();
|
||||
expect(agent).toMatchObject({ lifecycleState: "verifying", status: "paused", lifecycleOperation: { retryRequired: true } });
|
||||
@@ -75,6 +151,28 @@ const support = external ? { supported: true } : await getEmbeddedPostgresTestSu
|
||||
expect((await current(agent.id)).lifecycleOperation).toEqual(agent.lifecycleOperation);
|
||||
});
|
||||
|
||||
it("does not let an in-flight failed verifier overwrite a completed connection repair", async () => {
|
||||
const [agent] = await db.insert(agents).values({ companyId, name: "Repair while checking", status: "paused",
|
||||
lifecycleState: "verifying", lifecycleHolds: [], lifecycleRequiredPluginIds: [],
|
||||
lifecycleOperation: { id: randomUUID(), hostComplete: false, completedPluginIds: [], attempts: 0, verificationFailures: 2 } }).returning();
|
||||
const store = createLifecycleStore(db, createAgentLifecycleEffects());
|
||||
const owner = "old-account-check";
|
||||
const claimed = await store.claim(agent!.id, owner, new Date());
|
||||
expect(claimed).not.toBeNull();
|
||||
await db.transaction(async tx => {
|
||||
expect(await requestAgentLifecycleRecheckInTransaction(tx, { companyId, agentId: agent!.id,
|
||||
expectedVersion: agent!.lifecycleVersion, configurationRepaired: true },
|
||||
() => notifyDeliveryWork(tx, DELIVERY_QUEUES.agentLifecycle))).toBe(true);
|
||||
});
|
||||
await store.defer(claimed!, owner, "Old missing account", new Date(), { retryRequired: true });
|
||||
const repaired = await current(agent!.id);
|
||||
expect(repaired.lifecycleError).toBeNull();
|
||||
expect(repaired.lifecycleOperation).not.toHaveProperty("retryRequired");
|
||||
expect(repaired.lifecycleOperation).not.toHaveProperty("retryAt");
|
||||
expect(repaired.lifecycleOperation).not.toHaveProperty("verificationFailures");
|
||||
expect(await store.claim(agent!.id, "new-check", new Date())).not.toBeNull();
|
||||
});
|
||||
|
||||
it("permits an explicit retry of the same operation and completes verification before readiness", async () => {
|
||||
const { agent, work, runHost } = await failed();
|
||||
let release!: () => void;
|
||||
|
||||
@@ -17,7 +17,7 @@ type ChangeOptions = { version?: number; owner?: string; reason?: string };
|
||||
export const pendingLifecycleWork = and(
|
||||
inArray(agents.lifecycleState, ["preparing", "verifying", "pausing", "resuming", "terminating", "cleaning_up"]),
|
||||
sql`not (${agents.lifecycleState} = 'verifying' and ${agents.lifecycleHolds} @> '["setup"]'::jsonb)`,
|
||||
sql`not (${agents.lifecycleState} = 'verifying' and coalesce(${agents.lifecycleOperation}->>'retryRequired', 'false') = 'true')`,
|
||||
sql`not (${agents.lifecycleState} in ('verifying', 'resuming') and coalesce(${agents.lifecycleOperation}->>'retryRequired', 'false') = 'true')`,
|
||||
sql`not (${agents.lifecycleState} = 'verifying' and coalesce(${agents.lifecycleOperation}->>'waitingForSignal', 'false') = 'true')`,
|
||||
);
|
||||
|
||||
@@ -58,15 +58,15 @@ export async function releaseAgentLifecycleLeaseOfHolderInTransaction(
|
||||
|
||||
export async function requestAgentLifecycleRecheckInTransaction(
|
||||
tx: Pick<Db, "select" | "update">,
|
||||
input: { companyId: string; agentId: string; expectedVersion?: number; onlyIfWaiting?: boolean },
|
||||
input: { companyId: string; agentId: string; expectedVersion?: number; onlyIfWaiting?: boolean; configurationRepaired?: boolean },
|
||||
notify: () => Promise<void>,
|
||||
): Promise<boolean> {
|
||||
if (!("nestedIndex" in tx)) throw new AgentLifecycleConflict("Lifecycle rechecks require the producer transaction");
|
||||
const [agent] = await tx.select().from(agents).where(and(
|
||||
eq(agents.id, input.agentId), eq(agents.companyId, input.companyId),
|
||||
)).for("update");
|
||||
if (!agent || !["preparing", "verifying"].includes(agent.lifecycleState)
|
||||
|| agent.lifecycleOperation?.retryRequired
|
||||
if (!agent || !["preparing", "verifying", "resuming"].includes(agent.lifecycleState)
|
||||
|| agent.lifecycleOperation?.retryRequired && !input.configurationRepaired
|
||||
|| input.expectedVersion !== undefined && agent.lifecycleVersion !== input.expectedVersion
|
||||
|| input.onlyIfWaiting && !agent.lifecycleOperation?.waitingForSignal) return false;
|
||||
await notify();
|
||||
@@ -75,7 +75,9 @@ export async function requestAgentLifecycleRecheckInTransaction(
|
||||
await tx.update(agents).set({ lifecycleError: null,
|
||||
lifecycleOperation: owned
|
||||
? sql`${agents.lifecycleOperation} || '{"recheckRequested":true}'::jsonb`
|
||||
: sql`${agents.lifecycleOperation} - 'waitingForSignal' - 'recheckRequested' - 'retryAt'`,
|
||||
: input.configurationRepaired
|
||||
? sql`${agents.lifecycleOperation} - 'waitingForSignal' - 'recheckRequested' - 'retryAt' - 'retryRequired' - 'verificationFailures'`
|
||||
: sql`${agents.lifecycleOperation} - 'waitingForSignal' - 'recheckRequested' - 'retryAt'`,
|
||||
updatedAt: new Date(),
|
||||
}).where(and(eq(agents.id, agent.id), eq(agents.companyId, input.companyId), eq(agents.lifecycleVersion, agent.lifecycleVersion)));
|
||||
return true;
|
||||
@@ -266,16 +268,19 @@ export function createLifecycleStore(db: Db, effects: LifecycleEffects): Lifecyc
|
||||
sql`${agents.lifecycleOperation}->>'leaseOwner' = ${owner}`)).returning({ id: agents.id });
|
||||
return rows.length > 0;
|
||||
}
|
||||
const retry: Partial<AgentLifecycleOperation> = options?.retryRequired && agent.lifecycleState === "verifying"
|
||||
const retry: Partial<AgentLifecycleOperation> = options?.retryRequired && ["verifying", "resuming"].includes(agent.lifecycleState)
|
||||
? { retryRequired: true }
|
||||
: { retryAt: new Date(now.getTime() + (error ? 60_000 : 2_000)).toISOString() };
|
||||
if (options?.verificationFailures !== undefined && agent.lifecycleState === "verifying") {
|
||||
retry.verificationFailures = options.verificationFailures;
|
||||
}
|
||||
const rows = await db.update(agents).set({
|
||||
lifecycleError: error,
|
||||
lifecycleOperation: sql`(${agents.lifecycleOperation} - 'leaseOwner' - 'leaseUntil' - 'leaseHolder' - 'retryAt' - 'retryRequired' - 'waitingForSignal' - 'recheckRequested')
|
||||
|| ${JSON.stringify(retry)}::jsonb`,
|
||||
lifecycleError: sql`case when coalesce(${agents.lifecycleOperation}->>'recheckRequested', 'false') = 'true' then null else ${error}::text end`,
|
||||
lifecycleOperation: sql`((case when coalesce(${agents.lifecycleOperation}->>'recheckRequested', 'false') = 'true'
|
||||
then ${agents.lifecycleOperation} - 'verificationFailures' else ${agents.lifecycleOperation} end)
|
||||
- 'leaseOwner' - 'leaseUntil' - 'leaseHolder' - 'retryAt' - 'retryRequired' - 'waitingForSignal' - 'recheckRequested')
|
||||
|| case when coalesce(${agents.lifecycleOperation}->>'recheckRequested', 'false') = 'true'
|
||||
then '{}'::jsonb else ${JSON.stringify(retry)}::jsonb end`,
|
||||
updatedAt: now,
|
||||
}).where(and(eq(agents.id, agent.id), eq(agents.lifecycleVersion, agent.lifecycleVersion),
|
||||
sql`${agents.lifecycleOperation}->>'leaseOwner' = ${owner}`)).returning({ id: agents.id });
|
||||
|
||||
@@ -1,6 +1,15 @@
|
||||
import { readAiConnectionConfigurationFailure } from "../../../lib/ai-connection-configuration-failure.js";
|
||||
import type { LifecycleFailure } from "./ports.js";
|
||||
|
||||
// Only fixed text may leave this boundary. Provider errors can contain secrets.
|
||||
const connectionReasons = {
|
||||
ai_connection_responsible_user_missing: "Set the responsible user before selecting this agent's AI connection.",
|
||||
ai_connection_default_missing: "Connect an AI account and choose the responsible user's personal default for this provider.",
|
||||
ai_connection_missing: "The selected AI connection is unavailable. Choose an available connection in Configuration.",
|
||||
ai_connection_incompatible: "Choose an AI connection compatible with this agent's harness and model in Configuration.",
|
||||
ai_connection_unavailable: "Reconnect or validate the selected AI account in Configuration.",
|
||||
ai_connection_credential_not_shared: "The credential is not shared with the responsible user. Ask its owner for access or choose an authorized connection.",
|
||||
} as const;
|
||||
const reasons = {
|
||||
harness_test_failed: "The execution test did not pass. Open Configuration and run the agent test to inspect the failed checks.",
|
||||
ai_connection_required: "This agent's environment requires a managed AI connection and an isolated Runner home. Check both in Configuration, then retry.",
|
||||
@@ -24,11 +33,14 @@ const configurationFailures = new Set([
|
||||
|
||||
export function lifecycleVerificationNeedsRetry(context: LifecycleFailure, error: unknown): boolean {
|
||||
const code = lifecycleFailureCode(error) ?? "";
|
||||
return context.phase === "verifying" && context.stage === "host"
|
||||
&& (configurationFailures.has(code) || (code === "harness_test_failed" && (context.verificationFailures ?? 0) >= 3));
|
||||
return ["verifying", "resuming"].includes(context.phase ?? "") && context.stage === "host"
|
||||
&& (readAiConnectionConfigurationFailure(error) !== null || context.phase === "verifying"
|
||||
&& (configurationFailures.has(code) || (code === "harness_test_failed" && (context.verificationFailures ?? 0) >= 3)));
|
||||
}
|
||||
|
||||
export function lifecycleFailureCode(error: unknown): string | undefined {
|
||||
const selectionFailure = readAiConnectionConfigurationFailure(error);
|
||||
if (selectionFailure) return selectionFailure;
|
||||
const candidate = error as { code?: unknown; cause?: { code?: unknown } } | null;
|
||||
return [candidate?.code, candidate?.cause?.code].find((code): code is string =>
|
||||
typeof code === "string" && (Object.hasOwn(reasons, code) || transientCodes.has(code)));
|
||||
@@ -46,7 +58,8 @@ export function lifecycleFailureMessage(context: LifecycleFailure, error: unknow
|
||||
: ["save_plugins", "complete_host", "complete_plugin", "transition"].includes(context.stage)
|
||||
? `${phase}: saving progress` : phase;
|
||||
const code = lifecycleFailureCode(error);
|
||||
const reason = code && Object.hasOwn(reasons, code) ? reasons[code as keyof typeof reasons]
|
||||
const reason = code && Object.hasOwn(connectionReasons, code) ? connectionReasons[code as keyof typeof connectionReasons]
|
||||
: code && Object.hasOwn(reasons, code) ? reasons[code as keyof typeof reasons]
|
||||
: "Paperclip could not complete this step. It will retry automatically. If this continues, contact your instance administrator.";
|
||||
return `${step} failed. ${reason}${lifecycleVerificationNeedsRetry(context, error)
|
||||
? " Automatic verification is paused. Update Configuration or choose Retry now." : ""}`;
|
||||
|
||||
Reference in new issue
Block a user