mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-21 10:52:06 +01:00
## Thinking Path > - Paperclip manages AI agents doing work for people. > - Agent verification must finish before an accepted task can start. > - A missing AI account default can prevent verification before a provider starts. > - Lifecycle policy treated that owned configuration refusal as a temporary failure and retried each minute. > - The accepted turn remained Queued without explaining the prerequisite. > - Park account configuration failures and reuse the task's existing inline connection card. > - Successful repair rechecks verification and releases the original queued turn under its original authority. ## Linked Issues or Issue Description **What happened?** An OpenCode agent selected the responsible user's OpenRouter default, but that default did not exist. Verification retried unchanged configuration indefinitely and its accepted task stayed Queued. **Expected behavior:** Preserve accepted work, explain the setup prerequisite, and let the appropriate person repair it using the existing task connection UI. Completion must verify the agent before starting the same queued turn. Genuine pause, approval and access gates remain effective. **Reproduction:** Create an OpenCode Runner agent with a responsible-user OpenRouter binding and no personal default; assign a task while verification is pending. The base retries verification and shows an unexplained queue. Related: #18702 handles pending task connection recovery after account repair. This PR handles the earlier agent-verification boundary. #20330 addresses a separate Codex governed-wait timeout and is already merged. ## What Changed - Share owned account-selection failure evidence through a pure library contract. Lifecycle policy parks those configuration failures with fixed, redacted guidance; temporary infrastructure errors retain their retries. - Batch setup-state reads for unstarted queued turns. Distinguish manual repair from automatic setup retries and preserve Stop, started/history state and lifecycle holds. - Reuse the existing `connection_intent` card, account selection and completion APIs. Its controller-only `agentSetup` marker accepts only never-started legacy turns; runtime tokens still require a running run. - Address the card to the lifecycle verifier's account owner, preserving the task's execution principal and private-task visibility. Attach missing cards on admission, verification, startup and full recovery, including retained retries and continuations. Repeated delivery and publication remain bounded. A new queued turn supersedes the setup card of a stopped turn; generic connection cards retain cross-run reuse. - Recheck lifecycle verification after usable-account completion, then release the existing turn through the normal readiness path. Preserve decline, Stop, reassignment and changed setup-owner gates. Fence an in-flight verification failure from overwriting a repair; add no provider wake or recurring worker. - Park owned account refusals during resume as well as initial verification. Bind setup cards to the lifecycle operation, settle obsolete deliveries, and keep explicit Retry guidance accurate. - Reuse the router’s existing lifecycle pin for setup repair without rerouting the task, advancing the pool cursor, or replacing its saved binding. Legacy direct-account cards remain readable. - Hide the incorrect Resume banner during setup, retaining genuine manual/budget pause controls. Document the execution and retry contract. ## Verification - Red: eight original behavior assertions failed on the base, including the real selector's retry decision and the real batched queue projection. Six owned selection cases failed classification. Temporary-error presentation, different task/verifier users, retained retry/continuation recovery and the setup Resume banner also have failing-before/passing-after regressions. - Green: 430 affected backend, lifecycle, shared-contract and UI tests passed before the final review corrections. Subsequent focused runs cover one inline card, real account selection/completion, duplicate delivery/publication, verification before readiness, unchanged run authority, private tasks, no extra provider wake, decline/Stop/reassignment, company isolation, invalid runtime tokens, transient retries and redacted diagnostics. - Final setup/card and generic connection-service suites: 49 tests passed. The stopped-turn case failed before the correction, then passed with real account completion, the stopped source remaining cancelled, and the replacement source remaining queued. - After rebase, all 129 focused assertions passed. The local queue suite’s 88 execution assertions passed repeatedly, but its database-fixture teardown exceeded both 30-second and 120-second hooks. An isolated queue case completed, and the setup/connection suites fully passed. The original queue test and cleanup remain unchanged; broad CI supplies the full-suite result. - Server/UI typechecks, module boundaries, token gates and diff checks passed. `pnpm verify --explain` requested broad CI because the execution-contract document changed; no redundant full local suite was substituted. - Real source acceptance: macOS arm64, Node 26.4.0, isolated embedded PostgreSQL and local board identity. Created an OpenCode/OpenRouter agent and task through the API with no default. Admission retained one unstarted turn and produced the existing inline Connect OpenRouter card. The browser opened its credential form and closed it without entering credentials. The same card/run persisted after server restart and reload, with the setup label and no incorrect Resume. The browser also reproduced Stop followed by a new queued turn retaining the stale card on the prior head. On `12f609c98372fd1e50644028db75cfcc8deb210d`, startup superseded the stale card and retained the same replacement turn, with the original turn still cancelled and no provider execution. - The earlier CI case-search failure also reproduces on exact base `0e3fdebea8257bdff11573081c76b306f12db230` without this PR. Its repair is now in master via #20371; this branch is rebased onto it and contains no API-search changes. - Staging's actual Runtime test returned “Connect an account and choose your personal default” before provider startup. The original turn later started independently on deployed code, then hit a separate workspace-export failure. Automatic file recovery subsequently marked that run successful; its task remains Blocked and the trial’s useful output is unverified. This PR is not deployed; provider authentication through its card and downstream publication remain unverified. No account, credential, default, permission or binding was changed during diagnosis. - Previous revision review: Greptile 5/5 on `12f609c98372fd1e50644028db75cfcc8deb210d`, with all four inline findings addressed and resolved. [Full CI passed](https://github.com/paperclipai/paperclip/actions/runs/42713494508) on that revision: 59 successful checks, one neutral check and three intentional skips, with no pending or failed checks. - Latest revision `976bcae993805a8a2222395bbff9394f6791b424`: resume, router and Retry-copy regressions failed before repair. Green: 92 lifecycle/setup/shared tests, 83 setup/generic-connection/router tests, and 137 UI/shared tests. Server/UI typechecks, module boundaries and token gates passed. [Full CI passed](https://github.com/paperclipai/paperclip/actions/runs/42720731137) on this exact revision: 59 successful checks and three intentional skips, with no pending or failed checks. Greptile reviewed this head at 5/5; all six review findings are addressed and resolved. Real Stop/retry acceptance retained the same replacement turn and current-operation card through two actual server restarts, with no provider execution. ## Risks - Known account configuration failures now require repair or explicit Retry. Temporary failures keep automatic retries; accepted work and authority remain intact. - The projection adds one company-scoped batch read only for unstarted queued turns. Card attachment examines at most 100 active candidates and adds no timer, worker, schema or migration. - The optional stored setup-card marker requires an aware reader during rollback. No run/session/checkpoint identity format changes. Setup presentation grants no access and cannot bypass verification or other gates. ## Model Used - OpenAI GPT-6 (Codex), with reasoning, tool use, code editing and test execution. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
298 lines
20 KiB
TypeScript
298 lines
20 KiB
TypeScript
import { randomUUID } from "node:crypto";
|
|
import { and, asc, eq, inArray, sql } from "drizzle-orm";
|
|
import { agents, agentApiKeys, companies, budgetPolicies, type Db } from "@paperclipai/db";
|
|
import type { AgentLifecycleOperation } from "@paperclipai/shared";
|
|
import { AgentLifecycleConflict, canConfigureAgentConnection, compatibilityStatus, isLifecycleWorkPending, transition, type LifecycleCommand } from "../domain/policy.js";
|
|
import type { LifecycleEffects } from "./effects.js";
|
|
import type { LifecycleStore } from "../application/ports.js";
|
|
|
|
export function assertRootDatabase(db: Db) {
|
|
if ("nestedIndex" in db) throw new AgentLifecycleConflict("Lifecycle commands require the root database connection");
|
|
}
|
|
|
|
type LifecycleRow = typeof agents.$inferSelect;
|
|
type ChangeOptions = { version?: number; owner?: string; reason?: string };
|
|
|
|
// Prepared drafts wait for the wizard; they have no background verification obligation.
|
|
export const pendingLifecycleWork = and(
|
|
inArray(agents.lifecycleState, ["preparing", "verifying", "pausing", "resuming", "terminating", "cleaning_up"]),
|
|
sql`not (${agents.lifecycleState} = 'verifying' and ${agents.lifecycleHolds} @> '["setup"]'::jsonb)`,
|
|
sql`not (${agents.lifecycleState} in ('verifying', 'resuming') and coalesce(${agents.lifecycleOperation}->>'retryRequired', 'false') = 'true')`,
|
|
sql`not (${agents.lifecycleState} = 'verifying' and coalesce(${agents.lifecycleOperation}->>'waitingForSignal', 'false') = 'true')`,
|
|
);
|
|
|
|
/** A bounded startup pass rechecks this saved wait once, without a timer. */
|
|
export const lifecycleSignalWait = and(
|
|
eq(agents.lifecycleState, "verifying"),
|
|
sql`coalesce(${agents.lifecycleOperation}->>'waitingForSignal', 'false') = 'true'`,
|
|
sql`coalesce(${agents.lifecycleOperation}->>'retryRequired', 'false') <> 'true'`,
|
|
);
|
|
|
|
/** The producer owns this outer transaction. Register its intent before either
|
|
* the recheck write or the corresponding external fact commits. */
|
|
/**
|
|
* Releases a lifecycle lease whose holder process restart recovery proved gone.
|
|
* Compare-and-swap on that exact holder and lease owner; the lifecycle state,
|
|
* operation and progress are untouched, so the worker claims the step again.
|
|
*/
|
|
export async function releaseAgentLifecycleLeaseOfHolderInTransaction(
|
|
tx: Pick<Db, "select" | "update">,
|
|
input: { agentId: string; holder: { bootId: string | null; pid: number | null } },
|
|
notify: () => Promise<void>,
|
|
): Promise<{ released: false; reason: string } | { released: true; agent: LifecycleRow; leaseUntil: string }> {
|
|
if (!("nestedIndex" in tx)) throw new AgentLifecycleConflict("Lifecycle lease release requires the caller transaction");
|
|
const [agent] = await tx.select().from(agents).where(eq(agents.id, input.agentId)).for("update");
|
|
const operation = agent?.lifecycleOperation;
|
|
if (!agent || !operation?.leaseOwner || !operation.leaseUntil) return { released: false, reason: "lease_no_longer_held" };
|
|
if (operation.leaseHolder?.bootId !== input.holder.bootId || operation.leaseHolder?.pid !== input.holder.pid) {
|
|
return { released: false, reason: "owner_changed_since_proof" };
|
|
}
|
|
// Register the wake intent before the write so drain and sleep checks see it.
|
|
await notify();
|
|
const [row] = await tx.update(agents).set({
|
|
lifecycleOperation: sql`${agents.lifecycleOperation} - 'leaseOwner' - 'leaseUntil' - 'leaseHolder'`,
|
|
}).where(and(eq(agents.id, agent.id), sql`${agents.lifecycleOperation}->>'leaseOwner' = ${operation.leaseOwner}`))
|
|
.returning({ id: agents.id });
|
|
return row ? { released: true, agent, leaseUntil: operation.leaseUntil } : { released: false, reason: "lease_no_longer_held" };
|
|
}
|
|
|
|
export async function requestAgentLifecycleRecheckInTransaction(
|
|
tx: Pick<Db, "select" | "update">,
|
|
input: { companyId: string; agentId: string; expectedVersion?: number; onlyIfWaiting?: boolean; configurationRepaired?: boolean },
|
|
notify: () => Promise<void>,
|
|
): Promise<boolean> {
|
|
if (!("nestedIndex" in tx)) throw new AgentLifecycleConflict("Lifecycle rechecks require the producer transaction");
|
|
const [agent] = await tx.select().from(agents).where(and(
|
|
eq(agents.id, input.agentId), eq(agents.companyId, input.companyId),
|
|
)).for("update");
|
|
if (!agent || !["preparing", "verifying", "resuming"].includes(agent.lifecycleState)
|
|
|| agent.lifecycleOperation?.retryRequired && !input.configurationRepaired
|
|
|| input.expectedVersion !== undefined && agent.lifecycleVersion !== input.expectedVersion
|
|
|| input.onlyIfWaiting && !agent.lifecycleOperation?.waitingForSignal) return false;
|
|
await notify();
|
|
// Expiry permits a replacement claim; only an owner change fences a late park.
|
|
const owned = Boolean(agent.lifecycleOperation?.leaseOwner);
|
|
await tx.update(agents).set({ lifecycleError: null,
|
|
lifecycleOperation: owned
|
|
? sql`${agents.lifecycleOperation} || '{"recheckRequested":true}'::jsonb`
|
|
: input.configurationRepaired
|
|
? sql`${agents.lifecycleOperation} - 'waitingForSignal' - 'recheckRequested' - 'retryAt' - 'retryRequired' - 'verificationFailures'`
|
|
: sql`${agents.lifecycleOperation} - 'waitingForSignal' - 'recheckRequested' - 'retryAt'`,
|
|
updatedAt: new Date(),
|
|
}).where(and(eq(agents.id, agent.id), eq(agents.companyId, input.companyId), eq(agents.lifecycleVersion, agent.lifecycleVersion)));
|
|
return true;
|
|
}
|
|
|
|
/** The caller owns the accounting transaction and has locked this agent. */
|
|
async function changeLockedAgent(tx: Db, effects: LifecycleEffects, agent: LifecycleRow,
|
|
command: LifecycleCommand, options: ChangeOptions = {}, policyHolds?: string[], restartPause = false) {
|
|
if (options.version !== undefined && options.version !== agent.lifecycleVersion) return null;
|
|
if (command === "resume" && agent.lifecycleHolds.includes("setup")) throw new AgentLifecycleConflict("Finish agent setup before resuming");
|
|
if (command === "complete" && (!options.owner || !agent.lifecycleOperation?.leaseUntil || agent.lifecycleOperation?.leaseOwner !== options.owner ||
|
|
Date.parse(agent.lifecycleOperation?.leaseUntil ?? "") <= Date.now() ||
|
|
!agent.lifecycleOperation.hostComplete || agent.lifecycleRequiredPluginIds === null ||
|
|
agent.lifecycleRequiredPluginIds.some(id => !agent.lifecycleOperation!.completedPluginIds.includes(id)))) return null;
|
|
if (command === "retry") {
|
|
transition(agent.lifecycleState, command);
|
|
if (agent.lifecycleOperation?.leaseUntil && Date.parse(agent.lifecycleOperation.leaseUntil) > Date.now()) {
|
|
await effects.notifyWork(tx);
|
|
const [requested] = await tx.update(agents).set({ lifecycleOperation:
|
|
sql`${agents.lifecycleOperation} || '{"recheckRequested":true}'::jsonb`, updatedAt: new Date(),
|
|
}).where(eq(agents.id, agent.id)).returning();
|
|
return requested;
|
|
}
|
|
await effects.notifyWork(tx);
|
|
const [updated] = await tx.update(agents).set({ lifecycleError: null,
|
|
lifecycleOperation: sql`${agents.lifecycleOperation} - 'retryAt' - 'retryRequired' - 'verificationFailures' - 'leaseOwner' - 'leaseUntil' - 'leaseHolder' - 'waitingForSignal' - 'recheckRequested'`, updatedAt: new Date() }).where(eq(agents.id, agent.id)).returning();
|
|
return updated;
|
|
}
|
|
if (command === "resume" && !["ready", "paused", "pausing", "resuming"].includes(agent.lifecycleState)) transition(agent.lifecycleState, command);
|
|
let holds = policyHolds ?? agent.lifecycleHolds;
|
|
const reason = options.reason ?? "manual";
|
|
if (command === "reconcile" && policyHolds === undefined) {
|
|
if (["pending_approval", "terminating", "cleaning_up", "terminated", "rejected"].includes(agent.lifecycleState)) return agent;
|
|
const [company] = await tx.select().from(companies).where(eq(companies.id, agent.companyId));
|
|
const policies = await tx.select().from(budgetPolicies).where(and(eq(budgetPolicies.companyId, agent.companyId),
|
|
eq(budgetPolicies.scopeType, "agent"), eq(budgetPolicies.scopeId, agent.id)));
|
|
let budgetBlocked = false;
|
|
for (const policy of policies) if (await effects.policyBlocks(tx, policy)) budgetBlocked = true;
|
|
holds = holds.filter(hold => hold !== "budget" && hold !== "company_archived" && hold !== "company_paused");
|
|
if (budgetBlocked) holds.push("budget");
|
|
if (company.status === "archived") holds.push("company_archived");
|
|
if (company.status === "paused") holds.push("company_paused");
|
|
}
|
|
if (command === "pause") holds = [...new Set([...holds, reason])];
|
|
if (command === "resume") holds = holds.filter(hold => reason === "user" ? ["budget", "company_archived", "company_paused"].includes(hold) : hold !== reason);
|
|
const blockingHolds = holds.filter(hold => hold !== "setup" || !["preparing", "verifying"].includes(agent.lifecycleState));
|
|
let next = command === "reconcile"
|
|
? blockingHolds.length && !["pausing", "paused"].includes(agent.lifecycleState) ? "pausing" as const
|
|
: !holds.length && agent.lifecycleState === "paused" ? "resuming" as const : agent.lifecycleState
|
|
: command === "resume" && (holds.length || agent.lifecycleState === "pausing") ? agent.lifecycleState
|
|
: transition(agent.lifecycleState, command, agent.lifecycleOperation?.resumeState);
|
|
if (restartPause && next === "paused") next = "pausing";
|
|
if (command === "complete" && next === "ready" && holds.length) next = "pausing";
|
|
if (command === "complete" && next === "paused" && !holds.length) next = "resuming";
|
|
if (["complete", "reconcile"].includes(command) && next === "paused" && holds.length === 1 && holds[0] === "setup"
|
|
&& await effects.setupRequiresPreparation(tx, agent.companyId, agent.defaultEnvironmentId, agent)) next = "preparing";
|
|
if (next === agent.lifecycleState) {
|
|
if (JSON.stringify(holds) !== JSON.stringify(agent.lifecycleHolds) || agent.pauseReason !== (holds[0] ?? null)) {
|
|
const [updated] = await tx.update(agents).set({ lifecycleHolds: holds, pauseReason: holds[0] ?? null }).where(eq(agents.id, agent.id)).returning();
|
|
return updated;
|
|
}
|
|
return agent;
|
|
}
|
|
const operation: AgentLifecycleOperation = {
|
|
responsibleUserId: agent.lifecycleOperation?.responsibleUserId,
|
|
id: randomUUID(), hostComplete: false, completedPluginIds: [], attempts: 0,
|
|
resumeState: next === "pausing"
|
|
? agent.lifecycleState === "preparing" || agent.lifecycleState === "verifying" ? agent.lifecycleState : agent.lifecycleOperation?.resumeState ?? "ready"
|
|
: agent.lifecycleOperation?.resumeState,
|
|
};
|
|
if (next === "ready") operation.resumeState = "ready";
|
|
if (next === "pausing") {
|
|
const [company] = await tx.select({ pauseReason: companies.pauseReason }).from(companies)
|
|
.where(eq(companies.id, agent.companyId));
|
|
operation.pauseCause = { holds, companyPauseReason: company?.pauseReason ?? null };
|
|
}
|
|
const status = compatibilityStatus(next, agent.status);
|
|
if(!canConfigureAgentConnection(agent)&&canConfigureAgentConnection({lifecycleState:next,status}))await effects.beforeConnectionEligibilityRestore?.(tx,agent.companyId,agent.id);
|
|
if (status !== agent.status) await effects.beforeStatusChange(tx, agent.companyId, agent.id, agent.status, status);
|
|
if (isLifecycleWorkPending(next)) await effects.notifyWork(tx);
|
|
const [updated] = await tx.update(agents).set({
|
|
lifecycleHolds: holds, lifecycleState: next, lifecycleVersion: agent.lifecycleVersion + 1, lifecycleError: null,
|
|
lifecycleOperation: operation, status,
|
|
pauseReason: holds[0] ?? null,
|
|
pausedAt: next === "pausing" ? new Date() : next === "ready" ? null : agent.pausedAt,
|
|
updatedAt: new Date(),
|
|
}).where(eq(agents.id, agent.id)).returning();
|
|
if (next === "terminating" || next === "rejected") {
|
|
await tx.update(agentApiKeys).set({ revokedAt: new Date() }).where(eq(agentApiKeys.agentId, agent.id));
|
|
await effects.clearPrimary(tx, agent.companyId, agent.id);
|
|
}
|
|
if (command === "approve") await effects.recordCreation(tx, agent.companyId, agent.id);
|
|
if (next === "pausing" || next === "resuming" || next === "terminating" || next === "rejected") {
|
|
await effects.recordStatus(tx, agent.companyId, agent.id,
|
|
next === "resuming" ? "paused" : "idle",
|
|
next === "resuming" ? "idle" : next === "pausing" ? "paused" : "terminated");
|
|
}
|
|
return updated;
|
|
}
|
|
|
|
/** Apply a computed hold without resolving policies or opening another transaction. */
|
|
export async function applyAgentPolicyHoldsInTransaction(tx: Db, effects: LifecycleEffects,
|
|
companyId: string, agentId: string, patch: { budget?: boolean; companyStatus?: string; restartPause?: boolean }, lockedAgent?: LifecycleRow) {
|
|
if (!("nestedIndex" in tx)) throw new AgentLifecycleConflict("Policy holds require the caller's accounting transaction");
|
|
const agent = lockedAgent ?? (await tx.select().from(agents).where(and(
|
|
eq(agents.id, agentId), eq(agents.companyId, companyId),
|
|
)).for("update"))[0];
|
|
if (!agent || agent.companyId !== companyId || ["pending_approval", "terminating", "cleaning_up", "terminated", "rejected"].includes(agent.lifecycleState)) return null;
|
|
let holds = [...agent.lifecycleHolds];
|
|
if (patch.budget !== undefined) {
|
|
holds = holds.filter(hold => hold !== "budget");
|
|
if (patch.budget) holds.push("budget");
|
|
}
|
|
if (patch.companyStatus !== undefined) {
|
|
holds = holds.filter(hold => hold !== "company_paused" && hold !== "company_archived");
|
|
if (patch.companyStatus === "paused") holds.push("company_paused");
|
|
if (patch.companyStatus === "archived") holds.push("company_archived");
|
|
}
|
|
return changeLockedAgent(tx, effects, agent, "reconcile", {}, holds, patch.restartPause);
|
|
}
|
|
|
|
export function createLifecycleStore(db: Db, effects: LifecycleEffects): LifecycleStore {
|
|
const get = (id: string) => db.select().from(agents).where(eq(agents.id, id)).then(rows => rows[0] ?? null);
|
|
return {
|
|
get,
|
|
async change(id, command, options = {}) {
|
|
const existing = await get(id);
|
|
if (!existing) return null;
|
|
return effects.transaction(db, existing.companyId, async tx => {
|
|
const [agent] = await tx.select().from(agents).where(eq(agents.id, id)).for("update", command === "reconcile" ? { skipLocked: true } : undefined);
|
|
if (!agent) return null;
|
|
return changeLockedAgent(tx, effects, agent, command, options);
|
|
});
|
|
},
|
|
async claim(id, owner, now) {
|
|
// leaseHolder names the claiming process, so restart recovery settles a dead holder's lease at once.
|
|
const leaseHolder = effects.leaseHolder?.() ?? null;
|
|
const [agent] = await db.update(agents).set({
|
|
lifecycleOperation: sql`(coalesce(${agents.lifecycleOperation}, '{"id":"", "hostComplete":false, "completedPluginIds":[], "attempts":0}'::jsonb) - 'recheckRequested')
|
|
|| jsonb_build_object('leaseOwner', ${owner}::text, 'leaseUntil', ${(new Date(now.getTime() + 120_000)).toISOString()}::text,
|
|
'attempts', coalesce((${agents.lifecycleOperation}->>'attempts')::integer, 0) + 1)
|
|
|| ${JSON.stringify(leaseHolder ? { leaseHolder } : {})}::jsonb`,
|
|
}).where(and(eq(agents.id, id), pendingLifecycleWork,
|
|
sql`(${agents.lifecycleState} in ('pausing', 'terminating', 'cleaning_up') or exists (
|
|
select 1 from ${companies} where ${companies.id} = ${agents.companyId} and ${companies.status} = 'active'))`,
|
|
sql`coalesce((${agents.lifecycleOperation}->>'leaseUntil')::timestamptz, '-infinity') <= ${now.toISOString()}::timestamptz`,
|
|
sql`coalesce((${agents.lifecycleOperation}->>'retryAt')::timestamptz, '-infinity') <= ${now.toISOString()}::timestamptz`,
|
|
)).returning();
|
|
return agent ?? null;
|
|
},
|
|
async renew(agent, owner, now) {
|
|
const rows = await db.update(agents).set({ lifecycleOperation: sql`jsonb_set(${agents.lifecycleOperation}, '{leaseUntil}', ${JSON.stringify(new Date(now.getTime() + 120_000).toISOString())}::jsonb)` })
|
|
.where(and(eq(agents.id, agent.id), eq(agents.lifecycleVersion, agent.lifecycleVersion), sql`${agents.lifecycleOperation}->>'leaseOwner' = ${owner}`)).returning({ id: agents.id });
|
|
return rows.length > 0;
|
|
},
|
|
async setRequiredPlugins(agent, owner, pluginIds) {
|
|
const rows = await db.update(agents).set({ lifecycleRequiredPluginIds: pluginIds })
|
|
.where(and(eq(agents.id, agent.id), eq(agents.lifecycleVersion, agent.lifecycleVersion),
|
|
agent.lifecycleRequiredPluginIds === null ? sql`${agents.lifecycleRequiredPluginIds} is null`
|
|
: sql`${agents.lifecycleRequiredPluginIds} = ${JSON.stringify(agent.lifecycleRequiredPluginIds)}::jsonb`,
|
|
sql`${agents.lifecycleOperation}->>'leaseOwner' = ${owner}`)).returning({ id: agents.id });
|
|
return rows.length > 0;
|
|
},
|
|
async completeHost(agent, owner) {
|
|
const rows = await db.update(agents).set({
|
|
lifecycleOperation: sql`jsonb_set(${agents.lifecycleOperation}, '{hostComplete}', 'true'::jsonb)`,
|
|
}).where(and(eq(agents.id, agent.id), eq(agents.lifecycleVersion, agent.lifecycleVersion),
|
|
sql`${agents.lifecycleOperation}->>'leaseOwner' = ${owner}`)).returning({ id: agents.id });
|
|
return rows.length > 0;
|
|
},
|
|
async completePlugin(agent, owner, pluginId) {
|
|
const rows = await db.update(agents).set({
|
|
lifecycleOperation: sql`jsonb_set(${agents.lifecycleOperation}, '{completedPluginIds}',
|
|
coalesce(${agents.lifecycleOperation}->'completedPluginIds', '[]'::jsonb) || ${JSON.stringify([pluginId])}::jsonb)`,
|
|
}).where(and(eq(agents.id, agent.id), eq(agents.lifecycleVersion, agent.lifecycleVersion),
|
|
sql`${agents.lifecycleRequiredPluginIds} @> ${JSON.stringify([pluginId])}::jsonb`,
|
|
sql`${agents.lifecycleOperation}->>'leaseOwner' = ${owner}`)).returning({ id: agents.id });
|
|
return rows.length > 0;
|
|
},
|
|
async defer(agent, owner, error, now, options) {
|
|
if (options?.waitForSignal && agent.lifecycleState === "verifying") {
|
|
const rows = await db.update(agents).set({ lifecycleError: null,
|
|
lifecycleOperation: sql`(${agents.lifecycleOperation} - 'leaseOwner' - 'leaseUntil' - 'leaseHolder' - 'retryAt' - 'waitingForSignal' - 'recheckRequested')
|
|
|| case when coalesce(${agents.lifecycleOperation}->>'recheckRequested', 'false') = 'true'
|
|
then '{}'::jsonb else '{"waitingForSignal":true}'::jsonb end`,
|
|
updatedAt: now,
|
|
}).where(and(eq(agents.id, agent.id), eq(agents.lifecycleVersion, agent.lifecycleVersion),
|
|
sql`${agents.lifecycleOperation}->>'leaseOwner' = ${owner}`)).returning({ id: agents.id });
|
|
return rows.length > 0;
|
|
}
|
|
const retry: Partial<AgentLifecycleOperation> = options?.retryRequired && ["verifying", "resuming"].includes(agent.lifecycleState)
|
|
? { retryRequired: true }
|
|
: { retryAt: new Date(now.getTime() + (error ? 60_000 : 2_000)).toISOString() };
|
|
if (options?.verificationFailures !== undefined && agent.lifecycleState === "verifying") {
|
|
retry.verificationFailures = options.verificationFailures;
|
|
}
|
|
const rows = await db.update(agents).set({
|
|
lifecycleError: sql`case when coalesce(${agents.lifecycleOperation}->>'recheckRequested', 'false') = 'true' then null else ${error}::text end`,
|
|
lifecycleOperation: sql`((case when coalesce(${agents.lifecycleOperation}->>'recheckRequested', 'false') = 'true'
|
|
then ${agents.lifecycleOperation} - 'verificationFailures' else ${agents.lifecycleOperation} end)
|
|
- 'leaseOwner' - 'leaseUntil' - 'leaseHolder' - 'retryAt' - 'retryRequired' - 'waitingForSignal' - 'recheckRequested')
|
|
|| case when coalesce(${agents.lifecycleOperation}->>'recheckRequested', 'false') = 'true'
|
|
then '{}'::jsonb else ${JSON.stringify(retry)}::jsonb end`,
|
|
updatedAt: now,
|
|
}).where(and(eq(agents.id, agent.id), eq(agents.lifecycleVersion, agent.lifecycleVersion),
|
|
sql`${agents.lifecycleOperation}->>'leaseOwner' = ${owner}`)).returning({ id: agents.id });
|
|
return rows.length > 0;
|
|
},
|
|
async pending(limit, now) {
|
|
return db.select({ id: agents.id }).from(agents).where(and(
|
|
pendingLifecycleWork,
|
|
sql`coalesce((${agents.lifecycleOperation}->>'leaseUntil')::timestamptz, '-infinity') <= ${now.toISOString()}::timestamptz`,
|
|
sql`coalesce((${agents.lifecycleOperation}->>'retryAt')::timestamptz, '-infinity') <= ${now.toISOString()}::timestamptz`,
|
|
)).orderBy(asc(agents.updatedAt), asc(agents.id)).limit(limit).then(rows => rows.map(row => row.id));
|
|
},
|
|
};
|
|
}
|