mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Plugin Runner owners manage a physical task through an environment lease. > - The existing replacement path can assign a new physical task to that same lease. > - A delayed old owner can then stop the replacement or mark it stopped. > - Cleanup must keep the original physical task identity across RPC and database writes. > - This pull request pins that identity and keeps cleanup available to the new owner. ## Linked Issues or Issue Description Refs #19134 and #19157. This fix is based on master and does not enable repeated stopped-source recovery. A search found no duplicate fix. **What happened?** The maintained `createOwnedSandbox` replacement path changes `providerLeaseId` on one lease. An old task owner uses that mutable lease for a later Stop. A stopped-state response already in flight can also update the new task's flags. **Expected behavior** The old owner must address its original task. Its late cleanup writes must not alter the replacement. The replacement's own owner must still stop and complete it. **Steps to reproduce** 1. Create a plugin task owner for a stopped task. 2. Replace that task through the existing describe/startup path on the same lease. 3. Resume an old checkpoint or a held old status response. 4. Observe a Stop for the replacement or an incorrect stopped flag before this fix. ## What Changed - Require each task owner to keep the physical ID returned by its description. Reject a rebind to a different task. - Carry the ID through the existing internal task context after startup ends. Reject a changed physical task before RPC dispatch. - Match the physical task in stopped-state, completion and credential-handoff writes. - Hold the exact lease row through the retained-session database update and local checkpoint save. - Cover the maintained replacement path with real PostgreSQL tests. Preserve positive cleanup for the replacement owner. Document this boundary. ## Verification - Both replacement regressions failed on unmodified master `10a7d039c8`: delayed old-owner Stop targeted the replacement, and a held old status response marked it stopped. - Six focused files passed 160 tests: task authority, task runtime, warm tasks, execution target, credential writeback and task owner. - Four selected executor factory cases passed. They cover task-only providers and the physical task binding after the startup context ends. - Runner TypeScript, plugin SDK and adapter-utils builds passed. Server TypeScript check passed. Diff and added-line secret scans passed. - All 15 setup-probe cases passed with the actual Runner and fake provider binaries. The first local attempt lacked cargo on PATH. A sandboxed native attempt timed out in seven process cases; the unchanged run with loopback permission passed all 15. A separate initial DB integration attempt was denied a loopback listener; its permission-correct run passed. - Review found three missed constructors in the existing SDK/Core lifecycle fixture. They now use the actual describe result. All four cases in that real PostgreSQL and worker RPC file passed. - The first GitHub attempt failed in the unchanged GitHub recovery pagination case in `chat-channels.integration.test.ts`: its zero-request assertion observed a configuration GET. That exact case passed locally without code or assertion changes. The one failed-job-only retry passed; the initial failure log is retained. - [GitHub workflow 42082587576](https://github.com/paperclipai/paperclip/actions/runs/42082587576) passed on its second attempt. All CI checks completed, and Snyk passed. Standard Greptile scored exact head `fb914d98b99e` 5/5 with no unresolved threads. The branch is mergeable and ready for review. ## Risks The task context change is internal. It adds no plugin wire field, schema or Runner mode. A stale owner now gets an error and cannot clean up a replacement. A new owner can still clean up that task. The retained checkpoint write holds one existing lease row during a bounded local file write. Repeated stopped-source incarnation admission, runtime pins and deployments are outside this change. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), high reasoning. Used repository tools, TypeScript and real PostgreSQL tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR using the bug template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change - [x] I have run the focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no unresolved findings - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
146 lines
7.6 KiB
TypeScript
146 lines
7.6 KiB
TypeScript
import { aiConnectionMetadataSchema } from "@paperclipai/shared";
|
|
import { and, eq, sql } from "drizzle-orm";
|
|
import { companySecrets, connectionGrants, environmentLeases, toolConnections, type Db } from "@paperclipai/db";
|
|
import { z, type PluginEnvironmentTaskCredential } from "@paperclipai/plugin-sdk";
|
|
import { mkdtemp, writeFile, rm, realpath } from "node:fs/promises";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
import { isDeepStrictEqual } from "node:util";
|
|
import { decideCodexAuthMerge, withAccountHomeSecretMutationLock } from "@paperclipai/adapter-codex-local/server";
|
|
import { WORKSPACE_RESTORE_LOCK_TIMEOUT_CODE } from "@paperclipai/adapter-utils/workspace-restore-merge";
|
|
import { aiConnectionService } from "./ai-connections.js";
|
|
import { secretService } from "./secrets.js";
|
|
|
|
const bindingSchema = z.object({
|
|
grantId: z.string().uuid(),
|
|
secretId: z.string().uuid(),
|
|
sessionEpoch: z.number().int().nonnegative(),
|
|
versionSelector: z.union([z.literal("latest"), z.number().int().positive()]),
|
|
}).strict();
|
|
export type PluginTaskCredentialWritebackBinding = z.infer<typeof bindingSchema>;
|
|
const handoffSchema = z.object({
|
|
accountIdentity: z.string().min(1),
|
|
binding: bindingSchema,
|
|
state: z.enum(["pending", "complete"]),
|
|
}).strict();
|
|
const key = "runnerTaskCredentialHandoff";
|
|
type Scope = { companyId: string; leaseId: string; expectedTaskId?: string };
|
|
|
|
/** Save authority, never credential bytes or a controller-local temporary path. */
|
|
export async function registerPluginTaskCredentialHandoff(
|
|
db: Db,
|
|
scope: Scope,
|
|
accountIdentity: string,
|
|
binding: PluginTaskCredentialWritebackBinding,
|
|
) {
|
|
const handoff = handoffSchema.parse({ accountIdentity, binding, state: "pending" });
|
|
await db.transaction(async tx => {
|
|
const [lease] = await tx.select().from(environmentLeases).where(and(
|
|
eq(environmentLeases.companyId, scope.companyId), eq(environmentLeases.id, scope.leaseId),
|
|
)).for("update");
|
|
if (!lease || lease.status !== "active" || lease.metadata?.runnerTaskLease !== true ||
|
|
lease.metadata.runnerTaskRetiring === true) {
|
|
throw new Error("Plugin task credential lease unavailable");
|
|
}
|
|
const saved = lease.metadata[key];
|
|
if (saved) {
|
|
const prior = handoffSchema.parse(saved);
|
|
if (prior.accountIdentity !== accountIdentity || !isDeepStrictEqual(prior.binding, handoff.binding)) {
|
|
throw new Error("Plugin task credential authority changed");
|
|
}
|
|
return;
|
|
}
|
|
await tx.update(environmentLeases).set({ metadata: { ...lease.metadata, [key]: handoff } })
|
|
.where(eq(environmentLeases.id, lease.id));
|
|
});
|
|
}
|
|
|
|
/** Runs for live completion and the cleanup sweep, including after Core restarts. */
|
|
export async function settlePluginTaskCredentialHandoff(
|
|
db: Db,
|
|
scope: Scope,
|
|
metadata: Record<string, unknown>,
|
|
fetchCredential: (accountIdentity: string) => Promise<PluginEnvironmentTaskCredential | null>,
|
|
) {
|
|
if (!metadata[key]) return;
|
|
const handoff = handoffSchema.parse(metadata[key]);
|
|
if (handoff.state === "complete") return;
|
|
if (metadata.runnerTaskDispatched === true && metadata.runnerTaskExecutionStopped !== true) {
|
|
throw new Error("Plugin task credential process termination is unconfirmed");
|
|
}
|
|
const binding = handoff.binding;
|
|
const home = await mkdtemp(join(await realpath(tmpdir()), "paperclip-task-credential-"));
|
|
try {
|
|
// Revocation or an explicit account replacement intentionally discards an old refresh.
|
|
const account = async (database: Db) => {
|
|
const [grant] = await database.select().from(connectionGrants).where(and(
|
|
eq(connectionGrants.companyId, scope.companyId), eq(connectionGrants.id, binding.grantId),
|
|
)).for("update");
|
|
if (!grant || grant.status !== "active") return null;
|
|
const ref = grant.credentialSecretRefs.find(ref => ref.configPath === "ai.credential");
|
|
if (!ref || ref.secretId !== binding.secretId ||
|
|
(ref.versionSelector ?? "latest") !== binding.versionSelector) return null;
|
|
const [secret] = await database.select().from(companySecrets).where(and(
|
|
eq(companySecrets.companyId, scope.companyId), eq(companySecrets.id, binding.secretId),
|
|
)).for("update");
|
|
if (!secret || secret.aiSessionEpoch !== binding.sessionEpoch) return null;
|
|
const [connection] = await database.select().from(toolConnections).where(and(
|
|
eq(toolConnections.companyId, scope.companyId), eq(toolConnections.id, grant.connectionId),
|
|
));
|
|
const ai = aiConnectionMetadataSchema.safeParse(connection?.config.ai);
|
|
if (!connection || !ai.success || ai.data.provider !== "openai" || ai.data.method !== "subscription") return null;
|
|
return { connection, grant };
|
|
};
|
|
const mutate = <T>(operation: (database: Db) => Promise<T>) => withAccountHomeSecretMutationLock(
|
|
undefined, scope.companyId, () => db.transaction(tx => operation(tx as unknown as Db)),
|
|
);
|
|
const current = metadata.runnerTaskDispatched === true && await mutate(account);
|
|
if (current) {
|
|
// Do not hold database or account locks while the provider exports its credential.
|
|
const credential = await fetchCredential(handoff.accountIdentity);
|
|
if (credential !== null) {
|
|
if (credential.type !== "codex-auth-json") throw new Error("Plugin task credential provider mismatch");
|
|
const incoming = join(home, "incoming.json"), destination = join(home, "current.json");
|
|
await writeFile(incoming, credential.authJson, { mode: 0o600 });
|
|
const writeBack = () => mutate(async database => {
|
|
const selected = await account(database);
|
|
if (!selected) return;
|
|
const value = await aiConnectionService(database).credential(selected);
|
|
await writeFile(destination, value, { mode: 0o600 });
|
|
if (await decideCodexAuthMerge(incoming, destination, { errorLabel: "AI account refresh" }) !== 10) return;
|
|
await secretService(database).rotate(binding.secretId, {
|
|
value: credential.authJson, preserveAiSessionEpoch: true,
|
|
}, { userId: selected.grant.subjectUserId });
|
|
});
|
|
for (let attempt = 0; ; attempt++) {
|
|
try {
|
|
await writeBack();
|
|
break;
|
|
} catch (error) {
|
|
if (attempt >= 2 || (error as NodeJS.ErrnoException)?.code !== WORKSPACE_RESTORE_LOCK_TIMEOUT_CODE) throw error;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
const completed = await db.update(environmentLeases).set({
|
|
metadata: sql`coalesce(${environmentLeases.metadata}, '{}'::jsonb) || ${JSON.stringify({ [key]: { ...handoff, state: "complete" } })}::jsonb`,
|
|
}).where(and(
|
|
eq(environmentLeases.companyId, scope.companyId), eq(environmentLeases.id, scope.leaseId),
|
|
scope.expectedTaskId === undefined ? undefined : eq(environmentLeases.providerLeaseId, scope.expectedTaskId),
|
|
sql`${environmentLeases.metadata}->${key} = ${JSON.stringify(handoff)}::jsonb`,
|
|
)).returning({ id: environmentLeases.id });
|
|
if (completed.length !== 1) {
|
|
const [current] = await db.select({ metadata: environmentLeases.metadata, taskId: environmentLeases.providerLeaseId }).from(environmentLeases).where(and(
|
|
eq(environmentLeases.companyId, scope.companyId), eq(environmentLeases.id, scope.leaseId),
|
|
));
|
|
if (scope.expectedTaskId !== undefined && current?.taskId !== scope.expectedTaskId ||
|
|
!isDeepStrictEqual(handoffSchema.safeParse(current?.metadata?.[key]).data, { ...handoff, state: "complete" })) {
|
|
throw new Error("Plugin task credential handoff changed");
|
|
}
|
|
}
|
|
} finally {
|
|
// The provider retains the recoverable credential until completion succeeds.
|
|
await rm(home, { recursive: true, force: true });
|
|
}
|
|
}
|