Files
PaperClipAI/server/src/services/plugin-task-credential-writeback.ts
T
Nicky LeachandPaperclip 33cc567f75 Add lease-scoped Runner task hooks to environment plugins (#15406)
## Thinking Path

> - Paperclip coordinates AI agents and their runs.
> - Environment plugins own remote capacity.
> - A plugin needs a typed API to start and stop its installed Runner.
> - Core must derive authority from the saved lease.
> - A lost reply or controller restart must not lose cleanup authority.
> - This PR adds the worker contract and the host task service.
> - Native execution and Runner changes are separate PRs.

## Linked Issues or Issue Description

**Subsystem affected**

Plugin SDK and environment lease services.

**Current behavior**

Environment plugins have no typed Runner task hook. Core cannot dispatch
these tasks or recover their credential handoff through a saved lease.

**Proposed behavior**

Add validated task operations, worker negotiation, company and project
checks, retained-session authorization, and durable cleanup. Keep
credentials out of lease metadata.

**Additional context**

This PR is the first part of a smaller series. Runner installed-home
support is in #16977. Native execution and setup are in #16978. Both are
stacked drafts. This PR does not enable native plugin execution, change
default environments, add a route, or add a database migration.

## What Changed

- Add `supportsTasks` and `onEnvironmentTask` with seven typed
operations.
- Validate worker requests and responses. Reject shell commands and file
transfer operations.
- Pin dispatch and cleanup to the saved company, lease, plugin, and
driver.
- Authorize projects and retained sessions. Validate the current run
separately from the retained PRP identity.
- Save dispatch before the provider call. Fence admission during
cleanup. Require observed process termination.
- Save account writeback authority on the lease. Recover Codex refreshes
through the existing account and secret services.
- Document the task API. Keep the Runner, heartbeat, setup, workspace,
and provider changes in later PRs.

## Verification

- Full `pnpm -r typecheck` and `pnpm build` passed on the separated API
scope.
- 201 focused tests passed: SDK worker dispatch, task validation,
PostgreSQL task lineage, credential recovery, and existing environment
cleanup.
- The lineage test covers a third run, repeated same-run replacements, a
stale submitting run, and exact termination receipts.
- Initial commands hit sandbox socket/IPC restrictions. The permitted
runs executed PostgreSQL and passed. A typecheck attempt lacked Cargo in
PATH; the run with the installed toolchain passed.
- The earlier combined implementation had a full local run with 22,286
passing tests, 120 skips, and eight failed assertions. Task fixture
failures were fixed before this split. Remaining host Git and
temporary-path assertions are recorded in the previous PR revision. This
split does not claim a fresh full local suite pass.
- The standard Greptile review scored this API head 5/5 with no
unresolved threads. CI is green. The single Apex review also scored this
head 5/5 with no actionable findings.
- Fresh API CI lost a self-hosted runner during server shard 9. GitHub
reported lost communication, with no failed test step. The affected jobs
passed on retry without code changes.
- CI on the prior combined head failed an unchanged workspace dev-server
readiness case. No unrelated test fix is included here. Fresh CI is
required for this smaller head.
- This scope does not run a provider or a Cloud test. The user owns
deployment and the intermediate Cloud test.

## Risks

- The API is opt-in. Native plugin execution requires the later
integrations.
- Providers must make task submission idempotent and prove descendant
termination.
- A failed status read or credential export keeps the lease available
for cleanup.
- Codex refresh writeback requires the same active grant, secret
reference, and session epoch. Revocation or account replacement discards
the old refresh.

## Model Used

OpenAI GPT-6 through Codex. Used source inspection, code execution,
tests, and GitHub tools. The exact served model ID and context-window
size are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run local checks and recorded failures and their results
above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-20 09:14:23 -07:00

144 lines
7.3 KiB
TypeScript

import { aiConnectionMetadataSchema } from "@paperclipai/shared";
import { and, eq, sql } from "drizzle-orm";
import { companySecrets, connectionGrants, environmentLeases, toolConnections, type Db } from "@paperclipai/db";
import { z, type PluginEnvironmentTaskCredential } from "@paperclipai/plugin-sdk";
import { mkdtemp, writeFile, rm, realpath } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { isDeepStrictEqual } from "node:util";
import { decideCodexAuthMerge, withAccountHomeSecretMutationLock } from "@paperclipai/adapter-codex-local/server";
import { WORKSPACE_RESTORE_LOCK_TIMEOUT_CODE } from "@paperclipai/adapter-utils/workspace-restore-merge";
import { aiConnectionService } from "./ai-connections.js";
import { secretService } from "./secrets.js";
const bindingSchema = z.object({
grantId: z.string().uuid(),
secretId: z.string().uuid(),
sessionEpoch: z.number().int().nonnegative(),
versionSelector: z.union([z.literal("latest"), z.number().int().positive()]),
}).strict();
export type PluginTaskCredentialWritebackBinding = z.infer<typeof bindingSchema>;
const handoffSchema = z.object({
accountIdentity: z.string().min(1),
binding: bindingSchema,
state: z.enum(["pending", "complete"]),
}).strict();
const key = "runnerTaskCredentialHandoff";
type Scope = { companyId: string; leaseId: string };
/** Save authority, never credential bytes or a controller-local temporary path. */
export async function registerPluginTaskCredentialHandoff(
db: Db,
scope: Scope,
accountIdentity: string,
binding: PluginTaskCredentialWritebackBinding,
) {
const handoff = handoffSchema.parse({ accountIdentity, binding, state: "pending" });
await db.transaction(async tx => {
const [lease] = await tx.select().from(environmentLeases).where(and(
eq(environmentLeases.companyId, scope.companyId), eq(environmentLeases.id, scope.leaseId),
)).for("update");
if (!lease || lease.status !== "active" || lease.metadata?.runnerTaskLease !== true ||
lease.metadata.runnerTaskRetiring === true) {
throw new Error("Plugin task credential lease unavailable");
}
const saved = lease.metadata[key];
if (saved) {
const prior = handoffSchema.parse(saved);
if (prior.accountIdentity !== accountIdentity || !isDeepStrictEqual(prior.binding, handoff.binding)) {
throw new Error("Plugin task credential authority changed");
}
return;
}
await tx.update(environmentLeases).set({ metadata: { ...lease.metadata, [key]: handoff } })
.where(eq(environmentLeases.id, lease.id));
});
}
/** Runs for live completion and the cleanup sweep, including after Core restarts. */
export async function settlePluginTaskCredentialHandoff(
db: Db,
scope: Scope,
metadata: Record<string, unknown>,
fetchCredential: (accountIdentity: string) => Promise<PluginEnvironmentTaskCredential | null>,
) {
if (!metadata[key]) return;
const handoff = handoffSchema.parse(metadata[key]);
if (handoff.state === "complete") return;
if (metadata.runnerTaskDispatched === true && metadata.runnerTaskExecutionStopped !== true) {
throw new Error("Plugin task credential process termination is unconfirmed");
}
const binding = handoff.binding;
const home = await mkdtemp(join(await realpath(tmpdir()), "paperclip-task-credential-"));
try {
// Revocation or an explicit account replacement intentionally discards an old refresh.
const account = async (database: Db) => {
const [grant] = await database.select().from(connectionGrants).where(and(
eq(connectionGrants.companyId, scope.companyId), eq(connectionGrants.id, binding.grantId),
)).for("update");
if (!grant || grant.status !== "active") return null;
const ref = grant.credentialSecretRefs.find(ref => ref.configPath === "ai.credential");
if (!ref || ref.secretId !== binding.secretId ||
(ref.versionSelector ?? "latest") !== binding.versionSelector) return null;
const [secret] = await database.select().from(companySecrets).where(and(
eq(companySecrets.companyId, scope.companyId), eq(companySecrets.id, binding.secretId),
)).for("update");
if (!secret || secret.aiSessionEpoch !== binding.sessionEpoch) return null;
const [connection] = await database.select().from(toolConnections).where(and(
eq(toolConnections.companyId, scope.companyId), eq(toolConnections.id, grant.connectionId),
));
const ai = aiConnectionMetadataSchema.safeParse(connection?.config.ai);
if (!connection || !ai.success || ai.data.provider !== "openai" || ai.data.method !== "subscription") return null;
return { connection, grant };
};
const mutate = <T>(operation: (database: Db) => Promise<T>) => withAccountHomeSecretMutationLock(
undefined, scope.companyId, () => db.transaction(tx => operation(tx as unknown as Db)),
);
const current = metadata.runnerTaskDispatched === true && await mutate(account);
if (current) {
// Do not hold database or account locks while the provider exports its credential.
const credential = await fetchCredential(handoff.accountIdentity);
if (credential !== null) {
if (credential.type !== "codex-auth-json") throw new Error("Plugin task credential provider mismatch");
const incoming = join(home, "incoming.json"), destination = join(home, "current.json");
await writeFile(incoming, credential.authJson, { mode: 0o600 });
const writeBack = () => mutate(async database => {
const selected = await account(database);
if (!selected) return;
const value = await aiConnectionService(database).credential(selected);
await writeFile(destination, value, { mode: 0o600 });
if (await decideCodexAuthMerge(incoming, destination, { errorLabel: "AI account refresh" }) !== 10) return;
await secretService(database).rotate(binding.secretId, {
value: credential.authJson, preserveAiSessionEpoch: true,
}, { userId: selected.grant.subjectUserId });
});
for (let attempt = 0; ; attempt++) {
try {
await writeBack();
break;
} catch (error) {
if (attempt >= 2 || (error as NodeJS.ErrnoException)?.code !== WORKSPACE_RESTORE_LOCK_TIMEOUT_CODE) throw error;
}
}
}
}
const completed = await db.update(environmentLeases).set({
metadata: sql`coalesce(${environmentLeases.metadata}, '{}'::jsonb) || ${JSON.stringify({ [key]: { ...handoff, state: "complete" } })}::jsonb`,
}).where(and(
eq(environmentLeases.companyId, scope.companyId), eq(environmentLeases.id, scope.leaseId),
sql`${environmentLeases.metadata}->${key} = ${JSON.stringify(handoff)}::jsonb`,
)).returning({ id: environmentLeases.id });
if (completed.length !== 1) {
const [current] = await db.select({ metadata: environmentLeases.metadata }).from(environmentLeases).where(and(
eq(environmentLeases.companyId, scope.companyId), eq(environmentLeases.id, scope.leaseId),
));
if (!isDeepStrictEqual(handoffSchema.safeParse(current?.metadata?.[key]).data, { ...handoff, state: "complete" })) {
throw new Error("Plugin task credential handoff changed");
}
}
} finally {
// The provider retains the recoverable credential until completion succeeds.
await rm(home, { recursive: true, force: true });
}
}