mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 04:21:24 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A plugin can be a project workspace provider and also an environment
driver for the agent runs that use those workspaces
> - Core asks the workspace provider to prepare the project workspace
before the run acquires its environment lease
> - The provider knows the run, agent, project, and execution workspace,
but not the environment that the run selected
> - A provider whose compute exists per run (for example, a sandbox that
it creates from the environment's driver config) cannot start or find
the run's compute for the workspace work
> - This pull request lets a provider opt in to receive the run's
selected environment, when that environment is one of the provider's own
drivers
> - The benefit is that such a provider can prepare the workspace on the
run's own compute, with no change to the order of operations in Core
## Linked Issues or Issue Description
**What existing behavior does this improve?**
Plugin workspace preparation for runs whose environment is a driver of
the same plugin.
**Subsystem affected**
Plugin SDK workspace provider params, the plugin manifest workspace
provider declaration, and `callWorkspaceProvider` /
`preparePluginWorkspace` in the server.
**Current behavior**
`prepare` params carry `runId`, `agentId`, `projectId`, and
`executionWorkspaceId`. They carry `taskLease` only when Core acquired
the lease first. They never name the run's environment.
**Proposed behavior**
A provider that declares `workspaceProvider.workspaceEnvironmentVersion:
1` receives `environment: { id, driverKey, config }` when the run's
selected environment is a plugin environment of that same provider.
`config` is the environment's `driverConfig`, the value that the
driver's acquire receives.
## What Changed
- SDK: optional strict `environment` field on
`pluginWorkspaceParamsSchema`.
- Manifest: optional `workspaceProvider.workspaceEnvironmentVersion: 1`,
next to the sibling version fields. Core sends the new field only to
providers that declare it, because providers parse the params with the
strict schema.
- Server: `callWorkspaceProvider` adds `environment` only when the
provider opted in, the environment driver is `plugin`, its `pluginKey`
is this provider, and it names a `driverKey`. The field is not part of
the operation ID.
- Server: `preparePluginWorkspace` passes the run's environment to its
prepare call. This covers normal, verify-existing, fresh, and
replacement prepares.
- Calls that do not have the run's environment do not send it: finalize,
inspect, remove, reservation reconciliation, and the retry-candidate
verification in `fresh-workspace-continuation.ts`.
- Lease acquisition, the `sessionHandoffVersion` gate, and the order of
operations do not change.
- Docs: `doc/plugins/PLUGIN_RUNNER_EXECUTION.md` describes the
capability and its exceptions.
## Verification
- `packages/plugins/sdk/tests/workspace-provider.test.ts`: 11 passed.
The schema accepts the field and rejects malformed values.
- `packages/shared/src/validators/plugin.test.ts`: 65 passed. The
manifest accepts only version 1.
- `server`: `plugin-workspaces.integration`,
`plugin-fresh-workspaces.integration`, `project-environments`, and
`heartbeat/workspace-preparation`: 226 passed, against embedded
Postgres. The new integration test checks the exact shape, an unchanged
operation ID, verification prepares, and omission for an undeclared
provider, another plugin's environment, non-plugin drivers, a missing
`driverKey`, and no environment. It fails when the server stops passing
the environment.
- `pnpm typecheck` passes in `packages/shared`, `packages/plugins/sdk`,
and `server`.
- The full suite was not run locally. CI runs it.
## Risks
- Low risk. Providers that do not declare the capability get the same
params as before.
- `config` is the stored `driverConfig`, in the same form that the
driver's acquire gets. Secret references in it reach the provider in
that same form.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, code search, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
205 lines
10 KiB
TypeScript
205 lines
10 KiB
TypeScript
import { createHash, randomUUID } from "node:crypto";
|
|
import { setTimeout as delay } from "node:timers/promises";
|
|
import { and, eq } from "drizzle-orm";
|
|
import { executionWorkspaces, type Db } from "@paperclipai/db";
|
|
import { readWorkspaceProviderBinding, type Environment } from "@paperclipai/shared";
|
|
import {
|
|
parseWorkspacePrepareResult,
|
|
parseWorkspaceProviderResult,
|
|
workspaceProviderOperationSchema,
|
|
type PluginWorkspaceOperation,
|
|
type PluginWorkspaceParams,
|
|
type PluginWorkspaceResult,
|
|
} from "@paperclipai/plugin-sdk";
|
|
import { conflict } from "../errors.js";
|
|
import { requireWorkspaceProviderWorker, type WorkspaceWorkers } from "./project-workspace-provider.js";
|
|
import { WorkspaceProviderFailureError, workspaceProviderCallFailure } from "./workspace-provider-failure.js";
|
|
|
|
type Workspace = typeof executionWorkspaces.$inferSelect;
|
|
type ProviderRequest = Omit<PluginWorkspaceParams, "deadline">;
|
|
type SelectedEnvironment = Pick<Environment, "id" | "driver" | "config">;
|
|
const legacyWorkspaceOperationMs = 300_000;
|
|
const legacyRpcGraceMs = 10_000;
|
|
|
|
/**
|
|
* The run's environment as the provider's own driver sees it, or undefined when
|
|
* the provider has not opted in or the environment belongs to another driver.
|
|
*/
|
|
function providerRunEnvironment(
|
|
provider: { pluginKey: string; manifestJson: { workspaceProvider?: { workspaceEnvironmentVersion?: 1 } } },
|
|
environment: SelectedEnvironment | null | undefined,
|
|
): PluginWorkspaceParams["environment"] {
|
|
if (provider.manifestJson.workspaceProvider?.workspaceEnvironmentVersion !== 1) return undefined;
|
|
if (!environment || environment.driver !== "plugin") return undefined;
|
|
const config = environment.config ?? {};
|
|
if (config.pluginKey !== provider.pluginKey || typeof config.driverKey !== "string") return undefined;
|
|
const driverConfig = config.driverConfig;
|
|
return {
|
|
id: environment.id,
|
|
driverKey: config.driverKey,
|
|
config: driverConfig && typeof driverConfig === "object" && !Array.isArray(driverConfig)
|
|
? driverConfig as Record<string, unknown> : {},
|
|
};
|
|
}
|
|
|
|
/** No paths or provider credentials are accepted from task callers. */
|
|
export async function callWorkspaceProvider(
|
|
db: Db,
|
|
workers: WorkspaceWorkers | undefined,
|
|
input: {
|
|
workspace: Workspace;
|
|
operation: PluginWorkspaceOperation;
|
|
pluginId?: string;
|
|
advanceGeneration?: boolean;
|
|
runId?: string | null;
|
|
agentId?: string | null;
|
|
removalOperationId?: string;
|
|
taskLease?: PluginWorkspaceParams["taskLease"];
|
|
/** The run's selected environment; forwarded only to an opted-in provider that owns its driver. */
|
|
environment?: SelectedEnvironment | null;
|
|
assertOwned?: () => void;
|
|
},
|
|
) {
|
|
const { workspace } = input;
|
|
const binding = readWorkspaceProviderBinding(workspace.metadata);
|
|
const pluginId = binding?.pluginId ?? input.pluginId;
|
|
if (!pluginId || !workspace.projectId)
|
|
throw conflict("Workspace storage ownership is missing");
|
|
const selectedProvider = await requireWorkspaceProviderWorker(db, workers, workspace.companyId, pluginId, true);
|
|
const verifyExisting = input.operation.kind === "prepare" && input.operation.verifyExisting === true;
|
|
if (verifyExisting && selectedProvider.manifestJson.workspaceProvider?.workspaceVerificationVersion !== 1)
|
|
throw conflict("Workspace verification is unavailable");
|
|
if (input.operation.kind === "prepare" && input.operation.fresh && selectedProvider.manifestJson.workspaceProvider?.freshWorkspaceVersion !== 1)
|
|
throw conflict("Fresh workspace preparation is unavailable");
|
|
if (input.operation.kind === "finalize" &&
|
|
(selectedProvider.manifestJson.workspaceProvider?.finalizationVersion !== 1 || !input.runId || !input.agentId))
|
|
throw conflict("Workspace finalization is unavailable");
|
|
const generation =
|
|
typeof workspace.metadata?.lifecycleGeneration === "number"
|
|
? workspace.metadata.lifecycleGeneration
|
|
: 0;
|
|
const operationGeneration = generation + (input.advanceGeneration ? 1 : 0);
|
|
const operation = workspaceProviderOperationSchema.parse(input.operation);
|
|
if (input.removalOperationId !== undefined && (operation.kind !== "remove" ||
|
|
input.removalOperationId !== createHash("sha256").update(JSON.stringify([workspace.id, operationGeneration, operation])).digest("hex")))
|
|
throw conflict("Workspace removal identity does not match its recorded payload");
|
|
if ((operation.kind === "inspect" || operation.kind === "remove") && input.removalOperationId === undefined
|
|
&& selectedProvider.manifestJson.workspaceProvider?.recoveryPullRequestVersion !== 1) delete operation.recoveryPullRequest;
|
|
if (operation.kind === "prepare" && operation.fresh && !verifyExisting) {
|
|
if (!workspace.sourceIssueId) throw conflict("Fresh workspace task ownership is missing");
|
|
const { stageProviderFreshWorkspace } = await import("./provider-fresh-workspace.js");
|
|
operation.fresh = await stageProviderFreshWorkspace({ db, companyId: workspace.companyId,
|
|
issueId: workspace.sourceIssueId, ...operation.fresh, existing: operation.fresh });
|
|
}
|
|
const environment = providerRunEnvironment(selectedProvider, input.environment);
|
|
const params: ProviderRequest = {
|
|
pluginId,
|
|
companyId: workspace.companyId,
|
|
projectId: workspace.projectId,
|
|
executionWorkspaceId: workspace.id,
|
|
operationId: input.removalOperationId ?? (
|
|
operation.kind === "inspect"
|
|
? randomUUID()
|
|
: createHash("sha256")
|
|
.update(
|
|
JSON.stringify([workspace.id, operationGeneration, operation,
|
|
...(operation.kind === "finalize" ? [input.runId, input.agentId] : []),
|
|
...(input.taskLease ? [input.taskLease.environmentId, input.taskLease.driverKey, input.taskLease.providerLeaseId] : [])]),
|
|
)
|
|
.digest("hex")),
|
|
generation: operationGeneration,
|
|
runId: input.runId ?? null,
|
|
agentId: input.agentId ?? null,
|
|
...(input.taskLease ? { taskLease: input.taskLease } : {}),
|
|
// Context only: the operation ID above stays independent of the environment.
|
|
...(environment ? { environment } : {}),
|
|
operation,
|
|
};
|
|
input.assertOwned?.();
|
|
let result: PluginWorkspaceResult;
|
|
try {
|
|
result = await invokeWorkspaceProvider(workers!, params);
|
|
} catch (error) {
|
|
// A worker that never received the call, a lost reply and a provider-reported
|
|
// failure need different recovery; only a closed provider reason crosses over.
|
|
if (error instanceof WorkspaceProviderFailureError) throw error;
|
|
throw workspaceProviderCallFailure(error);
|
|
}
|
|
// Recheck enablement and ownership after an external effect, including an ambiguous retry.
|
|
await requireWorkspaceProviderWorker(db, workers, workspace.companyId, pluginId);
|
|
const [fresh] = await db
|
|
.select()
|
|
.from(executionWorkspaces)
|
|
.where(
|
|
and(
|
|
eq(executionWorkspaces.id, workspace.id),
|
|
eq(executionWorkspaces.companyId, workspace.companyId),
|
|
),
|
|
);
|
|
if (
|
|
!fresh ||
|
|
fresh.status !== workspace.status ||
|
|
(fresh.metadata?.lifecycleGeneration ?? 0) !== generation ||
|
|
fresh.providerRef !== workspace.providerRef
|
|
) {
|
|
throw conflict("Workspace changed during storage preparation");
|
|
}
|
|
if (result.kind === "prepared" && result.binding.pluginId !== pluginId)
|
|
throw conflict("Workspace provider ownership changed");
|
|
return result;
|
|
}
|
|
|
|
export async function invokeWorkspaceProvider(workers: WorkspaceWorkers, params: ProviderRequest) {
|
|
const verifyExisting = params.operation.kind === "prepare" && params.operation.verifyExisting;
|
|
if (!verifyExisting && params.operation.kind === "prepare" && workers.getWorker(params.pluginId)?.supportedMethods?.includes("workspacePrepare")) {
|
|
const result = await prepareWorkspaceWithPolling(workers, params);
|
|
if (result) return result;
|
|
}
|
|
const timeout = verifyExisting ? 10_000 : params.operation.kind === "finalize" ? 30_000 : legacyWorkspaceOperationMs;
|
|
const request = { ...params, deadline: new Date(Date.now() + timeout).toISOString() };
|
|
return parseWorkspaceProviderResult(request, await workers.call(
|
|
params.pluginId, "workspaceOperation", request, timeout + (verifyExisting ? 2_000 : legacyRpcGraceMs),
|
|
));
|
|
}
|
|
|
|
async function prepareWorkspaceWithPolling(workers: Pick<WorkspaceWorkers, "call">, params: ProviderRequest) {
|
|
const rpcTimeout = 15_000;
|
|
const submit = { ...params, phase: "submit" as const, deadline: new Date(Date.now() + rpcTimeout).toISOString() };
|
|
const receipt = parseWorkspacePrepareResult(submit, await workers.call(params.pluginId, "workspacePrepare", submit, rpcTimeout));
|
|
if (receipt.kind === "unsupported") return null;
|
|
if (receipt.kind === "prepared") return receipt;
|
|
const offeredMs = Date.parse(receipt.deadline) - Date.now();
|
|
if (offeredMs > legacyWorkspaceOperationMs + legacyRpcGraceMs)
|
|
throw new WorkspaceProviderFailureError({
|
|
kind: "provider", reason: "unsupported_operation", retryable: false,
|
|
message: "Workspace provider offered a deadline outside the supported lifecycle budget",
|
|
});
|
|
const expiresAt = performance.now() + offeredMs;
|
|
const remaining = () => {
|
|
const ms = Math.ceil(expiresAt - performance.now());
|
|
if (ms <= 0) throw new WorkspaceProviderFailureError({
|
|
kind: "provider", reason: "deadline_exceeded", retryable: true,
|
|
message: "Workspace storage operation expired; retry the same workspace",
|
|
});
|
|
return ms;
|
|
};
|
|
const poll = { ...params, phase: "poll" as const, deadline: receipt.deadline };
|
|
for (;;) {
|
|
await delay(Math.min(1000, remaining()));
|
|
const timeout = Math.min(rpcTimeout, remaining());
|
|
let response: unknown;
|
|
try {
|
|
response = await workers.call(params.pluginId, "workspacePrepare", poll, timeout);
|
|
} catch (error) {
|
|
const failure = workspaceProviderCallFailure(error);
|
|
if (failure.kind === "transport" || failure.kind === "timeout") continue;
|
|
throw failure;
|
|
}
|
|
remaining();
|
|
const status = parseWorkspacePrepareResult(poll, response);
|
|
if (status.kind === "prepared") return status;
|
|
if (status.kind === "unsupported" || status.deadline !== receipt.deadline)
|
|
throw new Error("Workspace provider changed the preparation deadline");
|
|
}
|
|
}
|