mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-25 09:41:04 +01:00
Support project-aware environment task admission
Submit accepts multiple project IDs and the host validates their company scope before dispatch. Keep PRP compatibility for the Runner connection while documenting provider-owned resource preparation and task lifecycle. Verified SDK build, server typecheck, and 21 focused task and worker contract tests including multi-project scope rejection. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
9 files changed
+75
-34
No files matched your search
@@ -1,18 +1,27 @@
|
||||
# Remote Paperclip Runner tasks
|
||||
# Environment task admission
|
||||
|
||||
An environment driver can submit tasks directly to a remote Paperclip Runner
|
||||
and connect to it using Paperclip Runner Protocol (PRP). Declare `supportsTasks: true`
|
||||
on the driver and implement `onEnvironmentTask`. The worker advertises
|
||||
`environmentTask` during initialization. Both declarations must be present. The existing
|
||||
An environment driver admits tasks on provider-managed execution resources.
|
||||
Admission prepares the task's project and repository storage, starts Runner, and
|
||||
provides its Paperclip Runner Protocol (PRP) connection. The plugin owns resource
|
||||
preparation and task lifecycle. Declare `supportsTasks: true` on the driver and
|
||||
implement `onEnvironmentTask`. The worker advertises `environmentTask` during
|
||||
initialization. Both declarations must be present. The existing
|
||||
`environment.drivers.register` capability applies.
|
||||
|
||||
The server-only `environmentRuntime.task({companyId, leaseId, operation})` method
|
||||
loads the persisted lease and its run. It resolves the exact plugin recorded at
|
||||
acquisition. It supplies the agent, issue, and project context. It does not accept
|
||||
these identities or the plugin identity from the operation. An environment edit
|
||||
cannot redirect an existing task to a replacement provider. The caller must
|
||||
acquisition. It supplies the agent and issue context and validates requested
|
||||
projects against the company. The persisted lease determines the plugin and
|
||||
execution identity. An environment edit cannot redirect an existing task to a
|
||||
replacement provider. The caller must
|
||||
already authorize access to the company.
|
||||
|
||||
Submit includes `projectIds`, the projects whose storage the task needs. The host
|
||||
checks that every ID belongs to the task's company and passes the validated list
|
||||
to the plugin. The plugin resolves those projects into provider-specific storage
|
||||
and mount inputs. Project and repository volume preparation belongs to task
|
||||
admission; PRP carries the subsequent Runner session.
|
||||
|
||||
This contract supplies execution capabilities. It does not select a provider for
|
||||
heartbeat runs, stage runtime assets, or replace native Runner startup. Consumers
|
||||
must integrate those steps before enabling task execution. Readiness checks remain
|
||||
@@ -25,10 +34,9 @@ the task's durable attempt ID. Save it before a remote call. Do not use a persis
|
||||
machine ID as this task ID. Multiple attempts can use the same underlying resource. Provider task IDs are
|
||||
opaque strings; each provider owns its addressing constraints.
|
||||
|
||||
- `submit`: supplies Runner identity, harness, the client's supported PRP version
|
||||
range (`runner.protocolMin` and `runner.protocolMax`, inclusive), and a transient
|
||||
bootstrap ticket. The Runner run and lease IDs
|
||||
must match the persisted host records. Only a running run with an active,
|
||||
- `submit`: supplies a unique list of up to 64 project UUIDs, Runner identity,
|
||||
harness, the client's supported PRP version range (`runner.protocolMin` and `runner.protocolMax`, inclusive), and a transient
|
||||
bootstrap ticket. The Runner run and lease IDs must match the persisted host records. Only a running run with an active,
|
||||
unexpired lease can submit.
|
||||
- `status`: returns the phase and optional exit code. Optional `executionStopped`
|
||||
is live provider evidence that the complete task process tree has stopped.
|
||||
@@ -74,3 +82,4 @@ environment or run has been deleted. The corresponding `environmentId`, `runId`,
|
||||
and `agentId` can be null. Providers must use their persisted lease binding for
|
||||
cleanup; they must not need a current project or environment configuration.
|
||||
Submission and connection require the environment and running run to still exist.
|
||||
Cleanup receives an empty `projectIds` list and uses the persisted task binding.
|
||||
@@ -680,8 +680,9 @@ menu when no unique contribution exists, discovery fails, the module is missing,
|
||||
or rendering throws. The slot is a React-only contract; do not use a custom
|
||||
element export. This replaces only the menu, not company policy or authorization.
|
||||
|
||||
## Remote Paperclip Runner tasks
|
||||
## Environment task admission
|
||||
|
||||
Environment drivers can [submit tasks directly to a remote Paperclip Runner](ENVIRONMENT_TASKS.md)
|
||||
and connect to it using Paperclip Runner Protocol (PRP). The driver manages Runner
|
||||
launch, status, connections, completion, and cancellation for each task lease.
|
||||
Environment drivers can [admit tasks on provider-managed resources](ENVIRONMENT_TASKS.md).
|
||||
The driver uses host-supplied project context to prepare project and repository
|
||||
storage, starts Runner, and exposes its PRP connection. It also manages task
|
||||
status, completion, cancellation, and cleanup.
|
||||
@@ -372,7 +372,7 @@ export interface PluginDefinition {
|
||||
params: PluginEnvironmentValidateConfigParams,
|
||||
): Promise<PluginEnvironmentValidationResult>;
|
||||
|
||||
/** Submit, inspect, connect to, or finish a remote Paperclip Runner task over PRP. */
|
||||
/** Admit, inspect, connect to, or finish a task on provider-managed execution resources. */
|
||||
onEnvironmentTask?(params: PluginEnvironmentTaskParams): Promise<PluginEnvironmentTaskResult>;
|
||||
|
||||
/** Called to test reachability or readiness of a plugin-hosted environment. */
|
||||
|
||||
@@ -9,10 +9,11 @@ const secureUrl = z.string().url().refine(value => {
|
||||
return url.protocol === "wss:" && !url.username && !url.password && !url.search && !url.hash;
|
||||
}, "Expected a credential-free WSS URL");
|
||||
|
||||
/** One remote Paperclip Runner execution attempt over PRP. Secrets are transient RPC input. */
|
||||
/** One provider-managed execution attempt with a Runner PRP connection. Secrets are transient RPC input. */
|
||||
export const environmentTaskOperationSchema = z.discriminatedUnion("kind", [
|
||||
z.object({
|
||||
kind: z.literal("submit"),
|
||||
projectIds: z.array(z.string().uuid()).max(64).refine(ids => new Set(ids).size === ids.length, "Duplicate project IDs"),
|
||||
runner: z.object({
|
||||
/** Inclusive PRP version range supported by the submitting client. */
|
||||
protocolMin: z.number().int().min(1).max(Number.MAX_SAFE_INTEGER),
|
||||
@@ -63,7 +64,8 @@ export interface PluginEnvironmentTaskParams extends Omit<PluginEnvironmentDrive
|
||||
/** Null during cleanup after the associated run is deleted. */
|
||||
runId: string | null;
|
||||
agentId: string | null;
|
||||
projectId: string | null;
|
||||
/** Company-validated projects to prepare on submit; empty during cleanup. */
|
||||
projectIds: string[];
|
||||
operation: PluginEnvironmentTaskOperation;
|
||||
}
|
||||
|
||||
|
||||
@@ -191,7 +191,7 @@ export interface SandboxProviderCapabilities {
|
||||
}
|
||||
|
||||
export interface PluginEnvironmentDriverDeclaration {
|
||||
/** Submits tasks directly to a remote Paperclip Runner over PRP via environmentTask. */
|
||||
/** Admits tasks, prepares execution resources, and exposes Runner PRP connections via environmentTask. */
|
||||
supportsTasks?: boolean;
|
||||
/** Stable driver key, unique within the plugin. Namespaced by plugin ID at runtime. */
|
||||
driverKey: string;
|
||||
|
||||
@@ -7,26 +7,28 @@ import { executeEnvironmentTask } from "../services/environment-task-runtime.js"
|
||||
const state = vi.hoisted(() => ({ plugin: {} as any }));
|
||||
vi.mock("../services/plugin-registry.js", () => ({ pluginRegistryService: () => ({ getById: vi.fn(async () => state.plugin) }) }));
|
||||
const leaseId = "10000000-0000-4000-8000-000000000001";
|
||||
const projectId = "10000000-0000-4000-8000-000000000002";
|
||||
const secondProjectId = "10000000-0000-4000-8000-000000000003";
|
||||
const row = () => ({
|
||||
lease: { id: leaseId, companyId: "company", environmentId: "environment", providerLeaseId: "attempt-1", heartbeatRunId: "run", issueId: "issue", status: "active", expiresAt: null as Date | null,
|
||||
metadata: { driver: "plugin", pluginId: "original", driverKey: "tasks" } },
|
||||
environment: { id: "environment", config: { pluginKey: "test.provider", driverKey: "tasks", driverConfig: {} } } as { id: string; config: Record<string, unknown> } | null,
|
||||
run: { id: "run", agentId: "agent", status: "running" } as { id: string; agentId: string; status: string } | null,
|
||||
});
|
||||
function database(value: ReturnType<typeof row> | null = row()) {
|
||||
const results = [value ? [value] : [], [{ projectId: "project" }]];
|
||||
const query: any = { from: () => query, leftJoin: () => query, where: vi.fn(() => query), limit: () => Promise.resolve(results.shift()) };
|
||||
function database(value: ReturnType<typeof row> | null = row(), projectRows = [{ id: projectId }]) {
|
||||
const results = [value ? [value] : [], [{ id: "issue" }]];
|
||||
const query: any = { from: () => query, leftJoin: () => query, where: vi.fn(() => query), limit: () => Promise.resolve(results.shift()), then: (resolve: (rows: typeof projectRows) => unknown) => Promise.resolve(projectRows).then(resolve) };
|
||||
return { db: { select: () => query } as unknown as Db, query };
|
||||
}
|
||||
function worker(result: unknown = { kind: "accepted", taskId: "attempt-1" }) {
|
||||
return { getWorker: vi.fn(() => ({ supportedMethods: ["environmentTask"] })), call: vi.fn(async () => result) };
|
||||
}
|
||||
const submit = { kind: "submit" as const, runner: { protocolMin: 1, protocolMax: 2, harness: "codex", runnerId: "runner", leaseId, runId: "run", sessionId: "session", turnId: "turn", itemId: "item" }, bootstrapTicket: "transient-test-ticket" };
|
||||
const submit = { kind: "submit" as const, projectIds: [projectId], runner: { protocolMin: 1, protocolMax: 2, harness: "codex", runnerId: "runner", leaseId, runId: "run", sessionId: "session", turnId: "turn", itemId: "item" }, bootstrapTicket: "transient-test-ticket" };
|
||||
|
||||
beforeEach(() => {
|
||||
state.plugin = { id: "original", pluginKey: "test.provider", status: "ready", manifestJson: { capabilities: ["environment.drivers.register"], environmentDrivers: [{ driverKey: "tasks", supportsTasks: true }] } };
|
||||
});
|
||||
describe("remote Paperclip Runner tasks", () => {
|
||||
describe("environment task admission", () => {
|
||||
it("dispatches with host-derived scope and a persisted attempt identity", async () => {
|
||||
const { db, query } = database(); const workers = worker();
|
||||
await expect(executeEnvironmentTask(db, workers as never, { companyId: "company", leaseId, operation: submit })).resolves.toEqual({ kind: "accepted", taskId: "attempt-1" });
|
||||
@@ -35,9 +37,30 @@ describe("remote Paperclip Runner tasks", () => {
|
||||
expect(scope.sql).toContain('"environment_leases"."id" =');
|
||||
expect(scope.params).toEqual([leaseId, "company"]);
|
||||
expect(workers.call).toHaveBeenCalledWith("original", "environmentTask", expect.objectContaining({
|
||||
taskId: "attempt-1", companyId: "company", agentId: "agent", projectId: "project", runId: "run", operation: submit,
|
||||
taskId: "attempt-1", companyId: "company", agentId: "agent", projectIds: [projectId], runId: "run", operation: submit,
|
||||
}), 15_000);
|
||||
});
|
||||
it("validates every requested project in the task company before dispatch", async () => {
|
||||
const operation = { ...submit, projectIds: [projectId, secondProjectId] };
|
||||
const { db, query } = database(row(), [{ id: projectId }, { id: secondProjectId }]);
|
||||
const workers = worker();
|
||||
await executeEnvironmentTask(db, workers as never, { companyId: "company", leaseId, operation });
|
||||
expect(workers.call).toHaveBeenCalledWith("original", "environmentTask", expect.objectContaining({ projectIds: operation.projectIds }), 15_000);
|
||||
const scope = new PgDialect().sqlToQuery(query.where.mock.calls[2][0]);
|
||||
expect(scope.sql).toContain('"projects"."company_id" =');
|
||||
expect(scope.params).toEqual(["company", projectId, secondProjectId]);
|
||||
const rejected = worker();
|
||||
await expect(executeEnvironmentTask(database(row(), [{ id: projectId }]).db, rejected as never, { companyId: "company", leaseId, operation })).rejects.toThrow("project unavailable");
|
||||
expect(rejected.call).not.toHaveBeenCalled();
|
||||
});
|
||||
it("validates unique project IDs and allows tasks with no project mounts", async () => {
|
||||
for (const projectIds of [[projectId, projectId], ["invalid"], Array(65).fill(projectId)]) {
|
||||
expect(environmentTaskOperationSchema.safeParse({ ...submit, projectIds }).success).toBe(false);
|
||||
}
|
||||
const workers = worker();
|
||||
await executeEnvironmentTask(database().db, workers as never, { companyId: "company", leaseId, operation: { ...submit, projectIds: [] } });
|
||||
expect(workers.call).toHaveBeenCalledWith("original", "environmentTask", expect.objectContaining({ projectIds: [] }), 15_000);
|
||||
});
|
||||
it("rejects an absent or cross-company lease before calling a worker", async () => {
|
||||
const workers = worker();
|
||||
await expect(executeEnvironmentTask(database(null).db, workers as never, { companyId: "other", leaseId, operation: submit })).rejects.toThrow("lease unavailable");
|
||||
@@ -62,7 +85,7 @@ describe("remote Paperclip Runner tasks", () => {
|
||||
const workers = worker(kind === "status" ? { kind: "status", taskId: "attempt-1", phase: "cancelled" } : undefined);
|
||||
await executeEnvironmentTask(database(value).db, workers as never, { companyId: "company", leaseId, operation: { kind } });
|
||||
expect(workers.call).toHaveBeenCalledWith("original", "environmentTask", expect.objectContaining({
|
||||
config: {}, environmentId: null, runId: null, agentId: null, projectId: null,
|
||||
config: {}, environmentId: null, runId: null, agentId: null, projectIds: [],
|
||||
}), 15_000);
|
||||
}
|
||||
await expect(executeEnvironmentTask(database(value).db, worker() as never, { companyId: "company", leaseId, operation: submit })).rejects.toThrow("not active");
|
||||
|
||||
@@ -94,7 +94,7 @@ describe("plugin environment driver seam", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("negotiates and validates remote Runner task RPCs", async () => {
|
||||
it("negotiates and validates environment task RPCs", async () => {
|
||||
const calls: unknown[] = [];
|
||||
const plugin = definePlugin({
|
||||
async setup() {},
|
||||
|
||||
@@ -3872,7 +3872,7 @@ export function environmentRuntimeService(
|
||||
return resolveSandboxDuplexBridgeInput(experimental);
|
||||
},
|
||||
|
||||
/** Submit and manage tasks on a remote Paperclip Runner over PRP. */
|
||||
/** Admit and manage provider tasks, including resource preparation and Runner connections. */
|
||||
async task(input: { companyId: string; leaseId: string; operation: PluginEnvironmentTaskOperation }) {
|
||||
if (!options.pluginWorkerManager) throw new Error("Environment task worker manager unavailable");
|
||||
return executeEnvironmentTask(db, options.pluginWorkerManager, input);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { and, eq } from "drizzle-orm";
|
||||
import { environmentLeases, environments, heartbeatRuns, issues, type Db } from "@paperclipai/db";
|
||||
import { and, eq, inArray } from "drizzle-orm";
|
||||
import { environmentLeases, environments, heartbeatRuns, issues, projects, type Db } from "@paperclipai/db";
|
||||
import { environmentTaskOperationSchema, parseEnvironmentTaskResult, type PluginEnvironmentTaskOperation } from "@paperclipai/plugin-sdk";
|
||||
import { pluginRegistryService } from "./plugin-registry.js";
|
||||
import type { PluginWorkerManager } from "./plugin-worker-manager.js";
|
||||
@@ -39,9 +39,15 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
|
||||
!workers.getWorker(plugin.id)?.supportedMethods.includes("environmentTask")) {
|
||||
throw new Error("Environment task provider unavailable");
|
||||
}
|
||||
const project = operation.kind === "submit" && lease.issueId ? await db.select({ projectId: issues.projectId }).from(issues)
|
||||
const issue = operation.kind === "submit" && lease.issueId ? await db.select({ id: issues.id }).from(issues)
|
||||
.where(and(eq(issues.id, lease.issueId), eq(issues.companyId, input.companyId))).limit(1).then(rows => rows[0]) : null;
|
||||
if (operation.kind === "submit" && lease.issueId && !project) throw new Error("Environment task issue unavailable");
|
||||
if (operation.kind === "submit" && lease.issueId && !issue) throw new Error("Environment task issue unavailable");
|
||||
const projectIds = operation.kind === "submit" ? operation.projectIds : [];
|
||||
if (projectIds.length > 0) {
|
||||
const authorizedProjects = await db.select({ id: projects.id }).from(projects)
|
||||
.where(and(eq(projects.companyId, input.companyId), inArray(projects.id, projectIds)));
|
||||
if (authorizedProjects.length !== projectIds.length) throw new Error("Environment task project unavailable");
|
||||
}
|
||||
// Provider identity comes from the lease. Editing the environment must never
|
||||
// redirect status or cleanup to a replacement plugin.
|
||||
const config = (environment?.config ?? {}) as Record<string, unknown>;
|
||||
@@ -51,7 +57,7 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
|
||||
const result = await workers.call(plugin.id, "environmentTask", {
|
||||
driverKey: metadata.driverKey, companyId: input.companyId, environmentId: environment?.id ?? null,
|
||||
issueId: lease.issueId, config: driverConfig,
|
||||
taskId, runId: run?.id ?? null, agentId: run?.agentId ?? null, projectId: project?.projectId ?? null,
|
||||
taskId, runId: run?.id ?? null, agentId: run?.agentId ?? null, projectIds,
|
||||
lease: { providerLeaseId: lease.providerLeaseId, metadata: lease.metadata ?? undefined }, operation,
|
||||
}, 15_000);
|
||||
return parseEnvironmentTaskResult(operation, taskId, result);
|
||||
|
||||
Reference in new issue
Block a user