From dc6dd23ea4be802edcf15ff471987c2f4b6e61b8 Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Tue, 20 Oct 2026 11:41:49 -0700 Subject: [PATCH] Add plugin Runner preparation task contract (#16993) ## Thinking Path > - Paperclip manages work for AI agents. > - Environment plugins can own Runner tasks. > - Core selects provider policy and runtime assignments. > - The execution environment must resolve its own paths and configuration. > - The task API needs a preparation exchange before execution. > - This PR adds that exchange under the existing lease authority. ## Linked Issues or Issue Description **Problem or motivation** The environment task API can start a Runner. It cannot return a prepared provider configuration from Core requirements. **Proposed solution** Add a versioned `prepare` request and `prepared` result. Keep the provider configuration bound to its task and requirements. Check the same company, run, project, and retained-session authority as submission. **Alternatives considered** A shell preparation command would expose execution details in the API. Provider-specific ownership flags would spread this contract across backends. **Roadmap alignment** This extends the existing plugin and remote execution APIs. It does not add a separate execution system. Refs #16977, #15406. The prerequisite #16977 is merged. This PR now targets master. Runner transport and provider implementations follow separately. ## What Changed - Add strict launch requirements and prepared result schemas. - Add the preparation capability version and saved configuration ID. - Apply existing lease and resource checks to preparation. - Document idempotency, cancellation, recovery, and cleanup requirements. ## Verification - `pnpm -r typecheck` passed. - Plugin SDK build passed. - All 55 Dot Runner and native Codex server integration tests passed with Cargo available on PATH. - All 18 environment task runtime tests passed. These include preparation dispatch, inactive leases, wrong run identity, unauthorized projects, result identity, and unknown command rejection. - The broader root suite on the parent refactor finished with 22,420 passing tests, seven failed tests in unchanged Git and diagnostic code, and two suite setup errors because Cargo was absent from its PATH. This PR does not claim a passing full root suite. - No live provider calls or deployment were run. - Standard Greptile and Apex each scored commit `d3ac15906240b7066f5ff505ed2ffd9ef5dd43c1` 5/5. There are no unresolved review threads. - All 55 CI checks passed. Two optional checks were skipped. The branch is ready and mergeable against master. - A fresh master fetch and merge-base check passed immediately before the Apex request. ## Risks - This defines a new optional worker operation. Existing providers keep their current operations. - A provider must validate policy and resource bindings. Schema validation alone does not validate a concrete provider descriptor. - This PR defines the exchange. It does not connect native execution to a preparation provider. ## Model Used - OpenAI Codex, GPT-6. Reasoning and repository tools. The exact serving model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip --- doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md | 32 ++++++++++++++++- packages/plugins/sdk/src/environment-tasks.ts | 35 ++++++++++++++++++- packages/plugins/sdk/src/index.ts | 4 ++- .../environment-task-runtime.test.ts | 31 ++++++++++++++++ .../src/services/environment-task-runtime.ts | 29 +++++++++------ 5 files changed, 117 insertions(+), 14 deletions(-) diff --git a/doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md b/doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md index efbd7b3aa4a..b646c518180 100644 --- a/doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md +++ b/doc/plugins/PLUGIN_ENVIRONMENT_TASKS.md @@ -32,6 +32,7 @@ See [the schemas](../../packages/plugins/sdk/src/environment-tasks.ts) for all f | Operation | Result | Required behavior | | --- | --- | --- | | `describe` | `description` | Return the installed Runner version and SHA-256 digest. | +| `prepare` | `prepared` | Prepare the provider configuration for the exact Core requirements. Do not start a process. | | `submit` | `accepted` | Start the same opaque task at most once. Use the structured launch inputs. | | `connection` | `connection` | Return an authenticated WSS endpoint and its generation. | | `status` | `status` | Report the phase and observed process termination. | @@ -48,13 +49,42 @@ authorized project IDs, and optional launch limits. It can also contain a saved task ID for session reuse and a typed Codex or Claude credential. Core does not supply an executable command or a guest path. +## Launch preparation + +A provider can return `launchPreparationVersion: 1` from `describe`. +Core can then send the `prepare` operation before `submit`. + +The requirements contain the provider, model, policy, run identity, instructions, +and runtime context. They do not contain provider credentials or executable +choices. Resource paths identify source locations. The plugin must resolve them +for its execution environment. + +The result contains a configuration ID, requirements digest, and provider +descriptor. The schema is `paperclip.runner.launch-requirements.v1`. +The digest is SHA-256 over the requirements JSON. Sort object keys in lexical +order at every level. Keep array order. Omit absent optional fields. +The execution owner must preserve Core policy and resource assignments. + +Preparation must be idempotent for the task and requirements. The plugin owns +configuration storage and cleanup. A late preparation result must not authorize +execution after cancellation. The `submit` operation can reference the saved +configuration with `preparedLaunchId`. + +Recovery can supply the prior prepared result. Recovery within the same run +must use the same configuration. A different run supplies its current +requirements. A saved task ID in `resumeFrom` requires the same retained-session +authority as submission. + +This API defines the preparation exchange. Runner transport integration is a +separate change. + ## Lease authority Core calls `environmentRuntimeService.task` with a company ID, a saved lease ID, and an operation. It derives the plugin and driver from the lease. A change to the environment configuration cannot redirect cleanup to a new provider. -Before submission, Core checks the active lease, run, issue, and every project +Before preparation or submission, Core checks the active lease, run, issue, and every project in the company. It checks the submitting run ID separately from a retained PRP identity. Session reuse requires the same agent, environment, workspace, and provider. It also requires a retained source task and proof of termination. diff --git a/packages/plugins/sdk/src/environment-tasks.ts b/packages/plugins/sdk/src/environment-tasks.ts index ae2cd9e7eed..350b36fbb5f 100644 --- a/packages/plugins/sdk/src/environment-tasks.ts +++ b/packages/plugins/sdk/src/environment-tasks.ts @@ -21,9 +21,36 @@ export const environmentTaskCredentialSchema = z.discriminatedUnion("type", [ ]); export type PluginEnvironmentTaskCredential = z.infer; +/** Runner owns this versioned data contract. The plugin resolves its execution-host locations. */ +export const environmentRunnerLaunchRequirementsSchema = z.object({ + schema: z.literal("paperclip.runner.launch-requirements.v1"), + identity: z.object({ runnerInstanceId: runnerIdentity, environmentLeaseId: runnerIdentity, + runId: runnerIdentity, normalizedSessionId: runnerIdentity, turnId: runnerIdentity, itemId: runnerIdentity }).strict(), + provider: z.enum(["codex", "opencode", "acpx", "claude_managed", "aws_agentcore"]), + model: z.string().min(1).nullable(), instructions: z.string().max(1024 * 1024), + runtimeContext: z.record(z.string(), z.unknown()).nullable(), + settings: z.object({ acpxAgent: identifier.optional(), acpxMode: z.string().optional(), + acpxPermissionMode: z.enum(["approve-all", "approve-paperclip", "approve-reads", "deny-all"]).optional(), + acpxPermissionModePinned: z.boolean().optional(), piThinkingLevel: z.enum(["off", "low", "high", "max"]).optional(), + claudeEffort: z.enum(["low", "medium", "high", "xhigh", "max"]).optional(), + opencodePermissionMode: z.enum(["allow", "ask", "deny"]).optional(), + managedProfile: z.record(z.string(), z.unknown()).optional(), agentCoreProfile: z.record(z.string(), z.unknown()).optional(), + }).strict(), + approvalPolicy: z.enum(["never", "on-request", "untrusted"]), readOnly: z.boolean(), + preparedConversation: z.boolean(), includeCollaborationModeInstructions: z.boolean(), +}).strict(); +const preparedLaunch = z.object({ configurationId: providerId.max(1024), + requirementsDigest: z.string().regex(/^sha256:[a-f0-9]{64}$/), + provider: z.record(z.string(), z.unknown()), +}).strict(); + /** One plugin-provided Runner execution attempt with a PRP connection. Secrets are transient RPC input. */ export const environmentTaskOperationSchema = z.discriminatedUnion("kind", [ z.object({ kind: z.literal("describe") }).strict(), + z.object({ kind: z.literal("prepare"), requirements: environmentRunnerLaunchRequirementsSchema, + recovery: preparedLaunch.optional(), resumeFrom: providerId.optional(), + projectIds: z.array(z.string().uuid()).max(64).refine(ids => new Set(ids).size === ids.length, "Duplicate project IDs"), + }).strict(), z.object({ kind: z.literal("submit"), projectIds: z.array(z.string().uuid()).max(64).refine(ids => new Set(ids).size === ids.length, "Duplicate project IDs"), @@ -38,6 +65,7 @@ export const environmentTaskOperationSchema = z.discriminatedUnion("kind", [ bootstrapTicket: z.string().min(1).max(65_536), /** Opaque prior provider task; Core authorizes the saved session before supplying it. */ resumeFrom: z.string().min(1).optional(), + preparedLaunchId: providerId.max(1024).optional(), /** Require the externally installed account to match this Core-selected credential revision. */ credentialIdentity: z.string().min(1).max(1024).optional(), /** Stable account selection; distinct from its current credential revision. */ @@ -61,10 +89,12 @@ export const environmentTaskOperationSchema = z.discriminatedUnion("kind", [ export const environmentTaskResultSchema = z.discriminatedUnion("kind", [ z.object({ kind: z.literal("accepted"), taskId: providerId }).strict(), + z.object({ kind: z.literal("prepared"), taskId: providerId, launch: preparedLaunch }).strict(), z.object({ kind: z.literal("credentials"), taskId: providerId, /** Null proves that cancellation fenced admission before the task could start. It must not mean an unavailable export. */ credential: environmentTaskCredentialSchema.nullable() }).strict(), z.object({ kind: z.literal("description"), taskId: providerId, runner: z.object({ version: z.string().min(1), digest: z.string().regex(/^sha256:[a-f0-9]{64}$/) }).strict(), + launchPreparationVersion: z.literal(1).optional(), }).strict(), z.object({ kind: z.literal("status"), taskId: providerId, @@ -88,6 +118,9 @@ export const environmentTaskResultSchema = z.discriminatedUnion("kind", [ }).strict(), ]); +export type PluginEnvironmentRunnerLaunchRequirements = z.infer; +export type PluginEnvironmentPreparedLaunch = z.infer; + export type PluginEnvironmentTaskOperation = z.infer; export type PluginEnvironmentTaskResult = z.infer; @@ -108,7 +141,7 @@ export interface PluginEnvironmentTaskParams extends Omit { expect(environmentTaskOperationSchema.safeParse({ kind }).success).toBe(false); } }); + it("prepares configuration under the same active company, run, and project authority", async () => { + const operation = { kind: "prepare" as const, projectIds: [projectId], requirements: { + schema: "paperclip.runner.launch-requirements.v1" as const, + identity: { runnerInstanceId: "runner", environmentLeaseId: leaseId, runId: "run", + normalizedSessionId: "session", turnId: "turn", itemId: "item" }, + provider: "codex" as const, model: "selected", instructions: "Core instructions", runtimeContext: null, + settings: {}, approvalPolicy: "on-request" as const, readOnly: false, + preparedConversation: false, includeCollaborationModeInstructions: true, + } }; + const result = { kind: "prepared", taskId: "attempt-1", launch: { configurationId: "configuration-1", + requirementsDigest: `sha256:${"a".repeat(64)}`, provider: { kind: "codex" } } }; + const workers = worker(result); + await expect(executeEnvironmentTask(database().db, workers as never, { companyId: "company", leaseId, operation })) + .resolves.toEqual(result); + expect(workers.call).toHaveBeenCalledWith("original", "environmentTask", expect.objectContaining({ + companyId: "company", runId: "run", agentId: "agent", projectIds: [projectId], operation, + }), 15_000); + const denied = worker(result); + const inactive = row(); inactive.lease.status = "released"; + for (const value of [null, inactive]) { + await expect(executeEnvironmentTask(database(value).db, denied as never, { companyId: "company", leaseId, operation })).rejects.toThrow(); + } + await expect(executeEnvironmentTask(database().db, denied as never, { companyId: "company", leaseId, + operation: { ...operation, requirements: { ...operation.requirements, identity: { ...operation.requirements.identity, runId: "other" } } } })) + .rejects.toThrow("identity mismatch"); + await expect(executeEnvironmentTask(database(row(), []).db, denied as never, { companyId: "company", leaseId, operation })) + .rejects.toThrow("project unavailable"); + expect(denied.call).not.toHaveBeenCalled(); + expect(() => parseEnvironmentTaskResult(operation, "attempt-1", { ...result, taskId: "other" })).toThrow(); + expect(environmentTaskOperationSchema.safeParse({ ...operation, command: "sh" }).success).toBe(false); + }); it("allows only the private, unexpired setup identity on an ad-hoc lease", async () => { const value = row(); Object.assign(value.lease, { heartbeatRunId: null, issueId: null }); value.run = null; diff --git a/server/src/services/environment-task-runtime.ts b/server/src/services/environment-task-runtime.ts index a8b080a37b5..4f46bd0afe0 100644 --- a/server/src/services/environment-task-runtime.ts +++ b/server/src/services/environment-task-runtime.ts @@ -41,22 +41,29 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage const activeProbe = lease.heartbeatRunId === null && lease.issueId === null && probe && typeof probe.runId === "string" && typeof probe.sessionId === "string" && typeof probe.expiresAtMs === "number" && probe.expiresAtMs > Date.now(); - if ((operation.kind === "describe" || operation.kind === "submit" || operation.kind === "connection") && + if ((operation.kind === "describe" || operation.kind === "prepare" || operation.kind === "submit" || operation.kind === "connection") && (metadata.runnerTaskRetiring === true || !environment || (!activeProbe && run?.status !== "running") || lease.status !== "active" || (lease.expiresAt && lease.expiresAt.getTime() <= Date.now()))) throw new Error("Plugin-provided Runner execution lease is not active"); - if (operation.kind === "submit" && (operation.runner.runId !== (activeProbe ? probe.runId : run?.id) || (!operation.resumeFrom && operation.runner.leaseId !== lease.id) - || (activeProbe && (operation.runner.sessionId !== probe.sessionId || operation.projectIds.length > 0)))) { + const runner = operation.kind === "submit" ? operation.runner : operation.kind === "prepare" ? { + runnerId: operation.requirements.identity.runnerInstanceId, + leaseId: operation.requirements.identity.environmentLeaseId, + runId: operation.requirements.identity.runId, sessionId: operation.requirements.identity.normalizedSessionId, + } : null; + const launches = operation.kind === "submit" || operation.kind === "prepare"; + const resumeFrom = launches ? operation.resumeFrom : undefined; + if (runner && (runner.runId !== (activeProbe ? probe.runId : run?.id) || (!resumeFrom && runner.leaseId !== lease.id) + || (activeProbe && (runner.sessionId !== probe.sessionId || (launches && operation.projectIds.length > 0))))) { throw new Error("Plugin-provided Runner execution identity mismatch"); } - if (operation.kind === "submit" && operation.resumeFrom) { + if (runner && resumeFrom) { const sources = await db.select({ runId: environmentLeases.heartbeatRunId }).from(environmentLeases).where(and( eq(environmentLeases.companyId, input.companyId), eq(environmentLeases.provider, lease.provider!), - eq(environmentLeases.providerLeaseId, operation.resumeFrom), + eq(environmentLeases.providerLeaseId, resumeFrom), )).limit(2); const priorRunId = sources.length === 1 ? sources[0]!.runId : null; - if (activeProbe || !priorRunId || operation.resumeFrom !== await resolvePluginTaskResume(db, { + if (activeProbe || !priorRunId || resumeFrom !== await resolvePluginTaskResume(db, { companyId: input.companyId, leaseId: lease.id, - priorIdentity: { runnerInstanceId: operation.runner.runnerId, environmentLeaseId: operation.runner.leaseId, - runId: priorRunId, normalizedSessionId: operation.runner.sessionId }, + priorIdentity: { runnerInstanceId: runner.runnerId, environmentLeaseId: runner.leaseId, + runId: priorRunId, normalizedSessionId: runner.sessionId }, })) throw new Error("Plugin Runner resume identity mismatch"); } if (metadata.driver !== "plugin" || typeof metadata.pluginId !== "string" || typeof metadata.driverKey !== "string") { @@ -69,10 +76,10 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage !workers.getWorker(plugin.id)?.supportedMethods.includes("environmentTask")) { throw new Error("Plugin-provided Runner execution provider unavailable"); } - const issue = operation.kind === "submit" && lease.issueId ? await db.select({ id: issues.id }).from(issues) + const issue = launches && lease.issueId ? await db.select({ id: issues.id }).from(issues) .where(and(eq(issues.id, lease.issueId), eq(issues.companyId, input.companyId))).limit(1).then(rows => rows[0]) : null; - if (operation.kind === "submit" && lease.issueId && !issue) throw new Error("Plugin-provided Runner execution issue unavailable"); - const projectIds = operation.kind === "submit" ? operation.projectIds : []; + if (launches && lease.issueId && !issue) throw new Error("Plugin-provided Runner execution issue unavailable"); + const projectIds = launches ? operation.projectIds : []; if (projectIds.length > 0) { const authorizedProjects = await db.select({ id: projects.id }).from(projects) .where(and(eq(projects.companyId, input.companyId), inArray(projects.id, projectIds)));