mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 09:07:46 +02:00
Read native plugin task output files on the execution host (#17285)
This commit is contained in:
1 parent
d69631b01b
commit
412914cfa7
15 files changed
+202
-24
No files matched your search
@@ -1,7 +1,8 @@
|
||||
# Environment task API
|
||||
|
||||
An environment driver can provide a Runner task through a worker hook.
|
||||
The task API has no shell command or file transfer operation.
|
||||
The task API has no shell command. Its optional file-read operation is scoped
|
||||
to the active task workspace.
|
||||
|
||||
This API is a server and worker contract. The [native execution integration](PLUGIN_RUNNER_EXECUTION.md)
|
||||
connects a task-capable environment to a Runner run.
|
||||
@@ -38,6 +39,7 @@ See [the schemas](../../packages/plugins/sdk/src/environment-tasks.ts) for all f
|
||||
| `connection` | `connection` | Return an authenticated WSS endpoint and its generation. |
|
||||
| `status` | `status` | Report the phase and observed process termination. |
|
||||
| `check_project_auth` | `project_auth` | Validate the prepared project configuration before managed credential transfer. |
|
||||
| `read_file` | `file` | Read one bounded, task-scoped file from the active task. |
|
||||
| `read_state` | `state` | Read the exact retained Runner checkpoint. |
|
||||
| `archive_state` | `state` | Archive suspended authority with an idempotent archive key. Keep provider session files. |
|
||||
| `credentials` | `credentials` | Return the selected account after process termination. |
|
||||
@@ -217,3 +219,23 @@ for fresh attempts; ordinary preparation and reuse retain their existing contrac
|
||||
Core keeps transfer files while preparation is uncertain. Confirmed task completion
|
||||
and provider workspace removal release these disposable copies. The descriptors
|
||||
remain stable: a later preparation downloads the same verified files again.
|
||||
|
||||
|
||||
## Task workspace files
|
||||
|
||||
Core uses `read_file` for `call_api` file uploads and `register_deliverable`
|
||||
when a plugin owns execution and advertises `workspaceFileReadVersion: 1` in
|
||||
`describe`. The request includes the submitted Runner identity,
|
||||
validated project IDs, a canonical relative or absolute path, and a byte limit (at most 10 MiB).
|
||||
The provider resolves paths within locations granted to the active task. Relative
|
||||
paths resolve from its prepared workspace. Absolute paths can select another
|
||||
granted location such as agent home, but never a supervisor path or another
|
||||
agent's files. Core does not prescribe provider filesystem layout.
|
||||
|
||||
The provider must reject inactive tasks, path escapes, symlinks, hard links,
|
||||
non-regular files, protected credentials, oversized files, and files that change
|
||||
while being read. Return canonical base64, byte size, and a SHA-256 digest in a
|
||||
`file` result for the same task. Core validates the result and rechecks authority
|
||||
before publishing it. Deliverables additionally verify the caller's expected
|
||||
size and digest. Providers without this operation fail closed; Core never falls
|
||||
back to reading remote paths on the server. Existing artifact uploads are unchanged.
|
||||
@@ -16,8 +16,10 @@ Implement these hooks:
|
||||
- `onEnvironmentDestroyLease`
|
||||
|
||||
Task execution does not require `onEnvironmentExecute` or
|
||||
`onEnvironmentRealizeWorkspace`. The task API has no shell or file transfer
|
||||
operation. Use a separate workflow to install and save agent files.
|
||||
`onEnvironmentRealizeWorkspace`. The task API has no shell command or file installation
|
||||
operation. Use a separate workflow to install agent files. Providers that
|
||||
advertise `workspaceFileReadVersion: 1` can return task output files through the
|
||||
bounded `read_file` operation for attachments and deliverables.
|
||||
|
||||
Core supports per-turn workspace sessions with Codex, OpenCode, and ACPX.
|
||||
OpenCode and ACPX require the existing qualified provider profiles.
|
||||
|
||||
@@ -131,7 +131,7 @@ export function nativeTaskConstraints(input: NativeExecutionInput): string[] {
|
||||
"Use Paperclip semantic tools for coordination and finalization.",
|
||||
"When available, use submit_complaint for the raw reaction to agent-work friction or submit_suggestion for a concrete improvement. Submit proactively when warranted, briefly and in your own voice; this is not a mandatory report. Feedback is internal and attributed. Exclude secrets, private prompts, customer data, and personal blame. Never submit the same incident through both tools; aim for at most three suggestions per run. Submit silently once, before finishing, then immediately continue the primary task even if submission fails. Answer truthfully if the user asks about feedback or what you submitted.",
|
||||
"Save requested plans and Paperclip documents directly with write_document. A saved Paperclip document is already a durable deliverable. Do not create a local file, compute file hashes, or call register_deliverable for it unless the user also requests a downloadable file. Cite the saved document in your completion evidence and include its returned documentHref as a clickable link in your final response.",
|
||||
"When the requested result is a file, use register_deliverable before paperclip_finish. Compute its exact byte size and SHA-256, register the workspace-relative file, cite deliverable:<attachmentId> from the receipt as completion evidence, and include /api/attachments/<attachmentId>/content as the download link in your answer. A bare workspace filename is not a delivered result. For repository edits, cite an accessible PR or registered work product. Preserve existing work; do not upload unrelated files. If file publication fails, fix it or report the concrete blocker instead of claiming the file is delivered.",
|
||||
"When the requested result is a file, use register_deliverable before paperclip_finish. Compute its exact byte size and SHA-256, register the file using a path allowed by the tool, cite deliverable:<attachmentId> from the receipt as completion evidence, and include /api/attachments/<attachmentId>/content as the download link in your answer. A bare workspace filename is not a delivered result. For repository edits, cite an accessible PR or registered work product. Preserve existing work; do not upload unrelated files. If file publication fails, fix it or report the concrete blocker instead of claiming the file is delivered.",
|
||||
...(answeredQuestionConstraint ? [answeredQuestionConstraint] : []),
|
||||
finalResponseConstraint,
|
||||
];
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
import { z } from "zod";
|
||||
import type { PluginEnvironmentDriverBaseParams, PluginEnvironmentLease } from "./protocol.js";
|
||||
|
||||
export const MAX_ENVIRONMENT_TASK_FILE_BYTES = 10 * 1024 * 1024;
|
||||
const taskFilePath = z.string().min(1).max(2000).refine(value =>
|
||||
!/[\\\u0000-\u001f\u007f]/u.test(value) && !/^[a-z][a-z0-9+.-]*:/iu.test(value) &&
|
||||
value.replace(/^\//u, "").split("/").every(part => part !== "" && part !== "." && part !== ".."), "Expected a canonical task file path");
|
||||
// Provider IDs are opaque; providers validate their own addressing constraints.
|
||||
const providerId = z.string().min(1);
|
||||
// PRP identity.schema.json stableId.
|
||||
@@ -85,6 +89,10 @@ export const environmentTaskOperationSchema = z.discriminatedUnion("kind", [
|
||||
z.object({ kind: z.literal("check_project_auth"), configurationId: providerId.max(1024),
|
||||
provider: identifier, projectIds: z.array(z.string().uuid()).max(64)
|
||||
.refine(ids => new Set(ids).size === ids.length, "Duplicate project IDs") }).strict(),
|
||||
z.object({ kind: z.literal("read_file"), identity: durableIdentity,
|
||||
path: taskFilePath, maxBytes: z.number().int().positive().max(MAX_ENVIRONMENT_TASK_FILE_BYTES),
|
||||
projectIds: z.array(z.string().uuid()).max(64).refine(ids => new Set(ids).size === ids.length, "Duplicate project IDs"),
|
||||
}).strict(),
|
||||
z.object({ kind: z.literal("read_state"), identity: durableIdentity, resumeFrom: providerId.optional() }).strict(),
|
||||
z.object({ kind: z.literal("archive_state"), priorIdentity: durableIdentity,
|
||||
archiveKey: z.string().regex(/^[a-f0-9]{24}$/), resumeFrom: providerId }).strict(),
|
||||
@@ -101,6 +109,11 @@ export const environmentTaskResultSchema = z.discriminatedUnion("kind", [
|
||||
executionStopped: z.literal(true) }).strict(),
|
||||
z.object({ kind: z.literal("project_auth"), taskId: providerId, configurationId: providerId.max(1024),
|
||||
provider: identifier, allowed: z.boolean() }).strict(),
|
||||
z.object({ kind: z.literal("file"), taskId: providerId,
|
||||
base64: z.string().max(4 * Math.ceil(MAX_ENVIRONMENT_TASK_FILE_BYTES / 3)),
|
||||
byteSize: z.number().int().nonnegative().max(MAX_ENVIRONMENT_TASK_FILE_BYTES),
|
||||
sha256: z.string().regex(/^[a-f0-9]{64}$/),
|
||||
}).strict(),
|
||||
z.object({ kind: z.literal("state"), taskId: providerId, state: z.record(z.string(), z.unknown()) }).strict(),
|
||||
z.object({ kind: z.literal("prepared"), taskId: providerId, launch: preparedLaunch }).strict(),
|
||||
z.object({ kind: z.literal("credentials"), taskId: providerId, /** Null proves that cancellation fenced admission before the task could start. It must not mean an unavailable export. */
|
||||
@@ -110,6 +123,7 @@ export const environmentTaskResultSchema = z.discriminatedUnion("kind", [
|
||||
launchPreparationVersion: z.literal(1).optional(),
|
||||
freshWorkspaceVersion: z.literal(1).optional(),
|
||||
projectAuthCheckVersion: z.literal(1).optional(),
|
||||
workspaceFileReadVersion: z.literal(1).optional(),
|
||||
}).strict(),
|
||||
z.object({
|
||||
kind: z.literal("status"), taskId: providerId,
|
||||
@@ -158,7 +172,7 @@ export interface PluginEnvironmentTaskParams extends Omit<PluginEnvironmentDrive
|
||||
/** An accepted stop is not proof of process termination. */
|
||||
export function parseEnvironmentTaskResult(operation: PluginEnvironmentTaskOperation, taskId: string, value: unknown): PluginEnvironmentTaskResult {
|
||||
const result = environmentTaskResultSchema.parse(value);
|
||||
const expected = operation.kind === "describe" ? "description" : operation.kind === "prepare" ? "prepared" : operation.kind === "check_project_auth" ? "project_auth" : ["read_state", "archive_state"].includes(operation.kind) ? "state" : ["status", "connection", "credentials", "copilot_metadata"].includes(operation.kind) ? operation.kind : "accepted";
|
||||
const expected = operation.kind === "read_file" ? "file" : operation.kind === "describe" ? "description" : operation.kind === "prepare" ? "prepared" : operation.kind === "check_project_auth" ? "project_auth" : ["read_state", "archive_state"].includes(operation.kind) ? "state" : ["status", "connection", "credentials", "copilot_metadata"].includes(operation.kind) ? operation.kind : "accepted";
|
||||
if (result.taskId !== taskId || result.kind !== expected) throw new Error("Invalid plugin-provided Runner execution response");
|
||||
return result;
|
||||
}
|
||||
@@ -463,7 +463,7 @@ export type { PluginEnvironmentSyncErrorDiagnostic } from "./environment-sync-er
|
||||
|
||||
export type { AiConnectionPool, AiConnectionPoolConfig, AiConnectionPoolMember, AiConnectionRouterRequest, AiConnectionRouterResult, AiConnectionRouterSelection } from "@paperclipai/shared";
|
||||
|
||||
export { environmentRunnerLaunchRequirementsSchema, environmentTaskOperationSchema, environmentTaskResultSchema, environmentTaskCredentialSchema, parseEnvironmentTaskResult } from "./environment-tasks.js";
|
||||
export { MAX_ENVIRONMENT_TASK_FILE_BYTES, environmentRunnerLaunchRequirementsSchema, environmentTaskOperationSchema, environmentTaskResultSchema, environmentTaskCredentialSchema, parseEnvironmentTaskResult } from "./environment-tasks.js";
|
||||
export type { PluginEnvironmentTaskOperation, PluginEnvironmentTaskParams, PluginEnvironmentTaskResult, PluginEnvironmentTaskCredential } from "./environment-tasks.js";
|
||||
export type { AgentLifecycleRequest, AgentLifecycleResult } from "@paperclipai/shared";
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ const row = () => ({
|
||||
});
|
||||
function database(value: ReturnType<typeof row> | null = row(), projectRows = [{ id: projectId }], workspaceRows: unknown[] = []) {
|
||||
let selectedRows: unknown[] = projectRows;
|
||||
const results = [value ? [value] : [], [{ id: "issue" }]];
|
||||
const results = [value ? [value] : [], [{ id: "issue" }], value ? [value] : []];
|
||||
const query: any = { from: (table: unknown) => { selectedRows = table === executionWorkspaces ? workspaceRows : projectRows; return query; }, leftJoin: () => query, where: vi.fn(() => query), limit: () => Promise.resolve(results.shift()), then: (resolve: (rows: typeof projectRows) => unknown) => Promise.resolve(selectedRows).then(resolve as (rows: unknown[]) => unknown) };
|
||||
const sets: Record<string, unknown>[] = [];
|
||||
const update: any = { set: (value: Record<string, unknown>) => { sets.push(value); return update; }, where: () => update,
|
||||
@@ -80,7 +80,7 @@ describe("plugin-provided Runner execution", () => {
|
||||
await expect(executeEnvironmentTask(database(value, [], [workspace]).db, denied as never, { companyId: "company", leaseId, operation: { ...submit, projectIds: [] } })).rejects.toThrow("binding changed");
|
||||
expect(denied.call).not.toHaveBeenCalled();
|
||||
});
|
||||
it("accepts artifact identity and excludes filesystem transfer from the contract", () => {
|
||||
it("accepts artifact identity without arbitrary filesystem or execution settings", () => {
|
||||
const operation = environmentTaskOperationSchema.parse({ kind: "describe" });
|
||||
const description = { kind: "description", taskId: "attempt-1", runner: { version: "fixture", digest: `sha256:${"a".repeat(64)}` } };
|
||||
expect(parseEnvironmentTaskResult(operation, "attempt-1", description)).toEqual(description);
|
||||
@@ -91,6 +91,28 @@ describe("plugin-provided Runner execution", () => {
|
||||
expect(environmentTaskOperationSchema.safeParse({ kind }).success).toBe(false);
|
||||
}
|
||||
});
|
||||
it("bounds file reads to the active task identity and validated project scope", async () => {
|
||||
const value=row();
|
||||
const identity={runnerInstanceId:"runner",environmentLeaseId:leaseId,runId:"run",normalizedSessionId:"session",turnId:"turn",itemId:"item"};
|
||||
Object.assign(value.lease.metadata,{runnerTaskIdentity:{...submit.runner}});
|
||||
const operation={kind:"read_file" as const,identity,projectIds:[projectId],path:"proof.txt",maxBytes:1024};
|
||||
const result={kind:"file",taskId:"attempt-1",base64:"",byteSize:0,sha256:"a".repeat(64)};
|
||||
const workers=worker(result);
|
||||
await expect(executeEnvironmentTask(database(value).db,workers as never,{companyId:"company",leaseId,operation})).resolves.toEqual(result);
|
||||
workers.call.mockClear();
|
||||
for(const invalid of [{...operation,projectIds:[]},{...operation,identity:{...identity,itemId:"foreign"}}]) {
|
||||
await expect(executeEnvironmentTask(database(value).db,workers as never,{companyId:"company",leaseId,operation:invalid})).rejects.toThrow();
|
||||
}
|
||||
value.lease.expiresAt=new Date(0);
|
||||
await expect(executeEnvironmentTask(database(value).db,workers as never,{companyId:"company",leaseId,operation})).rejects.toThrow("not active");
|
||||
expect(workers.call).not.toHaveBeenCalled();
|
||||
value.lease.expiresAt=null;
|
||||
workers.call.mockImplementation(async()=>{value.run!.status="cancelled";return result;});
|
||||
await expect(executeEnvironmentTask(database(value).db,workers as never,{companyId:"company",leaseId,operation})).rejects.toThrow("operation unavailable");
|
||||
for(const path of ["//etc/passwd","../escape","a/../b","a\\b","https://example.test/file"]) expect(environmentTaskOperationSchema.safeParse({...operation,path}).success).toBe(false);
|
||||
expect(environmentTaskOperationSchema.safeParse({...operation,maxBytes:11*1024*1024}).success).toBe(false);
|
||||
expect(()=>parseEnvironmentTaskResult(operation,"attempt-1",{...result,taskId:"other"})).toThrow();
|
||||
});
|
||||
it("reads only the current task's saved identity and rejects inactive checkpoint operations", async () => {
|
||||
const value = row();
|
||||
const identity = { runnerInstanceId: "runner", environmentLeaseId: "logical-lease", runId: "run",
|
||||
|
||||
@@ -43,10 +43,10 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
|
||||
const activeProbe = lease.heartbeatRunId === null && lease.issueId === null && probe
|
||||
&& typeof probe.runId === "string" && typeof probe.sessionId === "string"
|
||||
&& typeof probe.expiresAtMs === "number" && probe.expiresAtMs > Date.now();
|
||||
if ((operation.kind === "describe" || operation.kind === "prepare" || operation.kind === "submit" || operation.kind === "connection" || operation.kind === "read_state" || operation.kind === "archive_state" || operation.kind === "check_project_auth" || operation.kind === "copilot_metadata") &&
|
||||
if ((operation.kind === "read_file" || operation.kind === "describe" || operation.kind === "prepare" || operation.kind === "submit" || operation.kind === "connection" || operation.kind === "read_state" || operation.kind === "archive_state" || operation.kind === "check_project_auth" || operation.kind === "copilot_metadata") &&
|
||||
(metadata.runnerTaskRetiring === true || !environment || (!activeProbe && run?.status !== "running") || lease.status !== "active" || (lease.expiresAt && lease.expiresAt.getTime() <= Date.now()))) throw new Error("Plugin-provided Runner execution lease is not active");
|
||||
if (operation.kind === "copilot_metadata" && !activeProbe) throw new Error("Copilot metadata requires a private setup lease");
|
||||
const checkpointIdentity = operation.kind === "read_state" ? operation.identity
|
||||
const checkpointIdentity = operation.kind === "read_state" || operation.kind === "read_file" ? operation.identity
|
||||
: operation.kind === "archive_state" ? operation.priorIdentity : null;
|
||||
const runner = checkpointIdentity ? { runnerId: checkpointIdentity.runnerInstanceId, leaseId: checkpointIdentity.environmentLeaseId,
|
||||
runId: checkpointIdentity.runId, sessionId: checkpointIdentity.normalizedSessionId } : operation.kind === "submit" ? operation.runner : operation.kind === "prepare" ? {
|
||||
@@ -55,7 +55,7 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
|
||||
runId: operation.requirements.identity.runId, sessionId: operation.requirements.identity.normalizedSessionId,
|
||||
} : null;
|
||||
const launches = operation.kind === "submit" || operation.kind === "prepare";
|
||||
const scopesProjects = launches || operation.kind === "check_project_auth";
|
||||
const scopesProjects = launches || operation.kind === "check_project_auth" || operation.kind === "read_file";
|
||||
const resumeFrom = launches || checkpointIdentity ? (operation as { resumeFrom?: string }).resumeFrom : undefined;
|
||||
const savedIdentity = metadata.runnerTaskIdentity as Record<string, unknown> | undefined;
|
||||
const ownCheckpoint = checkpointIdentity && !resumeFrom && savedIdentity
|
||||
@@ -101,7 +101,7 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
|
||||
eq(executionWorkspaces.id, workspaceBinding.executionWorkspaceId), eq(executionWorkspaces.companyId, input.companyId)));
|
||||
const saved = readWorkspaceProviderBinding(workspace?.metadata);
|
||||
if (!workspace || workspace.id !== lease.executionWorkspaceId || workspace.providerType !== "plugin" || workspace.providerRef !== workspaceBinding.workspaceRef ||
|
||||
(launches && !["active", "idle", "in_review"].includes(workspace.status)) ||
|
||||
(scopesProjects && !["active", "idle", "in_review"].includes(workspace.status)) ||
|
||||
workspaceBinding.pluginId !== plugin.id || !saved || JSON.stringify(saved) !== JSON.stringify(workspaceBinding)) {
|
||||
throw new Error("Managed workspace execution binding changed");
|
||||
}
|
||||
@@ -127,6 +127,12 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
|
||||
const driverConfig = config.pluginKey === plugin.pluginKey && config.driverKey === metadata.driverKey
|
||||
? (config.driverConfig as Record<string, unknown> | undefined) ?? {} : {};
|
||||
let requiredProjectAuthProof: Record<string, unknown> | undefined;
|
||||
if (operation.kind === "read_file") {
|
||||
const proof = metadata.runnerTaskProjectAuthValidation as { projectIds?: string[] } | undefined;
|
||||
if (metadata.runnerTaskExecutionStopped === true || !Array.isArray(proof?.projectIds) ||
|
||||
JSON.stringify([...proof.projectIds].sort()) !== JSON.stringify([...operation.projectIds].sort()))
|
||||
throw new Error("Plugin Runner file scope unavailable");
|
||||
}
|
||||
if (operation.kind === "submit") {
|
||||
const proof = metadata.runnerTaskProjectAuthValidation as { configurationId?: unknown; provider?: unknown; projectIds?: string[] } | undefined;
|
||||
const provider = operation.runner.harness === "codex" ? "openai" : operation.runner.harness === "claude" ? "anthropic" : operation.runner.harness;
|
||||
@@ -174,9 +180,17 @@ export async function executeEnvironmentTask(db: Db, workers: PluginWorkerManage
|
||||
taskId, runId: run?.id ?? (activeProbe ? String(probe.runId) : null), agentId: run?.agentId ?? null, projectIds,
|
||||
lease: { providerLeaseId: lease.providerLeaseId, metadata: lease.metadata ?? undefined },
|
||||
...(workspaceBinding ? { workspace: workspaceBinding } : {}),
|
||||
...(freshWorkspace && (launches || operation.kind === "check_project_auth") ? { fresh: freshWorkspace } : {}), operation,
|
||||
...(freshWorkspace && scopesProjects ? { fresh: freshWorkspace } : {}), operation,
|
||||
}, operation.kind === "copilot_metadata" ? Math.max(1, Math.min(40_000, Number(probe?.expiresAtMs) - Date.now())) : 15_000);
|
||||
const parsed = parseEnvironmentTaskResult(operation, taskId, result);
|
||||
if (operation.kind === "read_file") {
|
||||
const [current] = await db.select({ lease: environmentLeases, run: heartbeatRuns }).from(environmentLeases)
|
||||
.leftJoin(heartbeatRuns, and(eq(heartbeatRuns.id, environmentLeases.heartbeatRunId), eq(heartbeatRuns.companyId, environmentLeases.companyId)))
|
||||
.where(and(eq(environmentLeases.id, lease.id), eq(environmentLeases.companyId, input.companyId))).limit(1);
|
||||
if (!current || current.lease.status !== "active" || current.run?.status !== "running" ||
|
||||
current.lease.metadata?.runnerTaskRetiring === true || current.lease.metadata?.runnerTaskExecutionStopped === true ||
|
||||
current.lease.expiresAt && current.lease.expiresAt.getTime() <= Date.now()) throw new Error("Plugin Runner file lease expired during transfer");
|
||||
}
|
||||
if (freshWorkspace && parsed.kind === "description" && parsed.freshWorkspaceVersion !== 1)
|
||||
throw new Error("Plugin-provided fresh workspace preparation unavailable");
|
||||
if (operation.kind === "check_project_auth" && parsed.kind === "project_auth") {
|
||||
|
||||
@@ -122,6 +122,7 @@ describe("native runner file handoff", () => {
|
||||
overrides: Partial<{
|
||||
companyId: string;
|
||||
executionTargetKind: "local" | "remote";
|
||||
readRemoteTaskFile: (input: { contentRef: string; byteSize: number; sha256: string }) => Promise<Buffer>;
|
||||
readRemoteWorkspaceFile: (input: { contentRef: string; byteSize: number; sha256: string }) => Promise<Buffer>;
|
||||
}> = {},
|
||||
) {
|
||||
@@ -133,6 +134,7 @@ describe("native runner file handoff", () => {
|
||||
workspaceRoot,
|
||||
executionTargetKind: overrides.executionTargetKind ?? "local",
|
||||
readRemoteWorkspaceFile: overrides.readRemoteWorkspaceFile,
|
||||
readRemoteTaskFile: overrides.readRemoteTaskFile,
|
||||
storage: createStorageService(
|
||||
createLocalDiskStorageProvider(storageRoot),
|
||||
),
|
||||
@@ -600,6 +602,15 @@ describe("native runner file handoff", () => {
|
||||
.rejects.toThrow(/size|hash/);
|
||||
});
|
||||
|
||||
it("registers an absolute file only through a task-location-authorizing reader", async () => {
|
||||
const body=Buffer.from("Home output\n");const reader=vi.fn(async()=>body);
|
||||
const call=callFor("/home/agent/proof.txt",body,"home-output");
|
||||
await expect(authority({executionTargetKind:"remote",readRemoteWorkspaceFile:reader}).execute(call)).rejects.toThrow("path_denied");
|
||||
expect(reader).not.toHaveBeenCalled();
|
||||
await expect(authority({executionTargetKind:"remote",readRemoteTaskFile:reader}).execute(call)).resolves.toBeDefined();
|
||||
expect(reader).toHaveBeenCalledWith(expect.objectContaining({contentRef:"/home/agent/proof.txt"}));
|
||||
});
|
||||
|
||||
it("stages only exact wake-bound inbound bytes without exposing an API credential", async () => {
|
||||
const storage = createStorageService(
|
||||
createLocalDiskStorageProvider(storageRoot),
|
||||
|
||||
@@ -47,6 +47,8 @@ export interface NativeRunnerFileHandoffBinding {
|
||||
readonly executionTargetKind: "local" | "remote";
|
||||
/** Server-bound reader. Never supplied by the model or a request body. */
|
||||
readonly readRemoteWorkspaceFile?: RemoteWorkspaceFileReader;
|
||||
/** Provider-bound reader which authorizes absolute paths within task-granted locations. */
|
||||
readonly readRemoteTaskFile?: RemoteWorkspaceFileReader;
|
||||
}
|
||||
|
||||
export interface NativeRunnerFileHandoffInput {
|
||||
@@ -326,14 +328,15 @@ async function readVerifiedWorkspaceFile(
|
||||
}
|
||||
|
||||
if (binding.executionTargetKind === "remote") {
|
||||
if (!binding.readRemoteWorkspaceFile) throw new Error("paperclip_runner_file_handoff_remote_unsupported");
|
||||
const readFile = binding.readRemoteTaskFile ?? binding.readRemoteWorkspaceFile;
|
||||
if (!readFile) throw new Error("paperclip_runner_file_handoff_remote_unsupported");
|
||||
const contentRef = requiredText(input.contentRef, "content_ref", 2_000);
|
||||
if (path.posix.isAbsolute(contentRef) || /^[a-z][a-z0-9+.-]*:/iu.test(contentRef)
|
||||
if ((!binding.readRemoteTaskFile && path.posix.isAbsolute(contentRef)) || /^[a-z][a-z0-9+.-]*:/iu.test(contentRef)
|
||||
|| contentRef.includes("\\") || path.posix.normalize(contentRef) === ".."
|
||||
|| path.posix.normalize(contentRef).startsWith("../")) {
|
||||
throw new Error("paperclip_runner_file_handoff_path_denied");
|
||||
}
|
||||
const body = await binding.readRemoteWorkspaceFile({ contentRef, byteSize: input.byteSize, sha256: expectedSha256 });
|
||||
const body = await readFile({ contentRef, byteSize: input.byteSize, sha256: expectedSha256 });
|
||||
if (body.length !== input.byteSize) throw new Error("paperclip_runner_file_handoff_size_denied");
|
||||
if (createHash("sha256").update(body).digest("hex") !== expectedSha256) throw new Error("paperclip_runner_file_handoff_hash_mismatch");
|
||||
return { body, contentType, filename, sha256: expectedSha256, title };
|
||||
|
||||
@@ -11074,10 +11074,12 @@ describe("runnerd provider runtime wiring", () => {
|
||||
const launcher = vi.fn();
|
||||
const registration = vi.fn();
|
||||
const driver = { prepareLaunch: vi.fn(), runnerProcessLauncher: launcher, controlPlaneRegistration: registration };
|
||||
const owner = vi.spyOn(pluginTaskRuntime, "createPluginRunnerTaskOwner").mockReturnValue({ launcher, registration, driver });
|
||||
state.toolAuthorityDefinitions.mockImplementation(async binding => binding.readRemoteTaskFile ? [{ name: "register_deliverable" }] as never : []);
|
||||
const readWorkspaceFile = vi.fn(async () => Buffer.from("proof"));
|
||||
const owner = vi.spyOn(pluginTaskRuntime, "createPluginRunnerTaskOwner").mockReturnValue({ launcher, registration, driver, readWorkspaceFile });
|
||||
const task = vi.fn(async (operation: unknown) => {
|
||||
environmentTaskOperationSchema.parse(operation);
|
||||
return { kind: "description", launchPreparationVersion: 1, projectAuthCheckVersion: 1, taskId: "task", runner: { version: "fixture", digest: `sha256:${"a".repeat(64)}` } };
|
||||
return { kind: "description", launchPreparationVersion: 1, projectAuthCheckVersion: 1, ...(kind === "codex" ? { workspaceFileReadVersion: 1 } : {}), taskId: "task", runner: { version: "fixture", digest: `sha256:${"a".repeat(64)}` } };
|
||||
});
|
||||
const target = { kind: "remote", transport: "sandbox", providerKey: "plugin", leaseId: "plugin-task-lease",
|
||||
remoteCwd: "/unused", runnerTask: task, runnerTaskProjectIds: ["10000000-0000-4000-8000-000000000001"] } as const;
|
||||
@@ -11098,6 +11100,9 @@ describe("runnerd provider runtime wiring", () => {
|
||||
expect(state.createBackend.mock.calls.at(-1)![1]).toMatchObject({ workingDirectoryAuthority: "runner_prepared_launch", environment: {} });
|
||||
state.createBackend.mock.calls.at(-1)![1].codexTransportFactory!();
|
||||
const options = state.createTransport.mock.calls.at(-1)![0];
|
||||
const expectedTools = kind === "codex" ? [{ name: "register_deliverable" }] : [];
|
||||
expect(state.createBackend.mock.calls.at(-1)![1]).toMatchObject({ dynamicTools: expectedTools });
|
||||
expect(options).toMatchObject({ resumeDynamicTools: expectedTools });
|
||||
expect(options).toMatchObject({ ...driver, runnerArtifact: { version: "fixture" },
|
||||
prpIdentity: { environmentLeaseId: "plugin-task-lease" } });
|
||||
if (kind === "acpx") expect(options).toMatchObject({ acpxAgent: "claude", claudeEffort: "high" });
|
||||
@@ -11105,6 +11110,18 @@ describe("runnerd provider runtime wiring", () => {
|
||||
expect(options).not.toHaveProperty(key);
|
||||
}
|
||||
expect(task.mock.calls).toEqual([[{ kind: "describe" }]]);
|
||||
const binding = state.toolAuthorityDefinitions.mock.calls.at(-1)![0] as unknown as {
|
||||
readRemoteApiFile?: (path: string, maxBytes: number) => Promise<Buffer>;
|
||||
readRemoteTaskFile?: (file: {contentRef: string;byteSize: number;sha256: string}) => Promise<Buffer>;
|
||||
};
|
||||
if (kind === "codex") {
|
||||
await expect(binding.readRemoteApiFile!("proof.txt",1024)).resolves.toEqual(Buffer.from("proof"));
|
||||
await expect(binding.readRemoteTaskFile!({contentRef:"proof.txt",byteSize:5,sha256:"a".repeat(64)})).resolves.toEqual(Buffer.from("proof"));
|
||||
expect(readWorkspaceFile).toHaveBeenCalledWith("proof.txt",5);
|
||||
} else {
|
||||
expect(binding.readRemoteApiFile).toBeUndefined();
|
||||
expect(binding.readRemoteTaskFile).toBeUndefined();
|
||||
}
|
||||
expect(owner.mock.calls.at(-1)![0]).not.toHaveProperty("assets");
|
||||
const identity: Record<string, unknown> = options.prpIdentity!;
|
||||
await owner.mock.calls.at(-1)![0].onSessionRetained!({ schema: "paperclip.runner.durable.state.v1", lifecycle: "suspended", ...identity });
|
||||
|
||||
@@ -11813,6 +11813,11 @@ async function createRunnerdBackendWithinSessionClaim(
|
||||
.where(and(eq(environmentLeases.id, remoteTarget.leaseId!), eq(environmentLeases.companyId, input.execution.binding.companyId)));
|
||||
}, onDispatched: input.onTaskDispatched,
|
||||
});
|
||||
if (description.workspaceFileReadVersion === 1) {
|
||||
authority.binding.readRemoteApiFile = owner.readWorkspaceFile;
|
||||
authority.binding.readRemoteTaskFile = file => owner.readWorkspaceFile(file.contentRef, file.byteSize);
|
||||
dynamicTools = await authorityEpoch.definitions();
|
||||
}
|
||||
const executeTool = (call: Parameters<SessionToolAuthorityEpoch["execute"]>[0]) => {
|
||||
const current = sessionToolAuthorityEpochs.get(sessionScopeId);
|
||||
if (!current) throw new Error("native_session_tool_authority_unavailable");
|
||||
|
||||
@@ -179,6 +179,8 @@ type Binding = {
|
||||
assertBridgeAuthority?: (database?: Db) => Promise<void>;
|
||||
executionTargetKind?: "local" | "remote";
|
||||
readRemoteWorkspaceFile?: RemoteWorkspaceFileReader;
|
||||
readRemoteTaskFile?: RemoteWorkspaceFileReader;
|
||||
readRemoteApiFile?: (path: string, maxBytes: number) => Promise<Buffer>;
|
||||
currentWakeComments?: CurrentWakeCommentsBinding;
|
||||
chatAttachmentReadScope?: NativeChatAttachmentReadScope;
|
||||
syncIssueExternalObjects?: (issueId: string) => Promise<void>;
|
||||
@@ -278,7 +280,7 @@ export class PaperclipRunnerToolAuthority {
|
||||
descriptor.allowedModes.includes(workMode) &&
|
||||
(descriptor.operationId !== "register_deliverable" ||
|
||||
(Boolean(this.binding.workspaceRoot) &&
|
||||
((this.binding.executionTargetKind ?? "local") === "local" || Boolean(this.binding.readRemoteWorkspaceFile)))),
|
||||
((this.binding.executionTargetKind ?? "local") === "local" || Boolean(this.binding.readRemoteWorkspaceFile || this.binding.readRemoteTaskFile)))),
|
||||
).map((descriptor) => ({
|
||||
name: descriptor.operationId,
|
||||
description:
|
||||
@@ -303,8 +305,9 @@ export class PaperclipRunnerToolAuthority {
|
||||
},
|
||||
contentRef: {
|
||||
...(descriptor.inputSchema.properties?.contentRef ?? {}),
|
||||
description:
|
||||
"Workspace-relative source path. Absolute paths, URLs, traversal, and symlinks are rejected.",
|
||||
description: this.binding.readRemoteTaskFile
|
||||
? "Path to the file on the execution host: workspace-relative or absolute within a location granted to this task, including its agent home."
|
||||
: "Workspace-relative source path. Absolute paths, URLs, traversal, and symlinks are rejected.",
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -1165,6 +1168,14 @@ export class PaperclipRunnerToolAuthority {
|
||||
await assertArtifactAccess();
|
||||
return { bytes: Buffer.concat(chunks), filename: asset.originalFilename ?? "file", contentType: asset.contentType };
|
||||
}
|
||||
if (this.binding.executionTargetKind === "remote") {
|
||||
if (!this.binding.readRemoteApiFile) throw badRequest("Remote workspace file uploads are unavailable for this execution provider");
|
||||
await this.#boundContext();
|
||||
const bytes = await this.binding.readRemoteApiFile(file.path!, RUNNER_API_MAX_BYTES);
|
||||
await this.#boundContext();
|
||||
if (bytes.length > RUNNER_API_MAX_BYTES) throw badRequest("Workspace file exceeds API transfer limit");
|
||||
return { bytes, filename: basename(file.path!), contentType: "application/octet-stream" };
|
||||
}
|
||||
const resolved = await workspaceFileResourceService(this.db).prepareDownload(this.binding.issueId, { path: file.path!, workspace: "auto" });
|
||||
const handle = await openRunnerApiWorkspaceFile(resolved.realPath);
|
||||
try {
|
||||
@@ -1940,6 +1951,7 @@ export class PaperclipRunnerToolAuthority {
|
||||
workspaceRoot,
|
||||
executionTargetKind: this.binding.executionTargetKind ?? "local",
|
||||
readRemoteWorkspaceFile: this.binding.readRemoteWorkspaceFile,
|
||||
readRemoteTaskFile: this.binding.readRemoteTaskFile,
|
||||
},
|
||||
deliverable: {
|
||||
filename: typeof input.filename === "string" ? input.filename : "",
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import type { RunnerProviderLaunchRequirements } from "@paperclipai/paperclip-runner";
|
||||
import type { RunnerProcessLaunchSpec } from "../../vendor/paperclip-runner/index.js";
|
||||
@@ -274,3 +275,31 @@ describe("plugin native Runner ownership", () => {
|
||||
} finally { process.child.kill(); await process.completion.catch(() => {}); }
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
describe("plugin task workspace file transfer", () => {
|
||||
it("reads only a submitted task, validates bytes, and rejects late responses after release", async () => {
|
||||
const {task}=provider(); const perform=task.getMockImplementation()!;
|
||||
const bytes=Buffer.from("VM proof\n");
|
||||
let file={kind:"file",taskId:"task",base64:bytes.toString("base64"),byteSize:bytes.length,sha256:createHash("sha256").update(bytes).digest("hex")};
|
||||
let afterRead=async()=>{};
|
||||
task.mockImplementation(async op=>{if(op.kind==="read_file"){await afterRead();return file;}return perform(op);});
|
||||
const owner=createPluginRunnerTaskOwner({projectAuthCheckVersion:1,task,projectIds:[],harness:"codex",pollMs:1});
|
||||
await expect(owner.readWorkspaceFile("proof.txt",1024)).rejects.toThrow("unavailable");
|
||||
await prepare(owner);
|
||||
ingress.connect.mockReturnValue({ready:Promise.resolve(),close:async()=>{}});
|
||||
const registration=await owner.registration({} as never,"run");
|
||||
const handle=owner.launcher(launchSpec());registration.activate();await registration.ready();
|
||||
try {
|
||||
await expect(owner.readWorkspaceFile("proof.txt",1024)).resolves.toEqual(bytes);
|
||||
await expect(owner.readWorkspaceFile("/home/agent/proof.txt",1024)).resolves.toEqual(bytes);
|
||||
expect(task.mock.calls.find(([op])=>op.kind==="read_file")?.[0]).toMatchObject({identity:requirements.identity,path:"proof.txt",projectIds:[]});
|
||||
for(const path of ["../escape","//etc/passwd","file:/secret","a/../b","a\\b"]) await expect(owner.readWorkspaceFile(path,1024)).rejects.toThrow();
|
||||
await expect(owner.readWorkspaceFile("proof.txt",1)).rejects.toThrow("unavailable");
|
||||
file={...file,sha256:"f".repeat(64)};
|
||||
await expect(owner.readWorkspaceFile("proof.txt",1024)).rejects.toThrow("integrity");
|
||||
afterRead=()=>registration.release();
|
||||
await expect(owner.readWorkspaceFile("proof.txt",1024)).rejects.toThrow("unavailable");
|
||||
} finally {await registration.release();await registration.checkpoint("unsettled");await handle.completion;}
|
||||
});
|
||||
});
|
||||
@@ -1,3 +1,4 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { setTimeout as delay } from "node:timers/promises";
|
||||
import { environmentTaskOperationSchema, environmentTaskResultSchema, type PluginEnvironmentTaskOperation, type PluginEnvironmentTaskResult, type PluginEnvironmentTaskCredential } from "@paperclipai/plugin-sdk";
|
||||
import type { RunnerIngressEndpoint } from "@paperclipai/adapter-utils/runner-connectivity";
|
||||
@@ -44,7 +45,7 @@ export function createPluginRunnerTaskOwner(input: {
|
||||
void running.catch(() => undefined);
|
||||
const call = async (operation: PluginEnvironmentTaskOperation) => {
|
||||
const result = environmentTaskResultSchema.parse(await input.task(operation));
|
||||
if (result.kind !== (operation.kind === "describe" ? "description" : operation.kind === "prepare" ? "prepared" : operation.kind === "check_project_auth" ? "project_auth" : ["read_state", "archive_state"].includes(operation.kind) ? "state" : ["status", "connection", "credentials"].includes(operation.kind) ? operation.kind : "accepted")) {
|
||||
if (result.kind !== (operation.kind === "read_file" ? "file" : operation.kind === "describe" ? "description" : operation.kind === "prepare" ? "prepared" : operation.kind === "check_project_auth" ? "project_auth" : ["read_state", "archive_state"].includes(operation.kind) ? "state" : ["status", "connection", "credentials"].includes(operation.kind) ? operation.kind : "accepted")) {
|
||||
throw new Error("Plugin Runner returned an invalid operation result");
|
||||
}
|
||||
return result;
|
||||
@@ -247,5 +248,15 @@ export function createPluginRunnerTaskOwner(input: {
|
||||
},
|
||||
controlPlaneRegistration: (authority, identity, signal) => registration(authority, identity?.runId ?? "", signal),
|
||||
};
|
||||
return { launcher, registration, driver };
|
||||
const readWorkspaceFile = async (path: string, maxBytes: number): Promise<Buffer> => {
|
||||
if (!submitted || !preparedIdentity || released || stopRequested) throw new Error("Plugin Runner file access unavailable");
|
||||
const result = await call(environmentTaskOperationSchema.parse({ kind: "read_file", identity: preparedIdentity,
|
||||
projectIds: [...input.projectIds], path, maxBytes }));
|
||||
if (released || stopRequested || result.kind !== "file" || result.byteSize > maxBytes) throw new Error("Plugin Runner file access unavailable");
|
||||
const bytes = Buffer.from(result.base64, "base64");
|
||||
if (bytes.length !== result.byteSize || bytes.toString("base64") !== result.base64 ||
|
||||
createHash("sha256").update(bytes).digest("hex") !== result.sha256) throw new Error("Plugin Runner file integrity check failed");
|
||||
return bytes;
|
||||
};
|
||||
return { launcher, registration, driver, readWorkspaceFile };
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { existsSync } from "node:fs";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { chmod, writeFile, symlink, mkdir, open } from "node:fs/promises";
|
||||
import { join, dirname } from "node:path";
|
||||
import { eq } from "drizzle-orm";
|
||||
@@ -933,6 +933,22 @@ else if(m.id!==undefined) send({id:m.id,result:{}});
|
||||
expect((await fixture.snapshot()).assets).toEqual(expect.arrayContaining([expect.objectContaining({ id: download.artifact.artifactId, companyId: fixture.companyId, createdByAgentId: fixture.agentId })]));
|
||||
});
|
||||
|
||||
it.each(["sample.txt", "/home/agent/proof.txt"])("uploads remote task bytes at %s and fails closed without the reader", async (sourcePath) => {
|
||||
const fixture=await server.fixture();
|
||||
const bytes=Buffer.from("proof exists only on the execution host\n");
|
||||
const read=vi.fn(async(path:string,maxBytes:number)=>{expect(path).toBe(sourcePath);expect(maxBytes).toBeGreaterThan(bytes.length);return bytes;});
|
||||
const binding={...fixture.authority.binding,executionTargetKind:"remote" as const,readRemoteApiFile:read};
|
||||
const authority=new PaperclipRunnerToolAuthority(server.db,binding);
|
||||
const call={tool:"call_api" as const,callId:randomUUID(),arguments:{operationId:"POST /api/companies/{companyId}/issues/{issueId}/attachments",pathParams:{issueId:fixture.issueId},files:[{path:sourcePath}]}};
|
||||
await expect(authority.execute(call)).resolves.toMatchObject({status:201});
|
||||
expect(read).toHaveBeenCalledOnce();
|
||||
expect((await fixture.snapshot()).assets).toEqual(expect.arrayContaining([expect.objectContaining({sha256:createHash("sha256").update(bytes).digest("hex"),byteSize:bytes.length})]));
|
||||
const unsupported=new PaperclipRunnerToolAuthority(server.db,{...binding,readRemoteApiFile:undefined});
|
||||
await expect(unsupported.execute({...call,callId:randomUUID()})).rejects.toThrow("Remote workspace file uploads are unavailable");
|
||||
const changing=new PaperclipRunnerToolAuthority(server.db,{...binding,readRemoteApiFile:async()=>{await server.db.update(heartbeatRuns).set({status:"cancelled"}).where(eq(heartbeatRuns.id,fixture.runId));return bytes;}});
|
||||
await expect(changing.execute({...call,callId:randomUUID()})).rejects.toThrow();
|
||||
});
|
||||
|
||||
it("does not accept caller-supplied identity in an API comment", async () => {
|
||||
const fixture = await server.fixture();
|
||||
const result = await fixture.authority.execute({ tool: "call_api", callId: "comment", arguments: { operationId: "POST /api/issues/{id}/comments", pathParams: { id: fixture.issueId }, body: { body: "Identity proof", authorAgentId: randomUUID(), authorUserId: "spoofed", runId: randomUUID() } } });
|
||||
|
||||
Reference in new issue
Block a user