mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 10:13:11 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps connections give agents controlled access to external services. > - Daytona provides sandbox, file, command, Git, and account APIs. > - Operators need to assign this access without exposing a server credential. > - A small task-specific tool list cannot serve general Daytona work. > - This pull request adds the published HTTP actions through the existing connection gateway. > - Operators keep control of identity, agent access, and each action. ## Linked Issues or Issue Description **Problem or motivation** Paperclip has a Daytona sandbox provider for its own execution infrastructure. It lacks a normal Apps connection for agents to manage an operator's Daytona account. Reusing the server's provider credential would bypass normal connection ownership and action permissions. **Proposed solution** Add a Daytona connection with personal or shared vault credentials. Generate its action catalog from the official platform and Toolbox API specifications. Use the existing run-bound gateway, grants, action policies, approvals, revocation, and audit paths. Sandbox IDs come from Daytona; tools need no Paperclip task or lease ID. **Alternatives considered** A four-action diagnostic subset cannot support general provider work. Launching the provider CLI would expose host filesystem behavior. Native OAuth is not included because a supported third-party registration path was not verified. The native method uses an organization API key. Existing Composio setup is a separate possible OAuth route and is not qualified by this change. **Roadmap alignment** This extends the existing Apps and Connections provider coverage in `ROADMAP.md`. It uses that system's credential and permission model. GitHub searches found no existing Daytona Apps connector PR or issue. ## What Changed - Add the Daytona catalog entry, provider source override, official theme logos, permission review, and setup documentation. - Generate 274 HTTP actions: 169 platform and 105 Toolbox operations. Record two WebSocket upgrades separately; do not advertise them as HTTP tools. Record source URLs and hashes. - Add provider-focused execution for sandboxes, snapshots, volumes, files, Git, commands, computer use, and account resources. Provider authority still limits which endpoints accept an organization key. - Validate action arguments. Resolve Toolbox targets through the fixed platform API. Check sandbox identity and organization before using an approved Daytona proxy. Keep normal guarded network requests and vault resolution. - Transfer file bytes without reading or writing host files. Share one caller deadline across prerequisite reads, dispatch, and response reading. Bound requests and responses. Do not automatically retry writes with unknown outcomes. Redact the selected credential from results and errors. - Classify credential access, access changes, interactive input, command execution, file permissions, and deletion (including TTL and auto-delete) as destructive actions. Keep new and changed actions under catalog review. - Upgrade saved catalogs and retire the earlier diagnostic names. Hide retired actions from the UI and agent discovery even when provider refresh fails. Preserve existing action choices and historical records. Reject stale same-name contracts during agent discovery and dispatch before a board refresh. - Add database-backed gateway and catalog-upgrade regressions, schema generator tests, a browser setup check, and both capability map updates. ## Verification - Passed on the review-fix head: 57 focused Vitest cases (18 Daytona database/client cases, five HTTP boundary cases, and 34 shared catalog cases), four generator tests, `pnpm -r typecheck`, and `pnpm build`. The five HTTP cases use a real local server through the pinned production transport. The normal browser setup check and 17 generator/artwork cases passed on the initial PR head; those UI and artwork paths are unchanged. - Passed on this branch: brand asset validation, token gates, map structure checks, and `git diff --check`. All latest-head GitHub checks pass on `ea979ccb3a64dc9508807823540f935c4316ea35`. Apex is 5/5, with all five findings addressed and zero unresolved threads. The branch has no merge conflicts. - The initial catalog-generation command could not find the external research corpus. The supported `--definitions-only` mode regenerated the definitions successfully. Unrelated generated drift was excluded. - An earlier full local Vitest attempt on the preview branch failed when bundled PostgreSQL could not start on macOS. After replay onto current master, all three unchanged event-sequencing cases pass with bundled PostgreSQL; the earlier startup cause remains uncertain. - The first full-suite command on this branch used my external test database override. Chat fixtures require a current migrated schema and failed because `agents.lifecycle_state` was absent in that disposable database. A focused comparison reproduces failure with the override and passes with normal setup. That attempt was stopped, and the full suite restarted without the override. Focused Daytona cases use their own migrated PostgreSQL 16 databases. - The clean local full-suite attempt recorded embedded PostgreSQL startup failures in two unchanged native-runtime cases and two failures in unchanged Git fixtures. It was stopped after these failures were isolated; no full local suite pass is claimed. The 15 matching native-runtime cases pass separately. The Git cases fail with the local `init.defaultBranch=main` setting and pass with `GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master` limited to the test command. No global Git setting was changed. Full Linux CI is green. - Fixtures use real database persistence and run-bound gateway sessions with provider responses replaced by fixtures. They cover personal/shared vault ownership, reads and writes, file bytes, Off policies, revocation, organization mismatch, untrusted proxy rejection, output bounds, saved catalog upgrades, failed refresh, and permission preservation. - The first Apex review found five valid issues: multipart transport compatibility, destructive classification, caller deadlines, audit filters, and stale saved contracts. All five were fixed with regressions before the second review. A typecheck then found the nullable saved schema hash; admission now accepts that stored type and rejects null as stale. - The browser check covers setup controls without submitting a provider key. No real Daytona account, provider write, native OAuth flow, or agent model run is qualified. Live provider read/write proof remains pending a test account. ## Risks - Daytona keys have broad Toolbox authority, including keys with no selected resource scopes. Only assign this connection to trusted agents and review its action permissions. - Remote writes or commands can continue after a timeout. Calls do not automatically retry an uncertain outcome. - The bridge targets Daytona Cloud. Proxy hosts are restricted; an unrecognized future regional host fails closed. - Files and responses have an 8 MiB bound. Each action has one caller deadline, capped at 30 seconds across all provider requests and response reading. Interactive WebSocket sessions are not exposed as HTTP actions. - The generated JSON snapshot exceeds 1,000 lines because it holds the provider input contracts. Generator logic and execution code remain in separate small modules. Updates require explicit regeneration and catalog review. - No database migration or lockfile change is required. ## Model Used OpenAI Codex, GPT-6. The runtime identifies the model family but does not expose the exact backend model ID or context window. Used reasoning, repository inspection, code editing, shell tools, database verification, and browser tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
259 lines
15 KiB
TypeScript
259 lines
15 KiB
TypeScript
import Ajv, { type ValidateFunction } from "ajv";
|
|
import addFormats from "ajv-formats";
|
|
import { z } from "zod";
|
|
import { HttpError } from "../errors.js";
|
|
import { redactSensitiveText } from "../redaction.js";
|
|
import { descriptorHash, stableHash } from "./tool-catalog-hash.js";
|
|
import catalog from "./daytona/operations.generated.json" with { type: "json" };
|
|
|
|
export const DAYTONA_API_URL = "https://app.daytona.io/api";
|
|
// Preserve the saved method identity when upgrading the earlier small catalog.
|
|
export const DAYTONA_METHOD = "workspace-diagnostics";
|
|
const MAX_BYTES = 8 * 1024 * 1024;
|
|
const record = (value: unknown): Record<string, unknown> =>
|
|
value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : {};
|
|
const failure = (status: number, code: string, message: string) => new HttpError(status, message, { code });
|
|
export type DaytonaRequest = (url: string, init: RequestInit) => Promise<Response>;
|
|
type Operation = typeof catalog.tools[number];
|
|
const operations = new Map(catalog.tools.map(operation => [operation.name, operation]));
|
|
const validators = new Map<string, ValidateFunction>();
|
|
// Match the interop used by the existing plugin configuration validator.
|
|
const AjvCtor = (Ajv as any).default ?? Ajv;
|
|
const ajv = new AjvCtor({ strict: false, allErrors: true, validateFormats: true });
|
|
((addFormats as any).default ?? addFormats)(ajv);
|
|
|
|
export const DAYTONA_TOOLS = catalog.tools.map(({ name, title, description, inputSchema, annotations }) => ({
|
|
name, title, description, inputSchema: inputSchema as Record<string, unknown>, annotations,
|
|
}));
|
|
|
|
export function isDaytonaToolName(name: string): boolean {
|
|
return operations.has(name);
|
|
}
|
|
|
|
const currentContracts = new Map(DAYTONA_TOOLS.map(tool => [tool.name, {
|
|
versionHash: descriptorHash(tool, daytonaRisk(tool.name)), schemaHash: stableHash(tool.inputSchema), riskLevel: daytonaRisk(tool.name),
|
|
}]));
|
|
|
|
export function isCurrentDaytonaCatalogEntry(entry: { toolName: string; versionHash: string; schemaHash: string | null; riskLevel: string }): boolean {
|
|
const current = currentContracts.get(entry.toolName);
|
|
return Boolean(current && current.versionHash === entry.versionHash && current.schemaHash === entry.schemaHash && current.riskLevel === entry.riskLevel);
|
|
}
|
|
|
|
export function isDaytonaConnection(connection: { transport: string; config?: Record<string, unknown> }) {
|
|
return connection.transport === "rest_api" && connection.config?.sourceTemplateKey === "daytona"
|
|
&& connection.config.connectionMethodKey === DAYTONA_METHOD && connection.config.url === DAYTONA_API_URL;
|
|
}
|
|
|
|
export function daytonaRisk(name: string): "read" | "write" | "destructive" {
|
|
const operation = operations.get(name);
|
|
if (!operation || operation.annotations.destructiveHint) return "destructive";
|
|
return operation.annotations.readOnlyHint ? "read" : "write";
|
|
}
|
|
|
|
function pathValue(value: unknown) {
|
|
const text = String(value);
|
|
if (!text || text.length > 8192 || /[\u0000-\u001f\\]/.test(text)
|
|
|| text.split("/").some(segment => segment === "." || segment === "..")) {
|
|
throw failure(422, "daytona_arguments_invalid", "Invalid Daytona path argument.");
|
|
}
|
|
return encodeURIComponent(text);
|
|
}
|
|
|
|
function decodeFile(value: unknown) {
|
|
if (typeof value !== "string" || value.length > Math.ceil(MAX_BYTES / 3) * 4
|
|
|| value.length % 4 !== 0 || /[^A-Za-z0-9+/=]/.test(value)) {
|
|
throw failure(422, "daytona_file_invalid", "File contents must be bounded base64 bytes.");
|
|
}
|
|
const bytes = Buffer.from(value, "base64");
|
|
if (bytes.toString("base64") !== value) throw failure(422, "daytona_file_invalid", "File contents must be canonical base64 bytes.");
|
|
return bytes;
|
|
}
|
|
|
|
/** A provider-focused REST bridge. It never reads host files, calls host shell
|
|
* commands, borrows provider-plugin credentials, or looks up Paperclip tasks. */
|
|
export function daytonaClient(headers: Record<string, string>, request: DaytonaRequest, pinnedOrganizationId?: unknown,
|
|
signal: AbortSignal = AbortSignal.timeout(30_000), deadlineAt: number = Date.now() + 30_000) {
|
|
const authorization = headers.Authorization;
|
|
if (!authorization || !/^Bearer [^\r\n]+$/.test(authorization)) {
|
|
throw failure(422, "daytona_credential_missing", "Reconnect Daytona to restore its API key.");
|
|
}
|
|
const secret = authorization.slice(7);
|
|
const expired = () => failure(504, "daytona_deadline_exceeded", "The Daytona action deadline expired. An operation already sent may still be running; inspect it before retrying.");
|
|
// Bound the caller even if a transport or body reader is slow to acknowledge
|
|
// cancellation. A late prerequisite result never advances to the next call.
|
|
async function withinDeadline<T>(work: () => Promise<T>): Promise<T> {
|
|
if (signal.aborted || Date.now() >= deadlineAt) throw expired();
|
|
let abort!: () => void;
|
|
try {
|
|
return await Promise.race([
|
|
Promise.resolve().then(() => { if (signal.aborted || Date.now() >= deadlineAt) throw expired(); return work(); }),
|
|
new Promise<never>((_resolve, reject) => {
|
|
abort = () => reject(expired());
|
|
signal.addEventListener("abort", abort, { once: true });
|
|
}),
|
|
]);
|
|
} finally { signal.removeEventListener("abort", abort); }
|
|
}
|
|
function sanitize(value: unknown): unknown {
|
|
if (typeof value === "string") return redactSensitiveText(value.split(secret).join("[REDACTED]"));
|
|
if (Array.isArray(value)) return value.map(sanitize);
|
|
if (value && typeof value === "object") return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, sanitize(item)]));
|
|
return value;
|
|
}
|
|
|
|
async function send(url: string, init: RequestInit, organizationId?: string): Promise<unknown> {
|
|
const projected = new Headers(init.headers);
|
|
projected.set("Authorization", authorization!);
|
|
projected.set("X-Daytona-Source", "paperclip-connector");
|
|
if (organizationId) projected.set("X-Daytona-Organization-ID", organizationId);
|
|
let response: Response;
|
|
try {
|
|
response = await withinDeadline(() => request(url, { ...init, redirect: "error", headers: Object.fromEntries(projected), signal }));
|
|
} catch {
|
|
if (signal.aborted || Date.now() >= deadlineAt) throw expired();
|
|
throw failure(502, "daytona_outcome_unknown", init.method === "GET"
|
|
? "Daytona could not complete the request."
|
|
: "Daytona did not return a result. The operation may have started; inspect its state before retrying.");
|
|
}
|
|
if (!response.ok) {
|
|
void response.body?.cancel().catch(() => {});
|
|
const status = [400, 401, 403, 404, 409, 422, 429].includes(response.status) ? response.status : 502;
|
|
throw failure(status, "daytona_request_failed", status === 401 || status === 403
|
|
? "Daytona denied access. Check this key's organization and permissions. Some account operations require provider user authentication."
|
|
: status === 404 ? "Daytona could not find this resource." : "Daytona could not complete this operation.");
|
|
}
|
|
if (response.status === 204 || !response.body) return null;
|
|
const reader = response.body.getReader();
|
|
const chunks: Uint8Array[] = [];
|
|
let length = 0;
|
|
try {
|
|
for (;;) {
|
|
const part = await withinDeadline(() => reader.read());
|
|
if (part.done) break;
|
|
length += part.value.byteLength;
|
|
if (length > MAX_BYTES) throw failure(502, "daytona_output_limit", "Daytona's response exceeded the limit. Request a smaller page or file range.");
|
|
chunks.push(part.value);
|
|
}
|
|
} catch (error) {
|
|
if (error instanceof HttpError) throw error;
|
|
throw failure(502, "daytona_outcome_unknown", "Daytona's response was interrupted. Inspect the operation before retrying.");
|
|
} finally { void reader.cancel().catch(() => {}); }
|
|
const bytes = Buffer.concat(chunks);
|
|
if (!bytes.length) return null;
|
|
const contentType = response.headers.get("Content-Type") ?? "application/octet-stream";
|
|
if (contentType.includes("json")) {
|
|
try { return sanitize(JSON.parse(bytes.toString("utf8"))); }
|
|
catch { throw failure(502, "daytona_invalid_response", "Daytona returned an invalid response."); }
|
|
}
|
|
if (contentType.startsWith("text/")) return sanitize(bytes.toString("utf8"));
|
|
if (bytes.includes(Buffer.from(secret))) throw failure(403, "daytona_credential_export_denied", "The response contains connection credential material.");
|
|
return { contentType, contentBase64: bytes.toString("base64"), byteLength: bytes.length };
|
|
}
|
|
|
|
async function identity() {
|
|
const value = record(await send(`${DAYTONA_API_URL}/api-keys/current`, { method: "GET" }));
|
|
const organizationId = value.organizationId;
|
|
if (!z.string().uuid().safeParse(organizationId).success || (pinnedOrganizationId && organizationId !== pinnedOrganizationId)) {
|
|
throw failure(403, "daytona_organization_mismatch", "The Daytona key does not identify the configured organization.");
|
|
}
|
|
return { organizationId: organizationId as string };
|
|
}
|
|
|
|
async function execute(operation: Operation, parameters: Record<string, unknown>) {
|
|
const account = await identity();
|
|
let base = DAYTONA_API_URL;
|
|
if (operation.kind === "toolbox") {
|
|
const sandbox = record(await send(`${DAYTONA_API_URL}/sandbox/${pathValue(parameters.sandboxId)}`, { method: "GET" }, account.organizationId));
|
|
if (!z.string().uuid().safeParse(sandbox.id).success || sandbox.organizationId !== account.organizationId
|
|
|| (sandbox.id !== parameters.sandboxId && sandbox.name !== parameters.sandboxId)) {
|
|
throw failure(403, "daytona_workspace_mismatch", "Daytona returned a sandbox outside the selected account or target.");
|
|
}
|
|
let proxyValue = sandbox.toolboxProxyUrl;
|
|
if (!proxyValue) proxyValue = record(await send(`${DAYTONA_API_URL}/sandbox/${pathValue(sandbox.id)}/toolbox-proxy-url`, { method: "GET" }, account.organizationId)).url;
|
|
let proxy: URL;
|
|
try { proxy = new URL(String(proxyValue)); }
|
|
catch { throw failure(502, "daytona_proxy_invalid", "Daytona returned an invalid toolbox address."); }
|
|
if (proxy.protocol !== "https:" || !/^proxy(?:-[a-z0-9-]+)?\.app\.daytona\.io$/.test(proxy.hostname)
|
|
|| proxy.username || proxy.password || (proxy.port && proxy.port !== "443")
|
|
|| !/^\/toolbox\/?$/.test(proxy.pathname) || proxy.search || proxy.hash) {
|
|
throw failure(502, "daytona_proxy_invalid", "Daytona returned an unrecognized toolbox address. No credentials were forwarded.");
|
|
}
|
|
base = `${proxy.origin}/toolbox/${pathValue(sandbox.id)}`;
|
|
}
|
|
const route = operation.path.replace(/\{([^}]+)\}/g, (_match, key: string) => pathValue(record(parameters.path)[key]));
|
|
const url = new URL(base + route);
|
|
const projectedHeaders: Record<string, string> = {};
|
|
for (const parameter of operation.parameters) {
|
|
if (parameter.group === "path") continue;
|
|
const value = record(parameters[parameter.group])[parameter.name];
|
|
if (value === undefined || value === null) continue;
|
|
if (parameter.group === "headers") {
|
|
const text = String(value);
|
|
if (/[\r\n]/.test(text)) throw failure(422, "daytona_arguments_invalid", "Invalid Daytona header argument.");
|
|
projectedHeaders[parameter.name] = text;
|
|
} else if (parameter.json || (value && typeof value === "object" && !Array.isArray(value))) {
|
|
if (parameter.style === "deepObject") {
|
|
for (const [key, child] of Object.entries(record(value))) {
|
|
if (Array.isArray(child)) child.forEach(item => url.searchParams.append(`${parameter.name}[${key}]`, String(item)));
|
|
else url.searchParams.append(`${parameter.name}[${key}]`, String(child));
|
|
}
|
|
} else url.searchParams.append(parameter.name, JSON.stringify(value));
|
|
} else if (Array.isArray(value)) {
|
|
if (parameter.explode) value.forEach(item => url.searchParams.append(parameter.name, String(item)));
|
|
else url.searchParams.append(parameter.name, value.join(parameter.style === "spaceDelimited" ? " " : parameter.style === "pipeDelimited" ? "|" : ","));
|
|
} else url.searchParams.append(parameter.name, String(value));
|
|
}
|
|
let body: BodyInit | undefined;
|
|
if (operation.bodyContentType === "multipart/form-data") {
|
|
const form = new FormData();
|
|
if (Array.isArray(parameters.files)) {
|
|
let total = 0;
|
|
for (const item of parameters.files) {
|
|
const file = record(item), bytes = decodeFile(file.contentBase64);
|
|
total += bytes.length;
|
|
if (total > MAX_BYTES) throw failure(422, "daytona_input_limit", "The total upload exceeds the limit.");
|
|
form.append("files", new Blob([bytes]), String(file.path));
|
|
}
|
|
} else {
|
|
for (const [key, value] of Object.entries(record(parameters.body))) {
|
|
if (key === "file") form.append(key, new Blob([decodeFile(value)]), "upload");
|
|
else form.append(key, String(value));
|
|
}
|
|
}
|
|
// The guarded, DNS-pinned transport accepts bytes, not FormData. Use the
|
|
// platform encoder to serialize parts and its matching boundary header.
|
|
const encoded = new Request(DAYTONA_API_URL, { method: "POST", body: form });
|
|
body = Buffer.from(await withinDeadline(() => encoded.arrayBuffer()));
|
|
if (body.byteLength > MAX_BYTES) throw failure(422, "daytona_input_limit", "The multipart upload exceeds the limit.");
|
|
projectedHeaders["Content-Type"] = encoded.headers.get("Content-Type")!;
|
|
} else if (parameters.body !== undefined) {
|
|
body = JSON.stringify(parameters.body);
|
|
if (Buffer.byteLength(body) > MAX_BYTES) throw failure(422, "daytona_input_limit", "The request body exceeds the limit.");
|
|
projectedHeaders["Content-Type"] = operation.bodyContentType === "*/*" ? "application/json" : operation.bodyContentType ?? "application/json";
|
|
}
|
|
// No automatic retry. A transport failure cannot prove a write did not run.
|
|
return send(url.toString(), { method: operation.method, headers: projectedHeaders, body }, account.organizationId);
|
|
}
|
|
return { probe: identity, execute };
|
|
}
|
|
|
|
type Binding = { companyId: string; agentId: string | null; runId?: string | null; responsibleUserId?: string | null };
|
|
type Connection = { companyId: string; config: Record<string, unknown> };
|
|
export async function executeDaytonaTool(input: {
|
|
binding: Binding; connection: Connection; headers: Record<string, string>;
|
|
request: DaytonaRequest; toolName: string; parameters: unknown;
|
|
signal?: AbortSignal;
|
|
deadlineAt?: number;
|
|
}) {
|
|
if (!input.binding.agentId || input.connection.companyId !== input.binding.companyId) {
|
|
throw failure(404, "daytona_connection_not_found", "Connection not found.");
|
|
}
|
|
const operation = operations.get(input.toolName);
|
|
if (!operation) throw failure(404, "daytona_tool_not_found", "Daytona action not found.");
|
|
let validate = validators.get(operation.name);
|
|
if (!validate) { validate = ajv.compile(operation.inputSchema); validators.set(operation.name, validate!); }
|
|
if (!validate!(input.parameters)) throw failure(422, "daytona_arguments_invalid", "Invalid Daytona action arguments.");
|
|
const config = record(input.connection.config.methodConfig);
|
|
return daytonaClient(input.headers, input.request, config.organizationId, input.signal, input.deadlineAt).execute(operation, record(input.parameters));
|
|
}
|