mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-09 06:15:21 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox providers carry agent work through controlled execution channels > - The Daytona callback bridge uses a bespoke line-framed protocol over its duplex channel > - The bespoke protocol adds framing work and does not use the Node transport that already supports multiplexed streams > - This pull request carries raw bytes across the channel, adds a Node HTTP/2 bridge, and selects it for Daytona > - The benefit is one authenticated, multiplexed callback session with queue_v1 as the bounded fallback ## Linked Issues or Issue Description **Subsystem affected** The packages/plugins Daytona provider and the shared duplex execution path. **Problem or motivation** The Daytona callback bridge uses a bespoke line-framed protocol over the provider duplex channel. This adds protocol work and limits stream handling. **Proposed solution** Carry raw bytes through the cross-layer channel. Add an authenticated Node HTTP/2 host server and sandbox client gateway. Select http2_v1 for Daytona and retain queue_v1 as the fallback. **Alternatives considered** Keep the current duplex_v1 protocol. This keeps the bespoke framing path and does not provide one HTTP/2 session for callback streams. **Roadmap alignment** ROADMAP.md lists Daytona under cloud and sandbox agents. This change improves the shipped Daytona provider path. **Additional context** The branch adds no dependency. Node 24 provides the http2 module. The host token check and canonical path parser remain the single dispatch path. ## What Changed - Carry raw Uint8Array chunks through the adapter, plugin, worker, runtime, and Daytona layers. - Encode bytes as base64 only across the JSON-RPC hop, because JSON has no binary type. - Add the bounded host HTTP/2 server and the in-sandbox HTTP/2 client gateway. - Authenticate every stream with the per-run bridge token before route work. - Parse the path once and reuse the canonical result for route and forwarding work. - Select http2_v1 for Daytona and fall back once to queue_v1 when the client preface is absent. - Add transport, session, stream, and fallback telemetry. - Mark HTTP/2 as the preferred transport and queue_v1 as the soft-deprecated fallback. ## Verification - `npx vitest run packages/adapter-utils/src` — 990 passed and 4 skipped. - `npx vitest run server/src/__tests__/plugin-worker-manager-duplex.test.ts` — 32 passed. - `npx vitest run --config packages/plugins/sandbox-providers/daytona/vitest.config.ts` — 220 passed and 6 skipped. - `npx tsc --noEmit` in `packages/adapter-utils`, `packages/shared`, `packages/plugins/sdk`, and `server` — clean. - No `package.json` or `pnpm-lock.yaml` file changed. - The live Daytona test skips when `DAYTONA_API_KEY` is absent. - The root `npx tsc --noEmit` command has a pre-existing missing `packages/adapters/droid-local` reference on this branch and on `master`. ## Risks - The transport change affects several duplex layers and could expose byte-boundary errors. - A missing HTTP/2 client preface falls back once to queue_v1 and records `preface_missing`. - The host token check and canonical path parser must remain on the shared dispatch path. - The live Daytona test needs `DAYTONA_API_KEY` and does not run in this agent sandbox. ## Model Used OpenAI GPT-5, tool-enabled coding agent with repository inspection, GitHub CLI, and shell execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
276 lines
9.8 KiB
TypeScript
276 lines
9.8 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
|
|
import {
|
|
createNotification,
|
|
decodeChannelBytes,
|
|
DUPLEX_CHANNEL_DATA_NOTIFICATION,
|
|
DUPLEX_CHANNEL_EXIT_NOTIFICATION,
|
|
encodeChannelBytes,
|
|
parseMessage,
|
|
serializeMessage,
|
|
type PluginDuplexChannelCloseParams,
|
|
type PluginDuplexChannelCloseResult,
|
|
type PluginDuplexChannelDataParams,
|
|
type PluginDuplexChannelExitParams,
|
|
type PluginDuplexChannelOpenParams,
|
|
type PluginDuplexChannelOpenResult,
|
|
type PluginDuplexChannelStopParams,
|
|
type PluginDuplexChannelWriteParams,
|
|
} from "./protocol.js";
|
|
|
|
// The generic duplex channel messages model the setup-token pseudo-terminal
|
|
// contract: open, write, stop, and close requests, plus data and exit
|
|
// notifications. The host owns the route identifier. The worker returns a worker
|
|
// session identifier that binds the data and the exit notification only. A close
|
|
// keys on the host route identifier, so a lost open reply still permits a
|
|
// host-keyed close.
|
|
//
|
|
// The messages are static types, so a valid payload assigns to its type and an
|
|
// invalid payload fails the type check. The `@ts-expect-error` directives make
|
|
// the compiler reject each invalid payload. The `tsc --noEmit` check validates
|
|
// the directives, so an invalid payload that the type accepts fails the build.
|
|
|
|
describe("duplex channel request schemas", () => {
|
|
it("accepts a valid open request and its reply", () => {
|
|
const open: PluginDuplexChannelOpenParams = {
|
|
hostRouteId: "route-1",
|
|
driverKey: "daytona",
|
|
companyId: "company-1",
|
|
environmentId: "env-1",
|
|
providerLeaseId: "lease-1",
|
|
command: ["paperclip-bridge"],
|
|
};
|
|
const reply: PluginDuplexChannelOpenResult = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
};
|
|
expect(open.hostRouteId).toBe("route-1");
|
|
expect(reply.hostRouteId).toBe("route-1");
|
|
expect(reply.workerSessionId).toBe("ws-1");
|
|
});
|
|
|
|
it("rejects an open reply that omits the echoed host route identifier", () => {
|
|
// @ts-expect-error — the open reply echoes the host route identifier.
|
|
const reply: PluginDuplexChannelOpenResult = { workerSessionId: "ws-1" };
|
|
expect(reply).toBeDefined();
|
|
});
|
|
|
|
it("rejects an open request that omits the host route identifier", () => {
|
|
// @ts-expect-error — hostRouteId is required.
|
|
const open: PluginDuplexChannelOpenParams = {
|
|
driverKey: "daytona",
|
|
companyId: "company-1",
|
|
environmentId: "env-1",
|
|
providerLeaseId: "lease-1",
|
|
command: ["paperclip-bridge"],
|
|
};
|
|
expect(open).toBeDefined();
|
|
});
|
|
|
|
it("rejects an open request whose command is not a string array", () => {
|
|
const open: PluginDuplexChannelOpenParams = {
|
|
hostRouteId: "route-1",
|
|
driverKey: "daytona",
|
|
companyId: "company-1",
|
|
environmentId: "env-1",
|
|
providerLeaseId: "lease-1",
|
|
// @ts-expect-error — command must be a string array.
|
|
command: "paperclip-bridge",
|
|
};
|
|
expect(open).toBeDefined();
|
|
});
|
|
|
|
it("accepts a valid write request that carries the exact pair", () => {
|
|
const write: PluginDuplexChannelWriteParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
data: "payload",
|
|
};
|
|
expect(write.data).toBe("payload");
|
|
});
|
|
|
|
it("rejects a write request that omits the host route identifier", () => {
|
|
// @ts-expect-error — hostRouteId is required on a post-bind write.
|
|
const write: PluginDuplexChannelWriteParams = {
|
|
workerSessionId: "ws-1",
|
|
data: "payload",
|
|
};
|
|
expect(write).toBeDefined();
|
|
});
|
|
|
|
it("rejects a write request that omits the data", () => {
|
|
// @ts-expect-error — data is required.
|
|
const write: PluginDuplexChannelWriteParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
};
|
|
expect(write).toBeDefined();
|
|
});
|
|
|
|
it("accepts a valid stop request that carries the exact pair", () => {
|
|
const stop: PluginDuplexChannelStopParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
};
|
|
expect(stop.workerSessionId).toBe("ws-1");
|
|
});
|
|
|
|
it("rejects a stop request that omits the host route identifier", () => {
|
|
// @ts-expect-error — hostRouteId is required on a post-bind stop.
|
|
const stop: PluginDuplexChannelStopParams = { workerSessionId: "ws-1" };
|
|
expect(stop).toBeDefined();
|
|
});
|
|
|
|
it("rejects a stop request that omits the worker session identifier", () => {
|
|
// @ts-expect-error — workerSessionId is required.
|
|
const stop: PluginDuplexChannelStopParams = { hostRouteId: "route-1" };
|
|
expect(stop).toBeDefined();
|
|
});
|
|
|
|
it("accepts a pre-bind close request and a route-only acknowledgement", () => {
|
|
const close: PluginDuplexChannelCloseParams = { hostRouteId: "route-1" };
|
|
const reply: PluginDuplexChannelCloseResult = { hostRouteId: "route-1" };
|
|
expect(close.hostRouteId).toBe("route-1");
|
|
expect(reply.hostRouteId).toBe("route-1");
|
|
});
|
|
|
|
it("accepts a bound close acknowledgement that echoes both identifiers", () => {
|
|
const reply: PluginDuplexChannelCloseResult = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
};
|
|
expect(reply.workerSessionId).toBe("ws-1");
|
|
});
|
|
|
|
it("accepts a close request that also carries the worker session identifier", () => {
|
|
const close: PluginDuplexChannelCloseParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
};
|
|
expect(close.workerSessionId).toBe("ws-1");
|
|
});
|
|
|
|
it("rejects a close request that omits the host route identifier", () => {
|
|
// @ts-expect-error — hostRouteId is the authoritative close key and is required.
|
|
const close: PluginDuplexChannelCloseParams = { workerSessionId: "ws-1" };
|
|
expect(close).toBeDefined();
|
|
});
|
|
});
|
|
|
|
describe("duplex channel notification schemas", () => {
|
|
it("uses the generic notification method names", () => {
|
|
expect(DUPLEX_CHANNEL_DATA_NOTIFICATION).toBe("duplexChannel.data");
|
|
expect(DUPLEX_CHANNEL_EXIT_NOTIFICATION).toBe("duplexChannel.exit");
|
|
});
|
|
|
|
it("accepts a valid data notification that carries the exact pair", () => {
|
|
const data: PluginDuplexChannelDataParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
chunk: "output bytes",
|
|
};
|
|
expect(data.chunk).toBe("output bytes");
|
|
});
|
|
|
|
it("rejects a data notification that omits the host route identifier", () => {
|
|
// @ts-expect-error — hostRouteId is required on a notification.
|
|
const data: PluginDuplexChannelDataParams = {
|
|
workerSessionId: "ws-1",
|
|
chunk: "output bytes",
|
|
};
|
|
expect(data).toBeDefined();
|
|
});
|
|
|
|
it("rejects a data notification that omits the chunk", () => {
|
|
// @ts-expect-error — chunk is required.
|
|
const data: PluginDuplexChannelDataParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
};
|
|
expect(data).toBeDefined();
|
|
});
|
|
|
|
it("accepts an exit notification with a numeric code and with null", () => {
|
|
const exit: PluginDuplexChannelExitParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
exitCode: 0,
|
|
};
|
|
const exitNull: PluginDuplexChannelExitParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
exitCode: null,
|
|
};
|
|
expect(exit.exitCode).toBe(0);
|
|
expect(exitNull.exitCode).toBeNull();
|
|
});
|
|
|
|
it("rejects an exit notification that omits the host route identifier", () => {
|
|
// @ts-expect-error — hostRouteId is required on a notification.
|
|
const exit: PluginDuplexChannelExitParams = {
|
|
workerSessionId: "ws-1",
|
|
exitCode: 0,
|
|
};
|
|
expect(exit).toBeDefined();
|
|
});
|
|
|
|
it("rejects an exit notification with a non-numeric, non-null exit code", () => {
|
|
const exit: PluginDuplexChannelExitParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
// @ts-expect-error — exitCode must be a number or null.
|
|
exitCode: "0",
|
|
};
|
|
expect(exit).toBeDefined();
|
|
});
|
|
});
|
|
|
|
// JSON-RPC travels as one line of JSON text (see `serializeMessage`), and JSON
|
|
// carries no binary type. `chunk` and `data` cross this hop as a base64 string
|
|
// (`ChannelBytesWireValue`); `encodeChannelBytes`/`decodeChannelBytes` are the
|
|
// one shared codec both ends use.
|
|
describe("duplex channel byte-safe wire representation", () => {
|
|
it("test_json_rpc_hop_preserves_all_byte_values", () => {
|
|
const allByteValues = Uint8Array.from({ length: 256 }, (_, value) => value);
|
|
|
|
// Encode the full byte corpus into a data notification, exactly as
|
|
// worker-rpc-host.ts does at the worker→host boundary, then send it through
|
|
// the same newline-delimited JSON serialization the host and the worker use
|
|
// over stdio.
|
|
const params: PluginDuplexChannelDataParams = {
|
|
hostRouteId: "route-1",
|
|
workerSessionId: "ws-1",
|
|
chunk: encodeChannelBytes(allByteValues),
|
|
};
|
|
const line = serializeMessage(
|
|
createNotification(DUPLEX_CHANNEL_DATA_NOTIFICATION, params),
|
|
);
|
|
|
|
// Parse the line back, exactly as the reading side does, and decode the
|
|
// chunk back to raw bytes.
|
|
const parsed = parseMessage(line);
|
|
expect("params" in parsed).toBe(true);
|
|
const receivedParams = (parsed as { params: PluginDuplexChannelDataParams }).params;
|
|
const decoded = decodeChannelBytes(receivedParams.chunk);
|
|
|
|
expect(decoded).not.toBeNull();
|
|
expect(decoded).toEqual(allByteValues);
|
|
// The byte value zero is the case a UTF-8 string hop loses or mistreats as a
|
|
// terminator. Assert it by name, not only as part of the full-corpus check.
|
|
expect(decoded?.[0]).toBe(0);
|
|
});
|
|
|
|
it("round-trips an empty byte chunk through encode and decode", () => {
|
|
const empty = new Uint8Array(0);
|
|
const encoded = encodeChannelBytes(empty);
|
|
expect(encoded).toBe("");
|
|
expect(decodeChannelBytes(encoded)).toBeNull();
|
|
});
|
|
|
|
it("decodeChannelBytes rejects a malformed wire value", () => {
|
|
expect(decodeChannelBytes("not valid base64!!")).toBeNull();
|
|
expect(decodeChannelBytes(undefined)).toBeNull();
|
|
expect(decodeChannelBytes(42)).toBeNull();
|
|
});
|
|
});
|