Files
PaperClipAI/packages/shared/src/validators/plugin.test.ts
T
Nicky LeachandPaperclip 233be4b36c feat: parallelize sandbox file-sync behind a provider opt-in capability (#11736)
## Thinking Path

> - Paperclip runs AI agents through local and remote execution
adapters.
> - Sandbox providers move workspace and asset files before and after
agent runs.
> - Serial file transfers delay startup and teardown when several
operations do not depend on each other.
> - Providers need an opt-in contract so existing providers keep their
serial behavior.
> - This pull request adds a bounded scheduler and routes inbound and
outbound sync operations through it.
> - The benefit is shorter sandbox setup and teardown with stable
errors, clear telemetry, and a safe opt-in path.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting (multiple of the above): packages/shared,
packages/adapter-utils, packages/plugins, and server.

**Problem or motivation**

Sandbox sync processes the workspace, assets, and referenced projects in
series. This adds avoidable wait time to agent startup and teardown.

**Proposed solution**

Add a fail-closed provider capability named concurrentSyncOperations.
Use a bounded scheduler with a limit of four operations. Preserve
operation order for error reporting. Keep non-opted-in providers on the
serial path.

**Alternatives considered**

Increase the serial transfer speed or add provider-specific schedulers.
Those options do not provide one shared contract or stable behavior
across providers.

**Roadmap alignment**

ROADMAP.md lists cloud and sandbox agents as a product area. This change
improves sandbox execution without changing the control-plane contract.

**Additional context**

The Daytona provider opts in. Board trials on this commit showed overlap
for inbound sync and outbound restore, with no referenced-project
staging failures.

## What Changed

- Add the concurrentSyncOperations sandbox capability and fail-closed
parsing.
- Add a bounded settle-all scheduler with stable input-order errors.
- Parallelize inbound workspace, asset, and referenced-project sync
operations when the provider opts in.
- Parallelize outbound workspace and asset restore operations when the
provider opts in.
- Surface referenced-project failure text in run logs and server
telemetry.
- Add Daytona sync spans and the capability declaration.
- Preserve in-flight upload scratch tarballs during workspace wipe.
- Add unit and regression tests for the scheduler, coordinators,
provider behavior, telemetry, and wipe race.

## Verification

- Run the adapter-utils and server type checks.
- Run the targeted adapter-utils, server, and Daytona test suites.
- Run the full automated sweep.
- Review six cold Daytona trials, with three serial and three parallel
runs.
- Confirm that parallel trials show inbound overlap and outbound restore
overlap.
- Confirm that providers without the capability keep serial behavior.

## Risks

- Providers must opt in only when their file operations can run safely
at the same time.
- A provider that declares the capability incorrectly can expose
transfer races.
- The scheduler keeps a limit of four to bound resource use.
- Providers without the capability keep the prior serial behavior.

## Model Used

OpenAI GPT-5 in the Codex runtime. The model used tool calls, code
inspection, and GitHub workflow support. The model did not author the
implementation commits.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: # / Closes #
/ Refs # OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub #NNN / github.com/paperclipai/paperclip URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-19 12:34:11 -07:00

425 lines
14 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { PLUGIN_CAPABILITIES } from "../constants.js";
import { resolveDeclaredSandboxCapabilities } from "../environment-support.js";
import { pluginManagedRoutineDeclarationSchema, pluginManifestV1Schema, pluginUiSlotDeclarationSchema } from "./plugin.js";
function buildSandboxProviderManifest(driver: Record<string, unknown>) {
return {
id: "paperclip.capability-provider",
apiVersion: 1,
version: "0.1.0",
displayName: "Capability Provider",
description: "Sandbox provider that declares fine-grained capabilities.",
author: "Paperclip",
categories: ["automation"],
capabilities: ["environment.drivers.register"],
entrypoints: { worker: "./dist/worker.js" },
environmentDrivers: [
{
driverKey: "capability-provider",
kind: "sandbox_provider",
displayName: "Capability Provider",
configSchema: { type: "object" },
...driver,
},
],
};
}
describe("plugin capability constants", () => {
it("exposes each capability once", () => {
expect(new Set(PLUGIN_CAPABILITIES).size).toBe(PLUGIN_CAPABILITIES.length);
});
});
describe("plugin manifest validators", () => {
it("accepts existing-style plugins that do not request access or authorization capabilities", () => {
const parsed = pluginManifestV1Schema.parse({
id: "paperclip.compat-dashboard",
apiVersion: 1,
version: "0.1.0",
displayName: "Compat Dashboard",
description: "Dashboard-only plugin without access or authorization host APIs.",
author: "Paperclip",
categories: ["ui"],
capabilities: ["ui.dashboardWidget.register"],
entrypoints: {
worker: "./dist/worker.js",
ui: "./dist/ui.js",
},
ui: {
slots: [
{
type: "dashboardWidget",
id: "compat-dashboard",
displayName: "Compat Dashboard",
exportName: "CompatDashboard",
},
],
},
});
expect(parsed.capabilities).toEqual(["ui.dashboardWidget.register"]);
});
it("accepts sandbox provider template config bindings", () => {
const parsed = pluginManifestV1Schema.parse({
id: "paperclip.template-provider",
apiVersion: 1,
version: "0.1.0",
displayName: "Template Provider",
description: "Sandbox provider with captured template config binding.",
author: "Paperclip",
categories: ["automation"],
capabilities: ["environment.drivers.register"],
entrypoints: { worker: "./dist/worker.js" },
environmentDrivers: [
{
driverKey: "template-provider",
kind: "sandbox_provider",
displayName: "Template Provider",
supportsTemplateCapture: true,
templateRefKind: "provider_template",
templateConfigBinding: {
field: "templateId",
unsetFields: ["image"],
},
configSchema: { type: "object" },
},
],
});
expect(parsed.environmentDrivers?.[0]?.templateConfigBinding).toEqual({
field: "templateId",
unsetFields: ["image"],
});
});
it("rejects template config bindings that replace provider identity", () => {
const parsed = pluginManifestV1Schema.safeParse({
id: "paperclip.bad-template-provider",
apiVersion: 1,
version: "0.1.0",
displayName: "Bad Template Provider",
categories: ["automation"],
capabilities: ["environment.drivers.register"],
entrypoints: { worker: "./dist/worker.js" },
environmentDrivers: [
{
driverKey: "bad-template-provider",
kind: "sandbox_provider",
displayName: "Bad Template Provider",
templateConfigBinding: {
field: "provider",
},
configSchema: { type: "object" },
},
],
});
expect(parsed.success).toBe(false);
if (parsed.success) return;
expect(parsed.error.issues.some((issue) => issue.message.includes("provider key"))).toBe(true);
});
});
describe("plugin managed routine validators", () => {
it("accepts core issue surface visibility values in routine templates", () => {
const parsed = pluginManagedRoutineDeclarationSchema.parse({
routineKey: "wiki.refresh",
title: "Refresh Wiki",
activityGatePolicy: "require_external_activity",
activityGateScope: "project",
issueTemplate: { surfaceVisibility: "default" },
});
expect(parsed.issueTemplate?.surfaceVisibility).toBe("default");
expect(parsed.activityGatePolicy).toBe("require_external_activity");
expect(parsed.activityGateScope).toBe("project");
});
it("rejects non-core issue surface visibility values in routine templates", () => {
const parsed = pluginManagedRoutineDeclarationSchema.safeParse({
routineKey: "wiki.refresh",
title: "Refresh Wiki",
issueTemplate: { surfaceVisibility: "normal" },
});
expect(parsed.success).toBe(false);
});
});
describe("plugin managed skill validators", () => {
const baseManifest = {
id: "paperclip.test-managed-skills",
apiVersion: 1,
version: "0.1.0",
displayName: "Managed Skills",
description: "Managed skills test plugin.",
author: "Paperclip",
categories: ["automation"],
entrypoints: { worker: "./dist/worker.js" },
} as const;
it("requires skills.managed when managed skills are declared", () => {
const parsed = pluginManifestV1Schema.safeParse({
...baseManifest,
capabilities: [],
skills: [{ skillKey: "wiki-maintainer", displayName: "Wiki Maintainer" }],
});
expect(parsed.success).toBe(false);
if (parsed.success) return;
expect(parsed.error.issues.some((issue) => issue.message.includes("skills.managed"))).toBe(true);
});
it("accepts managed skills with the skills.managed capability", () => {
const parsed = pluginManifestV1Schema.parse({
...baseManifest,
capabilities: ["skills.managed"],
skills: [{ skillKey: "wiki-maintainer", displayName: "Wiki Maintainer" }],
});
expect(parsed.skills?.[0]?.skillKey).toBe("wiki-maintainer");
});
});
describe("plugin UI slot validators", () => {
it("accepts route-scoped sidebar slots with a routePath", () => {
const parsed = pluginUiSlotDeclarationSchema.parse({
type: "routeSidebar",
id: "wiki-route-sidebar",
displayName: "Wiki Sidebar",
exportName: "WikiSidebar",
routePath: "wiki",
});
expect(parsed.routePath).toBe("wiki");
});
it("requires route-scoped sidebar slots to declare a routePath", () => {
const parsed = pluginUiSlotDeclarationSchema.safeParse({
type: "routeSidebar",
id: "wiki-route-sidebar",
displayName: "Wiki Sidebar",
exportName: "WikiSidebar",
});
expect(parsed.success).toBe(false);
if (parsed.success) return;
expect(parsed.error.issues[0]?.message).toBe("routeSidebar slots require routePath");
});
it("keeps reserved company route protection for route-scoped sidebars", () => {
const parsed = pluginUiSlotDeclarationSchema.safeParse({
type: "routeSidebar",
id: "settings-route-sidebar",
displayName: "Settings Sidebar",
exportName: "SettingsSidebar",
routePath: "settings",
});
expect(parsed.success).toBe(false);
if (parsed.success) return;
expect(parsed.error.issues.some((issue) => issue.message.includes("reserved by the host"))).toBe(true);
});
it("accepts workspace entity types as detailTab targets", () => {
const parsed = pluginUiSlotDeclarationSchema.parse({
type: "detailTab",
id: "workspace-diff-viewer",
displayName: "Diff",
exportName: "WorkspaceDiffViewer",
entityTypes: ["execution_workspace", "project_workspace"],
});
expect(parsed.entityTypes).toEqual(["execution_workspace", "project_workspace"]);
});
it("accepts execution_workspace as a toolbarButton entityType", () => {
const parsed = pluginUiSlotDeclarationSchema.parse({
type: "toolbarButton",
id: "workspace-open-diff",
displayName: "Open diff",
exportName: "OpenWorkspaceDiffButton",
entityTypes: ["execution_workspace"],
});
expect(parsed.entityTypes).toEqual(["execution_workspace"]);
});
it("accepts company settings page slots with a non-core settings route", () => {
const parsed = pluginUiSlotDeclarationSchema.parse({
type: "companySettingsPage",
id: "permissions-settings",
displayName: "Permissions",
exportName: "PermissionsSettingsPage",
routePath: "permissions",
});
expect(parsed.routePath).toBe("permissions");
});
it("prevents company settings page slots from shadowing core settings routes", () => {
const parsed = pluginUiSlotDeclarationSchema.safeParse({
type: "companySettingsPage",
id: "instance-settings",
displayName: "Instance",
exportName: "InstanceSettingsPage",
routePath: "instance",
});
expect(parsed.success).toBe(false);
if (parsed.success) return;
expect(parsed.error.issues.some((issue) => issue.message.includes("reserved by the host"))).toBe(true);
});
});
describe("sandbox provider capability declaration validators", () => {
it("test_manifest_accepts_sandbox_capabilities_and_rejects_unknown_capability_keys", () => {
const parsed = pluginManifestV1Schema.parse(
buildSandboxProviderManifest({
sandboxCapabilities: {
reusableLeases: true,
nativeSyncIn: true,
nativeSyncOut: false,
persistentProcessSessions: true,
independentControlCommands: false,
incrementalSessionOutput: true,
},
}),
);
expect(parsed.environmentDrivers?.[0]?.sandboxCapabilities).toEqual({
reusableLeases: true,
nativeSyncIn: true,
nativeSyncOut: false,
persistentProcessSessions: true,
independentControlCommands: false,
incrementalSessionOutput: true,
});
const rejected = pluginManifestV1Schema.safeParse(
buildSandboxProviderManifest({
sandboxCapabilities: {
reusableLeases: true,
// A typo or unknown capability name must fail validation, not drop
// silently. The nested schema is `.strict()`.
nativeSync: true,
},
}),
);
expect(rejected.success).toBe(false);
});
it("test_manifest_accepts_concurrent_sync_operations_capability", () => {
// A provider opts in to parallel bidirectional file sync with this key. The
// strict schema accepts it and keeps the declared value.
const parsed = pluginManifestV1Schema.parse(
buildSandboxProviderManifest({
sandboxCapabilities: { concurrentSyncOperations: true },
}),
);
expect(parsed.environmentDrivers?.[0]?.sandboxCapabilities).toEqual({
concurrentSyncOperations: true,
});
});
it("test_manifest_rejects_unknown_sync_concurrency_capability_key", () => {
// A neighboring but unknown concurrency key must fail validation, not drop
// silently. The strict schema rejects a capability the host does not honor.
const rejected = pluginManifestV1Schema.safeParse(
buildSandboxProviderManifest({
sandboxCapabilities: { concurrentSyncAndExec: true },
}),
);
expect(rejected.success).toBe(false);
});
it("test_supports_reusable_leases_compat_maps_to_reusable_leases", () => {
const parsed = pluginManifestV1Schema.parse(
buildSandboxProviderManifest({ supportsReusableLeases: true }),
);
const driver = parsed.environmentDrivers?.[0];
expect(driver?.sandboxCapabilities).toBeUndefined();
expect(resolveDeclaredSandboxCapabilities(driver!).reusableLeases).toBe(true);
});
it("test_sandbox_capabilities_reusable_leases_wins_over_compat_field", () => {
const parsed = pluginManifestV1Schema.parse(
buildSandboxProviderManifest({
supportsReusableLeases: true,
sandboxCapabilities: { reusableLeases: false },
}),
);
const driver = parsed.environmentDrivers?.[0];
// The nested declaration wins over the legacy compat flag when both exist.
expect(resolveDeclaredSandboxCapabilities(driver!).reusableLeases).toBe(false);
});
});
describe("login pty transport capability and legacy alias", () => {
it("test_login_pty_new_field_parses_and_carries_the_flag", () => {
const parsed = pluginManifestV1Schema.parse(
buildSandboxProviderManifest({ supportsLoginPty: true }),
);
expect(parsed.environmentDrivers?.[0]?.supportsLoginPty).toBe(true);
});
it("test_legacy_alias_only_canonicalizes_onto_login_pty", () => {
const parsed = pluginManifestV1Schema.parse(
buildSandboxProviderManifest({ supportsSetupTokenLogin: true }),
);
const driver = parsed.environmentDrivers?.[0];
// The validator maps the deprecated alias onto the canonical field at parse
// time, and it drops the alias so a downstream reader cannot read the old
// name.
expect(driver?.supportsLoginPty).toBe(true);
expect(driver).not.toHaveProperty("supportsSetupTokenLogin");
});
it("test_conflicting_alias_and_new_field_reject", () => {
const rejected = pluginManifestV1Schema.safeParse(
buildSandboxProviderManifest({
supportsLoginPty: true,
supportsSetupTokenLogin: false,
}),
);
expect(rejected.success).toBe(false);
});
it("test_matching_alias_and_new_field_pass", () => {
const parsed = pluginManifestV1Schema.parse(
buildSandboxProviderManifest({
supportsLoginPty: true,
supportsSetupTokenLogin: true,
}),
);
expect(parsed.environmentDrivers?.[0]?.supportsLoginPty).toBe(true);
});
it("test_unknown_login_field_misspelling_rejects", () => {
const rejected = pluginManifestV1Schema.safeParse(
buildSandboxProviderManifest({ supportsLoginPTY: true }),
);
expect(rejected.success).toBe(false);
});
it("test_login_pty_accepts_literal_booleans_only", () => {
const rejected = pluginManifestV1Schema.safeParse(
buildSandboxProviderManifest({ supportsLoginPty: "true" }),
);
expect(rejected.success).toBe(false);
});
});