Files
PaperClipAI/server/src/__tests__/heartbeat-execution-configuration.test.ts
T
DottaandClaude Opus 5.5 2d1c2db64a feat(mcp): host the MCP Skills extension for connected apps (#21219)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents use connected apps through MCP. Paperclip already hosts MCP
Apps views and speaks stateless MCP 2026-07-28.
> - The MCP Skills extension (SEP-2640, Final) lets an app publish
skills (instructions and files) next to its tools. Hugging Face already
serves 25.
> - Paperclip did not read them, so agents missed each app's own
workflow guidance.
> - Skills are a prompt-injection surface. The spec makes manifest
binding, verification, origin tagging and approval revocation host
requirements.
> - This pull request makes Paperclip a Skills host: discover, review,
verify, and deliver app skills to the agents that use the app.
> - The benefit is that connected apps can teach agents their workflows,
under board control.

## Linked Issues or Issue Description

**Problem**
Connected MCP servers can publish skills with the
`io.modelcontextprotocol/skills` extension. Paperclip ignored them. A
board could not see, review or approve them, and agents never received
them.

**Proposed solution**
Implement the host side of SEP-2640. List skills during catalog refresh,
record them per connection and company, bind board approval to the full
file manifest, and load approved skills into agent runs with verified,
lazily fetched content.

**Why it matters**
Apps ship workflow guidance with their tools. Without the extension,
each team must copy that guidance into company skills by hand, and it
goes stale.

Refs #21119, Refs #21140 (the MCP Apps and stateless MCP client work
this builds on).

## What Changed

- **Client capability.** Every stateless request declares
`io.modelcontextprotocol/skills`. Legacy handshake sessions do not. The
capabilities that `server/discover` advertises are remembered beside the
protocol era.
- **Discovery.** Catalog refresh calls `skills/list` (paged) only when
the server declared the extension. A listing inside its `ttlMs` is
reused. A skills failure never fails the tool refresh. Nothing runs
while a run starts.
- **Schema.** New `tool_mcp_skills` (company, connection, skill URI;
frontmatter, sorted manifest, manifest digest, review state) and
`tool_mcp_skill_files` (verified bytes keyed by connection, URI and
digest). Both cascade with the connection.
- **Governance.** New skills wait for review. Approval binds to the
order-free digest of every `{uri, digest, size}`; a request for any
other digest gets 409. A changed manifest or frontmatter revokes
approval and logs `tool_mcp_skill.approval_revoked`. Dynamic, oversized
and invalid skills are declined with a reason. Approve and decline use
the company skill policy (`skills.install`, `external_package`,
`mcp-connection:<id>`).
- **Run time.** Approved skills of connections the agent can use load in
parallel with other skill preparation, inside a 3 s budget. `SKILL.md`
is read with the agent's grant and checked for size, SHA-256 and
frontmatter. A failure calls `skills/get` and takes the skill out of
use. Each skill becomes a per-run bundle on the connector skill path,
named in its app's namespace, tagged with its origin. Paperclip writes
the frontmatter: `allowed-tools` only if the board approved it.
- **Supporting files.** `GET
/api/agents/me/mcp-skills/:skillId/files?path=` serves manifest-listed
files to an agent in an active run, through the skill's own connection.
The sandbox bridge allows the route.
- **Board API and UI.** `GET/POST /api/companies/:companyId/mcp-skills…`
(list, read instructions, approve, decline). The Skills page shows
"Skills from connected apps" with app, state, size, requested tools and
plain-language notes. It renders nothing when no app publishes skills.
- **Docs.** `doc/connections/MCP-SKILLS.md`, linked from
`doc/connections/README.md`.
- **Shared frontmatter parser.** It now decodes YAML single-quoted
scalars. Before, 5 of Hugging Face's 25 skills kept their quotes and
would fail the frontmatter check; now all 25 match. The generated runner
copy is regenerated.

File size notes: `server/src/services/tool-gateway.ts` grows by about
200 lines. The run-time reads must use its grant resolution and
`callAssignedConnectionProtocol`, like MCP Apps view reads. Splitting
the gateway is a separate refactor. `CompanySkills.tsx` and
`CompanySkills.production.tsx` gain 2–4 lines each; the section lives in
its own component.

## Verification

- `server/src/__tests__/mcp-skills.test.ts` (15 tests) uses a fake
stateless skills server and embedded Postgres. It covers: discovery only
after the server declares the extension; the capability in `_meta`;
`ttlMs` reuse; lazy fetch (no reads at listing or approval; `SKILL.md`
at load; supporting files on read); a stale-digest approval refused; a
reorder that keeps approval; a changed or added file that revokes it;
dynamic skills declined; a same-size tampered `SKILL.md` rejected,
refreshed with `skills/get`, and taken out of use; mismatched
frontmatter rejected; reads outside the manifest refused; the same URI
and name from two servers kept apart; company scoping; a slow app
skipped within the budget and cached later; HTTP authorization (board
only, agent needs an active run, run header must match) and activity
logging.
- Red/green: with the digest check disabled, the tamper test fails; with
the manifest comparison disabled, the revocation test fails. Both pass
with the code.
- `server/src/__tests__/mcp-protocol-versions.test.ts`,
`mcp-apps.test.ts`, `connection-instructions.test.ts`,
`tool-gateway-discovery-http.test.ts`: 47 pass (two capability
assertions updated for the new extension).
- `packages/shared/src/frontmatter.test.ts` adds a single-quoted scalar
case; the runner contract test passes (23 tests).
- `ui/src/components/skills/ConnectedAppSkills.test.tsx` (3) and
`ui/src/pages/CompanySkills.test.tsx` (44) pass.
`packages/adapter-utils/src/sandbox-callback-bridge.test.ts` (89)
passes.
- `pnpm --filter @paperclipai/server typecheck` passes. UI `tsc` passes
except the known missing generated Storybook catalog files.
- `pnpm verify` returned exit 2 (doc change is a global CI input);
required CI supplies broad coverage.
- Live check against `https://huggingface.co/mcp` without credentials,
through Paperclip's own services and the real egress guard: 25 skills
discovered as pending; three approved; cold load 135 ms, cached load 13
ms; a supporting file read through the manifest; the bundle tagged with
its origin.
- Storybook: `Agents and reusable capabilities/Skills and Skill
Studio/Connected app skills` (needs review, all states, tool request,
name collision, reading instructions).

## Risks

- **Migration:** two new tables only. No changes to existing tables.
Safe with the previous release running.
- **Run start:** companies without an approved app skill pay one indexed
read. With approved skills the wait is capped at 3 s; late or failing
skills are left out with a `skills.mcp_unavailable` run warning and
never fail the run.
- **Agent harness enforcement:** Paperclip cannot gate each
code-execution tool call an agent makes while following an app's skill.
Approval is explicit and per skill, the approval note says agents may
run commands the skill describes, and `allowed-tools` is never granted
without a separate tick.
- **Shared-home adapters** receive app skills like company skills. A
revoked skill leaves on the next skill sync.
- Not in this PR: per-connection opt-in for dynamic skills; serving
Paperclip's own skills from its public MCP endpoint;
`resources/directory/read`; app skills on an agent's skills tab.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, local test runs, live HTTP checks).
Extended reasoning enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 19:18:06 -06:00

880 lines
64 KiB
TypeScript

import { enableRunFailureReporting } from "./helpers/run-failure-reporting.js";
import { createHash, randomUUID } from "node:crypto";
import { trace } from "@opentelemetry/api";
import { InMemorySpanExporter, SimpleSpanProcessor, TracerProvider } from "@opentelemetry/sdk-trace";
import { access, chmod, mkdir, readdir, readFile, mkdtemp, rm, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { eq } from "drizzle-orm";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { agents, authUsers, companies, companyMemberships, companySkills, companySkillTestRuns, createDb, executionWorkspaces, heartbeatRunEvents, heartbeatRuns, issues, projects, plugins, environments } from "@paperclipai/db";
import { companySkillService } from "../services/company-skills.js";
import { skillVersionCachePaths, withSkillVersionCacheLock } from "../services/skill-version-cache.js";
import { registerServerAdapter, unregisterServerAdapter, type AdapterExecutionContext } from "../adapters/index.js";
import { HttpError } from "../errors.js";
import { heartbeatService } from "../services/heartbeat.js";
import { bindHeartbeatRunnerHomeMode, prepareHeartbeatExecutionConfiguration, type HeartbeatExecutionConfigurationInput } from "../services/heartbeat/execution-configuration.js";
import { buildEffectiveRunSessionConfigMetadata, resolveTaskSessionConfigFreshness } from "../services/heartbeat/workspaces.js";
import { aiConnectionBindingSchema, type Environment } from "@paperclipai/shared";
import { secretService } from "../services/secrets.js";
import { instanceSettingsService } from "../services/instance-settings.js";
import { aiConnectionService } from "../services/ai-connections.js";
import * as aiRuntime from "../services/ai-connection-runtime.js";
import * as connectorRuntime from "../services/connector-runtime.js";
import * as assignedMcp from "../services/native-runtime/assigned-mcp-tools.js";
import * as sentry from "../sentry.js";
import { waitForPendingRunFailureReports } from "../services/run-failure-report.js";
import { recoveryPostgresSupport, startRecoveryPostgres } from "./helpers/recovery-postgres.js";
import { selectHeartbeatEnvironment, type HeartbeatEnvironmentSelectionInput } from "../services/heartbeat/environment-selection.js";
import { environmentService } from "../services/environments.js";
import { executionWorkspaceService } from "../services/execution-workspaces.js";
import { buildExecutionWorkspaceAdapterConfig } from "../services/execution-workspace-policy.js";
import { buildNativeExecutionInput } from "../services/native-runtime/native-execution-input.js";
import { nativeRuntimeContextFixture } from "../services/native-runtime/runtime-context.test-fixture.js";
function deferred<T>() {
let resolve!: (value: T) => void;
const promise = new Promise<T>(complete => { resolve = complete; });
return { promise, resolve };
}
const skillFailures = vi.hoisted(() => new Map<string, string>());
const skillWaits = vi.hoisted(() => new Map<string, () => Promise<void>>());
describe("Runner home mode during recovery preparation", () => {
const fingerprint = (effectiveAdapterConfig: Record<string, unknown>) => buildEffectiveRunSessionConfigMetadata({
adapterType: "paperclip_runner", effectiveAdapterConfig, agentRuntimeConfig: {}, issueOverrides: null,
workspaceConfig: {}, environment: null, environmentEnv: null, projectEnv: null, routineEnv: null, runtimeSkills: [],
});
it.each([
{ schema: "paperclip.native-execution-input.v5" },
{ schema: "paperclip.native-execution-input.v7", homeMode: "isolated" },
])("preserves the implicit isolated fingerprint through recovery and the next run: %j", async saved => {
const config = { provider: "codex", model: "fixture-model" };
// This is the effective configuration shape built by preparation, including
// its optional property when no explicit home mode was configured.
const prepare = (input: unknown) => ({ ...config, homeMode: bindHeartbeatRunnerHomeMode(config, input).homeMode });
const fresh = await fingerprint(prepare(null));
const recovered = await fingerprint(prepare(saved));
const next = await fingerprint(prepare(null));
expect(recovered.fingerprint).toBe(fresh.fingerprint);
expect(next.fingerprint).toBe(recovered.fingerprint);
expect(resolveTaskSessionConfigFreshness({ hasTaskSession: true, configuredModel: config.model,
taskSessionParams: { __paperclipConfiguredModel: config.model, __paperclipConfigFingerprint: recovered.fingerprint,
__paperclipConfigFingerprintVersion: recovered.version, __paperclipConfigCategoryFingerprints: recovered.categoryFingerprints },
configMetadata: next })).toMatchObject({ reset: false, changedCategories: [] });
expect(config).not.toHaveProperty("homeMode");
});
it("pins the admitted isolated mode but invalidates a genuinely preconfigured next run", async () => {
const config = { provider: "codex", model: "fixture-model", homeMode: "preconfigured" };
const recoveredConfig = bindHeartbeatRunnerHomeMode(config, { schema: "paperclip.native-execution-input.v5" });
expect(recoveredConfig.homeMode).toBe("isolated");
expect(bindHeartbeatRunnerHomeMode(config, null).homeMode).toBe("preconfigured");
const recovered = await fingerprint(recoveredConfig);
const next = await fingerprint(config);
expect(next.fingerprint).not.toBe(recovered.fingerprint);
expect(resolveTaskSessionConfigFreshness({ hasTaskSession: true, configuredModel: config.model,
taskSessionParams: { __paperclipConfiguredModel: config.model, __paperclipConfigFingerprint: recovered.fingerprint,
__paperclipConfigFingerprintVersion: recovered.version, __paperclipConfigCategoryFingerprints: recovered.categoryFingerprints },
configMetadata: next })).toMatchObject({ reset: true, changedCategories: ["adapterConfig"] });
});
it.each([
["preconfigured", { schema: "paperclip.native-execution-input.v5" }, "isolated"],
["preconfigured", { schema: "paperclip.native-execution-input.v7", homeMode: "isolated" }, "isolated"],
["isolated", { schema: "paperclip.native-execution-input.v7", homeMode: "preconfigured" }, "preconfigured"],
])("prepares the saved mode after current configuration changes to %s", (current, saved, expected) => {
const config = { homeMode: current, model: "current-model", env: { TASK_VALUE: "kept" } };
// Called before router selection and again after mutable task overrides.
const routed = bindHeartbeatRunnerHomeMode(config, saved);
const prepared = bindHeartbeatRunnerHomeMode({ ...routed, homeMode: current }, saved);
expect(prepared).toEqual({ ...config, homeMode: expected });
expect(config.homeMode).toBe(current);
});
it.each(["isolated", "preconfigured", undefined])("uses current mode %s for a new run", homeMode => {
const config = { homeMode };
expect(bindHeartbeatRunnerHomeMode(config, null)).toBe(config);
expect(bindHeartbeatRunnerHomeMode(config, {})).toBe(config);
});
});
vi.mock("../services/company-skills.js", async importOriginal => {
const actual = await importOriginal<typeof import("../services/company-skills.js")>();
return { ...actual, companySkillService: (...args: Parameters<typeof actual.companySkillService>) => {
const service = actual.companySkillService(...args);
return { ...service, withRuntimeSkillEntries: async <T>(companyId: string,
options: Parameters<typeof service.withRuntimeSkillEntries>[1],
consume: (entries: Awaited<ReturnType<typeof service.listRuntimeSkillEntries>>) => T | Promise<T>) => {
await skillWaits.get(companyId)?.();
const failure = skillFailures.get(companyId);
if (failure) throw new Error(failure);
return service.withRuntimeSkillEntries(companyId, options, consume);
} };
} };
});
const support = recoveryPostgresSupport;
const describePostgres = support.supported ? describe : describe.skip;
if (!support.supported) console.warn(`Skipping execution configuration tests: ${support.reason}`);
describePostgres("heartbeat execution configuration through adapter dispatch", () => {
let temporary: Awaited<ReturnType<typeof startRecoveryPostgres>>;
let db: ReturnType<typeof createDb>;
let heartbeat: ReturnType<typeof heartbeatService>;
const captured = new Map<string, AdapterExecutionContext>();
const inspect = new Map<string, (input: AdapterExecutionContext) => Promise<void>>();
const adapterType = "codex_local";
const oldHome = process.env.PAPERCLIP_HOME;
let home: string;
beforeAll(async () => {
temporary = await startRecoveryPostgres("heartbeat-execution-config-");
db = createDb(temporary.connectionString);
home = await mkdtemp(path.join(tmpdir(), "paperclip-execution-skills-"));
process.env.PAPERCLIP_HOME = home;
heartbeat = heartbeatService(db);
// Exercise real preparation for a managed-capable adapter without calling a provider.
registerServerAdapter({
type: adapterType, supportsLocalAgentJwt: false,
execute: async input => {
captured.set(input.runId, input);
await db.update(issues).set({ status: "in_review" }).where(eq(issues.id, String(input.context.issueId)));
await inspect.get(input.agent.id)?.(input);
return { exitCode: 0, signal: null, timedOut: false, resultJson: {} };
},
testEnvironment: async () => ({ adapterType, status: "pass", checks: [], testedAt: new Date().toISOString() }),
});
}, 60_000);
afterEach(() => { vi.restoreAllMocks(); vi.unstubAllEnvs(); skillFailures.clear(); skillWaits.clear(); });
afterAll(async () => {
await heartbeat?.drainActiveRunExecutions();
unregisterServerAdapter(adapterType);
await db?.$client.end({ timeout: 0 });
await temporary?.cleanup();
if (oldHome === undefined) delete process.env.PAPERCLIP_HOME; else process.env.PAPERCLIP_HOME = oldHome;
if (home) await rm(home, { recursive: true, force: true });
});
async function seed(managed = false) {
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), projectId = randomUUID();
const userId = `configuration-user-${companyId}`;
const credential = `fixture-api-key-${companyId}`;
await db.insert(authUsers).values({ id: userId, name: "Configuration user", email: `${userId}@example.test`,
createdAt: new Date(), updatedAt: new Date() });
await db.insert(companies).values({ id: companyId, name: "Execution configuration fixture", issuePrefix: `C${companyId.slice(0, 6)}` });
await db.insert(companyMemberships).values({ companyId, principalId: userId, principalType: "user", status: "active", membershipRole: "member" });
const account = managed ? await aiConnectionService(db).save(companyId, userId, {
provider: "openai", method: "api_key", ownership: "personal", name: "Fixture account", apiKey: credential, agentIds: [], allAgents: true,
}, credential) : null;
await db.insert(projects).values({ id: projectId, companyId, name: "Config project", env: { SHARED_VALUE: "project", PROJECT_ONLY: "project-only" } });
await db.insert(agents).values({ id: agentId, companyId, name: "Configuration reader", adapterType, status: "idle",
adapterConfig: { model: "agent-model", env: { SHARED_VALUE: "agent", AGENT_ONLY: "agent-only", ...(managed ? { OPENAI_API_KEY: "stale-agent-key" } : {}) } },
runtimeConfig: account ? { aiConnection: { provider: "openai", method: "api_key", mode: "delegated", connectionId: account.connectionId, grantId: account.grantId } } : {},
});
await db.insert(issues).values({ id: issueId, companyId, projectId, title: "Configuration fixture", status: "todo", assigneeAgentId: agentId, responsibleUserId: userId, issueNumber: 1, assigneeAdapterOverrides: { adapterConfig: { model: "issue-model" } } });
return { companyId, agentId, issueId, userId, credential };
}
async function execute(f: Awaited<ReturnType<typeof seed>>) {
const run = await heartbeat.wakeup(f.agentId, { source: "on_demand", triggerDetail: "manual", reason: "manual", payload: { issueId: f.issueId }, contextSnapshot: {
issueId: f.issueId, paperclipSecrets: { manifest: [{ secretKey: "untrusted-wake-value" }] },
paperclipWake: { connectorSkillInstructions: "untrusted wake instructions" },
} });
expect(run).not.toBeNull();
await heartbeat.drainActiveRunExecutions();
const [finished] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run!.id));
expect(finished).toBeDefined();
return { finished, input: captured.get(run!.id) };
}
it.each([
{ saved: "isolated", managed: true, failure: "saved managed login required" },
{ saved: "isolated", managed: false, failure: "saved isolated skills required" },
{ saved: "preconfigured", managed: true, failure: null },
{ saved: null, managed: true, failure: "saved managed login required" },
] as const)("retains $saved preparation after configuration changes (managed: $managed)", async ({ saved, managed, failure }) => {
const f = await seed(managed);
const [agent] = await db.update(agents).set({ adapterType: "paperclip_runner", adapterConfig: {
provider: "codex", model: "gpt-6.1-sol", homeMode: saved === "isolated" ? "preconfigured" : "isolated",
} }).where(eq(agents.id, f.agentId)).returning();
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId,
status: "running", runtimeMode: "native", contextSnapshot: { issueId: f.issueId },
}).returning();
const prepare = vi.spyOn(aiRuntime, "prepareManagedAiRuntime").mockRejectedValue(new Error("saved managed login required"));
skillFailures.set(f.companyId, "saved isolated skills required");
// Preparation only reads the persisted mode and whether recovery is active.
const persisted = saved === null ? null : { schema: saved === "isolated" ? "paperclip.native-execution-input.v5" : "paperclip.native-execution-input.v7",
...(saved === "preconfigured" ? { homeMode: saved } : {}),
} as unknown as HeartbeatExecutionConfigurationInput["environment"]["persistedNativeExecutionInput"];
const localEnvironment = await environmentService(db).ensureLocalEnvironment();
const opening = prepareHeartbeatExecutionConfiguration(db, {
run, agent, task: { issueId: f.issueId, context: {}, responsibleUserId: f.userId,
acceptedPlanContinuationWake: false, acceptedPlanWakeRoutingDecision: null, taskProjectContext: null,
routineEnvContext: { routineId: null, env: null, responsibleUserId: null } },
config: { sourceWorkspaceConfig: { ...agent.adapterConfig, homeMode: "preconfigured" }, aiBinding: managed
? aiConnectionBindingSchema.parse(agent.runtimeConfig!.aiConnection) : undefined },
policy: { trustPreset: { kind: "standard", preset: "standard", boundary: null, sourcePresets: {} },
useIsolatedTaskDirectory: false, explicitTaskDirectory: false, requestedExecutionWorkspaceMode: "shared_workspace" },
environment: { selectedEnvironmentId: localEnvironment.id, selectedEnvironmentForConfig: localEnvironment,
resolvedInstanceSettings: await instanceSettingsService(db).get(), persistedNativeExecutionInput: persisted },
services: { secretsSvc: secretService(db), companySkills: companySkillService(db), appendRunEvent: vi.fn(), finalizeAiConnectionBusyDeferral: vi.fn() },
onFailureSecretsResolved: vi.fn(), onManagedAiRuntimeAcquired: vi.fn(), onIdentityRedactorPrepared: vi.fn(),
});
if (failure) await expect(opening).rejects.toThrow(failure);
else {
expect(await opening).toMatchObject({ prepared: true, runtimeConfig: { homeMode: "preconfigured" }, runtimeSkillEntries: [] });
expect(prepare).not.toHaveBeenCalled();
}
if (managed && saved === "isolated") expect(prepare).toHaveBeenCalledOnce();
});
it.each(["agent", "stored task override"] as const)("uses operator preconfigured mode despite %s configuration", async source => {
const f = await seed(true);
const nativeHome = await mkdtemp(path.join(tmpdir(), "paperclip-preconfigured-preparation-"));
const workspace = path.join(nativeHome, "workspace");
await mkdir(workspace);
process.env.PAPERCLIP_HOME = nativeHome;
await db.update(agents).set({ adapterType: "paperclip_runner", adapterConfig: {
provider: "codex", model: "gpt-6.1-sol", homeMode: "preconfigured", cwd: workspace,
} }).where(eq(agents.id, f.agentId));
if (source === "stored task override") {
await db.update(issues).set({ assigneeAdapterOverrides: { adapterConfig: { homeMode: "isolated" } } }).where(eq(issues.id, f.issueId));
// Old task data cannot change the operator's mode before AI routing.
await db.update(agents).set({ runtimeConfig: { aiConnection: { mode: "router", connectionId: "unconfigured-pool" } } }).where(eq(agents.id, f.agentId));
}
skillFailures.set(f.companyId, "Preconfigured mode must not resolve assigned skills");
const prepare = vi.spyOn(aiRuntime, "prepareManagedAiRuntime");
const beforeNative = vi.fn(async () => {
// The fixture intentionally stops preparation; do not schedule a retry.
await db.update(agents).set({ status: "paused" }).where(eq(agents.id, f.agentId));
throw new Error("fixture stopped before provider start");
});
const nativeHeartbeat = heartbeatService(db, { beforeNativeRuntimeSelection: beforeNative });
try {
const run = await nativeHeartbeat.wakeup(f.agentId, { source: "on_demand", triggerDetail: "manual", reason: "manual",
payload: { issueId: f.issueId }, contextSnapshot: { issueId: f.issueId } });
expect(run).not.toBeNull();
await nativeHeartbeat.drainActiveRunExecutions();
const saved = await nativeHeartbeat.getRun(run!.id);
expect(saved?.error).toContain("fixture stopped before provider start");
expect(beforeNative).toHaveBeenCalledOnce();
expect(prepare).not.toHaveBeenCalled();
expect(saved?.contextSnapshot?.aiConnection).toBeUndefined();
} finally {
await nativeHeartbeat.drainActiveRunExecutions();
process.env.PAPERCLIP_HOME = home;
// Native preparation seals context bundles; release only this test's tree.
const release = async (directory: string): Promise<void> => {
await chmod(directory, 0o700);
for (const entry of await readdir(directory, { withFileTypes: true })) {
if (entry.isDirectory()) await release(path.join(directory, entry.name));
}
};
await release(nativeHome);
await rm(nativeHome, { recursive: true, force: true });
}
});
async function runnerTaskEnvironment(supportsTasks = true): Promise<Environment> {
const pluginId = randomUUID(), pluginKey = `test.runner-tasks.${pluginId}`;
await db.insert(plugins).values({ id: pluginId, pluginKey, packageName: "@test/runner-tasks", version: "1.0.0", apiVersion: 1,
categories: ["automation"], status: "ready", manifestJson: { id: pluginKey, apiVersion: 1, version: "1.0.0",
displayName: "Runner tasks", description: "Runner task fixture", author: "Paperclip", categories: ["automation"],
capabilities: ["environment.drivers.register"], entrypoints: { worker: "worker.js" },
environmentDrivers: [{ driverKey: "tasks", displayName: "Tasks", supportsTasks, configSchema: { type: "object" } }] } });
const [environment] = await db.insert(environments).values({ name: `Runner tasks ${pluginId}`, driver: "plugin", status: "active",
config: { pluginKey, driverKey: "tasks", driverConfig: {} } }).returning();
return environment as unknown as Environment;
}
it.each([
{ name: "a run without a managed AI connection", managed: false, homeMode: undefined, persisted: false, supportsTasks: true, refused: true },
{ name: "a preconfigured home that drops the managed AI connection", managed: true, homeMode: "preconfigured", persisted: false, supportsTasks: true, refused: true },
{ name: "a run with a managed AI connection", managed: true, homeMode: undefined, persisted: false, supportsTasks: true, refused: false },
{ name: "resumed native input admitted before the rule", managed: false, homeMode: undefined, persisted: true, supportsTasks: true, refused: false },
{ name: "a plugin environment without Runner tasks", managed: false, homeMode: undefined, persisted: false, supportsTasks: false, refused: false },
] as const)("plugin Runner-task environments require a managed AI connection: $name", async ({ managed, homeMode, persisted, supportsTasks, refused }) => {
const f = await seed(managed);
const environment = await runnerTaskEnvironment(supportsTasks);
const config = { provider: "codex", model: "gpt-6.1-sol", ...(homeMode ? { homeMode } : {}) };
const [agent] = await db.update(agents).set({ adapterType: "paperclip_runner", adapterConfig: config }).where(eq(agents.id, f.agentId)).returning();
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, issueId: f.issueId, scopeKind: "issue",
status: "running", runtimeMode: "native", contextSnapshot: { issueId: f.issueId } }).returning();
const reached = "fixture passed managed AI admission";
const prepare = vi.spyOn(aiRuntime, "prepareManagedAiRuntime").mockRejectedValue(new Error(reached));
skillFailures.set(f.companyId, reached);
const secretsResolved = vi.fn(), redactorPrepared = vi.fn();
const opening = prepareHeartbeatExecutionConfiguration(db, {
run, agent, task: { issueId: f.issueId, context: {}, responsibleUserId: f.userId,
acceptedPlanContinuationWake: false, acceptedPlanWakeRoutingDecision: null, taskProjectContext: null,
routineEnvContext: { routineId: null, env: null, responsibleUserId: null } },
config: { sourceWorkspaceConfig: config, aiBinding: managed ? aiConnectionBindingSchema.parse(agent.runtimeConfig!.aiConnection) : undefined },
policy: { trustPreset: { kind: "standard", preset: "standard", boundary: null, sourcePresets: {} },
useIsolatedTaskDirectory: false, explicitTaskDirectory: false, requestedExecutionWorkspaceMode: "shared_workspace" },
environment: { selectedEnvironmentId: environment.id, selectedEnvironmentForConfig: environment,
resolvedInstanceSettings: await instanceSettingsService(db).get(),
persistedNativeExecutionInput: persisted ? { schema: "paperclip.native-execution-input.v5" } as unknown as HeartbeatExecutionConfigurationInput["environment"]["persistedNativeExecutionInput"] : null },
services: { secretsSvc: secretService(db), companySkills: companySkillService(db), appendRunEvent: vi.fn(), finalizeAiConnectionBusyDeferral: vi.fn() },
onFailureSecretsResolved: secretsResolved, onManagedAiRuntimeAcquired: vi.fn(), onIdentityRedactorPrepared: redactorPrepared,
});
if (refused) {
await expect(opening).rejects.toMatchObject({ name: "ConfigurationIncompleteFailure", code: "configuration_incomplete",
message: expect.stringContaining(homeMode ? "isolated Runner home" : "Choose an AI connection"),
resultJson: { configurationIncomplete: { reason: "ai_connection_required", actionUrl: `/agents/${f.agentId}/runtime`,
...(homeMode ? { homeMode: "preconfigured" } : {}) } } });
// Refused before observability, secrets, or the managed runtime.
expect(redactorPrepared).not.toHaveBeenCalled();
expect(secretsResolved).not.toHaveBeenCalled();
expect(prepare).not.toHaveBeenCalled();
} else {
await expect(opening).rejects.toThrow(reached);
expect(redactorPrepared).toHaveBeenCalledOnce();
}
});
async function configuration(f: Awaited<ReturnType<typeof seed>>, config: Record<string, unknown>, snapshot: Record<string, unknown> = {}, type = "paperclip_runner", explicitTaskDirectory = false, requestedExecutionWorkspaceMode: HeartbeatExecutionConfigurationInput["policy"]["requestedExecutionWorkspaceMode"] = "shared_workspace", selectedEnvironmentForConfig: HeartbeatExecutionConfigurationInput["environment"]["selectedEnvironmentForConfig"] = null, pinnedSkillTest?: HeartbeatExecutionConfigurationInput["task"]["pinnedSkillTest"], adoptingLiveRunner = false) {
const [agent] = await db.update(agents).set({ adapterType: type, adapterConfig: config }).where(eq(agents.id, f.agentId)).returning();
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, issueId: f.issueId, scopeKind: "issue", status: "running", contextSnapshot: snapshot }).returning();
const resolvedInstanceSettings = await instanceSettingsService(db).get();
const appendRunEvent = vi.fn();
const environment = selectedEnvironmentForConfig ?? await environmentService(db).ensureLocalEnvironment();
return { run, appendRunEvent, prepare: () => prepareHeartbeatExecutionConfiguration(db, {
run, agent,
task: { issueId: f.issueId, context: { ...snapshot, issueId: f.issueId }, responsibleUserId: null, pinnedSkillTest, adoptingLiveRunner,
acceptedPlanContinuationWake: false, acceptedPlanWakeRoutingDecision: null,
taskProjectContext: null, routineEnvContext: { routineId: null, env: null, responsibleUserId: null } },
config: { sourceWorkspaceConfig: config, aiBinding: undefined },
policy: { trustPreset: { kind: "standard", preset: "standard", boundary: null, sourcePresets: {} }, useIsolatedTaskDirectory: false, explicitTaskDirectory, requestedExecutionWorkspaceMode },
environment: { selectedEnvironmentId: environment.id, selectedEnvironmentForConfig: environment,
resolvedInstanceSettings, persistedNativeExecutionInput: adoptingLiveRunner
// Preparation reads only whether the admitted input carries its runtime context.
? { schema: "paperclip.native-execution-input.v7", runtimeContext: {} } as unknown as HeartbeatExecutionConfigurationInput["environment"]["persistedNativeExecutionInput"]
: null },
services: { secretsSvc: secretService(db), companySkills: companySkillService(db), appendRunEvent, finalizeAiConnectionBusyDeferral: vi.fn() },
onFailureSecretsResolved: () => {}, onManagedAiRuntimeAcquired: () => {}, onIdentityRedactorPrepared: () => {},
}) };
}
it("delivers the Studio pin instead of ordinary assignments and recovers that exact native selection", async () => {
const f = await seed(), skills = companySkillService(db);
const selected = await skills.createLocalSkill(f.companyId, { name: "Unassigned Studio skill", markdown: "# Studio\nOriginal selected bytes.\n" });
const other = await skills.createLocalSkill(f.companyId, { name: "Ordinary assignment" });
const pin = { key: selected.key, versionId: selected.currentVersionId! };
await skills.updateFile(f.companyId, selected.id, "SKILL.md", "# Studio\nNewer head.\n");
const config = { paperclipSkillSync: { desiredSkills: [other.key] } };
await instanceSettingsService(db).updateExperimental({ enableBetaSkills: false });
const opening = await configuration(f, config, {}, "paperclip_runner", false, "shared_workspace", null, pin);
const result = await opening.prepare();
expect(result.prepared).toBe(true);
if (!result.prepared) throw new Error("Configuration did not finish");
expect(result.runtimeSkillEntries).toEqual([expect.objectContaining({ key: selected.key, versionId: pin.versionId })]);
expect(await readFile(path.join(result.runtimeSkillEntries[0]!.source, "SKILL.md"), "utf8")).toContain("Original selected bytes.");
const [saved] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, opening.run.id));
expect(saved.contextSnapshot!.skillVersions).toEqual({ [selected.key]: pin.versionId });
const restoring = await configuration(f, { paperclipSkillSync: { desiredSkills: [] } }, saved.contextSnapshot!, "paperclip_runner");
expect(await restoring.prepare()).toMatchObject({ runtimeSkillEntries: [expect.objectContaining({ key: selected.key, versionId: pin.versionId })] });
const preconfigured = await configuration(f, { ...config, homeMode: "preconfigured" }, {}, "paperclip_runner", false, "shared_workspace", null, pin);
await expect(preconfigured.prepare()).rejects.toThrow("isolated Runner home");
await db.update(companySkills).set({ accessSourceId: randomUUID() }).where(eq(companySkills.id, selected.id));
const denied = await configuration(f, config, {}, "paperclip_runner", false, "shared_workspace", null, pin);
await expect(denied.prepare()).rejects.toThrow("pinned skill is unavailable or no longer permitted");
}, 60_000);
it("dispatches an unassigned Studio revision from its durable harness row", async () => {
const f = await seed(), skills = companySkillService(db);
const selected = await skills.createLocalSkill(f.companyId, { name: "Studio delivery", markdown: "# Studio\nRead scripts/prove.sh.\n" });
const script = "#!/bin/sh\nprintf 'studio-pinned-proof\\n'\n";
const saved = await skills.updateFile(f.companyId, selected.id, "scripts/prove.sh", script, null, { executable: true });
const other = await skills.createLocalSkill(f.companyId, { name: "Assigned instead" });
await db.update(agents).set({ adapterConfig: { paperclipSkillSync: { desiredSkills: [other.key] } } }).where(eq(agents.id, f.agentId));
await db.update(issues).set({ workMode: "skill_test", harnessKind: "skill_test" }).where(eq(issues.id, f.issueId));
await db.insert(companySkillTestRuns).values({ companyId: f.companyId, skillId: selected.id, skillVersionId: saved.versionId!,
agentId: f.agentId, issueId: f.issueId, inputSnapshot: "Run the saved script." });
await skills.updateFile(f.companyId, selected.id, "scripts/prove.sh", "#!/bin/sh\necho newer\n", null, { executable: true });
const { finished, input } = await execute(f);
expect(finished.status, finished.error ?? undefined).toBe("succeeded");
const delivered = input?.config.paperclipRuntimeSkills as Array<{ key: string; source: string; versionId: string }>;
const testSkill = delivered.find(entry => entry.key === selected.key)!;
expect(testSkill).toMatchObject({ versionId: saved.versionId });
expect(delivered.some(entry => entry.key === other.key)).toBe(false);
expect(await readFile(path.join(testSkill.source, "scripts/prove.sh"), "utf8")).toBe(script);
expect((await stat(path.join(testSkill.source, "scripts/prove.sh"))).mode & 0o111).not.toBe(0);
expect(finished.contextSnapshot).toMatchObject({ skillVersions: { [selected.key]: saved.versionId },
paperclipSkillTest: { skillKey: selected.key, skillVersionId: saved.versionId } });
const [unchangedAgent] = await db.select().from(agents).where(eq(agents.id, f.agentId));
expect(unchangedAgent.adapterConfig).toMatchObject({ paperclipSkillSync: { desiredSkills: [other.key] } });
}, 60_000);
it("overlaps skills with assignments and dependent connection instructions before composition", async () => {
const f = await seed();
const skillGate = deferred<void>();
skillWaits.set(f.companyId, () => skillGate.promise);
const assignments = vi.spyOn(connectorRuntime, "resolveConnectorAssignments");
const instructions = vi.spyOn(assignedMcp, "resolveAssignedConnectionInstructionsForRun");
const configured = await configuration(f, { provider: "codex", model: "gpt-6.1-sol", homeMode: "isolated" });
const opening = configured.prepare();
try {
await vi.waitFor(() => {
expect(assignments).toHaveBeenCalledOnce();
expect(instructions).toHaveBeenCalledOnce();
});
} finally {
skillGate.resolve();
}
expect(await opening).toMatchObject({ prepared: true, runtimeConfig: { homeMode: "isolated", model: "gpt-6.1-sol" } });
});
it("settles concurrent configuration reads before exposing the first preparation failure", async () => {
const f = await seed();
const assignmentGate = deferred<Awaited<ReturnType<typeof connectorRuntime.resolveConnectorAssignments>>>();
const assignments = vi.spyOn(connectorRuntime, "resolveConnectorAssignments").mockReturnValue(assignmentGate.promise);
const instructions = vi.spyOn(assignedMcp, "resolveAssignedConnectionInstructionsForRun").mockRejectedValue(new Error("later instruction failure"));
skillFailures.set(f.companyId, "first skill failure");
const configured = await configuration(f, { provider: "codex", homeMode: "isolated" });
let settled = false;
const opening = configured.prepare();
void opening.then(() => { settled = true; }, () => { settled = true; });
try {
await vi.waitFor(() => expect(assignments).toHaveBeenCalledOnce());
await Promise.resolve();
expect(settled).toBe(false);
expect(instructions).not.toHaveBeenCalled();
} finally {
assignmentGate.resolve([]);
}
await expect(opening).rejects.toThrow("first skill failure");
});
it("does not install new environment declarations when recovering a saved skill selection", async () => {
const f = await seed(), pluginId = randomUUID(), pluginKey = `fixture.${randomUUID()}`;
await db.insert(plugins).values({ id: pluginId, pluginKey, packageName: "fixture", version: "1.0.0", status: "ready",
manifestJson: { id: pluginKey, apiVersion: 1, version: "1.0.0", displayName: "Fixture", description: "Fixture", author: "Tests", categories: ["automation"],
capabilities: ["environment.drivers.register", "skills.managed"], entrypoints: { worker: "worker.js" },
environmentDrivers: [{ driverKey: "tasks", displayName: "Tasks", configSchema: {}, skillKeys: ["new-invalid"] }],
skills: [{ skillKey: "new-invalid", displayName: "New invalid skill", markdown: "---\nname: new-invalid\nmetadata:\n sources:\n - kind: github-dir\n repo: fixture/skills\n---\nUnpinned." }] } });
const [environment] = await db.insert(environments).values({ name: "Fixture", driver: "plugin", config: { pluginKey, driverKey: "tasks", driverConfig: {} } }).returning();
const config = { paperclipSkillSync: { desiredSkills: [] } };
const restoring = await configuration(f, config, { skillVersions: {}, skillSelectionSnapshot: true }, "paperclip_runner", false, "shared_workspace", environment as Environment);
expect(await restoring.prepare()).toMatchObject({ prepared: true, runtimeSkillEntries: [] });
const fresh = await configuration(f, config, {}, "paperclip_runner", false, "shared_workspace", environment as Environment);
const rejected = await fresh.prepare().then(() => false, error => { expect(error.message).toMatch(/pinned Git commit/); return true; });
expect(rejected).toBe(true);
});
it("reports a shadowed assignment and captures only the required plugin skill", async () => {
const f = await seed(), pluginId = randomUUID(), pluginKey = `fixture.${randomUUID()}`;
const name = `plugin-${pluginKey.split(".").at(-1)}-desktop`;
const existing = await companySkillService(db).createLocalSkill(f.companyId, { name, markdown: `---\nname: ${name}\n---\nCompany skill.` });
await db.insert(plugins).values({ id: pluginId, pluginKey, packageName: "fixture", version: "1.0.0", status: "ready",
manifestJson: { id: pluginKey, apiVersion: 1, version: "1.0.0", displayName: "Fixture", description: "Fixture", author: "Tests", categories: ["automation"],
capabilities: ["environment.drivers.register", "skills.managed"], entrypoints: { worker: "worker.js" },
environmentDrivers: [{ driverKey: "tasks", displayName: "Tasks", configSchema: {}, skillKeys: ["desktop"] }],
skills: [{ skillKey: "desktop", displayName: "Desktop" }] } });
const [environment] = await db.insert(environments).values({ name: randomUUID(), driver: "plugin", config: { pluginKey, driverKey: "tasks" } }).returning();
await db.update(issues).set({ description: `Use [company skill](skill://${existing.id}).` }).where(eq(issues.id, f.issueId));
const config = { paperclipSkillSync: { desiredSkills: [existing.key] } };
const run = await configuration(f, config, {}, "paperclip_runner", false, "shared_workspace", environment as Environment);
const result = await run.prepare();
expect(result.prepared).toBe(true);
if (!result.prepared) throw new Error("Configuration did not finish");
const key = `plugin/${pluginKey.replaceAll(".", "-")}/desktop`;
expect(result.runtimeSkillEntries.map(skill => skill.key)).toEqual([key]);
expect(run.appendRunEvent).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ eventType: "skills.shadowed", level: "warn",
message: expect.stringContaining(existing.key), payload: { shadowedSkills: [{ key: existing.key, shadowedBy: key }] } }));
const [saved] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.run.id));
expect(saved.contextSnapshot!.shadowedSkills).toEqual([{ key: existing.key, shadowedBy: key }]);
expect(Object.keys(saved.contextSnapshot!.skillVersions as object)).toEqual([key]);
const recovery = await configuration(f, config, saved.contextSnapshot!, "paperclip_runner", false, "shared_workspace", environment as Environment);
const resumed = await recovery.prepare();
expect(resumed.prepared).toBe(true);
if (!resumed.prepared) throw new Error("Recovery did not finish");
expect(resumed.runtimeSkillEntries.map(skill => skill.key)).toEqual([key]);
expect(recovery.appendRunEvent.mock.calls.some(([, event]) => event.eventType === "skills.shadowed")).toBe(false);
});
it("keeps default task-directory configuration stable when the directory is reused", async () => {
const f = await seed();
const initial = await (await configuration(f, {}, {}, "paperclip_runner", true)).prepare();
const reused = await (await configuration(f, {})).prepare();
expect(initial.prepared).toBe(true);
expect(reused.prepared).toBe(true);
if (!initial.prepared || !reused.prepared) throw new Error("Configuration was not prepared");
expect(initial.mergedConfig).toEqual(reused.mergedConfig);
expect(initial.configSnapshot).toEqual(reused.configSnapshot);
});
it("keeps shared provider workspace configuration across native recovery and the next turn", async () => {
const f = await seed();
const selectedSkills = [];
for (const name of ["Alphabetically first long skill", "B longer selected skill", "C", "D small"]) {
selectedSkills.push(await companySkillService(db).createLocalSkill(f.companyId, { name, markdown: "# Keep the saved result\n" }));
}
const instructionsRootPath = path.join(home, f.companyId);
await mkdir(instructionsRootPath);
await writeFile(path.join(instructionsRootPath, "AGENTS.md"), "Preserve the workspace across controller replacement.\n");
const config = { provider: "codex", model: "fixture-model", lifecycleMode: "warm", idleTimeoutMs: 300_000,
codexPermissionMode: "full", paperclipSkillSync: { desiredSkills: selectedSkills.map(skill => skill.key) },
instructionsRootPath, instructionsFilePath: path.join(instructionsRootPath, "AGENTS.md"),
instructionsEntryFile: "AGENTS.md", instructionsBundleMode: "managed", env: { OPENAI_API_KEY: "fixture-direct-key" } };
const [environment] = await db.insert(environments).values({ name: `Directory ${f.companyId}`, driver: "plugin",
config: { pluginKey: "test.directory", driverKey: "tasks" } }).returning();
const [agent] = await db.update(agents).set({ adapterType: "paperclip_runner", adapterConfig: config,
defaultEnvironmentId: environment.id }).where(eq(agents.id, f.agentId)).returning();
const [issue] = await db.select().from(issues).where(eq(issues.id, f.issueId));
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId,
issueId: f.issueId, scopeKind: "issue", status: "running", runtimeMode: "native", responsibleUserId: f.userId, contextSnapshot: {} }).returning();
const workspaceId = run.id;
const [workspace] = await db.insert(executionWorkspaces).values({ id: workspaceId, companyId: f.companyId,
projectId: issue.projectId, sourceIssueId: issue.id, name: "Shared task workspace", mode: "shared_workspace",
strategyType: "project_primary", providerType: "adapter_managed", cwd: `/workspaces/${workspaceId}` }).returning();
const issueSettings = { mode: "shared_workspace" as const };
const sourceConfig = (current: Record<string, unknown>) => buildExecutionWorkspaceAdapterConfig({ agentConfig: current,
projectPolicy: null, issueSettings, mode: "shared_workspace", legacyUseProjectWorkspace: null });
const selectionInput: HeartbeatEnvironmentSelectionInput = {
run, agent, task: { issueRef: { ...issue, executionWorkspaceId: null }, issueContext: issue, issueId: issue.id,
executionProjectId: issue.projectId, workspaceProjectContext: { hasWorkspace: false }, boundSourceWorkspace: null,
hasSelectedWorkspaceSource: false, taskWorkspaceIntent: null, responsibleUserId: f.userId, context: {},
freshWorkspace: null },
policy: { trustPreset: { kind: "standard", preset: "standard", boundary: null, sourcePresets: {} },
requestedExecutionWorkspaceMode: "shared_workspace", isolatedWorkspacesEnabled: true,
projectExecutionWorkspacePolicy: null, issueExecutionWorkspaceSettings: issueSettings },
config: { config, sourceWorkspaceConfig: sourceConfig(config), issueAssigneeOverrides: null },
services: { environmentsSvc: environmentService(db), executionWorkspacesSvc: executionWorkspaceService(db),
instanceSettings: instanceSettingsService(db), findSharedWorkspaceHolder: vi.fn(async () => null),
findExecutionWorkspaceHolder: vi.fn(async () => null) },
};
const prepare = async () => {
const selected = await selectHeartbeatEnvironment(db, selectionInput);
const result = await prepareHeartbeatExecutionConfiguration(db, { run, agent,
task: { issueId: issue.id, context: {}, responsibleUserId: f.userId, acceptedPlanContinuationWake: false,
acceptedPlanWakeRoutingDecision: null, taskProjectContext: null,
routineEnvContext: { routineId: null, env: null, responsibleUserId: null } },
config: { sourceWorkspaceConfig: selectionInput.config.sourceWorkspaceConfig, aiBinding: undefined },
policy: { trustPreset: selectionInput.policy.trustPreset, useIsolatedTaskDirectory: selected.useIsolatedTaskDirectory,
explicitTaskDirectory: selected.explicitTaskDirectory, requestedExecutionWorkspaceMode: selectionInput.policy.requestedExecutionWorkspaceMode },
environment: selected, services: { secretsSvc: secretService(db), companySkills: companySkillService(db),
appendRunEvent: vi.fn(), finalizeAiConnectionBusyDeferral: vi.fn() },
onFailureSecretsResolved: () => {}, onManagedAiRuntimeAcquired: () => {}, onIdentityRedactorPrepared: () => {} });
if (!result.prepared) throw new Error("Configuration was not prepared");
const metadata = await buildEffectiveRunSessionConfigMetadata({ adapterType: "paperclip_runner",
effectiveAdapterConfig: result.runtimeConfig, agentRuntimeConfig: {}, issueOverrides: null, workspaceConfig: {},
environment: null, environmentEnv: null, projectEnv: null, routineEnv: null, runtimeSkills: result.runtimeSkillEntries });
return { selected, result, metadata };
};
const first = await prepare();
expect(first.selected.useIsolatedTaskDirectory).toBe(false);
expect(first.result.runtimeConfig.workspaceStrategy).toBeUndefined();
// Preserve the pre-fix fresh name/key query order, including exact versions.
expect(first.result.runtimeConfig.paperclipSkillSync).toEqual({ desiredSkills: selectedSkills.map(skill => ({
key: skill.key, versionId: skill.currentVersionId,
})) });
// Read the actual JSONB snapshot rather than reusing the pre-persistence
// JavaScript map: PostgreSQL key order differs for these four real keys.
const [saved] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id));
run.contextSnapshot = saved.contextSnapshot;
selectionInput.task.freshWorkspace = null;
selectionInput.task.issueRef!.executionWorkspaceId = workspaceId;
selectionInput.task.issueRef!.executionWorkspacePreference = "reuse_existing";
await db.update(issues).set({ executionWorkspaceId: workspaceId, executionWorkspacePreference: "reuse_existing" }).where(eq(issues.id, issue.id));
selectionInput.run.runnerProfileJson = { nativeExecutionInput: buildNativeExecutionInput({
companyId: f.companyId, agentId: agent.id, runId: run.id, normalizedSessionId: randomUUID(), provider: "codex",
issue: { id: issue.id, identifier: "CFG-1", title: issue.title, description: null, workMode: "standard" },
taskPrompt: "Continue the admitted work", workspace: { id: workspaceId, cwd: workspace.cwd!, repoUrl: null, repoRef: null, branchName: null },
completionContract: { id: randomUUID(), sha256: `sha256:${"a".repeat(64)}`, schemaVersion: "paperclip.run-result.v1",
contract: { revision: "1", objective: "Retain the directory", criteria: [{ id: "objective", requirement: "Preserve the workspace" }] } },
runtimeContext: nativeRuntimeContextFixture(),
}) };
const recovered = await prepare();
expect(recovered.selected.nativeRecoveryExecutionWorkspaceId).toBe(workspaceId);
expect(recovered.selected.useIsolatedTaskDirectory).toBe(false);
expect(recovered.result.runtimeSkillEntries).toEqual(first.result.runtimeSkillEntries);
expect(resolveTaskSessionConfigFreshness({ hasTaskSession: true, configuredModel: config.model,
taskSessionParams: { __paperclipConfiguredModel: config.model, __paperclipConfigFingerprint: first.metadata.fingerprint,
__paperclipConfigFingerprintVersion: first.metadata.version, __paperclipConfigCategoryFingerprints: first.metadata.categoryFingerprints },
configMetadata: recovered.metadata })).toMatchObject({ reset: false, changedCategories: [] });
expect(recovered.result.runtimeConfig).toEqual(first.result.runtimeConfig);
expect(recovered.metadata.fingerprint).toBe(first.metadata.fingerprint);
selectionInput.run.runnerProfileJson = {};
run.contextSnapshot = {};
const next = await prepare();
expect(next.result.runtimeConfig).toEqual(first.result.runtimeConfig);
expect(next.metadata.fingerprint).toBe(first.metadata.fingerprint);
selectionInput.config.sourceWorkspaceConfig = sourceConfig({ ...config,
paperclipSkillSync: { desiredSkills: selectedSkills.slice(1).map(skill => skill.key) } });
const fewerSkills = await prepare();
expect(fewerSkills.result.runtimeSkillEntries).toHaveLength(3);
expect(fewerSkills.metadata.fingerprint).not.toBe(first.metadata.fingerprint);
// A real model change must still invalidate the next admission.
selectionInput.config.config = { ...config, model: "changed-model" };
selectionInput.config.sourceWorkspaceConfig = sourceConfig(selectionInput.config.config);
const changed = await prepare();
expect(changed.selected.useIsolatedTaskDirectory).toBe(false);
expect(changed.result.runtimeConfig.model).toBe("changed-model");
expect(changed.metadata.fingerprint).not.toBe(first.metadata.fingerprint);
}, 30_000);
it.each([
{ mode: "shared_workspace" as const, config: { workspaceStrategy: { type: "git_worktree", baseRef: "main" } } },
{ mode: "agent_default" as const, config: {} },
])("still selects directory transport instead of a $mode strategy", async ({ mode, config }) => {
const f = await seed();
const prepared = await (await configuration(f, config, {}, "paperclip_runner", true, mode)).prepare();
expect(prepared.prepared).toBe(true);
if (!prepared.prepared) throw new Error("Configuration was not prepared");
expect(prepared.mergedConfig.workspaceStrategy).toEqual({ type: "project_primary" });
});
it.each(["fresh", "saved empty"])("prepares a %s native skill selection while another run holds the company cache lock", async kind => {
const f = await seed();
const selected = kind === "saved empty" ? await companySkillService(db).createLocalSkill(f.companyId, { name: "Later assignment" }) : null;
const input = await configuration(f, selected ? { paperclipSkillSync: { desiredSkills: [selected.key] } } : {},
kind === "saved empty" ? { skillVersions: {}, skillSelectionSnapshot: true } : {});
let release!: () => void;
let acquired!: () => void;
const ready = new Promise<void>(resolve => { acquired = resolve; });
const barrier = new Promise<void>(resolve => { release = resolve; });
const holder = withSkillVersionCacheLock(db, f.companyId, async () => { acquired(); await barrier; });
await ready;
const started = performance.now();
const pending = input.prepare();
let timer: ReturnType<typeof setTimeout> | undefined;
try {
const result = await Promise.race([pending, new Promise<null>(resolve => { timer = setTimeout(() => resolve(null), 2000); })]);
expect(result, "An empty native selection must not queue behind skill hydration").not.toBeNull();
console.info("Empty-skill preparation under a held cache lock", { kind, elapsedMs: performance.now() - started });
expect(result?.prepared).toBe(true);
if (result?.prepared) expect(result.runtimeSkillEntries).toEqual([]);
const [saved] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, input.run.id));
expect(saved.contextSnapshot).toMatchObject({ skillVersions: {}, skillSelectionSnapshot: true });
} finally {
if (timer) clearTimeout(timer);
release();
await holder;
await pending;
}
});
it.each(["selected native", "legacy operational"])("retains the cache exclusion for %s skills", async kind => {
const f = await seed();
const selected = await companySkillService(db).createLocalSkill(f.companyId, { name: "Selected lock control" });
const input = await configuration(f, { paperclipSkillSync: { desiredSkills: kind === "selected native" ? [selected.key] : [] } }, {}, kind === "selected native" ? "paperclip_runner" : "codex_local");
let release!: () => void;
let acquired!: () => void;
const ready = new Promise<void>(resolve => { acquired = resolve; });
const barrier = new Promise<void>(resolve => { release = resolve; });
const holder = withSkillVersionCacheLock(db, f.companyId, async () => { acquired(); await barrier; });
await ready;
let settled = false;
const pending = input.prepare().finally(() => { settled = true; });
try {
await new Promise(resolve => setTimeout(resolve, 100));
expect(settled).toBe(false);
} finally { release(); await holder; }
const result = await pending;
expect(result.prepared).toBe(true);
if (result.prepared) expect(result.runtimeSkillEntries.some(entry => entry.key === (kind === "selected native" ? selected.key : "paperclipai/paperclip/paperclip"))).toBe(true);
});
it("reconnects to a live remote Runner with its admitted skills without re-downloading them or waiting on the cache", async () => {
const f = await seed(), skills = companySkillService(db);
const selected = await skills.createLocalSkill(f.companyId, { name: "Admitted skill", markdown: "# Admitted\nOriginal bytes.\n" });
const config = { provider: "codex", model: "gpt-6.1-sol", homeMode: "isolated", paperclipSkillSync: { desiredSkills: [selected.key] } };
const admitted = await configuration(f, config);
const first = await admitted.prepare();
if (!first.prepared) throw new Error("Admission did not finish");
const [saved] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, admitted.run.id));
const versionId = (saved.contextSnapshot!.skillVersions as Record<string, string>)[selected.key]!;
expect(versionId).toBeTruthy();
// A newly deployed server starts with an empty skill cache.
const cached = skillVersionCachePaths(f.companyId, selected.id, versionId).directory;
await rm(cached, { recursive: true, force: true });
const fingerprint = (entries: typeof first.runtimeSkillEntries) => buildEffectiveRunSessionConfigMetadata({
adapterType: "paperclip_runner", effectiveAdapterConfig: {}, agentRuntimeConfig: {}, issueOverrides: null,
workspaceConfig: {}, environment: null, environmentEnv: null, projectEnv: null, routineEnv: null, runtimeSkills: entries,
}).then(metadata => metadata.fingerprint);
let release!: () => void;
let acquired!: () => void;
const ready = new Promise<void>(resolve => { acquired = resolve; });
const barrier = new Promise<void>(resolve => { release = resolve; });
const holder = withSkillVersionCacheLock(db, f.companyId, async () => { acquired(); await barrier; });
await ready;
const adopting = await configuration(f, config, saved.contextSnapshot!, "paperclip_runner", false, "shared_workspace", null, undefined, true);
const normal = await configuration(f, config, saved.contextSnapshot!, "paperclip_runner");
let normalSettled = false;
const normalPending = normal.prepare().finally(() => { normalSettled = true; });
const adoptingPending = adopting.prepare();
let timer: ReturnType<typeof setTimeout> | undefined;
try {
const result = await Promise.race([adoptingPending, new Promise<null>(resolve => { timer = setTimeout(() => resolve(null), 5000); })]);
expect(result, "A live Runner reattach must not queue behind skill downloads").not.toBeNull();
if (!result?.prepared) throw new Error("Reattach preparation did not finish");
expect(result.runtimeSkillEntries).toEqual([expect.objectContaining({ key: selected.key, versionId, sourceStatus: "available",
runtimeName: first.runtimeSkillEntries[0]!.runtimeName })]);
expect(await fingerprint(result.runtimeSkillEntries)).toBe(await fingerprint(first.runtimeSkillEntries));
expect(await access(cached).then(() => true, () => false)).toBe(false);
expect(adopting.appendRunEvent).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
payload: { skillPreparation: "reused_admitted", skills: 1 } }));
// An ordinary start of the same saved selection still waits for the cache and materializes.
expect(normalSettled).toBe(false);
} finally {
if (timer) clearTimeout(timer);
release();
await holder;
await adoptingPending.catch(() => undefined);
}
const restored = await normalPending;
if (!restored.prepared) throw new Error("Normal preparation did not finish");
expect(await readFile(path.join(restored.runtimeSkillEntries[0]!.source, "SKILL.md"), "utf8")).toContain("Original bytes.");
expect(await fingerprint(restored.runtimeSkillEntries)).toBe(await fingerprint(first.runtimeSkillEntries));
expect(normal.appendRunEvent).not.toHaveBeenCalledWith(expect.anything(), expect.objectContaining({
payload: expect.objectContaining({ skillPreparation: "reused_admitted" }) }));
});
it("applies project environment and issue overrides and replaces caller-supplied runtime fields", async () => {
const f = await seed();
const { finished, input } = await execute(f);
expect(finished.status, finished.error ?? undefined).toBe("succeeded");
expect(input?.config).toMatchObject({ model: "issue-model", env: { SHARED_VALUE: "project", AGENT_ONLY: "agent-only", PROJECT_ONLY: "project-only" } });
expect(input?.context.paperclipSecrets).toBeUndefined();
expect(JSON.stringify(input?.context.paperclipWake)).not.toContain("untrusted wake instructions");
});
it("delivers the selected credential in an isolated home and removes it after completion", async () => {
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://collector.test");
const f = await seed(true);
let providerHome: string | undefined;
inspect.set(f.agentId, async input => {
const env = input.config.env as Record<string, string>;
providerHome = env.CODEX_HOME;
expect(env.OPENAI_API_KEY).toBe(f.credential);
expect(env.CODEX_API_KEY).toBe(f.credential);
expect(JSON.parse(await readFile(path.join(providerHome, "auth.json"), "utf8"))).toEqual({ OPENAI_API_KEY: f.credential });
await input.onMeta?.({ adapterType, command: "fixture", env, prompt: "fixture" });
});
const exporter = new InMemorySpanExporter();
const provider = new TracerProvider({ spanProcessors: [new SimpleSpanProcessor({ exporter })] });
expect(trace.setGlobalTracerProvider(provider)).toBe(true);
try {
const { finished, input } = await execute(f);
expect(finished.status, finished.error ?? undefined).toBe("succeeded");
expect(input?.context.aiConnection).toMatchObject({ accountName: "Fixture account", provider: "openai" });
expect(providerHome).toEqual(expect.any(String));
await expect(access(providerHome!)).rejects.toMatchObject({ code: "ENOENT" });
const events = await db.select().from(heartbeatRunEvents).where(eq(heartbeatRunEvents.runId, finished.id));
expect(events.some(event => event.eventType === "adapter.invoke")).toBe(true);
expect(JSON.stringify(events)).not.toContain(f.credential);
await provider.forceFlush();
const runIdHash = createHash("sha256").update(finished.id).digest("hex").slice(0, 12);
const steps = exporter.getFinishedSpans().filter(span => span.name.startsWith("heartbeat.prepare.configuration_")
&& span.attributes["paperclip.task.run.run_id"] === runIdHash);
expect(steps.map(span => span.name).sort()).toEqual([
"configuration_mentions", "configuration_github_identity", "configuration_secrets", "configuration_ai_runtime",
"configuration_skills", "configuration_connector_assignments", "configuration_connector_skills",
"configuration_connector_delivery", "configuration_connection_instructions", "configuration_agent_identity",
"configuration_mcp_skills",
].map(step => `heartbeat.prepare.${step}`).sort());
for (const span of steps) {
expect(span.status).toEqual({ code: 0 });
expect(span.attributes).toEqual({ "paperclip.task.run.run_id": runIdHash });
expect(span.events).toEqual([]);
}
} finally {
trace.disable();
await provider.shutdown();
}
});
it("cleans acquired credentials and redacts diagnostics when later skill preparation fails", async () => {
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://collector.test");
const f = await seed(true);
skillFailures.set(f.companyId, `Invalid fixture skill configuration: ${f.credential}`);
const capturedFailure = vi.spyOn(sentry, "captureRunFailure").mockImplementation(() => false);
const prepare = aiRuntime.prepareManagedAiRuntime;
let acquiredHome: string | undefined;
vi.spyOn(aiRuntime, "prepareManagedAiRuntime").mockImplementation(async (...args) => {
const runtime = await prepare(...args);
acquiredHome = runtime.home;
await access(acquiredHome!);
return runtime;
});
const exporter = new InMemorySpanExporter();
const provider = new TracerProvider({ spanProcessors: [new SimpleSpanProcessor({ exporter })] });
expect(trace.setGlobalTracerProvider(provider)).toBe(true);
try {
const { finished, input } = await execute(f);
expect(finished.status).toBe("failed");
expect(input).toBeUndefined();
expect(acquiredHome).toEqual(expect.any(String));
await expect(access(acquiredHome!)).rejects.toMatchObject({ code: "ENOENT" });
expect(finished.error).toContain("Invalid fixture skill configuration");
await waitForPendingRunFailureReports();
expect(capturedFailure).toHaveBeenCalled();
expect(JSON.stringify(capturedFailure.mock.calls)).toContain("Invalid fixture skill configuration");
expect(JSON.stringify(capturedFailure.mock.calls)).not.toContain(f.credential);
await provider.forceFlush();
const runIdHash = createHash("sha256").update(finished.id).digest("hex").slice(0, 12);
const steps = exporter.getFinishedSpans().filter(span => span.name.startsWith("heartbeat.prepare.configuration_")
&& span.attributes["paperclip.task.run.run_id"] === runIdHash);
expect(steps.find(span => span.name === "heartbeat.prepare.configuration_ai_runtime")?.status.code).toBe(0);
expect(steps.find(span => span.name === "heartbeat.prepare.configuration_skills")?.status).toEqual({ code: 2 });
expect(steps.some(span => span.name === "heartbeat.prepare.configuration_connector_assignments")).toBe(true);
for (const span of steps) {
expect(span.attributes).toEqual({ "paperclip.task.run.run_id": runIdHash });
expect(span.events).toEqual([]);
}
expect(JSON.stringify(steps.map(span => ({ name: span.name, status: span.status, attributes: span.attributes, events: span.events })))).not.toContain(f.credential);
} finally {
trace.disable();
await provider.shutdown();
}
});
it("defers a busy account without dispatch and preserves its scheduled retry", async () => {
const f = await seed(true);
vi.spyOn(aiRuntime, "prepareManagedAiRuntime").mockRejectedValueOnce(new HttpError(422, "AI credentials are being updated. Retry shortly.", { code: "ai_connection_busy" }));
const { finished, input } = await execute(f);
expect(input).toBeUndefined();
expect(finished).toMatchObject({ status: "cancelled", errorCode: "ai_connection_busy", resultJson: { executionRecovery: { kind: "ai_connection_wait", providerWorkStarted: false } }, contextSnapshot: { aiConnectionBusyDeferredWhileAssignee: true } });
const runs = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, f.agentId));
expect(runs.filter(run => run.id !== finished.id)).toEqual([expect.objectContaining({ status: "scheduled_retry", scheduledRetryReason: "ai_connection_busy" })]);
});
it("snapshots only selected immutable skills and omits inaccessible optional assignments", async () => {
const f = await seed();
const skills = companySkillService(db);
const selected = await skills.createLocalSkill(f.companyId, { name: "Selected workflow", markdown: "# Selected\nUse the original workflow.\n" });
const unused = await skills.createLocalSkill(f.companyId, { name: "Unused workflow" });
const restricted = await skills.createLocalSkill(f.companyId, { name: "Restricted workflow" });
await db.update(companySkills).set({ accessSourceId: randomUUID() }).where(eq(companySkills.id, restricted.id));
await db.update(agents).set({ adapterConfig: { paperclipSkillSync: { desiredSkills: [restricted.key] } } }).where(eq(agents.id, f.agentId));
await db.update(issues).set({ description: "Use [/selected](skill://" + selected.id + "?s=selected-workflow)." }).where(eq(issues.id, f.issueId));
const { finished, input } = await execute(f);
expect(finished.status, finished.error ?? undefined).toBe("succeeded");
const inventory = input?.config.paperclipRuntimeSkills as Array<{ key: string; source: string; versionId: string }>;
expect(inventory.some(entry => entry.key === unused.key || entry.key === restricted.key)).toBe(false);
const delivered = inventory.find(entry => entry.key === selected.key)!;
expect(delivered.versionId).toBe(selected.currentVersionId);
expect(delivered.source).toContain(selected.currentVersionId!);
expect(await readFile(path.join(delivered.source, "SKILL.md"), "utf8")).toContain("Use the original workflow.");
expect(finished.contextSnapshot).toMatchObject({ skillSelectionSnapshot: true, skillVersions: { [selected.key]: selected.currentVersionId } });
expect(Object.keys(finished.contextSnapshot!.skillVersions as object)).not.toContain(unused.key);
expect(Object.keys(finished.contextSnapshot!.skillVersions as object)).not.toContain(restricted.key);
});
it("rejects an inaccessible explicit skill reference before adapter dispatch", async () => {
const f = await seed();
const skill = await companySkillService(db).createLocalSkill(f.companyId, { name: "Private reference" });
await db.update(companySkills).set({ accessSourceId: randomUUID() }).where(eq(companySkills.id, skill.id));
await db.update(issues).set({ description: "Use [/private](skill://" + skill.id + "?s=private-reference)." }).where(eq(issues.id, f.issueId));
const { finished, input } = await execute(f);
expect(finished.status).toBe("failed");
expect(finished.error).toContain("referenced skill is unavailable");
expect(input).toBeUndefined();
});
});
enableRunFailureReporting();