Files
PaperClipAI/server/src/__tests__/dot-runner.test.ts
T
DottaandPaperclip b8ffeb973a feat(runner): add experimental Grok Bot agent participation (#16219)
## Thinking Path

> - Paperclip manages AI agents that perform work for a company.
> - The shared Runner owns task admission, authority, receipts and
finalization.
> - An existing Grok Bot needs to receive assignments and return results
through that Runner.
> - The Bot also needs governed access when it starts work from its own
conversation.
> - This PR adds agent-purpose MCP access and routine webhook dispatch.
> - It reuses the external-agent Rust Runner and keeps Grok capabilities
explicit.
> - The provider stays experimental until the real-client managed
journeys pass.

## Linked Issues or Issue Description

**Agent or provider**
Grok Bot, distinct from the Grok Build coding runtime in #13882.

**Why this adapter is useful**
Users can assign work to an existing Bot, receive progress, questions,
documents and files in Paperclip, and let the Bot initiate work under
its agent permissions.

**How the agent is invoked**
A webhook-only routine delivers bounded references. Dedicated
named-agent OAuth and MCP provide inbound calls. Setup follows [Grok
routines](https://cursor.com/help/grok-bot/routines) and [custom
plugins](https://cursor.com/help/grok-bot/connect-plugins). The current
desktop app needs its Bot-generated Webhook URL navigation link when
routine-row clicks fail.

The GitHub search found related Grok Build runtime work, but no existing
Grok Bot agent provider. This extends the roadmap’s existing agent,
Runner, governance and artifact foundations.

## What Changed

- Add default-off Grok Bot participation through the shared real Rust
external backend, preserving native provider identity guarantees and
distinguishing session closure from remote termination.
- Bind agent OAuth to company, agent, client, gateway and authorizer;
recheck permissions and task-bound execution authority without
manufacturing a native identity.
- Persist routine configuration, dispatch intents, authenticated
acknowledgement and recovery holds. Store URL/key together in encrypted
secret storage, use guarded public egress, and never automatically
replay ambiguous HTTP delivery.
- Reuse native tools, durable waiting, completion assessment and review.
Add task-bound file grants through existing artifact storage; suppress
proactive echo dispatch and synchronous result wake loops.
- Enforce the current native deliverable work-mode policy at file-ticket
issuance and use. Add operator retries for confirmed rejected wakes,
with retained receipts, stable retry intent and current-authority
checks. Recovery locks bindings before routine rows. Thread Stop
resolves the selected run and its agent.
- Cancel queued and preparing Stop targets before assignment creation,
restore saved setup after a failed request and successful retry, and
publish the Grok provider descriptor schema. Advance the versioned
Copilot source declaration for the shared schema change; its
qualification remains pending.
- Filter task summaries with the shared privacy policy, preserve
feedback admission when wake delivery is blocked, reconcile settled
delivery receipts without restoring work authority, and bind every Stop
control to its displayed run. Recovery records distinguish dispatch
uncertainty from a revoked session.
- Add resumable guided setup, credential replacement, consent,
management, observed work status and stop/recovery controls, plus an
interactive Storybook journey using the production task thread, question
form and report card. Grok’s own apps remain available; Paperclip
workspace and third-party tools are deferred.

## Verification

- Five focused protocol, authorization, person MCP and Runner files: 192
tests pass. The Grok assigned-work case executes the actual Rust Runner,
saves a document, uploads/deduplicates a file and reaches ordinary
finalization.
- Proactive intake/deduplication and renewed-session recovery pass.
These use scripted clients and synthetic storage; they are not
real-provider acceptance.
- Repository typecheck, production build, token gates, map checks and
Storybook build/catalog checks pass. The local full-suite campaign was
stopped after macOS shared-memory exhaustion repeatedly prevented
embedded PostgreSQL startup. No local aggregate pass is claimed. The
first clean-worker CI run identified avatar fixture/configuration, flag
defaults, feature-card ordering and OpenAPI route-coverage gaps; all
four focused regressions pass after correction. Those regressions pass.
The latest clean-worker CI run also found a Copilot source-hash
declaration mismatch after the shared descriptor update; the corrected
declaration passes all 28 integrity tests. The next review corrections
pass 144 rendered UI tests, 21 OpenAPI tests and 82 Runner
contract/integrity/history tests. The latest database fixtures were
blocked at startup by host shared-memory exhaustion; clean-worker CI
must execute their new regressions. Full current-head CI now passes: 54
successful checks and four intentional skips. The database regression
uses the PostgreSQL driver result shape; file-policy, rejected-feedback
and lock-order cases pass on clean workers. Current-head Greptile is 5/5
with no unresolved findings.
- Rendered setup steps and outbound-only verification were walked
through in Storybook. Simulation cannot enable Done without an
authenticated-reply state.
- Actual client OAuth, continued access, reconnect and one routine wake
with an exact chat acknowledgement were observed separately. This
candidate’s live authenticated challenge, assigned/proactive workflows,
Linux managed artifact, restart and cleanup are not yet qualified. The
provider stays disabled by default.
- The separate Cloud ingress change forwards only exact task-bound
upload/download paths and preserves host identity and header stripping.
- Current Cloud policy reserves Test for pre-merge app/harness
experiments; Staging accepts only master-derived revisions. Real-client
qualification needs a supported, authorized deployment target and
remains outstanding.

## Risks

One append-only migration adds routine/dispatch state and extends shared
bindings and file tickets. The unshipped migrations were regenerated
after upstream consumed their original numbers. Rolling back code
retains the durable rows; revoke grants and inspect unknown remote
activity before resuming dispatch. Grok cannot confirm remote
termination, stream its complete transcript or report spending. Plugins
are account-wide and do not isolate credentials between Bots.
Real-client managed acceptance remains required before enabling the
experimental provider.

## Model Used

OpenAI Codex based on GPT-6, with reasoning, code execution and
browser/native UI tools. The exact deployment model ID and
context-window size are not exposed by this runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-13 20:57:42 -05:00

979 lines
83 KiB
TypeScript

import { createGrokRoutineDispatcher } from "../services/grok-bot/dispatcher.js";
import { grokRoutineService } from "../services/grok-bot/routine.js";
import { accessService } from "../services/access.js";
import { Readable } from "node:stream";
import { createPublicMcpTransfers } from "../services/public-mcp/file-transfers.js";
import { issueService } from "../services/issues.js";
import { createHash, createHmac, randomBytes, randomUUID } from "node:crypto";
import { createServer } from "node:http";
import { execFileSync } from "node:child_process";
import { fileURLToPath } from "node:url";
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { and, eq, inArray, sql } from "drizzle-orm";
import express, { type Request } from "express";
import request from "supertest";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { grokBotRoutines, grokBotDispatches, agents, authUsers, companies, companyMemberships, createDb, dotAgentBindings, dotMailboxItems, dotRunnerAssignments, dotRunnerOperations,
heartbeatRuns, agentWakeupRequests, issueComments, issues, nativeRunResults, nativeRunFinalizations, completionContracts, mcpEventDeliveries, mcpEventSubscriptions, workspaceOperations, workAssessments, statusDecisions, issueThreadInteractions } from "@paperclipai/db";
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { createPublicMcpOAuth, DEVICE_GRANT } from "../services/public-mcp/oauth.js";
import { createPublicMcpExecutor } from "../services/public-mcp/capabilities.js";
import { publicMcpIngressRoutes, publicMcpManagementRoutes } from "../routes/public-mcp.js";
import { createPublicMcpEvents, type EventFetch } from "../services/public-mcp/events.js";
import { instanceSettingsService } from "../services/instance-settings.js";
import { canonicalNativeRuntimeContextDigest, type StrictCompletionContractInput } from "../vendor/paperclip-runner/index.js";
import { createDotRunnerMcpTools, dotRunnerBroker } from "../services/dot-runner-broker.js";
import { prepareNativeHeartbeatRun } from "../services/native-runtime/prepare-native-run.js";
import { buildNativeExecutionInput } from "../services/native-runtime/native-execution-input.js";
import { nativeRuntimeContextFixture } from "../services/native-runtime/runtime-context.test-fixture.js";
import { executePaperclipNativeSession } from "../services/native-runtime/native-session-executor.js";
import { registerAssignedMcpGateway } from "../services/native-runtime/assigned-mcp-tools.js";
import type { ToolGatewayService } from "../services/tool-gateway.js";
import { documentService } from "../services/documents.js";
import { setupRunnerPrpWebSocketServer } from "../realtime/runner-prp-ws.js";
import { finalizeNativeRun } from "../services/native-runtime/native-run-finalizer.js";
import { resolveHeartbeatNativeRuntimeMode, resolveNativeRuntimeMode } from "../services/native-runtime/runtime-mode.js";
import { heartbeatService } from "../services/heartbeat.js";
import { consumeDotHistoryReceipt, publishActiveDotComment } from "../services/dot-assignment-follow-up.js";
import { createPostgresWakeQueueAdapter } from "../modules/wake-queue/adapters/postgres.js";
import { createReleaseIssueExecution } from "../modules/wake-queue/application/use-cases.js";
describe("durable Dot Runner integration", () => {
let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let root: string;
beforeAll(async () => {
const runnerRoot = fileURLToPath(new URL("../../../packages/paperclip-runner/", import.meta.url));
execFileSync("cargo", ["build", "--release", "--locked", "--manifest-path", join(runnerRoot, "runner/Cargo.toml"), "-p", "paperclip-runner-core", "--bin", "paperclip-runnerd"], { cwd: runnerRoot, stdio: "inherit", timeout: 300000 });
temporary = await startEmbeddedPostgresTestDatabase("paperclip-dot-runner-");
db = createDb(temporary.connectionString);
root = await mkdtemp(join(tmpdir(), "paperclip-dot-state-"));
vi.stubEnv("PAPERCLIP_IN_WORKTREE", "false");
vi.stubEnv("PAPERCLIP_RUNNER_BINARY", join(runnerRoot, "runner/target/release", process.platform === "win32" ? "paperclip-runnerd.exe" : "paperclip-runnerd"));
vi.stubEnv("PAPERCLIP_RUNNER_STATE_DIR", join(root, "runner-state"));
vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", randomBytes(32).toString("base64"));
await instanceSettingsService(db).updateExperimental({ enablePublicMcp: true, enableOpenAiDot: true, enableNativeRunner: false });
}, 360000);
afterEach(async () => {
const { heartbeatService } = await import("../services/heartbeat.js");
dotRunnerBroker(db, { heartbeat: heartbeatService(db) });
await instanceSettingsService(db).updateExperimental({ enableGrokBot: false, enableNativeRunner: false });
});
afterAll(async () => {
try {
await temporary?.cleanup();
} finally {
try {
if (root) await rm(root, { recursive: true, force: true });
} finally {
vi.unstubAllEnvs();
}
}
});
it("requires each persisted prerequisite for pairing and new work without relying on the retired environment flag", async () => {
const settings = instanceSettingsService(db);
const broker = dotRunnerBroker(db);
for (const key of ["enableOpenAiDot", "enablePublicMcp"] as const) {
await settings.updateExperimental({ [key]: false });
try {
vi.stubEnv("PAPERCLIP_ENABLE_OPENAI_DOT", "1");
expect(await broker.enabled()).toBe(false);
await expect(broker.createPairing({ companyId: randomUUID(), agentId: randomUUID(), operatorId: randomUUID() })).rejects.toThrow("experimental settings");
} finally { await settings.updateExperimental({ [key]: true }); }
}
vi.stubEnv("PAPERCLIP_ENABLE_OPENAI_DOT", "0");
expect(await broker.enabled()).toBe(true);
});
it("gates fresh Dot runtime selection while preserving the existing run recovery path", () => {
const input = {
enabled: true,
runtimeConfig: {},
adapterConfig: { provider: "openai_dot", dotBindingId: randomUUID(), allowUnmeteredProvider: true },
agent: { status: "active", adapterType: "paperclip_runner" },
issue: { id: randomUUID(), workMode: "standard" },
target: { kind: "local" },
workspaceId: null,
};
expect(() => resolveNativeRuntimeMode(input)).toThrow("experimental settings");
expect(resolveNativeRuntimeMode({ ...input, enabled: false, dotEnabled: true })).toMatchObject({ kind: "native", profile: { backend: "openai_dot_mcp" } });
expect(() => resolveNativeRuntimeMode({ ...input, enabled: false, dotEnabled: true,
adapterConfig: { provider: "codex" },
})).toThrow("Paperclip Runner is experimental and disabled");
expect(resolveHeartbeatNativeRuntimeMode({ ...input, enabled: false, dotEnabled: false,
persisted: { runtimeMode: "native", runtimeModeReason: null, runtimeModeResolvedAt: new Date(), driverKind: "openai_dot_mcp" },
})).toMatchObject({ kind: "native", profile: { backend: "openai_dot_mcp" } });
});
async function fixture(options: { cloudOrigin?: string; provider?: "openai_dot" | "grok_bot" } = {}) {
const provider = options.provider ?? "openai_dot";
if (provider === "grok_bot") await instanceSettingsService(db).updateExperimental({ enableGrokBot: true, enableNativeRunner: true });
const userId = randomUUID();
await db.insert(authUsers).values({ id: userId, name: "Dot operator", email: userId + "@example.test", createdAt: new Date(), updatedAt: new Date() });
const [company] = await db.insert(companies).values({ name: "Dot test", issuePrefix: "DT" + randomBytes(3).toString("hex") }).returning();
await db.insert(companyMemberships).values({ companyId: company!.id, principalType: "user", principalId: userId, membershipRole: "owner", status: "active" });
await accessService(db).setPrincipalPermission(company!.id, "user", userId, "agents:create", true, userId);
const [agent] = await db.insert(agents).values({ companyId: company!.id, name: provider === "grok_bot" ? "Grok Bot" : "Dot", adapterType: "paperclip_runner", status: "active", adapterConfig: { provider, allowUnmeteredProvider: true } }).returning();
const config = { origin: "https://paperclip.example", resource: "https://paperclip.example/mcp/runner" };
const oauth = createPublicMcpOAuth(db, config);
const client = await oauth.register({ client_name: "Dedicated Dot", redirect_uris: ["https://chatgpt.com/connector_platform/oauth/callback"] }, randomUUID());
const verifier = randomBytes(32).toString("base64url");
const input = { client_id: client.client_id, redirect_uri: "https://chatgpt.com/connector_platform/oauth/callback", response_type: "code", resource: config.resource,
scope: "paperclip:agent offline_access", code_challenge: createHash("sha256").update(verifier).digest("base64url"), code_challenge_method: "S256" };
const requestId = (await oauth.authorize(input)).split("/").at(-1)!;
const consent = await oauth.consent(requestId, { type: "board", source: "session", userId }, { decision: "approve", companyId: company!.id, allowWrites: false });
const tokens = await oauth.token({ grant_type: "authorization_code", client_id: client.client_id, redirect_uri: input.redirect_uri,
resource: config.resource, code: new URL(consent.redirectUrl).searchParams.get("code"), code_verifier: verifier });
const unpaired = await oauth.authenticate(tokens.access_token);
expect(unpaired.actor.type).toBe("none");
const broker = dotRunnerBroker(db);
const pairing = await broker.createPairing({ companyId: company!.id, agentId: agent!.id, operatorId: userId, provider });
await broker.pair(unpaired, pairing.pairingCode);
expect((await db.select().from(agents).where(eq(agents.id, agent!.id)))[0]!.adapterConfig[provider === "grok_bot" ? "externalBindingId" : "dotBindingId"]).toBe(pairing.bindingId);
await expect(broker.pair(unpaired, pairing.pairingCode)).rejects.toThrow();
const principal = await oauth.authenticate(tokens.access_token);
expect(principal.actor).toMatchObject({ type: "agent", agentId: agent!.id, companyId: company!.id });
const personal = createPublicMcpOAuth(db, { ...config, resource: config.origin + "/mcp/paperclip" });
await expect(personal.authenticate(tokens.access_token)).rejects.toThrow();
const secret = "whsec_" + randomBytes(32).toString("base64");
const received: Array<Record<string, any>> = [];
let verifications = 0;
const fetcher: EventFetch = async (_url, init) => {
const headers = new Headers(init.headers); const bytes = String(init.body); const body = JSON.parse(bytes);
const signature = "v1," + createHmac("sha256", Buffer.from(secret.slice(6), "base64"))
.update(`${headers.get("webhook-id")}.${headers.get("webhook-timestamp")}.${bytes}`).digest("base64");
expect(headers.get("webhook-signature")).toContain(signature);
if (body.type === "verification") { verifications++; return Response.json({ challenge: body.challenge }); }
received.push(body); return new Response(null, { status: 204 });
};
const events = createPublicMcpEvents(db, oauth, async () => { throw new Error("personal API dispatch forbidden"); }, { enableDotRunner: true, fetch: fetcher, ...options });
const subscription = { name: "paperclip.dot.mailbox_updated", arguments: { companyId: company!.id, bindingId: pairing.bindingId }, delivery: { mode: "webhook", url: "https://example.com/dot-hook", secret } };
if (provider === "grok_bot") {
const routines = grokRoutineService(db, async () => ({ delivery: "accepted", httpStatus: 200 }));
await routines.configure({ companyId: company!.id, agentId: agent!.id, bindingId: pairing.bindingId, operatorId: userId,
expectedRevision: 0, botName: "Synthetic Grok", webhookUrl: "https://bot.example.test/routine", webhookKey: "synthetic-key" });
const wake = await routines.reserve({ companyId: company!.id, agentId: agent!.id, bindingId: pairing.bindingId, sourceEventId: randomUUID(), kind: "verify" });
await routines.send(company!.id, agent!.id, wake.id);
await routines.acknowledge(principal, { correlationId: wake.id, bindingId: wake.bindingId, bindingGeneration: wake.bindingGeneration, challenge: String(wake.payload.challenge) });
const snapshot = await broker.snapshot(company!.id, agent!.id, pairing.bindingId);
const [fresh] = await db.select().from(agents).where(eq(agents.id, agent!.id));
return { company: company!, agent: fresh!, userId, broker, principal, oauth, events, subscription, received, snapshot, tokens, client, verifications: () => verifications };
}
await events.subscribe(principal, subscription);
await events.tick();
if (!received.length) throw new Error("Dot readiness delivery missing: " + JSON.stringify(await db.select({ outcome: mcpEventDeliveries.outcome, event: mcpEventDeliveries.event }).from(mcpEventDeliveries)));
expect(received.at(-1)?.data.kind).toBe("readiness_challenge");
expect(received.at(-1)?.data).not.toHaveProperty("challenge");
const challenge = (await broker.mailbox(principal)).items.find(i => i.kind === "readiness_challenge")!;
await broker.confirmChallenge(principal, String(challenge.references.challenge));
const snapshot = await broker.snapshot(company!.id, agent!.id, pairing.bindingId);
await db.update(agents).set({ adapterConfig: { provider: "openai_dot", dotBindingId: pairing.bindingId, allowUnmeteredProvider: true, lifecycleMode: "per_turn" } }).where(eq(agents.id, agent!.id));
return { company: company!, agent: agent!, userId, broker, principal, oauth, events, subscription, received, snapshot, tokens, client, verifications: () => verifications };
}
it("keeps scoped Dot events on tenant OAuth when the Cloud personal broker is configured", async () => {
const f = await fixture({ cloudOrigin: "https://cloud-broker.example" });
expect(f.verifications()).toBe(1);
expect(f.received.at(-1)?.data.kind).toBe("readiness_challenge");
await f.broker.revoke(f.company.id, f.agent.id, f.userId);
await expect(f.events.subscribe(f.principal, f.subscription)).rejects.toThrow();
}, 30000);
async function offeredWork(f: Awaited<ReturnType<typeof fixture>>) {
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.company.id, agentId: f.agent.id,
status: "queued", invocationSource: "assignment", triggerDetail: "system" }).returning();
const [assignment] = await db.insert(dotRunnerAssignments).values({ companyId: f.company.id, bindingId: f.snapshot.bindingId,
bindingGeneration: f.snapshot.bindingGeneration, runId: run!.id, agentId: f.agent.id, normalizedSessionId: randomUUID(),
turnId: randomUUID(), controllerGeneration: 1, catalogDigest: "test", status: "offered", projection: {},
acceptBy: new Date(Date.now() + 10 * 60_000), expiresAt: new Date(Date.now() + 2 * 60 * 60_000) }).returning();
await db.insert(dotMailboxItems).values({ companyId: f.company.id, bindingId: f.snapshot.bindingId,
bindingGeneration: f.snapshot.bindingGeneration, assignmentId: assignment!.id, kind: "assignment", sourceEventId: randomUUID(),
references: { assignmentId: assignment!.id, runId: run!.id, revision: 1 } });
return { run: run!, assignment: assignment! };
}
function gateway(oauth: ReturnType<typeof createPublicMcpOAuth>, actor: Request["actor"]) {
const personal = createPublicMcpOAuth(db, { ...oauth.config, resource: oauth.config.origin + "/mcp/paperclip" });
const dispatch = async () => { throw new Error("Personal API dispatch forbidden"); };
const app = express();
app.use(express.json());
app.use(publicMcpIngressRoutes(personal, createPublicMcpExecutor(db, personal, dispatch)));
app.use(publicMcpIngressRoutes(oauth, createPublicMcpExecutor(db, oauth, dispatch)));
app.use((req, _res, next) => { req.actor = actor; next(); });
app.use("/api", publicMcpManagementRoutes(personal, oauth));
return { app, personal };
}
it("reapplies the Dot-only migration after the merged gateway schema", async () => {
const migration = await readFile(new URL("../../../packages/db/src/migrations/0317_messy_famine.sql", import.meta.url), "utf8");
for (const statement of migration.split("--> statement-breakpoint")) {
if (statement.trim()) await db.execute(sql.raw(statement));
}
});
it.each(["revoked binding", "removed subscription"])("handles an automatic event-test race with a %s", async change => {
const f = await fixture();
await db.update(dotAgentBindings).set({ readyAt: null, status: "connected" }).where(eq(dotAgentBindings.id, f.snapshot.bindingId));
const readBinding = f.broker.bindingForAgent.bind(f.broker);
const read = vi.spyOn(f.broker, "bindingForAgent").mockImplementationOnce(async (companyId, agentId) => {
const state = await readBinding(companyId, agentId);
if (change === "revoked binding") await db.update(dotAgentBindings).set({ revokedAt: new Date() }).where(eq(dotAgentBindings.id, f.snapshot.bindingId));
else await db.update(mcpEventSubscriptions).set({ stoppedAt: new Date() }).where(eq(mcpEventSubscriptions.bindingId, f.snapshot.bindingId));
return state;
});
try {
await expect(f.broker.challenge(f.company.id, f.agent.id, { automatic: true, bindingId: f.snapshot.bindingId })).resolves.toEqual({ status: "unavailable" });
await expect(f.broker.challenge(f.company.id, f.agent.id)).rejects.toThrow("Connect and subscribe");
} finally {
read.mockRestore();
await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop();
}
});
it.each(["openai_dot", "grok_bot"] as const)("admits one durable %s idle intake without requiring a preassigned task", async provider => {
const f = await fixture({ provider });
await db.update(companies).set({ defaultResponsibleUserId: f.userId }).where(eq(companies.id, f.company.id));
const holding = await offeredWork(f);
await db.update(heartbeatRuns).set({ status: "running", startedAt: new Date() }).where(eq(heartbeatRuns.id, holding.run.id));
const requestId = randomUUID();
try {
const requests = await Promise.all([1, 2, 3].map(() => f.broker.requestTurn(f.principal, "Create hello and assign it to my owner", requestId)));
expect(requests[1]).toEqual(requests[0]); expect(requests[2]).toEqual(requests[0]);
const intake = await db.select().from(issues).where(eq(issues.id, requests[0]!.issueId));
expect(intake).toHaveLength(1);
expect(intake[0]).toMatchObject({ assigneeAgentId: f.agent.id, createdByAgentId: f.agent.id, responsibleUserId: f.userId });
expect(await f.broker.capabilities(f.principal)).toMatchObject({ agentId: f.agent.id, responsibleUser: { id: f.userId }, idle: { start: provider === "grok_bot" ? "paperclip_grok_request_turn" : "paperclip_dot_request_turn" } });
await expect(f.broker.requestTurn(f.principal, "Different request", requestId)).rejects.toThrow("reused");
await db.update(companies).set({ budgetMonthlyCents: 1, spentMonthlyCents: 1 }).where(eq(companies.id, f.company.id));
await expect(f.broker.requestTurn(f.principal, "Spend more", randomUUID())).rejects.toThrow("budget");
await db.update(companies).set({ budgetMonthlyCents: 0, spentMonthlyCents: 0 }).where(eq(companies.id, f.company.id));
await f.broker.revoke(f.company.id, f.agent.id, f.userId);
expect((await db.select().from(agents).where(eq(agents.id, f.agent.id)))[0]!.adapterConfig).not.toHaveProperty(provider === "grok_bot" ? "externalBindingId" : "dotBindingId");
await expect(f.broker.requestTurn(f.principal, "After revoke", randomUUID())).rejects.toThrow();
} finally { await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, holding.run.id)); }
});
it("keeps competing Dot assignments queued and replays durable work requests", async () => {
const f = await fixture();
await db.update(companies).set({ defaultResponsibleUserId: f.userId }).where(eq(companies.id, f.company.id));
await db.update(agents).set({ runtimeConfig: { heartbeat: { maxConcurrentRuns: 20, wakeOnDemand: true } } }).where(eq(agents.id, f.agent.id));
const holding = await offeredWork(f);
await db.update(heartbeatRuns).set({ status: "running", startedAt: new Date() }).where(eq(heartbeatRuns.id, holding.run.id));
const task = async (title: string) => (await db.insert(issues).values({ companyId: f.company.id, title, status: "todo", assigneeAgentId: f.agent.id, responsibleUserId: f.userId }).returning())[0]!;
const first = await task("First queued task");
const second = await task("Second queued task");
try {
const id = randomUUID();
const responses = await Promise.all(Array.from({ length: 3 }, () => f.broker.requestWork(f.principal, first.id, id)));
expect(responses[0]?.runId).toBeTruthy();
expect(responses).toEqual([responses[0], responses[0], responses[0]]);
const wakes = await db.select().from(agentWakeupRequests).where(and(eq(agentWakeupRequests.agentId, f.agent.id), eq(agentWakeupRequests.idempotencyKey, `dot-work:${f.snapshot.bindingId}:${f.snapshot.bindingGeneration}:${id}`)));
expect(wakes).toHaveLength(1);
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, responses[0]!.runId!)))[0]?.status).toBe("queued");
await heartbeatService(db).resumeQueuedRuns();
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, responses[0]!.runId!)))[0]?.status).toBe("queued");
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, responses[0]!.runId!));
await db.update(issues).set({ status: "done" }).where(eq(issues.id, first.id));
expect(await f.broker.requestWork(f.principal, first.id, id)).toEqual(responses[0]);
await expect(f.broker.requestWork(f.principal, second.id, id)).rejects.toThrow("another task");
const third = await task("Third queued task");
const raceId = randomUUID();
const race = await Promise.allSettled([f.broker.requestWork(f.principal, second.id, raceId), f.broker.requestWork(f.principal, third.id, raceId)]);
expect(race.filter(r => r.status === "fulfilled")).toHaveLength(1);
expect(race.filter(r => r.status === "rejected")).toHaveLength(1);
const raceWakes = await db.select().from(agentWakeupRequests).where(and(eq(agentWakeupRequests.agentId, f.agent.id), eq(agentWakeupRequests.idempotencyKey, `dot-work:${f.snapshot.bindingId}:${f.snapshot.bindingGeneration}:${raceId}`)));
expect(raceWakes).toHaveLength(1);
const allRuns = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, f.agent.id));
expect(allRuns).toHaveLength(3); // Holding turn plus one admitted run per unique request.
expect(allRuns.filter(r => r.status === "queued")).toHaveLength(1);
} finally {
await db.update(heartbeatRuns).set({ status: "cancelled", finishedAt: new Date() }).where(eq(heartbeatRuns.agentId, f.agent.id));
await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop();
}
}, 30000);
it("delivers deduplicated follow-up references only for the accepted current assignment", async () => {
const f = await fixture(), { run, assignment } = await offeredWork(f);
const [task] = await db.insert(issues).values({ companyId: f.company.id, title: "Follow-up fixture", assigneeAgentId: f.agent.id }).returning();
await db.update(heartbeatRuns).set({ nativeIssueId: task!.id, status: "running" }).where(eq(heartbeatRuns.id, run.id));
await db.update(dotRunnerAssignments).set({ status: "accepted" }).where(eq(dotRunnerAssignments.id, assignment.id));
const [incoming] = await db.insert(issueComments).values({ companyId: f.company.id, issueId: task!.id, authorUserId: f.userId, body: "Synthetic comment body must not be in webhook data" }).returning();
await db.insert(issueComments).values({ companyId: f.company.id, issueId: task!.id, authorAgentId: f.agent.id, body: "Dot's own output" });
try {
await f.events.tick(); await f.events.tick();
const rows = await db.select().from(dotMailboxItems).where(and(eq(dotMailboxItems.assignmentId, assignment.id), eq(dotMailboxItems.kind, "follow_up")));
expect(rows).toHaveLength(1);
expect(rows[0]!.references).toEqual({ assignmentId: assignment.id, commentId: incoming!.id });
const delivered = f.received.filter(event => event.data?.kind === "follow_up");
expect(delivered).toHaveLength(1);
expect(JSON.stringify(delivered)).not.toContain("Synthetic comment body");
expect(JSON.stringify(delivered)).not.toContain("commentId");
await db.update(dotRunnerAssignments).set({ status: "fenced" }).where(eq(dotRunnerAssignments.id, assignment.id));
await db.insert(issueComments).values({ companyId: f.company.id, issueId: task!.id, authorUserId: f.userId, body: "Too late" });
await f.events.tick();
expect(await db.select().from(dotMailboxItems).where(and(eq(dotMailboxItems.assignmentId, assignment.id), eq(dotMailboxItems.kind, "follow_up")))).toHaveLength(1);
} finally { await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop(); }
});
it("coalesces a normal comment wake into the accepted Dot mailbox without a second run", async () => {
const f = await fixture(), { run, assignment } = await offeredWork(f);
const [task] = await db.insert(issues).values({ companyId: f.company.id, title: "Active Dot comment",
status: "in_progress", assigneeAgentId: f.agent.id, executionRunId: run.id, checkoutRunId: run.id }).returning();
await db.update(heartbeatRuns).set({ nativeIssueId: task!.id, issueId: task!.id, status: "running",
runtimeMode: "native", nativeSessionId: assignment.normalizedSessionId }).where(eq(heartbeatRuns.id, run.id));
await db.update(dotRunnerAssignments).set({ status: "accepted" }).where(eq(dotRunnerAssignments.id, assignment.id));
const [comment] = await db.insert(issueComments).values({ companyId: f.company.id, issueId: task!.id,
authorUserId: f.userId, body: "Continue the same assignment" }).returning();
try {
const admitted = await heartbeatService(db).wakeup(f.agent.id, { source: "assignment", triggerDetail: "system",
reason: "issue_commented", payload: { issueId: task!.id, commentId: comment!.id },
requestedByActorType: "user", requestedByActorId: f.userId,
contextSnapshot: { issueId: task!.id, wakeCommentId: comment!.id, wakeReason: "issue_commented" } });
expect(admitted?.id).toBe(run.id);
await f.events.tick(); await f.events.tick();
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, f.agent.id))).toHaveLength(1);
const wakes = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, f.agent.id));
expect(wakes).toHaveLength(1);
expect(wakes[0]).toMatchObject({ status: "deferred_issue_execution", runId: null, finishedAt: null });
const mailbox = await db.select().from(dotMailboxItems).where(and(eq(dotMailboxItems.assignmentId, assignment.id), eq(dotMailboxItems.kind, "follow_up")));
expect(mailbox).toHaveLength(1);
expect(mailbox[0]!.references).toEqual({ assignmentId: assignment.id, commentId: comment!.id });
expect(f.received.filter(event => event.data?.kind === "follow_up")).toHaveLength(1);
await f.broker.mailbox(f.principal);
expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, wakes[0]!.id)))[0]?.status).toBe("deferred_issue_execution");
const [receipt] = await db.insert(dotRunnerOperations).values({ companyId: f.company.id,
assignmentId: assignment.id, requestId: randomUUID(), digest: "history", status: "completed",
command: { action: "tool", input: { name: "get_task_history", arguments: {} } },
outcome: { status: "completed", isError: false, result: { comments: [{ id: comment!.id }] } } }).returning();
await consumeDotHistoryReceipt(db, receipt!);
await consumeDotHistoryReceipt(db, receipt!);
expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, wakes[0]!.id)))[0]).toMatchObject({ status: "coalesced", runId: run.id });
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, f.agent.id))).toHaveLength(1);
} finally {
await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.id, run.id));
await db.update(dotRunnerAssignments).set({ status: "fenced" }).where(eq(dotRunnerAssignments.id, assignment.id));
await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop();
}
}, 30000);
it.each(["finish before delivery", "read before wake admission", "partial history", "failed history"])(
"preserves comment delivery when %s", async scenario => {
const f = await fixture(), { run, assignment } = await offeredWork(f);
await db.update(companies).set({ defaultResponsibleUserId: f.userId }).where(eq(companies.id, f.company.id));
const [task] = await db.insert(issues).values({ companyId: f.company.id, title: "Dot comment race",
status: "in_progress", assigneeAgentId: f.agent.id, responsibleUserId: f.userId,
executionRunId: run.id, checkoutRunId: run.id }).returning();
await db.update(heartbeatRuns).set({ nativeIssueId: task!.id, issueId: task!.id, status: "running",
contextSnapshot: { issueId: task!.id }, runtimeMode: "native", nativeSessionId: assignment.normalizedSessionId }).where(eq(heartbeatRuns.id, run.id));
await db.update(dotRunnerAssignments).set({ status: "accepted" }).where(eq(dotRunnerAssignments.id, assignment.id));
const comments = await db.insert(issueComments).values([1, 2].map(index => ({ companyId: f.company.id,
issueId: task!.id, authorUserId: f.userId, body: "Input " + index }))).returning();
const [history] = await db.insert(dotRunnerOperations).values({ companyId: f.company.id,
assignmentId: assignment.id, requestId: randomUUID(), digest: "history", status: "completed",
command: { action: "tool", input: { name: "get_task_history", arguments: {} } },
outcome: { status: "completed", isError: scenario === "failed history",
result: { comments: [{ id: comments[0]!.id }] } } }).returning();
try {
if (scenario === "read before wake admission") await consumeDotHistoryReceipt(db, history!);
for (const comment of comments) await heartbeatService(db).wakeup(f.agent.id, {
source: "assignment", triggerDetail: "system", reason: "issue_commented",
payload: { issueId: task!.id, commentId: comment.id }, requestedByActorType: "user", requestedByActorId: f.userId,
contextSnapshot: { issueId: task!.id, wakeCommentId: comment.id, wakeReason: "issue_commented" },
});
if (scenario === "finish before delivery") {
const requestId = randomUUID();
await db.insert(dotRunnerOperations).values({ companyId: f.company.id, assignmentId: assignment.id,
requestId, digest: "finish", command: { action: "finish" }, status: "admitted" });
const port = f.broker.port({ binding: { companyId: f.company.id, agentId: f.agent.id, runId: run.id }, provider: { binding: f.snapshot } });
await port.settle({ sourceEventId: randomUUID(), payload: { requestId,
binding: { ...f.snapshot, runId: run.id, normalizedSessionId: assignment.normalizedSessionId,
turnId: assignment.turnId, assignmentRevision: assignment.revision }, outcome: { status: "completed" } } } as Parameters<typeof port.settle>[0]);
await f.events.tick();
expect(f.received.filter(event => event.data?.kind === "follow_up")).toHaveLength(0);
}
await consumeDotHistoryReceipt(db, history!);
let wakes = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, f.agent.id));
expect(wakes).toHaveLength(scenario === "read before wake admission" ? 2 : 1);
const pending = wakes.find(wake => wake.status === "deferred_issue_execution")!;
expect(pending.payload?._paperclipWakeContext).toMatchObject({ wakeCommentIds:
["finish before delivery", "failed history"].includes(scenario)
? comments.map(comment => comment.id) : [comments[1]!.id] });
if (scenario === "read before wake admission") expect(wakes.find(wake => wake.payload?.commentId === comments[0]!.id)?.status).toBe("coalesced");
if (scenario === "finish before delivery") {
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, run.id));
await db.update(issues).set({ status: "done" }).where(eq(issues.id, task!.id));
const release = createReleaseIssueExecution({ issueLock: createPostgresWakeQueueAdapter(db, {
resolveResponsibleUserId: async () => f.userId,
getRoutineEnv: async () => ({ routineId: null, env: null, responsibleUserId: null }),
resolveSessionBeforeForWakeup: async () => null,
}), recovery: {
escalateStrandedAssignedIssue: async () => { throw new Error("unexpected escalation"); },
escalateStrandedRecoveryIssueInPlace: async () => { throw new Error("unexpected escalation"); },
} });
await release({ companyId: f.company.id, runId: run.id, now: new Date(), suppressImmediateRecovery: true });
await release({ companyId: f.company.id, runId: run.id, now: new Date(), suppressImmediateRecovery: true });
const runs = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, f.agent.id));
expect(runs).toHaveLength(2);
expect(runs.find(row => row.id !== run.id)).toMatchObject({ status: "queued",
contextSnapshot: { wakeCommentIds: comments.map(comment => comment.id) } });
wakes = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, f.agent.id));
expect(wakes).toHaveLength(1);
expect(wakes[0]).toMatchObject({ status: "queued", runId: runs.find(row => row.id !== run.id)!.id });
}
} finally {
await db.update(heartbeatRuns).set({ status: "cancelled" }).where(eq(heartbeatRuns.agentId, f.agent.id));
await db.update(dotRunnerAssignments).set({ status: "fenced" }).where(eq(dotRunnerAssignments.id, assignment.id));
await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop();
}
}, 30000);
it.each(["foreign comment", "self comment", "revoked binding", "fenced assignment", "expired assignment", "prior generation"])(
"refuses active comment delivery for %s", async kind => {
const f = await fixture(), { run, assignment } = await offeredWork(f);
const [task] = await db.insert(issues).values({ companyId: f.company.id, title: "Bounded follow-up", assigneeAgentId: f.agent.id }).returning();
await db.update(heartbeatRuns).set({ nativeIssueId: task!.id, status: "running", runtimeMode: "native",
nativeSessionId: assignment.normalizedSessionId }).where(eq(heartbeatRuns.id, run.id));
await db.update(dotRunnerAssignments).set({ status: "accepted" }).where(eq(dotRunnerAssignments.id, assignment.id));
const [comment] = await db.insert(issueComments).values({ companyId: f.company.id, issueId: task!.id,
...(kind === "self comment" ? { authorAgentId: f.agent.id } : { authorUserId: f.userId }), body: "Bounded input" }).returning();
if (kind === "revoked binding") await db.update(dotAgentBindings).set({ revokedAt: new Date() }).where(eq(dotAgentBindings.id, f.snapshot.bindingId));
if (kind === "fenced assignment") await db.update(dotRunnerAssignments).set({ status: "fenced" }).where(eq(dotRunnerAssignments.id, assignment.id));
if (kind === "expired assignment") await db.update(dotRunnerAssignments).set({ expiresAt: new Date(0) }).where(eq(dotRunnerAssignments.id, assignment.id));
if (kind === "prior generation") await db.update(dotAgentBindings).set({ generation: f.snapshot.bindingGeneration + 1 }).where(eq(dotAgentBindings.id, f.snapshot.bindingId));
try {
expect(await db.transaction(tx => publishActiveDotComment(tx as unknown as typeof db, {
companyId: f.company.id, agentId: f.agent.id, bindingId: f.snapshot.bindingId, runId: run.id,
issueId: task!.id, commentId: kind === "foreign comment" ? randomUUID() : comment!.id }))).toBe(false);
expect(await db.select().from(dotMailboxItems).where(and(eq(dotMailboxItems.assignmentId, assignment.id), eq(dotMailboxItems.kind, "follow_up")))).toHaveLength(0);
} finally { await f.events.stop(); }
}, 30000);
it.each(["queued", "running"] as const)("stops a %s Grok run before an external assignment exists", async status => {
const f = await fixture({ provider: "grok_bot" });
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.company.id, agentId: f.agent.id, status, invocationSource: "assignment", triggerDetail: "system" }).returning();
const cancelRun = vi.fn().mockResolvedValue(null);
const broker = dotRunnerBroker(db, { heartbeat: { wakeup: vi.fn(), cancelRun } });
expect(await broker.stopGrokWork(f.company.id, f.agent.id, f.userId, run!.id)).toMatchObject({ stopped: true, remoteTerminationConfirmed: false });
expect(cancelRun).toHaveBeenCalledWith(run!.id, "Paperclip work stopped; Grok execution unconfirmed");
expect(await grokRoutineService(db).recoveryNeeded(f.company.id, f.agent.id)).toBe(status === "running");
expect(await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.runId, run!.id))).toHaveLength(0);
const other = await fixture({ provider: "grok_bot" });
await expect(broker.stopGrokWork(other.company.id, other.agent.id, other.userId, run!.id)).rejects.toThrow("changed");
expect(cancelRun).toHaveBeenCalledTimes(1);
await f.events.stop(); await other.events.stop();
});
it("cancels queued Grok work without fencing another active assignment", async () => {
const f = await fixture({ provider: "grok_bot" }), active = await offeredWork(f);
const [queued] = await db.insert(heartbeatRuns).values({ companyId: f.company.id, agentId: f.agent.id, status: "queued", invocationSource: "assignment", triggerDetail: "system" }).returning();
const cancelRun = vi.fn().mockResolvedValue(null);
const broker = dotRunnerBroker(db, { heartbeat: { wakeup: vi.fn(), cancelRun } });
await broker.stopGrokWork(f.company.id, f.agent.id, f.userId, queued!.id);
expect(cancelRun).toHaveBeenCalledTimes(1);
expect(cancelRun).toHaveBeenCalledWith(queued!.id, "Paperclip work stopped; Grok execution unconfirmed");
expect((await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.id, active.assignment.id)))[0]!.status).toBe("offered");
expect(await grokRoutineService(db).recoveryNeeded(f.company.id, f.agent.id)).toBe(false);
await f.events.stop();
});
it("rejects a stale Grok Stop click without fencing the newly displayed work", async () => {
const f = await fixture({ provider: "grok_bot" });
const old = await offeredWork(f);
await db.update(dotRunnerAssignments).set({ status: "settled" }).where(eq(dotRunnerAssignments.id, old.assignment.id));
await db.update(heartbeatRuns).set({ status: "succeeded" }).where(eq(heartbeatRuns.id, old.run.id));
const current = await offeredWork(f);
await expect(f.broker.stopGrokWork(f.company.id, f.agent.id, f.userId, old.run.id)).rejects.toThrow("changed");
expect((await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.id, current.assignment.id)))[0]!.status).toBe("offered");
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, current.run.id)))[0]!.status).toBe("queued");
await f.events.stop();
});
it("preserves Grok feedback and its deferred wake while routine recovery blocks notification", async () => {
const f = await fixture({ provider: "grok_bot" }), { run, assignment } = await offeredWork(f);
const [task] = await db.insert(issues).values({ companyId: f.company.id, title: "Grok feedback", assigneeAgentId: f.agent.id }).returning();
await db.update(heartbeatRuns).set({ nativeIssueId: task!.id, status: "running", runtimeMode: "native", nativeSessionId: assignment.normalizedSessionId }).where(eq(heartbeatRuns.id, run.id));
await db.update(dotRunnerAssignments).set({ status: "accepted" }).where(eq(dotRunnerAssignments.id, assignment.id));
await db.update(grokBotRoutines).set({ recoveryRequiredAt: new Date(), recoveryReason: "dispatch" }).where(eq(grokBotRoutines.bindingId, f.snapshot.bindingId));
const [comment] = await db.insert(issueComments).values({ companyId: f.company.id, issueId: task!.id, authorUserId: f.userId, body: "Please include the source dates." }).returning();
await db.transaction(async tx => {
expect(await publishActiveDotComment(tx as unknown as typeof db, { companyId: f.company.id, agentId: f.agent.id,
bindingId: f.snapshot.bindingId, runId: run.id, issueId: task!.id, commentId: comment!.id })).toMatchObject({ consumed: false });
await tx.insert(agentWakeupRequests).values({ companyId: f.company.id, agentId: f.agent.id, source: "assignment", triggerDetail: "system",
reason: "issue_comment", status: "deferred_issue_execution", payload: { issueId: task!.id, commentId: comment!.id } });
});
const delivery = vi.fn().mockResolvedValue({ delivery: "accepted", httpStatus: 200 });
const dispatcher = createGrokRoutineDispatcher(db, () => true, delivery);
await dispatcher.tick();
expect(delivery).not.toHaveBeenCalled();
expect(await db.select().from(dotMailboxItems).where(and(eq(dotMailboxItems.assignmentId, assignment.id), eq(dotMailboxItems.kind, "follow_up")))).toHaveLength(1);
expect(await db.select().from(agentWakeupRequests).where(and(eq(agentWakeupRequests.agentId, f.agent.id), eq(agentWakeupRequests.status, "deferred_issue_execution")))).toHaveLength(1);
await f.events.stop();
});
it("serializes tool-result writes and cursor reads with the binding mailbox lock", async () => {
const f = await fixture();
const { run, assignment } = await offeredWork(f);
const requestId = randomUUID();
await db.insert(dotRunnerOperations).values({ companyId: f.company.id, assignmentId: assignment.id, requestId,
digest: "test", command: { action: "tool" }, status: "admitted" });
let release!: () => void;
let held!: () => void;
const locked = new Promise<void>(resolve => { held = resolve; });
const gate = new Promise<void>(resolve => { release = resolve; });
const firstEvent = randomUUID();
const transaction = db.transaction(async tx => {
await tx.select().from(dotAgentBindings).where(eq(dotAgentBindings.id, f.snapshot.bindingId)).for("update");
await tx.insert(dotMailboxItems).values({ companyId: f.company.id, bindingId: f.snapshot.bindingId,
bindingGeneration: f.snapshot.bindingGeneration, kind: "operation_result", sourceEventId: firstEvent, references: {} });
held(); await gate;
});
await locked;
const port = f.broker.port({ binding: { companyId: f.company.id, agentId: f.agent.id, runId: run.id }, provider: { binding: f.snapshot } });
let written = false, read = false;
const secondEvent = randomUUID();
const writer = port.settle({ sourceEventId: secondEvent, payload: { requestId, binding: { ...f.snapshot, runId: run.id,
normalizedSessionId: assignment.normalizedSessionId, turnId: assignment.turnId, assignmentRevision: assignment.revision }, outcome: { status: "completed", result: {} } } } as Parameters<typeof port.settle>[0]).then(() => { written = true; });
const reader = f.broker.mailbox(f.principal).then(value => { read = true; return value; });
try {
await new Promise(resolve => setTimeout(resolve, 30));
expect(written).toBe(false); expect(read).toBe(false);
} finally { release(); }
await Promise.all([transaction, writer]);
const initial = await reader;
const next = await f.broker.mailbox(f.principal, initial.nextCursor);
expect([...initial.items, ...next.items].map(i => i.sourceEventId)).toEqual(expect.arrayContaining([firstEvent, secondEvent]));
await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop();
}, 30000);
it.each(["succeeded", "failed", "cancelled", "timed_out"] as const)("fences a stranded %s assignment before new admission without claiming external stop", async status => {
const f = await fixture();
const { run, assignment } = await offeredWork(f);
try {
await expect(f.broker.snapshot(f.company.id, f.agent.id, f.snapshot.bindingId)).rejects.toThrow("active assignment");
await db.update(heartbeatRuns).set({ status }).where(eq(heartbeatRuns.id, run.id));
await f.broker.snapshot(f.company.id, f.agent.id, f.snapshot.bindingId);
await f.broker.snapshot(f.company.id, f.agent.id, f.snapshot.bindingId);
expect((await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.id, assignment.id)))[0]?.status).toBe("fenced");
const inbox = await f.broker.mailbox(f.principal);
const fences = inbox.items.filter(item => item.assignmentId === assignment.id && item.kind === "authority_revoked");
expect(fences).toHaveLength(1);
expect(fences[0]?.references).toMatchObject({ assignmentId: assignment.id, runId: run.id, externalStopConfirmed: false });
await expect(createDotRunnerMcpTools(db).callTool(f.principal, "paperclip_dot_accept", { assignmentId: assignment.id, requestId: randomUUID() })).rejects.toThrow("revoked");
} finally { await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop(); }
}, 30000);
it("allows paused Dot fence delivery and acknowledgements without task authority", async () => {
const f = await fixture();
const { run, assignment } = await offeredWork(f);
const [issue] = await db.insert(issues).values({ companyId: f.company.id, title: "Work paused before inbox read", assigneeAgentId: f.agent.id }).returning();
await db.update(heartbeatRuns).set({ nativeIssueId: issue!.id }).where(eq(heartbeatRuns.id, run.id));
await db.insert(nativeRunFinalizations).values({ runId: run.id, companyId: f.company.id, issueId: issue!.id, phase: "observed" });
const port = f.broker.port({ binding: { companyId: f.company.id, agentId: f.agent.id, runId: run.id }, provider: { binding: f.snapshot } });
await port.dispatch({ sourceEventId: randomUUID(), payload: { kind: "authority_revoked", binding: {
...f.snapshot, runId: run.id, normalizedSessionId: assignment.normalizedSessionId,
turnId: assignment.turnId, assignmentRevision: assignment.revision,
} } });
await db.update(agents).set({ status: "paused" }).where(eq(agents.id, f.agent.id));
const principal = await f.oauth.authenticate(f.tokens.access_token);
const tools = createDotRunnerMcpTools(db);
await f.events.tick();
expect(f.received.at(-1)?.data.kind).toBe("authority_revoked");
const inbox = await tools.callTool(principal, "paperclip_dot_inbox", { after: 0 });
expect(inbox).toMatchObject({ result: { nextCursor: 0, items: [{ kind: "authority_revoked", assignmentId: assignment.id, references: { assignmentId: assignment.id } }] } });
const fencedAssignmentId = (inbox!.result as { items: Array<{ references: { assignmentId: string } }> }).items[0]!.references.assignmentId;
await expect(tools.callTool(principal, "paperclip_dot_tasks", {})).rejects.toThrow("authority");
await expect(tools.callTool(principal, "paperclip_dot_read", { assignmentId: assignment.id })).rejects.toThrow("authority");
await expect(tools.callTool(principal, "paperclip_dot_accept", { assignmentId: assignment.id, requestId: randomUUID() })).rejects.toThrow("authority");
expect(await tools.callTool(principal, "paperclip_dot_control_ack", { assignmentId: fencedAssignmentId, requestId: randomUUID() })).toMatchObject({ result: { status: "acknowledged", externalStopConfirmed: false } });
await f.events.unsubscribe(principal, f.subscription); await f.events.stop();
}, 30000);
it("authorizes admitted Dot reviews while preserving the worker and rejecting stale review authority", async () => {
const f = await fixture();
const companyId = f.company.id;
const [worker] = await db.insert(agents).values({ companyId, name: "Original worker", status: "active", adapterType: "paperclip_runner" }).returning();
const [issue] = await db.insert(issues).values({ companyId, title: "Review another agent's work", status: "in_review", statusVersion: 3, assigneeAgentId: worker!.id }).returning();
const issueId = issue!.id;
const [contract] = await db.insert(completionContracts).values({ companyId, issueId, revision: 1, schemaVersion: "paperclip.completion-contract.v1", policyVersion: "fixture",
risk: "low", completionAuthority: "agent", incompleteCriteriaPolicy: "block", contractJson: {}, canonicalSha256: randomUUID(), createdByActorType: "user", createdByActorId: f.userId }).returning();
const [source] = await db.insert(heartbeatRuns).values({ companyId, agentId: worker!.id, nativeIssueId: issueId, runtimeMode: "native", status: "succeeded",
completionContractId: contract!.id, completionContractSha256: contract!.canonicalSha256 }).returning();
const [result] = await db.insert(nativeRunResults).values({ companyId, issueId, runId: source!.id, completionContractId: contract!.id,
serverFingerprint: randomUUID(), schemaStatus: "accepted", resultJson: {}, canonicalSha256: randomUUID() }).returning();
const [assessment] = await db.insert(workAssessments).values({ companyId, issueId, runId: source!.id, contractId: contract!.id, resultId: result!.id,
triggerKind: "native_result", triggerActorCompanyId: companyId, priorIssueStatus: "in_progress", priorStatusVersion: 2, policyVersion: "fixture", assessmentJson: {}, inputDigest: randomUUID() }).returning();
const [decision] = await db.insert(statusDecisions).values({ companyId, issueId, runId: source!.id, assessmentId: assessment!.id, decisionVersion: 1,
policyVersion: "fixture", fromStatus: "in_progress", toStatus: "in_review", reasonCode: "explicit_review", decisionJson: { projectedStatusVersion: 3 }, decisionDigest: randomUUID(), applicationState: "applied" }).returning();
const [interaction] = await db.insert(issueThreadInteractions).values({ companyId, issueId, kind: "request_confirmation", sourceRunId: source!.id,
addresseeAgentId: f.agent.id, createdByAgentId: worker!.id,
payload: { version: 1, prompt: "Review the completed work.", target: { type: "custom", key: "native_completion_review", revisionId: decision!.id } } }).returning();
const { run, assignment } = await offeredWork(f);
const contextSnapshot = { issueId, nativeReviewInteractionId: interaction!.id, nativeReviewDecisionId: decision!.id };
await db.update(heartbeatRuns).set({ status: "running", runtimeMode: "native", nativeIssueId: issueId, nativeSessionId: assignment.normalizedSessionId, contextSnapshot }).where(eq(heartbeatRuns.id, run.id));
await db.update(issues).set({ lastStatusDecisionId: decision!.id, executionRunId: run.id, checkoutRunId: source!.id }).where(eq(issues.id, issueId));
await db.insert(nativeRunFinalizations).values({ runId: run.id, companyId, issueId, phase: "observed", controllerGeneration: assignment.controllerGeneration, leaseExpiresAt: new Date(Date.now() + 60_000) });
await expect(f.broker.read(f.principal, assignment.id)).resolves.toMatchObject({ assignmentId: assignment.id });
await expect(f.broker.operation(f.principal, assignment.id, randomUUID(), "accept", {})).resolves.toMatchObject({ status: "pending" });
await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, issueId));
await expect(f.broker.read(f.principal, assignment.id)).rejects.toThrow("execution authority");
await db.update(issues).set({ executionRunId: run.id }).where(eq(issues.id, issueId));
await db.update(issueThreadInteractions).set({ addresseeAgentId: worker!.id }).where(eq(issueThreadInteractions.id, interaction!.id));
await expect(f.broker.read(f.principal, assignment.id)).rejects.toThrow("execution authority");
await db.update(issueThreadInteractions).set({ addresseeAgentId: f.agent.id, status: "accepted", resolvedByAgentId: f.agent.id, resolvedByRunId: run.id }).where(eq(issueThreadInteractions.id, interaction!.id));
await db.update(issues).set({ status: "done", executionRunId: null }).where(eq(issues.id, issueId));
await expect(f.broker.read(f.principal, assignment.id)).resolves.toMatchObject({ assignmentId: assignment.id });
expect((await db.select().from(issues).where(eq(issues.id, issueId)))[0]?.assigneeAgentId).toBe(worker!.id);
await db.update(heartbeatRuns).set({ contextSnapshot: { ...contextSnapshot, nativeReviewDecisionId: randomUUID() } }).where(eq(heartbeatRuns.id, run.id));
await expect(f.broker.read(f.principal, assignment.id)).rejects.toThrow("execution authority");
await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop();
}, 30000);
it("verifies reconnects and wakes the same outstanding assignment after exhausted delivery", async () => {
const f = await fixture();
const { assignment } = await offeredWork(f);
await f.events.tick();
const previousEventId = f.received.at(-1)?.eventId;
const before = await f.broker.mailbox(f.principal);
const oldItem = before.items.find(item => item.assignmentId === assignment.id)!;
await db.update(mcpEventDeliveries).set({ attempts: 6, finishedAt: new Date(), outcome: "delivery_exhausted" })
.where(eq(mcpEventDeliveries.mailboxItemId, oldItem.id));
const failedReconnect = createPublicMcpEvents(db, f.oauth, async () => { throw new Error("personal dispatch forbidden"); },
{ enableDotRunner: true, fetch: async () => Response.json({ challenge: "wrong" }) });
await expect(failedReconnect.subscribe(f.principal, f.subscription)).rejects.toThrow();
expect((await f.broker.mailbox(f.principal, before.nextCursor)).items).toEqual([]);
const verificationCount = f.verifications();
await f.events.subscribe(f.principal, f.subscription);
expect(f.verifications()).toBe(verificationCount + 1);
const after = await f.broker.mailbox(f.principal, before.nextCursor);
expect(after.items).toHaveLength(1);
expect(after.items[0]).toMatchObject({ kind: "assignment", assignmentId: assignment.id });
expect(await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.bindingId, f.snapshot.bindingId))).toHaveLength(1);
await f.events.tick();
expect(f.received.at(-1)?.data.mailboxItemId).toBe(after.items[0]!.id);
expect(f.received.at(-1)?.eventId).not.toBe(previousEventId);
}, 30000);
it.each(["OAuth revocation", "refresh-token replay"])("%s fences Dot and cancels waiting work", async reason => {
const f = await fixture();
const { run, assignment } = await offeredWork(f);
const { app, personal } = gateway(f.oauth, { type: "board", source: "session", userId: f.userId });
// A token submitted to the other resource's revocation endpoint has no effect.
await personal.revokeToken(f.tokens.access_token, f.client.client_id);
expect(await f.broker.bindingForAgent(f.company.id, f.agent.id)).not.toBeNull();
if (reason === "OAuth revocation") {
expect((await request(app).post("/mcp/runner/oauth/revoke").send({ token: f.tokens.access_token, client_id: f.client.client_id })).status).toBe(200);
} else {
const refresh = { grant_type: "refresh_token", refresh_token: f.tokens.refresh_token, client_id: f.client.client_id, resource: f.oauth.config.resource };
await f.oauth.token(refresh);
await expect(f.oauth.token(refresh)).rejects.toThrow();
}
expect(await f.broker.bindingForAgent(f.company.id, f.agent.id)).toBeNull();
expect((await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.id, assignment.id)))[0]?.status).toBe("fenced");
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run.id)))[0]?.status).toBe("cancelled");
await expect(f.oauth.authenticate(f.tokens.access_token)).rejects.toThrow();
const next = await f.broker.createPairing({ companyId: f.company.id, agentId: f.agent.id, operatorId: f.userId });
await f.oauth.revokeToken(f.tokens.access_token, f.client.client_id);
expect((await f.broker.bindingForAgent(f.company.id, f.agent.id))?.id).toBe(next.bindingId);
}, 30000);
it("uses merged client metadata and scoped browser consent with a distinct Dot issuer", async () => {
const f = await fixture();
const clientId = `https://dot-${randomUUID()}.example/oauth.json`;
const redirectUri = "https://chatgpt.com/connector_platform/oauth/callback";
const oauth = createPublicMcpOAuth(db, f.oauth.config, { metadataFetch: async () => Response.json({
client_id: clientId, client_name: "Dot metadata client", redirect_uris: [redirectUri],
grant_types: ["authorization_code", "refresh_token", DEVICE_GRANT],
}) });
const { app, personal } = gateway(oauth, { type: "board", source: "session", userId: f.userId });
const metadata = await request(app).get("/.well-known/oauth-protected-resource/mcp/runner");
expect(metadata.body.authorization_servers).toEqual([oauth.config.origin + "/mcp/runner/oauth"]);
const issuer = await request(app).get("/.well-known/oauth-authorization-server/mcp/runner/oauth");
expect(issuer.body).toMatchObject({ issuer: oauth.config.origin + "/mcp/runner/oauth", client_id_metadata_document_supported: true,
scopes_supported: ["paperclip:agent", "offline_access"], device_authorization_endpoint: oauth.config.origin + "/mcp/runner/oauth/device_authorization" });
expect((await request(app).get("/.well-known/oauth-authorization-server")).body.issuer).toBe(oauth.config.origin);
const verifier = randomBytes(32).toString("base64url");
const input = { client_id: clientId, redirect_uri: redirectUri, response_type: "code", resource: oauth.config.resource,
company_id: f.company.id, code_challenge: createHash("sha256").update(verifier).digest("base64url"), code_challenge_method: "S256" };
const id = (await oauth.authorize(input)).split("/").at(-1)!;
await expect(personal.describeRequest(id, { type: "board", source: "session", userId: f.userId }, null)).rejects.toThrow();
const description = await request(app).get("/api/mcp/requests/" + id);
expect(description.body).toMatchObject({ agentConnection: true, requestedCompanyId: f.company.id, clientOrigin: new URL(clientId).origin });
expect(description.body.companies.map((c: { id: string }) => c.id)).toEqual([f.company.id]);
expect((await request(app).post(`/api/mcp/requests/${id}/consent`).set("Origin", oauth.config.origin)
.send({ decision: "approve", companyId: randomUUID(), allowWrites: false })).status).toBe(403);
const consent = await request(app).post(`/api/mcp/requests/${id}/consent`).set("Origin", oauth.config.origin)
.send({ decision: "approve", companyId: f.company.id, allowWrites: false });
expect(consent.status).toBe(200);
const redirect = new URL(consent.body.redirectUrl);
expect(redirect.searchParams.get("iss")).toBe(issuer.body.issuer);
const tokens = await oauth.token({ grant_type: "authorization_code", client_id: clientId, redirect_uri: redirectUri,
resource: oauth.config.resource, code: redirect.searchParams.get("code"), code_verifier: verifier });
expect((await oauth.authenticate(tokens.access_token)).grant).toMatchObject({ purpose: "agent", agentId: null, scopes: ["paperclip:agent"] });
await expect(personal.authenticate(tokens.access_token)).rejects.toThrow();
await expect(oauth.authorize({ ...input, scope: "paperclip:read" })).rejects.toMatchObject({ code: "invalid_scope" });
await f.events.unsubscribe(f.principal, f.subscription);
}, 30000);
it("routes merged device consent to Dot without granting personal or cross-company access", async () => {
const f = await fixture();
const { app, personal } = gateway(f.oauth, { type: "board", source: "session", userId: f.userId });
const client = await f.oauth.register({ client_name: "Dot device", redirect_uris: [], response_types: [], grant_types: [DEVICE_GRANT, "refresh_token"] }, randomUUID());
const started = await request(app).post("/mcp/runner/oauth/device_authorization")
.send({ client_id: client.client_id, resource: f.oauth.config.resource, scope: "paperclip:agent offline_access", company_id: f.company.id });
expect(started.status).toBe(200);
const codes = started.body;
expect(await f.oauth.ownsDevice(codes.user_code)).toBe(true);
await expect(personal.describeDevice(codes.user_code, { type: "none" })).rejects.toThrow();
const description = await request(app).get("/api/mcp/device").query({ user_code: codes.user_code });
expect(description.body).toMatchObject({ agentConnection: true, requestedCompanyId: f.company.id });
await db.update(companyMemberships).set({ membershipRole: "viewer" }).where(eq(companyMemberships.principalId, f.userId));
expect((await request(app).post("/api/mcp/device/consent").set("Origin", f.oauth.config.origin)
.send({ userCode: codes.user_code, decision: "approve", companyId: f.company.id, allowWrites: false })).status).toBe(403);
await db.update(companyMemberships).set({ membershipRole: "owner" }).where(eq(companyMemberships.principalId, f.userId));
const approved = await request(app).post("/api/mcp/device/consent").set("Origin", f.oauth.config.origin)
.send({ userCode: codes.user_code, decision: "approve", companyId: f.company.id, allowWrites: false });
expect(approved.body).toEqual({ status: "approved" });
const exchange = { grant_type: DEVICE_GRANT, client_id: client.client_id, resource: f.oauth.config.resource, device_code: codes.device_code };
const tokens = (await request(app).post("/mcp/runner/oauth/token").send(exchange)).body;
expect((await f.oauth.authenticate(tokens.access_token)).actor.type).toBe("none");
await expect(personal.authenticate(tokens.access_token)).rejects.toThrow();
const renewed = await f.oauth.token({ grant_type: "refresh_token", client_id: client.client_id, resource: f.oauth.config.resource, refresh_token: tokens.refresh_token });
expect((await f.oauth.authenticate(renewed.access_token)).grant.purpose).toBe("agent");
await expect(f.oauth.token(exchange)).rejects.toMatchObject({ code: "invalid_grant" });
await instanceSettingsService(db).updateExperimental({ enableOpenAiDot: false });
try {
expect((await request(app).get("/.well-known/oauth-protected-resource/mcp/runner")).status).toBe(503);
expect((await request(app).get("/.well-known/oauth-protected-resource/mcp/paperclip")).status).toBe(200);
} finally { await instanceSettingsService(db).updateExperimental({ enableOpenAiDot: true }); }
await f.events.unsubscribe(f.principal, f.subscription);
}, 30000);
it.each([
{ name: "Dot local Rust", managed: false, provider: "openai_dot" as const },
{ name: "Grok local Rust", managed: false, provider: "grok_bot" as const },
...(process.env.PAPERCLIP_DOT_DAYTONA_LIVE === "1" ? [{ name: "private Daytona Rust", managed: true, provider: "openai_dot" as const }] : []),
])("signed readiness, native tools and finalization through $name", async ({ managed, provider }) => {
const remote = managed ? await (await import("./helpers/dot-daytona.js")).startDotDaytona(root) : null;
try {
const f = await fixture({ provider });
const [issue] = await db.insert(issues).values({ companyId: f.company.id, title: "Save the arithmetic report", description: "Save 17 + 25 = 42 as a task document.",
status: "in_progress", workMode: "standard", assigneeAgentId: f.agent.id }).returning();
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.company.id, agentId: f.agent.id, status: "running", invocationSource: "assignment", triggerDetail: "system", contextSnapshot: { issueId: issue!.id } }).returning();
await db.update(issues).set({ executionRunId: run!.id, checkoutRunId: run!.id }).where(eq(issues.id, issue!.id));
const prepared = await prepareNativeHeartbeatRun({ db, run: run!, issue: issue!, environmentLeaseId: randomUUID() });
await db.insert(nativeRunFinalizations).values({ runId: run!.id, companyId: f.company.id, issueId: issue!.id, phase: "observed" });
const [boundRun] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, run!.id));
const [contract] = await db.select().from(completionContracts).where(eq(completionContracts.id, boundRun!.completionContractId!));
const runtimeContext = nativeRuntimeContextFixture();
await writeFile(join(root, "AGENTS.md"), "Use Paperclip tools to save the report.");
runtimeContext.instructions.bundle.rootPath = root;
runtimeContext.aggregateDigest = canonicalNativeRuntimeContextDigest(runtimeContext);
const execution = buildNativeExecutionInput({ companyId: f.company.id, agentId: f.agent.id, runId: run!.id, issue: issue!,
taskPrompt: issue!.description!, normalizedSessionId: prepared.normalizedSessionId, provider, dotBinding: f.snapshot,
workspace: { id: randomUUID(), cwd: root, repoUrl: null, repoRef: null, branchName: null }, runtimeContext,
completionContract: { id: contract!.id, sha256: contract!.canonicalSha256,
schemaVersion: contract!.schemaVersion, contract: contract!.contractJson as unknown as StrictCompletionContractInput } });
await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeExecutionInput: execution } }).where(eq(heartbeatRuns.id, run!.id));
const server = createServer(); await new Promise<void>(done => server.listen(0, "127.0.0.1", done));
const address = server.address(); if (!address || typeof address === "string") throw new Error("Missing test server");
setupRunnerPrpWebSocketServer(server, { apiUrl: `http://127.0.0.1:${address.port}` });
const observed: Array<Record<string, any>> = [];
const appCall = vi.fn().mockResolvedValue({ status: "completed", result: { content: [{ type: "text", text: "synthetic gateway result" }] } });
const gatewayTools = [{ name: "fixture:read_status", displayName: "Read fixture status", description: "Read a synthetic app", parametersSchema: { type: "object", properties: {}, additionalProperties: false }, risk: "read" }];
registerAssignedMcpGateway(db, { listToolsForNamedGateway: vi.fn().mockResolvedValue(gatewayTools), executeTool: appCall } as unknown as ToolGatewayService);
if (provider === "openai_dot") await db.update(agents).set({ adapterConfig: { ...f.agent.adapterConfig, dotBindingId: f.snapshot.bindingId, dotWorkspaceAccess: true, dotAttachmentAccess: true } }).where(eq(agents.id, f.agent.id));
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "fixture-cloud");
try {
for (const useRunnerd of [false, true]) {
await expect(executePaperclipNativeSession({ db, execution, runnerInstanceId: prepared.runnerInstanceId,
useRunnerd, turnTimeoutMs: 1000, onEvent: async () => {} }))
.rejects.toThrow("dot_cloud_requires_managed_runner");
}
} finally { vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", ""); }
const resultPromise = executePaperclipNativeSession({ db, execution,
...(remote ? { runnerExecutionTarget: remote.target, runnerRemoteBinaryPath: remote.runnerBinary, runnerIngressAuthorized: true }
: { dotWorkspaceRoot: root }),
runnerInstanceId: prepared.runnerInstanceId,
runnerEnvironment: { PAPERCLIP_NATIVE_MCP_NAME: "paperclip-assigned", PAPERCLIP_NATIVE_MCP_URL: "http://127.0.0.1:3217/mcp/gateways/dot-fixture", PAPERCLIP_NATIVE_MCP_TOKEN: "fixture-gateway-secret" },
useRunnerd: true, turnTimeoutMs: 45000, onEvent: async event => { observed.push(event); }, onLog: async () => {} });
// Attach a handler immediately: a bootstrap error must never become an unhandled rejection.
let startupError: unknown;
let startupOutcome: unknown;
void resultPromise.then(result => { startupOutcome = result; }, () => {});
void resultPromise.catch(error => { startupError = error; });
try {
await vi.waitFor(async () => { if (startupError) throw startupError; expect(await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.runId, run!.id))).toHaveLength(1); }, { timeout: 30000, interval: 50 });
const [assignment] = await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.runId, run!.id));
if (provider === "openai_dot") await f.events.tick();
expect(JSON.stringify(f.received)).not.toContain("17 + 25");
const work = await f.broker.read(f.principal, assignment!.id);
expect(work.accounting).toEqual({ usage: null, cost: null });
expect(work.status).toBe("offered");
expect(observed.filter(event => event.eventType === "turn.started")).toHaveLength(0);
expect(JSON.stringify(work)).not.toContain(root);
expect((work.tools as Array<{ operationId: string }>).some(tool => tool.operationId === "register_deliverable")).toBe(provider === "openai_dot");
expect(await f.broker.operation(f.principal, assignment!.id, randomUUID(), "tool", { name: "write_document", arguments: {} })).toMatchObject({ status: "rejected" });
await f.broker.operation(f.principal, assignment!.id, randomUUID(), "accept", {});
expect(JSON.stringify(work)).not.toContain("fixture-gateway-secret");
if (provider === "grok_bot") {
const files = new Map<string, Buffer>();
const transfers = createPublicMcpTransfers(db, f.oauth, async () => { throw new Error("Personal API dispatch forbidden"); }, {
provider: "local_disk", putFile: async input => {
if (!Buffer.isBuffer(input.body)) throw new Error("Expected bounded bytes");
const objectKey = input.objectKey!; files.set(objectKey, input.body);
return { provider: "local_disk", objectKey, contentType: input.contentType, byteSize: input.body.length,
sha256: createHash("sha256").update(input.body).digest("hex"), originalFilename: input.originalFilename };
}, getObject: async (_company, key) => ({ stream: Readable.from(files.get(key)!) }),
headObject: async () => ({ exists: true }), deleteObject: async (_company, key) => { files.delete(key); },
}, { pathPrefix: "/mcp/runner/files", agentSession: { authorize: (principal, sessionId, taskId, write, database) => dotRunnerBroker(database ?? db).authorizeFileSession(principal, sessionId, taskId, write) } });
const api = express(); api.use(transfers.router);
const body = Buffer.from("Grok arithmetic file: 42\n"), uploadId = randomUUID();
const args = { companyId: f.company.id, taskId: issue!.id, requestId: uploadId, filename: "report.txt", contentType: "text/plain", byteSize: body.length, sha256: createHash("sha256").update(body).digest("hex") };
const grant = await transfers.getUploadUrl(f.principal, args, assignment!.id);
const url = new URL(String(grant.url));
const uploaded = await request(api).put(url.pathname + url.search).set("Content-Type", "text/plain").send(body);
expect(uploaded.status).toBe(200);
const duplicate = await request(api).put(url.pathname + url.search).set("Content-Type", "text/plain").send(body);
expect(duplicate.body.attachment.id).toBe(uploaded.body.attachment.id);
expect(await issueService(db).getAttachmentById(uploaded.body.attachment.id)).toMatchObject({ createdByAgentId: f.agent.id, originatingRunId: run!.id, createdByUserId: null });
await expect(transfers.getUploadUrl(f.principal, { ...args, taskId: randomUUID() }, assignment!.id)).rejects.toThrow();
const download = new URL(String((await transfers.getDownloadUrl(f.principal, uploaded.body.attachment.id, assignment!.id)).url));
expect((await request(api).get(download.pathname + download.search)).status).toBe(200);
const pendingUpload = new URL(String((await transfers.getUploadUrl(f.principal, { ...args, requestId: randomUUID() }, assignment!.id)).url));
await db.update(issues).set({ workMode: "ask" }).where(eq(issues.id, issue!.id));
await expect(transfers.getUploadUrl(f.principal, { ...args, requestId: randomUUID() }, assignment!.id)).rejects.toThrow("work mode");
expect((await request(api).put(pendingUpload.pathname + pendingUpload.search).set("Content-Type", "text/plain").send(body)).status).toBeGreaterThanOrEqual(400);
await db.update(issues).set({ workMode: "standard" }).where(eq(issues.id, issue!.id));
await db.update(agents).set({ status: "paused" }).where(eq(agents.id, f.agent.id));
expect((await request(api).get(download.pathname + download.search)).status).toBeGreaterThanOrEqual(400);
await db.update(agents).set({ status: "active" }).where(eq(agents.id, f.agent.id));
}
const app = (work.tools as Array<{ operationId: string }>).find(tool => tool.operationId.startsWith("app_"))!;
if (provider === "openai_dot") {
expect(app).toBeTruthy();
if (remote) {
expect((work.tools as Array<{ operationId: string }>).some(tool => tool.operationId.startsWith("workspace_"))).toBe(false);
} else {
const missingReadId = randomUUID();
await f.broker.operation(f.principal, assignment!.id, missingReadId, "tool", { name: "workspace_read", arguments: { path: "missing-fixture.txt", offset: 0 } });
await vi.waitFor(async () => expect(await f.broker.operationStatus(f.principal, assignment!.id, missingReadId)).toMatchObject({ status: "completed", isError: true, result: { outcome: "failed", code: "runner_bridge_file_not_found" } }), { timeout: 10000 });
}
// A definite read failure must not poison later calls or finalization.
const attachmentListId = randomUUID();
expect((work.tools as Array<{ operationId: string }>).some(tool => tool.operationId === "read_task_attachment")).toBe(true);
await f.broker.operation(f.principal, assignment!.id, attachmentListId, "tool", { name: "list_task_attachments", arguments: {} });
await vi.waitFor(async () => expect(await f.broker.operationStatus(f.principal, assignment!.id, attachmentListId)).toMatchObject({ status: "completed", isError: false, result: { attachments: [] } }), { timeout: 10000 });
const appRequestId = randomUUID();
await f.broker.operation(f.principal, assignment!.id, appRequestId, "tool", { name: app.operationId, arguments: {} });
await vi.waitFor(async () => expect(await f.broker.operationStatus(f.principal, assignment!.id, appRequestId)).toMatchObject({ status: "completed", isError: false }), { timeout: 10000 });
expect(appCall).toHaveBeenCalledWith(expect.objectContaining({ gatewayPublicId: "dot-fixture", sessionToken: "fixture-gateway-secret", tool: "fixture:read_status" }));
} else {
expect(app).toBeUndefined();
expect((work.tools as Array<{ operationId: string }>).some(tool => ["get_workspace", "prepare_repository", "workspace_read"].includes(tool.operationId))).toBe(false);
}
await db.update(agents).set({ budgetMonthlyCents: 100, spentMonthlyCents: 100 }).where(eq(agents.id, f.agent.id));
await expect(f.broker.read(f.principal, assignment!.id)).rejects.toThrow("execution authority");
await db.update(agents).set({ budgetMonthlyCents: 0, spentMonthlyCents: 0, status: "paused" }).where(eq(agents.id, f.agent.id));
await expect(f.broker.read(f.principal, assignment!.id)).rejects.toThrow("authority");
await db.update(agents).set({ status: "active" }).where(eq(agents.id, f.agent.id));
await db.update(issues).set({ executionRunId: null, checkoutRunId: null }).where(eq(issues.id, issue!.id));
await expect(f.broker.read(f.principal, assignment!.id)).rejects.toThrow("execution authority");
await db.update(issues).set({ executionRunId: run!.id, checkoutRunId: run!.id }).where(eq(issues.id, issue!.id));
const writeId = randomUUID();
const args = { name: "write_document", arguments: { key: "report", title: "Arithmetic", body: "17 + 25 = 42.", baseRevisionId: null, idempotencyKey: writeId } };
await f.broker.operation(f.principal, assignment!.id, writeId, "tool", args);
await vi.waitFor(async () => expect(await f.broker.operationStatus(f.principal, assignment!.id, writeId)).toMatchObject({ status: "completed", isError: false }), { timeout: 10000 });
await f.broker.operation(f.principal, assignment!.id, writeId, "tool", args);
await expect(f.broker.operation(f.principal, assignment!.id, writeId, "tool", { ...args, arguments: { ...args.arguments, body: "changed" } })).rejects.toThrow("different arguments");
const document = await documentService(db).getIssueDocumentByKey(issue!.id, "report");
expect(document?.body).toBe("17 + 25 = 42.");
const result = { reportedWorkDisposition: "completed", summary: "Saved the arithmetic report.",
completionClaim: { contractRevision: execution.completionContract.contract.revision, objectiveSatisfied: true,
criteria: execution.completionContract.contract.criteria.map(c => ({ criterionId: c.id, status: "passed", evidenceRefs: [] })), remainingWork: [] },
evidence: [], verification: [{ commandOrCheck: "17 + 25", status: "pass" }] };
const completionId = randomUUID();
await f.broker.operation(f.principal, assignment!.id, completionId, "tool", { name: "paperclip_finish", arguments: result });
await vi.waitFor(async () => expect(await f.broker.operationStatus(f.principal, assignment!.id, completionId)).toMatchObject({ status: "completed", isError: false }), { timeout: 10000 });
expect(await f.broker.operationStatus(f.principal, assignment!.id, completionId)).toMatchObject({ result: { completionReport: {
schema: "paperclip.run_result.v1", reportedWorkDisposition: "done", artifacts: [], attentionRequests: [],
verification: [{ commandOrCheck: "17 + 25", status: "passed" }],
} } });
if (provider === "grok_bot") await db.update(grokBotDispatches).set({ status: "accepted", sentAt: new Date() })
.where(and(eq(grokBotDispatches.assignmentId, assignment!.id), eq(grokBotDispatches.kind, "mailbox_updated")));
await f.broker.operation(f.principal, assignment!.id, randomUUID(), "finish", { result });
if (provider === "grok_bot") {
await vi.waitFor(async () => expect(await db.select().from(grokBotDispatches).where(and(eq(grokBotDispatches.assignmentId, assignment!.id),
eq(grokBotDispatches.kind, "mailbox_updated"), inArray(grokBotDispatches.status, ["prepared", "sending", "accepted", "unknown"])))).toHaveLength(0));
expect(await grokRoutineService(db).recoveryNeeded(f.company.id, f.agent.id)).toBe(false);
}
await resultPromise;
// The heartbeat records its controller workspace settlement before the
// ordinary status finalizer commits the task disposition.
await db.insert(workspaceOperations).values({ companyId: f.company.id, heartbeatRunId: run!.id, issueId: issue!.id,
phase: "workspace_finalize", status: "succeeded", exitCode: 0, cwd: root, finishedAt: new Date() });
await finalizeNativeRun({ db, runId: run!.id, workspaceFinalizeStatus: "succeeded" });
await finalizeNativeRun({ db, runId: run!.id, workspaceFinalizeStatus: "succeeded" });
expect(await db.select().from(nativeRunResults).where(eq(nativeRunResults.runId, run!.id))).toHaveLength(1);
expect((await db.select().from(issues).where(eq(issues.id, issue!.id)))[0]?.status).toBe("done");
expect(await db.select().from(dotRunnerOperations).where(eq(dotRunnerOperations.requestId, writeId))).toHaveLength(1);
expect(startupOutcome).toMatchObject({ exitCode: 0, model: null, nativeFinalization: { providerSessionId: null, driverKind: provider === "grok_bot" ? "grok_bot_mcp" : "openai_dot_mcp" } });
expect(observed.filter(event => event.eventType === "turn.started")).toHaveLength(1);
} catch (error) {
if (startupError) throw startupError;
await Promise.race([resultPromise.catch(() => {}), new Promise(resolve => setTimeout(resolve, 1000))]);
if (startupError) throw startupError;
throw error;
} finally { if (provider === "openai_dot") await f.events.unsubscribe(f.principal, f.subscription); await f.events.stop(); server.closeAllConnections(); await new Promise<void>(done => server.close(() => done())); }
} finally { await remote?.close(); }
}, 360000);
it.each(["failed", "cancelled", "timed_out"])("reports %s admission without creating another wake or waiting for a nonexistent assignment", async status => {
const f = await fixture();
const { run } = await offeredWork(f);
await db.update(heartbeatRuns).set({ status }).where(eq(heartbeatRuns.id, run.id));
const requestId = randomUUID(), issueId = randomUUID();
await db.insert(agentWakeupRequests).values({ companyId: f.company.id, agentId: f.agent.id,
source: "assignment", status: "admitted", runId: run.id, payload: { issueId },
idempotencyKey: `dot-work:${f.snapshot.bindingId}:${f.snapshot.bindingGeneration}:${requestId}` });
const result = await f.broker.requestWork(f.principal, issueId, requestId);
expect(result).toMatchObject({ status: "admission_failed", runId: run.id, admissionStatus: status });
expect(result.message).toContain("Stop waiting");
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, f.agent.id))).toHaveLength(1);
});
it("uses the app-configured heartbeat for Dot-originated managed admission and cancellation", async () => {
const f = await fixture();
const wakeup = vi.fn().mockResolvedValue(null);
const cancelRun = vi.fn().mockResolvedValue(null);
createDotRunnerMcpTools(db, { wakeup, cancelRun });
try {
const requestId = randomUUID();
const intake = await f.broker.requestTurn(f.principal, "Managed intake context test", requestId);
expect(wakeup).toHaveBeenCalledExactlyOnceWith(f.agent.id, expect.objectContaining({
source: "assignment", requestedByActorType: "agent", requestedByActorId: f.agent.id,
durableDotRequest: expect.objectContaining({ issueId: intake.issueId, requestId }),
}));
const assignment = await offeredWork(f);
await f.broker.revoke(f.company.id, f.agent.id, f.userId);
expect(cancelRun).toHaveBeenCalledExactlyOnceWith(assignment.run.id, "Dot connection revoked");
} finally { dotRunnerBroker(db, { heartbeat: heartbeatService(db) }); }
});
it("revocation, membership loss and a foreign company cannot recover authority", async () => {
const f = await fixture(); const other = await fixture();
await expect(f.broker.authorizeBinding(other.principal, f.company.id, f.snapshot.bindingId)).rejects.toThrow();
await db.update(companyMemberships).set({ status: "suspended" }).where(eq(companyMemberships.principalId, f.userId));
await expect(f.broker.mailbox(f.principal)).rejects.toThrow();
await db.update(companyMemberships).set({ status: "active" }).where(eq(companyMemberships.principalId, f.userId));
await f.broker.revoke(f.company.id, f.agent.id, f.userId);
await expect(f.broker.mailbox(f.principal)).rejects.toThrow();
await expect(f.oauth.authorizeGrant(f.principal.grant.id)).rejects.toThrow();
expect(await db.select().from(dotMailboxItems).where(eq(dotMailboxItems.bindingId, f.snapshot.bindingId))).toHaveLength(1);
await other.events.unsubscribe(other.principal, other.subscription);
}, 30000);
});