fix: carry AI connections through hiring and unblock task execution (#13438)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents hire other agents and assign tasks to them.
> - Managed AI connections must follow those hires across legacy and
native runners.
> - Missing accounts should pause task execution and let the user
connect from the task.
> - Subscription contention must wait without asking for new
credentials.
> - This pull request fixes these paths and the native tool and Daytona
staging failures found during live tests.
> - The result is a working hire, subtask, and connection setup flow on
local and remote runners.

## Linked Issues or Issue Description

**What happened?**

A managed Claude or Codex agent could hire a teammate without a usable
AI binding. Cross-provider hiring could fail before the user had a
chance to connect the new provider. First-time task setup did not show
the existing AI credential form inline. A busy subscription could
request a new connection. Native API replies could stop the parent after
a hire had already committed. Fresh Daytona sandboxes could fail to
extract read-only skill directories created on macOS.

**Expected behavior**

Compatible hires inherit the managed connection choice. A hire for
another provider uses the responsible user's default. If that account is
missing, the hire succeeds and the task asks for a connection.
Completing setup in the task resumes work automatically. Explicit child
auth settings and existing unmanaged login paths keep precedence.
Shared-account access checks remain in force.

**Steps to reproduce**

1. Connect a Claude or Codex parent with a managed AI account.
2. Ask it to hire one agent of each provider and create a self-assigned
subtask.
3. Assign work to both hires without connecting the second provider
first.
4. Connect the missing provider from its task card.
5. Check that all tasks finish and same-provider work uses the original
account.
6. Repeat with native runners and fresh Daytona sandboxes. The opt-in
browser suite in `tests/hiring-ai-connections/README.md` performs these
steps.

**Paperclip version or commit**

The live failures were reproduced from `f2c5e54dc`. The branch is
rebased onto `5282cabde`.

**Deployment mode**

Isolated local development instance. Legacy CLI and native runners.
Local execution and ephemeral Daytona sandboxes.

Related work: Refs #13247 for managed AI connections. Refs #13268 for
legacy credential-reference inheritance, which this branch preserves.
Refs #13432 for a concurrent managed-inheritance fix. This PR also
covers cross-provider task setup, subscription waits, native API
replies, and Daytona extraction. It permits missing responsible-user
defaults at hire time; restricted shared selections still fail.

## What Changed

- Apply managed connection defaults to both agent creation routes.
Preserve explicit auth choices and legacy credential-reference
inheritance.
- Allow hires before their responsible user connects the provider. Keep
approval gates, company boundaries, and shared-account access checks.
- Reuse the production AI credential form inside the pending task card.
Resume the task after setup.
- Retry subscription lease contention without consuming the
provider-failure allowance or creating a connection request.
- Require task execution-lock ownership when scheduling, promoting, and
dispatching subscription retries. Recheck ownership under the issue row
lock.
- Rename the HTTP operation identity at the native tool boundary so it
cannot override the runner's operation identity.
- Delay directory permission restoration during Daytona extraction.
Preserve the final read-only modes.
- Add database-backed regressions, real browser acceptance tests, and
Storybook states. Document setup and run-log behavior.

## Verification

- Six real browser scenarios passed: both parent providers on legacy
local and legacy Daytona; native Codex locally; native Claude on
Daytona. Each scenario hires both providers, completes a self-subtask
and assigned work, and connects the missing provider inline with
automatic continuation.
- Successful runs verify the account, responsible user, runner mode, and
Daytona lease. All 18 test sandboxes were deleted.
- Live authentication used API keys. Subscription inheritance, lease
contention, and retry have integration coverage. Fresh subscription
OAuth sign-in was not automated.
- Red/green tests reproduced missing bindings, missing inline forms,
subscription contention, native API reply failure, and GNU tar
permission failure.
- Seven Storybook browser checks passed. They cover both providers,
method selection, narrow layout, completion, cancellation, and invalid
credentials.
- Full local suite coverage completed before rebase. Initial timing and
fixture startup failures passed unchanged on isolated reruns. The first
full command did not exit cleanly; the remaining workspace and
serialized groups were completed separately.
- After rebase, 107 hiring/auth/retry tests and 59 native API,
task-card, and Daytona tests passed. The full workspace typecheck,
production build, and token gates passed again. Storybook build passed
before rebase.

- Review fixes: 169 hiring/retry/dispatch tests, 37 adjacent tests, and
four explicit cancellation-race cases passed. Eight cross-provider cases
cover stale auth keys on both creation routes and both runner types.
Server typecheck and build passed.

- Final CI on `ee4890837a8a4913e07453392b9a75969580dae1`: 32 checks
passed. Two optional Storybook jobs were skipped. The full server,
workspace, browser, native runner, build, typecheck, and release checks
passed.
- Three unchanged tests initially failed on a busy port, a chat row-lock
race, and preview-server readiness. Each affected job passed after one
CI rerun. Isolated local checks also passed: 41 credential tests, the
chat-concurrency case, and 25 preview-runtime tests.
- Greptile reviewed the final commit at 5/5. Both review threads are
resolved. GitHub reports no merge conflicts.

## Risks

- A missing personal account now defers authentication to the first
task. Explicit incompatible bindings and restricted shared accounts
still fail at hire time.
- An inherited personal default uses the responsible user's existing
authorization to install access for the new agent. It never copies
credentials or another user's identity.
- Subscription contention retries after a delay and rechecks task
eligibility. It does not consume the provider-failure budget.
- Native hiring uses the existing managed API-tools opt-in. Remote
native runners require a matching Linux binary and provider pack, as
documented in the acceptance README.
- No schema changes. Live tests make paid provider calls and remain
opt-in.

## Model Used

OpenAI GPT-6 (`gpt-6-astra`) in Codex, with reasoning, repository
inspection, code execution, browser automation, and API tools. The
context-window size is not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-09-14 16:15:48 -05:00
1 parent 5282cabde8
commit f912ecaacf
26 files changed
+1001 -26

No files matched your search

@@ -0,0 +1,242 @@
import { randomUUID } from "node:crypto";
import { mkdtemp, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import express from "express";
import request from "supertest";
import { eq } from "drizzle-orm";
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { agents, companies, companyMemberships, createDb, heartbeatRuns, issues, issueThreadInteractions, principalPermissionGrants, toolConnectionInstalls } from "@paperclipai/db";
import { type AiConnectionBinding } from "@paperclipai/shared";
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { agentRoutes } from "../routes/agents.js";
import { errorHandler } from "../middleware/index.js";
import { aiConnectionService } from "../services/ai-connections.js";
import { heartbeatService } from "../services/heartbeat.js";
import { getServerAdapter, registerServerAdapter, unregisterServerAdapter } from "../adapters/index.js";
import { prepareManagedAiRuntime } from "../services/ai-connection-runtime.js";
import { secretService } from "../services/secrets.js";
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let home: string;
beforeAll(async () => {
home = await mkdtemp(path.join(os.tmpdir(), "paperclip-hire-ai-"));
vi.stubEnv("PAPERCLIP_HOME", home);
vi.stubEnv("PAPERCLIP_INSTANCE_ID", "hire-ai");
database = await startEmbeddedPostgresTestDatabase("paperclip-hire-ai-db-");
db = createDb(database.connectionString);
}, 90_000);
afterAll(async () => {
await database?.cleanup();
vi.unstubAllEnvs();
if (home) await rm(home, { recursive: true, force: true });
});
async function fixture(provider: "anthropic" | "openai", method: "api_key" | "subscription" = "api_key") {
const companyId = randomUUID();
const agentId = randomUUID();
const userId = `owner-${companyId}`;
const binding = { provider, method, mode: "responsible_user" } as const;
const adapterType = provider === "anthropic" ? "claude_local" : "codex_local";
await db.insert(companies).values({ id: companyId, name: "Hiring connection test", issuePrefix: `H${companyId.slice(0, 7)}`, defaultResponsibleUserId: userId });
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: userId, membershipRole: "owner", status: "active" });
await db.insert(principalPermissionGrants).values({ companyId, principalType: "user", principalId: userId, permissionKey: "agents:create" });
await db.insert(agents).values({ id: agentId, companyId, name: "Manager", role: "ceo", adapterType, runtimeConfig: { aiConnection: binding } });
const [run] = await db.insert(heartbeatRuns).values({ companyId, agentId, status: "running", responsibleUserId: userId }).returning();
const app = express();
app.use(express.json());
app.use((req, _res, next) => {
req.actor = { type: "agent", agentId, companyId, runId: run!.id, source: "agent_jwt", onBehalfOfUserId: userId, onBehalfOfMemberships: [{ companyId, membershipRole: "owner", status: "active" }] };
next();
});
app.use("/api", agentRoutes(db));
app.use(errorHandler);
const credential = method === "api_key" ? "fixture-api-key" : provider === "anthropic" ? "fixture-subscription-token" : JSON.stringify({ tokens: { access_token: "fixture-access", refresh_token: "fixture-refresh", id_token: "fixture-id", account_id: "fixture-account" } });
const account = await aiConnectionService(db).save(companyId, userId, {
provider, method, name: "Manager's connection", ownership: "personal", agentIds: [agentId], allAgents: false,
...(method === "api_key" ? { apiKey: credential } : { loginSessionId: "fixture" }),
}, credential);
return { app, companyId, agentId, userId, binding, adapterType, account, runId: run!.id };
}
function hired(response: request.Response) {
expect(response.status, JSON.stringify(response.body)).toBe(201);
return response.body.agent ?? response.body;
}
describe("agent-created hires use managed AI connections", () => {
for (const endpoint of ["agent-hires", "agents"]) {
it.each([
["anthropic", "api_key"], ["anthropic", "subscription"],
["openai", "api_key"], ["openai", "subscription"],
] as const)(`${endpoint}: %s hires its own provider using the same %s binding`, async (provider, method) => {
const f = await fixture(provider, method);
const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({ name: "Teammate", role: "engineer", adapterType: f.adapterType, reportsTo: f.agentId }));
expect(agent.runtimeConfig.aiConnection).toEqual(f.binding);
const runtime = await prepareManagedAiRuntime(db, { companyId: f.companyId, agentId: agent.id, responsibleUserId: f.userId, adapterType: agent.adapterType, binding: agent.runtimeConfig.aiConnection, config: agent.adapterConfig });
try {
expect(runtime.attribution).toMatchObject({ connectionId: f.account.connectionId, grantId: f.account.grantId, method, responsibleUserId: f.userId });
} finally { await runtime.cleanup(); }
});
}
for (const endpoint of ["agent-hires", "agents"]) {
it.each([
["ANTHROPIC_API_KEY", "child-key"],
["ANTHROPIC_API_KEY", ""],
["CLAUDE_CONFIG_DIR", "/tmp/child-claude-home"],
["ANTHROPIC_BASE_URL", "https://example.invalid"],
])(`${endpoint}: preserves an explicit child auth setting %s=%s`, async (key, value) => {
const f = await fixture("anthropic");
const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({
name: "Explicit auth", role: "engineer", adapterType: f.adapterType,
adapterConfig: { env: { [key]: value } },
}));
expect(agent.runtimeConfig.aiConnection).toBeUndefined();
const [saved] = await db.select().from(agents).where(eq(agents.id, agent.id));
expect((saved.adapterConfig.env as Record<string, unknown>)[key]).toEqual({ type: "plain", value });
});
}
it.each([true, false])("managed bindings do not inherit legacy credentials (managed parent: %s)", async (managedParent) => {
const f = await fixture("openai");
const secret = await secretService(db).create(f.companyId, {
name: "Legacy parent key", provider: "local_encrypted", value: "fixture-legacy-key",
});
await db.update(agents).set({
runtimeConfig: managedParent ? { aiConnection: f.binding } : {},
adapterConfig: { env: { OPENAI_API_KEY: { type: "secret_ref", secretId: secret.id } } },
}).where(eq(agents.id, f.agentId));
const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({
name: "Managed child", role: "engineer", adapterType: f.adapterType,
...(managedParent ? {} : { runtimeConfig: { aiConnection: f.binding } }),
}));
expect(agent.runtimeConfig.aiConnection).toEqual(f.binding);
expect(agent.adapterConfig.env?.OPENAI_API_KEY).toBeUndefined();
});
it("keeps unmanaged parent hires on their existing authentication path", async () => {
const f = await fixture("openai");
await db.update(agents).set({ runtimeConfig: {} }).where(eq(agents.id, f.agentId));
const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({
name: "Legacy authentication", role: "engineer", adapterType: f.adapterType,
}));
expect(agent.runtimeConfig.aiConnection).toBeUndefined();
});
it.each(["anthropic", "openai"] as const)("%s can hire the other provider before that user connects it", async (provider) => {
const f = await fixture(provider);
const otherProvider = provider === "anthropic" ? "openai" : "anthropic";
const adapterType = otherProvider === "anthropic" ? "claude_local" : "codex_local";
const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Other provider", role: "engineer", adapterType }));
expect(agent.runtimeConfig.aiConnection).toMatchObject({ provider: otherProvider, mode: "responsible_user" });
await expect(prepareManagedAiRuntime(db, { companyId: f.companyId, agentId: agent.id, responsibleUserId: f.userId, adapterType, binding: agent.runtimeConfig.aiConnection, config: agent.adapterConfig })).rejects.toMatchObject({ details: { code: "ai_connection_default_missing" } });
expect(agent.status).toBe("idle");
});
for (const endpoint of ["agent-hires", "agents"]) {
it.each([
["anthropic", "codex_local", {}, "ANTHROPIC_API_KEY"],
["openai", "claude_local", {}, "OPENAI_API_KEY"],
["anthropic", "paperclip_runner", { provider: "codex" }, "ANTHROPIC_API_KEY"],
["openai", "paperclip_runner", { provider: "acpx", acpxAgent: "claude" }, "OPENAI_API_KEY"],
] as const)(`${endpoint}: ignores the %s auth key for a different provider in %s`, async (provider, adapterType, config, key) => {
const f = await fixture(provider);
const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/${endpoint}`).send({
name: "Cross-provider config", role: "engineer", adapterType,
adapterConfig: { ...config, env: { [key]: "leftover-parent-setting" } },
}));
expect(agent.runtimeConfig.aiConnection).toMatchObject({
provider: provider === "anthropic" ? "openai" : "anthropic", mode: "responsible_user",
});
});
}
it("accepts an explicit personal default before authentication, including approval-gated hires", async () => {
const f = await fixture("anthropic");
await db.update(companies).set({ requireBoardApprovalForNewAgents: true }).where(eq(companies.id, f.companyId));
const binding: AiConnectionBinding = { provider: "openai", method: "subscription", mode: "responsible_user" };
const response = await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Future Codex", role: "engineer", adapterType: "codex_local", runtimeConfig: { aiConnection: binding } });
const agent = hired(response);
expect(agent.runtimeConfig.aiConnection).toEqual(binding);
expect(agent.status).toBe("pending_approval");
expect(response.body.approval.payload.runtimeConfig.aiConnection).toEqual(binding);
expect(await db.select().from(toolConnectionInstalls).where(eq(toolConnectionInstalls.targetId, agent.id))).toEqual([]);
});
it.each(["anthropic", "openai"] as const)("inherits %s when the hire uses the native runner", async (provider) => {
const f = await fixture(provider, "subscription");
const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Native teammate", role: "engineer", adapterType: "paperclip_runner", adapterConfig: provider === "anthropic" ? { provider: "acpx", acpxAgent: "claude" } : { provider: "codex" } }));
expect(agent.runtimeConfig.aiConnection).toEqual(f.binding);
const runtime = await prepareManagedAiRuntime(db, { companyId: f.companyId, agentId: agent.id, responsibleUserId: f.userId, adapterType: agent.adapterType, binding: agent.runtimeConfig.aiConnection, config: agent.adapterConfig });
try { expect(runtime.attribution.connectionId).toBe(f.account.connectionId); } finally { await runtime.cleanup(); }
});
it.each([true, false])("preserves shared connection access boundaries (company access: %s)", async (allAgents) => {
const f = await fixture("anthropic");
const account = await aiConnectionService(db).save(f.companyId, f.userId, { provider: "anthropic", method: "api_key", name: "Shared Claude", ownership: "shared", apiKey: "fixture", agentIds: [f.agentId], allAgents }, "fixture");
const binding = { provider: "anthropic", method: "api_key", mode: "shared", ...account };
await db.update(agents).set({ runtimeConfig: { aiConnection: binding } }).where(eq(agents.id, f.agentId));
const response = await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Shared teammate", role: "engineer", adapterType: f.adapterType });
if (!allAgents) {
expect(response.status).toBe(403);
expect(await db.select().from(agents).where(eq(agents.companyId, f.companyId))).toHaveLength(1);
} else {
const agent = hired(response);
expect(agent.runtimeConfig.aiConnection).toEqual(binding);
const runtime = await prepareManagedAiRuntime(db, { companyId: f.companyId, agentId: agent.id, responsibleUserId: f.userId, adapterType: agent.adapterType, binding: agent.runtimeConfig.aiConnection, config: agent.adapterConfig });
try { expect(runtime.attribution.connectionId).toBe(account.connectionId); } finally { await runtime.cleanup(); }
}
});
it("still rejects an explicitly incompatible provider without creating a hire", async () => {
const f = await fixture("anthropic");
const response = await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Wrong provider", role: "engineer", adapterType: "codex_local", runtimeConfig: { aiConnection: f.binding } });
expect(response.status).toBe(422);
expect(response.body.details.code).toBe("ai_connection_incompatible");
expect(await db.select().from(agents).where(eq(agents.companyId, f.companyId))).toHaveLength(1);
});
});
describe("hired agents sharing a subscription", () => {
it.each(["anthropic", "openai"] as const)("waits for the %s parent and resumes without asking for a new connection", async (provider) => {
const f = await fixture(provider, "subscription");
const agent = hired(await request(f.app).post(`/api/companies/${f.companyId}/agent-hires`).send({ name: "Waiting teammate", role: "engineer", adapterType: f.adapterType, reportsTo: f.agentId, adapterConfig: { cwd: home, engine: "cli" }, runtimeConfig: { heartbeat: { enabled: false } } }));
const [issue] = await db.insert(issues).values({ companyId: f.companyId, title: "Subscription child task", status: "todo", assigneeAgentId: agent.id, responsibleUserId: f.userId, createdByUserId: f.userId }).returning();
const parentRuntime = await prepareManagedAiRuntime(db, { companyId: f.companyId, agentId: f.agentId, responsibleUserId: f.userId, adapterType: f.adapterType, binding: f.binding, config: { cwd: home } });
const execute = vi.fn(async () => {
await db.update(issues).set({ status: "done", completedAt: new Date() }).where(eq(issues.id, issue.id));
return { exitCode: 0, signal: null, timedOut: false, resultJson: {} };
});
registerServerAdapter({ ...getServerAdapter(f.adapterType), execute });
const heartbeat = heartbeatService(db);
let parentReleased = false;
try {
const run = await heartbeat.invoke(agent.id, "assignment", { issueId: issue.id, wakeReason: "issue_assigned", responsibleUserId: f.userId }, "system");
expect(run).not.toBeNull();
await expect.poll(async () => (await heartbeat.getRun(run!.id))?.status, { timeout: 20_000 }).not.toBe("running");
expect(await heartbeat.getRun(run!.id)).toMatchObject({ status: "cancelled", errorCode: "ai_connection_busy" });
let retry: typeof heartbeatRuns.$inferSelect | undefined;
await expect.poll(async () => { [retry] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.retryOfRunId, run!.id)); return retry?.status; }, { timeout: 10_000 }).toBe("scheduled_retry");
expect(execute).not.toHaveBeenCalled();
expect(await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.issueId, issue.id))).toHaveLength(0);
expect((await db.select().from(issues).where(eq(issues.id, issue.id)))[0].executionRunId).toBe(retry!.id);
parentReleased = true;
await parentRuntime.cleanup();
const promoted = await heartbeat.promoteDueScheduledRetries(new Date(retry!.scheduledRetryAt!.getTime() + 1000));
expect(promoted.runIds).toContain(retry!.id);
await heartbeat.resumeQueuedRuns();
await expect.poll(async () => (await heartbeat.getRun(retry!.id))?.status, { timeout: 20_000 }).toBe("succeeded");
expect(execute).toHaveBeenCalledTimes(1);
expect((await heartbeat.getRun(retry!.id))?.contextSnapshot?.aiConnection).toMatchObject({ connectionId: f.account.connectionId, responsibleUserId: f.userId, method: "subscription" });
} finally {
if (!parentReleased) await parentRuntime.cleanup();
await db.update(heartbeatRuns).set({ status: "cancelled", finishedAt: new Date() }).where(eq(heartbeatRuns.id, f.runId));
await heartbeat.drainActiveRunExecutions();
unregisterServerAdapter(f.adapterType);
}
});
});
@@ -1374,6 +1374,55 @@ describeEmbeddedPostgres("heartbeat bounded retry scheduling", () => {
expect(issue?.executionRunId).toBe(retryRuns[0]?.id);
});
it.each(["schedule", "transaction", "promote", "dispatch"] as const)(
"suppresses a busy-subscription retry whose task lock was cleared before %s",
async (phase) => {
const { companyId, issueId, runId, now } = await seedMaxTurnFixture();
await db.update(heartbeatRuns).set({
status: "cancelled", errorCode: "ai_connection_busy",
resultJson: { executionRecovery: { kind: "ai_connection_wait", providerWorkStarted: false } },
}).where(eq(heartbeatRuns.id, runId));
const clearLock = () => db.update(issues).set({ executionRunId: null }).where(eq(issues.id, issueId));
if (phase === "schedule") await clearLock();
const transaction = db.transaction.bind(db);
let raceApplied = false;
const transactionSpy = phase === "transaction"
? vi.spyOn(db, "transaction").mockImplementationOnce(async (callback, config) => {
await clearLock();
raceApplied = true;
return transaction(callback, config);
})
: null;
const scheduled = await heartbeat.scheduleBoundedRetry(runId, {
now, retryReason: "ai_connection_busy", wakeReason: "ai_connection_busy_retry",
maxAttempts: 1, delayMs: 1_000,
}).finally(() => transactionSpy?.mockRestore());
if (phase === "transaction") expect(raceApplied).toBe(true);
if (phase === "schedule" || phase === "transaction") {
expect(scheduled).toMatchObject({ outcome: "not_scheduled", errorCode: "issue_execution_lock_changed" });
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.retryOfRunId, runId))).toHaveLength(0);
return;
}
expect(scheduled.outcome).toBe("scheduled");
if (scheduled.outcome !== "scheduled") return;
if (phase === "promote") await clearLock();
const promotion = await heartbeat.promoteDueScheduledRetries(scheduled.dueAt);
if (phase === "promote") {
expect(promotion).toEqual({ promoted: 0, runIds: [] });
} else {
expect(promotion.runIds).toContain(scheduled.run.id);
await clearLock();
const adapter = createPostgresRunDispatchAdapter(db);
expect(await adapter.cancelStaleQueuedRun({
companyId, runId: scheduled.run.id, expectedStatus: "queued", now: scheduled.dueAt,
})).toMatchObject({ outcome: "cancelled", errorCode: "issue_execution_lock_changed" });
}
expect(await heartbeat.getRun(scheduled.run.id)).toMatchObject({
status: "cancelled", errorCode: "issue_execution_lock_changed",
});
},
);
it("does not promote a duplicate max-turn continuation that does not own the issue lock", async () => {
const { companyId, agentId, issueId, runId, now } = await seedMaxTurnFixture();
@@ -113,6 +113,7 @@ function readNonEmptyString(value: unknown): string | null {
function classifyRetryReasonKind(retryReason: string | null): RetryReasonKind {
if (retryReason === MAX_TURN_CONTINUATION_RETRY_REASON) return "max_turn_continuation";
if (retryReason === ISSUE_DISPOSITION_REPAIR_RETRY_REASON) return "disposition_repair";
if (retryReason === "ai_connection_busy") return "ai_connection_wait";
if (retryReason === "native_safe_replacement") return "native_safe_replacement";
return "other";
}
@@ -274,7 +275,7 @@ export function createPostgresRunDispatchAdapter(
runAgentId: input.agentId,
issueId,
retryReasonKind,
enforceIssueExecutionLock: retryReasonKind === "max_turn_continuation",
enforceIssueExecutionLock: retryReasonKind === "max_turn_continuation" || retryReasonKind === "ai_connection_wait",
isNonAssigneeWorkspaceBusyRetry: isNonAssigneeWorkspaceBusyRetry(retryReason, input.contextSnapshot),
budgetBlock: null,
agentInvokable: true,
@@ -5,11 +5,12 @@
// then packs the result into a facts object. This file only branches on
// that facts object; it never queries a database or reads the clock.
/** The retry reason a run carries, reduced to the three kinds a gate cares about. */
/** The retry reason a run carries, reduced to the kinds a gate cares about. */
export type RetryReasonKind =
| "max_turn_continuation"
| "disposition_repair"
| "native_safe_replacement"
| "ai_connection_wait"
| "other";
export type BudgetBlockFacts = {
@@ -403,7 +404,7 @@ export function decideScheduledRetryGate(
}
const lockOutcome = decideExecutionLock({
requiresExecutionLock: requiresInProgress && facts.enforceIssueExecutionLock,
requiresExecutionLock: (requiresInProgress || facts.retryReasonKind === "ai_connection_wait") && facts.enforceIssueExecutionLock,
runId: facts.runId,
issueExecutionRunId: facts.issueExecutionRunId,
});
@@ -411,7 +412,7 @@ export function decideScheduledRetryGate(
return {
allowed: false,
reason:
"Scheduled max-turn continuation suppressed because the issue execution lock belongs to a different run",
"Scheduled retry suppressed because the issue execution lock belongs to a different run",
errorCode: "issue_execution_lock_changed",
issueId: facts.issueId,
details: {
@@ -629,7 +630,7 @@ export function decideQueuedRunStaleness(
}
const lockOutcome = decideExecutionLock({
requiresExecutionLock: requiresInProgress,
requiresExecutionLock: requiresInProgress || facts.retryReasonKind === "ai_connection_wait",
runId: facts.runId,
issueExecutionRunId: facts.issueExecutionRunId,
});
@@ -638,7 +639,7 @@ export function decideQueuedRunStaleness(
stale: true,
errorCode: "issue_execution_lock_changed",
reason:
"Cancelled because max-turn continuation no longer owns the issue execution lock before the queued run could start",
"Cancelled because the retry no longer owns the issue execution lock before the queued run could start",
details: {
issueId: facts.issueId,
expectedExecutionRunId: facts.runId,
+35 -3
View File
@@ -3,6 +3,7 @@ import { prepareManagedAiRuntime, assertManagedAiProjectAuth, stripAiAuthBinding
import { ADAPTER_AUTH_MISSING_CHECK_CODE, AI_CONNECTION_CAPABILITIES, aiConnectionBindingSchema, type AiConnectionBinding } from "@paperclipai/shared";
import { toolConnections } from "@paperclipai/db";
import { aiConnectionService } from "../services/ai-connections.js";
import { defaultAiConnectionForHire } from "../services/agent-ai-connection-default.js";
import { assertAiConnectionCreateAccess, canInstallSharedAiConnectionForNewAgent, responsibleUserForAiRequest, validateAiApiKey } from "./ai-connections.js";
import { isAiConnectionCompatible } from "@paperclipai/shared";
import { applyConnectorSkills, resolveConnectorAssignments, annotateConnectorSkills, isConnectorSkill } from "../services/connector-runtime.js";
@@ -2412,6 +2413,22 @@ export function agentRoutes(
return normalizedRuntimeConfig;
}
async function normalizeCreatedAgentRuntimeConfig(
req: Request,
companyId: string,
adapterType: string,
adapterConfig: Record<string, unknown>,
runtimeConfig: unknown,
) {
const normalized = normalizeNewAgentRuntimeConfig(runtimeConfig);
if (req.actor.type !== "agent" || normalized.aiConnection) return normalized;
const manager = req.actor.agentId ? await svc.getById(req.actor.agentId) : null;
if (!manager || manager.companyId !== companyId) throw forbidden("Hiring agent is unavailable");
const binding = defaultAiConnectionForHire(adapterType, adapterConfig, manager.runtimeConfig?.aiConnection);
if (binding) normalized.aiConnection = binding;
return normalized;
}
async function normalizeMediatedAdapterConfigForPersistence(input: {
companyId: string;
adapterType: string | null | undefined;
@@ -2528,14 +2545,18 @@ export function agentRoutes(
companyId: string,
adapterType: string | null | undefined,
adapterConfig: Record<string, unknown>,
runtimeConfig: unknown,
): Promise<{ adapterConfig: Record<string, unknown>; inheritedFixedClaudeOAuthBinding: boolean }> {
const noInheritance = { adapterConfig, inheritedFixedClaudeOAuthBinding: false };
if (asRecord(runtimeConfig)?.aiConnection) return noInheritance;
if (req.actor.type !== "agent" || !req.actor.agentId) return noInheritance;
const credentialKeys = adapterType ? INHERITABLE_AGENT_CREDENTIAL_ENV_KEYS[adapterType] : undefined;
if (!credentialKeys) return noInheritance;
const parent = await svc.getById(req.actor.agentId);
if (!parent || parent.companyId !== companyId || parent.adapterType !== adapterType) return noInheritance;
// Managed parents must not pass stale legacy credential references to hires.
if (aiConnectionBindingSchema.safeParse(parent.runtimeConfig.aiConnection).success) return noInheritance;
const parentEnv = asRecord(asRecord(parent.adapterConfig)?.env);
if (!parentEnv) return noInheritance;
@@ -3297,7 +3318,17 @@ export function agentRoutes(
async function validateManagedAgentBinding(req: Request, companyId: string, agentId: string, adapterType: string, config: Record<string, unknown>, binding: AiConnectionBinding, environmentId: string | null | undefined, test: boolean, newAgent = false) {
const userId = responsibleUserForAiRequest(req);
const allowUninstalledShared = newAgent && await canInstallSharedAiConnectionForNewAgent(db, req, companyId, binding);
const selection = await aiConnectionService(db).select({ companyId, agentId, userId, adapterType, model: config.model, runnerProvider: config.provider, acpxAgent: config.acpxAgent, binding, allowUninstalledPersonal: newAgent, allowUninstalledShared, allowLegacyValidation: test });
const selection = await aiConnectionService(db).select({ companyId, agentId, userId, adapterType, model: config.model, runnerProvider: config.provider, acpxAgent: config.acpxAgent, binding, allowUninstalledPersonal: newAgent, allowUninstalledShared, allowLegacyValidation: test }).catch((error: unknown) => {
// Hiring is allowed before the responsible user has connected this
// provider. Execution still resolves credentials and creates the normal
// task connection request; compatibility and access denials stay errors.
if (newAgent && !test && binding.mode === "responsible_user" && error instanceof HttpError
&& ["ai_connection_default_missing", "ai_connection_missing", "ai_connection_unavailable", "ai_connection_responsible_user_missing"].includes(String(asRecord(error.details)?.code))) {
return null;
}
throw error;
});
if (!selection) return undefined;
if (test) {
if (environmentId) await assertAdapterTestEnvironmentForCompany(companyId, environmentId);
const target = await resolveAdapterTestExecutionContext({ companyId, adapterType, environmentId: environmentId ?? null });
@@ -4399,6 +4430,7 @@ export function agentRoutes(
hireInput.adapterType,
rawHireAdapterConfig,
),
hireInput.runtimeConfig,
);
const requestedAdapterConfig = applyCodexLocalKeyIsolation(
companyId,
@@ -4431,7 +4463,7 @@ export function agentRoutes(
adapterType: hireInput.adapterType,
adapterConfig: desiredSkillAssignment.adapterConfig,
});
const normalizedRuntimeConfig = normalizeNewAgentRuntimeConfig(hireInput.runtimeConfig);
const normalizedRuntimeConfig = await normalizeCreatedAgentRuntimeConfig(req, companyId, hireInput.adapterType, normalizedAdapterConfig, hireInput.runtimeConfig);
const normalizedHireInput = {
...hireInput,
adapterConfig: normalizedAdapterConfig,
@@ -4726,7 +4758,7 @@ export function agentRoutes(
adapterType: createInput.adapterType,
adapterConfig: desiredSkillAssignment.adapterConfig,
});
const normalizedRuntimeConfig = normalizeNewAgentRuntimeConfig(createInput.runtimeConfig);
const normalizedRuntimeConfig = await normalizeCreatedAgentRuntimeConfig(req, companyId, createInput.adapterType, normalizedAdapterConfig, createInput.runtimeConfig);
await assertAgentEnvironmentSelection(companyId, createInput.adapterType, createInput.defaultEnvironmentId);
await assertAgentDefaultEnvironmentSelection(companyId, createInput.defaultEnvironmentId, {
allowedDrivers: allowedEnvironmentDriversForAgent(createInput.adapterType),
@@ -0,0 +1,42 @@
import {
AI_PROVIDERS,
aiConnectionBindingSchema,
isAiConnectionCompatible,
type AiConnectionBinding,
type AiProvider,
} from "@paperclipai/shared";
// Only keys read by the child's provider express a child auth override.
// A config copied from another provider can retain unrelated keys.
const PROVIDER_AUTH_ENV_KEYS: Record<AiProvider, readonly string[]> = {
anthropic: ["ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "CLAUDE_CODE_OAUTH_TOKEN", "CLAUDE_CONFIG_DIR", "ANTHROPIC_BASE_URL", "CLAUDE_CODE_USE_BEDROCK", "CLAUDE_CODE_USE_VERTEX", "CLAUDE_CODE_USE_FOUNDRY"],
openai: ["OPENAI_API_KEY", "CODEX_API_KEY", "CODEX_HOME", "OPENAI_BASE_URL"],
openrouter: ["OPENROUTER_API_KEY", "OPENCODE_AUTH_JSON", "OPENCODE_CONFIG_CONTENT", "OPENCODE_CONFIG", "OPENCODE_CONFIG_DIR", "PAPERCLIP_OPENCODE_PROVIDERS"],
xai: ["XAI_API_KEY", "GROK_API_KEY", "GROK_HOME", "XAI_BASE_URL"],
};
/** A hire inherits a connection choice, never its manager's credentials or identity. */
export function defaultAiConnectionForHire(
adapterType: string,
config: Record<string, unknown>,
managerBinding: unknown,
): AiConnectionBinding | undefined {
const compatible = (binding: AiConnectionBinding) =>
isAiConnectionCompatible(binding, adapterType, config.model, config.provider, config.acpxAgent);
const inherited = aiConnectionBindingSchema.safeParse(managerBinding);
// Unmanaged parents keep their existing login and credential-reference paths.
if (!inherited.success) return undefined;
const env = config.env && typeof config.env === "object" ? config.env as Record<string, unknown> : {};
const withChildAuthPrecedence = (binding: AiConnectionBinding) =>
PROVIDER_AUTH_ENV_KEYS[binding.provider].some((key) => env[key] !== undefined) ? undefined : binding;
if (inherited.data.mode !== "delegated" && compatible(inherited.data)) {
return withChildAuthPrecedence(inherited.data);
}
// The selected provider's personal default supplies the actual sign-in method
// at run time. A provider without an account can be connected on the first task.
for (const provider of AI_PROVIDERS) {
const binding = { provider, method: "api_key", mode: "responsible_user" } as const;
if (compatible(binding)) return withChildAuthPrecedence(binding);
}
return undefined;
}
@@ -2,6 +2,12 @@ import { describe, expect, it } from "vitest";
import { executionFailureRetryCount } from "./execution-recovery-attempt.js";
describe("failure attempts across resource waits", () => {
it("preserves prior failures through repeated AI subscription waits", () => {
expect(executionFailureRetryCount({ scheduledRetryReason: "ai_connection_busy", scheduledRetryAttempt: 12,
contextSnapshot: { failureRetriesBeforeAiConnectionWait: 1 } })).toBe(1);
expect(executionFailureRetryCount({ scheduledRetryReason: "transient_failure", scheduledRetryAttempt: 2,
contextSnapshot: { failureRetriesBeforeAiConnectionWait: 0 } })).toBe(2);
});
it("preserves prior failures through repeated workspace waits", () => {
expect(executionFailureRetryCount({ scheduledRetryReason: "workspace_busy", scheduledRetryAttempt: 12,
contextSnapshot: { failureRetriesBeforeWorkspaceWait: 1 } })).toBe(1);
@@ -5,6 +5,10 @@ export function executionFailureRetryCount(run: {
contextSnapshot?: Record<string, unknown> | null;
}): number {
if (run.scheduledRetryReason === "max_turns_continuation") return 0;
if (run.scheduledRetryReason === "ai_connection_busy") {
const count = run.contextSnapshot?.failureRetriesBeforeAiConnectionWait;
if (typeof count === "number" && Number.isInteger(count) && count >= 0) return count;
}
if (run.scheduledRetryReason === "workspace_busy") {
// Only a server-created workspace retry can consume this field. Its
// scheduler overwrites caller context with the predecessor's durable count.
+63
View File
@@ -842,6 +842,10 @@ const MAX_TURN_CONTINUATION_LIVE_RUN_STATUSES = [
export { WORKSPACE_BUSY_RETRY_REASON };
export const WORKSPACE_BUSY_RETRY_WAKE_REASON = "workspace_busy_retry";
export const WORKSPACE_BUSY_ERROR_CODE = "workspace_busy";
const AI_CONNECTION_BUSY_RETRY_REASON = "ai_connection_busy";
function isAiConnectionBusy(error: unknown): error is HttpError {
return error instanceof HttpError && parseObject(error.details).code === AI_CONNECTION_BUSY_RETRY_REASON;
}
export const WORKSPACE_BUSY_RETRY_BASE_DELAY_MS = 60 * 1000;
export const WORKSPACE_BUSY_RETRY_JITTER_MS = 60 * 1000;
// A running run stops counting as a shared-workspace holder once it has been
@@ -15018,6 +15022,7 @@ export function heartbeatService(
);
const nextAttempt =
(retryReason === WORKSPACE_BUSY_RETRY_REASON ||
retryReason === AI_CONNECTION_BUSY_RETRY_REASON ||
retryReason === MAX_TURN_CONTINUATION_RETRY_REASON
? (run.scheduledRetryAttempt ?? 0)
: executionFailureRetryCount(run)) + 1;
@@ -15144,6 +15149,7 @@ export function heartbeatService(
const requiresIssueGate =
hasConversationContinuationPolicy(run.resultJson) ||
retryReason === AI_CONNECTION_BUSY_RETRY_REASON ||
retryReason === MAX_TURN_CONTINUATION_RETRY_REASON ||
retryReason === INTERACTION_CONTINUATION_INFRA_RETRY_REASON;
if (requiresIssueGate) {
@@ -15212,6 +15218,9 @@ export function heartbeatService(
executionFailureRetryCount(run),
}
: {}),
...(retryReason === AI_CONNECTION_BUSY_RETRY_REASON
? { failureRetriesBeforeAiConnectionWait: executionFailureRetryCount(run) }
: {}),
...(shouldQuarantineWorkspaceForRetry
? {
workspaceValidationRecovery: {
@@ -15534,6 +15543,20 @@ export function heartbeatService(
}
}
if (retryReason === AI_CONNECTION_BUSY_RETRY_REASON && issueId) {
// The issue row is locked above. Recheck after the preflight gate so
// cancellation or recovery cannot leave a successor without its lock.
const [lockedIssue] = await tx.select({ executionRunId: issues.executionRunId })
.from(issues).where(and(eq(issues.id, issueId), eq(issues.companyId, run.companyId)));
if (lockedIssue?.executionRunId !== run.id) {
return {
outcome: "not_scheduled", issueId, errorCode: "issue_execution_lock_changed",
reason: "Subscription retry suppressed because the task execution lock changed",
details: { issueId, expectedExecutionRunId: run.id, currentExecutionRunId: lockedIssue?.executionRunId ?? null },
};
}
}
const wakeupRequest = await tx
.insert(agentWakeupRequests)
.values({
@@ -15916,6 +15939,41 @@ export function heartbeatService(
.then((rows) => rows[0] ?? null);
}
// Subscription homes are serialized so refresh tokens cannot be overwritten
// by another run. Contention is a resource wait, not broken authentication.
// The database lease is released on completion/disconnect; retrying remains
// safe across processes and each attempt revalidates the selected account.
async function finalizeAiConnectionBusyDeferral(run: typeof heartbeatRuns.$inferSelect, error: HttpError) {
const now = new Date();
const cancelled = await setRunStatusIfRunning(run.id, "cancelled", {
error: error.message, errorCode: AI_CONNECTION_BUSY_RETRY_REASON, finishedAt: now,
resultJson: { executionRecovery: { kind: "ai_connection_wait", providerWorkStarted: false } },
});
if (!cancelled.updated) return;
await setWakeupStatus(run.wakeupRequestId, "cancelled", { finishedAt: now, error: error.message }).catch(() => undefined);
const cancelledRun = cancelled.run ?? await getRun(run.id);
const agent = await getAgent(run.agentId);
let scheduled = false;
try {
if (cancelledRun && agent) {
const retry = await scheduleBoundedRetryForRun(cancelledRun, agent, {
now, retryReason: AI_CONNECTION_BUSY_RETRY_REASON, wakeReason: "ai_connection_busy_retry",
maxAttempts: (cancelledRun.scheduledRetryAttempt ?? 0) + 1,
delayMs: computeWorkspaceBusyRetryDelayMs(),
});
scheduled = retry.outcome === "scheduled";
await appendRunEvent(cancelledRun, {
eventType: "lifecycle", stream: "system", level: "info",
message: scheduled ? "Waiting for the shared AI subscription. This task will retry automatically." : "The AI subscription is busy; this task can no longer retry automatically.",
payload: { retryScheduled: scheduled },
});
}
} finally {
if (cancelledRun && !scheduled) await releaseIssueExecutionAndPromote(cancelledRun);
await finalizeAgentStatus(run.agentId, "cancelled", null, { wasFirstHeartbeat: timerClaimWasFirstHeartbeat(run) });
}
}
// Terminal handling for a WorkspaceBusyDeferral thrown by the pre-dispatch
// gate: cancel the run (contention is not a failure), schedule a
// workspace_busy retry, and leave the agent idle. The issue execution lock
@@ -20711,6 +20769,7 @@ export function heartbeatService(
try {
managedAiRuntime = await prepareManagedAiRuntime(db, { companyId: agent.companyId, agentId: agent.id, responsibleUserId, adapterType: agent.adapterType, binding: aiBinding, config: resolvedConfig });
} catch (error) {
if (isAiConnectionBusy(error)) throw error;
if (responsibleUserId && issueId && aiBinding.mode === "responsible_user") {
await connectionIntentService(db).request({ sub: agent.id, company_id: agent.companyId, run_id: run.id, responsible_user_id: responsibleUserId }, aiBinding.provider, { purpose: "ai" }).catch(() => {
logger.warn({ runId: run.id, agentId: agent.id }, "Could not attach AI connection request; runtime configuration action remains available");
@@ -25018,6 +25077,10 @@ export function heartbeatService(
? outerErr.reason
: "adopted_runner_authentication_timeout",
}).catch(() => undefined);
} else if (isAiConnectionBusy(outerErr)) {
await finalizeAiConnectionBusyDeferral(run, outerErr).catch((error) => {
logger.error({ err: error, runId }, "failed to schedule a retry for the busy AI subscription");
});
} else if (isWorkspaceBusyDeferral(outerErr)) {
// Expected contention on a shared project workspace, not a
// failure: park the run as a bounded scheduled retry and leave the
@@ -38,3 +38,11 @@ it("retains the hold until the provider actually acknowledges cancellation", ()
})).toBe(false);
}
});
it("retries a busy AI subscription only when no provider work started", () => {
const waiting = { runtimeMode: "legacy", status: "cancelled", errorCode: "ai_connection_busy", scheduledRetryAttempt: 10,
resultJson: { executionRecovery: { kind: "ai_connection_wait", providerWorkStarted: false } } };
expect(legacyExecutionNeedsReconciliation(waiting)).toBe(false);
expect(legacyExecutionNeedsReconciliation({ ...waiting, resultJson: {} })).toBe(true);
expect(legacyExecutionNeedsReconciliation({ ...waiting, resultJson: { executionRecovery: { kind: "ai_connection_wait", providerWorkStarted: true } } })).toBe(true);
});
@@ -31,8 +31,10 @@ export function legacyExecutionNeedsReconciliation(
&& evidence.providerStopped === true && evidence.sessionPreserved === true
&& evidence.actionOutcomes === "settled"
&& (run.resultJson?.executionCancellation as Record<string, unknown> | undefined)?.state === "acknowledged") return false;
// Waiting for a live workspace holder precedes provider execution. It is a
// resource wait, not a failed provider attempt or permission to replay work.
// Waiting for a subscription or workspace precedes provider execution. It is
// a resource wait, not a failed provider attempt or permission to replay work.
if (run.status === "cancelled" && run.errorCode === "ai_connection_busy" &&
evidence?.kind === "ai_connection_wait" && evidence.providerWorkStarted === false) return false;
if (run.status === "cancelled" && run.errorCode === "workspace_busy" &&
evidence?.kind === "workspace_wait" && evidence.providerWorkStarted === false) return false;
if (executionFailureRetryCount(run) >= 2) return true;
@@ -320,7 +320,13 @@ export class PaperclipRunnerToolAuthority {
conversation: Boolean(context.issue.conversationAgentId) });
}
case "search_api": return searchRunnerApi(call.arguments);
case "call_api": return this.#callApi(call.callId, call.arguments);
case "call_api": {
// PRP reserves operationId/callId for semantic result identity. The
// HTTP operation is metadata, including in previously saved receipts;
// exposing it as operationId makes the runner reject a valid response.
const { operationId, ...response } = record(await this.#callApi(call.callId, call.arguments));
return { ...response, apiOperationId: operationId };
}
case "get_task_context": return {
company: { id: this.binding.companyId },
actor: redactedActor(context.actor),
@@ -34,7 +34,7 @@ describe("runner API against real HTTP routes", () => {
import { createInterface } from 'node:readline';
const send = value => process.stdout.write(JSON.stringify(value)+'\\n');
let step = 0;
const calls = [{tool:'search_api',arguments:{query:'list projects'}},{tool:'call_api',arguments:{operationId:'GET /api/companies/{companyId}/projects'}}];
const calls = [{tool:'search_api',arguments:{query:'list projects'}},{tool:'call_api',arguments:{operationId:'GET /api/companies/{companyId}/projects'}},{tool:'call_api',arguments:{operationId:'GET /api/projects/{id}',pathParams:{id:'${fixture.foreignProjectId}'}}}];
const next = () => { const c=calls[step++]; if(c) send({id:'call-'+step,method:'item/tool/call',params:{threadId:'api-thread',turnId:'api-turn',itemId:'api-item-'+step,callId:'api-call-'+step,...c}}); else send({method:'turn/completed',params:{turn:{id:'api-turn',status:'completed'}}}); };
for await (const line of createInterface({input:process.stdin})) {
const m=JSON.parse(line);
@@ -57,15 +57,24 @@ else if(m.id!==undefined) send({id:m.id,result:{}});
const params = request.params as any;
const result = await fixture.authority.execute({ tool: params.tool, arguments: params.arguments, callId: params.callId });
results.push(result);
return { success: true, contentItems: [{ type: "inputText", text: JSON.stringify({ ok: true, result }) }] };
// Match production's dynamicToolResponse: an extra test-only envelope
// hides collisions between API response fields and PRP tool identity.
return { success: true, contentItems: [{ type: "inputText", text: JSON.stringify(result) }] };
});
try {
await bundle.transport.request("initialize", {});
await bundle.transport.request("thread/start", { cwd: fixture.workspace, dynamicTools: await fixture.authority.definitions() });
await bundle.transport.request("turn/start", { input: [{ type: "text", text: "Find the project" }] });
for await (const notification of bundle.transport.notifications()) if (notification.method === "turn/completed") break;
expect(results).toHaveLength(2);
expect(results[1]).toMatchObject({ status: 200, data: [{ id: fixture.projectId, name: "Aurora" }] });
let timeout: ReturnType<typeof setTimeout> | undefined;
try {
await Promise.race([
(async () => { for await (const notification of bundle.transport.notifications()) if (notification.method === "turn/completed") return; })(),
new Promise<never>((_, reject) => { timeout = setTimeout(() => reject(new Error("Native provider did not continue after the API tool result")), 10_000); }),
]);
} finally { clearTimeout(timeout); }
expect(results).toHaveLength(3);
expect(results[1]).toMatchObject({ status: 200, apiOperationId: "GET /api/companies/{companyId}/projects", data: [{ id: fixture.projectId, name: "Aurora" }] });
expect(results[2]).toMatchObject({ ok: false, status: 404 });
expect(bundle.evidence().diagnostics).toContain("runnerd authenticated to the durable PRP control plane");
} finally { await bundle.transport.close(); }
}, 30_000);