mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
Third PR: depends on #15728 and #15727. The user has reviewed and authorized merging this experimental feature stack. ## Thinking Path > - Paperclip manages agent work through tasks and governed connections. > - Chat connections already bind external conversations to those tasks. > - Speko handles voice and telephony, while the selected Paperclip agent does the work. > - A call needs durable session authority, ordered follow-ups, and safe result delivery. > - This PR adds repeatable qualification for the Speko voice task integration. > - This PR depends on #15728, which depends on #15727. Speko stays experimental. ## Linked Issues or Issue Description **Problem or motivation** Phone and browser voice conversations cannot use the chat connection task lifecycle. A separate voice runtime is needed to preserve task authority, retries, and reply history. **Proposed solution** Add company-scoped voice sessions, signed callbacks, durable replies, inbound intake, and an agent-facing outbound-call tool. Reuse existing tasks, queues, secrets, and publications. **Roadmap alignment** This extends the experimental chat connector entry points under Agent Chat and Connected Apps. The project owner requested this feature. Refs #11418 for the original provider catalog proposal. This implementation uses the newer chat connection infrastructure rather than an MCP catalog-only integration. Refs #15721 for remote OpenCode startup. ## What Changed - Add a real Paperclip server/database E2E fixture that stubs only external Speko transport and the execution provider. - Add authenticated multi-user and cross-company qualification journeys. - Add Storybook interaction, accessibility, responsive, and visual-suite integration. - Add an E01–E16 runbook, component-to-story inventory, and current redacted qualification report. - Keep historical private call traces out of the PR. Provider proof tools are opt-in. ## Verification - All 21 standalone proof tests and browser client/server syntax checks pass. They cover signatures, redaction, cleanup retry, final audio saves, bounded report transport and late server write ordering. No live calls were placed. - All 27 focused UI/controller regressions, UI typecheck, token gates, and 28 map/catalog structural tests pass. The shared connection cache test covers pause/resume invalidation during the 30-second fresh period. - Storybook builds 157 Speko production-component stories. The complete prior 156-story set passed all 624 checks on UI source249cbc616a. After the final cache repair, all 152 affected page/launcher checks passed across both themes, 390/1200 widths and reduced motion. Real microphone access and external provider requests are blocked. - Hosted Storybook origin isolation and representative narrow-screen stories pass. Provider requests remain blocked. - Source annotations follow the current generated-catalog contract. No generated catalog or lockfile is included. Three inherited GitHub-heading tests passed after master reconciliation. - Commit905bc5da5dhas 55 passing checks, including required verify/E2E gates, and a fresh Greptile 5/5 review with zero unresolved findings. Local fixture success does not establish provider or spoken-playback acceptance. - The separate integrated browser run passed two journeys and failed browser answer retrieval. Real browser/telephone playback and timing acceptance remain outstanding. See the dated qualification report and E01–E16 runbook. Speko stays experimental. ## Risks - Voice sessions add migrations and provider-dependent lifecycle behavior. Migrations preserve existing chat and AI transports. - Callback origins must be reachable HTTPS. Provider keys and signing secrets remain vaulted. - Unverified callers receive task-scoped low-trust authority. Caller ID does not grant private task access. - Outbound telephony incurs provider charges. Provider acceptance is not proof that the caller heard an answer. - Keep the feature behind the existing experimental chat setting until the real browser, inbound, outbound, and delayed-result acceptance matrix passes. ## Model Used OpenAI Codex, GPT-6, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
365 lines
34 KiB
TypeScript
365 lines
34 KiB
TypeScript
import { test, expect } from "@playwright/test";
|
||
import { readFile, readdir } from "node:fs/promises";
|
||
import { resolve } from "node:path";
|
||
const root = resolve(import.meta.dirname, "../..");
|
||
const port = Number(process.env.SPEKO_NATIVE_TEST_PORT ?? 3480);
|
||
const origin = `http://127.0.0.1:${port}`;
|
||
|
||
test("E01/E03–E11/E13/E15/E16: setup, delayed work, duplicate input, navigation, repeat, reload and hangup use real task ledgers", async ({ page }, info) => {
|
||
const { home, supervisorPid } = JSON.parse(await readFile(resolve(import.meta.dirname, ".instance.json"), "utf8"));
|
||
const evidence = async () => JSON.parse(await readFile(resolve(home, "evidence.json"), "utf8"));
|
||
const post = async (path: string, data: unknown) => {
|
||
const response = await page.request.post(`${origin}/api${path}`, { headers: { Origin: origin }, data });
|
||
expect(response.ok(), `${path}: ${response.status()} ${await response.text()}`).toBe(true);
|
||
return response.json();
|
||
};
|
||
const company = await post("/companies", { name: "Speko integrated", issuePrefix: "SPE" });
|
||
const other = await post("/companies", { name: "Separate company", issuePrefix: "OTH" });
|
||
const policy = await page.request.patch(`${origin}/api/companies/${company.id}`, { headers: { Origin: origin }, data: { requireBoardApprovalForNewAgents: false } });
|
||
expect(policy.ok()).toBe(true);
|
||
const agent = await post(`/companies/${company.id}/agents`, { name: "Existing Paperclip Agent", role: "general", adapterType: "process", adapterConfig: { command: "fixture" }, runtimeConfig: { heartbeat: { enabled: false, wakeOnDemand: true } } });
|
||
// Discover the built SDK chunk by its export, rather than mock any Paperclip API.
|
||
const assets = resolve(root, "ui/dist/assets");
|
||
const candidates = await Promise.all((await readdir(assets)).filter(name => name.endsWith(".js")).map(async name => ({ name, source: await readFile(resolve(assets, name), "utf8") })));
|
||
const sdk = candidates.filter(file => /export\{[^}]* as VoiceConversation\}/.test(file.source));
|
||
expect(sdk).toHaveLength(1);
|
||
const sdkSource = (await readFile(resolve(import.meta.dirname, "stub-browser-sdk.mjs"), "utf8")).replaceAll("__STUB_PORT__", String(port + 1));
|
||
await page.route(`**/assets/${sdk[0].name}`, route => route.fulfill({ contentType: "application/javascript", body: sdkSource }));
|
||
const external: string[] = [];
|
||
await page.route(/^https?:\/\//, route => {
|
||
const url = new URL(route.request().url());
|
||
return ["127.0.0.1", "localhost"].includes(url.hostname) ? route.fallback() : route.abort();
|
||
});
|
||
page.on("request", request => { const url = new URL(request.url()); if (["http:", "https:", "wss:"].includes(url.protocol) && !["127.0.0.1", "localhost"].includes(url.hostname)) external.push(url.hostname); });
|
||
await page.addInitScript(() => { navigator.mediaDevices.getUserMedia = async () => { throw new Error("Integrated fixture must not request a real microphone"); }; });
|
||
await page.goto(`/${company.issuePrefix}/apps/chat/connect?provider=speko`);
|
||
await page.getByRole("button", { name: "Choose an active agent", exact: true }).click();
|
||
await page.getByRole("button", { name: `Select ${agent.name}`, exact: true }).click();
|
||
await page.getByRole("button", { name: "Continue", exact: true }).click();
|
||
await expect(page.getByLabel("Speko API key", { exact: true })).toHaveAttribute("type", "password");
|
||
await page.getByRole("textbox", { name: "Speko agent ID", exact: true }).fill("agent_fixture");
|
||
await page.getByLabel("Speko API key", { exact: true }).fill("fixture-speko-key");
|
||
const setupResponse = page.waitForResponse(r => r.request().method() === "POST" && /\/chat-endpoints\/[^/]+\/setup$/.test(new URL(r.url()).pathname));
|
||
await page.getByRole("button", { name: "Connect Speko", exact: true }).click();
|
||
const endpoint = await (await setupResponse).json(); expect(endpoint.id).toBeTruthy();
|
||
expect(JSON.stringify(endpoint)).not.toContain("fixture-speko-key");
|
||
await page.reload();
|
||
await expect(page.getByRole("button", { name: "Start voice", exact: true })).toBeVisible();
|
||
const nextSession = () => page.waitForResponse(r => r.request().method() === "POST" && /\/voice-sessions$/.test(new URL(r.url()).pathname) && r.ok()).then(r => r.json());
|
||
const pending = nextSession(); await page.getByRole("button", { name: "Start voice", exact: true }).click(); const started = await pending;
|
||
expect(started.session.assignedAgentId).toBe(agent.id); expect(started.media.transportToken).toMatch(/^fixture:/);
|
||
expect(JSON.stringify(started)).not.toContain("fixture-speko-key");
|
||
await expect(page.getByRole("button", { name: "Mute", exact: true })).toBeEnabled();
|
||
const speak = (text: string, id: string) => page.evaluate(({ text, id }) => (window as any).__spekoTransport.speak(text, id), { text, id });
|
||
const accepted = await speak("Run the acceptance benchmark and report its verification number.", "request-one");
|
||
expect(accepted.status).toBe("accepted");
|
||
expect(await speak("Run the acceptance benchmark and report its verification number.", "request-one")).toEqual(accepted);
|
||
await speak("Also count this follow-up on the same task.", "request-two");
|
||
// A client-side route transition must preserve the host and the original media.
|
||
await page.getByRole("link", { name: "View conversation task", exact: true }).click();
|
||
await expect(page).toHaveURL(new RegExp(`/issues/${started.session.issueId}$`));
|
||
await expect(page.getByRole("button", { name: "Mute", exact: true })).toBeEnabled();
|
||
expect(await page.evaluate(() => (window as any).__spekoTransport.creates)).toBe(1);
|
||
await expect.poll(async () => (await evidence()).tools.filter((tool: any) => tool.sessionId === started.session.id && tool.tool === "submit_request").length).toBe(2);
|
||
const transcript = page.getByRole("list", { name: "Conversation transcript", exact: true });
|
||
await expect(transcript).toContainText("Verification number", { timeout: 100_000 });
|
||
await expect.poll(async () => {
|
||
const rows = (await readFile(resolve(home, "execution.jsonl"), "utf8")).trim().split("\n").map(line => JSON.parse(line));
|
||
return rows.some(row => row.issueId === started.session.issueId && row.event === "completed" && row.elapsedMs >= 60_000 && row.requests === 2);
|
||
}).toBe(true);
|
||
await expect.poll(async () => (await evidence()).comments.filter((row: any) => row.issueId === started.session.issueId && !row.authorAgentId).length).toBe(2);
|
||
await expect(transcript).toContainText("Your follow-up was included", { timeout: 30_000 });
|
||
const beforeRepeat = await transcript.getByRole("listitem").count();
|
||
await page.getByRole("button", { name: "Repeat answer", exact: true }).click();
|
||
await expect(transcript.getByRole("listitem")).toHaveCount(beforeRepeat + 1);
|
||
await page.getByRole("button", { name: "Mute", exact: true }).click();
|
||
expect(await page.evaluate(() => (window as any).__spekoTransport.muted)).toBe(true);
|
||
await page.getByRole("button", { name: "End call", exact: true }).click();
|
||
await expect(page.getByRole("status").filter({ hasText: /^Call ended$/ })).toBeVisible();
|
||
expect(await page.evaluate(() => (window as any).__spekoTransport.open)).toBe(false);
|
||
// Prove connection guidance reaches the ordinary execution provider without
|
||
// installing a template or replacing the selected agent's instructions.
|
||
await expect.poll(async () => (await evidence()).runs.find((row: any) => row.context.issueId === started.session.issueId)?.context.paperclipTaskCommunicationGuidance).toContain("Keep your existing identity, runtime, tools, and task assignment");
|
||
const deliveredGuidance = (await evidence()).runs.find((row: any) => row.context.issueId === started.session.issueId).context.paperclipTaskCommunicationGuidance;
|
||
expect(deliveredGuidance).toContain("one focused clarification");
|
||
expect(deliveredGuidance).toContain("durable Paperclip continuation");
|
||
expect(deliveredGuidance).toContain("does not cancel work");
|
||
expect(deliveredGuidance).toContain("Spoken consent does not replace governed approval");
|
||
const foreign = await page.request.get(`${origin}/api/companies/${other.id}/voice-sessions/${started.session.id}`);
|
||
expect(foreign.status()).toBe(404);
|
||
await page.getByRole("button", { name: "Close voice panel", exact: true }).click();
|
||
await page.goto(`/${company.issuePrefix}/apps/chat/connect?provider=speko&resume=${endpoint.id}`);
|
||
const rejoining = nextSession(); await page.getByRole("button", { name: "Start voice", exact: true }).click(); const resumed = await rejoining;
|
||
expect(resumed.session.issueId).toBe(started.session.issueId); expect(resumed.session.id).not.toBe(started.session.id);
|
||
await expect(page.getByRole("button", { name: "Mute", exact: true })).toBeEnabled();
|
||
const journal = await page.evaluate(() => Object.entries(sessionStorage).filter(([key]) => key.startsWith("paperclip:voice-attempt:")));
|
||
expect(journal).toHaveLength(1);
|
||
expect(journal[0][1]).not.toContain(resumed.media.transportToken);
|
||
// Reload must reconcile the exact old call before it offers a new connection.
|
||
await page.reload();
|
||
await page.getByRole("button", { name: "Start voice", exact: true }).click();
|
||
await expect(page.getByRole("alert").filter({ hasText: "The previous call was closed" })).toBeVisible();
|
||
await expect.poll(async () => (await evidence()).sessions.find((row: any) => row.id === resumed.session.id)?.state).toBe("ended");
|
||
expect((await evidence()).sessions).toHaveLength(2);
|
||
const recoveredStart = nextSession();
|
||
await page.getByRole("button", { name: "Try again", exact: true }).click();
|
||
const recovered = await recoveredStart;
|
||
expect(recovered.session.issueId).toBe(started.session.issueId);
|
||
await expect(page.getByRole("button", { name: "Repeat answer", exact: true })).toBeEnabled();
|
||
await page.getByRole("button", { name: "Repeat answer", exact: true }).click();
|
||
await expect(transcript).toContainText("Your follow-up was included");
|
||
await page.getByRole("button", { name: "End call", exact: true }).click();
|
||
await expect.poll(async () => (await evidence()).sessions.filter((row: any) => row.issueId === started.session.issueId).every((row: any) => row.state === "ended")).toBe(true);
|
||
await page.getByRole("button", { name: "Close voice panel", exact: true }).click();
|
||
// Explicit New creates one new task; hanging up does not cancel accepted work.
|
||
await page.getByRole("combobox", { name: "Conversation", exact: true }).selectOption("new");
|
||
const newStart = nextSession(); await page.getByRole("button", { name: "Start voice", exact: true }).click(); const fresh = await newStart;
|
||
expect(fresh.session.issueId).not.toBe(started.session.issueId);
|
||
await expect(page.getByRole("button", { name: "Mute", exact: true })).toBeEnabled();
|
||
await speak("Do the benchmark; I will return after it finishes.", "after-hangup");
|
||
const executionRows = async () => (await readFile(resolve(home, "execution.jsonl"), "utf8")).trim().split("\n").map(line => JSON.parse(line));
|
||
await expect.poll(async () => (await executionRows()).some(row => row.issueId === fresh.session.issueId && row.event === "started")).toBe(true);
|
||
await page.getByRole("button", { name: "End call", exact: true }).click();
|
||
await expect(page.getByRole("status").filter({ hasText: /^Call ended$/ })).toBeVisible();
|
||
await expect.poll(async () => (await executionRows()).some(row => row.issueId === fresh.session.issueId && row.event === "completed" && row.elapsedMs >= 60_000), { timeout: 90_000 }).toBe(true);
|
||
await page.getByRole("button", { name: "Close voice panel", exact: true }).click();
|
||
await page.getByRole("combobox", { name: "Conversation", exact: true }).selectOption("continue");
|
||
const missedStart = nextSession(); await page.getByRole("button", { name: "Start voice", exact: true }).click(); const missed = await missedStart;
|
||
expect(missed.session.issueId).toBe(fresh.session.issueId);
|
||
await expect(transcript).toContainText("Verification number");
|
||
expect((await evidence()).comments.filter((row: any) => row.issueId === fresh.session.issueId && !row.authorAgentId)).toHaveLength(1);
|
||
await speak("Ask the structured acceptance question before continuing.", "request-question");
|
||
await expect.poll(() => page.evaluate(() => (window as any).__spekoTransport.question?.questions[0]?.id)).toBe("color");
|
||
await expect(transcript).toContainText("Which color should I use?");
|
||
const answer = () => page.evaluate(() => {
|
||
const transport = (window as any).__spekoTransport;
|
||
return transport.answer(transport.question.interactionId, [{ questionId: "color", optionIds: ["cobalt"] }], "answer-one");
|
||
});
|
||
const answered = await answer(); expect(answered.status).toBe("answered");
|
||
expect(await answer()).toEqual(answered);
|
||
await expect(transcript).toContainText("Your structured answer was received", { timeout: 30_000 });
|
||
await expect(transcript).toContainText("cobalt");
|
||
await expect.poll(async () => (await evidence()).questionDeliveries.filter((row: any) => row.interactionId === answered.interactionId).length).toBe(1);
|
||
const questionEvidence = await evidence();
|
||
expect(questionEvidence.interactions.find((row: any) => row.id === answered.interactionId)?.status).toBe("answered");
|
||
expect(questionEvidence.tools.filter((row: any) => row.tool === "answer_question")).toHaveLength(1);
|
||
const delivery = questionEvidence.questionDeliveries.find((row: any) => row.interactionId === answered.interactionId);
|
||
expect(["delivered", "fallback_queued"]).toContain(delivery.status);
|
||
expect(delivery.targetRunId).toBeTruthy();
|
||
await expect.poll(async () => (await evidence()).runs.find((row: any) => row.id === delivery.targetRunId)?.status).toBe("succeeded");
|
||
const beforeRestart = await evidence();
|
||
process.kill(supervisorPid, "SIGUSR2");
|
||
// Restart has the same startup budget as the initial fixture server.
|
||
await expect.poll(async () => (await evidence()).appPid, { timeout: 120_000 }).not.toBe(beforeRestart.appPid);
|
||
await expect.poll(async () => {
|
||
try { return (await page.request.get(`${origin}/api/health`)).ok(); } catch { return false; }
|
||
}).toBe(true);
|
||
// Same session and signed tool identity cross the real process boundary.
|
||
expect(await answer()).toEqual(answered);
|
||
const beforeRecoveredRepeat = await transcript.getByRole("listitem").count();
|
||
await page.getByRole("button", { name: "Repeat answer", exact: true }).click();
|
||
await expect(transcript.getByRole("listitem")).toHaveCount(beforeRecoveredRepeat + 1);
|
||
await expect(transcript).toContainText("Your structured answer was received");
|
||
expect(await page.evaluate(() => (window as any).__spekoTransport.creates)).toBe(3);
|
||
const afterRestart = await evidence();
|
||
expect(afterRestart.sessions).toHaveLength(beforeRestart.sessions.length);
|
||
expect(afterRestart.comments).toHaveLength(beforeRestart.comments.length);
|
||
expect(afterRestart.runs).toHaveLength(beforeRestart.runs.length);
|
||
expect(afterRestart.tools.filter((row: any) => row.tool === "answer_question")).toHaveLength(1);
|
||
expect(afterRestart.questionDeliveries).toHaveLength(1);
|
||
await page.getByRole("button", { name: "End call", exact: true }).click();
|
||
await expect.poll(async () => (await evidence()).sessions.every((row: any) => row.state === "ended")).toBe(true);
|
||
await page.getByRole("button", { name: "Close voice panel", exact: true }).click();
|
||
const activated = await post(`/chat-endpoints/${endpoint.id}/test`, {});
|
||
expect(activated.status).toBe("active");
|
||
await page.goto(`/${company.issuePrefix}/apps/chat/${endpoint.id}/settings`);
|
||
await expect(page.getByRole("textbox", { name: "Your callback number", exact: true })).toBeEnabled();
|
||
await expect(page.getByRole("button", { name: "Start voice", exact: true })).toHaveCount(0);
|
||
await expect(page.getByRole("combobox", { name: "Conversation", exact: true })).toHaveCount(0);
|
||
await expect(page.getByRole("link", {name: "Open Speko"})).toHaveAttribute("href", "https://platform.speko.ai/agents");
|
||
await expect(page.locator('header img[src="/brands/apps/speko.svg"]').first()).toBeVisible();
|
||
await page.getByRole("textbox", { name: "Your callback number", exact: true }).fill("+12015551234");
|
||
await page.getByRole("checkbox", { name: "Allow this agent to call me about my tasks", exact: true }).check();
|
||
await page.getByRole("button", { name: "Save phone setting", exact: true }).click();
|
||
await expect(page.getByRole("status").filter({ hasText: "Callback setting saved" })).toBeVisible();
|
||
await page.reload();
|
||
await expect(page.getByRole("textbox", { name: "Your callback number", exact: true })).toHaveValue("+12015551234");
|
||
await expect(page.getByRole("checkbox", { name: "Allow this agent to call me about my tasks", exact: true })).toBeChecked();
|
||
const updatedAgent = await page.request.patch(`${origin}/api/agents/${agent.id}`, { headers: { Origin: origin }, data: { adapterConfig: { command: "fixture-phone" } } });
|
||
expect(updatedAgent.ok()).toBe(true);
|
||
const callTask = await post(`/companies/${company.id}/issues`, { title: "Call my saved phone", status: "todo", assigneeAgentId: agent.id, responsibleUserId: "local-board" });
|
||
await expect.poll(async () => (await executionRows()).some(row => row.event === "phone_called" && row.issueId === callTask.id), { timeout: 30_000 }).toBe(true);
|
||
const phoneExecution = (await executionRows()).find(row => row.event === "phone_called" && row.issueId === callTask.id);
|
||
expect(phoneExecution.secondRejected).toBe(true);
|
||
expect(JSON.stringify(phoneExecution.result)).not.toContain("+12015551234");
|
||
expect(JSON.stringify(phoneExecution.result)).not.toContain("fixture-speko-key");
|
||
expect(JSON.stringify(phoneExecution.result)).not.toContain("transportToken");
|
||
await expect.poll(async () => (await evidence()).sessions.filter((row: any) => row.issueId === callTask.id && row.mode === "outbound_phone").length).toBe(1);
|
||
const phone = (await evidence()).sessions.find((row: any) => row.issueId === callTask.id && row.mode === "outbound_phone");
|
||
const providerState = JSON.parse(await readFile(resolve(home, "speko-fixture-state.json"), "utf8"));
|
||
expect(providerState.sessions.filter((entry: any) => entry[1].mode === "outbound_phone")).toHaveLength(1);
|
||
const inspectedPhone = await (await page.request.get(`${origin}/api/companies/${company.id}/voice-sessions/${phone.id}`)).json();
|
||
expect(inspectedPhone).toMatchObject({ id: phone.id, issueId: callTask.id, mode: "outbound_phone" });
|
||
expect(JSON.stringify(inspectedPhone)).not.toContain("transportToken");
|
||
const endedPhone = await post(`/companies/${company.id}/voice-sessions/${phone.id}/end`, {});
|
||
expect(endedPhone.state).toBe("ended");
|
||
const retainedTask = await (await page.request.get(`${origin}/api/issues/${callTask.id}`)).json();
|
||
expect(retainedTask.status).not.toBe("cancelled");
|
||
expect(retainedTask.assigneeAgentId).toBe(agent.id);
|
||
await page.getByRole("checkbox", { name: "Allow this agent to call me about my tasks", exact: true }).uncheck();
|
||
await page.getByRole("button", { name: "Save phone setting", exact: true }).click();
|
||
await expect(page.getByRole("status").filter({ hasText: "Callback setting saved" })).toBeVisible();
|
||
await expect.poll(async () => (await evidence()).sessions.every((row: any) => row.state === "ended")).toBe(true);
|
||
// E08: a real signed pre-call route creates NO task before explicit live approval.
|
||
await page.getByRole("combobox", {name: "Company phone number", exact: true}).selectOption("pn_fixture");
|
||
await page.getByRole("checkbox", {name: "Enable incoming calls on this number", exact: true}).check();
|
||
await page.getByRole("button", {name: "Save incoming call setting", exact: true}).click();
|
||
await expect(page.getByRole("status").filter({hasText: "Incoming call setting saved"})).toBeVisible();
|
||
await page.request.patch(`${origin}/api/agents/${agent.id}`, {headers: {Origin: origin}, data: {adapterConfig: {command: "fixture"}}});
|
||
const providerCommand = async (data: unknown) => {
|
||
const response = await page.request.post(`http://127.0.0.1:${port + 1}/command`, {headers: {Origin: origin}, data});
|
||
expect(response.ok(), await response.text()).toBe(true); return response.json();
|
||
};
|
||
const beforeIncoming = (await evidence()).sessions.length;
|
||
const incoming = await providerCommand({action: "inbound", agentId: "agent_fixture", callerId: "+12015551234"});
|
||
await expect(page.getByRole("heading", {name: "Incoming call awaiting approval", exact: true})).toBeVisible();
|
||
await expect.poll(async () => (await evidence()).incoming.filter((call: any) => call.endpointId === endpoint.id).length).toBe(1);
|
||
expect((await evidence()).sessions).toHaveLength(beforeIncoming);
|
||
const unapproved = await providerCommand({sessionId: incoming.sessionId, tool: "submit_request", args: {text: "Spoofed caller ID must not grant task access"}, toolCallId: "before-approval"});
|
||
expect(unapproved).toMatchObject({requestAccepted: false, authorization: "awaiting_approval", updates: []});
|
||
await page.getByRole("textbox", {name: "Code spoken on your call", exact: true}).fill("000000");
|
||
await page.getByRole("button", {name: "Approve live call", exact: true}).click();
|
||
await expect(page.getByRole("alert").filter({hasText: "does not match"})).toBeVisible();
|
||
const code = incoming.firstMessage.replace(/\D/g, ""); expect(code).toHaveLength(6);
|
||
await page.getByRole("textbox", {name: "Code spoken on your call", exact: true}).fill(code);
|
||
const inboundTask = await post(`/companies/${company.id}/issues`, {title: "Approved incoming phone conversation", assigneeAgentId: agent.id, status: "todo"});
|
||
// Refresh task inventory after this fixture task is created.
|
||
await page.reload();
|
||
await page.getByRole("textbox", {name: "Code spoken on your call", exact: true}).fill(code);
|
||
await page.getByRole("combobox", {name: "Conversation task", exact: true}).selectOption(inboundTask.id);
|
||
await page.getByRole("button", {name: "Approve live call", exact: true}).click();
|
||
await expect(page.getByRole("heading", {name: "Call approved", exact: true})).toBeFocused();
|
||
await expect.poll(async () => (await evidence()).sessions.find((call: any) => call.providerSessionId === incoming.sessionId)?.issueId).toBe(inboundTask.id);
|
||
const phoneWork = {sessionId: incoming.sessionId, tool: "submit_request", args: {text: "Perform this approved intake task"}, toolCallId: "approved-work"};
|
||
const incomingAccepted = await providerCommand(phoneWork); expect(incomingAccepted).toMatchObject({status: "accepted", authorization: "approved"});
|
||
expect(await providerCommand(phoneWork)).toEqual(incomingAccepted);
|
||
const boundIncoming = (await evidence()).sessions.find((call: any) => call.providerSessionId === incoming.sessionId);
|
||
await expect.poll(async () => (await evidence()).tools.filter((call: any) => call.sessionId === boundIncoming.id && call.tool === "submit_request").length).toBe(1);
|
||
// A saved task answer must reach the live phone transport before hangup.
|
||
// Provider acceptance is asserted separately from actual spoken playback.
|
||
await expect.poll(async () => (await evidence()).comments.some((row: any) => row.issueId === inboundTask.id && row.authorAgentId === agent.id && row.body.includes("Verification number")), {timeout: 100_000}).toBe(true);
|
||
await expect.poll(async () => (await providerCommand({action: "messages", sessionId: incoming.sessionId})).some((message: any) => message.text.includes("Verification number")), {timeout: 30_000}).toBe(true);
|
||
await expect.poll(async () => (await evidence()).pushActions.some((action: any) => action.payload.sessionId === boundIncoming.id && action.status === "completed" && action.result?.messageId && action.result.playback === "unknown")).toBe(true);
|
||
await post(`/companies/${company.id}/voice-sessions/${boundIncoming.id}/end`, {});
|
||
// E15: report reconciliation persists one report, excludes recordings and keeps task assignment.
|
||
await expect.poll(async () => (await evidence()).reports.find((report: any) => report.sessionId === boundIncoming.id)?.status, {timeout: 45000}).toBe("available");
|
||
const reportResponse = await page.request.get(`${origin}/api/companies/${company.id}/voice-sessions/${boundIncoming.id}/report`);
|
||
const report = await reportResponse.json(); expect(report.session.issueId).toBe(inboundTask.id);
|
||
expect(report.report).toMatchObject({costMicroUsd: "152340", transcript: [{text: "Fixture call transcript"}]});
|
||
expect(JSON.stringify(report)).not.toContain("recording"); expect(JSON.stringify(report)).not.toContain("fixture-speko-key");
|
||
await expect(page.getByRole("region", {name: "Your calls", exact: true})).toHaveCount(0);
|
||
await page.goto(`/${company.issuePrefix}/apps/chat/${endpoint.id}/activity`);
|
||
const history = page.getByRole("region", {name: "Connection activity", exact: true});
|
||
await history.getByRole("button", {name: "Refresh", exact: true}).click();
|
||
await expect(history).toContainText("Incoming call · Completed");
|
||
await history.locator("summary").filter({hasText: "Incoming call · Completed"}).click();
|
||
await expect(history.getByRole("list", {name: "Call transcript", exact: true})).toContainText("Fixture call transcript");
|
||
expect((await evidence()).reports.filter((row: any) => row.sessionId === boundIncoming.id)).toHaveLength(1);
|
||
const retainedIncomingTask = await (await page.request.get(`${origin}/api/issues/${inboundTask.id}`)).json();
|
||
expect(retainedIncomingTask.assigneeAgentId).toBe(agent.id); expect(retainedIncomingTask.status).not.toBe("cancelled");
|
||
await page.goto(`/${company.issuePrefix}/apps/chat/${endpoint.id}/settings`);
|
||
// E09: opted-in callers start a task-scoped, low-trust conversation without sign-in.
|
||
await page.getByRole("checkbox", {name: "Let callers start new low-trust tasks without sign-in", exact: true}).check();
|
||
await page.getByRole("button", {name: "Save incoming call setting", exact: true}).click();
|
||
await expect(page.getByRole("status").filter({hasText: "Incoming call setting saved"})).toBeVisible();
|
||
const guest = await providerCommand({action: "inbound", agentId: "agent_fixture", callerId: "+12015559876"});
|
||
await expect(page.getByRole("heading", {name: "Incoming conversation", exact: true})).toBeVisible();
|
||
const guestRequest = {sessionId: guest.sessionId, tool: "submit_request", args: {text: "Please contact me about a shipping question. Do not start private work."}, toolCallId: "guest-message"};
|
||
const guestAccepted = await providerCommand(guestRequest);
|
||
expect(guestAccepted).toMatchObject({authorization: "guest_intake", status: "accepted"});
|
||
expect(await providerCommand(guestRequest)).toEqual(guestAccepted);
|
||
expect(await providerCommand({sessionId: guest.sessionId, tool: "get_updates", args: {cursor: 0}, toolCallId: "guest-read"})).toMatchObject({authorization: "guest_intake", updates: []});
|
||
await expect.poll(async () => (await evidence()).sessions.filter((call: any) => call.providerSessionId === guest.sessionId).length).toBe(1);
|
||
const guestSession = (await evidence()).sessions.find((call: any) => call.providerSessionId === guest.sessionId);
|
||
const guestTaskResponse = await page.request.get(`${origin}/api/issues/${guestSession.issueId}`);
|
||
const guestTask = await guestTaskResponse.json();
|
||
expect(guestTask).toMatchObject({ assigneeAgentId: agent.id, responsibleUserId: null, sourceTrust: {disposition: "quarantined"}, executionPolicy: {authorizationPolicy: {trustBoundary: {allowedToolClasses: [], allowedSecretBindingIds: []}}}});
|
||
await expect.poll(async () => (await evidence()).comments.filter((row: any) => row.issueId === guestTask.id && row.body === (guestRequest.args as {text: string}).text).length).toBe(1);
|
||
// This fixture has only Local, so containment must block execution rather
|
||
// than silently grant the public caller the selected agent's host privileges.
|
||
await expect.poll(async () => (await evidence()).runs.find((run: any) => run.context?.issueId === guestTask.id)?.errorCode).toBe("low_trust_isolation_unavailable");
|
||
const guestRun = (await evidence()).runs.find((run: any) => run.context?.issueId === guestTask.id);
|
||
expect((await evidence()).wakeups.some((row: any) => row.runId === guestRun.id)).toBe(true);
|
||
expect((await evidence()).tools.filter((row: any) => row.sessionId === guestSession.id && row.tool === "submit_request")).toHaveLength(1);
|
||
await page.getByRole("button", {name: "End call", exact: true}).click();
|
||
await expect(page.getByRole("heading", {name: "Caller hung up", exact: true})).toBeFocused();
|
||
await expect.poll(async () => (await evidence()).sessions.find((call: any) => call.id === guestSession.id)?.state, {timeout: 45000}).toBe("ended");
|
||
await expect.poll(async () => (await evidence()).reports.find((report: any) => report.sessionId === guestSession.id)?.status, {timeout: 45000}).toBe("available");
|
||
await page.goto(`/${company.issuePrefix}/apps/chat/${endpoint.id}/activity`);
|
||
await history.getByRole("button", {name: "Refresh", exact: true}).click();
|
||
await expect(history.locator("summary").filter({hasText: "Incoming call · Completed"})).toHaveCount(2);
|
||
expect(external).toEqual([]);
|
||
await info.attach("durable-records", { body: JSON.stringify(await evidence(), null, 2), contentType: "application/json" });
|
||
await page.screenshot({ path: info.outputPath("completed.png"), fullPage: true });
|
||
});
|
||
|
||
// Connection setup selects an existing agent without changing its runtime or
|
||
// reassigning tasks created before the connection.
|
||
test("E02: selecting an existing agent preserves its configuration and historical assignment", async ({ page }) => {
|
||
const company = await (await page.request.post("/api/companies", { data: { name: "Speko existing agent" } })).json();
|
||
const created = await page.request.post(`/api/companies/${company.id}/agents`, { data: { name: "Existing support agent", role: "general", adapterType: "process", adapterConfig: { command: "true" }, runtimeConfig: { heartbeat: { enabled: false, wakeOnDemand: false } } } });
|
||
expect(created.ok()).toBe(true);
|
||
const agent = await created.json();
|
||
const task = await (await page.request.post(`/api/companies/${company.id}/issues`, { data: { title: "Preserve existing work", status: "backlog", assigneeAgentId: agent.id } })).json();
|
||
await page.goto(`/${company.issuePrefix}/apps/chat/connect?provider=speko`);
|
||
await expect(page.getByRole("button", { name: "Review recommended agent" })).toHaveCount(0);
|
||
await page.getByRole("button", { name: "Choose an active agent", exact: true }).click();
|
||
await page.getByRole("button", { name: "Select Existing support agent", exact: true }).click();
|
||
await page.getByRole("button", { name: "Continue", exact: true }).click();
|
||
await expect(page.getByText(/Existing support agent keeps its existing runtime/)).toBeVisible();
|
||
await page.reload();
|
||
await expect(page.getByText(/Existing support agent keeps its existing runtime/)).toBeVisible();
|
||
const agents = await (await page.request.get(`/api/companies/${company.id}/agents`)).json();
|
||
expect(agents).toHaveLength(1);
|
||
expect(agents[0].id).toBe(agent.id);
|
||
expect(agents[0].adapterConfig).toEqual(agent.adapterConfig);
|
||
expect(agents[0].reportsTo).toBe(agent.reportsTo);
|
||
expect((await (await page.request.get(`/api/issues/${task.id}`)).json()).assigneeAgentId).toBe(agent.id);
|
||
const endpoints = await (await page.request.get(`/api/companies/${company.id}/chat-endpoints`)).json();
|
||
expect(endpoints).toHaveLength(1);
|
||
expect(endpoints[0].assignedAgentId).toBe(agent.id);
|
||
});
|
||
|
||
|
||
test("E12: insufficient credits explains funding and never blindly retries a rejected call", async ({page}) => {
|
||
const company=await (await page.request.post("/api/companies",{data:{name:"Speko credit rejection"}})).json();
|
||
const agent=await (await page.request.post(`/api/companies/${company.id}/agents`,{data:{name:"Credit-test agent",role:"general",adapterType:"process",adapterConfig:{command:"true"},runtimeConfig:{heartbeat:{enabled:false,wakeOnDemand:false}}}})).json();
|
||
await page.addInitScript(() => { navigator.mediaDevices.getUserMedia=async () => {throw new Error("Rejected call must not request microphone");}; });
|
||
await page.goto(`/${company.issuePrefix}/apps/chat/connect?provider=speko`);
|
||
await page.getByRole("button",{name:"Choose an active agent",exact:true}).click();
|
||
await page.getByRole("button",{name:"Select Credit-test agent",exact:true}).click();
|
||
await page.getByRole("button",{name:"Continue",exact:true}).click();
|
||
await page.getByRole("textbox",{name:"Speko agent ID",exact:true}).fill("agent_credit_fixture");
|
||
await page.getByLabel("Speko API key",{exact:true}).fill("fixture-speko-key");
|
||
await page.getByRole("button",{name:"Connect Speko",exact:true}).click();
|
||
await expect(page.getByRole("button",{name:"Start voice",exact:true})).toBeVisible();
|
||
const creditState=async () => (await page.request.post(`http://127.0.0.1:${port+1}/command`,{headers:{Origin:origin},data:{action:"credits_required",agentId:"agent_credit_fixture",enabled:true}})).json();
|
||
expect((await creditState()).creationAttempts).toBe(0);
|
||
const responsePromise=page.waitForResponse(r => r.request().method()==="POST" && /\/voice-sessions$/.test(new URL(r.url()).pathname));
|
||
await page.getByRole("button",{name:"Start voice",exact:true}).click();
|
||
const response=await responsePromise;expect(response.status()).toBe(409);
|
||
const error=await response.json();expect(error.details.code).toBe("voice_credits_required");
|
||
expect(JSON.stringify(error)).not.toContain("private fixture provider detail");
|
||
await expect(page.getByRole("alert").filter({hasText:"Add credits in Speko"})).toBeVisible();
|
||
await expect(page.getByRole("button",{name:"Mute",exact:true})).toHaveCount(0);
|
||
const inspected=await (await page.request.get(`/api/companies/${company.id}/voice-sessions/${error.details.sessionId}`)).json();
|
||
expect(inspected).not.toHaveProperty("providerSessionId");expect(inspected.errorCode).toBe("insufficient_credits");
|
||
const {home}=JSON.parse(await readFile(resolve(import.meta.dirname,".instance.json"),"utf8"));
|
||
await expect.poll(async () => JSON.parse(await readFile(resolve(home,"evidence.json"),"utf8")).sessions.find((row: {id: string}) => row.id===error.details.sessionId)?.providerSessionId).toBeNull();
|
||
expect(["failed","ended"]).toContain(inspected.state);
|
||
// Replay the exact already-rejected creation, as after a lost response.
|
||
const replay=await page.request.post(response.url(),{data:response.request().postDataJSON()});
|
||
expect(replay.ok()).toBe(true);expect((await replay.json()).session.id).toBe(error.details.sessionId);
|
||
expect((await creditState()).creationAttempts).toBe(1);
|
||
});
|