Files
PaperClipAI/tests/speko-native/stub-provider.ts
T
DottaandPaperclip 2238831ecd test(voice): qualify Speko task conversations and Storybook surfaces (#15729)
Third PR: depends on #15728 and #15727. The user has reviewed and
authorized merging this experimental feature stack.

## Thinking Path

> - Paperclip manages agent work through tasks and governed connections.
> - Chat connections already bind external conversations to those tasks.
> - Speko handles voice and telephony, while the selected Paperclip
agent does the work.
> - A call needs durable session authority, ordered follow-ups, and safe
result delivery.
> - This PR adds repeatable qualification for the Speko voice task
integration.
> - This PR depends on #15728, which depends on #15727. Speko stays
experimental.

## Linked Issues or Issue Description

**Problem or motivation**

Phone and browser voice conversations cannot use the chat connection
task lifecycle. A separate voice runtime is needed to preserve task
authority, retries, and reply history.

**Proposed solution**

Add company-scoped voice sessions, signed callbacks, durable replies,
inbound intake, and an agent-facing outbound-call tool. Reuse existing
tasks, queues, secrets, and publications.

**Roadmap alignment**

This extends the experimental chat connector entry points under Agent
Chat and Connected Apps. The project owner requested this feature.

Refs #11418 for the original provider catalog proposal. This
implementation uses the newer chat connection infrastructure rather than
an MCP catalog-only integration. Refs #15721 for remote OpenCode
startup.

## What Changed

- Add a real Paperclip server/database E2E fixture that stubs only
external Speko transport and the execution provider.
- Add authenticated multi-user and cross-company qualification journeys.
- Add Storybook interaction, accessibility, responsive, and visual-suite
integration.
- Add an E01–E16 runbook, component-to-story inventory, and current
redacted qualification report.
- Keep historical private call traces out of the PR. Provider proof
tools are opt-in.

## Verification

- All 21 standalone proof tests and browser client/server syntax checks
pass. They cover signatures, redaction, cleanup retry, final audio
saves, bounded report transport and late server write ordering. No live
calls were placed.
- All 27 focused UI/controller regressions, UI typecheck, token gates,
and 28 map/catalog structural tests pass. The shared connection cache
test covers pause/resume invalidation during the 30-second fresh period.
- Storybook builds 157 Speko production-component stories. The complete
prior 156-story set passed all 624 checks on UI source 249cbc616a. After
the final cache repair, all 152 affected page/launcher checks passed
across both themes, 390/1200 widths and reduced motion. Real microphone
access and external provider requests are blocked.
- Hosted Storybook origin isolation and representative narrow-screen
stories pass. Provider requests remain blocked.
- Source annotations follow the current generated-catalog contract. No
generated catalog or lockfile is included. Three inherited
GitHub-heading tests passed after master reconciliation.
- Commit 905bc5da5d has 55 passing checks,
including required verify/E2E gates, and a fresh Greptile 5/5 review
with zero unresolved findings. Local fixture success does not establish
provider or spoken-playback acceptance.
- The separate integrated browser run passed two journeys and failed
browser answer retrieval. Real browser/telephone playback and timing
acceptance remain outstanding. See the dated qualification report and
E01–E16 runbook. Speko stays experimental.

## Risks

- Voice sessions add migrations and provider-dependent lifecycle
behavior. Migrations preserve existing chat and AI transports.
- Callback origins must be reachable HTTPS. Provider keys and signing
secrets remain vaulted.
- Unverified callers receive task-scoped low-trust authority. Caller ID
does not grant private task access.
- Outbound telephony incurs provider charges. Provider acceptance is not
proof that the caller heard an answer.
- Keep the feature behind the existing experimental chat setting until
the real browser, inbound, outbound, and delayed-result acceptance
matrix passes.

## Model Used

OpenAI Codex, GPT-6, with tool use and code execution. The exact serving
model ID and context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-13 16:54:41 -05:00

131 lines
11 KiB
TypeScript

/** Test-only external Speko transport. Paperclip routes/DB/permissions stay real. */
import { createServer } from "node:http";
import { createHmac, randomUUID } from "node:crypto";
import { once } from "node:events";
import { existsSync, readFileSync, writeFileSync, renameSync } from "node:fs";
import { resolve } from "node:path";
type Tool = { id: string; name: string; source: { kind: string; url: string; secret: string }; [key: string]: unknown };
type Session = { id: string; agentId: string; token: string; endedAt: string | null; mode: "browser" | "outbound_phone" | "inbound_phone" };
export async function installSpekoTestProvider(origin: string, port: number) {
const originalFetch = globalThis.fetch;
const tools = new Map<string, Tool[]>(), sessions = new Map<string, Session>(), hooks = new Map<string, any>();
const messages = new Map<string, {id: string; sessionId: string; text: string; mode: string}[]>();
let numberAgentId: string | null = null;
const creditRejectedAgents = new Set<string>(), creationAttempts = new Map<string, number>();
// The simulated external provider outlives an app restart, just as Speko does.
// This file contains only deterministic fixture credentials, never a real key.
const stateFile = resolve(process.env.PAPERCLIP_HOME!, "speko-fixture-state.json");
if (existsSync(stateFile)) {
const state = JSON.parse(readFileSync(stateFile, "utf8"));
for (const [key, value] of state.tools) tools.set(key, value);
numberAgentId = state.numberAgentId ?? null;
for (const [key, value] of state.hooks ?? []) hooks.set(key, value);
for (const [key, value] of state.messages ?? []) messages.set(key, value);
for (const [key, value] of state.sessions) sessions.set(key, value);
}
const persist = () => {
writeFileSync(`${stateFile}.pending`, JSON.stringify({ tools: [...tools], sessions: [...sessions], hooks: [...hooks], messages: [...messages], numberAgentId }), { mode: 0o600 });
renameSync(`${stateFile}.pending`, stateFile);
};
const json = (body: unknown, status = 200) => new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json" } });
globalThis.fetch = async (input, init) => {
const url = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url);
if (url.origin !== "https://api.speko.dev") return originalFetch(input, init);
if (new Headers(init?.headers).get("authorization") !== "Bearer fixture-speko-key") return json({ error: "fixture credential rejected" }, 401);
const method = init?.method ?? "GET", body = init?.body ? JSON.parse(String(init.body)) : undefined;
if (url.pathname === "/v1/phone-numbers" && method === "GET") return json([{id: "pn_fixture", organizationId: "org_fixture", e164: "+12015550123", label: "Fixture company line", agentId: numberAgentId, suspendedAt: null, direction: "both", setupStatus: {status: "ready", inboundReady: true, outboundReady: true, issues: []}}]);
if (url.pathname === "/v1/phone-numbers/pn_fixture" && method === "PATCH") {numberAgentId = body.agentId; persist(); return json({id: "pn_fixture"});}
if (url.pathname === "/v1/webhooks" && method === "GET") return json({data: [...hooks.values()].map(({signingSecret: _secret, ...hook}) => hook)});
const hook = /^\/v1\/webhooks(?:\/([^/]+))?$/.exec(url.pathname);
if (hook && ["POST", "PATCH"].includes(method)) {const id = hook[1] ?? `hook_${randomUUID()}`; hooks.set(id, {filterTags: {}, ...body, id}); persist(); return json({id});}
const agent = /^\/v1\/agents\/([^/]+)$/.exec(url.pathname);
if (agent && method === "GET") return json({ id: agent[1], organizationId: "org_fixture", name: "Fixture voice persona" });
if (agent && method === "PATCH") return json({ id: agent[1] });
const tool = /^\/v1\/agents\/([^/]+)\/tools(?:\/([^/]+))?$/.exec(url.pathname);
if (tool) {
const entries = tools.get(tool[1]) ?? [];
if (method === "GET") return json(entries.map(({ source: { secret: _secret, ...source }, ...value }) => ({ ...value, source })));
if (["POST", "PATCH"].includes(method)) {
const id = tool[2] ?? `tool_${randomUUID()}`;
tools.set(tool[1], [...entries.filter(entry => entry.id !== id), { ...body, id }]);
persist();
return json({ id });
}
}
if (["/v1/sessions", "/v1/sessions/phone"].includes(url.pathname) && method === "POST") {
creationAttempts.set(body.agentId, (creationAttempts.get(body.agentId) ?? 0) + 1);
if (creditRejectedAgents.has(body.agentId)) return json({error: "private fixture provider detail", code: "INSUFFICIENT_CREDITS"},402);
const id = randomUUID();
sessions.set(id, { id, agentId: body.agentId, token: body.toolSecrets.paperclip_session_token, endedAt: null, mode: url.pathname.endsWith("/phone") ? "outbound_phone" : "browser" });
persist();
if (url.pathname.endsWith("/phone")) return json({ sessionId: id, status: "dialing" });
return json({ sessionId: id, transportToken: `fixture:${id}`, transportUrl: "wss://speko-transport.invalid" });
}
const messageCall = /^\/v1\/calls\/([^/]+)\/messages$/.exec(url.pathname);
if (messageCall && method === "POST") {
const session = sessions.get(messageCall[1]);
if (!session) return json({error: "unknown fixture call"}, 404);
if (session.endedAt) return json({error: "call already ended"}, 409);
if (typeof body.text !== "string" || !body.text.trim() || body.mode !== "respond") return json({error: "invalid message"}, 400);
const message = {id: `message_${randomUUID()}`, sessionId: session.id, text: body.text, mode: body.mode};
messages.set(session.id, [...messages.get(session.id) ?? [], message]); persist();
return json({message_id: message.id}, 202);
}
const call = /^\/v1\/calls\/([^/]+)(\/end)?$/.exec(url.pathname);
if (call) {
const session = sessions.get(call[1]); if (!session) return json({ error: "unknown fixture call" }, 404);
if (call[2] && method === "POST") { session.endedAt = new Date().toISOString(); persist(); return json({ status: "already_ended" }); }
if (method === "GET") return json({ id: session.id, status: session.endedAt ? "ended" : "active", endedAt: session.endedAt, ended_at: session.endedAt, duration_seconds: session.endedAt ? 60 : null, report: session.endedAt ? {session_id: session.id, cost_micro_usd: "152340", updated_at: session.endedAt, transcript: {entries: [{id: "fixture-report-turn", index: 0, source: "user", text: "Fixture call transcript", started_at: session.endedAt, ended_at: session.endedAt}]} } : null, recording_resource_uri: "private-fixture-recording-do-not-fetch" });
}
throw new Error(`Unimplemented Speko fixture request: ${method} ${url.pathname}`);
};
const server = createServer(async (req, res) => {
if (req.headers.origin !== origin) { res.writeHead(403).end(); return; }
res.setHeader("access-control-allow-origin", origin);
res.setHeader("access-control-allow-headers", "content-type");
res.setHeader("access-control-allow-methods", "POST, OPTIONS");
if (req.method === "OPTIONS") { res.writeHead(204).end(); return; }
if (req.method !== "POST" || req.url !== "/command") { res.writeHead(404).end(); return; }
try {
const chunks: Buffer[] = []; let size = 0;
for await (const chunk of req) { size += chunk.length; if (size > 64 * 1024) throw new Error("Fixture payload too large"); chunks.push(chunk); }
const command = JSON.parse(Buffer.concat(chunks).toString());
if (command.action === "credits_required") {
if (command.enabled) creditRejectedAgents.add(command.agentId); else creditRejectedAgents.delete(command.agentId);
res.writeHead(200, {"content-type": "application/json"}).end(JSON.stringify({creationAttempts: creationAttempts.get(command.agentId) ?? 0})); return;
}
if (command.action === "inbound") {
const hook = [...hooks.values()].find(h => h.agentIds.includes(command.agentId) && h.events.includes("call.pre_call"));
if (!hook || numberAgentId !== command.agentId) {res.writeHead(409).end(); return;}
const id = randomUUID(), body = JSON.stringify({type: "call.pre_call", call_id: id, session_id: id, organization_id: "org_fixture", direction: "inbound", phone_number_id: "pn_fixture", dialed_number: "+12015550123", from: command.callerId ?? "+12015551234"});
const timestamp = String(Math.floor(Date.now()/1000)), webhookId = `msg_${id}`;
const signature = createHmac("sha256", Buffer.from(hook.signingSecret.slice(6), "base64")).update(`${webhookId}.${timestamp}.${body}`).digest("base64");
const response = await originalFetch(`${origin}${new URL(hook.url).pathname}`, {method: "POST", headers: {"content-type": "application/json", "webhook-id": webhookId, "webhook-timestamp": timestamp, "webhook-signature": `v1,${signature}`}, body});
const payload = await response.json() as any;
if (!response.ok) {res.writeHead(response.status).end(JSON.stringify(payload)); return;}
sessions.set(id, {id, agentId: command.agentId, token: payload.toolSecrets.paperclip_session_token, endedAt: null, mode: "inbound_phone"}); persist();
res.writeHead(200, {"content-type": "application/json"}).end(JSON.stringify({sessionId: id, firstMessage: payload.firstMessage})); return;
}
if (command.action === "messages") {
res.writeHead(200, {"content-type": "application/json"}).end(JSON.stringify(messages.get(command.sessionId) ?? [])); return;
}
const session = sessions.get(command.sessionId);
const tool = session && tools.get(session.agentId)?.find(item => item.name === command.tool);
if (!session || session.endedAt || !tool) { res.writeHead(404).end(); return; }
const toolId = command.toolCallId ?? randomUUID();
const body = JSON.stringify({ session_id: session.id, tool_call_id: toolId, idempotency_key: `${session.id}:${toolId}`, tool: command.tool, args: command.args });
const timestamp = String(Math.floor(Date.now() / 1000)), webhookId = `msg_${toolId}`;
const signature = createHmac("sha256", Buffer.from(tool.source.secret.slice(6), "base64")).update(`${webhookId}.${timestamp}.${body}`).digest("base64");
// Only external transport is stubbed: enter the production signed route.
const response = await originalFetch(`${origin}${new URL(tool.source.url).pathname}`, { method: "POST", headers: {
"content-type": "application/json", "webhook-id": webhookId, "webhook-timestamp": timestamp,
"webhook-signature": `v1,${signature}`, authorization: `Bearer ${session.token}`,
}, body });
res.writeHead(response.status, { "content-type": "application/json" }).end(await response.text());
} catch { res.writeHead(500).end(JSON.stringify({ error: "Fixture command failed" })); }
}).listen(port, "127.0.0.1");
await once(server, "listening");
return async () => { globalThis.fetch = originalFetch; server.close(); server.closeAllConnections(); };
}