mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 09:07:46 +02:00
Third PR: depends on #15728 and #15727. The user has reviewed and authorized merging this experimental feature stack. ## Thinking Path > - Paperclip manages agent work through tasks and governed connections. > - Chat connections already bind external conversations to those tasks. > - Speko handles voice and telephony, while the selected Paperclip agent does the work. > - A call needs durable session authority, ordered follow-ups, and safe result delivery. > - This PR adds repeatable qualification for the Speko voice task integration. > - This PR depends on #15728, which depends on #15727. Speko stays experimental. ## Linked Issues or Issue Description **Problem or motivation** Phone and browser voice conversations cannot use the chat connection task lifecycle. A separate voice runtime is needed to preserve task authority, retries, and reply history. **Proposed solution** Add company-scoped voice sessions, signed callbacks, durable replies, inbound intake, and an agent-facing outbound-call tool. Reuse existing tasks, queues, secrets, and publications. **Roadmap alignment** This extends the experimental chat connector entry points under Agent Chat and Connected Apps. The project owner requested this feature. Refs #11418 for the original provider catalog proposal. This implementation uses the newer chat connection infrastructure rather than an MCP catalog-only integration. Refs #15721 for remote OpenCode startup. ## What Changed - Add a real Paperclip server/database E2E fixture that stubs only external Speko transport and the execution provider. - Add authenticated multi-user and cross-company qualification journeys. - Add Storybook interaction, accessibility, responsive, and visual-suite integration. - Add an E01–E16 runbook, component-to-story inventory, and current redacted qualification report. - Keep historical private call traces out of the PR. Provider proof tools are opt-in. ## Verification - All 21 standalone proof tests and browser client/server syntax checks pass. They cover signatures, redaction, cleanup retry, final audio saves, bounded report transport and late server write ordering. No live calls were placed. - All 27 focused UI/controller regressions, UI typecheck, token gates, and 28 map/catalog structural tests pass. The shared connection cache test covers pause/resume invalidation during the 30-second fresh period. - Storybook builds 157 Speko production-component stories. The complete prior 156-story set passed all 624 checks on UI source249cbc616a. After the final cache repair, all 152 affected page/launcher checks passed across both themes, 390/1200 widths and reduced motion. Real microphone access and external provider requests are blocked. - Hosted Storybook origin isolation and representative narrow-screen stories pass. Provider requests remain blocked. - Source annotations follow the current generated-catalog contract. No generated catalog or lockfile is included. Three inherited GitHub-heading tests passed after master reconciliation. - Commit905bc5da5dhas 55 passing checks, including required verify/E2E gates, and a fresh Greptile 5/5 review with zero unresolved findings. Local fixture success does not establish provider or spoken-playback acceptance. - The separate integrated browser run passed two journeys and failed browser answer retrieval. Real browser/telephone playback and timing acceptance remain outstanding. See the dated qualification report and E01–E16 runbook. Speko stays experimental. ## Risks - Voice sessions add migrations and provider-dependent lifecycle behavior. Migrations preserve existing chat and AI transports. - Callback origins must be reachable HTTPS. Provider keys and signing secrets remain vaulted. - Unverified callers receive task-scoped low-trust authority. Caller ID does not grant private task access. - Outbound telephony incurs provider charges. Provider acceptance is not proof that the caller heard an answer. - Keep the feature behind the existing experimental chat setting until the real browser, inbound, outbound, and delayed-result acceptance matrix passes. ## Model Used OpenAI Codex, GPT-6, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
131 lines
11 KiB
TypeScript
131 lines
11 KiB
TypeScript
/** Test-only external Speko transport. Paperclip routes/DB/permissions stay real. */
|
|
import { createServer } from "node:http";
|
|
import { createHmac, randomUUID } from "node:crypto";
|
|
import { once } from "node:events";
|
|
import { existsSync, readFileSync, writeFileSync, renameSync } from "node:fs";
|
|
import { resolve } from "node:path";
|
|
|
|
type Tool = { id: string; name: string; source: { kind: string; url: string; secret: string }; [key: string]: unknown };
|
|
type Session = { id: string; agentId: string; token: string; endedAt: string | null; mode: "browser" | "outbound_phone" | "inbound_phone" };
|
|
export async function installSpekoTestProvider(origin: string, port: number) {
|
|
const originalFetch = globalThis.fetch;
|
|
const tools = new Map<string, Tool[]>(), sessions = new Map<string, Session>(), hooks = new Map<string, any>();
|
|
const messages = new Map<string, {id: string; sessionId: string; text: string; mode: string}[]>();
|
|
let numberAgentId: string | null = null;
|
|
const creditRejectedAgents = new Set<string>(), creationAttempts = new Map<string, number>();
|
|
// The simulated external provider outlives an app restart, just as Speko does.
|
|
// This file contains only deterministic fixture credentials, never a real key.
|
|
const stateFile = resolve(process.env.PAPERCLIP_HOME!, "speko-fixture-state.json");
|
|
if (existsSync(stateFile)) {
|
|
const state = JSON.parse(readFileSync(stateFile, "utf8"));
|
|
for (const [key, value] of state.tools) tools.set(key, value);
|
|
numberAgentId = state.numberAgentId ?? null;
|
|
for (const [key, value] of state.hooks ?? []) hooks.set(key, value);
|
|
for (const [key, value] of state.messages ?? []) messages.set(key, value);
|
|
for (const [key, value] of state.sessions) sessions.set(key, value);
|
|
}
|
|
const persist = () => {
|
|
writeFileSync(`${stateFile}.pending`, JSON.stringify({ tools: [...tools], sessions: [...sessions], hooks: [...hooks], messages: [...messages], numberAgentId }), { mode: 0o600 });
|
|
renameSync(`${stateFile}.pending`, stateFile);
|
|
};
|
|
const json = (body: unknown, status = 200) => new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json" } });
|
|
globalThis.fetch = async (input, init) => {
|
|
const url = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url);
|
|
if (url.origin !== "https://api.speko.dev") return originalFetch(input, init);
|
|
if (new Headers(init?.headers).get("authorization") !== "Bearer fixture-speko-key") return json({ error: "fixture credential rejected" }, 401);
|
|
const method = init?.method ?? "GET", body = init?.body ? JSON.parse(String(init.body)) : undefined;
|
|
if (url.pathname === "/v1/phone-numbers" && method === "GET") return json([{id: "pn_fixture", organizationId: "org_fixture", e164: "+12015550123", label: "Fixture company line", agentId: numberAgentId, suspendedAt: null, direction: "both", setupStatus: {status: "ready", inboundReady: true, outboundReady: true, issues: []}}]);
|
|
if (url.pathname === "/v1/phone-numbers/pn_fixture" && method === "PATCH") {numberAgentId = body.agentId; persist(); return json({id: "pn_fixture"});}
|
|
if (url.pathname === "/v1/webhooks" && method === "GET") return json({data: [...hooks.values()].map(({signingSecret: _secret, ...hook}) => hook)});
|
|
const hook = /^\/v1\/webhooks(?:\/([^/]+))?$/.exec(url.pathname);
|
|
if (hook && ["POST", "PATCH"].includes(method)) {const id = hook[1] ?? `hook_${randomUUID()}`; hooks.set(id, {filterTags: {}, ...body, id}); persist(); return json({id});}
|
|
const agent = /^\/v1\/agents\/([^/]+)$/.exec(url.pathname);
|
|
if (agent && method === "GET") return json({ id: agent[1], organizationId: "org_fixture", name: "Fixture voice persona" });
|
|
if (agent && method === "PATCH") return json({ id: agent[1] });
|
|
const tool = /^\/v1\/agents\/([^/]+)\/tools(?:\/([^/]+))?$/.exec(url.pathname);
|
|
if (tool) {
|
|
const entries = tools.get(tool[1]) ?? [];
|
|
if (method === "GET") return json(entries.map(({ source: { secret: _secret, ...source }, ...value }) => ({ ...value, source })));
|
|
if (["POST", "PATCH"].includes(method)) {
|
|
const id = tool[2] ?? `tool_${randomUUID()}`;
|
|
tools.set(tool[1], [...entries.filter(entry => entry.id !== id), { ...body, id }]);
|
|
persist();
|
|
return json({ id });
|
|
}
|
|
}
|
|
if (["/v1/sessions", "/v1/sessions/phone"].includes(url.pathname) && method === "POST") {
|
|
creationAttempts.set(body.agentId, (creationAttempts.get(body.agentId) ?? 0) + 1);
|
|
if (creditRejectedAgents.has(body.agentId)) return json({error: "private fixture provider detail", code: "INSUFFICIENT_CREDITS"},402);
|
|
const id = randomUUID();
|
|
sessions.set(id, { id, agentId: body.agentId, token: body.toolSecrets.paperclip_session_token, endedAt: null, mode: url.pathname.endsWith("/phone") ? "outbound_phone" : "browser" });
|
|
persist();
|
|
if (url.pathname.endsWith("/phone")) return json({ sessionId: id, status: "dialing" });
|
|
return json({ sessionId: id, transportToken: `fixture:${id}`, transportUrl: "wss://speko-transport.invalid" });
|
|
}
|
|
const messageCall = /^\/v1\/calls\/([^/]+)\/messages$/.exec(url.pathname);
|
|
if (messageCall && method === "POST") {
|
|
const session = sessions.get(messageCall[1]);
|
|
if (!session) return json({error: "unknown fixture call"}, 404);
|
|
if (session.endedAt) return json({error: "call already ended"}, 409);
|
|
if (typeof body.text !== "string" || !body.text.trim() || body.mode !== "respond") return json({error: "invalid message"}, 400);
|
|
const message = {id: `message_${randomUUID()}`, sessionId: session.id, text: body.text, mode: body.mode};
|
|
messages.set(session.id, [...messages.get(session.id) ?? [], message]); persist();
|
|
return json({message_id: message.id}, 202);
|
|
}
|
|
const call = /^\/v1\/calls\/([^/]+)(\/end)?$/.exec(url.pathname);
|
|
if (call) {
|
|
const session = sessions.get(call[1]); if (!session) return json({ error: "unknown fixture call" }, 404);
|
|
if (call[2] && method === "POST") { session.endedAt = new Date().toISOString(); persist(); return json({ status: "already_ended" }); }
|
|
if (method === "GET") return json({ id: session.id, status: session.endedAt ? "ended" : "active", endedAt: session.endedAt, ended_at: session.endedAt, duration_seconds: session.endedAt ? 60 : null, report: session.endedAt ? {session_id: session.id, cost_micro_usd: "152340", updated_at: session.endedAt, transcript: {entries: [{id: "fixture-report-turn", index: 0, source: "user", text: "Fixture call transcript", started_at: session.endedAt, ended_at: session.endedAt}]} } : null, recording_resource_uri: "private-fixture-recording-do-not-fetch" });
|
|
}
|
|
throw new Error(`Unimplemented Speko fixture request: ${method} ${url.pathname}`);
|
|
};
|
|
const server = createServer(async (req, res) => {
|
|
if (req.headers.origin !== origin) { res.writeHead(403).end(); return; }
|
|
res.setHeader("access-control-allow-origin", origin);
|
|
res.setHeader("access-control-allow-headers", "content-type");
|
|
res.setHeader("access-control-allow-methods", "POST, OPTIONS");
|
|
if (req.method === "OPTIONS") { res.writeHead(204).end(); return; }
|
|
if (req.method !== "POST" || req.url !== "/command") { res.writeHead(404).end(); return; }
|
|
try {
|
|
const chunks: Buffer[] = []; let size = 0;
|
|
for await (const chunk of req) { size += chunk.length; if (size > 64 * 1024) throw new Error("Fixture payload too large"); chunks.push(chunk); }
|
|
const command = JSON.parse(Buffer.concat(chunks).toString());
|
|
if (command.action === "credits_required") {
|
|
if (command.enabled) creditRejectedAgents.add(command.agentId); else creditRejectedAgents.delete(command.agentId);
|
|
res.writeHead(200, {"content-type": "application/json"}).end(JSON.stringify({creationAttempts: creationAttempts.get(command.agentId) ?? 0})); return;
|
|
}
|
|
if (command.action === "inbound") {
|
|
const hook = [...hooks.values()].find(h => h.agentIds.includes(command.agentId) && h.events.includes("call.pre_call"));
|
|
if (!hook || numberAgentId !== command.agentId) {res.writeHead(409).end(); return;}
|
|
const id = randomUUID(), body = JSON.stringify({type: "call.pre_call", call_id: id, session_id: id, organization_id: "org_fixture", direction: "inbound", phone_number_id: "pn_fixture", dialed_number: "+12015550123", from: command.callerId ?? "+12015551234"});
|
|
const timestamp = String(Math.floor(Date.now()/1000)), webhookId = `msg_${id}`;
|
|
const signature = createHmac("sha256", Buffer.from(hook.signingSecret.slice(6), "base64")).update(`${webhookId}.${timestamp}.${body}`).digest("base64");
|
|
const response = await originalFetch(`${origin}${new URL(hook.url).pathname}`, {method: "POST", headers: {"content-type": "application/json", "webhook-id": webhookId, "webhook-timestamp": timestamp, "webhook-signature": `v1,${signature}`}, body});
|
|
const payload = await response.json() as any;
|
|
if (!response.ok) {res.writeHead(response.status).end(JSON.stringify(payload)); return;}
|
|
sessions.set(id, {id, agentId: command.agentId, token: payload.toolSecrets.paperclip_session_token, endedAt: null, mode: "inbound_phone"}); persist();
|
|
res.writeHead(200, {"content-type": "application/json"}).end(JSON.stringify({sessionId: id, firstMessage: payload.firstMessage})); return;
|
|
}
|
|
if (command.action === "messages") {
|
|
res.writeHead(200, {"content-type": "application/json"}).end(JSON.stringify(messages.get(command.sessionId) ?? [])); return;
|
|
}
|
|
const session = sessions.get(command.sessionId);
|
|
const tool = session && tools.get(session.agentId)?.find(item => item.name === command.tool);
|
|
if (!session || session.endedAt || !tool) { res.writeHead(404).end(); return; }
|
|
const toolId = command.toolCallId ?? randomUUID();
|
|
const body = JSON.stringify({ session_id: session.id, tool_call_id: toolId, idempotency_key: `${session.id}:${toolId}`, tool: command.tool, args: command.args });
|
|
const timestamp = String(Math.floor(Date.now() / 1000)), webhookId = `msg_${toolId}`;
|
|
const signature = createHmac("sha256", Buffer.from(tool.source.secret.slice(6), "base64")).update(`${webhookId}.${timestamp}.${body}`).digest("base64");
|
|
// Only external transport is stubbed: enter the production signed route.
|
|
const response = await originalFetch(`${origin}${new URL(tool.source.url).pathname}`, { method: "POST", headers: {
|
|
"content-type": "application/json", "webhook-id": webhookId, "webhook-timestamp": timestamp,
|
|
"webhook-signature": `v1,${signature}`, authorization: `Bearer ${session.token}`,
|
|
}, body });
|
|
res.writeHead(response.status, { "content-type": "application/json" }).end(await response.text());
|
|
} catch { res.writeHead(500).end(JSON.stringify({ error: "Fixture command failed" })); }
|
|
}).listen(port, "127.0.0.1");
|
|
await once(server, "listening");
|
|
return async () => { globalThis.fetch = originalFetch; server.close(); server.closeAllConnections(); };
|
|
}
|