Files
PaperClipAI/tests/speko-voice-proof/browser-server.mjs
T
DottaandPaperclip 2238831ecd test(voice): qualify Speko task conversations and Storybook surfaces (#15729)
Third PR: depends on #15728 and #15727. The user has reviewed and
authorized merging this experimental feature stack.

## Thinking Path

> - Paperclip manages agent work through tasks and governed connections.
> - Chat connections already bind external conversations to those tasks.
> - Speko handles voice and telephony, while the selected Paperclip
agent does the work.
> - A call needs durable session authority, ordered follow-ups, and safe
result delivery.
> - This PR adds repeatable qualification for the Speko voice task
integration.
> - This PR depends on #15728, which depends on #15727. Speko stays
experimental.

## Linked Issues or Issue Description

**Problem or motivation**

Phone and browser voice conversations cannot use the chat connection
task lifecycle. A separate voice runtime is needed to preserve task
authority, retries, and reply history.

**Proposed solution**

Add company-scoped voice sessions, signed callbacks, durable replies,
inbound intake, and an agent-facing outbound-call tool. Reuse existing
tasks, queues, secrets, and publications.

**Roadmap alignment**

This extends the experimental chat connector entry points under Agent
Chat and Connected Apps. The project owner requested this feature.

Refs #11418 for the original provider catalog proposal. This
implementation uses the newer chat connection infrastructure rather than
an MCP catalog-only integration. Refs #15721 for remote OpenCode
startup.

## What Changed

- Add a real Paperclip server/database E2E fixture that stubs only
external Speko transport and the execution provider.
- Add authenticated multi-user and cross-company qualification journeys.
- Add Storybook interaction, accessibility, responsive, and visual-suite
integration.
- Add an E01–E16 runbook, component-to-story inventory, and current
redacted qualification report.
- Keep historical private call traces out of the PR. Provider proof
tools are opt-in.

## Verification

- All 21 standalone proof tests and browser client/server syntax checks
pass. They cover signatures, redaction, cleanup retry, final audio
saves, bounded report transport and late server write ordering. No live
calls were placed.
- All 27 focused UI/controller regressions, UI typecheck, token gates,
and 28 map/catalog structural tests pass. The shared connection cache
test covers pause/resume invalidation during the 30-second fresh period.
- Storybook builds 157 Speko production-component stories. The complete
prior 156-story set passed all 624 checks on UI source 249cbc616a. After
the final cache repair, all 152 affected page/launcher checks passed
across both themes, 390/1200 widths and reduced motion. Real microphone
access and external provider requests are blocked.
- Hosted Storybook origin isolation and representative narrow-screen
stories pass. Provider requests remain blocked.
- Source annotations follow the current generated-catalog contract. No
generated catalog or lockfile is included. Three inherited
GitHub-heading tests passed after master reconciliation.
- Commit 905bc5da5d has 55 passing checks,
including required verify/E2E gates, and a fresh Greptile 5/5 review
with zero unresolved findings. Local fixture success does not establish
provider or spoken-playback acceptance.
- The separate integrated browser run passed two journeys and failed
browser answer retrieval. Real browser/telephone playback and timing
acceptance remain outstanding. See the dated qualification report and
E01–E16 runbook. Speko stays experimental.

## Risks

- Voice sessions add migrations and provider-dependent lifecycle
behavior. Migrations preserve existing chat and AI transports.
- Callback origins must be reachable HTTPS. Provider keys and signing
secrets remain vaulted.
- Unverified callers receive task-scoped low-trust authority. Caller ID
does not grant private task access.
- Outbound telephony incurs provider charges. Provider acceptance is not
proof that the caller heard an answer.
- Keep the feature behind the existing experimental chat setting until
the real browser, inbound, outbound, and delayed-result acceptance
matrix passes.

## Model Used

OpenAI Codex, GPT-6, with tool use and code execution. The exact serving
model ID and context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-13 16:54:41 -05:00

98 lines
7.5 KiB
JavaScript

/** One-session loopback browser qualification harness. No Paperclip authority. */
import { createServer } from 'node:http';
import { once } from 'node:events';
import { readFile, writeFile, mkdir } from 'node:fs/promises';
import { randomBytes, createHash } from 'node:crypto';
import { homedir } from 'node:os';
import { resolve, dirname } from 'node:path';
import { build } from 'esbuild';
import { createReportSaver } from './cleanup.mjs';
import { createProof, createProofServer, DELAY_MS } from './proof.mjs';
const key = process.env.SPEKO_MCP_API_KEY;
if (!key) throw new Error('Credential required in environment');
const state = JSON.parse(await readFile(`${homedir()}/.paperclip/speko-proof/state.json`, 'utf8'));
const out = resolve(process.env.SPEKO_PROOF_OUTPUT ?? `${homedir()}/.paperclip/speko-proof/browser-${Date.now()}`);
await mkdir(dirname(out), { recursive: true, mode: 0o700 }); await mkdir(out, { mode: 0o700 });
const sourceDigests = {};
for (const file of ['browser-server.mjs', 'browser-client.mjs', 'cleanup.mjs', 'browser.html', 'proof.mjs', 'application-notification.txt']) sourceDigests[file] = createHash('sha256').update(await readFile(new URL(file, import.meta.url))).digest('hex');
await writeFile(resolve(out, 'source.json'), JSON.stringify({ startedAt: new Date().toISOString(), sourceDigests }), { mode: 0o600 });
const bundle = await build({ entryPoints: [new URL('./browser-client.mjs', import.meta.url).pathname], bundle: true, write: false, platform: 'browser', format: 'esm' });
// Disk writes stay serialized even when a browser request times out.
const saveReport = createReportSaver(async body => {
await writeFile(resolve(out, 'browser-report.json'), JSON.stringify(body, null, 2), {mode: 0o600});
await writeFile(resolve(out, 'proof.json'), JSON.stringify(proof.evidence(), null, 2), {mode: 0o600});
});
const inputs = new Map();
for (const name of ['start', 'followup']) {
const data = await readFile(new URL(`../../.paperclip-local/speko-proof-audio/${name}.pcm`, import.meta.url));
if (!data.length || data.length % 2 || data.length > 48000 * 2 * 10) throw new Error('Invalid synthetic fixture');
inputs.set(name, data);
}
const port = Number(process.env.SPEKO_BROWSER_PORT ?? 3199);
const origin = `http://127.0.0.1:${port}`;
const launch = randomBytes(24).toString('hex'), csrf = randomBytes(24).toString('hex'), cookie = randomBytes(24).toString('hex');
const proof = createProof({ signingSecret: state.signingSecret, onEvent: (event) => { if (event.kind === 'request_accepted') startTime = Date.now(); } });
const hook = createProofServer(proof);
let sessionId, attempted = false, startTime, ending, stopped = false;
async function provider(path, body) {
const r = await fetch(`https://api.speko.dev${path}`, { method: 'POST', headers: { Authorization: `Bearer ${key}`, 'content-type': 'application/json' }, body: JSON.stringify(body), signal: AbortSignal.timeout(20000) });
if (!r.ok) throw new Error(`Provider returned ${r.status}`);
return r.json();
}
async function endSession() {
if (!sessionId) return;
ending ??= provider(`/v1/calls/${sessionId}/end`, {});
try { await ending; } catch (error) { ending = undefined; throw error; }
}
const server = createServer(async (req, res) => {
const reply = (code, data, type = 'application/json') => { res.writeHead(code, { 'content-type': type, 'cache-control': 'no-store', 'x-content-type-options': 'nosniff' }); res.end(type === 'application/json' ? JSON.stringify(data) : data); };
try {
if (req.headers.host !== `127.0.0.1:${port}`) return reply(403, { error: 'Wrong host' });
if (req.method === 'GET' && req.url === `/launch/${launch}`) {
res.setHeader('set-cookie', `proof=${cookie}; HttpOnly; SameSite=Strict; Path=/`);
res.writeHead(302, { location: '/', 'referrer-policy': 'no-referrer' }); return res.end();
}
if (!req.headers.cookie?.split(';').some((part) => part.trim() === `proof=${cookie}`)) return reply(401, { error: 'Launch this isolated proof from its private link' });
if (req.method === 'GET' && req.url === '/') return reply(200, (await readFile(new URL('./browser.html', import.meta.url), 'utf8')).replace('CONFIG_JSON', JSON.stringify({ csrf })), 'text/html');
if (req.method === 'GET' && req.url === '/client.js') return reply(200, bundle.outputFiles[0].contents, 'text/javascript');
if (req.method === 'GET' && ['/input/start', '/input/followup'].includes(req.url)) return reply(200, inputs.get(req.url.split('/').at(-1)), 'application/octet-stream');
if (req.method !== 'POST' || req.headers.origin !== origin || req.headers['x-proof-csrf'] !== csrf) return reply(403, { error: 'Same-origin request required' });
const limit = req.url === '/audio' ? 20 * 1024 * 1024 : 256 * 1024;
const chunks = []; let size = 0;
for await (const chunk of req) { size += chunk.length; if (size > limit) return reply(413, { error: 'Request too large' }); chunks.push(chunk); }
const raw = Buffer.concat(chunks);
if (req.url === '/audio') { await writeFile(resolve(out, 'received.webm'), raw, { mode: 0o600 }); return reply(200, { saved: true }); }
const body = JSON.parse(raw.toString('utf8'));
if (req.url === '/session') {
if (attempted) return reply(409, { error: 'One session per probe. Creation is never blindly retried.' });
attempted = true;
await writeFile(resolve(out, 'intent.json'), JSON.stringify({ action: 'create_browser_session', at: new Date().toISOString() }), { mode: 0o600 });
const session = await provider('/v1/sessions', { mode: 'cascade', agentId: state.agentId, ttlSeconds: 120, maxDurationSeconds: 180, constraints: { allowedProviders: { llm: ['openai:gpt-4.1'] } }, systemPrompt: await readFile(new URL('./application-notification.txt', import.meta.url), 'utf8') });
sessionId = session.sessionId;
await writeFile(resolve(out, 'session.json'), JSON.stringify({ sessionId }), { mode: 0o600 });
return reply(200, { sessionId, transportToken: session.transportToken, transportUrl: session.transportUrl });
}
if (req.url === '/evidence') {
const evidence = proof.evidence();
// A server-owned clock, bound to the first signed submission, schedules
// the synthetic completion hint; the page cannot advance it.
return reply(200, { acceptedRequests: evidence.acceptedRequests, resultReturned: evidence.resultReturned, notificationReady: startTime !== undefined && Date.now() >= startTime + DELAY_MS + 1000,
expectedNumber: evidence.expectedSyntheticResult?.match(/Verification number (\d+)/)?.[1] ?? null });
}
if (req.url === '/report') { await saveReport(body); return reply(200, { saved: true }); }
if (req.url === '/end') { await endSession(); return reply(200, { ended: true }); }
return reply(404, { error: 'Not found' });
} catch { return reply(502, { error: 'Probe request failed; inspect private evidence. Do not retry session creation.' }); }
});
try {
hook.listen(3198, '127.0.0.1'); await once(hook, 'listening');
server.listen(port, '127.0.0.1'); await once(server, 'listening');
} catch (error) { hook.close(); server.close(); throw error; }
console.log(JSON.stringify({ url: `${origin}/launch/${launch}`, output: out }));
async function stop() {
if (stopped) return; stopped = true;
try { await endSession(); } finally { proof.close(); hook.close(); hook.closeAllConnections(); server.close(); server.closeAllConnections(); clearTimeout(cap); }
}
const cap = setTimeout(() => { void stop(); }, 270000);
process.once('SIGINT', () => { void stop(); }); process.once('SIGTERM', () => { void stop(); });