mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 07:12:00 +01:00
Third PR: depends on #15728 and #15727. The user has reviewed and authorized merging this experimental feature stack. ## Thinking Path > - Paperclip manages agent work through tasks and governed connections. > - Chat connections already bind external conversations to those tasks. > - Speko handles voice and telephony, while the selected Paperclip agent does the work. > - A call needs durable session authority, ordered follow-ups, and safe result delivery. > - This PR adds repeatable qualification for the Speko voice task integration. > - This PR depends on #15728, which depends on #15727. Speko stays experimental. ## Linked Issues or Issue Description **Problem or motivation** Phone and browser voice conversations cannot use the chat connection task lifecycle. A separate voice runtime is needed to preserve task authority, retries, and reply history. **Proposed solution** Add company-scoped voice sessions, signed callbacks, durable replies, inbound intake, and an agent-facing outbound-call tool. Reuse existing tasks, queues, secrets, and publications. **Roadmap alignment** This extends the experimental chat connector entry points under Agent Chat and Connected Apps. The project owner requested this feature. Refs #11418 for the original provider catalog proposal. This implementation uses the newer chat connection infrastructure rather than an MCP catalog-only integration. Refs #15721 for remote OpenCode startup. ## What Changed - Add a real Paperclip server/database E2E fixture that stubs only external Speko transport and the execution provider. - Add authenticated multi-user and cross-company qualification journeys. - Add Storybook interaction, accessibility, responsive, and visual-suite integration. - Add an E01–E16 runbook, component-to-story inventory, and current redacted qualification report. - Keep historical private call traces out of the PR. Provider proof tools are opt-in. ## Verification - All 21 standalone proof tests and browser client/server syntax checks pass. They cover signatures, redaction, cleanup retry, final audio saves, bounded report transport and late server write ordering. No live calls were placed. - All 27 focused UI/controller regressions, UI typecheck, token gates, and 28 map/catalog structural tests pass. The shared connection cache test covers pause/resume invalidation during the 30-second fresh period. - Storybook builds 157 Speko production-component stories. The complete prior 156-story set passed all 624 checks on UI source249cbc616a. After the final cache repair, all 152 affected page/launcher checks passed across both themes, 390/1200 widths and reduced motion. Real microphone access and external provider requests are blocked. - Hosted Storybook origin isolation and representative narrow-screen stories pass. Provider requests remain blocked. - Source annotations follow the current generated-catalog contract. No generated catalog or lockfile is included. Three inherited GitHub-heading tests passed after master reconciliation. - Commit905bc5da5dhas 55 passing checks, including required verify/E2E gates, and a fresh Greptile 5/5 review with zero unresolved findings. Local fixture success does not establish provider or spoken-playback acceptance. - The separate integrated browser run passed two journeys and failed browser answer retrieval. Real browser/telephone playback and timing acceptance remain outstanding. See the dated qualification report and E01–E16 runbook. Speko stays experimental. ## Risks - Voice sessions add migrations and provider-dependent lifecycle behavior. Migrations preserve existing chat and AI transports. - Callback origins must be reachable HTTPS. Provider keys and signing secrets remain vaulted. - Unverified callers receive task-scoped low-trust authority. Caller ID does not grant private task access. - Outbound telephony incurs provider charges. Provider acceptance is not proof that the caller heard an answer. - Keep the feature behind the existing experimental chat setting until the real browser, inbound, outbound, and delayed-result acceptance matrix passes. ## Model Used OpenAI Codex, GPT-6, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
87 lines
4.9 KiB
TypeScript
87 lines
4.9 KiB
TypeScript
import { readFileSync } from "node:fs";
|
|
import { createRequire } from "node:module";
|
|
import { resolve } from "node:path";
|
|
import { test, expect, type Page } from "@playwright/test";
|
|
|
|
const require = createRequire(import.meta.url);
|
|
const root = resolve(import.meta.dirname, "../..");
|
|
const addon = require.resolve("@storybook/addon-a11y/package.json", { paths: [resolve(root, "ui")] });
|
|
const axePath = require.resolve("axe-core/axe.min.js", { paths: [addon] });
|
|
const index = JSON.parse(readFileSync(resolve(root, "ui/storybook-static/index.json"), "utf8"));
|
|
const stories = (Object.values(index.entries) as { id: string; type: string }[]).filter((entry) => entry.type === "story" && entry.id.startsWith("connections-speko-"));
|
|
async function protectStoryOrigin(page: Page, baseURL: string, external: string[]) {
|
|
const storyOrigin = new URL(baseURL).origin;
|
|
await page.route(/^https?:\/\//, (route) => {
|
|
const url = new URL(route.request().url());
|
|
if (url.origin !== storyOrigin) {
|
|
external.push(url.hostname);
|
|
return route.abort();
|
|
}
|
|
return route.fallback();
|
|
});
|
|
}
|
|
|
|
test("Speko network isolation permits hosted Storybook and blocks providers", async ({ page }) => {
|
|
const baseURL = "https://storybook.example.invalid";
|
|
const external: string[] = [];
|
|
const served: string[] = [];
|
|
await page.route(/^https?:\/\//, (route) => {
|
|
served.push(route.request().url());
|
|
return route.fulfill({ contentType: "text/html", body: "<!doctype html><title>Hosted fixture</title>" });
|
|
});
|
|
await protectStoryOrigin(page, baseURL, external);
|
|
await page.goto(`${baseURL}/iframe.html`);
|
|
await expect(page).toHaveTitle("Hosted fixture");
|
|
expect(external).toEqual([]);
|
|
expect(await page.evaluate(async (origin) => (await fetch(`${origin}/asset`)).ok, baseURL)).toBe(true);
|
|
expect(await page.evaluate(async () => {
|
|
try { await fetch("https://api.speko.dev/v1/calls"); return false; }
|
|
catch { return true; }
|
|
})).toBe(true);
|
|
expect(external).toEqual(["api.speko.dev"]);
|
|
expect(served).toEqual([`${baseURL}/iframe.html`, `${baseURL}/asset`]);
|
|
});
|
|
|
|
for (const story of stories) for (const theme of ["light", "dark"]) for (const width of [390, 1200]) {
|
|
test(`Speko accessibility ${story.id} ${theme} ${width}`, async ({ page }, info) => {
|
|
const errors: string[] = [], external: string[] = [];
|
|
page.on("pageerror", (error) => errors.push(error.message));
|
|
await protectStoryOrigin(page, info.project.use.baseURL!, external);
|
|
await page.addInitScript(() => {
|
|
Object.defineProperty(navigator.mediaDevices, "getUserMedia", { value: () => { throw new Error("Story attempted real microphone access"); } });
|
|
});
|
|
await page.setViewportSize({ width, height: 844 });
|
|
await page.goto(`/iframe.html?id=${story.id}&viewMode=story&globals=theme:${theme}`);
|
|
await expect(page.locator("#storybook-root")).not.toBeEmpty({ timeout: 15_000 });
|
|
await expect(page.locator(".sb-errordisplay")).not.toBeVisible();
|
|
// Storybook's play function exercises the actual controls and controller.
|
|
await page.waitForFunction((id) => document.body.dataset.spekoStoryReady === id, story.id);
|
|
expect(await page.locator("body").getAttribute("data-speko-story-error")).toBeNull();
|
|
if (story.id === "connections-speko-call-history--keyboard-disclosure") {
|
|
const summary = page.locator("#storybook-root summary");
|
|
await summary.press("Enter");
|
|
await expect(page.getByRole("link", {name: "Conversation task", exact: true})).toBeVisible();
|
|
await summary.press("Enter");
|
|
await expect(page.getByRole("link", {name: "Conversation task", exact: true})).not.toBeVisible();
|
|
}
|
|
await page.evaluate(() => document.fonts.ready);
|
|
await page.addScriptTag({ path: axePath });
|
|
let violations: unknown[] = [];
|
|
// The addon also scans after play. Wait for that scan rather than racing
|
|
// its shared axe instance; retry only the explicit concurrent-run signal.
|
|
await expect.poll(async () => page.evaluate(async () => {
|
|
try {
|
|
const result = await (window as any).axe.run({ include: ["#storybook-root", "[role=dialog]"] }, { runOnly: { type: "tag", values: ["wcag2a", "wcag2aa", "wcag21aa"] } });
|
|
return { complete: true, violations: result.violations.map((v: any) => ({ id: v.id, impact: v.impact, nodes: v.nodes.map((n: any) => n.target) })) };
|
|
} catch (error) {
|
|
if (error instanceof Error && error.message.startsWith("Axe is already running")) return { complete: false, violations: [] };
|
|
throw error;
|
|
}
|
|
}).then((result) => { violations = result.violations; return result.complete; })).toBe(true);
|
|
expect(violations).toEqual([]);
|
|
expect(errors).toEqual([]); expect(external).toEqual([]);
|
|
expect(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth)).toBe(true);
|
|
await page.screenshot({ path: info.outputPath(`${story.id}-${theme}-${width}.png`), fullPage: true });
|
|
});
|
|
}
|