Files
PaperClipAI/packages/shared/src/validators/app-definition.ts
T
DottaandPaperclip 3ec7309d91 feat(voice): add experimental Speko task connection runtime (#15727)
First PR: runtime. UI: #15728. Qualification: #15729.

## Thinking Path

> - Paperclip manages agent work through tasks and governed connections.
> - Chat connections already bind external conversations to those tasks.
> - Speko handles voice and telephony, while the selected Paperclip
agent does the work.
> - A call needs durable session authority, ordered follow-ups, and safe
result delivery.
> - This PR adds an explicit voice transport to the existing connection
runtime.
> - The UI and qualification suites follow in linked PRs. Speko stays
experimental.

## Linked Issues or Issue Description

**Problem or motivation**

Phone and browser voice conversations cannot use the chat connection
task lifecycle. A separate voice runtime is needed to preserve task
authority, retries, and reply history.

**Proposed solution**

Add company-scoped voice sessions, signed callbacks, durable replies,
inbound intake, and an agent-facing outbound-call tool. Reuse existing
tasks, queues, secrets, and publications.

**Roadmap alignment**

This extends the experimental chat connector entry points under Agent
Chat and Connected Apps. The project owner requested this feature.

Refs #11418 for the original provider catalog proposal. This
implementation uses the newer chat connection infrastructure rather than
an MCP catalog-only integration. Refs #15721 for remote OpenCode
startup.

## What Changed

- Add additive voice schema, shared contracts, session APIs, and
provider callbacks.
- Bind browser, inbound, and outbound sessions to an endpoint, task, and
selected agent.
- Add low-trust guest intake in an explicitly selected sandbox and keep
ordinary approval rules.
- Submit task work through session-scoped tools and push approved
replies through Speko POST /v1/calls/{id}/messages.
- Distinguish provider acceptance from confirmed spoken playback.
Reconcile reports without downloading recordings.
- Add protocol, migration, persistence, provider, and
delivery-diagnostic tests.

## Verification

- Master 3389bea708 is integrated. Database/shared builds, server
typecheck, migration upgrade/replay checks, and 46
communication-guidance/provider/protocol tests passed.
- The final persistence suite passed 85 real-database tests. Five
focused native continuation/authorization regressions passed. Five
callback ownership/rotation tests passed after the last runtime repair.
- Review repairs cover reconnect ownership, current caller authority,
duplicate delivery, multipart answers, task-page answers, guest question
handling, and saved sandbox recovery.
- A real OpenCode/OpenRouter DeepSeek task ran in Daytona and saved a
correct disk-space answer. This did not establish live-call playback.
- Full local typecheck/build passed before the final repairs; focused
typechecks passed after them. Latest-head CI runs the repository gates.
- The separate integrated browser qualification passed two journeys and
failed browser answer retrieval. The feature remains experimental. The
report in #15729 records this failure and outstanding real-provider
acceptance.
- No lockfiles are included. Speko migrations follow master as
0324–0330.

## Risks

- Voice sessions add migrations and provider-dependent lifecycle
behavior. Migrations preserve existing chat and AI transports.
- Callback origins must be reachable HTTPS. Provider keys and signing
secrets remain vaulted.
- Unverified callers receive task-scoped low-trust authority. Caller ID
does not grant private task access.
- Outbound telephony incurs provider charges. Provider acceptance is not
proof that the caller heard an answer.
- Keep the feature behind the existing experimental chat setting until
the real browser, inbound, outbound, and delayed-result acceptance
matrix passes.

## Model Used

OpenAI Codex, GPT-6, with tool use and code execution. The exact serving
model ID and context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-09 21:54:11 -05:00

20 lines
10 KiB
TypeScript

import { z } from "zod";
import { connectionInstructionTemplateSchema } from "../connection-instructions.js";
import { AI_CONNECTION_CAPABILITIES, aiConnectionMetadataSchema } from "../ai-connections.js";
import { connectionGrantKindSchema, toolConnectionOwnershipSchema, toolConnectionPurposeSchema, toolConnectionTransportSchema } from "./tool-access.js";
const appBrandAssetUrlSchema=z.string().refine((value)=>{
if(/^\/brands\/apps\/[a-z0-9][a-z0-9._-]*\.(?:svg|png)$/i.test(value))return true;
try{return new URL(value).protocol==="https:";}catch{return false;}
},{message:"Brand assets must be HTTPS URLs or local /brands/apps SVG/PNG paths"});
const field=z.object({key:z.string().min(1),label:z.string().min(1),type:z.enum(["text","password","textarea","datetime","select","checkbox"]),required:z.boolean().optional(),advanced:z.boolean().optional(),hidden:z.boolean().optional(),placeholder:z.string().optional(),helperMd:z.string().optional(),secret:z.boolean().optional(),prefix:z.string().optional(),defaultValue:z.union([z.string(),z.boolean()]).optional(),validation:z.object({pattern:z.string().optional(),maxLength:z.number().int().positive().optional()}).optional(),options:z.array(z.object({value:z.string(),label:z.string()})).optional(),transport:z.object({location:z.enum(["query","header"]),name:z.string().min(1),format:z.enum(["string","csv","boolean"]).optional(),omitFalse:z.boolean().optional()}).optional()}).superRefine((v,c)=>{if(v.required&&v.type!=="checkbox"&&!v.placeholder)c.addIssue({code:"custom",message:"Required fields need placeholders",path:["placeholder"]});if(v.type==="select"&&(!v.options||v.options.length===0))c.addIssue({code:"custom",message:"Select fields need options",path:["options"]});if(v.hidden&&v.defaultValue===undefined)c.addIssue({code:"custom",message:"Hidden fields need defaults",path:["defaultValue"]})});
export const connectionMethodDefSchema=z.object({key:z.string().min(1),label:z.string().min(1).optional(),purpose:z.union([toolConnectionPurposeSchema,z.literal("ai")]).optional(),provider:z.enum(["slack","github","discord","microsoft-teams","telegram","speko","agentmail","imessage-photon"]).optional(),transport:z.union([toolConnectionTransportSchema,z.literal("runtime_auth")]),ai:aiConnectionMetadataSchema.optional(),auth:z.enum(["oauth","api_key","none"]),oauthStrategy:z.enum(["paperclip_cloud_connector","paperclip_id_connector"]).optional(),connectorProfile:z.string().regex(/^[a-z0-9]+(?:[.-][a-z0-9]+)*$/).optional(),oauthClientSecretRequired:z.boolean().optional(),oauthClientRegistration:z.literal("dcr").optional(),capabilityProfile:z.object({key:z.string().min(1),label:z.string().min(1),description:z.string().min(1).optional()}).optional(),grantKinds:z.array(connectionGrantKindSchema).min(1).optional(),ownershipModes:z.array(toolConnectionOwnershipSchema).min(1),whenToUse:z.string().min(1),defaults:z.object({serverUrl:z.string().url().optional(),serverUrlTemplate:z.string().regex(/^https:\/\//).optional(),discoveryUrl:z.string().url().nullable().optional(),serviceHost:z.string().optional(),templateKey:z.string().optional(),authorizationEndpoint:z.string().url().optional(),tokenEndpoint:z.string().url().optional(),metadataUrl:z.string().url().optional(),scopesHint:z.array(z.string()).optional(),oauthAuthorizationParams:z.object({access_type:z.literal("offline").optional(),prompt:z.literal("consent").optional()}).optional(),toolArgumentDefaults:z.record(z.string(),z.unknown()).optional(),mcpSessionRequired:z.boolean().optional()}).optional(),tenantFields:z.array(field).optional(),extensionFields:z.array(field).optional(),configRequirements:z.object({atLeastOneOf:z.array(z.string().min(1)).min(1).optional()}).optional(),credentialFields:z.array(field).optional(),keyPlacement:z.object({location:z.enum(["header","query","body_json","env"]),name:z.string().min(1),prefix:z.string().nullable().optional()}).optional(),credentialSources:z.object({vercelConnect:z.object({services:z.array(z.string().min(1)).min(1),principalModes:z.array(z.enum(["app","user"])).min(1),scopes:z.array(z.string().min(1)).min(1),header:z.object({name:z.string().min(1),prefix:z.string().nullable().optional()})}).optional()}).optional(),guidanceMd:z.string().min(1),consoleLinks:z.object({register:z.string().url().optional(),keys:z.string().url().optional(),settings:z.string().url().optional(),docs:z.string().url().optional()}).optional(),warnings:z.array(z.string()).optional(),variants:z.array(z.object({key:z.string(),label:z.string(),whenToUse:z.string(),tenantFields:z.array(field).optional()})).optional(),riskTier:z.enum(["S1","S2","S3","S4"]),requiredResourceFilters:z.array(z.string()).optional()}).superRefine((v,c)=>{if((v.transport==="runtime_auth")!==Boolean(v.ai))c.addIssue({code:"custom",message:"Runtime authentication requires AI metadata and AI metadata requires runtime_auth",path:["ai"]});if(v.ai&&!AI_CONNECTION_CAPABILITIES[v.ai.provider].methods[v.ai.method])c.addIssue({code:"custom",message:"Unsupported AI sign-in method",path:["ai","method"]});if(v.ai&&v.auth!==(v.ai.method==="subscription"?"oauth":"api_key"))c.addIssue({code:"custom",message:"AI sign-in method must match authentication",path:["auth"]});if(v.oauthClientRegistration&&(v.auth!=="oauth"||v.transport!=="mcp_remote"||!v.ownershipModes.includes("dcr")))c.addIssue({code:"custom",message:"DCR client registration requires a remote MCP OAuth method with DCR ownership",path:["oauthClientRegistration"]});if(v.defaults?.mcpSessionRequired&&v.transport!=="mcp_remote")c.addIssue({code:"custom",message:"MCP session initialization preferences require remote MCP transport",path:["defaults","mcpSessionRequired"]});const purpose=v.purpose??"tool";if((purpose==="ai")!==(v.transport==="runtime_auth"))c.addIssue({code:"custom",message:"AI methods require runtime_auth and runtime_auth requires AI purpose",path:["purpose"]});if(["chat_sdk","voice"].includes(v.transport)&&purpose!=="channel")c.addIssue({code:"custom",message:"Chat SDK methods must be channel connections",path:["purpose"]});if(purpose==="channel"&&!["chat_sdk","voice"].includes(v.transport)&&!(v.provider==="agentmail"&&v.transport==="rest_api"))c.addIssue({code:"custom",message:"Channel connections must use the Chat SDK transport",path:["transport"]});if(purpose==="channel"&&((v.provider==="speko")!==(v.transport==="voice")))c.addIssue({code:"custom",message:"Speko channels require the voice transport",path:["transport"]});if(purpose==="channel"&&!v.provider)c.addIssue({code:"custom",message:"Channel connections require a chat provider",path:["provider"]});if(v.auth==="api_key"&&!v.keyPlacement&&purpose!=="channel")c.addIssue({code:"custom",message:"API-key tool methods require keyPlacement",path:["keyPlacement"]});if(v.oauthStrategy&&v.auth!=="oauth")c.addIssue({code:"custom",message:"OAuth strategies require OAuth auth",path:["oauthStrategy"]});if(v.oauthStrategy&&!v.connectorProfile)c.addIssue({code:"custom",message:"Paperclip Cloud connector methods require connectorProfile",path:["connectorProfile"]});if(v.connectorProfile&&!v.oauthStrategy)c.addIssue({code:"custom",message:"connectorProfile requires a Paperclip Cloud OAuth strategy",path:["connectorProfile"]});if(v.credentialSources?.vercelConnect&&(v.transport!=="mcp_remote"||v.auth==="none"))c.addIssue({code:"custom",message:"Vercel Connect requires an authenticated remote MCP method",path:["credentialSources","vercelConnect"]});const keys=new Set([...(v.tenantFields??[]),...(v.extensionFields??[])].map((entry)=>entry.key));for(const key of v.configRequirements?.atLeastOneOf??[])if(!keys.has(key))c.addIssue({code:"custom",message:"Config requirement references an unknown field",path:["configRequirements","atLeastOneOf"]});if(v.defaults?.serverUrl&&v.defaults.serverUrlTemplate)c.addIssue({code:"custom",message:"Use either serverUrl or serverUrlTemplate",path:["defaults"]});for(const placeholder of v.defaults?.serverUrlTemplate?.matchAll(/\{([a-zA-Z0-9_-]+)\}/g)??[])if(!keys.has(placeholder[1]))c.addIssue({code:"custom",message:"Server URL template references an unknown field",path:["defaults","serverUrlTemplate"]})});
export const appDefinitionSchema=z.object({agentInstructions:connectionInstructionTemplateSchema.optional(),aiConnectionRouter:z.object({pluginKey:z.string().min(1)}).optional(),schemaVersion:z.literal(1),slug:z.string().regex(/^[a-z0-9]+(?:-[a-z0-9]+)*$/),name:z.string().min(1),description:z.string().min(1),categories:z.array(z.enum(["ai","analytics","commerce","communication","content","data","developer","productivity","other"])).min(1),tags:z.array(z.string().min(1)).optional(),featured:z.boolean().optional(),branding:z.object({logoUrl:appBrandAssetUrlSchema,darkLogoUrl:appBrandAssetUrlSchema.optional(),backgroundColor:z.string().optional(),accentColor:z.string().optional()}),urlPatterns:z.array(z.string()),docsUrl:z.string().url().optional(),setupPrerequisite:z.object({title:z.string().min(1),description:z.string().min(1),steps:z.array(z.string().min(1)).min(1).optional(),actionLabel:z.string().min(1),actionUrl:z.string().url()} ).optional(),redirectConstraints:z.enum(["https-or-loopback-http"]).optional(),methods:z.array(connectionMethodDefSchema),suggestable:z.boolean().optional(),availability:z.object({available:z.boolean(),reason:z.string().optional(),robotEmail:z.string().optional()}).optional(),ownershipAvailability:z.object({platform_shared:z.boolean().optional(),platform_provisioned:z.boolean().optional(),customer:z.boolean().optional(),dcr:z.boolean().optional()}).optional()}).superRefine((value, ctx) => {
if (value.aiConnectionRouter) {
if (value.methods.length !== 0) ctx.addIssue({ code: "custom", message: "Pooled connectors use existing connections, not authentication methods", path: ["methods"] });
if (!value.categories.includes("ai")) ctx.addIssue({ code: "custom", message: "Pooled connectors must use the AI category", path: ["categories"] });
} else if (value.methods.length === 0) {
ctx.addIssue({ code: "custom", message: "Connectors require at least one connection method", path: ["methods"] });
}
});
export const appDefinitionsSchema=z.array(appDefinitionSchema).superRefine((v,c)=>{const s=new Set<string>();v.forEach((a,i)=>{if(s.has(a.slug))c.addIssue({code:"custom",message:"Duplicate slug",path:[i,"slug"]});s.add(a.slug)})});