Files
PaperClipAI/server/src/services/native-runtime/external-chat-question-response.ts
T
DottaandPaperclip 3ec7309d91 feat(voice): add experimental Speko task connection runtime (#15727)
First PR: runtime. UI: #15728. Qualification: #15729.

## Thinking Path

> - Paperclip manages agent work through tasks and governed connections.
> - Chat connections already bind external conversations to those tasks.
> - Speko handles voice and telephony, while the selected Paperclip
agent does the work.
> - A call needs durable session authority, ordered follow-ups, and safe
result delivery.
> - This PR adds an explicit voice transport to the existing connection
runtime.
> - The UI and qualification suites follow in linked PRs. Speko stays
experimental.

## Linked Issues or Issue Description

**Problem or motivation**

Phone and browser voice conversations cannot use the chat connection
task lifecycle. A separate voice runtime is needed to preserve task
authority, retries, and reply history.

**Proposed solution**

Add company-scoped voice sessions, signed callbacks, durable replies,
inbound intake, and an agent-facing outbound-call tool. Reuse existing
tasks, queues, secrets, and publications.

**Roadmap alignment**

This extends the experimental chat connector entry points under Agent
Chat and Connected Apps. The project owner requested this feature.

Refs #11418 for the original provider catalog proposal. This
implementation uses the newer chat connection infrastructure rather than
an MCP catalog-only integration. Refs #15721 for remote OpenCode
startup.

## What Changed

- Add additive voice schema, shared contracts, session APIs, and
provider callbacks.
- Bind browser, inbound, and outbound sessions to an endpoint, task, and
selected agent.
- Add low-trust guest intake in an explicitly selected sandbox and keep
ordinary approval rules.
- Submit task work through session-scoped tools and push approved
replies through Speko POST /v1/calls/{id}/messages.
- Distinguish provider acceptance from confirmed spoken playback.
Reconcile reports without downloading recordings.
- Add protocol, migration, persistence, provider, and
delivery-diagnostic tests.

## Verification

- Master 3389bea708 is integrated. Database/shared builds, server
typecheck, migration upgrade/replay checks, and 46
communication-guidance/provider/protocol tests passed.
- The final persistence suite passed 85 real-database tests. Five
focused native continuation/authorization regressions passed. Five
callback ownership/rotation tests passed after the last runtime repair.
- Review repairs cover reconnect ownership, current caller authority,
duplicate delivery, multipart answers, task-page answers, guest question
handling, and saved sandbox recovery.
- A real OpenCode/OpenRouter DeepSeek task ran in Daytona and saved a
correct disk-space answer. This did not establish live-call playback.
- Full local typecheck/build passed before the final repairs; focused
typechecks passed after them. Latest-head CI runs the repository gates.
- The separate integrated browser qualification passed two journeys and
failed browser answer retrieval. The feature remains experimental. The
report in #15729 records this failure and outstanding real-provider
acceptance.
- No lockfiles are included. Speko migrations follow master as
0324–0330.

## Risks

- Voice sessions add migrations and provider-dependent lifecycle
behavior. Migrations preserve existing chat and AI transports.
- Callback origins must be reachable HTTPS. Provider keys and signing
secrets remain vaulted.
- Unverified callers receive task-scoped low-trust authority. Caller ID
does not grant private task access.
- Outbound telephony incurs provider charges. Provider acceptance is not
proof that the caller heard an answer.
- Keep the feature behind the existing experimental chat setting until
the real browser, inbound, outbound, and delayed-result acceptance
matrix passes.

## Model Used

OpenAI Codex, GPT-6, with tool use and code execution. The exact serving
model ID and context window are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-09 21:54:11 -05:00

935 lines
39 KiB
TypeScript

import { photonAnswersMatch } from "../photon/interactions.js";
import { and, or, eq, lte, inArray, notExists, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
agentWakeupRequests,
chatActions,
chatConversations,
chatDeliveries,
chatEndpoints,
chatExternalPrincipals,
chatIdentityLinks,
chatVoiceSessions,
chatVoiceToolCalls,
chatVoiceReplies,
toolConnections,
chatMessageLinks,
chatPublications,
heartbeatRuns,
issueComments,
issueQuestionResponseDeliveries,
issueThreadInteractions,
} from "@paperclipai/db";
import type { AskUserQuestionsInteraction } from "@paperclipai/shared";
import {
parseChatQuestionFormSubmitTokenPayload,
validateChatQuestionFormSubmission,
} from "../chat-question-forms.js";
import { questionResponseDeliveryValues } from "../question-response-delivery.js";
import { voiceSessionStore, type VoiceTransaction, currentVoiceCredentialFingerprint } from "../voice/voice-session-store.js";
import { nativeSha256 } from "./canonical.js";
export const EXTERNAL_CHAT_QUESTION_RESPONSE_KEY =
"paperclipExternalChatQuestionResponse";
type Binding = {
companyId: string;
issueId: string;
runId: string;
agentId: string;
};
type LockMode = "blocking" | "nonblocking" | "read";
type Marker = {
schema: "paperclip.external_chat_question_response.v1";
interactionId: string;
responseDeliveryId: string;
sourceRunId: string;
sourceCommentId: string;
endpointId: string;
conversationId: string;
bindingSha256: string;
};
type ResolvedQuestionResponse = {
marker: Marker;
provider: typeof chatEndpoints.$inferSelect.provider;
/** Server-recorded current answer time, already bound into the marker hash. */
answeredAtMs: number;
authorizationContext: Record<string, unknown>;
/** Authoritative answered interactions, oldest first; not persisted answer text. */
interactionIds: string[];
};
// Keep authorization work/lock duration bounded. A ninth linked answer is
// denied, never admitted with a silently truncated ancestor history.
const MAX_QUESTION_RESPONSE_CHAIN_DEPTH = 8;
type Chain = {
runIds: Set<string>;
interactionIds: Set<string>;
deliveryIds: Set<string>;
actionIds: Set<string>;
};
const record = (value: unknown): Record<string, unknown> =>
value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
const ids = (value: unknown): string[] =>
Array.isArray(value)
? value.filter((id): id is string => typeof id === "string")
: [];
function completedQuestionFormMatchesInteraction(
interaction: AskUserQuestionsInteraction,
action: typeof chatActions.$inferSelect,
): boolean {
const payload = parseChatQuestionFormSubmitTokenPayload(action.payload);
const result = record(action.result);
const answers = record(interaction.result).answers;
const resolvedAt =
interaction.resolvedAt instanceof Date
? interaction.resolvedAt
: new Date(interaction.resolvedAt ?? Number.NaN);
if (
!payload ||
interaction.status !== "answered" ||
action.providerActionId !== payload.formActionId ||
payload.interactionId !== interaction.id ||
result.code !== "question_form_answered" ||
result.interactionId !== interaction.id ||
!Number.isFinite(resolvedAt.getTime()) ||
!Array.isArray(answers) ||
answers.length !== payload.fields.length
)
return false;
const answerByQuestionId = new Map<string, Record<string, unknown>>();
for (const value of answers) {
const answer = record(value);
const questionId =
typeof answer.questionId === "string" ? answer.questionId : null;
if (!questionId || answerByQuestionId.has(questionId)) return false;
answerByQuestionId.set(questionId, answer);
}
const values: Record<string, string> = {};
for (const field of payload.fields) {
const answer = answerByQuestionId.get(field.questionId);
if (!answer) return false;
const optionIds = ids(answer.optionIds);
if (
!Array.isArray(answer.optionIds) ||
optionIds.length !== answer.optionIds.length
)
return false;
if (field.kind === "single_select") {
if (
optionIds.length > 1 ||
(answer.otherText !== undefined &&
answer.otherText !== null &&
answer.otherText !== "")
)
return false;
const option = optionIds[0]
? field.options.find((candidate) => candidate.optionId === optionIds[0])
: null;
if (optionIds.length === 1 && !option) return false;
values[field.fieldId] = option?.value ?? "";
continue;
}
if (
optionIds.length !== 0 ||
(answer.otherText !== undefined &&
answer.otherText !== null &&
typeof answer.otherText !== "string")
)
return false;
values[field.fieldId] =
typeof answer.otherText === "string" ? answer.otherText : "";
}
const validation = validateChatQuestionFormSubmission({
callbackId: payload.formActionId,
privateMetadata: payload.formActionId,
interaction: { ...interaction, status: "pending" },
payload,
values,
// Recheck the durable answer time against the opaque token's lifetime;
// continuing later must not depend on the current wall clock.
now: resolvedAt,
});
return (
validation.ok &&
nativeSha256(validation.answers) === nativeSha256(answers)
);
}
/**
* Resolve a durable provider answer or GitHub's authenticated Board fallback,
* never a caller-supplied wake marker. This
* is routing/reading authority only: it never checks out a task or resolves a
* review. Callers retain their issue/run/actor and current destination checks.
*/
export async function resolveExternalChatQuestionResponse(
tx: Db,
binding: Binding,
contextSnapshot: unknown,
lockMode: LockMode = "blocking",
mint = false,
): Promise<ResolvedQuestionResponse | null> {
return resolveQuestionResponseChain(
tx,
binding,
contextSnapshot,
lockMode,
mint,
{
runIds: new Set(),
interactionIds: new Set(),
deliveryIds: new Set(),
actionIds: new Set(),
},
);
}
async function resolveQuestionResponseChain(
tx: Db,
binding: Binding,
contextSnapshot: unknown,
lockMode: LockMode,
mint: boolean,
chain: Chain,
): Promise<ResolvedQuestionResponse | null> {
if (
chain.runIds.has(binding.runId) ||
chain.runIds.size >= MAX_QUESTION_RESPONSE_CHAIN_DEPTH
)
return null;
chain.runIds.add(binding.runId);
const context = record(contextSnapshot);
if (
context.source !== "issue.interaction.respond" ||
context.issueId !== binding.issueId ||
context.wakeReason !== "issue_commented" ||
context.interactionKind !== "ask_user_questions" ||
context.interactionStatus !== "answered" ||
context.externalChatContinuation !== true ||
typeof context.interactionId !== "string" ||
typeof context.sourceRunId !== "string" ||
typeof context.sourceCommentId !== "string" ||
context.wakeCommentId !== context.sourceCommentId ||
ids(context.wakeCommentIds).length !== 1 ||
ids(context.wakeCommentIds)[0] !== context.sourceCommentId
)
return null;
const runQuery = tx
.select()
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.id, binding.runId),
eq(heartbeatRuns.companyId, binding.companyId),
eq(heartbeatRuns.agentId, binding.agentId),
),
);
const [run] = await (lockMode === "read"
? runQuery
: lockMode === "nonblocking"
? runQuery.for("update", { noWait: true })
: runQuery.for("update"));
if (!run?.wakeupRequestId) return null;
const admitted = record(run.contextSnapshot);
for (const key of [
"source",
"issueId",
"interactionId",
"interactionKind",
"interactionStatus",
"sourceRunId",
"sourceCommentId",
"externalChatContinuation",
"wakeCommentId",
"wakeCommentIds",
]) {
if (
nativeSha256(admitted[key] ?? null) !== nativeSha256(context[key] ?? null)
)
return null;
}
const query = tx
.select({
interaction: issueThreadInteractions,
delivery: issueQuestionResponseDeliveries,
wake: agentWakeupRequests,
source: heartbeatRuns,
})
.from(issueThreadInteractions)
.innerJoin(
issueQuestionResponseDeliveries,
and(
eq(
issueQuestionResponseDeliveries.interactionId,
issueThreadInteractions.id,
),
eq(
issueQuestionResponseDeliveries.companyId,
issueThreadInteractions.companyId,
),
eq(
issueQuestionResponseDeliveries.issueId,
issueThreadInteractions.issueId,
),
eq(
issueQuestionResponseDeliveries.sourceRunId,
issueThreadInteractions.sourceRunId,
),
),
)
.innerJoin(
heartbeatRuns,
and(
eq(heartbeatRuns.id, issueThreadInteractions.sourceRunId),
eq(heartbeatRuns.companyId, issueThreadInteractions.companyId),
),
)
.innerJoin(
agentWakeupRequests,
and(
eq(agentWakeupRequests.id, run.wakeupRequestId),
eq(agentWakeupRequests.companyId, issueThreadInteractions.companyId),
eq(agentWakeupRequests.agentId, binding.agentId),
eq(agentWakeupRequests.runId, binding.runId),
),
)
.where(
and(
eq(issueThreadInteractions.id, context.interactionId),
eq(issueThreadInteractions.companyId, binding.companyId),
eq(issueThreadInteractions.issueId, binding.issueId),
eq(issueThreadInteractions.sourceRunId, context.sourceRunId),
),
);
const rows = await (lockMode === "read"
? query
: lockMode === "nonblocking"
? query.for("update", { noWait: true })
: query.for("update"));
if (rows.length !== 1) return null;
const { interaction, delivery, source, wake } = rows[0]!;
if (
chain.interactionIds.has(interaction.id) ||
chain.deliveryIds.has(delivery.id) ||
chain.runIds.has(source.id)
)
return null;
chain.interactionIds.add(interaction.id);
chain.deliveryIds.add(delivery.id);
const sourceContext = record(source.contextSnapshot);
const sourceWake = record(sourceContext.paperclipWake);
// A follow-up question inherits no authority from its marker alone. Rebuild
// every parent proof from current durable state until the direct-chat root.
const parent =
sourceContext.source === "issue.interaction.respond"
? await resolveQuestionResponseChain(
tx,
{
companyId: binding.companyId,
agentId: binding.agentId,
issueId: binding.issueId,
runId: source.id,
},
sourceContext,
lockMode,
false,
chain,
)
: null;
if (sourceContext.source === "issue.interaction.respond" && !parent)
return null;
const provider =
parent?.provider ??
(
["slack", "github", "discord", "microsoft-teams", "telegram", "imessage-photon", "speko"] as const
).find(
(candidate) =>
sourceContext.source === `chat:${candidate}` ||
sourceContext.source === `chat:${candidate}:recovery`,
);
const sourceIds = ids(sourceContext.wakeCommentIds);
const wakePayload = record(wake.payload);
// Legacy CLI adapters create the same durable provider interactions through
// the API. Their task binding lives in the saved wake rather than nativeIssueId.
// Keep the source task, answer receipt, actor and provider proofs below intact.
const sourceTaskMatches = source.runtimeMode === "native"
? source.nativeIssueId === binding.issueId
: source.runtimeMode === "legacy"
&& source.nativeIssueId === null
&& record(sourceWake.issue).id === binding.issueId;
if (
!provider ||
source.agentId !== binding.agentId ||
(!sourceTaskMatches && !(provider === "speko" && source.runtimeMode === "legacy" && source.nativeIssueId === null)) ||
sourceContext.issueId !== binding.issueId ||
(source.status !== "succeeded" &&
!(
source.status === "cancelled" &&
source.errorCode === "external_chat_continuation"
)) ||
sourceWake.externalChatProvider !== provider ||
!(
(sourceContext.paperclipHarnessCheckedOut === true &&
sourceWake.checkedOutByHarness === true) ||
(sourceContext.paperclipExternalChatExecutionBound === true &&
sourceWake.externalChatExecutionBound === true)
) ||
sourceIds.length !== 1 ||
sourceIds[0] !== context.sourceCommentId ||
(parent !== null &&
parent.marker.sourceCommentId !== context.sourceCommentId) ||
interaction.kind !== "ask_user_questions" ||
interaction.status !== "answered" ||
interaction.createdByAgentId !== binding.agentId ||
!interaction.resolvedByUserId ||
(interaction.sourceCommentId !== null &&
interaction.sourceCommentId !== context.sourceCommentId) ||
interaction.resolvedByAgentId !== null ||
!interaction.resolvedAt ||
record(interaction.result).cancelled === true ||
wake.requestedByActorType !== "user" ||
wake.requestedByActorId !== interaction.resolvedByUserId ||
wake.idempotencyKey !== `question-response:${interaction.id}` ||
["skipped", "cancelled"].includes(wake.status) ||
wakePayload.issueId !== binding.issueId ||
wakePayload.interactionId !== interaction.id ||
wakePayload.sourceRunId !== source.id ||
wakePayload.sourceCommentId !== context.sourceCommentId ||
wakePayload.externalChatContinuation !== true ||
!(
(delivery.status === "fallback_queued" &&
delivery.deliveryMode === "wake_fallback" &&
delivery.targetRunId === binding.runId) ||
(delivery.status === "delivering" &&
delivery.targetRunId === null &&
delivery.deliveryMode === null)
)
)
return null;
let expected: ReturnType<typeof questionResponseDeliveryValues>;
try {
expected = questionResponseDeliveryValues(
interaction as unknown as AskUserQuestionsInteraction,
);
} catch {
return null;
}
if (
delivery.payloadSha256 !== expected.payloadSha256 ||
delivery.correlationId !== expected.correlationId
)
return null;
if (provider === "speko") {
// A model-selected interaction ID is insufficient. Require the exact
// canonical answer, a presented publication, and the durable signed-tool
// receipt committed atomically with that answer. No synthetic identity link.
const proofs = await tx.select({ receipt: chatVoiceToolCalls, session: chatVoiceSessions,
reply: chatVoiceReplies, publication: chatPublications, endpoint: chatEndpoints, connection: toolConnections,
conversation: chatConversations }).from(chatVoiceToolCalls)
.innerJoin(chatVoiceSessions, and(eq(chatVoiceSessions.id, chatVoiceToolCalls.sessionId), eq(chatVoiceSessions.companyId, binding.companyId)))
.innerJoin(chatVoiceReplies, and(eq(chatVoiceReplies.sessionId, chatVoiceSessions.id), eq(chatVoiceReplies.companyId, binding.companyId)))
.innerJoin(chatPublications, and(eq(chatPublications.id, chatVoiceReplies.publicationId), eq(chatPublications.companyId, binding.companyId)))
.innerJoin(chatEndpoints, and(eq(chatEndpoints.id, chatVoiceSessions.endpointId), eq(chatEndpoints.companyId, binding.companyId)))
.innerJoin(toolConnections, and(eq(toolConnections.id, chatEndpoints.connectionId), eq(toolConnections.companyId, binding.companyId)))
.innerJoin(chatConversations, and(eq(chatConversations.id, chatVoiceSessions.conversationId), eq(chatConversations.companyId, binding.companyId)))
.where(and(eq(chatVoiceToolCalls.companyId, binding.companyId), eq(chatVoiceToolCalls.tool, "answer_question"),
sql`${chatVoiceToolCalls.response}->>'interactionId' = ${interaction.id}`,
sql`${chatPublications.payload}->>'interactionId' = ${interaction.id}`,
eq(chatVoiceSessions.issueId, binding.issueId),
// Terminal acknowledgment publications use the same interaction ID.
// Only a question actually delivered before its answer can be proof.
lte(chatVoiceReplies.deliveredAt, interaction.resolvedAt)));
if (proofs.length === 0 && interaction.resolvedByRunId === null) {
// The authenticated task-page answer commits the canonical interaction
// and response delivery too. Bind it to the original voice requester;
// do not require hearing the question or fabricate a signed-tool receipt.
const origins = await tx.select({ session: chatVoiceSessions, endpoint: chatEndpoints,
connection: toolConnections, conversation: chatConversations, comment: issueComments,
inbound: chatDeliveries, link: chatMessageLinks }).from(chatMessageLinks)
.innerJoin(issueComments, and(eq(issueComments.id, chatMessageLinks.commentId), eq(issueComments.companyId, binding.companyId), eq(issueComments.issueId, binding.issueId)))
.innerJoin(chatDeliveries, and(eq(chatDeliveries.id, chatMessageLinks.deliveryId), eq(chatDeliveries.companyId, binding.companyId), eq(chatDeliveries.endpointId, chatMessageLinks.endpointId), eq(chatDeliveries.conversationId, chatMessageLinks.conversationId)))
.innerJoin(chatExternalPrincipals, and(eq(chatExternalPrincipals.id, chatDeliveries.principalId), eq(chatExternalPrincipals.companyId, binding.companyId), eq(chatExternalPrincipals.provider, "speko")))
.innerJoin(chatVoiceSessions, and(sql`${chatExternalPrincipals.externalId} = 'voice:' || ${chatVoiceSessions.id}::text`, eq(chatVoiceSessions.companyId, binding.companyId), eq(chatVoiceSessions.endpointId, chatMessageLinks.endpointId), eq(chatVoiceSessions.conversationId, chatMessageLinks.conversationId), eq(chatVoiceSessions.issueId, binding.issueId)))
.innerJoin(chatEndpoints, and(eq(chatEndpoints.id, chatVoiceSessions.endpointId), eq(chatEndpoints.companyId, binding.companyId)))
.innerJoin(toolConnections, and(eq(toolConnections.id, chatEndpoints.connectionId), eq(toolConnections.companyId, binding.companyId)))
.innerJoin(chatConversations, and(eq(chatConversations.id, chatVoiceSessions.conversationId), eq(chatConversations.companyId, binding.companyId), eq(chatConversations.endpointId, chatEndpoints.id)))
.where(and(eq(chatMessageLinks.companyId, binding.companyId), eq(chatMessageLinks.commentId, context.sourceCommentId), eq(chatMessageLinks.direction, "inbound")));
if (origins.length !== 1) return null;
const { session, endpoint, connection, conversation, comment, inbound, link } = origins[0]!;
if (comment.deletedAt || inbound.state !== "processed" || comment.authorUserId !== interaction.resolvedByUserId
|| session.callerId !== interaction.resolvedByUserId || !["member", "instance_admin", "local_board"].includes(session.callerAuthority)
|| session.assignedAgentId !== binding.agentId || endpoint.assignedAgentId !== binding.agentId
|| endpoint.provider !== "speko" || !["active", "verifying"].includes(endpoint.status)
|| !connection.enabled || connection.status !== "active" || connection.transport !== "voice"
|| session.generation !== record(endpoint.setup).runtimeGeneration
|| session.credentialFingerprint !== await currentVoiceCredentialFingerprint(tx, binding.companyId, connection.credentialSecretRefs)
|| !["active", "waiting"].includes(conversation.state) || conversation.issueId !== binding.issueId
|| (parent && (parent.marker.endpointId !== endpoint.id || parent.marker.conversationId !== conversation.id))) return null;
try {
const store = voiceSessionStore(tx, { allowLocalBoard: session.callerAuthority === "local_board" });
await store.authorizePrincipal(tx as unknown as VoiceTransaction, binding.companyId, endpoint.id, `voice:${session.id}`);
} catch (error) {
if (error && typeof error === "object" && "code" in error && error.code === "55P03") throw error;
return null;
}
const marker: Marker = {
schema: "paperclip.external_chat_question_response.v1", interactionId: interaction.id,
responseDeliveryId: delivery.id, sourceRunId: source.id, sourceCommentId: context.sourceCommentId,
endpointId: endpoint.id, conversationId: conversation.id,
bindingSha256: nativeSha256({ binding: {companyId: binding.companyId, issueId: binding.issueId, runId: binding.runId, agentId: binding.agentId}, responseDeliveryId: delivery.id, payloadSha256: delivery.payloadSha256,
sourceRunId: source.id, sourceCommentId: context.sourceCommentId, sourceOrigin: sourceContext.source,
...(parent ? { sourceQuestionResponseSha256: parent.marker.bindingSha256 } : {}),
interaction: { id: interaction.id, payload: interaction.payload, result: interaction.result,
resolvedAt: interaction.resolvedAt.toISOString(), resolvedByUserId: interaction.resolvedByUserId },
boardResponse: { sessionId: session.id, callerId: session.callerId, generation: session.generation,
credentialFingerprint: session.credentialFingerprint, inboundId: inbound.id, linkId: link.id },
conversationGeneration: conversation.sessionGeneration, wakeId: wake.id }),
};
if (!mint && nativeSha256(record(context[EXTERNAL_CHAT_QUESTION_RESPONSE_KEY])) !== nativeSha256(marker)) return null;
return { marker, provider, answeredAtMs: interaction.resolvedAt.getTime(),
interactionIds: [...(parent?.interactionIds ?? []), interaction.id],
authorizationContext: { ...context, source: "chat:speko", paperclipHarnessCheckedOut: false, paperclipExternalChatExecutionBound: true } };
}
if (proofs.length !== 1) return null;
const proof = proofs[0]!;
const { receipt, session, reply, publication, endpoint, connection, conversation } = proof;
const [original] = await tx.select({ comment: issueComments, inbound: chatDeliveries, link: chatMessageLinks })
.from(chatMessageLinks).innerJoin(issueComments, eq(issueComments.id, chatMessageLinks.commentId))
.innerJoin(chatDeliveries, eq(chatDeliveries.id, chatMessageLinks.deliveryId))
.where(and(eq(chatMessageLinks.companyId, binding.companyId), eq(chatMessageLinks.endpointId, endpoint.id),
eq(chatMessageLinks.conversationId, conversation.id), eq(chatMessageLinks.direction, "inbound"),
eq(chatMessageLinks.commentId, context.sourceCommentId), eq(issueComments.companyId, binding.companyId),
eq(issueComments.issueId, binding.issueId), eq(chatDeliveries.companyId, binding.companyId)));
if (!original || original.comment.deletedAt || original.inbound.state !== "processed"
|| original.comment.authorUserId !== interaction.resolvedByUserId
|| session.callerId !== interaction.resolvedByUserId || !["member", "instance_admin", "local_board"].includes(session.callerAuthority)
|| session.assignedAgentId !== binding.agentId || endpoint.assignedAgentId !== binding.agentId
|| endpoint.provider !== "speko" || !["active", "verifying"].includes(endpoint.status)
|| !connection.enabled || connection.status !== "active" || connection.transport !== "voice"
|| session.generation !== record(endpoint.setup).runtimeGeneration
|| session.credentialFingerprint !== await currentVoiceCredentialFingerprint(tx, binding.companyId, connection.credentialSecretRefs)
|| !["active", "waiting"].includes(conversation.state) || conversation.issueId !== binding.issueId
|| publication.conversationId !== conversation.id || publication.endpointId !== endpoint.id || publication.issueId !== binding.issueId
|| publication.state !== "published" || !publication.providerMessageId || !reply.deliveredAt
|| reply.deliveredAt > interaction.resolvedAt || receipt.response.status !== "answered"
|| receipt.response.resultSha256 !== nativeSha256(interaction.result)
|| (parent && (parent.marker.endpointId !== endpoint.id || parent.marker.conversationId !== conversation.id))) return null;
const marker: Marker = {
schema: "paperclip.external_chat_question_response.v1", interactionId: interaction.id,
responseDeliveryId: delivery.id, sourceRunId: source.id, sourceCommentId: context.sourceCommentId,
endpointId: endpoint.id, conversationId: conversation.id,
bindingSha256: nativeSha256({ binding: { companyId: binding.companyId, issueId: binding.issueId, runId: binding.runId, agentId: binding.agentId }, responseDeliveryId: delivery.id, payloadSha256: delivery.payloadSha256,
sourceRunId: source.id, sourceCommentId: context.sourceCommentId, sourceOrigin: sourceContext.source,
...(parent ? { sourceQuestionResponseSha256: parent.marker.bindingSha256 } : {}),
interaction: { id: interaction.id, payload: interaction.payload, result: interaction.result,
resolvedAt: interaction.resolvedAt.toISOString(), resolvedByUserId: interaction.resolvedByUserId },
voiceResponse: { receiptId: receipt.id, fingerprint: receipt.fingerprint, response: receipt.response,
sessionId: session.id, callerId: session.callerId, generation: session.generation, credentialFingerprint: session.credentialFingerprint },
publicationId: publication.id, replyId: reply.id, inboundId: original.inbound.id, linkId: original.link.id,
conversationGeneration: conversation.sessionGeneration, wakeId: wake.id }),
};
if (!mint && nativeSha256(record(context[EXTERNAL_CHAT_QUESTION_RESPONSE_KEY])) !== nativeSha256(marker)) return null;
return { marker, provider, answeredAtMs: interaction.resolvedAt.getTime(),
interactionIds: [...(parent?.interactionIds ?? []), interaction.id],
authorizationContext: { ...context, source: "chat:speko", paperclipHarnessCheckedOut: false, paperclipExternalChatExecutionBound: true } };
}
const actionQuery = tx
.select({
action: chatActions,
publication: chatPublications,
link: chatMessageLinks,
inbound: chatDeliveries,
comment: issueComments,
conversation: chatConversations,
endpoint: chatEndpoints,
identity: chatIdentityLinks,
})
.from(chatActions)
.innerJoin(
chatPublications,
and(
sql`${chatActions.payload}->>'publicationId' = ${chatPublications.id}::text`,
eq(chatPublications.companyId, chatActions.companyId),
eq(chatPublications.endpointId, chatActions.endpointId),
eq(chatPublications.conversationId, chatActions.conversationId),
),
)
.innerJoin(
chatMessageLinks,
and(
eq(chatMessageLinks.companyId, chatActions.companyId),
eq(chatMessageLinks.endpointId, chatActions.endpointId),
eq(chatMessageLinks.conversationId, chatActions.conversationId),
eq(chatMessageLinks.commentId, context.sourceCommentId),
eq(chatMessageLinks.direction, "inbound"),
),
)
.innerJoin(
chatDeliveries,
and(
eq(chatDeliveries.id, chatMessageLinks.deliveryId),
eq(chatDeliveries.companyId, chatActions.companyId),
eq(chatDeliveries.endpointId, chatActions.endpointId),
eq(chatDeliveries.conversationId, chatActions.conversationId),
or(eq(chatActions.kind, "photon_interaction"), eq(chatDeliveries.principalId, chatActions.principalId)),
),
)
.innerJoin(
issueComments,
and(
eq(issueComments.id, chatMessageLinks.commentId),
eq(issueComments.companyId, chatActions.companyId),
eq(issueComments.issueId, binding.issueId),
),
)
.innerJoin(
chatConversations,
and(
eq(chatConversations.id, chatActions.conversationId),
eq(chatConversations.companyId, chatActions.companyId),
eq(chatConversations.endpointId, chatActions.endpointId),
),
)
.innerJoin(
chatEndpoints,
and(
eq(chatEndpoints.id, chatActions.endpointId),
eq(chatEndpoints.companyId, chatActions.companyId),
),
)
.innerJoin(
chatIdentityLinks,
and(
eq(chatIdentityLinks.companyId, chatActions.companyId),
eq(chatIdentityLinks.endpointId, chatActions.endpointId),
eq(chatIdentityLinks.principalId, chatActions.principalId),
),
)
.where(
and(
eq(chatActions.companyId, binding.companyId),
inArray(chatActions.kind, ["question_answer", "question_form_submit", "photon_interaction"]),
eq(chatActions.status, "processed"),
sql`${chatActions.payload}->>'interactionId' = ${interaction.id}`,
sql`${chatActions.result}->>'interactionId' = ${interaction.id}`,
),
);
// GitHub has no executable question callback. Its published link leads to
// the ordinary authenticated Board answer route, whose pending→answered CAS
// creates the response delivery above atomically. Do not fabricate a provider
// action: bind that distinct receipt to the original linked person instead.
const boardQuery = tx
.select({
action: sql<null>`null`,
publication: chatPublications,
link: chatMessageLinks,
inbound: chatDeliveries,
comment: issueComments,
conversation: chatConversations,
endpoint: chatEndpoints,
identity: chatIdentityLinks,
})
.from(chatPublications)
.innerJoin(
chatMessageLinks,
and(
eq(chatMessageLinks.companyId, chatPublications.companyId),
eq(chatMessageLinks.endpointId, chatPublications.endpointId),
eq(chatMessageLinks.conversationId, chatPublications.conversationId),
eq(chatMessageLinks.commentId, context.sourceCommentId),
eq(chatMessageLinks.direction, "inbound"),
),
)
.innerJoin(
chatDeliveries,
and(
eq(chatDeliveries.id, chatMessageLinks.deliveryId),
eq(chatDeliveries.companyId, chatMessageLinks.companyId),
eq(chatDeliveries.endpointId, chatMessageLinks.endpointId),
eq(chatDeliveries.conversationId, chatMessageLinks.conversationId),
),
)
.innerJoin(
issueComments,
and(
eq(issueComments.id, chatMessageLinks.commentId),
eq(issueComments.companyId, chatMessageLinks.companyId),
eq(issueComments.issueId, binding.issueId),
),
)
.innerJoin(
chatConversations,
and(
eq(chatConversations.id, chatPublications.conversationId),
eq(chatConversations.companyId, chatPublications.companyId),
eq(chatConversations.endpointId, chatPublications.endpointId),
),
)
.innerJoin(
chatEndpoints,
and(
eq(chatEndpoints.id, chatPublications.endpointId),
eq(chatEndpoints.companyId, chatPublications.companyId),
eq(chatEndpoints.provider, "github"),
),
)
.innerJoin(
chatIdentityLinks,
and(
eq(chatIdentityLinks.companyId, chatDeliveries.companyId),
eq(chatIdentityLinks.endpointId, chatDeliveries.endpointId),
eq(chatIdentityLinks.principalId, chatDeliveries.principalId),
),
)
.where(
and(
eq(chatPublications.companyId, binding.companyId),
eq(chatPublications.issueId, binding.issueId),
sql`${chatPublications.idempotencyKey} = 'interaction:' || ${interaction.id} || ':' || ${chatEndpoints.id}::text`,
notExists(
tx
.select({ id: chatActions.id })
.from(chatActions)
.where(
and(
eq(chatActions.companyId, binding.companyId),
sql`${chatActions.payload}->>'interactionId' = ${interaction.id}`,
inArray(chatActions.kind, [
"question_answer",
"question_form_submit",
]),
),
),
),
),
);
const evidenceQuery = provider === "github" ? boardQuery : actionQuery;
let expectedPrincipalId: string | null = null;
if (lockMode !== "read") {
// Identity changes take this advisory lock before their row lock. Resolve
// the candidate without row locks, acquire that same policy fence, then
// reread everything under locks. A changed candidate is never adopted.
const candidates = await evidenceQuery;
if (candidates.length !== 1 || !candidates[0]!.inbound.principalId)
return null;
expectedPrincipalId = candidates[0]!.inbound.principalId;
const key = `chat-identity:${binding.companyId}:${expectedPrincipalId}`;
if (lockMode === "blocking") {
await tx.execute(
sql`select pg_advisory_xact_lock(hashtextextended(${key}, 0))`,
);
} else {
const [lock] = await tx.execute(
sql`select pg_try_advisory_xact_lock(hashtextextended(${key}, 0)) as acquired`,
);
if (!lock?.acquired)
throw Object.assign(
new Error("paperclip_external_chat_wait_authorization_contended"),
{ code: "55P03" },
);
}
}
const actions = await (lockMode === "read"
? evidenceQuery
: lockMode === "nonblocking"
? evidenceQuery.for("update", { noWait: true })
: evidenceQuery.for("update"));
// One exact provider response, never whichever responder happened to be last.
if (
actions.length !== 1 ||
(expectedPrincipalId !== null &&
actions[0]!.inbound.principalId !== expectedPrincipalId)
)
return null;
const {
action,
publication,
link,
inbound,
comment,
conversation,
endpoint,
identity,
} = actions[0]!;
if (action) {
if (chain.actionIds.has(action.id)) return null;
chain.actionIds.add(action.id);
}
if (
parent &&
(parent.marker.endpointId !== endpoint.id ||
parent.marker.conversationId !== conversation.id)
)
return null;
const boardRuntime = record(record(inbound.normalizedEvent).runtimeContext);
const boardGeneration = record(endpoint.setup).runtimeGeneration;
if (!action) {
const card = record(record(publication.payload).card);
const links = Array.isArray(card.actions) ? card.actions.map(record) : [];
if (
provider !== "github" ||
interaction.resolvedByRunId !== null ||
publication.payload.progressState !== "waiting_for_input" ||
card.kind !== "question" ||
(card.actions !== undefined && !Array.isArray(card.actions)) ||
links.some((link) => link.type !== "link") ||
!publication.publishedAt ||
publication.publishedAt > interaction.resolvedAt ||
typeof boardGeneration !== "number" ||
!Number.isSafeInteger(boardGeneration) ||
boardGeneration < 0 ||
boardRuntime.generation !== boardGeneration ||
typeof boardRuntime.credentialFingerprint !== "string" ||
!/^[a-f0-9]{64}$/.test(boardRuntime.credentialFingerprint)
)
return null;
} else if (action.kind === "question_answer") {
const answers = record(interaction.result).answers;
const selectedAnswer = Array.isArray(answers)
? answers
.map(record)
.find((answer) => answer.questionId === action.payload.questionId)
: null;
const questions = record(interaction.payload).questions;
const selectedQuestion = Array.isArray(questions)
? questions
.map(record)
.find((question) => question.id === action.payload.questionId)
: null;
if (
record(action.result).interactionStatus !== "answered" ||
!selectedAnswer ||
!selectedQuestion ||
selectedQuestion.selectionMode !== "single" ||
ids(selectedAnswer.optionIds).length !== 1 ||
ids(selectedAnswer.optionIds)[0] !== action.payload.optionId ||
!Array.isArray(selectedQuestion.options) ||
!selectedQuestion.options.some(
(option) => record(option).id === action.payload.optionId,
)
)
return null;
} else if (action.kind === "photon_interaction") {
if (provider !== "imessage-photon" || !photonAnswersMatch(interaction as unknown as AskUserQuestionsInteraction, action.result) || action.payload.version !== 1 || action.payload.sessionGeneration !== conversation.sessionGeneration || typeof action.payload.expiresAt !== "string" || Date.parse(action.payload.expiresAt) <= interaction.resolvedAt.getTime() || !Number.isFinite(Date.parse(action.payload.expiresAt))) return null;
} else if (
!completedQuestionFormMatchesInteraction(
interaction as unknown as AskUserQuestionsInteraction,
action,
)
) {
return null;
}
if (
inbound.state !== "processed" ||
comment.deletedAt !== null ||
(action?.kind !== "photon_interaction" && comment.authorUserId !== interaction.resolvedByUserId) ||
conversation.issueId !== binding.issueId ||
!["active", "waiting"].includes(conversation.state) ||
endpoint.provider !== provider ||
endpoint.assignedAgentId !== binding.agentId ||
(endpoint.status !== "active" &&
!(provider === "imessage-photon" && endpoint.status === "verifying" && record(endpoint.setup).step === "test")) ||
publication.state !== "published" ||
!publication.providerMessageId ||
publication.issueId !== binding.issueId ||
record(publication.payload).interactionId !== interaction.id ||
identity.status !== "linked" ||
identity.paperclipUserId !== interaction.resolvedByUserId
)
return null;
const marker: Marker = {
schema: "paperclip.external_chat_question_response.v1",
interactionId: interaction.id,
responseDeliveryId: delivery.id,
sourceRunId: source.id,
sourceCommentId: comment.id,
endpointId: endpoint.id,
conversationId: conversation.id,
bindingSha256: nativeSha256({
binding: {
companyId: binding.companyId,
issueId: binding.issueId,
runId: binding.runId,
agentId: binding.agentId,
},
responseDeliveryId: delivery.id,
payloadSha256: delivery.payloadSha256,
sourceRunId: source.id,
sourceIds,
sourceOrigin: sourceContext.source,
// Omit this field for a direct-chat parent so deployed v1 proofs remain
// byte-compatible. A chained proof commits to every validated ancestor.
...(parent
? { sourceQuestionResponseSha256: parent.marker.bindingSha256 }
: {}),
interaction: {
id: interaction.id,
payload: interaction.payload,
result: interaction.result,
resolvedAt: interaction.resolvedAt.toISOString(),
resolvedByUserId: interaction.resolvedByUserId,
createdByAgentId: interaction.createdByAgentId,
},
wake: {
id: wake.id,
actor: wake.requestedByActorId,
payload: wake.payload,
},
...(action
? {
action: {
id: action.id,
principalId: action.principalId,
payload: action.payload,
result: action.result,
},
}
: {
boardResponse: {
schema: "paperclip.github_board_question_response.v1",
principalId: inbound.principalId,
runtimeGeneration: boardGeneration,
credentialFingerprint: boardRuntime.credentialFingerprint,
},
}),
publication: {
id: publication.id,
providerMessageId: publication.providerMessageId,
},
linkId: link.id,
inboundId: inbound.id,
comment: { id: comment.id, body: comment.body },
conversation: {
id: conversation.id,
generation: conversation.sessionGeneration,
},
endpointId: endpoint.id,
identityId: identity.id,
}),
};
if (
!mint &&
nativeSha256(record(context[EXTERNAL_CHAT_QUESTION_RESPONSE_KEY])) !==
nativeSha256(marker)
)
return null;
return {
marker,
provider,
answeredAtMs: interaction.resolvedAt.getTime(),
interactionIds: [...(parent?.interactionIds ?? []), interaction.id],
authorizationContext: {
...context,
source: `chat:${provider}`,
paperclipHarnessCheckedOut: false,
paperclipExternalChatExecutionBound: true,
},
};
}