mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents can work on tasks that start from an external chat, such as Slack. When the agent waits for the next chat message, the task shows "in review" with an external response wait. > - Issue lists and inbox attention show these waits. To show a wait, the server checks that the chat conversation is still authorized: the messages, the destination, the endpoint, and each person who wrote. > - The server did this check one time for each waiting task, and it started all the checks at the same time. A page with 19 waits sent 19 copies of the message-link statement at the same moment, each on its own pool connection, and then 19 copies of each policy read. > - On a hosted database, 19 identical copies of this statement were active at the same moment, about 670 ms each. They were 19 of the 20 active queries. > - This pull request reads the messages of all waits of a page with one statement and shares the policy reads between the waits. Each wait gets the same authorization result as before. > - The benefit is fewer statements and fewer connections for each issue page, so one page cannot fill the pool with copies of one read. ## Linked Issues or Issue Description Refs #20400 (shares wait reads between the callers of one request; this change removes the per-wait fan-out inside that shared read). Refs #18966 (reuses the prepared message-link statement; this change keeps that statement for single runs). **What happened** `listExternalReviewWaits` ran `authorizeChatConversationForBoundRun` one time for each waiting task, inside `Promise.all`. Each call sends the message-link statement, then a resource read, and three reads for each person. A page with N waits sent N copies of each statement at the same time, on up to N pool connections. **Expected behavior** One issue page sends one message-link statement for all of its waits. A resource or a person that is common to many waits is read one time. The authorization result of each wait does not change. **Steps to reproduce** 1. Put many tasks of one company in review with an external chat response wait. 2. Open the issue list or the inbox. 3. Look at the active statements of the database. One copy of the message-link statement runs for each wait. ## What Changed - `server/src/services/native-runtime/chat-attachment-reuse.ts`: - Split `authorizeChatConversationForBoundRun` into steps: read the bound provider and wake comments, read the message rows, and apply the policy to the rows (`authorizeBoundRunLinks`). The single-run path uses the same prepared statement, the same lock modes and the same policy code as before. - Move the message-link join and its policy filters to one query builder, `boundRunLinksQuery`. The single-run statement and the batch statement both use it. - Add `authorizeChatConversationsForBoundRuns(db, companyId, runs)`. It is read-only and takes no locks. It reads the messages of all runs with one statement, then gives each run only the rows that the single-run statement returns for that run (same issue, agent, provider and comments). The runs share each distinct resource and principal read. Each run gets its conversation, or `null` for a policy denial. Other errors are thrown, as before. - Add `isBoundChatAuthorizationDenial` for the three policy denial errors. - `server/src/services/native-runtime/external-review-wait.ts`: `listExternalReviewWaits` uses the batch authorization. The inherited GitHub review-wait reads do not change. - `server/src/__tests__/batched-chat-authorization.integration.test.ts`: new tests on real PostgreSQL. For each case, the batch result must equal the result of separate single-run reads. The cases include each denial: a missing or foreign comment, no comments, no execution marker, a wrong provider, agent or issue, a person without a link or without a membership, a viewer member, a revoked link, a disabled or unavailable destination, a paused endpoint, a closed conversation, an unprocessed delivery, a delivery without a person, and a verifying endpoint. A test also makes sure that one person's decision never applies to another person in the same batch, and that a batch sends one message-link statement. - `scripts/benchmark-chat-bound-authorization.ts`: a local benchmark that loads the pre-fix modules from a baseline commit, checks that both versions return the same waits, and measures them. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/batched-chat-authorization.integration.test.ts src/__tests__/prepared-chat-authorization.integration.test.ts` passes (10 tests). - Mutation check: when I removed the issue filter from the per-run row selection, or shared one principal decision for all people of an endpoint, the new tests failed (1 and 3 failures). - Related suites pass: `heartbeat-reviewed-chat-binding`, `external-chat-wait`, `preserved-native-response-wait`, `chat-attachment-reuse` and `external-review-wait` (392 tests). - `pnpm verify` asks for broader verification because the change adds a file under `scripts/`. Required CI supplies that coverage. - Benchmark: `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-chat-bound-authorization.ts --baseline-ref=f4337bb5fb` (the measured pre-fix commit) Benchmark fixture (synthetic, local PostgreSQL 18.1, pool of 10): one company, one Slack endpoint, 20 tasks waiting for a chat response from 5 people, 4 of those waits denied (the person is a suspended member), and 2,000 earlier conversations with 50,000 processed deliveries (6 KiB events, 419 MB). Both versions return the same 16 waits. 15 alternating pairs after warm-up: | One issue page (20 waits) | Before | After | | --- | --- | --- | | Statements sent | 101 | 18 | | Message-link statements | 20 | 1 | | Server time of the message-link statements | 0.50 ms (sum of 20) | 0.13 ms | | Page time, median (min to max) | 8.86 ms (6.94 to 12.05) | 4.05 ms (3.62 to 5.54) | | 8 pages at the same time, median (min to max) | 54.18 ms (48.36 to 89.24) | 14.99 ms (12.60 to 19.59) | ## Risks - The batch must give each wait the same result as a separate read. The batch statement uses the same join and policy filters. It then selects each run's rows with the equality filters of the single-run statement. The policy code is the same function for both paths. The tests compare both paths for each denial case. - Shared reads: the batch reads each resource and each person one time. All reads are plain reads with no locks, as before. A policy change that occurs during one page read can now apply to all waits of the page at the same time, instead of to some of them. - An invalid wake comment id in one wait makes the whole page read fail. This was already true: the failure of one wait rejected the whole `Promise.all`. - The single-run path (attachment reuse, status decisions, run preparation and others) keeps its prepared statement and lock modes. The order of the conditions in its WHERE clause changed; the conditions are the same. - Not proven: why each copy took about 670 ms on the hosted database. Locally one copy takes about 0.03 ms with the expected index plan. The likely causes are the burst of copies on a small compute and cold pages. Verify after deployment that the statement runs at most one time for each issue page request. - No migration and no schema change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1831 lines
64 KiB
TypeScript
1831 lines
64 KiB
TypeScript
import { notifyIssueRecovery } from "../issue-recovery-notifications.js";
|
|
import { createHash } from "node:crypto";
|
|
|
|
import { and, desc, eq, inArray, isNull, sql, type SQL } from "drizzle-orm";
|
|
import type { SelectedFields } from "drizzle-orm/pg-core";
|
|
import type { Db } from "@paperclipai/db";
|
|
import {
|
|
assets,
|
|
agents,
|
|
chatConversations,
|
|
chatDeliveries,
|
|
chatEndpointResources,
|
|
chatEndpoints,
|
|
chatExternalPrincipals,
|
|
chatIdentityLinks,
|
|
chatMessageLinks,
|
|
chatPublications,
|
|
companyMemberships,
|
|
heartbeatRuns,
|
|
issueAttachments,
|
|
issueComments,
|
|
issues,
|
|
issueWorkProducts,
|
|
runIdentityContexts,
|
|
} from "@paperclipai/db";
|
|
|
|
import {
|
|
isAllowedContentType,
|
|
MAX_ATTACHMENT_BYTES,
|
|
normalizeContentType,
|
|
} from "../../attachment-types.js";
|
|
import { getStorageService } from "../../storage/index.js";
|
|
import type { StorageService } from "../../storage/types.js";
|
|
import { loadConfig } from "../../config.js";
|
|
import { HttpError } from "../../errors.js";
|
|
import { voiceSessionStore, type VoiceTransaction } from "../voice/voice-session-store.js";
|
|
import { issueService } from "../issues.js";
|
|
import { readInheritedGitHubReviewWait } from "../chat-channels/github/review-wait.js";
|
|
import { boundExternalChatProvider } from "./external-chat-provider.js";
|
|
import { resolveExternalChatQuestionResponse } from "./external-chat-question-response.js";
|
|
import { githubRunReplyState } from "../chat-run-publications.js";
|
|
import { readPreservedNativeResultFinalization } from "./preserved-native-result-finalization.js";
|
|
|
|
// This boundary reads identity and wake authority, never launch renderings,
|
|
// saved provider profiles, or run output. Keep the nested wake proof narrow too.
|
|
const responseWaitContextKeys = ["source", "issueId", "wakeReason", "wakeCommentId", "wakeCommentIds",
|
|
"interactionId", "interactionKind", "interactionStatus", "sourceRunId", "sourceCommentId",
|
|
"externalChatContinuation", "paperclipExternalChatQuestionResponse", "paperclipHarnessCheckedOut",
|
|
"paperclipExternalChatExecutionBound"] as const;
|
|
const responseWaitRun = {
|
|
id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
|
|
nativeIssueId: heartbeatRuns.nativeIssueId, issueId: heartbeatRuns.issueId, status: heartbeatRuns.status,
|
|
responsibleUserId: heartbeatRuns.responsibleUserId, activeIdentityContextId: heartbeatRuns.activeIdentityContextId,
|
|
contextSnapshot: sql<Record<string, unknown> | null>`case when jsonb_typeof(${heartbeatRuns.contextSnapshot}) = 'object'
|
|
then (select coalesce(jsonb_object_agg(field.key, field.value), '{}'::jsonb)
|
|
from jsonb_each(${heartbeatRuns.contextSnapshot}) as field(key, value)
|
|
where field.key in (${sql.join(responseWaitContextKeys.map(key => sql`${key}`), sql`, `)}))
|
|
|| jsonb_build_object('paperclipWake', jsonb_build_object(
|
|
'checkedOutByHarness', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,checkedOutByHarness}',
|
|
'externalChatExecutionBound', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,externalChatExecutionBound}',
|
|
'externalChatProvider', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,externalChatProvider}',
|
|
'issue', jsonb_build_object('id', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,issue,id}'),
|
|
'commentIds', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,commentIds}'))
|
|
else null end`,
|
|
};
|
|
|
|
export const LIST_CHAT_ATTACHMENTS_TOOL_NAME = "list_chat_attachments";
|
|
export const REUSE_CHAT_ATTACHMENT_TOOL_NAME = "reuse_chat_attachment";
|
|
|
|
export const LIST_CHAT_ATTACHMENTS_TOOL_DEFINITION = Object.freeze({
|
|
name: LIST_CHAT_ATTACHMENTS_TOOL_NAME,
|
|
description:
|
|
"List metadata for files already received or delivered in this same external-chat conversation. Call with {} for the latest page (default 20), or set limit to an integer from 1 through 50. Page through nextCursor to find older files, or filter by an exact sourceCommentId when known. File bytes and private storage locations are never returned.",
|
|
inputSchema: {
|
|
type: "object",
|
|
properties: {
|
|
sourceCommentId: {
|
|
type: ["string", "null"],
|
|
description:
|
|
"Optional exact sourceCommentId returned by this tool on an earlier page.",
|
|
},
|
|
limit: { type: "integer", minimum: 1, maximum: 50, default: 20 },
|
|
cursor: {
|
|
type: ["string", "null"],
|
|
maxLength: 1024,
|
|
description: "Opaque nextCursor from the immediately preceding page.",
|
|
},
|
|
},
|
|
required: [],
|
|
additionalProperties: false,
|
|
},
|
|
annotations: {
|
|
semanticContract: "paperclip.server-chat-attachment-reuse.v1",
|
|
operationId: LIST_CHAT_ATTACHMENTS_TOOL_NAME,
|
|
version: 1,
|
|
exposure: "run_scoped",
|
|
requiredClaims: [],
|
|
},
|
|
});
|
|
|
|
export const REUSE_CHAT_ATTACHMENT_TOOL_DEFINITION = Object.freeze({
|
|
name: REUSE_CHAT_ATTACHMENT_TOOL_NAME,
|
|
description:
|
|
"Prepare one exact file previously received or delivered in this same external-chat conversation for the current response. All four arguments are required: copy sourceCommentId and attachmentId from list_chat_attachments, provide a nonempty title, and choose a nonempty idempotencyKey of at most 200 characters once for this reuse; send that same key unchanged on every retry. This queues a verified copy for final publication; it does not confirm provider delivery.",
|
|
inputSchema: {
|
|
type: "object",
|
|
properties: {
|
|
idempotencyKey: { type: "string", minLength: 1, maxLength: 200 },
|
|
sourceCommentId: { type: "string", format: "uuid" },
|
|
attachmentId: { type: "string", format: "uuid" },
|
|
title: { type: "string", minLength: 1, maxLength: 500 },
|
|
},
|
|
required: ["idempotencyKey", "sourceCommentId", "attachmentId", "title"],
|
|
additionalProperties: false,
|
|
},
|
|
annotations: {
|
|
semanticContract: "paperclip.server-chat-attachment-reuse.v1",
|
|
operationId: REUSE_CHAT_ATTACHMENT_TOOL_NAME,
|
|
version: 1,
|
|
exposure: "run_scoped",
|
|
requiredClaims: [],
|
|
},
|
|
});
|
|
|
|
export type ChatReuseBinding = {
|
|
companyId: string;
|
|
issueId: string;
|
|
runId: string;
|
|
agentId: string;
|
|
};
|
|
|
|
type AuthorizedConversation = {
|
|
conversationId: string;
|
|
endpointId: string;
|
|
};
|
|
|
|
export type ExternalChatResponseWaitAuthorization =
|
|
| "authorized"
|
|
| "revoked"
|
|
| "not_applicable";
|
|
|
|
type AuthorizationLockMode = "blocking" | "nonblocking" | "read";
|
|
|
|
type AuthorizationEndpoint = Pick<typeof chatEndpoints.$inferSelect,
|
|
"id" | "companyId" | "provider" | "providerAccountId" | "status" | "setup"
|
|
| "allowDirectMessages" | "allowGroupChats" | "allowUnlinkedPeople" | "sponsorUserId"
|
|
>;
|
|
type AuthorizationConversation = Pick<typeof chatConversations.$inferSelect,
|
|
"id" | "resourceId" | "isDirectMessage"
|
|
>;
|
|
type AuthorizationResource = Pick<typeof chatEndpointResources.$inferSelect,
|
|
"availability" | "type" | "enabled"
|
|
>;
|
|
|
|
class ExternalChatWaitAuthorizationContentionError extends Error {
|
|
constructor() {
|
|
super("paperclip_external_chat_wait_authorization_contended");
|
|
this.name = "ExternalChatWaitAuthorizationContentionError";
|
|
}
|
|
}
|
|
|
|
export function isExternalChatWaitAuthorizationContention(
|
|
error: unknown,
|
|
): boolean {
|
|
let current: unknown = error;
|
|
for (let depth = 0; depth < 4 && current; depth += 1) {
|
|
if (current instanceof ExternalChatWaitAuthorizationContentionError) {
|
|
return true;
|
|
}
|
|
const value = record(current);
|
|
if (value.code === "55P03") return true;
|
|
current = value.cause;
|
|
}
|
|
return false;
|
|
}
|
|
|
|
export type ChatAttachmentReuseSource = {
|
|
sourceCommentId: string;
|
|
attachmentId: string;
|
|
filename: string;
|
|
contentType: string;
|
|
byteSize: number;
|
|
sha256: string;
|
|
objectKey: string;
|
|
createdAt: Date;
|
|
};
|
|
|
|
export type ListedChatAttachment = Omit<
|
|
ChatAttachmentReuseSource,
|
|
"objectKey" | "createdAt"
|
|
> & {
|
|
createdAt: string;
|
|
contentAccess: "metadata_only";
|
|
};
|
|
|
|
export type ListedChatAttachmentPage = {
|
|
attachments: ListedChatAttachment[];
|
|
nextCursor: string | null;
|
|
complete: boolean;
|
|
};
|
|
|
|
type ListCursor = {
|
|
schema: "paperclip.chat-attachment-list-cursor.v1";
|
|
conversationId: string;
|
|
sourceCommentId: string | null;
|
|
createdAt: string;
|
|
attachmentId: string;
|
|
sourceCommentIdTieBreak: string;
|
|
};
|
|
|
|
export type PreparedReusedChatAttachment = {
|
|
result: {
|
|
commandId: string;
|
|
disposition: "applied";
|
|
stateRevision: number;
|
|
entityRefs: string[];
|
|
scheduledWakeIds: never[];
|
|
source: {
|
|
commentId: string;
|
|
attachmentId: string;
|
|
sha256: string;
|
|
};
|
|
prepared: {
|
|
attachmentId: string;
|
|
workProductId: string;
|
|
commentId: string;
|
|
filename: string;
|
|
byteSize: number;
|
|
sha256: string;
|
|
};
|
|
};
|
|
rollbackDefinitePreCommitFailure: (() => Promise<void>) | null;
|
|
};
|
|
|
|
function record(value: unknown): Record<string, unknown> {
|
|
return typeof value === "object" && value !== null && !Array.isArray(value)
|
|
? (value as Record<string, unknown>)
|
|
: {};
|
|
}
|
|
|
|
function wakeCommentIds(contextSnapshot: unknown): string[] {
|
|
const context = record(contextSnapshot);
|
|
const raw = context.wakeCommentIds;
|
|
if (!Array.isArray(raw)) return [];
|
|
return [
|
|
...new Set(
|
|
raw
|
|
.filter(
|
|
(value): value is string =>
|
|
typeof value === "string" && value.trim().length > 0,
|
|
)
|
|
.map((value) => value.trim()),
|
|
),
|
|
];
|
|
}
|
|
|
|
function publicationAttachmentIds(payload: unknown): string[] {
|
|
const raw = record(payload).attachmentIds;
|
|
return Array.isArray(raw)
|
|
? raw.filter((value): value is string => typeof value === "string")
|
|
: [];
|
|
}
|
|
|
|
function isUuid(value: unknown): value is string {
|
|
return (
|
|
typeof value === "string" &&
|
|
/^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/iu.test(
|
|
value,
|
|
)
|
|
);
|
|
}
|
|
|
|
function encodeListCursor(cursor: ListCursor): string {
|
|
return Buffer.from(JSON.stringify(cursor), "utf8").toString("base64url");
|
|
}
|
|
|
|
function decodeListCursor(
|
|
value: unknown,
|
|
conversationId: string,
|
|
sourceCommentId: string | null,
|
|
): ListCursor | null {
|
|
if (value === null || value === undefined) return null;
|
|
if (typeof value !== "string" || value.length === 0 || value.length > 1024) {
|
|
throw new Error("paperclip_runner_chat_attachment_cursor_invalid");
|
|
}
|
|
try {
|
|
const parsed = record(
|
|
JSON.parse(Buffer.from(value, "base64url").toString("utf8")),
|
|
);
|
|
const createdAt =
|
|
typeof parsed.createdAt === "string" ? parsed.createdAt : "";
|
|
const parsedDate = new Date(createdAt);
|
|
if (
|
|
parsed.schema !== "paperclip.chat-attachment-list-cursor.v1" ||
|
|
parsed.conversationId !== conversationId ||
|
|
(parsed.sourceCommentId ?? null) !== sourceCommentId ||
|
|
Number.isNaN(parsedDate.getTime()) ||
|
|
!isUuid(parsed.attachmentId) ||
|
|
!isUuid(parsed.sourceCommentIdTieBreak)
|
|
) {
|
|
throw new Error("invalid");
|
|
}
|
|
return parsed as ListCursor;
|
|
} catch {
|
|
throw new Error("paperclip_runner_chat_attachment_cursor_invalid");
|
|
}
|
|
}
|
|
|
|
function destinationAllowed(
|
|
endpoint: AuthorizationEndpoint,
|
|
conversation: AuthorizationConversation,
|
|
resource: AuthorizationResource | null,
|
|
): boolean {
|
|
if (conversation.isDirectMessage) return endpoint.allowDirectMessages;
|
|
if (!resource || resource.availability !== "available") return false;
|
|
if (
|
|
endpoint.provider === "microsoft-teams" &&
|
|
resource.type === "group_chat"
|
|
) {
|
|
return endpoint.allowGroupChats;
|
|
}
|
|
return resource.enabled;
|
|
}
|
|
|
|
function currentSetupTestAcceptsBoundDeliveries(
|
|
endpoint: AuthorizationEndpoint,
|
|
deliveries: Array<{
|
|
normalizedEvent: unknown;
|
|
processedAt: Date | null;
|
|
receivedAt: Date;
|
|
}>,
|
|
): boolean {
|
|
if (endpoint.status === "active") return true;
|
|
if (endpoint.status !== "verifying" || deliveries.length === 0) return false;
|
|
const setup = record(endpoint.setup);
|
|
const testStartedAt =
|
|
typeof setup.testStartedAt === "string"
|
|
? new Date(setup.testStartedAt)
|
|
: null;
|
|
const generation = setup.runtimeGeneration;
|
|
if (
|
|
setup.step !== "test" ||
|
|
!testStartedAt ||
|
|
Number.isNaN(testStartedAt.getTime()) ||
|
|
typeof generation !== "number" ||
|
|
!Number.isSafeInteger(generation) ||
|
|
generation < 0
|
|
) {
|
|
return false;
|
|
}
|
|
return deliveries.every((delivery) => {
|
|
const runtimeContext = record(
|
|
record(delivery.normalizedEvent).runtimeContext,
|
|
);
|
|
return (
|
|
delivery.receivedAt >= testStartedAt &&
|
|
delivery.processedAt !== null &&
|
|
delivery.processedAt >= testStartedAt &&
|
|
delivery.processedAt >= delivery.receivedAt &&
|
|
runtimeContext.generation === generation &&
|
|
typeof runtimeContext.credentialFingerprint === "string" &&
|
|
/^[a-f0-9]{64}$/u.test(runtimeContext.credentialFingerprint)
|
|
);
|
|
});
|
|
}
|
|
|
|
async function principalAuthorized(
|
|
tx: Db,
|
|
endpoint: AuthorizationEndpoint,
|
|
principalId: string,
|
|
lockMode: AuthorizationLockMode = "blocking",
|
|
): Promise<boolean> {
|
|
if (lockMode === "blocking") {
|
|
await tx.execute(
|
|
sql`select pg_advisory_xact_lock(hashtextextended(${`chat-identity:${endpoint.companyId}:${principalId}`}, 0))`,
|
|
);
|
|
} else if (lockMode === "nonblocking") {
|
|
const [lock] = (await tx.execute(
|
|
sql`select pg_try_advisory_xact_lock(hashtextextended(${`chat-identity:${endpoint.companyId}:${principalId}`}, 0)) as acquired`,
|
|
)) as unknown as Array<{ acquired: boolean }>;
|
|
if (!lock?.acquired) {
|
|
throw new ExternalChatWaitAuthorizationContentionError();
|
|
}
|
|
}
|
|
const principalQuery = tx
|
|
.select({ id: chatExternalPrincipals.id, externalId: chatExternalPrincipals.externalId })
|
|
.from(chatExternalPrincipals)
|
|
.where(
|
|
and(
|
|
eq(chatExternalPrincipals.id, principalId),
|
|
eq(chatExternalPrincipals.companyId, endpoint.companyId),
|
|
eq(chatExternalPrincipals.provider, endpoint.provider),
|
|
eq(
|
|
chatExternalPrincipals.providerAccountId,
|
|
endpoint.providerAccountId ?? "",
|
|
),
|
|
),
|
|
);
|
|
const [principal] = await (lockMode === "read"
|
|
? principalQuery.limit(1)
|
|
: lockMode === "nonblocking"
|
|
? principalQuery.for("update", { noWait: true }).limit(1)
|
|
: principalQuery.for("update").limit(1));
|
|
if (!principal) return false;
|
|
if (endpoint.provider === "speko") {
|
|
try {
|
|
await voiceSessionStore(tx, { allowLocalBoard: loadConfig().deploymentMode === "local_trusted", nonblockingAuthority: lockMode === "nonblocking" })
|
|
.authorizePrincipal(tx as unknown as VoiceTransaction, endpoint.companyId, endpoint.id, principal.externalId);
|
|
return true;
|
|
} catch (error) {
|
|
if (error instanceof HttpError && [403, 404].includes(error.status)) return false;
|
|
throw error;
|
|
}
|
|
}
|
|
const linkQuery = tx
|
|
.select({
|
|
status: chatIdentityLinks.status,
|
|
userId: chatIdentityLinks.paperclipUserId,
|
|
})
|
|
.from(chatIdentityLinks)
|
|
.where(
|
|
and(
|
|
eq(chatIdentityLinks.companyId, endpoint.companyId),
|
|
eq(chatIdentityLinks.endpointId, endpoint.id),
|
|
eq(chatIdentityLinks.principalId, principalId),
|
|
),
|
|
);
|
|
const [link] = await (lockMode === "read"
|
|
? linkQuery.limit(1)
|
|
: lockMode === "nonblocking"
|
|
? linkQuery.for("update", { noWait: true }).limit(1)
|
|
: linkQuery.for("update").limit(1));
|
|
const activeMember = async (userId: string) => {
|
|
const membershipQuery = tx
|
|
.select({
|
|
status: companyMemberships.status,
|
|
role: companyMemberships.membershipRole,
|
|
})
|
|
.from(companyMemberships)
|
|
.where(
|
|
and(
|
|
eq(companyMemberships.companyId, endpoint.companyId),
|
|
eq(companyMemberships.principalType, "user"),
|
|
eq(companyMemberships.principalId, userId),
|
|
),
|
|
);
|
|
const [membership] = await (lockMode === "read"
|
|
? membershipQuery.limit(1)
|
|
: lockMode === "nonblocking"
|
|
? membershipQuery.for("update", { noWait: true }).limit(1)
|
|
: membershipQuery.for("update").limit(1));
|
|
return membership?.status === "active" && membership.role !== "viewer";
|
|
};
|
|
if (link?.status === "linked") {
|
|
return Boolean(link.userId && (await activeMember(link.userId)));
|
|
}
|
|
if (!endpoint.allowUnlinkedPeople) return false;
|
|
return endpoint.sponsorUserId ? activeMember(endpoint.sponsorUserId) : true;
|
|
}
|
|
|
|
const boundRunLinkFields = {
|
|
commentId: chatMessageLinks.commentId,
|
|
conversation: {
|
|
id: chatConversations.id,
|
|
resourceId: chatConversations.resourceId,
|
|
isDirectMessage: chatConversations.isDirectMessage,
|
|
},
|
|
endpoint: {
|
|
id: chatEndpoints.id,
|
|
companyId: chatEndpoints.companyId,
|
|
provider: chatEndpoints.provider,
|
|
providerAccountId: chatEndpoints.providerAccountId,
|
|
status: chatEndpoints.status,
|
|
allowDirectMessages: chatEndpoints.allowDirectMessages,
|
|
allowGroupChats: chatEndpoints.allowGroupChats,
|
|
allowUnlinkedPeople: chatEndpoints.allowUnlinkedPeople,
|
|
sponsorUserId: chatEndpoints.sponsorUserId,
|
|
setup: sql<AuthorizationEndpoint["setup"]>`jsonb_build_object(
|
|
'step', ${chatEndpoints.setup}->'step',
|
|
'testStartedAt', ${chatEndpoints.setup}->'testStartedAt',
|
|
'runtimeGeneration', ${chatEndpoints.setup}->'runtimeGeneration'
|
|
)`,
|
|
},
|
|
principalId: chatDeliveries.principalId,
|
|
normalizedEvent: sql<unknown>`jsonb_build_object('runtimeContext', jsonb_build_object(
|
|
'generation', ${chatDeliveries.normalizedEvent}->'runtimeContext'->'generation',
|
|
'credentialFingerprint', ${chatDeliveries.normalizedEvent}->'runtimeContext'->'credentialFingerprint'
|
|
))`,
|
|
processedAt: chatDeliveries.processedAt,
|
|
receivedAt: chatDeliveries.receivedAt,
|
|
};
|
|
|
|
/** The one join and policy filter for the inbound messages of a bound run.
|
|
* The caller adds the run's company, comments, issue, provider and agent. */
|
|
function boundRunLinksQuery<T extends SelectedFields>(
|
|
tx: Db,
|
|
extraFields: T,
|
|
bindingFilter: SQL,
|
|
) {
|
|
return tx
|
|
.select({ ...boundRunLinkFields, ...extraFields })
|
|
.from(chatMessageLinks)
|
|
.innerJoin(
|
|
chatDeliveries,
|
|
and(
|
|
eq(chatDeliveries.id, chatMessageLinks.deliveryId),
|
|
eq(chatDeliveries.companyId, chatMessageLinks.companyId),
|
|
eq(chatDeliveries.endpointId, chatMessageLinks.endpointId),
|
|
eq(chatDeliveries.conversationId, chatMessageLinks.conversationId),
|
|
),
|
|
)
|
|
.innerJoin(
|
|
chatConversations,
|
|
and(
|
|
eq(chatConversations.id, chatMessageLinks.conversationId),
|
|
eq(chatConversations.companyId, chatMessageLinks.companyId),
|
|
eq(chatConversations.endpointId, chatMessageLinks.endpointId),
|
|
),
|
|
)
|
|
.innerJoin(
|
|
chatEndpoints,
|
|
and(
|
|
eq(chatEndpoints.id, chatConversations.endpointId),
|
|
eq(chatEndpoints.companyId, chatConversations.companyId),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
bindingFilter,
|
|
eq(chatMessageLinks.direction, "inbound"),
|
|
eq(chatDeliveries.state, "processed"),
|
|
inArray(chatConversations.state, ["active", "waiting"]),
|
|
inArray(chatEndpoints.status, ["active", "verifying"]),
|
|
),
|
|
);
|
|
}
|
|
|
|
function prepareBoundRunLinks(tx: Db, lockMode: AuthorizationLockMode) {
|
|
const linksQuery = boundRunLinksQuery(tx, {}, and(
|
|
eq(chatMessageLinks.companyId, sql.placeholder("companyId")),
|
|
sql`${chatMessageLinks.commentId} = any(${sql.placeholder("commentIds")}::uuid[])`,
|
|
eq(chatConversations.issueId, sql.placeholder("issueId")),
|
|
eq(chatEndpoints.provider, sql.placeholder("provider")),
|
|
eq(chatEndpoints.assignedAgentId, sql.placeholder("agentId")),
|
|
)!);
|
|
return (lockMode === "read"
|
|
? linksQuery
|
|
: lockMode === "nonblocking"
|
|
? linksQuery.for("update", { noWait: true })
|
|
: linksQuery.for("update")).prepare("chat_bound_run_links");
|
|
}
|
|
|
|
type BoundRunLink = Awaited<ReturnType<ReturnType<typeof prepareBoundRunLinks>["execute"]>>[number];
|
|
|
|
const boundRunLinksTemplates = new WeakMap<
|
|
Db,
|
|
Partial<Record<AuthorizationLockMode, ReturnType<typeof prepareBoundRunLinks>>>
|
|
>();
|
|
|
|
function boundRunLinksTemplate(tx: Db, lockMode: AuthorizationLockMode) {
|
|
let templates = boundRunLinksTemplates.get(tx);
|
|
if (!templates) {
|
|
templates = {};
|
|
boundRunLinksTemplates.set(tx, templates);
|
|
}
|
|
return templates[lockMode] ??= prepareBoundRunLinks(tx, lockMode);
|
|
}
|
|
|
|
const boundChatAuthorizationDenials = new Set([
|
|
"paperclip_runner_chat_attachment_binding_denied",
|
|
"paperclip_runner_chat_attachment_destination_denied",
|
|
"paperclip_runner_chat_attachment_principal_denied",
|
|
]);
|
|
|
|
/** True for the three policy denials of bound-run chat authorization. Database
|
|
* errors, lock contention and malformed input are not denials. */
|
|
export function isBoundChatAuthorizationDenial(error: unknown): boolean {
|
|
return error instanceof Error && boundChatAuthorizationDenials.has(error.message);
|
|
}
|
|
|
|
/** The provider and wake comments that the saved context of a run binds. */
|
|
async function boundRunAuthorizationRequest(
|
|
tx: Db,
|
|
binding: ChatReuseBinding,
|
|
contextSnapshot: unknown,
|
|
lockMode: AuthorizationLockMode,
|
|
): Promise<{ provider: string; commentIds: string[] }> {
|
|
let context = record(contextSnapshot);
|
|
if (context.source === "issue.interaction.respond") {
|
|
const answer = await resolveExternalChatQuestionResponse(tx, binding, context, lockMode);
|
|
if (!answer) throw new Error("paperclip_runner_chat_attachment_binding_denied");
|
|
context = answer.authorizationContext;
|
|
}
|
|
const source = typeof context.source === "string" ? context.source : "";
|
|
const provider = boundExternalChatProvider(source);
|
|
const commentIds = wakeCommentIds(context);
|
|
if (
|
|
!provider ||
|
|
(context.paperclipHarnessCheckedOut !== true &&
|
|
context.paperclipExternalChatExecutionBound !== true) ||
|
|
commentIds.length === 0
|
|
) {
|
|
throw new Error("paperclip_runner_chat_attachment_binding_denied");
|
|
}
|
|
return { provider, commentIds };
|
|
}
|
|
|
|
type BoundRunPolicyReads = {
|
|
resource: (endpoint: AuthorizationEndpoint, resourceId: string) => Promise<AuthorizationResource | null>;
|
|
principal: (endpoint: AuthorizationEndpoint, principalId: string) => Promise<boolean>;
|
|
};
|
|
|
|
function boundRunPolicyReads(
|
|
tx: Db,
|
|
companyId: string,
|
|
lockMode: AuthorizationLockMode,
|
|
): BoundRunPolicyReads {
|
|
return {
|
|
resource: (endpoint, resourceId) => {
|
|
const query = tx
|
|
.select({
|
|
availability: chatEndpointResources.availability,
|
|
type: chatEndpointResources.type,
|
|
enabled: chatEndpointResources.enabled,
|
|
})
|
|
.from(chatEndpointResources)
|
|
.where(
|
|
and(
|
|
eq(chatEndpointResources.id, resourceId),
|
|
eq(chatEndpointResources.companyId, companyId),
|
|
eq(chatEndpointResources.endpointId, endpoint.id),
|
|
),
|
|
);
|
|
return lockMode === "read"
|
|
? query.then((rows) => rows[0] ?? null)
|
|
: lockMode === "nonblocking"
|
|
? query
|
|
.for("update", { noWait: true })
|
|
.then((rows) => rows[0] ?? null)
|
|
: query.for("update").then((rows) => rows[0] ?? null);
|
|
},
|
|
principal: (endpoint, principalId) =>
|
|
principalAuthorized(tx, endpoint, principalId, lockMode),
|
|
};
|
|
}
|
|
|
|
/** Read-only policy reads that the runs of one batch share. The batch reads
|
|
* each distinct resource and principal one time. */
|
|
function sharedBoundRunPolicyReads(reads: BoundRunPolicyReads): BoundRunPolicyReads {
|
|
const resources = new Map<string, Promise<AuthorizationResource | null>>();
|
|
const principals = new Map<string, Promise<boolean>>();
|
|
const once = <T>(cache: Map<string, Promise<T>>, key: string, read: () => Promise<T>) => {
|
|
let value = cache.get(key);
|
|
if (!value) {
|
|
value = read();
|
|
cache.set(key, value);
|
|
}
|
|
return value;
|
|
};
|
|
return {
|
|
resource: (endpoint, resourceId) =>
|
|
once(resources, `${endpoint.id}:${resourceId}`, () => reads.resource(endpoint, resourceId)),
|
|
principal: (endpoint, principalId) =>
|
|
once(principals, `${endpoint.id}:${principalId}`, () => reads.principal(endpoint, principalId)),
|
|
};
|
|
}
|
|
|
|
/** Apply the bound-run policy to the inbound message rows of one run. */
|
|
async function authorizeBoundRunLinks(
|
|
links: BoundRunLink[],
|
|
commentIds: string[],
|
|
reads: BoundRunPolicyReads,
|
|
): Promise<AuthorizedConversation> {
|
|
const linkedCommentIds = new Set(links.map((row) => row.commentId));
|
|
const conversationIds = new Set(links.map((row) => row.conversation.id));
|
|
const endpointIds = new Set(links.map((row) => row.endpoint.id));
|
|
if (
|
|
links.length === 0 ||
|
|
conversationIds.size !== 1 ||
|
|
endpointIds.size !== 1 ||
|
|
!commentIds.every((id) => linkedCommentIds.has(id)) ||
|
|
links.some((row) => !row.principalId)
|
|
) {
|
|
throw new Error("paperclip_runner_chat_attachment_binding_denied");
|
|
}
|
|
const conversation = links[0]!.conversation;
|
|
const endpoint = links[0]!.endpoint;
|
|
if (!currentSetupTestAcceptsBoundDeliveries(endpoint, links)) {
|
|
throw new Error("paperclip_runner_chat_attachment_binding_denied");
|
|
}
|
|
const resource = conversation.resourceId
|
|
? await reads.resource(endpoint, conversation.resourceId)
|
|
: null;
|
|
if (!destinationAllowed(endpoint, conversation, resource)) {
|
|
throw new Error("paperclip_runner_chat_attachment_destination_denied");
|
|
}
|
|
for (const principalId of new Set(links.map((row) => row.principalId!))) {
|
|
if (!(await reads.principal(endpoint, principalId))) {
|
|
throw new Error("paperclip_runner_chat_attachment_principal_denied");
|
|
}
|
|
}
|
|
return { conversationId: conversation.id, endpointId: endpoint.id };
|
|
}
|
|
|
|
/** Caller must independently verify the current run/issue execution owner. */
|
|
export async function authorizeChatConversationForBoundRun(
|
|
tx: Db,
|
|
binding: ChatReuseBinding,
|
|
contextSnapshot: unknown,
|
|
lockMode: AuthorizationLockMode = "blocking",
|
|
): Promise<AuthorizedConversation> {
|
|
const { provider, commentIds } = await boundRunAuthorizationRequest(
|
|
tx, binding, contextSnapshot, lockMode,
|
|
);
|
|
const links = await boundRunLinksTemplate(tx, lockMode).execute({
|
|
companyId: binding.companyId,
|
|
issueId: binding.issueId,
|
|
agentId: binding.agentId,
|
|
provider,
|
|
commentIds,
|
|
});
|
|
return authorizeBoundRunLinks(
|
|
links, commentIds, boundRunPolicyReads(tx, binding.companyId, lockMode),
|
|
);
|
|
}
|
|
|
|
/**
|
|
* The read mode of `authorizeChatConversationForBoundRun` for several runs of
|
|
* one company. One statement reads the inbound messages of all the runs, and
|
|
* the runs share their resource and principal reads. Each run gets the same
|
|
* outcome as a separate read-mode call: its conversation, or null for a
|
|
* policy denial. Other errors are thrown. This takes no locks.
|
|
*/
|
|
export async function authorizeChatConversationsForBoundRuns(
|
|
db: Db,
|
|
companyId: string,
|
|
runs: Array<{ binding: ChatReuseBinding; contextSnapshot: unknown }>,
|
|
): Promise<Array<AuthorizedConversation | null>> {
|
|
if (runs.some((run) => run.binding.companyId !== companyId)) {
|
|
throw new Error("paperclip_runner_chat_authorization_batch_company_mismatch");
|
|
}
|
|
const deniedAsNull = (error: unknown) => {
|
|
if (isBoundChatAuthorizationDenial(error)) return null;
|
|
throw error;
|
|
};
|
|
const requests = await Promise.all(runs.map((run) =>
|
|
boundRunAuthorizationRequest(db, run.binding, run.contextSnapshot, "read").catch(deniedAsNull)));
|
|
const pending = runs.flatMap((run, index) => {
|
|
const request = requests[index];
|
|
return request ? [{ binding: run.binding, ...request }] : [];
|
|
});
|
|
if (pending.length === 0) return runs.map(() => null);
|
|
const unique = (values: string[]) => [...new Set(values)];
|
|
const rows = await boundRunLinksQuery(db, {
|
|
issueId: chatConversations.issueId,
|
|
assignedAgentId: chatEndpoints.assignedAgentId,
|
|
}, and(
|
|
eq(chatMessageLinks.companyId, companyId),
|
|
sql`${chatMessageLinks.commentId} = any(${sql.param(unique(pending.flatMap((run) => run.commentIds)))}::uuid[])`,
|
|
sql`${chatConversations.issueId} = any(${sql.param(unique(pending.map((run) => run.binding.issueId)))}::uuid[])`,
|
|
sql`${chatEndpoints.provider} = any(${sql.param(unique(pending.map((run) => run.provider)))}::text[])`,
|
|
sql`${chatEndpoints.assignedAgentId} = any(${sql.param(unique(pending.map((run) => run.binding.agentId)))}::uuid[])`,
|
|
)!);
|
|
const reads = sharedBoundRunPolicyReads(boundRunPolicyReads(db, companyId, "read"));
|
|
return Promise.all(requests.map(async (request, index) => {
|
|
if (!request) return null;
|
|
const { binding } = runs[index]!;
|
|
const comments = new Set(request.commentIds);
|
|
// The equality filters of the single-run statement, for this run's rows.
|
|
const links = rows.filter((row) =>
|
|
row.issueId === binding.issueId &&
|
|
row.assignedAgentId === binding.agentId &&
|
|
row.endpoint.provider === request.provider &&
|
|
row.commentId !== null && comments.has(row.commentId));
|
|
return authorizeBoundRunLinks(links, request.commentIds, reads).catch(deniedAsNull);
|
|
}));
|
|
}
|
|
|
|
function externalChatWaitCandidate(
|
|
contextSnapshot: unknown,
|
|
binding: ChatReuseBinding,
|
|
): { provider: string; commentIds: string[] } | null {
|
|
const context = record(contextSnapshot);
|
|
const source = typeof context.source === "string" ? context.source : "";
|
|
const provider = boundExternalChatProvider(source);
|
|
const commentIds = wakeCommentIds(context);
|
|
const wake = record(context.paperclipWake);
|
|
const wakeIssue = record(wake.issue);
|
|
const payloadCommentIds = Array.isArray(wake.commentIds)
|
|
? wake.commentIds.filter(
|
|
(value): value is string =>
|
|
typeof value === "string" && value.trim().length > 0,
|
|
)
|
|
: [];
|
|
if (
|
|
!provider ||
|
|
!(
|
|
(context.paperclipHarnessCheckedOut === true &&
|
|
wake.checkedOutByHarness === true) ||
|
|
(context.paperclipExternalChatExecutionBound === true &&
|
|
wake.externalChatExecutionBound === true)
|
|
) ||
|
|
commentIds.length === 0 ||
|
|
wake.externalChatProvider !== provider ||
|
|
wakeIssue.id !== binding.issueId ||
|
|
payloadCommentIds.length !== commentIds.length ||
|
|
payloadCommentIds.some((id, index) => id !== commentIds[index])
|
|
) {
|
|
return null;
|
|
}
|
|
return { provider, commentIds };
|
|
}
|
|
|
|
/**
|
|
* Authorize a no-work external-chat wait from the same durable boundary used
|
|
* by historical file reuse. The nested wake payload proves that this was a
|
|
* server-built chat turn; current destination and principal policy remain
|
|
* authoritative at the instant the status decision commits.
|
|
*/
|
|
export async function resolveExternalChatResponseWaitAuthorizationInTransaction(
|
|
tx: Db,
|
|
binding: ChatReuseBinding,
|
|
lockMode: AuthorizationLockMode = "blocking",
|
|
): Promise<ExternalChatResponseWaitAuthorization> {
|
|
// Preserve the native mutation lock order (issue -> run -> actor). The
|
|
// status committer already holds the issue lock; PostgreSQL treats this
|
|
// repeated lock as a no-op.
|
|
const issueQuery = tx
|
|
.select({
|
|
assigneeAgentId: issues.assigneeAgentId,
|
|
executionRunId: issues.executionRunId,
|
|
})
|
|
.from(issues)
|
|
.where(
|
|
and(
|
|
eq(issues.id, binding.issueId),
|
|
eq(issues.companyId, binding.companyId),
|
|
),
|
|
);
|
|
const [issue] = await (lockMode === "read"
|
|
? issueQuery.limit(1)
|
|
: lockMode === "nonblocking"
|
|
? issueQuery.for("update", { noWait: true }).limit(1)
|
|
: issueQuery.for("update").limit(1));
|
|
|
|
const runQuery = tx
|
|
.select(responseWaitRun)
|
|
.from(heartbeatRuns)
|
|
.where(
|
|
and(
|
|
eq(heartbeatRuns.id, binding.runId),
|
|
eq(heartbeatRuns.companyId, binding.companyId),
|
|
eq(heartbeatRuns.agentId, binding.agentId),
|
|
eq(heartbeatRuns.nativeIssueId, binding.issueId),
|
|
eq(heartbeatRuns.runtimeMode, "native"),
|
|
),
|
|
);
|
|
const [run] = await (lockMode === "read"
|
|
? runQuery.limit(1)
|
|
: lockMode === "nonblocking"
|
|
? runQuery.for("update", { noWait: true }).limit(1)
|
|
: runQuery.for("update").limit(1));
|
|
const answerContext =
|
|
run && record(run.contextSnapshot).source === "issue.interaction.respond"
|
|
? await resolveExternalChatQuestionResponse(
|
|
tx,
|
|
binding,
|
|
run.contextSnapshot,
|
|
lockMode,
|
|
)
|
|
: null;
|
|
if (
|
|
run &&
|
|
record(run.contextSnapshot).source === "issue.interaction.respond" &&
|
|
record(run.contextSnapshot).paperclipExternalChatQuestionResponse &&
|
|
!answerContext
|
|
)
|
|
return "revoked";
|
|
const directContext = answerContext?.authorizationContext ?? run?.contextSnapshot;
|
|
const inheritedReviewWait = run && !externalChatWaitCandidate(directContext, binding)
|
|
? await readInheritedGitHubReviewWait(tx, run) : null;
|
|
if (inheritedReviewWait?.kind === "revoked") return "revoked";
|
|
const authorizationContext = inheritedReviewWait?.context ?? directContext;
|
|
const candidate = run ? externalChatWaitCandidate(authorizationContext, binding) : null;
|
|
if (!candidate) return "not_applicable";
|
|
|
|
const actorQuery = tx
|
|
.select({ status: agents.status })
|
|
.from(agents)
|
|
.where(
|
|
and(
|
|
eq(agents.id, binding.agentId),
|
|
eq(agents.companyId, binding.companyId),
|
|
),
|
|
);
|
|
const [actor] = await (lockMode === "read"
|
|
? actorQuery.limit(1)
|
|
: lockMode === "nonblocking"
|
|
? actorQuery.for("update", { noWait: true }).limit(1)
|
|
: actorQuery.for("update").limit(1));
|
|
const preserved = run && run.status !== "running"
|
|
? await readPreservedNativeResultFinalization(tx, binding) : null;
|
|
if (
|
|
!issue ||
|
|
!actor ||
|
|
(!preserved && run.status !== "running") ||
|
|
issue.assigneeAgentId !== binding.agentId ||
|
|
(!preserved && issue.executionRunId !== binding.runId) ||
|
|
["paused", "terminated", "pending_approval", "error"].includes(
|
|
actor.status,
|
|
)
|
|
) {
|
|
return "revoked";
|
|
}
|
|
|
|
const links = await tx
|
|
.select({
|
|
commentId: chatMessageLinks.commentId,
|
|
conversationId: chatConversations.id,
|
|
endpointId: chatEndpoints.id,
|
|
principalId: chatDeliveries.principalId,
|
|
})
|
|
.from(chatMessageLinks)
|
|
.innerJoin(
|
|
chatDeliveries,
|
|
and(
|
|
eq(chatDeliveries.id, chatMessageLinks.deliveryId),
|
|
eq(chatDeliveries.companyId, chatMessageLinks.companyId),
|
|
eq(chatDeliveries.endpointId, chatMessageLinks.endpointId),
|
|
eq(chatDeliveries.conversationId, chatMessageLinks.conversationId),
|
|
),
|
|
)
|
|
.innerJoin(
|
|
chatConversations,
|
|
and(
|
|
eq(chatConversations.id, chatMessageLinks.conversationId),
|
|
eq(chatConversations.companyId, chatMessageLinks.companyId),
|
|
eq(chatConversations.endpointId, chatMessageLinks.endpointId),
|
|
),
|
|
)
|
|
.innerJoin(
|
|
chatEndpoints,
|
|
and(
|
|
eq(chatEndpoints.id, chatConversations.endpointId),
|
|
eq(chatEndpoints.companyId, chatConversations.companyId),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
eq(chatMessageLinks.companyId, binding.companyId),
|
|
eq(chatMessageLinks.direction, "inbound"),
|
|
inArray(chatMessageLinks.commentId, candidate.commentIds),
|
|
eq(chatDeliveries.state, "processed"),
|
|
eq(chatConversations.issueId, binding.issueId),
|
|
eq(
|
|
chatEndpoints.provider,
|
|
candidate.provider as typeof chatEndpoints.$inferSelect.provider,
|
|
),
|
|
eq(chatEndpoints.assignedAgentId, binding.agentId),
|
|
),
|
|
);
|
|
const linkedCommentIds = new Set(links.map((row) => row.commentId));
|
|
if (
|
|
links.length === 0 ||
|
|
new Set(links.map((row) => row.conversationId)).size !== 1 ||
|
|
new Set(links.map((row) => row.endpointId)).size !== 1 ||
|
|
links.some((row) => !row.principalId) ||
|
|
!candidate.commentIds.every((id) => linkedCommentIds.has(id))
|
|
) {
|
|
return "revoked";
|
|
}
|
|
try {
|
|
await authorizeChatConversationForBoundRun(
|
|
tx,
|
|
binding,
|
|
authorizationContext,
|
|
lockMode,
|
|
);
|
|
return "authorized";
|
|
} catch (error) {
|
|
if (
|
|
error instanceof Error &&
|
|
[
|
|
"paperclip_runner_chat_attachment_binding_denied",
|
|
"paperclip_runner_chat_attachment_destination_denied",
|
|
"paperclip_runner_chat_attachment_principal_denied",
|
|
].includes(error.message)
|
|
) {
|
|
return "revoked";
|
|
}
|
|
throw error;
|
|
}
|
|
}
|
|
|
|
export async function resolveExternalChatResponseWaitAuthorization(input: {
|
|
db: Db;
|
|
binding: ChatReuseBinding;
|
|
}): Promise<ExternalChatResponseWaitAuthorization> {
|
|
return input.db.transaction((tx) =>
|
|
resolveExternalChatResponseWaitAuthorizationInTransaction(
|
|
tx as unknown as Db,
|
|
input.binding,
|
|
"read",
|
|
),
|
|
);
|
|
}
|
|
|
|
const publishedGitHubReviewWaitKey = "github-review-response:";
|
|
|
|
export function isPublishedGitHubReviewCompletionContinuation(value: unknown): boolean {
|
|
const continuation = record(value);
|
|
return continuation.kind === "response_wake" && typeof continuation.idempotencyKey === "string" &&
|
|
continuation.idempotencyKey.startsWith(publishedGitHubReviewWaitKey);
|
|
}
|
|
|
|
/** Project a wait from the same saved publication and current authority used
|
|
* by finalization. Inherited waits need an accepted tree resume or a confirmed
|
|
* reply from this run to its exact current source. An arbitrary old review does
|
|
* not assign an unanswered new turn a wait. */
|
|
export async function readPublishedGitHubReviewCompletionContinuationInTransaction(
|
|
tx: Db, binding: ChatReuseBinding, lockMode: AuthorizationLockMode = "read",
|
|
) {
|
|
const [run] = await tx.select(responseWaitRun).from(heartbeatRuns).where(and(
|
|
eq(heartbeatRuns.id, binding.runId), eq(heartbeatRuns.companyId, binding.companyId),
|
|
eq(heartbeatRuns.agentId, binding.agentId), eq(heartbeatRuns.nativeIssueId, binding.issueId),
|
|
eq(heartbeatRuns.runtimeMode, "native"),
|
|
)).limit(1);
|
|
if (!run || run.status !== "running" && !await readPreservedNativeResultFinalization(tx, binding)) return null;
|
|
const wait = await readInheritedGitHubReviewWait(tx, run);
|
|
if (wait?.kind !== "published") return null;
|
|
if (wait.reviewRunId !== run.id) {
|
|
const [identity] = await tx.select({ cause: runIdentityContexts.cause, parentContextId: runIdentityContexts.parentContextId })
|
|
.from(runIdentityContexts).where(and(eq(runIdentityContexts.id, run.activeIdentityContextId!),
|
|
eq(runIdentityContexts.companyId, run.companyId), eq(runIdentityContexts.runId, run.id),
|
|
eq(runIdentityContexts.status, "accepted"))).limit(1);
|
|
const resumed = run.contextSnapshot?.source === "issue.tree_resume" && identity?.cause === "issue_tree_resumed" &&
|
|
Boolean(identity.parentContextId);
|
|
if (!resumed && !(run.contextSnapshot?.source === "chat:github" &&
|
|
await githubRunReplyState(tx, { ...binding, endpointId: wait.endpointId,
|
|
sourceCommentId: wait.context.wakeCommentId, requireSettled: true }) === "confirmed")) return null;
|
|
}
|
|
if (await resolveExternalChatResponseWaitAuthorizationInTransaction(tx, binding, lockMode) !== "authorized") return null;
|
|
return { kind: "response_wake" as const, summary: "Wait for the next authorized GitHub response to the published review.",
|
|
idempotencyKey: `${publishedGitHubReviewWaitKey}${run.id}:${wait.reviewId}:${wait.headSha}` };
|
|
}
|
|
|
|
export async function readPublishedGitHubReviewCompletionContinuation(input: { db: Db; binding: ChatReuseBinding }) {
|
|
return input.db.transaction(tx => readPublishedGitHubReviewCompletionContinuationInTransaction(tx as unknown as Db, input.binding));
|
|
}
|
|
|
|
async function sourceLineageExists(
|
|
tx: Db,
|
|
binding: ChatReuseBinding,
|
|
conversation: AuthorizedConversation,
|
|
source: Pick<
|
|
ChatAttachmentReuseSource,
|
|
"sourceCommentId" | "attachmentId"
|
|
> & {
|
|
parentCommentId: string;
|
|
},
|
|
): Promise<boolean> {
|
|
const [inbound] = await tx
|
|
.select({ providerEventId: chatDeliveries.providerEventId })
|
|
.from(chatMessageLinks)
|
|
.innerJoin(
|
|
chatDeliveries,
|
|
and(
|
|
eq(chatDeliveries.id, chatMessageLinks.deliveryId),
|
|
eq(chatDeliveries.companyId, chatMessageLinks.companyId),
|
|
eq(chatDeliveries.endpointId, chatMessageLinks.endpointId),
|
|
eq(chatDeliveries.conversationId, chatMessageLinks.conversationId),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
eq(chatMessageLinks.companyId, binding.companyId),
|
|
eq(chatMessageLinks.endpointId, conversation.endpointId),
|
|
eq(chatMessageLinks.conversationId, conversation.conversationId),
|
|
eq(chatMessageLinks.direction, "inbound"),
|
|
eq(chatMessageLinks.commentId, source.sourceCommentId),
|
|
eq(chatDeliveries.state, "processed"),
|
|
),
|
|
)
|
|
.limit(1);
|
|
if (inbound && source.parentCommentId === source.sourceCommentId) {
|
|
const [lifecycle] = await tx
|
|
.select({ id: chatDeliveries.id })
|
|
.from(chatDeliveries)
|
|
.where(
|
|
and(
|
|
eq(chatDeliveries.companyId, binding.companyId),
|
|
eq(chatDeliveries.endpointId, conversation.endpointId),
|
|
eq(chatDeliveries.conversationId, conversation.conversationId),
|
|
inArray(chatDeliveries.eventKind, [
|
|
"message_updated",
|
|
"message_deleted",
|
|
"message_restored",
|
|
]),
|
|
eq(chatDeliveries.state, "processed"),
|
|
sql`${chatDeliveries.normalizedEvent}->'message'->>'targetProviderEventId' = ${inbound.providerEventId}`,
|
|
),
|
|
)
|
|
.limit(1);
|
|
return !lifecycle;
|
|
}
|
|
const publications = await tx
|
|
.select({
|
|
payload: chatPublications.payload,
|
|
providerMessageId: chatPublications.providerMessageId,
|
|
publishedAt: chatPublications.publishedAt,
|
|
})
|
|
.from(chatPublications)
|
|
.where(
|
|
and(
|
|
eq(chatPublications.companyId, binding.companyId),
|
|
eq(chatPublications.endpointId, conversation.endpointId),
|
|
eq(chatPublications.conversationId, conversation.conversationId),
|
|
eq(chatPublications.issueId, binding.issueId),
|
|
eq(chatPublications.commentId, source.sourceCommentId),
|
|
eq(chatPublications.state, "published"),
|
|
),
|
|
);
|
|
const matchingPublications = publications.filter(
|
|
(row) =>
|
|
row.providerMessageId !== null &&
|
|
row.publishedAt !== null &&
|
|
publicationAttachmentIds(row.payload).includes(source.attachmentId),
|
|
);
|
|
if (matchingPublications.length === 0) return false;
|
|
for (const publication of matchingPublications) {
|
|
const [lifecycle] = await tx
|
|
.select({ id: chatDeliveries.id })
|
|
.from(chatDeliveries)
|
|
.where(
|
|
and(
|
|
eq(chatDeliveries.companyId, binding.companyId),
|
|
eq(chatDeliveries.endpointId, conversation.endpointId),
|
|
eq(chatDeliveries.conversationId, conversation.conversationId),
|
|
inArray(chatDeliveries.eventKind, [
|
|
"message_updated",
|
|
"message_deleted",
|
|
"message_restored",
|
|
]),
|
|
eq(chatDeliveries.state, "processed"),
|
|
eq(
|
|
sql<string>`${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId'`,
|
|
publication.providerMessageId!,
|
|
),
|
|
),
|
|
)
|
|
.limit(1);
|
|
if (!lifecycle) return true;
|
|
}
|
|
return false;
|
|
}
|
|
|
|
const HISTORICAL_ATTACHMENT_FILE_EXTENSIONS: ReadonlyMap<string, string> =
|
|
new Map([
|
|
["image/png", "png"],
|
|
["image/jpeg", "jpg"],
|
|
["image/jpg", "jpg"],
|
|
["image/webp", "webp"],
|
|
["image/gif", "gif"],
|
|
["audio/mpeg", "mp3"],
|
|
["audio/mp4", "m4a"],
|
|
["audio/ogg", "ogg"],
|
|
["audio/wav", "wav"],
|
|
["audio/webm", "webm"],
|
|
["video/mp4", "mp4"],
|
|
["video/webm", "webm"],
|
|
["video/quicktime", "mov"],
|
|
["video/x-m4v", "m4v"],
|
|
["application/pdf", "pdf"],
|
|
["application/zip", "zip"],
|
|
["application/json", "json"],
|
|
["text/plain", "txt"],
|
|
["text/markdown", "md"],
|
|
["text/csv", "csv"],
|
|
["text/html", "html"],
|
|
]);
|
|
|
|
async function loadSource(
|
|
tx: Db,
|
|
binding: ChatReuseBinding,
|
|
conversation: AuthorizedConversation,
|
|
sourceCommentId: string,
|
|
attachmentId: string,
|
|
allowEmpty = false,
|
|
): Promise<ChatAttachmentReuseSource> {
|
|
const [sourceComment] = await tx
|
|
.select({ id: issueComments.id })
|
|
.from(issueComments)
|
|
.where(
|
|
and(
|
|
eq(issueComments.id, sourceCommentId),
|
|
eq(issueComments.companyId, binding.companyId),
|
|
eq(issueComments.issueId, binding.issueId),
|
|
isNull(issueComments.deletedAt),
|
|
),
|
|
)
|
|
.for("update")
|
|
.limit(1);
|
|
if (!sourceComment) {
|
|
throw new Error("paperclip_runner_chat_attachment_source_denied");
|
|
}
|
|
const [row] = await tx
|
|
.select({
|
|
attachmentId: issueAttachments.id,
|
|
parentCommentId: issueComments.id,
|
|
filename: assets.originalFilename,
|
|
contentType: assets.contentType,
|
|
byteSize: assets.byteSize,
|
|
sha256: assets.sha256,
|
|
objectKey: assets.objectKey,
|
|
createdAt: issueAttachments.createdAt,
|
|
})
|
|
.from(issueAttachments)
|
|
.innerJoin(
|
|
assets,
|
|
and(
|
|
eq(assets.id, issueAttachments.assetId),
|
|
eq(assets.companyId, binding.companyId),
|
|
),
|
|
)
|
|
.innerJoin(
|
|
issueComments,
|
|
and(
|
|
eq(issueComments.id, issueAttachments.issueCommentId),
|
|
eq(issueComments.companyId, binding.companyId),
|
|
eq(issueComments.issueId, binding.issueId),
|
|
isNull(issueComments.deletedAt),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
eq(issueAttachments.id, attachmentId),
|
|
eq(issueAttachments.companyId, binding.companyId),
|
|
eq(issueAttachments.issueId, binding.issueId),
|
|
),
|
|
)
|
|
.for("update")
|
|
.limit(1);
|
|
if (
|
|
!row ||
|
|
row.byteSize < (allowEmpty ? 0 : 1) ||
|
|
row.byteSize > MAX_ATTACHMENT_BYTES ||
|
|
!isAllowedContentType(normalizeContentType(row.contentType)) ||
|
|
!/^[a-f0-9]{64}$/iu.test(row.sha256)
|
|
) {
|
|
throw new Error("paperclip_runner_chat_attachment_source_denied");
|
|
}
|
|
const sourceWithParent = {
|
|
sourceCommentId,
|
|
attachmentId: row.attachmentId,
|
|
parentCommentId: row.parentCommentId,
|
|
// Provider photos/voice messages legitimately have no original filename.
|
|
// Derive display metadata from the already-validated MIME and opaque id;
|
|
// never infer a storage path or change the selected source bytes.
|
|
filename: row.filename?.trim()
|
|
? row.filename
|
|
: `attachment-${row.attachmentId}.${HISTORICAL_ATTACHMENT_FILE_EXTENSIONS.get(normalizeContentType(row.contentType)) ?? "bin"}`,
|
|
contentType: normalizeContentType(row.contentType),
|
|
byteSize: row.byteSize,
|
|
sha256: row.sha256,
|
|
objectKey: row.objectKey,
|
|
createdAt: row.createdAt,
|
|
};
|
|
if (
|
|
!(await sourceLineageExists(tx, binding, conversation, sourceWithParent))
|
|
) {
|
|
throw new Error("paperclip_runner_chat_attachment_source_denied");
|
|
}
|
|
const { parentCommentId: _parentCommentId, ...source } = sourceWithParent;
|
|
return source;
|
|
}
|
|
|
|
export async function listAuthorizedChatAttachments(input: {
|
|
db: Db;
|
|
binding: ChatReuseBinding;
|
|
sourceCommentId?: string | null;
|
|
cursor?: string | null;
|
|
limit: number;
|
|
}): Promise<ListedChatAttachmentPage> {
|
|
return input.db.transaction(async (tx) => {
|
|
const [current] = await tx
|
|
.select({ run: heartbeatRuns, actorStatus: agents.status })
|
|
.from(heartbeatRuns)
|
|
.innerJoin(
|
|
issues,
|
|
and(
|
|
eq(issues.id, input.binding.issueId),
|
|
eq(issues.companyId, heartbeatRuns.companyId),
|
|
),
|
|
)
|
|
.innerJoin(
|
|
agents,
|
|
and(
|
|
eq(agents.id, input.binding.agentId),
|
|
eq(agents.companyId, heartbeatRuns.companyId),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
eq(heartbeatRuns.id, input.binding.runId),
|
|
eq(heartbeatRuns.companyId, input.binding.companyId),
|
|
eq(heartbeatRuns.agentId, input.binding.agentId),
|
|
eq(heartbeatRuns.nativeIssueId, input.binding.issueId),
|
|
eq(heartbeatRuns.runtimeMode, "native"),
|
|
eq(heartbeatRuns.status, "running"),
|
|
eq(issues.assigneeAgentId, input.binding.agentId),
|
|
eq(issues.executionRunId, input.binding.runId),
|
|
),
|
|
)
|
|
.for("update")
|
|
.limit(1);
|
|
if (
|
|
!current ||
|
|
["paused", "terminated", "pending_approval", "error"].includes(
|
|
current.actorStatus,
|
|
)
|
|
) {
|
|
throw new Error("paperclip_runner_tool_binding_not_authorized");
|
|
}
|
|
const conversation = await authorizeChatConversationForBoundRun(
|
|
tx as unknown as Db,
|
|
input.binding,
|
|
current.run.contextSnapshot,
|
|
);
|
|
const sourceFilter = input.sourceCommentId ?? null;
|
|
const cursor = decodeListCursor(
|
|
input.cursor,
|
|
conversation.conversationId,
|
|
sourceFilter,
|
|
);
|
|
const rawCandidates = await tx.execute(sql<{
|
|
attachment_id: string;
|
|
source_comment_id: string;
|
|
created_at: Date | string;
|
|
}>`
|
|
with candidate_pairs as (
|
|
select
|
|
attachment.id as attachment_id,
|
|
link.comment_id as source_comment_id,
|
|
attachment.created_at
|
|
from issue_attachments attachment
|
|
join issue_comments parent_comment
|
|
on parent_comment.id = attachment.issue_comment_id
|
|
and parent_comment.company_id = ${input.binding.companyId}::uuid
|
|
and parent_comment.issue_id = ${input.binding.issueId}::uuid
|
|
and parent_comment.deleted_at is null
|
|
join chat_message_links link
|
|
on link.company_id = ${input.binding.companyId}::uuid
|
|
and link.endpoint_id = ${conversation.endpointId}::uuid
|
|
and link.conversation_id = ${conversation.conversationId}::uuid
|
|
and link.direction = 'inbound'
|
|
and link.comment_id = attachment.issue_comment_id
|
|
join chat_deliveries delivery
|
|
on delivery.id = link.delivery_id
|
|
and delivery.company_id = link.company_id
|
|
and delivery.endpoint_id = link.endpoint_id
|
|
and delivery.conversation_id = link.conversation_id
|
|
and delivery.state = 'processed'
|
|
where attachment.company_id = ${input.binding.companyId}::uuid
|
|
and attachment.issue_id = ${input.binding.issueId}::uuid
|
|
and (${sourceFilter}::uuid is null or link.comment_id = ${sourceFilter}::uuid)
|
|
and not exists (
|
|
select 1
|
|
from chat_deliveries lifecycle
|
|
where lifecycle.company_id = ${input.binding.companyId}::uuid
|
|
and lifecycle.endpoint_id = ${conversation.endpointId}::uuid
|
|
and lifecycle.conversation_id = ${conversation.conversationId}::uuid
|
|
and lifecycle.state = 'processed'
|
|
and lifecycle.event_kind in ('message_updated', 'message_deleted', 'message_restored')
|
|
and lifecycle.normalized_event->'message'->>'targetProviderEventId' = delivery.provider_event_id
|
|
)
|
|
|
|
union all
|
|
|
|
select
|
|
attachment.id as attachment_id,
|
|
publication.comment_id as source_comment_id,
|
|
attachment.created_at
|
|
from issue_attachments attachment
|
|
join issue_comments parent_comment
|
|
on parent_comment.id = attachment.issue_comment_id
|
|
and parent_comment.company_id = ${input.binding.companyId}::uuid
|
|
and parent_comment.issue_id = ${input.binding.issueId}::uuid
|
|
and parent_comment.deleted_at is null
|
|
join chat_publications publication
|
|
on publication.company_id = ${input.binding.companyId}::uuid
|
|
and publication.endpoint_id = ${conversation.endpointId}::uuid
|
|
and publication.conversation_id = ${conversation.conversationId}::uuid
|
|
and publication.issue_id = ${input.binding.issueId}::uuid
|
|
and publication.state = 'published'
|
|
and publication.published_at is not null
|
|
and publication.provider_message_id is not null
|
|
and publication.comment_id is not null
|
|
and publication.payload->'attachmentIds' ? attachment.id::text
|
|
join issue_comments source_comment
|
|
on source_comment.id = publication.comment_id
|
|
and source_comment.company_id = publication.company_id
|
|
and source_comment.issue_id = publication.issue_id
|
|
and source_comment.deleted_at is null
|
|
where attachment.company_id = ${input.binding.companyId}::uuid
|
|
and attachment.issue_id = ${input.binding.issueId}::uuid
|
|
and (${sourceFilter}::uuid is null or publication.comment_id = ${sourceFilter}::uuid)
|
|
and not exists (
|
|
select 1
|
|
from chat_deliveries lifecycle
|
|
where lifecycle.company_id = ${input.binding.companyId}::uuid
|
|
and lifecycle.endpoint_id = ${conversation.endpointId}::uuid
|
|
and lifecycle.conversation_id = ${conversation.conversationId}::uuid
|
|
and lifecycle.state = 'processed'
|
|
and lifecycle.event_kind in ('message_updated', 'message_deleted', 'message_restored')
|
|
and lifecycle.normalized_event->'message'->>'providerMessageId' = publication.provider_message_id
|
|
)
|
|
), candidates as (
|
|
select distinct on (attachment_id)
|
|
attachment_id,
|
|
source_comment_id,
|
|
created_at
|
|
from candidate_pairs
|
|
order by attachment_id, created_at desc, source_comment_id desc
|
|
)
|
|
select attachment_id, source_comment_id, created_at
|
|
from candidates
|
|
where (
|
|
${cursor?.createdAt ?? null}::timestamptz is null
|
|
or (created_at, attachment_id, source_comment_id) <
|
|
(${cursor?.createdAt ?? null}::timestamptz, ${cursor?.attachmentId ?? null}::uuid, ${cursor?.sourceCommentIdTieBreak ?? null}::uuid)
|
|
)
|
|
order by created_at desc, attachment_id desc, source_comment_id desc
|
|
limit ${input.limit + 1}
|
|
`);
|
|
const candidates = (
|
|
Array.from(rawCandidates) as Array<{
|
|
attachment_id: string;
|
|
source_comment_id: string;
|
|
created_at: Date | string;
|
|
}>
|
|
).map((row) => ({
|
|
sourceCommentId: row.source_comment_id,
|
|
attachmentId: row.attachment_id,
|
|
createdAt:
|
|
row.created_at instanceof Date
|
|
? row.created_at
|
|
: new Date(row.created_at),
|
|
}));
|
|
const listed: ListedChatAttachment[] = [];
|
|
const scanned = candidates.slice(0, input.limit);
|
|
for (const candidate of scanned) {
|
|
if (listed.length >= input.limit) break;
|
|
try {
|
|
const source = await loadSource(
|
|
tx as unknown as Db,
|
|
input.binding,
|
|
conversation,
|
|
candidate.sourceCommentId,
|
|
candidate.attachmentId,
|
|
true,
|
|
);
|
|
listed.push({
|
|
sourceCommentId: source.sourceCommentId,
|
|
attachmentId: source.attachmentId,
|
|
filename: source.filename,
|
|
contentType: source.contentType,
|
|
byteSize: source.byteSize,
|
|
sha256: source.sha256,
|
|
createdAt: source.createdAt.toISOString(),
|
|
contentAccess: "metadata_only",
|
|
});
|
|
} catch (error) {
|
|
// Metadata listing omits deleted, oversized, or no-longer-lineaged files.
|
|
if (
|
|
!(error instanceof Error) ||
|
|
error.message !== "paperclip_runner_chat_attachment_source_denied"
|
|
) {
|
|
throw error;
|
|
}
|
|
}
|
|
}
|
|
const lastScanned = scanned.at(-1) ?? null;
|
|
const hasMore = candidates.length > input.limit;
|
|
return {
|
|
attachments: listed,
|
|
nextCursor:
|
|
hasMore && lastScanned
|
|
? encodeListCursor({
|
|
schema: "paperclip.chat-attachment-list-cursor.v1",
|
|
conversationId: conversation.conversationId,
|
|
sourceCommentId: sourceFilter,
|
|
createdAt: lastScanned.createdAt.toISOString(),
|
|
attachmentId: lastScanned.attachmentId,
|
|
sourceCommentIdTieBreak: lastScanned.sourceCommentId,
|
|
})
|
|
: null,
|
|
complete: !hasMore,
|
|
};
|
|
});
|
|
}
|
|
|
|
export async function authorizeChatAttachmentReuse(input: {
|
|
db: Db;
|
|
binding: ChatReuseBinding;
|
|
contextSnapshot: unknown;
|
|
sourceCommentId: string;
|
|
attachmentId: string;
|
|
/** Inspection may open empty files; publication retains its nonempty bound. */
|
|
allowEmpty?: boolean;
|
|
}): Promise<ChatAttachmentReuseSource> {
|
|
const conversation = await authorizeChatConversationForBoundRun(
|
|
input.db,
|
|
input.binding,
|
|
input.contextSnapshot,
|
|
);
|
|
return loadSource(
|
|
input.db,
|
|
input.binding,
|
|
conversation,
|
|
input.sourceCommentId,
|
|
input.attachmentId,
|
|
input.allowEmpty,
|
|
);
|
|
}
|
|
|
|
async function readSourceBytes(
|
|
storage: StorageService,
|
|
companyId: string,
|
|
source: ChatAttachmentReuseSource,
|
|
timeoutMs: number,
|
|
): Promise<Buffer> {
|
|
let acquisitionTimedOut = false;
|
|
let rejectAcquisition!: (error: Error) => void;
|
|
const acquisitionTimeout = new Promise<never>((_resolve, reject) => {
|
|
rejectAcquisition = reject;
|
|
});
|
|
const acquisitionTimer = setTimeout(() => {
|
|
acquisitionTimedOut = true;
|
|
rejectAcquisition(
|
|
new Error("paperclip_runner_chat_attachment_source_read_timed_out"),
|
|
);
|
|
}, timeoutMs);
|
|
acquisitionTimer.unref?.();
|
|
const objectPromise = storage
|
|
.getObject(companyId, source.objectKey)
|
|
.then((object) => {
|
|
if (acquisitionTimedOut) object.stream.destroy();
|
|
return object;
|
|
});
|
|
let object: Awaited<ReturnType<StorageService["getObject"]>>;
|
|
try {
|
|
object = await Promise.race([objectPromise, acquisitionTimeout]);
|
|
} finally {
|
|
clearTimeout(acquisitionTimer);
|
|
}
|
|
const timeout = setTimeout(() => {
|
|
object.stream.destroy(
|
|
new Error("paperclip_runner_chat_attachment_source_read_timed_out"),
|
|
);
|
|
}, timeoutMs);
|
|
timeout.unref?.();
|
|
const chunks: Buffer[] = [];
|
|
let total = 0;
|
|
try {
|
|
for await (const chunk of object.stream) {
|
|
const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
|
|
total += buffer.length;
|
|
if (total > source.byteSize || total > MAX_ATTACHMENT_BYTES) {
|
|
object.stream.destroy();
|
|
throw new Error(
|
|
"paperclip_runner_chat_attachment_source_size_mismatch",
|
|
);
|
|
}
|
|
chunks.push(buffer);
|
|
}
|
|
} finally {
|
|
clearTimeout(timeout);
|
|
}
|
|
const body = Buffer.concat(chunks);
|
|
if (
|
|
body.length !== source.byteSize ||
|
|
createHash("sha256").update(body).digest("hex") !==
|
|
source.sha256.toLowerCase()
|
|
) {
|
|
if (!object.stream.destroyed) {
|
|
object.stream.destroy();
|
|
}
|
|
throw new Error(
|
|
"paperclip_runner_chat_attachment_source_integrity_mismatch",
|
|
);
|
|
}
|
|
return body;
|
|
}
|
|
|
|
const DEFAULT_STORAGE_TIMEOUT_MS = 10_000;
|
|
|
|
async function deleteStorageObjectWithin(
|
|
storage: StorageService,
|
|
companyId: string,
|
|
objectKey: string,
|
|
timeoutMs: number,
|
|
): Promise<void> {
|
|
const deletion = storage
|
|
.deleteObject(companyId, objectKey)
|
|
.catch(() => undefined);
|
|
let timer: ReturnType<typeof setTimeout> | null = null;
|
|
try {
|
|
await Promise.race([
|
|
deletion,
|
|
new Promise<void>((resolve) => {
|
|
timer = setTimeout(resolve, timeoutMs);
|
|
timer.unref?.();
|
|
}),
|
|
]);
|
|
} finally {
|
|
if (timer) clearTimeout(timer);
|
|
}
|
|
}
|
|
|
|
async function putStorageObjectWithin(
|
|
storage: StorageService,
|
|
input: Parameters<StorageService["putFile"]>[0],
|
|
timeoutMs: number,
|
|
): Promise<Awaited<ReturnType<StorageService["putFile"]>>> {
|
|
let timedOut = false;
|
|
let rejectTimeout!: (error: Error) => void;
|
|
const timeout = new Promise<never>((_resolve, reject) => {
|
|
rejectTimeout = reject;
|
|
});
|
|
const timer = setTimeout(() => {
|
|
timedOut = true;
|
|
rejectTimeout(
|
|
new Error("paperclip_runner_chat_attachment_storage_write_timed_out"),
|
|
);
|
|
}, timeoutMs);
|
|
timer.unref?.();
|
|
const write = storage.putFile(input).then((stored) => {
|
|
if (timedOut) {
|
|
void deleteStorageObjectWithin(
|
|
storage,
|
|
input.companyId,
|
|
stored.objectKey,
|
|
timeoutMs,
|
|
);
|
|
}
|
|
return stored;
|
|
});
|
|
try {
|
|
return await Promise.race([write, timeout]);
|
|
} finally {
|
|
clearTimeout(timer);
|
|
}
|
|
}
|
|
|
|
export async function prepareReusedChatAttachment(input: {
|
|
db: Db;
|
|
binding: ChatReuseBinding;
|
|
source: ChatAttachmentReuseSource;
|
|
title: string;
|
|
storage?: StorageService;
|
|
storageTimeoutMs?: number;
|
|
}): Promise<PreparedReusedChatAttachment> {
|
|
const [issue] = await input.db
|
|
.select({ statusVersion: issues.statusVersion })
|
|
.from(issues)
|
|
.innerJoin(
|
|
heartbeatRuns,
|
|
and(
|
|
eq(heartbeatRuns.id, input.binding.runId),
|
|
eq(heartbeatRuns.companyId, issues.companyId),
|
|
eq(heartbeatRuns.nativeIssueId, issues.id),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
eq(issues.id, input.binding.issueId),
|
|
eq(issues.companyId, input.binding.companyId),
|
|
eq(issues.assigneeAgentId, input.binding.agentId),
|
|
eq(issues.executionRunId, input.binding.runId),
|
|
eq(heartbeatRuns.agentId, input.binding.agentId),
|
|
eq(heartbeatRuns.status, "running"),
|
|
eq(heartbeatRuns.runtimeMode, "native"),
|
|
),
|
|
)
|
|
.limit(1);
|
|
if (!issue)
|
|
throw new Error("paperclip_runner_chat_attachment_binding_denied");
|
|
const storage = input.storage ?? getStorageService();
|
|
const storageTimeoutMs =
|
|
typeof input.storageTimeoutMs === "number" &&
|
|
Number.isFinite(input.storageTimeoutMs) &&
|
|
input.storageTimeoutMs > 0
|
|
? input.storageTimeoutMs
|
|
: DEFAULT_STORAGE_TIMEOUT_MS;
|
|
const body = await readSourceBytes(
|
|
storage,
|
|
input.binding.companyId,
|
|
input.source,
|
|
storageTimeoutMs,
|
|
);
|
|
const stored = await putStorageObjectWithin(
|
|
storage,
|
|
{
|
|
companyId: input.binding.companyId,
|
|
namespace: `issues/${input.binding.issueId}`,
|
|
originalFilename: input.source.filename,
|
|
contentType: input.source.contentType,
|
|
body,
|
|
},
|
|
storageTimeoutMs,
|
|
);
|
|
try {
|
|
if (
|
|
stored.byteSize !== body.length ||
|
|
stored.sha256.toLowerCase() !== input.source.sha256.toLowerCase() ||
|
|
stored.contentType !== input.source.contentType
|
|
) {
|
|
throw new Error("paperclip_runner_chat_attachment_storage_mismatch");
|
|
}
|
|
const attachment = await issueService(input.db).createAttachment({
|
|
issueId: input.binding.issueId,
|
|
provider: stored.provider,
|
|
objectKey: stored.objectKey,
|
|
contentType: stored.contentType,
|
|
byteSize: stored.byteSize,
|
|
sha256: stored.sha256,
|
|
originalFilename: stored.originalFilename,
|
|
createdByAgentId: input.binding.agentId,
|
|
createdByRunId: input.binding.runId,
|
|
});
|
|
if (
|
|
!attachment.artifactWorkProductId ||
|
|
attachment.originatingRunId !== input.binding.runId
|
|
) {
|
|
throw new Error("paperclip_runner_chat_attachment_origin_not_persisted");
|
|
}
|
|
const [workProduct] = await input.db
|
|
.select({ metadata: issueWorkProducts.metadata })
|
|
.from(issueWorkProducts)
|
|
.where(
|
|
and(
|
|
eq(issueWorkProducts.id, attachment.artifactWorkProductId),
|
|
eq(issueWorkProducts.companyId, input.binding.companyId),
|
|
eq(issueWorkProducts.issueId, input.binding.issueId),
|
|
eq(issueWorkProducts.createdByRunId, input.binding.runId),
|
|
),
|
|
)
|
|
.for("update")
|
|
.limit(1);
|
|
if (!workProduct) {
|
|
throw new Error("paperclip_runner_chat_attachment_origin_not_persisted");
|
|
}
|
|
await input.db.transaction(async tx => {
|
|
await notifyIssueRecovery(tx, { companyId: input.binding.companyId });
|
|
await tx
|
|
.update(issueWorkProducts)
|
|
.set({
|
|
title: input.title,
|
|
metadata: {
|
|
...record(workProduct.metadata),
|
|
reusedFromAttachmentId: input.source.attachmentId,
|
|
reusedFromCommentId: input.source.sourceCommentId,
|
|
reusedFromSha256: input.source.sha256,
|
|
},
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(
|
|
and(
|
|
eq(issueWorkProducts.id, attachment.artifactWorkProductId),
|
|
eq(issueWorkProducts.companyId, input.binding.companyId),
|
|
eq(issueWorkProducts.issueId, input.binding.issueId),
|
|
eq(issueWorkProducts.createdByRunId, input.binding.runId),
|
|
),
|
|
);
|
|
});
|
|
const comment = await issueService(input.db).addComment(
|
|
input.binding.issueId,
|
|
`Prepared ${input.title} for this response.`,
|
|
{ agentId: input.binding.agentId, runId: input.binding.runId },
|
|
{
|
|
attachmentIds: [attachment.id],
|
|
authorizationReason: "paperclip_runner_protocol",
|
|
metadata: { version: 1, sections: [], attachmentPreparation: { attachmentId: attachment.id } },
|
|
},
|
|
input.db,
|
|
);
|
|
return {
|
|
result: {
|
|
commandId: `chat-attachment-reused:${attachment.id}`,
|
|
disposition: "applied",
|
|
stateRevision: issue.statusVersion,
|
|
entityRefs: [
|
|
attachment.id,
|
|
attachment.artifactWorkProductId,
|
|
comment.id,
|
|
],
|
|
scheduledWakeIds: [],
|
|
source: {
|
|
commentId: input.source.sourceCommentId,
|
|
attachmentId: input.source.attachmentId,
|
|
sha256: input.source.sha256,
|
|
},
|
|
prepared: {
|
|
attachmentId: attachment.id,
|
|
workProductId: attachment.artifactWorkProductId,
|
|
commentId: comment.id,
|
|
filename: input.source.filename,
|
|
byteSize: body.length,
|
|
sha256: attachment.sha256,
|
|
},
|
|
},
|
|
rollbackDefinitePreCommitFailure: async () => {
|
|
await deleteStorageObjectWithin(
|
|
storage,
|
|
input.binding.companyId,
|
|
stored.objectKey,
|
|
storageTimeoutMs,
|
|
);
|
|
},
|
|
};
|
|
} catch (error) {
|
|
await deleteStorageObjectWithin(
|
|
storage,
|
|
input.binding.companyId,
|
|
stored.objectKey,
|
|
storageTimeoutMs,
|
|
);
|
|
throw error;
|
|
}
|
|
}
|