Files
PaperClipAI/server/src/services/native-runtime/chat-attachment-reuse.ts
T
DottaandPaperclip 208a093d19 fix(chat): preserve resumed review waits and retry newer saved input (#17725)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/36/changes/paperclip/36
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub review agents publish findings and wait for an authorized
reply or head change.
> - Review tools could retain the requester after resume while
completion lost that same wait proof.
> - A yielded report could then queue its own continuation or repeat a
missing-wait error.
> - Ordinary Retry could also select an older failed turn while a newer
saved request had no admission.
> - This change shares the existing authority proofs and canonical
admission owners.
> - Current access, publication, execution and budget gates remain
required.

## Linked Issues or Issue Description

Refs #17318. Builds on the canonical saved-input retry owner in #17733.
Related #16550 covers deferred legacy comments after ownership waits.
The separate recovery repair in #17724 owns the cleanup-to-saved-input
handoff; this PR does not add a second admission owner.

A resumed review could publish an assessment, then turn its
`response_wake` into an immediate `issue_status_changed` wake. A
provider that omitted its continuation could instead repeat a completion
error. A newer linked-human message saved during a budget hold could
also remain unstarted because Retry selected the older failed source.

## What Changed

- Resolve passive review waits through the accepted requester ledger
used by GitHub tools. Require the latest reviewed head, exact company,
task, agent, repository and processed publication under current access.
- Fill omitted/null yielded continuation from that server proof. Permit
an inherited normal tree resume only through its accepted parent
identity. Permit an ordinary follow-up only after this run has a
processed useful reply linked to its exact inbound message. Reject
acknowledgements, partial assessments, pending or uncertain writes as
that reply proof.
- Recheck the same run/review/head/source proof during feedback,
finalization and final status commit. Preserve explicit malformed-input
rejection and Done policy. Add no monitor permission or publication
grant.
- Keep heartbeat and publication projections narrow. Preserve both
published Copilot v62 identities and generate source-bound v63 from the
pinned merged base. No v63 live qualification is claimed.
- Certify the structured isolated-directory binding failure only at its
pre-allocation boundary and only through canonical task Retry. Other
workspace validation reasons and unknown effects remain denied.
- Route Retry to eligible newer saved input through the existing
canonical owner. Preserve the old result and skipped receipt. Require
current principal, source, App, runtime fence, budget, native ownership
and exact cleanup proof.
- Recover absent historical App metadata only through an earlier trusted
processed response receipt and an immutable non-null endpoint claim.
Explicit malformed or conflicting metadata stays denied.
- Revalidate execution through the exact claimed wake, coalesced task
Retry and resolved user-continuation receipt after admission. A second
normal Retry can recover the exact never-started authorization failure
with unchanged saved input. Require empty execution evidence, exact
diagnostics and no allocated or active executor.
- Explain when a recorded result prevents exact replay. Ordinary board
Retry directs a new task message to continue from saved work. Inherited
requester identity alone never grants replay.
- Keep accepted board-message preparation failures with the canonical
task Retry owner. Certify the exact unchanged comment, original user
wake, accepted same-user identity and absence of provider execution. Pin
that source, recheck it at admission, and bind identity initialization
to the coalesced intent and exact claimed successor receipt.
Uncertifiable sources retain an actionable fresh-message refusal.
- Return safe named authority categories for board Retry refusals.
Update execution documentation and capability maps. Add no schema or
polling loop.

## Verification

- A fresh external review request exposed a separate isolated-directory
preparation failure before allocation. Its first normal Retry saved an
intent but admitted no successor. This follow-up certifies only that
exact structured preparation boundary through the existing canonical
Retry owner, with trusted failed settlement and no executor, lease,
result or output. A real HTTP/queue regression reproduces the missing
successor before the repair. Twenty-three current-source safety cases
and the final 98-case Retry/HTTP group pass, including a fresh click
after a denied intent, source revocation, accepted results and the older
direct API remaining denied. The warning case uses actual configuration
preparation and production event append; unknown warning shape or
provider provenance stays denied. Full workspace typecheck, build, maps
and diff checks pass. Renewed hosted normal Retry now succeeds on the
final combined source: it preserves the exact saved request and linked
requester, starts one native successor in a fresh attempt workspace,
publishes the current-head 2/5 assessment and commits In review with
passive response wake.

- A later hosted normal Retry published a low-score assessment but lost
its accepted source identity because the previous attempt failed before
identity initialization. It repeated continuation errors, then submitted
Done. This is failed acceptance, not a successful qualification. The
initializer now retains the exact source only after execution-time
failed-chat Retry authorization; generic Retry and Done policy are
unchanged. A real queue/initializer regression fails with a null source
on the previous code and passes with the newer saved source. Two real
database completion cases verify that source yields the authoritative
wait and commits In review. Renewed hosted normal Retry now retains the
accepted source and exposes the authoritative current-review wait. It
completed after two semantic feedback corrections (wrong contract
revision, then wrong criterion IDs); the correct contract was present in
emitted input. The third finish was accepted and committed In review,
with no missing-wait loop or Done escalation.
- Repeated board-message preparation failures now walk their canonical
intent, failed dispatch and accepted parent identity, rechecking each
contained predecessor. A production queue regression fails the second
ordinary HTTP Retry before the repair and passes afterward. Changed
source, contradictory execution evidence and cyclic receipts remain
denied; traversal stops after twenty links.

- New inherited omission cases failed before the repair. The final full
PostgreSQL 18 wait/status suite passes **252 tests** under Node 24.22.
It covers omitted/null input, malformed-first recovery, same-head
resume, exact-source replies, changed heads, revoked access,
pending/uncertain/mixed publications, final commit revalidation, no
duplicate review and no self-wake.
- **15 existing publication tests passed before the pinned merge** on a
separate fresh database. They retain default partial-delivery and
failure-fallback behavior. Context reads remain below the unchanged 16
KiB heartbeat and publication response limits with large saved output
present.
- Saved-input tests cover a real signed PR webhook, production
accounting-budget refusal, normal policy recovery and canonical Retry.
Historical producer-shaped tests prove the missing-App case red, then
check trusted prior receipts and malformed, late, revoked or mismatched
proof. A broader order-coupled test was reproduced on the prior source;
its global fault injector now targets its own authorization query. The
43-case sequence passes without changing production behavior or its
safety assertions.
- New signed-ingress and real heartbeat execution tests reproduced the
admitted-successor failure, then the next normal Retry failure. The
final pinned merged source passes 249 full GitHub review/publication and
Retry cases on a fresh database, including all 49 saved-input cases. A
separate 74-case Retry/HTTP run also passes. Exact dispatch negatives
cover source, company, agent, task, successor, receipt, pause,
assignment, accepted success, controller and cleanup proof. Bootstrap
contradictions remain denied. The canonical Retry suite passes 16 cases
through its supported external PostgreSQL test connection.
- The normal board HTTP route reproduced the unusable inherited-result
refusal. It now returns actionable guidance without starting a
successor. The actual UI Retry hook and banner tests pass 12 cases.
- A production-shaped accepted board comment followed by a pre-provider
qualification failure reproduced HTTP 409 before the repair. The final
real HTTP/queue test certifies one canonical successor and preserves the
accepted identity. It rejects changed source, actor, task, agent,
result, controller, process, profile, allocation and receipt evidence. A
post-acceptance source change keeps the same intent pending. Execution
revalidation rejects removed or mismatched proof. The probe stops
deliberately after production identity initialization; it does not claim
provider execution. The final source passes the 75-case Retry group, the
focused two-case board/inherited-policy group and 19 canonical Retry
cases. Full workspace typecheck, build and map checks pass.
- Full workspace `pnpm -r typecheck` and `pnpm build` pass. Map,
module-boundary and diff checks pass. Full local `pnpm test:run` remains
unverified. The 256-case explicit-native suite cannot initialize its
packaged PostgreSQL because libzstd is absent; an existing
command-scoped fallback did not resolve it. Current-head remote
full-suite checks are required. Merged-source full GitHub+Retry (249),
separate Retry/HTTP (74), full wait/status (252), profile/history and
completion (75), and visible Retry hook/banner (12) checks pass. The
fresh owned PostgreSQL 18 databases applied all 377 migrations.
- Actual Rust Runner, OpenCode 1.18.34 and exact DeepSeek used unchanged
production-generated synthetic contexts. Resume finished with an
explicit server wait. The author-reply trial omitted continuation twice:
its first report failed the existing empty-criteria check; the corrected
second report still omitted continuation and received the exact server
wait. It completed once with no missing-wait loop or Done escalation.
Controller acknowledgments were synthetic; these probes do not establish
hosted publication or board state.
- Hosted combined testing proved automatic admission of the exact saved
author reply, a useful GitHub answer and committed passive wait. It
required two malformed-continuation rejections before a nullish fallback
succeeded. A later normal Pause/Resume completed in one finish with
explicit response_wake and no new review/comment. The admitted normal
Retry then exposed the execution guard gap repaired here. Those remain
checkpoint results. On the final combined source, renewed normal Retry
completed actual native provider work and GitHub publication, committed
the passive wait, released ownership and cleaned up its environment. A
subsequent ordinary unmentioned author reply woke once, made no GitHub
tool calls or new assessment, and completed its first finish with an
explicit response wake. Both exact leases expired with successful
cleanup and the queue was empty. However, final destination inspection
found a separate automatic fallback overwrote the existing summary with
a generic no-final-response message. External summary preservation
therefore remains failed acceptance; the separate publication repair in
#17726 owns that producer. Full combined publication preservation has
not passed. These hosted runs include the pinned-base native additions;
they do not qualify Copilot v63.
- Current-head automated review is 5/5 with zero unresolved threads. All
current-head CI gates passed (54 successful checks/contexts, four
skipped, none pending). The earlier login-cleanup idle-work assertion
also reproduced locally while its controlled baseline passed; its cause
remains unknown, but the exact assertion passed in the fresh
current-head CI shard. No timeout, assertion or production workaround
was added. This PR is ready for review within its source, authority,
native finalization, state, wake and cleanup scope. The independent
automatic fallback publication regression is tracked by #17726; no
complete combined workflow pass is claimed.

## Risks

- The latest saved review defines the durable head boundary. Ingress
must record a new head before admission. A reply receipt proves delivery
only for its exact saved source; a shared GitHub comment ID alone cannot
authorize this wait.
- Historical App recovery cannot infer identity without direct proof or
an earlier valid server response receipt. It never replays that
response.
- This does not infer review-score policy from instructions or override
Done. No deployment, merge or repair of historical tasks is claimed by
the PR.

## Model Used

OpenAI `gpt-6.1-sol`, with reasoning, repository inspection, code
editing and test execution. The context-window size is not exposed in
this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused database and
native suites; full local suite limitation described above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green (verified on immutable latest
head)
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(verified on the latest head)
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-26 22:22:47 -05:00

1659 lines
57 KiB
TypeScript

import { createHash } from "node:crypto";
import { and, desc, eq, inArray, isNull, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
assets,
agents,
chatConversations,
chatDeliveries,
chatEndpointResources,
chatEndpoints,
chatExternalPrincipals,
chatIdentityLinks,
chatMessageLinks,
chatPublications,
companyMemberships,
heartbeatRuns,
issueAttachments,
issueComments,
issues,
issueWorkProducts,
runIdentityContexts,
} from "@paperclipai/db";
import {
isAllowedContentType,
MAX_ATTACHMENT_BYTES,
normalizeContentType,
} from "../../attachment-types.js";
import { getStorageService } from "../../storage/index.js";
import type { StorageService } from "../../storage/types.js";
import { loadConfig } from "../../config.js";
import { HttpError } from "../../errors.js";
import { voiceSessionStore, type VoiceTransaction } from "../voice/voice-session-store.js";
import { issueService } from "../issues.js";
import { readInheritedGitHubReviewWait } from "../chat-channels/github/review-wait.js";
import { boundExternalChatProvider } from "./external-chat-provider.js";
import { resolveExternalChatQuestionResponse } from "./external-chat-question-response.js";
import { githubRunReplyState } from "../chat-run-publications.js";
// This boundary reads identity and wake authority, never launch renderings,
// saved provider profiles, or run output. Keep the nested wake proof narrow too.
const responseWaitContextKeys = ["source", "issueId", "wakeReason", "wakeCommentId", "wakeCommentIds",
"interactionId", "interactionKind", "interactionStatus", "sourceRunId", "sourceCommentId",
"externalChatContinuation", "paperclipExternalChatQuestionResponse", "paperclipHarnessCheckedOut",
"paperclipExternalChatExecutionBound"] as const;
const responseWaitRun = {
id: heartbeatRuns.id, companyId: heartbeatRuns.companyId, agentId: heartbeatRuns.agentId,
nativeIssueId: heartbeatRuns.nativeIssueId, issueId: heartbeatRuns.issueId, status: heartbeatRuns.status,
responsibleUserId: heartbeatRuns.responsibleUserId, activeIdentityContextId: heartbeatRuns.activeIdentityContextId,
contextSnapshot: sql<Record<string, unknown> | null>`case when jsonb_typeof(${heartbeatRuns.contextSnapshot}) = 'object'
then (select coalesce(jsonb_object_agg(field.key, field.value), '{}'::jsonb)
from jsonb_each(${heartbeatRuns.contextSnapshot}) as field(key, value)
where field.key in (${sql.join(responseWaitContextKeys.map(key => sql`${key}`), sql`, `)}))
|| jsonb_build_object('paperclipWake', jsonb_build_object(
'checkedOutByHarness', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,checkedOutByHarness}',
'externalChatExecutionBound', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,externalChatExecutionBound}',
'externalChatProvider', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,externalChatProvider}',
'issue', jsonb_build_object('id', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,issue,id}'),
'commentIds', ${heartbeatRuns.contextSnapshot}#>'{paperclipWake,commentIds}'))
else null end`,
};
export const LIST_CHAT_ATTACHMENTS_TOOL_NAME = "list_chat_attachments";
export const REUSE_CHAT_ATTACHMENT_TOOL_NAME = "reuse_chat_attachment";
export const LIST_CHAT_ATTACHMENTS_TOOL_DEFINITION = Object.freeze({
name: LIST_CHAT_ATTACHMENTS_TOOL_NAME,
description:
"List metadata for files already received or delivered in this same external-chat conversation. Call with {} for the latest page (default 20), or set limit to an integer from 1 through 50. Page through nextCursor to find older files, or filter by an exact sourceCommentId when known. File bytes and private storage locations are never returned.",
inputSchema: {
type: "object",
properties: {
sourceCommentId: {
type: ["string", "null"],
description:
"Optional exact sourceCommentId returned by this tool on an earlier page.",
},
limit: { type: "integer", minimum: 1, maximum: 50, default: 20 },
cursor: {
type: ["string", "null"],
maxLength: 1024,
description: "Opaque nextCursor from the immediately preceding page.",
},
},
required: [],
additionalProperties: false,
},
annotations: {
semanticContract: "paperclip.server-chat-attachment-reuse.v1",
operationId: LIST_CHAT_ATTACHMENTS_TOOL_NAME,
version: 1,
exposure: "run_scoped",
requiredClaims: [],
},
});
export const REUSE_CHAT_ATTACHMENT_TOOL_DEFINITION = Object.freeze({
name: REUSE_CHAT_ATTACHMENT_TOOL_NAME,
description:
"Prepare one exact file previously received or delivered in this same external-chat conversation for the current response. All four arguments are required: copy sourceCommentId and attachmentId from list_chat_attachments, provide a nonempty title, and choose a nonempty idempotencyKey of at most 200 characters once for this reuse; send that same key unchanged on every retry. This queues a verified copy for final publication; it does not confirm provider delivery.",
inputSchema: {
type: "object",
properties: {
idempotencyKey: { type: "string", minLength: 1, maxLength: 200 },
sourceCommentId: { type: "string", format: "uuid" },
attachmentId: { type: "string", format: "uuid" },
title: { type: "string", minLength: 1, maxLength: 500 },
},
required: ["idempotencyKey", "sourceCommentId", "attachmentId", "title"],
additionalProperties: false,
},
annotations: {
semanticContract: "paperclip.server-chat-attachment-reuse.v1",
operationId: REUSE_CHAT_ATTACHMENT_TOOL_NAME,
version: 1,
exposure: "run_scoped",
requiredClaims: [],
},
});
export type ChatReuseBinding = {
companyId: string;
issueId: string;
runId: string;
agentId: string;
};
type AuthorizedConversation = {
conversationId: string;
endpointId: string;
};
export type ExternalChatResponseWaitAuthorization =
| "authorized"
| "revoked"
| "not_applicable";
type AuthorizationLockMode = "blocking" | "nonblocking" | "read";
type AuthorizationEndpoint = Pick<typeof chatEndpoints.$inferSelect,
"id" | "companyId" | "provider" | "providerAccountId" | "status" | "setup"
| "allowDirectMessages" | "allowGroupChats" | "allowUnlinkedPeople" | "sponsorUserId"
>;
type AuthorizationConversation = Pick<typeof chatConversations.$inferSelect,
"id" | "resourceId" | "isDirectMessage"
>;
type AuthorizationResource = Pick<typeof chatEndpointResources.$inferSelect,
"availability" | "type" | "enabled"
>;
class ExternalChatWaitAuthorizationContentionError extends Error {
constructor() {
super("paperclip_external_chat_wait_authorization_contended");
this.name = "ExternalChatWaitAuthorizationContentionError";
}
}
export function isExternalChatWaitAuthorizationContention(
error: unknown,
): boolean {
let current: unknown = error;
for (let depth = 0; depth < 4 && current; depth += 1) {
if (current instanceof ExternalChatWaitAuthorizationContentionError) {
return true;
}
const value = record(current);
if (value.code === "55P03") return true;
current = value.cause;
}
return false;
}
export type ChatAttachmentReuseSource = {
sourceCommentId: string;
attachmentId: string;
filename: string;
contentType: string;
byteSize: number;
sha256: string;
objectKey: string;
createdAt: Date;
};
export type ListedChatAttachment = Omit<
ChatAttachmentReuseSource,
"objectKey" | "createdAt"
> & {
createdAt: string;
contentAccess: "metadata_only";
};
export type ListedChatAttachmentPage = {
attachments: ListedChatAttachment[];
nextCursor: string | null;
complete: boolean;
};
type ListCursor = {
schema: "paperclip.chat-attachment-list-cursor.v1";
conversationId: string;
sourceCommentId: string | null;
createdAt: string;
attachmentId: string;
sourceCommentIdTieBreak: string;
};
export type PreparedReusedChatAttachment = {
result: {
commandId: string;
disposition: "applied";
stateRevision: number;
entityRefs: string[];
scheduledWakeIds: never[];
source: {
commentId: string;
attachmentId: string;
sha256: string;
};
prepared: {
attachmentId: string;
workProductId: string;
commentId: string;
filename: string;
byteSize: number;
sha256: string;
};
};
rollbackDefinitePreCommitFailure: (() => Promise<void>) | null;
};
function record(value: unknown): Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
function wakeCommentIds(contextSnapshot: unknown): string[] {
const context = record(contextSnapshot);
const raw = context.wakeCommentIds;
if (!Array.isArray(raw)) return [];
return [
...new Set(
raw
.filter(
(value): value is string =>
typeof value === "string" && value.trim().length > 0,
)
.map((value) => value.trim()),
),
];
}
function publicationAttachmentIds(payload: unknown): string[] {
const raw = record(payload).attachmentIds;
return Array.isArray(raw)
? raw.filter((value): value is string => typeof value === "string")
: [];
}
function isUuid(value: unknown): value is string {
return (
typeof value === "string" &&
/^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/iu.test(
value,
)
);
}
function encodeListCursor(cursor: ListCursor): string {
return Buffer.from(JSON.stringify(cursor), "utf8").toString("base64url");
}
function decodeListCursor(
value: unknown,
conversationId: string,
sourceCommentId: string | null,
): ListCursor | null {
if (value === null || value === undefined) return null;
if (typeof value !== "string" || value.length === 0 || value.length > 1024) {
throw new Error("paperclip_runner_chat_attachment_cursor_invalid");
}
try {
const parsed = record(
JSON.parse(Buffer.from(value, "base64url").toString("utf8")),
);
const createdAt =
typeof parsed.createdAt === "string" ? parsed.createdAt : "";
const parsedDate = new Date(createdAt);
if (
parsed.schema !== "paperclip.chat-attachment-list-cursor.v1" ||
parsed.conversationId !== conversationId ||
(parsed.sourceCommentId ?? null) !== sourceCommentId ||
Number.isNaN(parsedDate.getTime()) ||
!isUuid(parsed.attachmentId) ||
!isUuid(parsed.sourceCommentIdTieBreak)
) {
throw new Error("invalid");
}
return parsed as ListCursor;
} catch {
throw new Error("paperclip_runner_chat_attachment_cursor_invalid");
}
}
function destinationAllowed(
endpoint: AuthorizationEndpoint,
conversation: AuthorizationConversation,
resource: AuthorizationResource | null,
): boolean {
if (conversation.isDirectMessage) return endpoint.allowDirectMessages;
if (!resource || resource.availability !== "available") return false;
if (
endpoint.provider === "microsoft-teams" &&
resource.type === "group_chat"
) {
return endpoint.allowGroupChats;
}
return resource.enabled;
}
function currentSetupTestAcceptsBoundDeliveries(
endpoint: AuthorizationEndpoint,
deliveries: Array<{
normalizedEvent: unknown;
processedAt: Date | null;
receivedAt: Date;
}>,
): boolean {
if (endpoint.status === "active") return true;
if (endpoint.status !== "verifying" || deliveries.length === 0) return false;
const setup = record(endpoint.setup);
const testStartedAt =
typeof setup.testStartedAt === "string"
? new Date(setup.testStartedAt)
: null;
const generation = setup.runtimeGeneration;
if (
setup.step !== "test" ||
!testStartedAt ||
Number.isNaN(testStartedAt.getTime()) ||
typeof generation !== "number" ||
!Number.isSafeInteger(generation) ||
generation < 0
) {
return false;
}
return deliveries.every((delivery) => {
const runtimeContext = record(
record(delivery.normalizedEvent).runtimeContext,
);
return (
delivery.receivedAt >= testStartedAt &&
delivery.processedAt !== null &&
delivery.processedAt >= testStartedAt &&
delivery.processedAt >= delivery.receivedAt &&
runtimeContext.generation === generation &&
typeof runtimeContext.credentialFingerprint === "string" &&
/^[a-f0-9]{64}$/u.test(runtimeContext.credentialFingerprint)
);
});
}
async function principalAuthorized(
tx: Db,
endpoint: AuthorizationEndpoint,
principalId: string,
lockMode: AuthorizationLockMode = "blocking",
): Promise<boolean> {
if (lockMode === "blocking") {
await tx.execute(
sql`select pg_advisory_xact_lock(hashtextextended(${`chat-identity:${endpoint.companyId}:${principalId}`}, 0))`,
);
} else if (lockMode === "nonblocking") {
const [lock] = (await tx.execute(
sql`select pg_try_advisory_xact_lock(hashtextextended(${`chat-identity:${endpoint.companyId}:${principalId}`}, 0)) as acquired`,
)) as unknown as Array<{ acquired: boolean }>;
if (!lock?.acquired) {
throw new ExternalChatWaitAuthorizationContentionError();
}
}
const principalQuery = tx
.select({ id: chatExternalPrincipals.id, externalId: chatExternalPrincipals.externalId })
.from(chatExternalPrincipals)
.where(
and(
eq(chatExternalPrincipals.id, principalId),
eq(chatExternalPrincipals.companyId, endpoint.companyId),
eq(chatExternalPrincipals.provider, endpoint.provider),
eq(
chatExternalPrincipals.providerAccountId,
endpoint.providerAccountId ?? "",
),
),
);
const [principal] = await (lockMode === "read"
? principalQuery.limit(1)
: lockMode === "nonblocking"
? principalQuery.for("update", { noWait: true }).limit(1)
: principalQuery.for("update").limit(1));
if (!principal) return false;
if (endpoint.provider === "speko") {
try {
await voiceSessionStore(tx, { allowLocalBoard: loadConfig().deploymentMode === "local_trusted", nonblockingAuthority: lockMode === "nonblocking" })
.authorizePrincipal(tx as unknown as VoiceTransaction, endpoint.companyId, endpoint.id, principal.externalId);
return true;
} catch (error) {
if (error instanceof HttpError && [403, 404].includes(error.status)) return false;
throw error;
}
}
const linkQuery = tx
.select({
status: chatIdentityLinks.status,
userId: chatIdentityLinks.paperclipUserId,
})
.from(chatIdentityLinks)
.where(
and(
eq(chatIdentityLinks.companyId, endpoint.companyId),
eq(chatIdentityLinks.endpointId, endpoint.id),
eq(chatIdentityLinks.principalId, principalId),
),
);
const [link] = await (lockMode === "read"
? linkQuery.limit(1)
: lockMode === "nonblocking"
? linkQuery.for("update", { noWait: true }).limit(1)
: linkQuery.for("update").limit(1));
const activeMember = async (userId: string) => {
const membershipQuery = tx
.select({
status: companyMemberships.status,
role: companyMemberships.membershipRole,
})
.from(companyMemberships)
.where(
and(
eq(companyMemberships.companyId, endpoint.companyId),
eq(companyMemberships.principalType, "user"),
eq(companyMemberships.principalId, userId),
),
);
const [membership] = await (lockMode === "read"
? membershipQuery.limit(1)
: lockMode === "nonblocking"
? membershipQuery.for("update", { noWait: true }).limit(1)
: membershipQuery.for("update").limit(1));
return membership?.status === "active" && membership.role !== "viewer";
};
if (link?.status === "linked") {
return Boolean(link.userId && (await activeMember(link.userId)));
}
if (!endpoint.allowUnlinkedPeople) return false;
return endpoint.sponsorUserId ? activeMember(endpoint.sponsorUserId) : true;
}
/** Caller must independently verify the current run/issue execution owner. */
export async function authorizeChatConversationForBoundRun(
tx: Db,
binding: ChatReuseBinding,
contextSnapshot: unknown,
lockMode: AuthorizationLockMode = "blocking",
): Promise<AuthorizedConversation> {
let context = record(contextSnapshot);
if (context.source === "issue.interaction.respond") {
const answer = await resolveExternalChatQuestionResponse(tx, binding, context, lockMode);
if (!answer) throw new Error("paperclip_runner_chat_attachment_binding_denied");
context = answer.authorizationContext;
}
const source = typeof context.source === "string" ? context.source : "";
const provider = boundExternalChatProvider(source);
const commentIds = wakeCommentIds(context);
if (
!provider ||
(context.paperclipHarnessCheckedOut !== true &&
context.paperclipExternalChatExecutionBound !== true) ||
commentIds.length === 0
) {
throw new Error("paperclip_runner_chat_attachment_binding_denied");
}
const linksQuery = tx
.select({
commentId: chatMessageLinks.commentId,
conversation: {
id: chatConversations.id,
resourceId: chatConversations.resourceId,
isDirectMessage: chatConversations.isDirectMessage,
},
endpoint: {
id: chatEndpoints.id,
companyId: chatEndpoints.companyId,
provider: chatEndpoints.provider,
providerAccountId: chatEndpoints.providerAccountId,
status: chatEndpoints.status,
allowDirectMessages: chatEndpoints.allowDirectMessages,
allowGroupChats: chatEndpoints.allowGroupChats,
allowUnlinkedPeople: chatEndpoints.allowUnlinkedPeople,
sponsorUserId: chatEndpoints.sponsorUserId,
setup: sql<AuthorizationEndpoint["setup"]>`jsonb_build_object(
'step', ${chatEndpoints.setup}->'step',
'testStartedAt', ${chatEndpoints.setup}->'testStartedAt',
'runtimeGeneration', ${chatEndpoints.setup}->'runtimeGeneration'
)`,
},
principalId: chatDeliveries.principalId,
normalizedEvent: sql<unknown>`jsonb_build_object('runtimeContext', jsonb_build_object(
'generation', ${chatDeliveries.normalizedEvent}->'runtimeContext'->'generation',
'credentialFingerprint', ${chatDeliveries.normalizedEvent}->'runtimeContext'->'credentialFingerprint'
))`,
processedAt: chatDeliveries.processedAt,
receivedAt: chatDeliveries.receivedAt,
})
.from(chatMessageLinks)
.innerJoin(
chatDeliveries,
and(
eq(chatDeliveries.id, chatMessageLinks.deliveryId),
eq(chatDeliveries.companyId, chatMessageLinks.companyId),
eq(chatDeliveries.endpointId, chatMessageLinks.endpointId),
eq(chatDeliveries.conversationId, chatMessageLinks.conversationId),
),
)
.innerJoin(
chatConversations,
and(
eq(chatConversations.id, chatMessageLinks.conversationId),
eq(chatConversations.companyId, chatMessageLinks.companyId),
eq(chatConversations.endpointId, chatMessageLinks.endpointId),
),
)
.innerJoin(
chatEndpoints,
and(
eq(chatEndpoints.id, chatConversations.endpointId),
eq(chatEndpoints.companyId, chatConversations.companyId),
),
)
.where(
and(
eq(chatMessageLinks.companyId, binding.companyId),
eq(chatMessageLinks.direction, "inbound"),
inArray(chatMessageLinks.commentId, commentIds),
eq(chatDeliveries.state, "processed"),
eq(chatConversations.issueId, binding.issueId),
inArray(chatConversations.state, ["active", "waiting"]),
eq(
chatEndpoints.provider,
provider as typeof chatEndpoints.$inferSelect.provider,
),
eq(chatEndpoints.assignedAgentId, binding.agentId),
inArray(chatEndpoints.status, ["active", "verifying"]),
),
);
const links = await (lockMode === "read"
? linksQuery
: lockMode === "nonblocking"
? linksQuery.for("update", { noWait: true })
: linksQuery.for("update"));
const linkedCommentIds = new Set(links.map((row) => row.commentId));
const conversationIds = new Set(links.map((row) => row.conversation.id));
const endpointIds = new Set(links.map((row) => row.endpoint.id));
if (
links.length === 0 ||
conversationIds.size !== 1 ||
endpointIds.size !== 1 ||
!commentIds.every((id) => linkedCommentIds.has(id)) ||
links.some((row) => !row.principalId)
) {
throw new Error("paperclip_runner_chat_attachment_binding_denied");
}
const conversation = links[0]!.conversation;
const endpoint = links[0]!.endpoint;
if (!currentSetupTestAcceptsBoundDeliveries(endpoint, links)) {
throw new Error("paperclip_runner_chat_attachment_binding_denied");
}
const resource = conversation.resourceId
? await (() => {
const query = tx
.select({
availability: chatEndpointResources.availability,
type: chatEndpointResources.type,
enabled: chatEndpointResources.enabled,
})
.from(chatEndpointResources)
.where(
and(
eq(chatEndpointResources.id, conversation.resourceId!),
eq(chatEndpointResources.companyId, binding.companyId),
eq(chatEndpointResources.endpointId, endpoint.id),
),
);
return lockMode === "read"
? query.then((rows) => rows[0] ?? null)
: lockMode === "nonblocking"
? query
.for("update", { noWait: true })
.then((rows) => rows[0] ?? null)
: query.for("update").then((rows) => rows[0] ?? null);
})()
: null;
if (!destinationAllowed(endpoint, conversation, resource)) {
throw new Error("paperclip_runner_chat_attachment_destination_denied");
}
for (const principalId of new Set(links.map((row) => row.principalId!))) {
if (!(await principalAuthorized(tx, endpoint, principalId, lockMode))) {
throw new Error("paperclip_runner_chat_attachment_principal_denied");
}
}
return { conversationId: conversation.id, endpointId: endpoint.id };
}
function externalChatWaitCandidate(
contextSnapshot: unknown,
binding: ChatReuseBinding,
): { provider: string; commentIds: string[] } | null {
const context = record(contextSnapshot);
const source = typeof context.source === "string" ? context.source : "";
const provider = boundExternalChatProvider(source);
const commentIds = wakeCommentIds(context);
const wake = record(context.paperclipWake);
const wakeIssue = record(wake.issue);
const payloadCommentIds = Array.isArray(wake.commentIds)
? wake.commentIds.filter(
(value): value is string =>
typeof value === "string" && value.trim().length > 0,
)
: [];
if (
!provider ||
!(
(context.paperclipHarnessCheckedOut === true &&
wake.checkedOutByHarness === true) ||
(context.paperclipExternalChatExecutionBound === true &&
wake.externalChatExecutionBound === true)
) ||
commentIds.length === 0 ||
wake.externalChatProvider !== provider ||
wakeIssue.id !== binding.issueId ||
payloadCommentIds.length !== commentIds.length ||
payloadCommentIds.some((id, index) => id !== commentIds[index])
) {
return null;
}
return { provider, commentIds };
}
/**
* Authorize a no-work external-chat wait from the same durable boundary used
* by historical file reuse. The nested wake payload proves that this was a
* server-built chat turn; current destination and principal policy remain
* authoritative at the instant the status decision commits.
*/
export async function resolveExternalChatResponseWaitAuthorizationInTransaction(
tx: Db,
binding: ChatReuseBinding,
lockMode: AuthorizationLockMode = "blocking",
): Promise<ExternalChatResponseWaitAuthorization> {
// Preserve the native mutation lock order (issue -> run -> actor). The
// status committer already holds the issue lock; PostgreSQL treats this
// repeated lock as a no-op.
const issueQuery = tx
.select({
assigneeAgentId: issues.assigneeAgentId,
executionRunId: issues.executionRunId,
})
.from(issues)
.where(
and(
eq(issues.id, binding.issueId),
eq(issues.companyId, binding.companyId),
),
);
const [issue] = await (lockMode === "read"
? issueQuery.limit(1)
: lockMode === "nonblocking"
? issueQuery.for("update", { noWait: true }).limit(1)
: issueQuery.for("update").limit(1));
const runQuery = tx
.select(responseWaitRun)
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.id, binding.runId),
eq(heartbeatRuns.companyId, binding.companyId),
eq(heartbeatRuns.agentId, binding.agentId),
eq(heartbeatRuns.nativeIssueId, binding.issueId),
eq(heartbeatRuns.runtimeMode, "native"),
),
);
const [run] = await (lockMode === "read"
? runQuery.limit(1)
: lockMode === "nonblocking"
? runQuery.for("update", { noWait: true }).limit(1)
: runQuery.for("update").limit(1));
const answerContext =
run && record(run.contextSnapshot).source === "issue.interaction.respond"
? await resolveExternalChatQuestionResponse(
tx,
binding,
run.contextSnapshot,
lockMode,
)
: null;
if (
run &&
record(run.contextSnapshot).source === "issue.interaction.respond" &&
record(run.contextSnapshot).paperclipExternalChatQuestionResponse &&
!answerContext
)
return "revoked";
const directContext = answerContext?.authorizationContext ?? run?.contextSnapshot;
const inheritedReviewWait = run && !externalChatWaitCandidate(directContext, binding)
? await readInheritedGitHubReviewWait(tx, run) : null;
if (inheritedReviewWait?.kind === "revoked") return "revoked";
const authorizationContext = inheritedReviewWait?.context ?? directContext;
const candidate = run ? externalChatWaitCandidate(authorizationContext, binding) : null;
if (!candidate) return "not_applicable";
const actorQuery = tx
.select({ status: agents.status })
.from(agents)
.where(
and(
eq(agents.id, binding.agentId),
eq(agents.companyId, binding.companyId),
),
);
const [actor] = await (lockMode === "read"
? actorQuery.limit(1)
: lockMode === "nonblocking"
? actorQuery.for("update", { noWait: true }).limit(1)
: actorQuery.for("update").limit(1));
if (
!issue ||
!actor ||
run.status !== "running" ||
issue.assigneeAgentId !== binding.agentId ||
issue.executionRunId !== binding.runId ||
["paused", "terminated", "pending_approval", "error"].includes(
actor.status,
)
) {
return "revoked";
}
const links = await tx
.select({
commentId: chatMessageLinks.commentId,
conversationId: chatConversations.id,
endpointId: chatEndpoints.id,
principalId: chatDeliveries.principalId,
})
.from(chatMessageLinks)
.innerJoin(
chatDeliveries,
and(
eq(chatDeliveries.id, chatMessageLinks.deliveryId),
eq(chatDeliveries.companyId, chatMessageLinks.companyId),
eq(chatDeliveries.endpointId, chatMessageLinks.endpointId),
eq(chatDeliveries.conversationId, chatMessageLinks.conversationId),
),
)
.innerJoin(
chatConversations,
and(
eq(chatConversations.id, chatMessageLinks.conversationId),
eq(chatConversations.companyId, chatMessageLinks.companyId),
eq(chatConversations.endpointId, chatMessageLinks.endpointId),
),
)
.innerJoin(
chatEndpoints,
and(
eq(chatEndpoints.id, chatConversations.endpointId),
eq(chatEndpoints.companyId, chatConversations.companyId),
),
)
.where(
and(
eq(chatMessageLinks.companyId, binding.companyId),
eq(chatMessageLinks.direction, "inbound"),
inArray(chatMessageLinks.commentId, candidate.commentIds),
eq(chatDeliveries.state, "processed"),
eq(chatConversations.issueId, binding.issueId),
eq(
chatEndpoints.provider,
candidate.provider as typeof chatEndpoints.$inferSelect.provider,
),
eq(chatEndpoints.assignedAgentId, binding.agentId),
),
);
const linkedCommentIds = new Set(links.map((row) => row.commentId));
if (
links.length === 0 ||
new Set(links.map((row) => row.conversationId)).size !== 1 ||
new Set(links.map((row) => row.endpointId)).size !== 1 ||
links.some((row) => !row.principalId) ||
!candidate.commentIds.every((id) => linkedCommentIds.has(id))
) {
return "revoked";
}
try {
await authorizeChatConversationForBoundRun(
tx,
binding,
authorizationContext,
lockMode,
);
return "authorized";
} catch (error) {
if (
error instanceof Error &&
[
"paperclip_runner_chat_attachment_binding_denied",
"paperclip_runner_chat_attachment_destination_denied",
"paperclip_runner_chat_attachment_principal_denied",
].includes(error.message)
) {
return "revoked";
}
throw error;
}
}
export async function resolveExternalChatResponseWaitAuthorization(input: {
db: Db;
binding: ChatReuseBinding;
}): Promise<ExternalChatResponseWaitAuthorization> {
return input.db.transaction((tx) =>
resolveExternalChatResponseWaitAuthorizationInTransaction(
tx as unknown as Db,
input.binding,
"read",
),
);
}
const publishedGitHubReviewWaitKey = "github-review-response:";
export function isPublishedGitHubReviewCompletionContinuation(value: unknown): boolean {
const continuation = record(value);
return continuation.kind === "response_wake" && typeof continuation.idempotencyKey === "string" &&
continuation.idempotencyKey.startsWith(publishedGitHubReviewWaitKey);
}
/** Project a wait from the same saved publication and current authority used
* by finalization. Inherited waits need an accepted tree resume or a confirmed
* reply from this run to its exact current source. An arbitrary old review does
* not assign an unanswered new turn a wait. */
export async function readPublishedGitHubReviewCompletionContinuationInTransaction(
tx: Db, binding: ChatReuseBinding, lockMode: AuthorizationLockMode = "read",
) {
const [run] = await tx.select(responseWaitRun).from(heartbeatRuns).where(and(
eq(heartbeatRuns.id, binding.runId), eq(heartbeatRuns.companyId, binding.companyId),
eq(heartbeatRuns.agentId, binding.agentId), eq(heartbeatRuns.nativeIssueId, binding.issueId),
eq(heartbeatRuns.runtimeMode, "native"),
)).limit(1);
if (!run || run.status !== "running") return null;
const wait = await readInheritedGitHubReviewWait(tx, run);
if (wait?.kind !== "published") return null;
if (wait.reviewRunId !== run.id) {
const [identity] = await tx.select({ cause: runIdentityContexts.cause, parentContextId: runIdentityContexts.parentContextId })
.from(runIdentityContexts).where(and(eq(runIdentityContexts.id, run.activeIdentityContextId!),
eq(runIdentityContexts.companyId, run.companyId), eq(runIdentityContexts.runId, run.id),
eq(runIdentityContexts.status, "accepted"))).limit(1);
const resumed = run.contextSnapshot?.source === "issue.tree_resume" && identity?.cause === "issue_tree_resumed" &&
Boolean(identity.parentContextId);
if (!resumed && !(run.contextSnapshot?.source === "chat:github" &&
await githubRunReplyState(tx, { ...binding, endpointId: wait.endpointId,
sourceCommentId: wait.context.wakeCommentId, requireSettled: true }) === "confirmed")) return null;
}
if (await resolveExternalChatResponseWaitAuthorizationInTransaction(tx, binding, lockMode) !== "authorized") return null;
return { kind: "response_wake" as const, summary: "Wait for the next authorized GitHub response to the published review.",
idempotencyKey: `${publishedGitHubReviewWaitKey}${run.id}:${wait.reviewId}:${wait.headSha}` };
}
export async function readPublishedGitHubReviewCompletionContinuation(input: { db: Db; binding: ChatReuseBinding }) {
return input.db.transaction(tx => readPublishedGitHubReviewCompletionContinuationInTransaction(tx as unknown as Db, input.binding));
}
async function sourceLineageExists(
tx: Db,
binding: ChatReuseBinding,
conversation: AuthorizedConversation,
source: Pick<
ChatAttachmentReuseSource,
"sourceCommentId" | "attachmentId"
> & {
parentCommentId: string;
},
): Promise<boolean> {
const [inbound] = await tx
.select({ providerEventId: chatDeliveries.providerEventId })
.from(chatMessageLinks)
.innerJoin(
chatDeliveries,
and(
eq(chatDeliveries.id, chatMessageLinks.deliveryId),
eq(chatDeliveries.companyId, chatMessageLinks.companyId),
eq(chatDeliveries.endpointId, chatMessageLinks.endpointId),
eq(chatDeliveries.conversationId, chatMessageLinks.conversationId),
),
)
.where(
and(
eq(chatMessageLinks.companyId, binding.companyId),
eq(chatMessageLinks.endpointId, conversation.endpointId),
eq(chatMessageLinks.conversationId, conversation.conversationId),
eq(chatMessageLinks.direction, "inbound"),
eq(chatMessageLinks.commentId, source.sourceCommentId),
eq(chatDeliveries.state, "processed"),
),
)
.limit(1);
if (inbound && source.parentCommentId === source.sourceCommentId) {
const [lifecycle] = await tx
.select({ id: chatDeliveries.id })
.from(chatDeliveries)
.where(
and(
eq(chatDeliveries.companyId, binding.companyId),
eq(chatDeliveries.endpointId, conversation.endpointId),
eq(chatDeliveries.conversationId, conversation.conversationId),
inArray(chatDeliveries.eventKind, [
"message_updated",
"message_deleted",
"message_restored",
]),
eq(chatDeliveries.state, "processed"),
sql`${chatDeliveries.normalizedEvent}->'message'->>'targetProviderEventId' = ${inbound.providerEventId}`,
),
)
.limit(1);
return !lifecycle;
}
const publications = await tx
.select({
payload: chatPublications.payload,
providerMessageId: chatPublications.providerMessageId,
publishedAt: chatPublications.publishedAt,
})
.from(chatPublications)
.where(
and(
eq(chatPublications.companyId, binding.companyId),
eq(chatPublications.endpointId, conversation.endpointId),
eq(chatPublications.conversationId, conversation.conversationId),
eq(chatPublications.issueId, binding.issueId),
eq(chatPublications.commentId, source.sourceCommentId),
eq(chatPublications.state, "published"),
),
);
const matchingPublications = publications.filter(
(row) =>
row.providerMessageId !== null &&
row.publishedAt !== null &&
publicationAttachmentIds(row.payload).includes(source.attachmentId),
);
if (matchingPublications.length === 0) return false;
for (const publication of matchingPublications) {
const [lifecycle] = await tx
.select({ id: chatDeliveries.id })
.from(chatDeliveries)
.where(
and(
eq(chatDeliveries.companyId, binding.companyId),
eq(chatDeliveries.endpointId, conversation.endpointId),
eq(chatDeliveries.conversationId, conversation.conversationId),
inArray(chatDeliveries.eventKind, [
"message_updated",
"message_deleted",
"message_restored",
]),
eq(chatDeliveries.state, "processed"),
eq(
sql<string>`${chatDeliveries.normalizedEvent}->'message'->>'providerMessageId'`,
publication.providerMessageId!,
),
),
)
.limit(1);
if (!lifecycle) return true;
}
return false;
}
const HISTORICAL_ATTACHMENT_FILE_EXTENSIONS: ReadonlyMap<string, string> =
new Map([
["image/png", "png"],
["image/jpeg", "jpg"],
["image/jpg", "jpg"],
["image/webp", "webp"],
["image/gif", "gif"],
["audio/mpeg", "mp3"],
["audio/mp4", "m4a"],
["audio/ogg", "ogg"],
["audio/wav", "wav"],
["audio/webm", "webm"],
["video/mp4", "mp4"],
["video/webm", "webm"],
["video/quicktime", "mov"],
["video/x-m4v", "m4v"],
["application/pdf", "pdf"],
["application/zip", "zip"],
["application/json", "json"],
["text/plain", "txt"],
["text/markdown", "md"],
["text/csv", "csv"],
["text/html", "html"],
]);
async function loadSource(
tx: Db,
binding: ChatReuseBinding,
conversation: AuthorizedConversation,
sourceCommentId: string,
attachmentId: string,
allowEmpty = false,
): Promise<ChatAttachmentReuseSource> {
const [sourceComment] = await tx
.select({ id: issueComments.id })
.from(issueComments)
.where(
and(
eq(issueComments.id, sourceCommentId),
eq(issueComments.companyId, binding.companyId),
eq(issueComments.issueId, binding.issueId),
isNull(issueComments.deletedAt),
),
)
.for("update")
.limit(1);
if (!sourceComment) {
throw new Error("paperclip_runner_chat_attachment_source_denied");
}
const [row] = await tx
.select({
attachmentId: issueAttachments.id,
parentCommentId: issueComments.id,
filename: assets.originalFilename,
contentType: assets.contentType,
byteSize: assets.byteSize,
sha256: assets.sha256,
objectKey: assets.objectKey,
createdAt: issueAttachments.createdAt,
})
.from(issueAttachments)
.innerJoin(
assets,
and(
eq(assets.id, issueAttachments.assetId),
eq(assets.companyId, binding.companyId),
),
)
.innerJoin(
issueComments,
and(
eq(issueComments.id, issueAttachments.issueCommentId),
eq(issueComments.companyId, binding.companyId),
eq(issueComments.issueId, binding.issueId),
isNull(issueComments.deletedAt),
),
)
.where(
and(
eq(issueAttachments.id, attachmentId),
eq(issueAttachments.companyId, binding.companyId),
eq(issueAttachments.issueId, binding.issueId),
),
)
.for("update")
.limit(1);
if (
!row ||
row.byteSize < (allowEmpty ? 0 : 1) ||
row.byteSize > MAX_ATTACHMENT_BYTES ||
!isAllowedContentType(normalizeContentType(row.contentType)) ||
!/^[a-f0-9]{64}$/iu.test(row.sha256)
) {
throw new Error("paperclip_runner_chat_attachment_source_denied");
}
const sourceWithParent = {
sourceCommentId,
attachmentId: row.attachmentId,
parentCommentId: row.parentCommentId,
// Provider photos/voice messages legitimately have no original filename.
// Derive display metadata from the already-validated MIME and opaque id;
// never infer a storage path or change the selected source bytes.
filename: row.filename?.trim()
? row.filename
: `attachment-${row.attachmentId}.${HISTORICAL_ATTACHMENT_FILE_EXTENSIONS.get(normalizeContentType(row.contentType)) ?? "bin"}`,
contentType: normalizeContentType(row.contentType),
byteSize: row.byteSize,
sha256: row.sha256,
objectKey: row.objectKey,
createdAt: row.createdAt,
};
if (
!(await sourceLineageExists(tx, binding, conversation, sourceWithParent))
) {
throw new Error("paperclip_runner_chat_attachment_source_denied");
}
const { parentCommentId: _parentCommentId, ...source } = sourceWithParent;
return source;
}
export async function listAuthorizedChatAttachments(input: {
db: Db;
binding: ChatReuseBinding;
sourceCommentId?: string | null;
cursor?: string | null;
limit: number;
}): Promise<ListedChatAttachmentPage> {
return input.db.transaction(async (tx) => {
const [current] = await tx
.select({ run: heartbeatRuns, actorStatus: agents.status })
.from(heartbeatRuns)
.innerJoin(
issues,
and(
eq(issues.id, input.binding.issueId),
eq(issues.companyId, heartbeatRuns.companyId),
),
)
.innerJoin(
agents,
and(
eq(agents.id, input.binding.agentId),
eq(agents.companyId, heartbeatRuns.companyId),
),
)
.where(
and(
eq(heartbeatRuns.id, input.binding.runId),
eq(heartbeatRuns.companyId, input.binding.companyId),
eq(heartbeatRuns.agentId, input.binding.agentId),
eq(heartbeatRuns.nativeIssueId, input.binding.issueId),
eq(heartbeatRuns.runtimeMode, "native"),
eq(heartbeatRuns.status, "running"),
eq(issues.assigneeAgentId, input.binding.agentId),
eq(issues.executionRunId, input.binding.runId),
),
)
.for("update")
.limit(1);
if (
!current ||
["paused", "terminated", "pending_approval", "error"].includes(
current.actorStatus,
)
) {
throw new Error("paperclip_runner_tool_binding_not_authorized");
}
const conversation = await authorizeChatConversationForBoundRun(
tx as unknown as Db,
input.binding,
current.run.contextSnapshot,
);
const sourceFilter = input.sourceCommentId ?? null;
const cursor = decodeListCursor(
input.cursor,
conversation.conversationId,
sourceFilter,
);
const rawCandidates = await tx.execute(sql<{
attachment_id: string;
source_comment_id: string;
created_at: Date | string;
}>`
with candidate_pairs as (
select
attachment.id as attachment_id,
link.comment_id as source_comment_id,
attachment.created_at
from issue_attachments attachment
join issue_comments parent_comment
on parent_comment.id = attachment.issue_comment_id
and parent_comment.company_id = ${input.binding.companyId}::uuid
and parent_comment.issue_id = ${input.binding.issueId}::uuid
and parent_comment.deleted_at is null
join chat_message_links link
on link.company_id = ${input.binding.companyId}::uuid
and link.endpoint_id = ${conversation.endpointId}::uuid
and link.conversation_id = ${conversation.conversationId}::uuid
and link.direction = 'inbound'
and link.comment_id = attachment.issue_comment_id
join chat_deliveries delivery
on delivery.id = link.delivery_id
and delivery.company_id = link.company_id
and delivery.endpoint_id = link.endpoint_id
and delivery.conversation_id = link.conversation_id
and delivery.state = 'processed'
where attachment.company_id = ${input.binding.companyId}::uuid
and attachment.issue_id = ${input.binding.issueId}::uuid
and (${sourceFilter}::uuid is null or link.comment_id = ${sourceFilter}::uuid)
and not exists (
select 1
from chat_deliveries lifecycle
where lifecycle.company_id = ${input.binding.companyId}::uuid
and lifecycle.endpoint_id = ${conversation.endpointId}::uuid
and lifecycle.conversation_id = ${conversation.conversationId}::uuid
and lifecycle.state = 'processed'
and lifecycle.event_kind in ('message_updated', 'message_deleted', 'message_restored')
and lifecycle.normalized_event->'message'->>'targetProviderEventId' = delivery.provider_event_id
)
union all
select
attachment.id as attachment_id,
publication.comment_id as source_comment_id,
attachment.created_at
from issue_attachments attachment
join issue_comments parent_comment
on parent_comment.id = attachment.issue_comment_id
and parent_comment.company_id = ${input.binding.companyId}::uuid
and parent_comment.issue_id = ${input.binding.issueId}::uuid
and parent_comment.deleted_at is null
join chat_publications publication
on publication.company_id = ${input.binding.companyId}::uuid
and publication.endpoint_id = ${conversation.endpointId}::uuid
and publication.conversation_id = ${conversation.conversationId}::uuid
and publication.issue_id = ${input.binding.issueId}::uuid
and publication.state = 'published'
and publication.published_at is not null
and publication.provider_message_id is not null
and publication.comment_id is not null
and publication.payload->'attachmentIds' ? attachment.id::text
join issue_comments source_comment
on source_comment.id = publication.comment_id
and source_comment.company_id = publication.company_id
and source_comment.issue_id = publication.issue_id
and source_comment.deleted_at is null
where attachment.company_id = ${input.binding.companyId}::uuid
and attachment.issue_id = ${input.binding.issueId}::uuid
and (${sourceFilter}::uuid is null or publication.comment_id = ${sourceFilter}::uuid)
and not exists (
select 1
from chat_deliveries lifecycle
where lifecycle.company_id = ${input.binding.companyId}::uuid
and lifecycle.endpoint_id = ${conversation.endpointId}::uuid
and lifecycle.conversation_id = ${conversation.conversationId}::uuid
and lifecycle.state = 'processed'
and lifecycle.event_kind in ('message_updated', 'message_deleted', 'message_restored')
and lifecycle.normalized_event->'message'->>'providerMessageId' = publication.provider_message_id
)
), candidates as (
select distinct on (attachment_id)
attachment_id,
source_comment_id,
created_at
from candidate_pairs
order by attachment_id, created_at desc, source_comment_id desc
)
select attachment_id, source_comment_id, created_at
from candidates
where (
${cursor?.createdAt ?? null}::timestamptz is null
or (created_at, attachment_id, source_comment_id) <
(${cursor?.createdAt ?? null}::timestamptz, ${cursor?.attachmentId ?? null}::uuid, ${cursor?.sourceCommentIdTieBreak ?? null}::uuid)
)
order by created_at desc, attachment_id desc, source_comment_id desc
limit ${input.limit + 1}
`);
const candidates = (
Array.from(rawCandidates) as Array<{
attachment_id: string;
source_comment_id: string;
created_at: Date | string;
}>
).map((row) => ({
sourceCommentId: row.source_comment_id,
attachmentId: row.attachment_id,
createdAt:
row.created_at instanceof Date
? row.created_at
: new Date(row.created_at),
}));
const listed: ListedChatAttachment[] = [];
const scanned = candidates.slice(0, input.limit);
for (const candidate of scanned) {
if (listed.length >= input.limit) break;
try {
const source = await loadSource(
tx as unknown as Db,
input.binding,
conversation,
candidate.sourceCommentId,
candidate.attachmentId,
true,
);
listed.push({
sourceCommentId: source.sourceCommentId,
attachmentId: source.attachmentId,
filename: source.filename,
contentType: source.contentType,
byteSize: source.byteSize,
sha256: source.sha256,
createdAt: source.createdAt.toISOString(),
contentAccess: "metadata_only",
});
} catch (error) {
// Metadata listing omits deleted, oversized, or no-longer-lineaged files.
if (
!(error instanceof Error) ||
error.message !== "paperclip_runner_chat_attachment_source_denied"
) {
throw error;
}
}
}
const lastScanned = scanned.at(-1) ?? null;
const hasMore = candidates.length > input.limit;
return {
attachments: listed,
nextCursor:
hasMore && lastScanned
? encodeListCursor({
schema: "paperclip.chat-attachment-list-cursor.v1",
conversationId: conversation.conversationId,
sourceCommentId: sourceFilter,
createdAt: lastScanned.createdAt.toISOString(),
attachmentId: lastScanned.attachmentId,
sourceCommentIdTieBreak: lastScanned.sourceCommentId,
})
: null,
complete: !hasMore,
};
});
}
export async function authorizeChatAttachmentReuse(input: {
db: Db;
binding: ChatReuseBinding;
contextSnapshot: unknown;
sourceCommentId: string;
attachmentId: string;
/** Inspection may open empty files; publication retains its nonempty bound. */
allowEmpty?: boolean;
}): Promise<ChatAttachmentReuseSource> {
const conversation = await authorizeChatConversationForBoundRun(
input.db,
input.binding,
input.contextSnapshot,
);
return loadSource(
input.db,
input.binding,
conversation,
input.sourceCommentId,
input.attachmentId,
input.allowEmpty,
);
}
async function readSourceBytes(
storage: StorageService,
companyId: string,
source: ChatAttachmentReuseSource,
timeoutMs: number,
): Promise<Buffer> {
let acquisitionTimedOut = false;
let rejectAcquisition!: (error: Error) => void;
const acquisitionTimeout = new Promise<never>((_resolve, reject) => {
rejectAcquisition = reject;
});
const acquisitionTimer = setTimeout(() => {
acquisitionTimedOut = true;
rejectAcquisition(
new Error("paperclip_runner_chat_attachment_source_read_timed_out"),
);
}, timeoutMs);
acquisitionTimer.unref?.();
const objectPromise = storage
.getObject(companyId, source.objectKey)
.then((object) => {
if (acquisitionTimedOut) object.stream.destroy();
return object;
});
let object: Awaited<ReturnType<StorageService["getObject"]>>;
try {
object = await Promise.race([objectPromise, acquisitionTimeout]);
} finally {
clearTimeout(acquisitionTimer);
}
const timeout = setTimeout(() => {
object.stream.destroy(
new Error("paperclip_runner_chat_attachment_source_read_timed_out"),
);
}, timeoutMs);
timeout.unref?.();
const chunks: Buffer[] = [];
let total = 0;
try {
for await (const chunk of object.stream) {
const buffer = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
total += buffer.length;
if (total > source.byteSize || total > MAX_ATTACHMENT_BYTES) {
object.stream.destroy();
throw new Error(
"paperclip_runner_chat_attachment_source_size_mismatch",
);
}
chunks.push(buffer);
}
} finally {
clearTimeout(timeout);
}
const body = Buffer.concat(chunks);
if (
body.length !== source.byteSize ||
createHash("sha256").update(body).digest("hex") !==
source.sha256.toLowerCase()
) {
if (!object.stream.destroyed) {
object.stream.destroy();
}
throw new Error(
"paperclip_runner_chat_attachment_source_integrity_mismatch",
);
}
return body;
}
const DEFAULT_STORAGE_TIMEOUT_MS = 10_000;
async function deleteStorageObjectWithin(
storage: StorageService,
companyId: string,
objectKey: string,
timeoutMs: number,
): Promise<void> {
const deletion = storage
.deleteObject(companyId, objectKey)
.catch(() => undefined);
let timer: ReturnType<typeof setTimeout> | null = null;
try {
await Promise.race([
deletion,
new Promise<void>((resolve) => {
timer = setTimeout(resolve, timeoutMs);
timer.unref?.();
}),
]);
} finally {
if (timer) clearTimeout(timer);
}
}
async function putStorageObjectWithin(
storage: StorageService,
input: Parameters<StorageService["putFile"]>[0],
timeoutMs: number,
): Promise<Awaited<ReturnType<StorageService["putFile"]>>> {
let timedOut = false;
let rejectTimeout!: (error: Error) => void;
const timeout = new Promise<never>((_resolve, reject) => {
rejectTimeout = reject;
});
const timer = setTimeout(() => {
timedOut = true;
rejectTimeout(
new Error("paperclip_runner_chat_attachment_storage_write_timed_out"),
);
}, timeoutMs);
timer.unref?.();
const write = storage.putFile(input).then((stored) => {
if (timedOut) {
void deleteStorageObjectWithin(
storage,
input.companyId,
stored.objectKey,
timeoutMs,
);
}
return stored;
});
try {
return await Promise.race([write, timeout]);
} finally {
clearTimeout(timer);
}
}
export async function prepareReusedChatAttachment(input: {
db: Db;
binding: ChatReuseBinding;
source: ChatAttachmentReuseSource;
title: string;
storage?: StorageService;
storageTimeoutMs?: number;
}): Promise<PreparedReusedChatAttachment> {
const [issue] = await input.db
.select({ statusVersion: issues.statusVersion })
.from(issues)
.innerJoin(
heartbeatRuns,
and(
eq(heartbeatRuns.id, input.binding.runId),
eq(heartbeatRuns.companyId, issues.companyId),
eq(heartbeatRuns.nativeIssueId, issues.id),
),
)
.where(
and(
eq(issues.id, input.binding.issueId),
eq(issues.companyId, input.binding.companyId),
eq(issues.assigneeAgentId, input.binding.agentId),
eq(issues.executionRunId, input.binding.runId),
eq(heartbeatRuns.agentId, input.binding.agentId),
eq(heartbeatRuns.status, "running"),
eq(heartbeatRuns.runtimeMode, "native"),
),
)
.limit(1);
if (!issue)
throw new Error("paperclip_runner_chat_attachment_binding_denied");
const storage = input.storage ?? getStorageService();
const storageTimeoutMs =
typeof input.storageTimeoutMs === "number" &&
Number.isFinite(input.storageTimeoutMs) &&
input.storageTimeoutMs > 0
? input.storageTimeoutMs
: DEFAULT_STORAGE_TIMEOUT_MS;
const body = await readSourceBytes(
storage,
input.binding.companyId,
input.source,
storageTimeoutMs,
);
const stored = await putStorageObjectWithin(
storage,
{
companyId: input.binding.companyId,
namespace: `issues/${input.binding.issueId}`,
originalFilename: input.source.filename,
contentType: input.source.contentType,
body,
},
storageTimeoutMs,
);
try {
if (
stored.byteSize !== body.length ||
stored.sha256.toLowerCase() !== input.source.sha256.toLowerCase() ||
stored.contentType !== input.source.contentType
) {
throw new Error("paperclip_runner_chat_attachment_storage_mismatch");
}
const attachment = await issueService(input.db).createAttachment({
issueId: input.binding.issueId,
provider: stored.provider,
objectKey: stored.objectKey,
contentType: stored.contentType,
byteSize: stored.byteSize,
sha256: stored.sha256,
originalFilename: stored.originalFilename,
createdByAgentId: input.binding.agentId,
createdByRunId: input.binding.runId,
});
if (
!attachment.artifactWorkProductId ||
attachment.originatingRunId !== input.binding.runId
) {
throw new Error("paperclip_runner_chat_attachment_origin_not_persisted");
}
const [workProduct] = await input.db
.select({ metadata: issueWorkProducts.metadata })
.from(issueWorkProducts)
.where(
and(
eq(issueWorkProducts.id, attachment.artifactWorkProductId),
eq(issueWorkProducts.companyId, input.binding.companyId),
eq(issueWorkProducts.issueId, input.binding.issueId),
eq(issueWorkProducts.createdByRunId, input.binding.runId),
),
)
.for("update")
.limit(1);
if (!workProduct) {
throw new Error("paperclip_runner_chat_attachment_origin_not_persisted");
}
await input.db
.update(issueWorkProducts)
.set({
title: input.title,
metadata: {
...record(workProduct.metadata),
reusedFromAttachmentId: input.source.attachmentId,
reusedFromCommentId: input.source.sourceCommentId,
reusedFromSha256: input.source.sha256,
},
updatedAt: new Date(),
})
.where(
and(
eq(issueWorkProducts.id, attachment.artifactWorkProductId),
eq(issueWorkProducts.companyId, input.binding.companyId),
eq(issueWorkProducts.issueId, input.binding.issueId),
eq(issueWorkProducts.createdByRunId, input.binding.runId),
),
);
const comment = await issueService(input.db).addComment(
input.binding.issueId,
`Prepared ${input.title} for this response.`,
{ agentId: input.binding.agentId, runId: input.binding.runId },
{
attachmentIds: [attachment.id],
authorizationReason: "paperclip_runner_protocol",
},
input.db,
);
return {
result: {
commandId: `chat-attachment-reused:${attachment.id}`,
disposition: "applied",
stateRevision: issue.statusVersion,
entityRefs: [
attachment.id,
attachment.artifactWorkProductId,
comment.id,
],
scheduledWakeIds: [],
source: {
commentId: input.source.sourceCommentId,
attachmentId: input.source.attachmentId,
sha256: input.source.sha256,
},
prepared: {
attachmentId: attachment.id,
workProductId: attachment.artifactWorkProductId,
commentId: comment.id,
filename: input.source.filename,
byteSize: body.length,
sha256: attachment.sha256,
},
},
rollbackDefinitePreCommitFailure: async () => {
await deleteStorageObjectWithin(
storage,
input.binding.companyId,
stored.objectKey,
storageTimeoutMs,
);
},
};
} catch (error) {
await deleteStorageObjectWithin(
storage,
input.binding.companyId,
stored.objectKey,
storageTimeoutMs,
);
throw error;
}
}