Files
PaperClipAI/server/src/services/chat-channels/publication-replacement.ts
T
DottaandPaperclip 0ead74b4f3 feat(slack): show useful progress without extra messages (#20827)
## Thinking Path

> - Paperclip helps people manage AI agents and their work.
> - Chat connectors show agent work in the source conversation.
> - Slack progress uses a small set of general status messages.
> - Native runtimes already mark some completed assistant notes as
public progress.
> - The server can select those notes and control their size and rate.
> - This pull request keeps one progress message and sends the final
answer as a new reply.
> - Users can follow useful changes without reading each tool event.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Automatic progress updates in Slack task threads.

**Current behavior**

Users see general messages such as “Maya is making progress”. The first
answer can replace that message. A source update that arrives during the
progress delay can be lost if no later event arrives.

**Proposed behavior**

Show the latest short public note in one message. Wait 10 seconds before
the first update and at least 30 seconds between updates. Recover
pending work after a restart. Send the final answer as a fresh reply.

**Reason and benefit**

Users can see what the agent checks and what it found. The thread stays
short. The model needs no new tool call or summary model.

Related change: [resolved-card chat
bindings](https://github.com/paperclipai/paperclip/pull/20825). This
branch includes that master revision. The progress change uses the
existing source authority checks.

## What Changed

- Select completed `agentMessage` items from the normalized PRP
`progress` channel. Keep unknown channels, reasoning, tool output, and
raw provider fields out of progress text.
- Sanitize each note. Keep at most two sentences and 400 characters. Add
a task link when a public board URL is available.
- Prefer public notes over general tool status. Remove repeated text and
combine bursts into the latest pending note.
- Use run events and the publication outbox as saved state. Use the
shared deadline worker for the grace period and update delay. A delivery
receipt wakes the worker when an earlier update is still in flight.
Settled history does not poll.
- Apply the run's registered-secret filter before text selection or
storage. Reuse the latest usable note within a sweep. Invalidate the
cache when the source event or secret registry changes.
- Reuse the exact source-bound Slack fast-response receipt when it
replaced the working placeholder. Check its current outbound message
link before editing it.
- Send Slack answers as new replies. After delivery, queue a separate
cleanup edit for a generic working message. Keep its current outbound
link as the edit authority. Leave useful public notes in place. Cancel
pending public progress when an answer, question, or terminal run takes
over. Keep the existing delivery policy for other connectors.
- Add one short Slack prompt instruction. Update product documentation
and regression tests.

The Slack module owns public-note selection, cadence, lane eligibility,
and cleanup. The shared progress service holds its existing issue and
run locks and calls a narrow provider policy. The secret registry
supplies redaction and a version token for the per-sweep note cache. The
existing outbox owns delivery, retries, and authority checks. The Slack
module owns receipt reuse. Shared delivery changes supply the receipt
wake and select the Slack reply policy. No schema or configuration
change is required.

## Verification

- The initial content-selection regression failed before the
implementation and passed after it.
- The follow-up reproduced two review defects before the fixes. A
registered synthetic secret appeared in saved progress. A generic
working message stayed active after the answer. Both regressions now
pass.
- The final content, database-worker, query, protocol-boundary, and
redaction selection passes 78 tests across four files.
- The earlier affected transport selection passes all 65 cases. It
covers public progress, native cadence, separate answers, Slack and
Telegram retry isolation, FIFO restart, and cleanup ownership. Its first
run passed 64 cases and found one old assertion that rejected the
intended cleanup edit. The corrected assertion checks the exact
successor message ID.
- Full server typecheck passes: `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm
--filter @paperclipai/server typecheck`. An earlier attempt found two
transaction type errors. The helper contracts now accept their actual
database operations. The final run passes all source and test layers.
- The broad Slack selection stopped after 189 passing cases when the
sandbox killed its worker with SIGKILL. That attempt is not a pass. The
focused transport run above supplies affected local coverage. Required
CI must supply broad coverage on the final head.
- `pnpm verify --explain` returns exit 2 because the product
documentation change requires broad CI coverage. No full local suite was
started.
- Preflight installed dependencies and built the SDK. Cargo is absent,
so local Rust checks could not run. No Rust source changed.
- [CI on
`02552bb584`](https://github.com/paperclipai/paperclip/actions/runs/43493597242)
passed build, typecheck, policy, all browser shards, Runner/Rust checks,
and the other test lanes. One chat shard failed in three Slack cases.
Each expected one edit after an answer was already sent. The intended
cleanup adds a “Response sent.” edit before the later failure notice.
All three failures reproduced locally. The test assertions now check
both exact edit texts and the original run's message ID. They also check
that another sweep adds no edits.
- The latest local run passes all 85 selected transport cases. It
includes all 18 Slack/Telegram answer and failure orders, plus progress,
deferred FIFO, retry isolation, and cleanup coverage. Command: `pnpm
exec vitest run server/src/__tests__/chat-channels.integration.test.ts
-t 'preserves selected answer and failure lanes|public progress|native
progress|keeps intermediate.*agent comments|keeps
a.*retry.*queue|FIFO'`.
- [Final
CI](https://github.com/paperclipai/paperclip/actions/runs/43499797545)
passed on `55ec4c053fe5263d9033d807777dbf2e679407e3`. All 52 executed
jobs passed. This includes build, typecheck, general tests, serialized
tests, all eight browser shards, Runner/Rust checks, and the required
aggregates. The affected chat shard now passes. The test-file selection
job was skipped because this run used full coverage.
- [Greptile](https://github.com/paperclipai/paperclip/runs/132142301124)
completed on the same head with 5/5 and no new findings. All four inline
review threads are resolved. The review job and Socket checks passed. Of
61 check runs, 57 succeeded, three were skipped, and one was neutral. No
checks remain pending or failed. The PR is mergeable with a clean state.
- [Superagent Security
Scan](https://github.com/paperclipai/paperclip/runs/132142345827)
returned a neutral result with “PR scan workflow failed”. It supplied no
security verdict. Do not count it as a successful scan.
- Live Slack notification and unread behavior has not been tested.
Service tests use real PostgreSQL and a fake provider runtime.

## Risks

- Only runtimes that map commentary to the explicit public `progress`
channel supply contextual prose. Other runtimes retain general status
messages.
- Slack final answers now create a fresh reply. This is an intended
delivery change.
- Public commentary must suit the current conversation audience. Text
sanitization adds protection; it does not authorize sharing unrelated
private material.
- Source events and outbox rows supply recovery state. No new timer
table or recurring idle scan is added.
- The existing large integration suite retains its provider fixtures and
lifecycle assertions. No source file crosses the 1,000-line threshold in
this change.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning and code tools. The exact
deployment ID was not supplied.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-13 15:48:05 -06:00

968 lines
38 KiB
TypeScript

import { slackPublicProgressReceiptToReplace, slackPublicProgressSupersessionReason, isSlackPublicProgress, slackProgressCompletionTarget } from "./slack/public-progress.js";
import { imessageProgressSupersessionReason } from "./imessage-photon/publications.js";
import { fastResponseRequests } from "@paperclipai/db";
import {
and,
desc,
eq,
gt,
inArray,
isNotNull,
isNull,
like,
ne,
or,
sql,
} from "drizzle-orm";
import { alias } from "drizzle-orm/pg-core";
import type { Db } from "@paperclipai/db";
import {
agentWakeupRequests,
chatEndpoints,
chatMessageLinks,
chatPublications,
heartbeatRuns,
issueComments,
issueThreadInteractions,
issues,
} from "@paperclipai/db";
import type {
SafeChatPublicationPayload,
} from "@paperclipai/shared";
import {
isUuidLike,
} from "@paperclipai/shared";
import {
NativeChatReviewPresentationContentionError,
} from "../native-runtime/native-chat-review-presentation.js";
import { isExternalChatWaitAuthorizationContention } from "../native-runtime/chat-attachment-reuse.js";
import {
inboundWakePublicationKey,
parseInboundWakePublicationKey,
} from "../chat-inbound-wakeup-publications.js";
type EndpointRow = typeof chatEndpoints.$inferSelect;
type DbOrTransaction = Db | Parameters<Parameters<Db["transaction"]>[0]>[0];
type ChatPublicationReplacementOptions = {
endpointRecord: (endpointId: string) => Promise<{ endpoint: EndpointRow } | null>;
authorizeRetainedChatSourcePublication: (database: DbOrTransaction, publication: typeof chatPublications.$inferSelect) => Promise<boolean>;
};
export function createChatPublicationReplacement(db: Db, options: ChatPublicationReplacementOptions) {
const { endpointRecord, authorizeRetainedChatSourcePublication } = options;
function runIdFromMilestonePublication(
publication: typeof chatPublications.$inferSelect,
): string | null {
if (!publication.payload.progressState) return null;
const match =
/^run:([^:]+):(?:queued|working|waiting_for_input|completed|failed):/.exec(
publication.idempotencyKey,
);
const runId = match?.[1] ?? null;
// This identifier is also compared against a UUID column below. Old or
// manually repaired rows must not be allowed to turn the global
// publication sweep into a PostgreSQL cast error.
return isUuidLike(runId) ? runId : null;
}
async function runOwnershipMilestoneSupersessionReason(
publication: typeof chatPublications.$inferSelect,
): Promise<string | null> {
const completion = await slackProgressCompletionTarget(db, publication);
if (completion === null) return "Slack progress message is no longer replaceable";
const progress = publication.payload.progressState;
const typingReason = imessageProgressSupersessionReason(progress);
if (typingReason) {
const endpoint = await endpointRecord(publication.endpointId);
if (endpoint?.endpoint.provider === "imessage-photon") return typingReason;
}
if (
!progress ||
!["queued", "working", "waiting_for_input"].includes(progress)
) {
return null;
}
const runId = runIdFromMilestonePublication(publication);
if (!runId) return null;
const run = await db.transaction(async (tx) => {
// Use native finalization's issue -> run lock order. If terminalization
// won, the current status suppresses this obsolete provider update. If
// this short authorization snapshot wins, the progress was still true
// at its durable send boundary and the later final remains authoritative.
const currentIssue = await tx
.select({ id: issues.id })
.from(issues)
.where(
and(
eq(issues.id, publication.issueId),
eq(issues.companyId, publication.companyId),
),
)
.for("update")
.then((rows) => rows[0] ?? null);
if (!currentIssue) return null;
return tx
.select({
status: heartbeatRuns.status,
errorCode: heartbeatRuns.errorCode,
})
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.id, runId),
eq(heartbeatRuns.companyId, publication.companyId),
sql`${heartbeatRuns.contextSnapshot} ->> 'issueId' = ${publication.issueId}`,
),
)
.for("update")
.then((rows) => rows[0] ?? null);
});
if ((await endpointRecord(publication.endpointId))?.endpoint.provider === "slack") {
const reason = await slackPublicProgressSupersessionReason(db, publication, runId);
if (reason) return reason;
}
const baseProgressKey = `run:${runId}:${progress}:${publication.endpointId}`;
const nativeProgressPrefix = `run:${runId}:working:${publication.endpointId}:native:`;
const isTerminalizableProgress =
(progress === "queued" || progress === "working") &&
(publication.idempotencyKey === baseProgressKey ||
publication.idempotencyKey.startsWith(nativeProgressPrefix));
if (
isTerminalizableProgress &&
run &&
["succeeded", "interrupted", "failed", "cancelled", "timed_out"].includes(
run.status,
)
) {
return "Run reached a terminal state before progress delivery";
}
// Match only the closed ownership-attention projection. Historical progress
// without an extant run retains its existing delivery semantics.
const ownershipBlocked =
run?.status === "running" &&
[
"native_execution_ownership_unverified",
"native_adopted_runner_authentication_timeout",
].includes(run.errorCode ?? "");
if (progress === "waiting_for_input") {
return ownershipBlocked
? null
: "Run ownership attention no longer applies to the bound task";
}
return ownershipBlocked
? "Run progress was superseded by ownership recovery attention"
: null;
}
async function providerProgressLaneConsumed(
publication: typeof chatPublications.$inferSelect,
providerMessageId: string,
): Promise<boolean> {
// Published progress rows retain their old provider ID after an edit.
// Consult that exact message's current outbound link, not the conversation
// tail: an authored answer or failure has consumed the lane, while an
// interleaved task-status update may still be replaced normally.
const [consumed] = await db
.select({ id: chatPublications.id })
.from(chatMessageLinks)
.innerJoin(
chatPublications,
and(
eq(chatPublications.id, chatMessageLinks.publicationId),
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
eq(chatPublications.providerMessageId, providerMessageId),
eq(chatPublications.state, "published"),
),
)
.where(
and(
eq(chatMessageLinks.companyId, publication.companyId),
eq(chatMessageLinks.endpointId, publication.endpointId),
eq(chatMessageLinks.conversationId, publication.conversationId),
eq(chatMessageLinks.direction, "outbound"),
eq(chatMessageLinks.providerMessageId, providerMessageId),
or(
and(
isNotNull(chatPublications.commentId),
sql`${chatPublications.payload}->>'progressState' is null`,
),
sql`${chatPublications.payload}->>'progressState' = 'failed'`,
),
),
)
.limit(1);
return Boolean(consumed);
}
async function committedResponseMayReplaceFailure(
publication: typeof chatPublications.$inferSelect,
runId: string,
providerMessageId: string,
): Promise<boolean> {
if (
!publication.commentId ||
publication.payload.progressState !== undefined
)
return false;
return db
.transaction(async (tx) => {
const [run] = await tx
.select()
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.id, runId),
eq(heartbeatRuns.companyId, publication.companyId),
eq(heartbeatRuns.nativeIssueId, publication.issueId),
eq(heartbeatRuns.runtimeMode, "native"),
eq(heartbeatRuns.status, "succeeded"),
isNotNull(heartbeatRuns.finishedAt),
),
)
.for("share", { noWait: true });
const presentation = run?.resultJson?.presentationDecision as
Record<string, unknown> | undefined;
if (
!run?.resultJson?.nativeCommittedChatResponse ||
presentation?.commentId !== publication.commentId
)
return false;
const [comment] = await tx
.select({ id: issueComments.id })
.from(issueComments)
.where(
and(
eq(issueComments.id, publication.commentId!),
eq(issueComments.companyId, publication.companyId),
eq(issueComments.issueId, publication.issueId),
eq(issueComments.createdByRunId, runId),
eq(issueComments.authorAgentId, run.agentId),
isNull(issueComments.deletedAt),
),
)
.for("share", { noWait: true });
if (!comment) return false;
const [priorAnswer] = await tx
.select({ id: chatPublications.id })
.from(chatPublications)
.innerJoin(
issueComments,
and(
eq(issueComments.id, chatPublications.commentId),
eq(issueComments.companyId, publication.companyId),
eq(issueComments.createdByRunId, runId),
),
)
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
ne(chatPublications.id, publication.id),
inArray(chatPublications.state, [
"published",
"streaming",
"delivery_unknown",
]),
sql`${chatPublications.payload}->>'progressState' is null`,
),
)
.limit(1);
if (priorAnswer) return false;
// A historical progress row can retain this ID after a later edit.
// Only its CURRENT outbound link may grant this narrow exception; an
// authored answer or another run's failure must remain consumed.
const [failure] = await tx
.select({ id: chatPublications.id })
.from(chatMessageLinks)
.innerJoin(
chatPublications,
and(
eq(chatPublications.id, chatMessageLinks.publicationId),
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
eq(chatPublications.providerMessageId, providerMessageId),
eq(
chatPublications.idempotencyKey,
`run:${runId}:failed:${publication.endpointId}`,
),
eq(chatPublications.state, "published"),
isNull(chatPublications.commentId),
sql`${chatPublications.payload}->>'progressState' = 'failed'`,
),
)
.where(
and(
eq(chatMessageLinks.companyId, publication.companyId),
eq(chatMessageLinks.endpointId, publication.endpointId),
eq(chatMessageLinks.conversationId, publication.conversationId),
eq(chatMessageLinks.providerMessageId, providerMessageId),
eq(chatMessageLinks.direction, "outbound"),
),
);
if (!failure) return false;
// The marker is read from the authoritative run, never the publication
// payload. Recheck accepted result/decision plus the complete current
// source batch, actor, access and epoch before borrowing its failure lane.
// The transport claim repeats those checks at the I/O boundary.
return authorizeRetainedChatSourcePublication(tx, publication);
})
.catch((error: unknown) => {
if (isExternalChatWaitAuthorizationContention(error))
throw new NativeChatReviewPresentationContentionError();
throw error;
});
}
async function runPublicationToReplace(
publication: typeof chatPublications.$inferSelect,
payload: SafeChatPublicationPayload,
): Promise<string | null> {
if (payload.attachmentIds?.length) return null;
const currentRunId =
runIdFromMilestonePublication(publication) ??
(publication.commentId
? await db
.select({ runId: issueComments.createdByRunId })
.from(issueComments)
.where(eq(issueComments.id, publication.commentId))
.then((rows) => rows[0]?.runId ?? null)
: null);
if (!currentRunId) return null;
const completion = await slackProgressCompletionTarget(db, publication);
if (completion !== undefined) return completion;
return db
.select({
id: chatPublications.id,
commentId: chatPublications.commentId,
providerMessageId: chatPublications.providerMessageId,
payload: chatPublications.payload,
})
.from(chatPublications)
.leftJoin(issueComments, eq(issueComments.id, chatPublications.commentId))
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.state, "published"),
isNotNull(chatPublications.providerMessageId),
or(
like(chatPublications.idempotencyKey, `run:${currentRunId}:%`),
eq(issueComments.createdByRunId, currentRunId),
),
),
)
.orderBy(desc(chatPublications.createdAt), desc(chatPublications.id))
.then(async (rows) => {
// Progress updates are one replaceable provider-message lane per run.
// The first durable agent comment may turn that placeholder into the
// terminal response, but later comments from the same run are distinct
// user-visible outputs and must be posted separately. Re-editing the
// placeholder for each comment silently erases the earlier replies.
if (
publication.commentId &&
rows.some(
(row) =>
row.commentId !== null && row.payload.progressState === undefined,
)
)
return null;
const replacement = rows.find(
(row) =>
Boolean(row.providerMessageId) &&
row.payload.progressState !== undefined,
);
if (!replacement?.providerMessageId) {
if ((await endpointRecord(publication.endpointId))?.endpoint.provider === "slack") {
return slackPublicProgressReceiptToReplace(db, publication, currentRunId);
}
return null;
}
if (
(await providerProgressLaneConsumed(
publication,
replacement.providerMessageId,
)) &&
!(await committedResponseMayReplaceFailure(
publication,
currentRunId,
replacement.providerMessageId,
))
)
return null;
// Replacement identity belongs to the run, not to the provider-visible
// tail. A status/control reply may legitimately interleave while the run
// is active; making the tail the edit candidate would strand this run's
// working placeholder forever. The query is bounded by endpoint,
// conversation (the task generation), and run id, so an interleaved
// control or another run can never donate its provider message here.
return replacement.providerMessageId;
});
}
async function receiptReactionCompletionRunId(
tx: Db,
publication: typeof chatPublications.$inferSelect,
payload: SafeChatPublicationPayload,
): Promise<string | null> {
if (
payload.progressState === "completed" ||
payload.progressState === "failed"
) {
return runIdFromMilestonePublication(publication);
}
if (publication.commentId) {
const runId = await tx
.select({ runId: issueComments.createdByRunId })
.from(issueComments)
.where(
and(
eq(issueComments.id, publication.commentId),
eq(issueComments.companyId, publication.companyId),
),
)
.then((rows) => rows[0]?.runId ?? null);
return isUuidLike(runId) ? runId : null;
}
if (
payload.interactionId &&
publication.idempotencyKey ===
`interaction:${payload.interactionId}:${publication.endpointId}`
) {
const runId = await tx
.select({ runId: issueThreadInteractions.sourceRunId })
.from(issueThreadInteractions)
.where(
and(
eq(issueThreadInteractions.id, payload.interactionId),
eq(issueThreadInteractions.companyId, publication.companyId),
eq(issueThreadInteractions.issueId, publication.issueId),
),
)
.then((rows) => rows[0]?.runId ?? null);
return isUuidLike(runId) ? runId : null;
}
return null;
}
async function inboundWakePublicationToReplace(
publication: typeof chatPublications.$inferSelect,
payload: SafeChatPublicationPayload,
): Promise<string | null> {
if (payload.attachmentIds?.length) return null;
const notice = parseInboundWakePublicationKey(publication.idempotencyKey);
let wakeId: string | null =
notice && notice.state !== "queued" ? notice.wakeId : null;
let runId = runIdFromMilestonePublication(publication);
if (
!notice &&
!runId &&
payload.interactionId &&
publication.idempotencyKey ===
`interaction:${payload.interactionId}:${publication.endpointId}`
) {
const [interaction] = await db
.select({ runId: issueThreadInteractions.sourceRunId })
.from(issueThreadInteractions)
.where(
and(
eq(issueThreadInteractions.id, payload.interactionId),
eq(issueThreadInteractions.companyId, publication.companyId),
eq(issueThreadInteractions.issueId, publication.issueId),
),
);
runId = interaction?.runId ?? null;
}
if (!notice && !runId && publication.commentId) {
const [comment] = await db
.select({ runId: issueComments.createdByRunId })
.from(issueComments)
.where(
and(
eq(issueComments.id, publication.commentId),
eq(issueComments.companyId, publication.companyId),
eq(issueComments.issueId, publication.issueId),
),
);
runId = comment?.runId ?? null;
}
let runContext: Record<string, unknown> | null = null;
if (runId) {
if (publication.commentId) {
const [priorAnswer] = await db
.select({ id: chatPublications.id })
.from(chatPublications)
.innerJoin(
issueComments,
eq(issueComments.id, chatPublications.commentId),
)
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.state, "published"),
eq(issueComments.createdByRunId, runId),
sql`${chatPublications.payload}->>'progressState' is null`,
),
)
.limit(1);
if (priorAnswer) return null;
}
const [run] = await db
.select({
wakeId: heartbeatRuns.wakeupRequestId,
context: heartbeatRuns.contextSnapshot,
})
.from(heartbeatRuns)
.innerJoin(
agentWakeupRequests,
and(
eq(agentWakeupRequests.id, heartbeatRuns.wakeupRequestId),
eq(agentWakeupRequests.runId, heartbeatRuns.id),
eq(agentWakeupRequests.companyId, heartbeatRuns.companyId),
eq(agentWakeupRequests.agentId, heartbeatRuns.agentId),
// Failed/cancelled is the admitted run's outcome, not loss of
// its original admission. Its terminal milestone still owns
// this exact lane even if no working update was sent first.
ne(agentWakeupRequests.status, "skipped"),
),
)
.where(
and(
eq(heartbeatRuns.id, runId),
eq(heartbeatRuns.companyId, publication.companyId),
sql`${heartbeatRuns.contextSnapshot}->>'issueId' = ${publication.issueId}`,
),
);
wakeId = run?.wakeId ?? null;
runContext = run?.context ?? null;
}
if (!wakeId) return null;
const [queued] = await db
.select({
commentId: chatPublications.commentId,
providerMessageId: chatPublications.providerMessageId,
})
.from(chatPublications)
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
eq(chatPublications.state, "published"),
isNotNull(chatPublications.providerMessageId),
eq(
chatPublications.idempotencyKey,
inboundWakePublicationKey(
wakeId,
"queued",
publication.endpointId,
publication.conversationId,
),
),
),
);
if (!queued?.commentId || !queued.providerMessageId) return null;
// The run helper may have rejected this same provider ID because its
// queued/working placeholder is already an answer or terminal failure.
// Falling back to the older wake row must not erase that newer output.
if (
await providerProgressLaneConsumed(publication, queued.providerMessageId)
)
return null;
if (notice && notice.state !== "queued")
return queued.commentId === publication.commentId
? queued.providerMessageId
: null;
// A deferred owner can be a Board wake into which the exact external
// comment coalesced. The run must actually contain that source comment;
// wake ID alone cannot authorize a different successor's edit lane.
return runContext &&
(runContext.wakeCommentId === queued.commentId ||
runContext.commentId === queued.commentId ||
(Array.isArray(runContext.wakeCommentIds) &&
runContext.wakeCommentIds.includes(queued.commentId)))
? queued.providerMessageId
: null;
}
async function interactionResolutionPublicationToReplace(
publication: typeof chatPublications.$inferSelect,
payload: SafeChatPublicationPayload,
): Promise<string | null> {
if (
!publication.idempotencyKey.startsWith("interaction-resolution:") ||
!payload.interactionId
)
return null;
return db
.select({ providerMessageId: chatPublications.providerMessageId })
.from(chatPublications)
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
eq(
chatPublications.idempotencyKey,
`interaction:${payload.interactionId}:${publication.endpointId}`,
),
eq(chatPublications.state, "published"),
isNotNull(chatPublications.providerMessageId),
),
)
.then((rows) => rows[0]?.providerMessageId ?? null);
}
async function interactionPromptPublicationToReplace(
publication: typeof chatPublications.$inferSelect,
payload: SafeChatPublicationPayload,
): Promise<string | null> {
if (
!payload.interactionId ||
publication.idempotencyKey !==
`interaction:${payload.interactionId}:${publication.endpointId}`
) {
return null;
}
const interaction = await db
.select({
kind: issueThreadInteractions.kind,
sourceRunId: issueThreadInteractions.sourceRunId,
})
.from(issueThreadInteractions)
.where(
and(
eq(issueThreadInteractions.id, payload.interactionId),
eq(issueThreadInteractions.companyId, publication.companyId),
eq(issueThreadInteractions.issueId, publication.issueId),
),
)
.then((rows) => rows[0] ?? null);
if (!interaction?.sourceRunId) return null;
// A provider-visible interaction is this run's response, not an additional
// message beside its progress indicator. Move the durable interaction
// publication onto the exact source run's provider-message lane. This
// retires both the normal working placeholder and the queued placeholder
// when a very fast run asks before the working update is published.
for (const progressState of ["working", "queued"] as const) {
const providerMessageId = await db
.select({ providerMessageId: chatPublications.providerMessageId })
.from(chatPublications)
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
eq(
chatPublications.idempotencyKey,
`run:${interaction.sourceRunId}:${progressState}:${publication.endpointId}`,
),
eq(chatPublications.state, "published"),
isNotNull(chatPublications.providerMessageId),
),
)
.then((rows) => rows[0]?.providerMessageId ?? null);
if (providerMessageId) return providerMessageId;
}
return null;
}
async function progressSupersededByFastResponse(publication: typeof chatPublications.$inferSelect): Promise<boolean> {
if (!["queued", "working"].includes(publication.payload.progressState ?? "")) return false;
if (isSlackPublicProgress(publication)) return false;
const runId = runIdFromMilestonePublication(publication);
const [run] = runId ? await db.select({ context: heartbeatRuns.contextSnapshot }).from(heartbeatRuns).where(and(eq(heartbeatRuns.companyId, publication.companyId), eq(heartbeatRuns.id, runId))) : [];
const sourceCommentId = typeof run?.context?.wakeCommentId === "string" ? run.context.wakeCommentId : typeof run?.context?.commentId === "string" ? run.context.commentId : publication.commentId;
if (!sourceCommentId) return false;
const [receipt] = await db.select({ id: fastResponseRequests.id }).from(fastResponseRequests).innerJoin(chatPublications,
and(eq(chatPublications.companyId, publication.companyId), eq(chatPublications.commentId, fastResponseRequests.commentId), eq(chatPublications.state, "published")))
.where(and(eq(fastResponseRequests.companyId, publication.companyId), eq(fastResponseRequests.sourceCommentId, sourceCommentId), eq(fastResponseRequests.endpointId, publication.endpointId), eq(fastResponseRequests.conversationId, publication.conversationId))).limit(1);
return Boolean(receipt);
}
async function runProgressSupersededByPublishedInteraction(
publication: typeof chatPublications.$inferSelect,
): Promise<boolean> {
if (
!["queued", "working"].includes(publication.payload.progressState ?? "")
) {
return false;
}
const sourceRunId = runIdFromMilestonePublication(publication);
if (!sourceRunId) return false;
const interactionIds = await db
.select({ id: issueThreadInteractions.id })
.from(issueThreadInteractions)
.where(
and(
eq(issueThreadInteractions.companyId, publication.companyId),
eq(issueThreadInteractions.issueId, publication.issueId),
eq(issueThreadInteractions.sourceRunId, sourceRunId),
),
)
.then((rows) => rows.map((row) => row.id));
if (interactionIds.length === 0) return false;
return db
.select({ id: chatPublications.id })
.from(chatPublications)
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
eq(chatPublications.state, "published"),
inArray(
chatPublications.idempotencyKey,
interactionIds.map(
(interactionId) =>
`interaction:${interactionId}:${publication.endpointId}`,
),
),
),
)
.limit(1)
.then((rows) => rows.length > 0);
}
async function taskStatusPublicationToReplace(
publication: typeof chatPublications.$inferSelect,
payload: SafeChatPublicationPayload,
): Promise<string | null> {
if (
!publication.idempotencyKey.startsWith("control:status:") ||
payload.attachmentIds?.length
)
return null;
const rows = await db
.select({
commentId: chatPublications.commentId,
commentRunId: issueComments.createdByRunId,
idempotencyKey: chatPublications.idempotencyKey,
payload: chatPublications.payload,
providerMessageId: chatPublications.providerMessageId,
})
.from(chatPublications)
.leftJoin(issueComments, eq(issueComments.id, chatPublications.commentId))
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.state, "published"),
isNotNull(chatPublications.providerMessageId),
),
)
.orderBy(desc(chatPublications.createdAt), desc(chatPublications.id));
const rowRunId = (row: (typeof rows)[number]) => {
const milestoneMatch =
/^run:([^:]+):(?:queued|working|waiting_for_input|completed|failed):/.exec(
row.idempotencyKey,
);
return milestoneMatch?.[1] ?? row.commentRunId ?? null;
};
for (const candidate of rows) {
if (
!candidate.providerMessageId ||
!["queued", "working"].includes(candidate.payload.progressState ?? "")
) {
continue;
}
const runId = rowRunId(candidate);
if (!runId) continue;
const laneClosed = rows.some((row) => {
if (rowRunId(row) !== runId) return false;
return (
["waiting_for_input", "completed", "failed"].includes(
row.payload.progressState ?? "",
) ||
(row.commentId !== null && row.payload.progressState === undefined)
);
});
if (!laneClosed) return candidate.providerMessageId;
}
return null;
}
async function closeProgressToReplace(
publication: typeof chatPublications.$inferSelect,
endpoint: EndpointRow,
): Promise<{ providerMessageId: string; runId: string } | null> {
if (
!["telegram", "slack", "discord", "microsoft-teams"].includes(
endpoint.provider,
) ||
!publication.idempotencyKey.startsWith("control:close:")
)
return null;
const current = alias(chatPublications, "close_current_progress");
const plainText = (table: typeof chatPublications | typeof current) =>
and(
sql`jsonb_typeof(${table.payload}->'text') = 'string'`,
sql`${table.payload}->'card' is null`,
sql`${table.payload}->'interactionId' is null`,
sql`${table.payload}->'transportPart' is null`,
sql`(${table.payload}->'attachmentIds' is null or ${table.payload}->'attachmentIds' = '[]'::jsonb)`,
);
const ownedMilestone = (table: typeof chatPublications | typeof current) =>
or(
sql`${table.idempotencyKey} = 'run:' || ${heartbeatRuns.id}::text || ':queued:' || ${endpoint.id}`,
sql`${table.idempotencyKey} = 'run:' || ${heartbeatRuns.id}::text || ':working:' || ${endpoint.id}`,
sql`${table.idempotencyKey} like 'run:' || ${heartbeatRuns.id}::text || ':working:' || ${endpoint.id} || ':native:%'`,
);
// A historical progress row alone is not edit authority: its exact current
// outbound link must still belong to that run's progress (or an interleaved
// status control). Read under the same credential lease as final delivery.
const lanes = await db
.selectDistinct({
providerMessageId: chatMessageLinks.providerMessageId,
runId: heartbeatRuns.id,
})
.from(chatPublications)
.innerJoin(
heartbeatRuns,
and(
eq(heartbeatRuns.companyId, publication.companyId),
eq(heartbeatRuns.agentId, endpoint.assignedAgentId),
sql`${heartbeatRuns.contextSnapshot}->>'issueId' = ${publication.issueId}`,
sql`split_part(${chatPublications.idempotencyKey}, ':', 2) = ${heartbeatRuns.id}::text`,
),
)
.innerJoin(
chatMessageLinks,
and(
eq(chatMessageLinks.companyId, publication.companyId),
eq(chatMessageLinks.endpointId, publication.endpointId),
eq(chatMessageLinks.conversationId, publication.conversationId),
eq(chatMessageLinks.direction, "outbound"),
eq(
chatMessageLinks.providerMessageId,
chatPublications.providerMessageId,
),
),
)
.innerJoin(
current,
and(
eq(current.id, chatMessageLinks.publicationId),
eq(current.companyId, publication.companyId),
eq(current.endpointId, publication.endpointId),
eq(current.conversationId, publication.conversationId),
eq(current.issueId, publication.issueId),
eq(current.state, "published"),
eq(current.providerMessageId, chatMessageLinks.providerMessageId),
isNull(current.commentId),
plainText(current),
or(
and(
sql`${current.payload}->>'progressState' in ('queued', 'working')`,
ownedMilestone(current),
),
like(current.idempotencyKey, "control:status:%"),
),
),
)
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
eq(chatPublications.state, "published"),
isNull(chatPublications.commentId),
plainText(chatPublications),
sql`${chatPublications.payload}->>'progressState' in ('queued', 'working')`,
ownedMilestone(chatPublications),
),
)
.limit(2);
// One close receipt owns one edit. Multiple independent lanes require a
// separate durable multi-effect protocol, never an arbitrary latest guess.
if (lanes.length !== 1) return null;
const lane = lanes[0]!;
const [possiblyConsumed] = await db
.select({ id: chatPublications.id })
.from(chatPublications)
.leftJoin(
issueComments,
and(
eq(issueComments.id, chatPublications.commentId),
eq(issueComments.companyId, publication.companyId),
eq(issueComments.issueId, publication.issueId),
),
)
.where(
and(
eq(chatPublications.companyId, publication.companyId),
eq(chatPublications.endpointId, publication.endpointId),
eq(chatPublications.conversationId, publication.conversationId),
eq(chatPublications.issueId, publication.issueId),
or(
inArray(chatPublications.state, [
"pending",
"retry",
"streaming",
"delivery_unknown",
"published",
]),
// Stopping retries (including an operator's unknown-delivery
// cancellation) does not prove an attempted edit never arrived.
and(
inArray(chatPublications.state, ["cancelled", "failed"]),
gt(chatPublications.attempts, 0),
),
),
or(
eq(issueComments.createdByRunId, lane.runId),
// Question/confirmation prompts consume this same run's progress
// ID through interactionPromptPublicationToReplace, but have no
// commentId. A stale working link is not proof that their edit
// never reached the provider (including an unconfirmed receipt).
sql`exists (
select 1 from issue_thread_interactions consuming_interaction
where consuming_interaction.company_id = ${publication.companyId}::uuid
and consuming_interaction.issue_id = ${publication.issueId}::uuid
and consuming_interaction.source_run_id = ${lane.runId}::uuid
and ${chatPublications.payload}->>'interactionId' = consuming_interaction.id::text
and ${chatPublications.idempotencyKey} = 'interaction:' || consuming_interaction.id::text || ':' || ${publication.endpointId}
)`,
eq(
chatPublications.idempotencyKey,
`run:${lane.runId}:completed:${endpoint.id}`,
),
eq(
chatPublications.idempotencyKey,
`run:${lane.runId}:failed:${endpoint.id}`,
),
),
),
)
.limit(1);
// A final may have reached the provider before its database receipt. Even an
// unchanged working link cannot disprove that ambiguous provider effect.
return possiblyConsumed ? null : lane;
}
return { runIdFromMilestonePublication, runOwnershipMilestoneSupersessionReason, runPublicationToReplace, receiptReactionCompletionRunId, inboundWakePublicationToReplace, interactionResolutionPublicationToReplace, interactionPromptPublicationToReplace, progressSupersededByFastResponse, runProgressSupersededByPublishedInteraction, taskStatusPublicationToReplace, closeProgressToReplace };
}