mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-20 09:50:48 +01:00
## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Chat connectors show agent work in the source conversation. > - Slack progress uses a small set of general status messages. > - Native runtimes already mark some completed assistant notes as public progress. > - The server can select those notes and control their size and rate. > - This pull request keeps one progress message and sends the final answer as a new reply. > - Users can follow useful changes without reading each tool event. ## Linked Issues or Issue Description **What existing behavior does this improve?** Automatic progress updates in Slack task threads. **Current behavior** Users see general messages such as “Maya is making progress”. The first answer can replace that message. A source update that arrives during the progress delay can be lost if no later event arrives. **Proposed behavior** Show the latest short public note in one message. Wait 10 seconds before the first update and at least 30 seconds between updates. Recover pending work after a restart. Send the final answer as a fresh reply. **Reason and benefit** Users can see what the agent checks and what it found. The thread stays short. The model needs no new tool call or summary model. Related change: [resolved-card chat bindings](https://github.com/paperclipai/paperclip/pull/20825). This branch includes that master revision. The progress change uses the existing source authority checks. ## What Changed - Select completed `agentMessage` items from the normalized PRP `progress` channel. Keep unknown channels, reasoning, tool output, and raw provider fields out of progress text. - Sanitize each note. Keep at most two sentences and 400 characters. Add a task link when a public board URL is available. - Prefer public notes over general tool status. Remove repeated text and combine bursts into the latest pending note. - Use run events and the publication outbox as saved state. Use the shared deadline worker for the grace period and update delay. A delivery receipt wakes the worker when an earlier update is still in flight. Settled history does not poll. - Apply the run's registered-secret filter before text selection or storage. Reuse the latest usable note within a sweep. Invalidate the cache when the source event or secret registry changes. - Reuse the exact source-bound Slack fast-response receipt when it replaced the working placeholder. Check its current outbound message link before editing it. - Send Slack answers as new replies. After delivery, queue a separate cleanup edit for a generic working message. Keep its current outbound link as the edit authority. Leave useful public notes in place. Cancel pending public progress when an answer, question, or terminal run takes over. Keep the existing delivery policy for other connectors. - Add one short Slack prompt instruction. Update product documentation and regression tests. The Slack module owns public-note selection, cadence, lane eligibility, and cleanup. The shared progress service holds its existing issue and run locks and calls a narrow provider policy. The secret registry supplies redaction and a version token for the per-sweep note cache. The existing outbox owns delivery, retries, and authority checks. The Slack module owns receipt reuse. Shared delivery changes supply the receipt wake and select the Slack reply policy. No schema or configuration change is required. ## Verification - The initial content-selection regression failed before the implementation and passed after it. - The follow-up reproduced two review defects before the fixes. A registered synthetic secret appeared in saved progress. A generic working message stayed active after the answer. Both regressions now pass. - The final content, database-worker, query, protocol-boundary, and redaction selection passes 78 tests across four files. - The earlier affected transport selection passes all 65 cases. It covers public progress, native cadence, separate answers, Slack and Telegram retry isolation, FIFO restart, and cleanup ownership. Its first run passed 64 cases and found one old assertion that rejected the intended cleanup edit. The corrected assertion checks the exact successor message ID. - Full server typecheck passes: `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm --filter @paperclipai/server typecheck`. An earlier attempt found two transaction type errors. The helper contracts now accept their actual database operations. The final run passes all source and test layers. - The broad Slack selection stopped after 189 passing cases when the sandbox killed its worker with SIGKILL. That attempt is not a pass. The focused transport run above supplies affected local coverage. Required CI must supply broad coverage on the final head. - `pnpm verify --explain` returns exit 2 because the product documentation change requires broad CI coverage. No full local suite was started. - Preflight installed dependencies and built the SDK. Cargo is absent, so local Rust checks could not run. No Rust source changed. - [CI on `02552bb584`](https://github.com/paperclipai/paperclip/actions/runs/43493597242) passed build, typecheck, policy, all browser shards, Runner/Rust checks, and the other test lanes. One chat shard failed in three Slack cases. Each expected one edit after an answer was already sent. The intended cleanup adds a “Response sent.” edit before the later failure notice. All three failures reproduced locally. The test assertions now check both exact edit texts and the original run's message ID. They also check that another sweep adds no edits. - The latest local run passes all 85 selected transport cases. It includes all 18 Slack/Telegram answer and failure orders, plus progress, deferred FIFO, retry isolation, and cleanup coverage. Command: `pnpm exec vitest run server/src/__tests__/chat-channels.integration.test.ts -t 'preserves selected answer and failure lanes|public progress|native progress|keeps intermediate.*agent comments|keeps a.*retry.*queue|FIFO'`. - [Final CI](https://github.com/paperclipai/paperclip/actions/runs/43499797545) passed on `55ec4c053fe5263d9033d807777dbf2e679407e3`. All 52 executed jobs passed. This includes build, typecheck, general tests, serialized tests, all eight browser shards, Runner/Rust checks, and the required aggregates. The affected chat shard now passes. The test-file selection job was skipped because this run used full coverage. - [Greptile](https://github.com/paperclipai/paperclip/runs/132142301124) completed on the same head with 5/5 and no new findings. All four inline review threads are resolved. The review job and Socket checks passed. Of 61 check runs, 57 succeeded, three were skipped, and one was neutral. No checks remain pending or failed. The PR is mergeable with a clean state. - [Superagent Security Scan](https://github.com/paperclipai/paperclip/runs/132142345827) returned a neutral result with “PR scan workflow failed”. It supplied no security verdict. Do not count it as a successful scan. - Live Slack notification and unread behavior has not been tested. Service tests use real PostgreSQL and a fake provider runtime. ## Risks - Only runtimes that map commentary to the explicit public `progress` channel supply contextual prose. Other runtimes retain general status messages. - Slack final answers now create a fresh reply. This is an intended delivery change. - Public commentary must suit the current conversation audience. Text sanitization adds protection; it does not authorize sharing unrelated private material. - Source events and outbox rows supply recovery state. No new timer table or recurring idle scan is added. - The existing large integration suite retains its provider fixtures and lifecycle assertions. No source file crosses the 1,000-line threshold in this change. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code tools. The exact deployment ID was not supplied. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
466 lines
20 KiB
TypeScript
466 lines
20 KiB
TypeScript
import { and, asc, desc, eq, gt, gte, inArray, like, notExists, or, sql, type Param, type Placeholder } from "drizzle-orm";
|
|
import { alias } from "drizzle-orm/pg-core";
|
|
import type { Db } from "@paperclipai/db";
|
|
import { agents, chatConversations, chatEndpoints, chatPublications, heartbeatRunEvents, heartbeatRuns, issueComments, issues } from "@paperclipai/db";
|
|
import { notifyChatPublicationWork } from "./chat-work-notifications.js";
|
|
import { projectSafeChatPublication } from "./chat-publication-projection.js";
|
|
import { hasChatRunOwnedProviderInteraction } from "./chat-interaction-arbitration.js";
|
|
import { resolveChatOriginPublicationBindings } from "./issues.js";
|
|
import { boundRunIssueIdSql } from "./run-task-binding.js";
|
|
import { allowsAutomaticRunPublication, chatRunProjectionWindow, type ChatRunProjectionScope } from "./chat-run-publication-policy.js";
|
|
import { SAFE_NATIVE_CHAT_PROGRESS_EVENT_TYPES, safeNativeChatProgressForEvent } from "./safe-native-chat-progress.js";
|
|
import { columnPlaceholder, preparedStatements } from "./prepared-statements.js";
|
|
|
|
import { runSecretRedactionVersionSql } from "./run-secret-redaction.js";
|
|
import { nativeChatProgressPolicy, nativeProgressBaseEligible, nativeProgressEventEligible } from "./chat-native-progress-policy.js";
|
|
import { publicChatTaskUrl } from "./chat-task-url.js";
|
|
|
|
type SafeNativeChatProgressCandidate = {
|
|
eventId: number;
|
|
eventSeq: number;
|
|
eventType: string;
|
|
eventCreatedAt: string;
|
|
runId: string;
|
|
agentId: string;
|
|
issueId: string;
|
|
companyId: string;
|
|
endpointId: string;
|
|
conversationId: string;
|
|
agentName: string;
|
|
provider: string;
|
|
};
|
|
|
|
type NativeProgressCursor = Pick<SafeNativeChatProgressCandidate, "eventCreatedAt" | "runId" | "conversationId">;
|
|
type NativeProgressInput = { since?: Date; limit?: number; scope?: ChatRunProjectionScope; signal?: AbortSignal };
|
|
|
|
type Bound<Value> = Value | Placeholder;
|
|
export type NativeProgressPageBinding = {
|
|
since: Date | Param;
|
|
companyId?: Bound<string>;
|
|
issueId?: Bound<string>;
|
|
cursor: { eventCreatedAt: Bound<string>; runId: Bound<string>; conversationId: Bound<string> } | null;
|
|
pageSize: Bound<number>;
|
|
};
|
|
|
|
export function nativeProgressPageQuery(db: Db, bind: NativeProgressPageBinding) {
|
|
const basePublication = alias(
|
|
chatPublications,
|
|
"safe_native_progress_base_publication",
|
|
);
|
|
const laterEvent = alias(
|
|
heartbeatRunEvents,
|
|
"safe_native_progress_later_event",
|
|
);
|
|
const issueIdFromContext = boundRunIssueIdSql();
|
|
return db
|
|
.select({
|
|
eventId: heartbeatRunEvents.id,
|
|
eventSeq: heartbeatRunEvents.seq,
|
|
eventType: heartbeatRunEvents.eventType,
|
|
// Keep PostgreSQL microseconds in the cursor. Date truncation can
|
|
// repeatedly select the same suppressed page without advancing.
|
|
eventCreatedAt: sql<string>`${heartbeatRunEvents.createdAt}::text`,
|
|
runId: heartbeatRuns.id,
|
|
agentId: heartbeatRuns.agentId,
|
|
issueId: chatConversations.issueId,
|
|
companyId: chatConversations.companyId,
|
|
endpointId: chatConversations.endpointId,
|
|
conversationId: chatConversations.id,
|
|
agentName: agents.name,
|
|
provider: chatEndpoints.provider,
|
|
})
|
|
.from(heartbeatRunEvents)
|
|
.innerJoin(
|
|
heartbeatRuns,
|
|
and(
|
|
eq(heartbeatRuns.id, heartbeatRunEvents.runId),
|
|
eq(heartbeatRuns.companyId, heartbeatRunEvents.companyId),
|
|
eq(heartbeatRuns.agentId, heartbeatRunEvents.agentId),
|
|
eq(heartbeatRuns.runtimeMode, "native"),
|
|
eq(heartbeatRuns.status, "running"),
|
|
),
|
|
)
|
|
.innerJoin(
|
|
chatConversations,
|
|
and(
|
|
eq(chatConversations.companyId, heartbeatRuns.companyId),
|
|
sql`coalesce(${heartbeatRuns.issueId}::text, ${issueIdFromContext}) = ${chatConversations.issueId}::text`,
|
|
inArray(chatConversations.state, ["active", "waiting"]),
|
|
),
|
|
)
|
|
.innerJoin(
|
|
chatEndpoints,
|
|
and(
|
|
eq(chatEndpoints.companyId, chatConversations.companyId),
|
|
eq(chatEndpoints.id, chatConversations.endpointId),
|
|
eq(chatEndpoints.publicationMode, "automatic"),
|
|
allowsAutomaticRunPublication(),
|
|
eq(chatEndpoints.assignedAgentId, heartbeatRuns.agentId),
|
|
),
|
|
)
|
|
.innerJoin(agents, eq(agents.id, heartbeatRuns.agentId))
|
|
.innerJoin(
|
|
basePublication,
|
|
and(
|
|
eq(basePublication.companyId, heartbeatRuns.companyId),
|
|
eq(basePublication.endpointId, chatConversations.endpointId),
|
|
eq(basePublication.conversationId, chatConversations.id),
|
|
eq(basePublication.issueId, chatConversations.issueId),
|
|
sql`${basePublication.idempotencyKey} = 'run:' || ${heartbeatRuns.id}::text || ':working:' || ${chatConversations.endpointId}::text`,
|
|
nativeProgressBaseEligible(chatEndpoints.provider, basePublication),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
inArray(heartbeatRunEvents.eventType, [
|
|
...SAFE_NATIVE_CHAT_PROGRESS_EVENT_TYPES,
|
|
]),
|
|
gte(heartbeatRunEvents.createdAt, bind.since),
|
|
bind.companyId !== undefined ? eq(heartbeatRuns.companyId, bind.companyId) : undefined,
|
|
bind.issueId !== undefined ? eq(chatConversations.issueId, bind.issueId) : undefined,
|
|
nativeProgressEventEligible(chatEndpoints.provider, heartbeatRunEvents.createdAt, basePublication.createdAt),
|
|
notExists(
|
|
db
|
|
.select({ id: laterEvent.id })
|
|
.from(laterEvent)
|
|
.where(
|
|
and(
|
|
eq(laterEvent.companyId, heartbeatRunEvents.companyId),
|
|
eq(laterEvent.runId, heartbeatRunEvents.runId),
|
|
gt(laterEvent.seq, heartbeatRunEvents.seq),
|
|
inArray(laterEvent.eventType, [
|
|
...SAFE_NATIVE_CHAT_PROGRESS_EVENT_TYPES,
|
|
]),
|
|
),
|
|
),
|
|
),
|
|
bind.cursor
|
|
? sql`(${heartbeatRunEvents.createdAt}, ${heartbeatRuns.id}, ${chatConversations.id}) >
|
|
(${bind.cursor.eventCreatedAt}::timestamptz, ${bind.cursor.runId}::uuid, ${bind.cursor.conversationId}::uuid)`
|
|
: undefined,
|
|
),
|
|
)
|
|
.orderBy(
|
|
asc(heartbeatRunEvents.createdAt),
|
|
asc(heartbeatRuns.id),
|
|
asc(chatConversations.id),
|
|
)
|
|
.limit(bind.pageSize);
|
|
}
|
|
|
|
type NativeProgressPageInput = { since: Date; scope?: ChatRunProjectionScope; cursor: NativeProgressCursor | null; pageSize: number };
|
|
type NativeProgressPageShape = `${"all" | "company" | "issue"}:${"first" | "after"}`;
|
|
|
|
const nativeProgressPageStatements = preparedStatements((db, shape: NativeProgressPageShape) => nativeProgressPageQuery(db, {
|
|
since: columnPlaceholder("since", heartbeatRunEvents.createdAt),
|
|
companyId: shape.startsWith("all:") ? undefined : sql.placeholder("companyId"),
|
|
issueId: shape.startsWith("issue:") ? sql.placeholder("issueId") : undefined,
|
|
cursor: shape.endsWith(":after") ? { eventCreatedAt: sql.placeholder("cursorEventCreatedAt"),
|
|
runId: sql.placeholder("cursorRunId"), conversationId: sql.placeholder("cursorConversationId") } : null,
|
|
pageSize: sql.placeholder("pageSize"),
|
|
}).prepare(`safe_native_chat_progress_page_${shape.replace(":", "_")}`));
|
|
|
|
export function nativeProgressPageStatement(db: Db, input: NativeProgressPageInput) {
|
|
const filter = input.scope ? (input.scope.issueId ? "issue" : "company") : "all";
|
|
return {
|
|
statement: nativeProgressPageStatements(db, `${filter}:${input.cursor ? "after" : "first"}`),
|
|
values: {
|
|
since: input.since, pageSize: input.pageSize,
|
|
...(input.scope ? { companyId: input.scope.companyId } : {}),
|
|
...(input.scope?.issueId ? { issueId: input.scope.issueId } : {}),
|
|
...(input.cursor ? { cursorEventCreatedAt: input.cursor.eventCreatedAt, cursorRunId: input.cursor.runId,
|
|
cursorConversationId: input.cursor.conversationId } : {}),
|
|
},
|
|
};
|
|
}
|
|
|
|
function nativeProgressPages(db: Db, since: Date, scope?: ChatRunProjectionScope) {
|
|
return async (pageSize: number, cursor: NativeProgressCursor | null): Promise<SafeNativeChatProgressCandidate[]> => {
|
|
const { statement, values } = nativeProgressPageStatement(db, { since, scope, cursor, pageSize });
|
|
return await statement.execute(values);
|
|
};
|
|
}
|
|
|
|
export async function loadNativeChatProgressScopes(db: Db, scope?: ChatRunProjectionScope, signal?: AbortSignal) {
|
|
const pages = nativeProgressPages(db, chatRunProjectionWindow({}).since, scope);
|
|
const scopes = new Map<string, { companyId: string; issueId: string }>();
|
|
let cursor: NativeProgressCursor | null = null;
|
|
for (;;) {
|
|
signal?.throwIfAborted();
|
|
const rows = await pages(200, cursor);
|
|
for (const row of rows) scopes.set(`${row.companyId}:${row.issueId}`, { companyId: row.companyId, issueId: row.issueId });
|
|
if (rows.length < 200) return [...scopes.values()];
|
|
cursor = rows[rows.length - 1];
|
|
}
|
|
}
|
|
|
|
export async function reconcileNativeChatProgress(db: Db, options: NativeProgressInput = {}) {
|
|
const input = chatRunProjectionWindow(options);
|
|
const pages = nativeProgressPages(db, input.since, options.scope);
|
|
const publicNotes = new Map<string, { seq: number; text: string } | null>();
|
|
let inserted = 0, contended = false;
|
|
let nextDueAt: number | null = null;
|
|
const defer = (dueAt: number) => { nextDueAt = nextDueAt === null ? dueAt : Math.min(nextDueAt, dueAt); };
|
|
let cursor: NativeProgressCursor | null = null;
|
|
while (inserted < input.limit) {
|
|
options.signal?.throwIfAborted();
|
|
const pageSize = Math.max(25, Math.min(200, input.limit - inserted));
|
|
const rows = await pages(pageSize, cursor);
|
|
if (rows.length === 0) break;
|
|
const lastRow = rows.at(-1)!;
|
|
cursor = {
|
|
eventCreatedAt: lastRow.eventCreatedAt,
|
|
runId: lastRow.runId,
|
|
conversationId: lastRow.conversationId,
|
|
};
|
|
|
|
for (const row of rows) {
|
|
options.signal?.throwIfAborted();
|
|
if (inserted >= input.limit) break;
|
|
const result = await db.transaction(async (tx) => {
|
|
// Keep the normal issue -> run order used by native finalization. The
|
|
// publication insert takes an issue FK lock, so locking the run first
|
|
// would invert that order against a concurrent terminal commit. A
|
|
// contended issue/run is optional progress, not a reason to hold other
|
|
// chats: skip it now and revisit it with a fresh cursor next sweep.
|
|
const currentIssue = await tx
|
|
.select({ id: issues.id })
|
|
.from(issues)
|
|
.where(
|
|
and(
|
|
eq(issues.id, row.issueId),
|
|
eq(issues.companyId, row.companyId),
|
|
),
|
|
)
|
|
.for("update", { skipLocked: true })
|
|
.then((currentRows) => currentRows[0] ?? null);
|
|
if (!currentIssue) {
|
|
const [existing] = await tx.select({ id: issues.id }).from(issues)
|
|
.where(and(eq(issues.id, row.issueId), eq(issues.companyId, row.companyId)));
|
|
contended ||= Boolean(existing);
|
|
return 0;
|
|
}
|
|
// Native event admission serializes on the run row. Taking the same
|
|
// short lock makes terminal status, a later event, or a final comment
|
|
// win before this non-terminal projection can be inserted.
|
|
const currentRun = await tx
|
|
.select({ id: heartbeatRuns.id,
|
|
redactionVersion: runSecretRedactionVersionSql(),
|
|
})
|
|
.from(heartbeatRuns)
|
|
.where(
|
|
and(
|
|
eq(heartbeatRuns.id, row.runId),
|
|
eq(heartbeatRuns.companyId, row.companyId),
|
|
eq(heartbeatRuns.agentId, row.agentId),
|
|
eq(heartbeatRuns.runtimeMode, "native"),
|
|
eq(heartbeatRuns.status, "running"),
|
|
),
|
|
)
|
|
.for("update", { skipLocked: true })
|
|
.then((currentRows) => currentRows[0] ?? null);
|
|
if (!currentRun) {
|
|
const [existing] = await tx.select({ id: heartbeatRuns.id }).from(heartbeatRuns).where(and(
|
|
eq(heartbeatRuns.id, row.runId), eq(heartbeatRuns.companyId, row.companyId),
|
|
eq(heartbeatRuns.agentId, row.agentId), eq(heartbeatRuns.runtimeMode, "native"),
|
|
eq(heartbeatRuns.status, "running"),
|
|
));
|
|
contended ||= Boolean(existing);
|
|
return 0;
|
|
}
|
|
|
|
const destination = await tx
|
|
.select({ id: chatConversations.id })
|
|
.from(chatConversations)
|
|
.innerJoin(
|
|
chatEndpoints,
|
|
and(
|
|
eq(chatEndpoints.companyId, chatConversations.companyId),
|
|
eq(chatEndpoints.id, chatConversations.endpointId),
|
|
eq(chatEndpoints.publicationMode, "automatic"),
|
|
allowsAutomaticRunPublication(),
|
|
eq(chatEndpoints.assignedAgentId, row.agentId),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
eq(chatConversations.companyId, row.companyId),
|
|
eq(chatConversations.id, row.conversationId),
|
|
eq(chatConversations.endpointId, row.endpointId),
|
|
eq(chatConversations.issueId, row.issueId),
|
|
inArray(chatConversations.state, ["active", "waiting"]),
|
|
),
|
|
)
|
|
.limit(1)
|
|
.then((currentRows) => currentRows[0] ?? null);
|
|
if (!destination) return 0;
|
|
|
|
const [currentEvent] = await tx
|
|
.select({
|
|
id: heartbeatRunEvents.id,
|
|
seq: heartbeatRunEvents.seq,
|
|
eventType: heartbeatRunEvents.eventType,
|
|
createdAt: heartbeatRunEvents.createdAt,
|
|
})
|
|
.from(heartbeatRunEvents)
|
|
.where(
|
|
and(
|
|
eq(heartbeatRunEvents.companyId, row.companyId),
|
|
eq(heartbeatRunEvents.runId, row.runId),
|
|
inArray(heartbeatRunEvents.eventType, [
|
|
...SAFE_NATIVE_CHAT_PROGRESS_EVENT_TYPES,
|
|
]),
|
|
),
|
|
)
|
|
.orderBy(desc(heartbeatRunEvents.seq), desc(heartbeatRunEvents.id))
|
|
.limit(1);
|
|
if (
|
|
!currentEvent ||
|
|
currentEvent.id !== row.eventId ||
|
|
currentEvent.seq !== row.eventSeq ||
|
|
currentEvent.eventType !== row.eventType
|
|
) {
|
|
return 0;
|
|
}
|
|
const policy = nativeChatProgressPolicy(row.provider);
|
|
const noteKey = `${row.companyId}:${row.runId}:${currentEvent.seq}:${currentRun.redactionVersion}`;
|
|
if (policy.latestPublicNote && !publicNotes.has(noteKey)) {
|
|
publicNotes.set(noteKey, await policy.latestPublicNote(tx, row, options.signal));
|
|
}
|
|
const publicNote = policy.latestPublicNote ? publicNotes.get(noteKey) ?? null : null;
|
|
const publicText = publicNote?.text ?? null;
|
|
const progress = publicText ? { phase: "public", text: publicText } : safeNativeChatProgressForEvent(
|
|
currentEvent.eventType,
|
|
row.agentName,
|
|
);
|
|
if (!progress) return 0;
|
|
|
|
const baseKey = `run:${row.runId}:working:${row.endpointId}`;
|
|
const [currentBase] = await tx
|
|
.select({ createdAt: chatPublications.createdAt, state: chatPublications.state, redactedError: chatPublications.redactedError })
|
|
.from(chatPublications)
|
|
.where(
|
|
and(
|
|
eq(chatPublications.companyId, row.companyId),
|
|
eq(chatPublications.endpointId, row.endpointId),
|
|
eq(chatPublications.conversationId, row.conversationId),
|
|
eq(chatPublications.issueId, row.issueId),
|
|
eq(chatPublications.idempotencyKey, baseKey),
|
|
),
|
|
)
|
|
.limit(1);
|
|
if (!currentBase || !policy.acceptsBase(currentBase, publicText !== null)) return 0;
|
|
|
|
const progressKeyPrefix = `${baseKey}:native:`;
|
|
const previousProgress = await tx
|
|
.select({
|
|
createdAt: chatPublications.createdAt,
|
|
idempotencyKey: chatPublications.idempotencyKey,
|
|
payload: chatPublications.payload,
|
|
state: chatPublications.state,
|
|
publishedAt: chatPublications.publishedAt,
|
|
})
|
|
.from(chatPublications)
|
|
.where(
|
|
and(
|
|
eq(chatPublications.companyId, row.companyId),
|
|
eq(chatPublications.endpointId, row.endpointId),
|
|
eq(chatPublications.conversationId, row.conversationId),
|
|
eq(chatPublications.issueId, row.issueId),
|
|
like(chatPublications.idempotencyKey, `${progressKeyPrefix}%`),
|
|
),
|
|
);
|
|
if (publicText) {
|
|
if (previousProgress.some(publication => publication.payload.text.split("\n\n")[0] === publicText)) return 0;
|
|
} else if (previousProgress.some(publication => publication.idempotencyKey.startsWith(`${progressKeyPrefix}${progress.phase}:`))) return 0;
|
|
const bindings = await resolveChatOriginPublicationBindings(
|
|
tx,
|
|
row.companyId,
|
|
row.issueId,
|
|
row.runId,
|
|
);
|
|
if (
|
|
!bindings.some(
|
|
(binding) =>
|
|
binding.endpointId === row.endpointId &&
|
|
binding.conversationId === row.conversationId,
|
|
)
|
|
) {
|
|
return 0;
|
|
}
|
|
if (
|
|
await hasChatRunOwnedProviderInteraction(tx, {
|
|
companyId: row.companyId,
|
|
issueId: row.issueId,
|
|
runId: row.runId,
|
|
})
|
|
) {
|
|
return 0;
|
|
}
|
|
const explicitlyAuthoredFinal = await tx
|
|
.select({ id: chatPublications.id })
|
|
.from(chatPublications)
|
|
.innerJoin(
|
|
issueComments,
|
|
and(
|
|
eq(issueComments.companyId, chatPublications.companyId),
|
|
eq(issueComments.id, chatPublications.commentId),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
eq(chatPublications.companyId, row.companyId),
|
|
eq(chatPublications.endpointId, row.endpointId),
|
|
eq(chatPublications.conversationId, row.conversationId),
|
|
eq(chatPublications.issueId, row.issueId),
|
|
eq(issueComments.authorType, "agent"),
|
|
eq(issueComments.createdByRunId, row.runId),
|
|
sql`(
|
|
${issueComments.metadata} ->> 'authorizationReason' = 'paperclip_runner_protocol'
|
|
or left(coalesce(${issueComments.metadata} ->> 'authorizationReason', ''), 6) = 'allow_'
|
|
)`,
|
|
),
|
|
)
|
|
.limit(1);
|
|
if (explicitlyAuthoredFinal.length > 0) return 0;
|
|
|
|
const delivery = policy.nextDelivery({ base: currentBase, previous: previousProgress,
|
|
eventCreatedAt: currentEvent.createdAt, now: Date.now() });
|
|
if (delivery.dueAt !== undefined) defer(delivery.dueAt);
|
|
if (!delivery.ready) return 0;
|
|
|
|
const taskUrl = publicText ? publicChatTaskUrl(row.issueId) : null;
|
|
await notifyChatPublicationWork(tx);
|
|
const insertedRows = await tx
|
|
.insert(chatPublications)
|
|
.values({
|
|
companyId: row.companyId,
|
|
endpointId: row.endpointId,
|
|
conversationId: row.conversationId,
|
|
issueId: row.issueId,
|
|
idempotencyKey: `${progressKeyPrefix}${progress.phase}:${publicNote?.seq ?? currentEvent.seq}`,
|
|
payload: projectSafeChatPublication({
|
|
classification: "external",
|
|
source: "safe_milestone",
|
|
text: publicText && taskUrl
|
|
? `${publicText}\n\n[View task](${taskUrl})` : progress.text,
|
|
progressState: "working",
|
|
}),
|
|
state: "pending",
|
|
createdAt: new Date(),
|
|
})
|
|
.onConflictDoNothing()
|
|
.returning({ id: chatPublications.id });
|
|
return insertedRows.length;
|
|
});
|
|
inserted += result;
|
|
}
|
|
if (rows.length < pageSize) break;
|
|
}
|
|
return { inserted, contended, nextDueAt };
|
|
}
|
|
|
|
export async function enqueueSafeNativeChatProgress(db: Db, options: NativeProgressInput = {}): Promise<number> {
|
|
return (await reconcileNativeChatProgress(db, options)).inserted;
|
|
}
|