Files
PaperClipAI/server/src/services/activity.ts
T
DottaandPaperclip 5c7a025032 fix(ui): clear consumed messages from the pending queue (#16317)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Task conversations keep follow-up messages pending while an agent
works.
> - The screen combines optimistic sends, queue snapshots, and run input
records.
> - A late send response can restore a local queue flag after a run
takes the message.
> - A stale queue snapshot can then keep the same message in pending.
> - This pull request matches the input record to its source queue and
clears stale pending flags.
> - The benefit is that consumed messages leave pending while unsent
messages stay visible.

## Linked Issues or Issue Description

**What happened?**

A follow-up can remain in the task composer's pending row after a run
takes it as input. Local queue state can outlive the send request. The
old projection cleared that state for steering, but not for ordinary
run-start input. A stale queue read can also retain the consumed entry.

**Expected behavior**

A message that a started run takes as input appears once in history and
leaves pending. Other pending messages remain available. A message
assigned to a run that has not started remains pending.

**Steps to reproduce**

1. Send a follow-up while a task has a live run.
2. Let a started run record that message as input before or after the
send response arrives.
3. Let a queue read still include the consumed message.
4. Observe the message remain pending on the base revision.

**Paperclip version or commit**

Reproduced against master at `33d24e48ad` with controlled request
ordering.

**Deployment mode**

Source checkout. Browser verification used the production task screen
with a simulated native runner and delayed API responses. It did not run
a live model or change a deployed instance.

Related: [PR
#15030](https://github.com/paperclipai/paperclip/pull/15030) addresses
acknowledgement matching in the runner console. This change addresses
the task-detail composer and does not modify that console.

## What Changed

- Clear local pending flags when a started run records the comment as
input.
- Use live-run input IDs while the run-history query catches up.
- Match stale queue entries by their primary or coalesced source wake
ID, even when the snapshot targets the previous run or has no target.
- Retain coalesced source queue IDs in run context and include receipt
IDs in compact task run reads.
- Keep unstarted run inputs and authoritative retries pending. Cover
send ordering, same-run and successor delivery, queued snapshots,
coalesced inputs, and retry consumption.
- Add the delivery-race checks to the steering feature map.

## Verification

- Before the fix, the first four request-order regressions failed with
the consumed message still in pending. Coverage now includes 16
combinations of receipt source, send-response order, and stale snapshot
kind.
- Focused UI checks: 5 files and 233 tests passed. These cover task
detail, queue projection, optimistic comments, queued-message controls,
and the task-chat adapter.
- `pnpm -r typecheck` passed.
- Server receipt checks: 5 files and 139 tests passed with
`PAPERCLIP_TEST_POSTGRES_TEMPLATE=0`. These exercise real database
reads, coalesced heartbeat input, and polling routes. The default
cached-template path timed out during setup on this host; the
fresh-cluster path passed without longer timeouts.
- Token gates and `node scripts/check-maps.mjs` passed.
- Browser: sent a follow-up with a delayed response in the production
task screen. The consumed message moved into history. A second unsent
follow-up stayed queued. Reload preserved both states. The API and
runner were simulated.
- `pnpm build` passed.
- The full local repository test run was started, then stopped to
isolate database setup timeouts. All 55 GitHub CI checks passed on
`4ec6820cf1f89880b801fd5f938d72f54591c5e2`, including the full test
matrix. One serialized server shard passed on its single rerun after a
15-second route-test timeout; the same 97-test route suite also passed
locally.
- Greptile reviewed the final head at 5/5. All review threads are
resolved.

## Risks

- Queue display depends on matching comment IDs, source queue IDs, and a
recorded start time. Rows without start evidence stay pending. Newer
queues remain visible until their own input receipt arrives.
- Run context gains additive receipt metadata. No schema migration is
required. Dispatch order, stored queue state, company access, and
steering authority stay unchanged.
- Browser evidence covers controlled response ordering. It does not
establish deployment or live-provider behavior.

## Model Used

- OpenAI Codex, GPT-6, with reasoning, tool use, and code execution. The
exact serving model ID and context-window limit are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-14 16:11:29 -05:00

655 lines
27 KiB
TypeScript

import { skillAccessService } from "./skill-access.js";
import { credentialAccessConnectionNameSql } from "./credential-access-visibility.js";
import { executionProjectionsForRuns } from "./execution-projection.js";
import { and, asc, desc, eq, inArray, isNull, or, sql, type SQL } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
activityLog,
agents,
documentRevisions,
documents,
environmentLeases,
environments,
heartbeatRunEvents,
heartbeatRuns,
issueComments,
issueDocuments,
issues,
issueWorkProducts,
workspaceOperations,
} from "@paperclipai/db";
import { hasWorkspaceRestoreFailure, safeWorkspaceRestorePath, ISSUE_CONTINUATION_SUMMARY_DOCUMENT_KEY } from "@paperclipai/shared";
import { logger } from "../middleware/logger.js";
import { visibleIssueCondition } from "./issue-visibility.js";
import { classifyRunLiveness } from "./run-liveness.js";
export interface ActivityFilters {
companyId: string;
agentId?: string;
entityType?: string;
entityId?: string;
limit?: number;
readCondition?: SQL<boolean>;
}
const DEFAULT_ACTIVITY_LIMIT = 100;
const MAX_ACTIVITY_LIMIT = 500;
export function normalizeActivityLimit(limit: number | undefined) {
if (!Number.isFinite(limit)) return DEFAULT_ACTIVITY_LIMIT;
return Math.max(1, Math.min(MAX_ACTIVITY_LIMIT, Math.floor(limit ?? DEFAULT_ACTIVITY_LIMIT)));
}
export function activityService(db: Db) {
const scheduledLivenessBackfills = new Set<string>();
const issueIdAsText = sql<string>`${issues.id}::text`;
const summarizedUsageJson = sql<Record<string, unknown> | null>`
case
when ${heartbeatRuns.usageJson} is null then null
else jsonb_strip_nulls(jsonb_build_object(
'inputTokens', coalesce(${heartbeatRuns.usageJson} -> 'inputTokens', ${heartbeatRuns.usageJson} -> 'input_tokens'),
'input_tokens', coalesce(${heartbeatRuns.usageJson} -> 'input_tokens', ${heartbeatRuns.usageJson} -> 'inputTokens'),
'outputTokens', coalesce(${heartbeatRuns.usageJson} -> 'outputTokens', ${heartbeatRuns.usageJson} -> 'output_tokens'),
'output_tokens', coalesce(${heartbeatRuns.usageJson} -> 'output_tokens', ${heartbeatRuns.usageJson} -> 'outputTokens'),
'cachedInputTokens', coalesce(
${heartbeatRuns.usageJson} -> 'cachedInputTokens',
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens'
),
'cached_input_tokens', coalesce(
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
${heartbeatRuns.usageJson} -> 'cachedInputTokens',
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens'
),
'cache_read_input_tokens', coalesce(
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens',
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
${heartbeatRuns.usageJson} -> 'cachedInputTokens'
),
'billingType', coalesce(${heartbeatRuns.usageJson} -> 'billingType', ${heartbeatRuns.usageJson} -> 'billing_type'),
'billing_type', coalesce(${heartbeatRuns.usageJson} -> 'billing_type', ${heartbeatRuns.usageJson} -> 'billingType'),
'costUsd', coalesce(
${heartbeatRuns.usageJson} -> 'costUsd',
${heartbeatRuns.usageJson} -> 'cost_usd',
${heartbeatRuns.usageJson} -> 'total_cost_usd'
),
'cost_usd', coalesce(
${heartbeatRuns.usageJson} -> 'cost_usd',
${heartbeatRuns.usageJson} -> 'costUsd',
${heartbeatRuns.usageJson} -> 'total_cost_usd'
),
'total_cost_usd', coalesce(
${heartbeatRuns.usageJson} -> 'total_cost_usd',
${heartbeatRuns.usageJson} -> 'cost_usd',
${heartbeatRuns.usageJson} -> 'costUsd'
)
))
end
`.as("usageJson");
// A denied user needs the selected connection's name to repair their task.
// Other readers need the connection's existing human sharing permission.
const summarizedResultJson = (viewerUserId: string | null) => sql<Record<string, unknown> | null>`
case
when ${heartbeatRuns.resultJson} is null then null
else jsonb_strip_nulls(jsonb_build_object(
'conversationReset', ${heartbeatRuns.resultJson} -> 'conversationReset',
'configurationIncomplete', case
when ${heartbeatRuns.resultJson} -> 'configurationIncomplete' ->> 'reason' = 'workspace_base_ref_unresolved'
then jsonb_build_object('reason', 'workspace_base_ref_unresolved')
when ${heartbeatRuns.errorCode} = 'configuration_incomplete'
and (${heartbeatRuns.resultJson} #>> '{configurationIncomplete,selectionFailure}' = 'ai_connection_credential_not_shared'
or ${heartbeatRuns.error} = 'This credential is not shared with the responsible user')
then jsonb_build_object(
'selectionFailure', case when ${heartbeatRuns.resultJson} #>> '{configurationIncomplete,selectionFailure}' = 'ai_connection_credential_not_shared'
then 'ai_connection_credential_not_shared' end,
'credentialAccess', jsonb_build_object(
'connectionName', ${credentialAccessConnectionNameSql(viewerUserId)}
)
) end,
'workspaceRestoreFailure', case when ${heartbeatRuns.resultJson} ->> 'workspaceRestoreFailure'
in ('restore_permission_denied', 'restore_lock_timeout', 'restore_unsafe_archive', 'restore_failed')
then ${heartbeatRuns.resultJson} -> 'workspaceRestoreFailure' end,
'workspaceRestorePath', case when length(${heartbeatRuns.resultJson} ->> 'workspaceRestorePath') <= 180
then ${heartbeatRuns.resultJson} -> 'workspaceRestorePath' end,
'finalResponseRecorded', case when jsonb_typeof(${heartbeatRuns.resultJson} -> 'finalResponseRecorded') = 'boolean'
then ${heartbeatRuns.resultJson} -> 'finalResponseRecorded' end,
'billingType', coalesce(${heartbeatRuns.resultJson} -> 'billingType', ${heartbeatRuns.resultJson} -> 'billing_type'),
'billing_type', coalesce(${heartbeatRuns.resultJson} -> 'billing_type', ${heartbeatRuns.resultJson} -> 'billingType'),
'costUsd', coalesce(
${heartbeatRuns.resultJson} -> 'costUsd',
${heartbeatRuns.resultJson} -> 'cost_usd',
${heartbeatRuns.resultJson} -> 'total_cost_usd'
),
'cost_usd', coalesce(
${heartbeatRuns.resultJson} -> 'cost_usd',
${heartbeatRuns.resultJson} -> 'costUsd',
${heartbeatRuns.resultJson} -> 'total_cost_usd'
),
'total_cost_usd', coalesce(
${heartbeatRuns.resultJson} -> 'total_cost_usd',
${heartbeatRuns.resultJson} -> 'cost_usd',
${heartbeatRuns.resultJson} -> 'costUsd'
),
'stopReason', ${heartbeatRuns.resultJson} -> 'stopReason',
'effectiveTimeoutSec', ${heartbeatRuns.resultJson} -> 'effectiveTimeoutSec',
'effectiveTimeoutMs', ${heartbeatRuns.resultJson} -> 'effectiveTimeoutMs',
'timeoutConfigured', ${heartbeatRuns.resultJson} -> 'timeoutConfigured',
'timeoutSource', ${heartbeatRuns.resultJson} -> 'timeoutSource',
'timeoutFired', ${heartbeatRuns.resultJson} -> 'timeoutFired'
))
end
`.as("resultJson");
function countValue(value: unknown) {
const parsed = Number(value ?? 0);
return Number.isFinite(parsed) ? Math.max(0, Math.floor(parsed)) : 0;
}
function dateValue(value: unknown) {
if (value instanceof Date) return Number.isNaN(value.getTime()) ? null : value;
if (typeof value === "string" || typeof value === "number") {
const parsed = new Date(value);
return Number.isNaN(parsed.getTime()) ? null : parsed;
}
return null;
}
function latestDate(...values: unknown[]) {
let latest: Date | null = null;
for (const value of values) {
const parsed = dateValue(value);
if (!parsed) continue;
if (!latest || parsed.getTime() > latest.getTime()) latest = parsed;
}
return latest;
}
function asRecord(value: unknown): Record<string, unknown> | null {
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
return value as Record<string, unknown>;
}
function readNumber(value: unknown) {
return typeof value === "number" && Number.isFinite(value) ? value : null;
}
async function backfillMissingRunLivenessForIssue(companyId: string, issueId: string) {
const runs = await db
.select({
id: heartbeatRuns.id,
companyId: heartbeatRuns.companyId,
status: heartbeatRuns.status,
contextSnapshot: heartbeatRuns.contextSnapshot,
resultJson: heartbeatRuns.resultJson,
stdoutExcerpt: heartbeatRuns.stdoutExcerpt,
stderrExcerpt: heartbeatRuns.stderrExcerpt,
error: heartbeatRuns.error,
errorCode: heartbeatRuns.errorCode,
continuationAttempt: heartbeatRuns.continuationAttempt,
})
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.companyId, companyId),
isNull(heartbeatRuns.livenessState),
sql`${heartbeatRuns.status} not in ('queued', 'running')`,
or(
sql`${heartbeatRuns.contextSnapshot} ->> 'issueId' = ${issueId}`,
sql`exists (
select 1
from ${activityLog}
where ${activityLog.companyId} = ${companyId}
and ${activityLog.entityType} = 'issue'
and ${activityLog.entityId} = ${issueId}
and ${activityLog.runId} = ${heartbeatRuns.id}
)`,
),
),
)
.limit(20);
if (runs.length === 0) return;
const issue = await db
.select({
status: issues.status,
title: issues.title,
description: issues.description,
workMode: issues.workMode,
})
.from(issues)
.where(and(eq(issues.companyId, companyId), eq(issues.id, issueId)))
.then((rows) => rows[0] ?? null);
for (const run of runs) {
const context = asRecord(run.contextSnapshot);
const continuationAttempt =
readNumber(context?.continuationAttempt) ??
readNumber(context?.livenessContinuationAttempt) ??
run.continuationAttempt ??
0;
const [commentStats] = await db
.select({
count: sql<number>`count(*)::int`,
latestAt: sql<Date | null>`max(${issueComments.createdAt})`,
})
.from(issueComments)
.where(
and(
eq(issueComments.companyId, companyId),
eq(issueComments.issueId, issueId),
eq(issueComments.createdByRunId, run.id),
),
);
const [documentStats] = await db
.select({
count: sql<number>`count(*)::int`,
planCount: sql<number>`count(*) filter (where ${issueDocuments.key} = 'plan')::int`,
latestAt: sql<Date | null>`max(${documentRevisions.createdAt})`,
})
.from(documentRevisions)
.innerJoin(issueDocuments, eq(documentRevisions.documentId, issueDocuments.documentId))
.where(
and(
eq(documentRevisions.companyId, companyId),
eq(documentRevisions.createdByRunId, run.id),
eq(issueDocuments.companyId, companyId),
eq(issueDocuments.issueId, issueId),
sql`${issueDocuments.key} != ${ISSUE_CONTINUATION_SUMMARY_DOCUMENT_KEY}`,
),
);
const [workProductStats] = await db
.select({
count: sql<number>`count(*)::int`,
latestAt: sql<Date | null>`max(${issueWorkProducts.createdAt})`,
})
.from(issueWorkProducts)
.where(
and(
eq(issueWorkProducts.companyId, companyId),
eq(issueWorkProducts.issueId, issueId),
eq(issueWorkProducts.createdByRunId, run.id),
),
);
const [workspaceOperationStats] = await db
.select({
count: sql<number>`count(*)::int`,
latestAt: sql<Date | null>`max(${workspaceOperations.startedAt})`,
})
.from(workspaceOperations)
.where(and(eq(workspaceOperations.companyId, companyId), eq(workspaceOperations.heartbeatRunId, run.id)));
const [activityStats] = await db
.select({
count: sql<number>`count(*)::int`,
latestAt: sql<Date | null>`max(${activityLog.createdAt})`,
})
.from(activityLog)
.where(and(eq(activityLog.companyId, companyId), eq(activityLog.runId, run.id)));
const [eventStats] = await db
.select({
count: sql<number>`count(*) filter (where ${heartbeatRunEvents.eventType} not in ('lifecycle', 'adapter.invoke', 'error'))::int`,
latestAt: sql<Date | null>`max(${heartbeatRunEvents.createdAt}) filter (where ${heartbeatRunEvents.eventType} not in ('lifecycle', 'adapter.invoke', 'error'))`,
})
.from(heartbeatRunEvents)
.where(and(eq(heartbeatRunEvents.companyId, companyId), eq(heartbeatRunEvents.runId, run.id)));
const classification = classifyRunLiveness({
runStatus: run.status,
issue,
resultJson: asRecord(run.resultJson),
stdoutExcerpt: run.stdoutExcerpt,
stderrExcerpt: run.stderrExcerpt,
error: run.error,
errorCode: run.errorCode,
continuationAttempt,
evidence: {
issueCommentsCreated: countValue(commentStats?.count),
documentRevisionsCreated: countValue(documentStats?.count),
planDocumentRevisionsCreated: countValue(documentStats?.planCount),
workProductsCreated: countValue(workProductStats?.count),
workspaceOperationsCreated: countValue(workspaceOperationStats?.count),
activityEventsCreated: countValue(activityStats?.count),
toolOrActionEventsCreated: countValue(eventStats?.count),
latestEvidenceAt: latestDate(
commentStats?.latestAt,
documentStats?.latestAt,
workProductStats?.latestAt,
workspaceOperationStats?.latestAt,
activityStats?.latestAt,
eventStats?.latestAt,
),
},
});
await db
.update(heartbeatRuns)
.set({
livenessState: classification.livenessState,
livenessReason: classification.livenessReason,
continuationAttempt: classification.continuationAttempt,
lastUsefulActionAt: classification.lastUsefulActionAt,
nextAction: classification.nextAction,
updatedAt: new Date(),
})
.where(and(eq(heartbeatRuns.id, run.id), isNull(heartbeatRuns.livenessState)));
}
}
function scheduleRunLivenessBackfill(companyId: string, issueId: string) {
const key = `${companyId}:${issueId}`;
if (scheduledLivenessBackfills.has(key)) return;
scheduledLivenessBackfills.add(key);
void backfillMissingRunLivenessForIssue(companyId, issueId)
.catch((err: unknown) => {
logger.warn({ err, companyId, issueId }, "run liveness backfill failed");
})
.finally(() => {
scheduledLivenessBackfills.delete(key);
});
}
return {
list: async (filters: ActivityFilters) => {
const conditions = [eq(activityLog.companyId, filters.companyId), await skillAccessService(db).activityCondition(filters.companyId)];
const limit = normalizeActivityLimit(filters.limit);
if (filters.agentId) {
conditions.push(eq(activityLog.agentId, filters.agentId));
}
if (filters.entityType) {
conditions.push(eq(activityLog.entityType, filters.entityType));
}
if (filters.entityId) {
conditions.push(eq(activityLog.entityId, filters.entityId));
}
return db
.select({ activityLog })
.from(activityLog)
.leftJoin(
issues,
and(
eq(activityLog.entityType, sql`'issue'`),
eq(activityLog.entityId, issueIdAsText),
),
)
.where(
and(
...conditions,
or(
sql`${activityLog.entityType} != 'issue'`,
and(visibleIssueCondition(), filters.readCondition ?? sql<boolean>`true`),
),
),
)
.orderBy(desc(activityLog.createdAt))
.limit(limit)
.then((rows) => rows.map((r) => r.activityLog));
},
forIssue: (issueId: string) =>
db
.select()
.from(activityLog)
.where(
or(
and(eq(activityLog.entityType, "issue"), eq(activityLog.entityId, issueId)),
and(or(eq(activityLog.action, "project.created"), eq(activityLog.action, "company.skill_created")), sql`${activityLog.details}->>'sourceIssueId' = ${issueId}`,
sql`${activityLog.companyId} = (select company_id from issues where id = ${issueId})`),
),
)
.orderBy(desc(activityLog.createdAt)),
runsForIssue: async (companyId: string, issueId: string, viewerUserId: string | null = null) => {
scheduleRunLivenessBackfill(companyId, issueId);
const runs = await db
.select({
runId: heartbeatRuns.id,
runtimeMode: heartbeatRuns.runtimeMode,
wakeupRequestId: heartbeatRuns.wakeupRequestId,
status: heartbeatRuns.status,
agentId: heartbeatRuns.agentId,
adapterType: agents.adapterType,
startedAt: heartbeatRuns.startedAt,
finishedAt: heartbeatRuns.finishedAt,
createdAt: heartbeatRuns.createdAt,
invocationSource: heartbeatRuns.invocationSource,
responsibleUserId: heartbeatRuns.responsibleUserId,
errorCode: heartbeatRuns.errorCode,
error: sql<string | null>`case when ${heartbeatRuns.status} = 'failed'
and ${heartbeatRuns.errorCode} = 'native_provider_model_rejected'
then left(${heartbeatRuns.error}, 2000)
when ${heartbeatRuns.status} = 'failed'
and ${heartbeatRuns.error} = 'This credential is not shared with the responsible user'
then 'This credential is not shared with the responsible user' else null end`,
usageJson: summarizedUsageJson,
resultJson: summarizedResultJson(viewerUserId),
logBytes: heartbeatRuns.logBytes,
retryOfRunId: heartbeatRuns.retryOfRunId,
scheduledRetryAt: heartbeatRuns.scheduledRetryAt,
scheduledRetryAttempt: heartbeatRuns.scheduledRetryAttempt,
scheduledRetryReason: heartbeatRuns.scheduledRetryReason,
livenessState: heartbeatRuns.livenessState,
livenessReason: heartbeatRuns.livenessReason,
continuationAttempt: heartbeatRuns.continuationAttempt,
lastUsefulActionAt: heartbeatRuns.lastUsefulActionAt,
nextAction: heartbeatRuns.nextAction,
wakeCommentIds: sql<string[] | null>`${heartbeatRuns.contextSnapshot} -> 'wakeCommentIds'`,
queuedCommentSourceWakeIds: sql<string[] | null>`${heartbeatRuns.contextSnapshot} -> 'queuedCommentSourceWakeIds'`,
wakeCommentId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'wakeCommentId'`,
contextCommentId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'commentId'`,
contextIssueId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'issueId'`,
})
.from(heartbeatRuns)
.innerJoin(
agents,
and(
eq(agents.id, heartbeatRuns.agentId),
eq(agents.companyId, heartbeatRuns.companyId),
),
)
.where(
and(
eq(heartbeatRuns.companyId, companyId),
or(
eq(heartbeatRuns.issueId, issueId),
sql`exists (
select 1
from ${activityLog}
where ${activityLog.companyId} = ${companyId}
and ${activityLog.entityType} = 'issue'
and ${activityLog.entityId} = ${issueId}
and ${activityLog.runId} = ${heartbeatRuns.id}
)`,
),
),
)
.orderBy(desc(heartbeatRuns.createdAt));
if (runs.length === 0) return runs;
const runIds = runs.map((run) => run.runId);
if (runIds.length === 0) return runs;
const exhaustionRowsQuery = db
.select({
runId: heartbeatRunEvents.runId,
message: heartbeatRunEvents.message,
})
.from(heartbeatRunEvents)
.where(
and(
inArray(heartbeatRunEvents.runId, runIds),
eq(heartbeatRunEvents.eventType, "lifecycle"),
sql`${heartbeatRunEvents.message} like 'Bounded retry exhausted%'`,
),
)
.orderBy(asc(heartbeatRunEvents.runId), desc(heartbeatRunEvents.id));
const leaseRowsQuery = db
.select({
lease: environmentLeases,
environment: {
id: environments.id,
name: environments.name,
driver: environments.driver,
},
})
.from(environmentLeases)
.innerJoin(environments, eq(environmentLeases.environmentId, environments.id))
.where(
and(
eq(environmentLeases.companyId, companyId),
inArray(environmentLeases.heartbeatRunId, runIds),
),
)
.orderBy(desc(environmentLeases.lastUsedAt), desc(environmentLeases.createdAt));
// Only stored, current plan revisions can support a saved-plan link.
// Do not trust an adapter's claim that it wrote a document.
const savedPlanQuery = runs.some((run) => hasWorkspaceRestoreFailure(run.resultJson))
? db.select({ revisionId: documentRevisions.id, runId: documentRevisions.createdByRunId })
.from(issueDocuments)
.innerJoin(documents, and(eq(documents.id, issueDocuments.documentId), eq(documents.companyId, companyId)))
.innerJoin(documentRevisions, and(eq(documentRevisions.id, documents.latestRevisionId), eq(documentRevisions.documentId, documents.id), eq(documentRevisions.companyId, companyId)))
.where(and(eq(issueDocuments.companyId, companyId), eq(issueDocuments.issueId, issueId), eq(issueDocuments.key, "plan")))
.limit(1)
: Promise.resolve([]);
const [exhaustionRows, leaseRows, executionByRunId, [savedPlan]] = await Promise.all([
exhaustionRowsQuery,
leaseRowsQuery,
executionProjectionsForRuns(db, companyId, runIds, new Date(), { retryDatabaseReads: true }),
savedPlanQuery,
]);
const retryExhaustedReasonByRunId = new Map<string, string>();
for (const row of exhaustionRows) {
if (!row.message || retryExhaustedReasonByRunId.has(row.runId)) continue;
retryExhaustedReasonByRunId.set(row.runId, row.message);
}
const leaseByRunId = new Map<string, (typeof leaseRows)[number]>();
for (const row of leaseRows) {
if (row.lease.heartbeatRunId && !leaseByRunId.has(row.lease.heartbeatRunId)) {
leaseByRunId.set(row.lease.heartbeatRunId, row);
}
}
return runs.map((run) => {
const leaseRow = leaseByRunId.get(run.runId);
const leaseMetadata = leaseRow?.lease.metadata ?? null;
const workspacePath =
typeof leaseMetadata?.remoteCwd === "string" && leaseMetadata.remoteCwd.trim().length > 0
? leaseMetadata.remoteCwd
: typeof leaseMetadata?.remoteWorkspacePath === "string" && leaseMetadata.remoteWorkspacePath.trim().length > 0
? leaseMetadata.remoteWorkspacePath
: null;
return {
...run,
resultJson: run.resultJson ? {
...run.resultJson,
...(Object.hasOwn(run.resultJson, "workspaceRestorePath") ? {
workspaceRestorePath: safeWorkspaceRestorePath(run.resultJson.workspaceRestorePath),
} : {}),
...(hasWorkspaceRestoreFailure(run.resultJson) ? {
...(savedPlan?.runId === run.runId ? { savedPlanRevisionId: savedPlan.revisionId } : {}),
} : {}),
} : null,
execution: executionByRunId.get(run.runId) ?? null,
environment: leaseRow
? {
id: leaseRow.environment.id,
name: leaseRow.environment.name,
driver: leaseRow.environment.driver,
}
: null,
environmentLease: leaseRow
? {
id: leaseRow.lease.id,
status: leaseRow.lease.status,
leasePolicy: leaseRow.lease.leasePolicy,
provider: leaseRow.lease.provider,
providerLeaseId: leaseRow.lease.providerLeaseId,
executionWorkspaceId: leaseRow.lease.executionWorkspaceId,
workspacePath,
failureReason: leaseRow.lease.failureReason,
cleanupStatus: leaseRow.lease.cleanupStatus,
acquiredAt: leaseRow.lease.acquiredAt,
releasedAt: leaseRow.lease.releasedAt,
}
: null,
retryExhaustedReason: retryExhaustedReasonByRunId.get(run.runId) ?? null,
};
});
},
issuesForRun: async (runId: string) => {
const run = await db
.select({
companyId: heartbeatRuns.companyId,
issueId: heartbeatRuns.issueId,
})
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, runId))
.then((rows) => rows[0] ?? null);
if (!run) return [];
const fromActivity = await db
.selectDistinctOn([issueIdAsText], {
issueId: issues.id,
identifier: issues.identifier,
title: issues.title,
status: issues.status,
priority: issues.priority,
})
.from(activityLog)
.innerJoin(issues, eq(activityLog.entityId, issueIdAsText))
.where(
and(
eq(activityLog.companyId, run.companyId),
eq(activityLog.runId, runId),
eq(activityLog.entityType, "issue"),
visibleIssueCondition(),
),
)
.orderBy(issueIdAsText);
const contextIssueId = run.issueId;
if (!contextIssueId) return fromActivity;
if (fromActivity.some((issue) => issue.issueId === contextIssueId)) return fromActivity;
const fromContext = await db
.select({
issueId: issues.id,
identifier: issues.identifier,
title: issues.title,
status: issues.status,
priority: issues.priority,
})
.from(issues)
.where(
and(
eq(issues.companyId, run.companyId),
eq(issues.id, contextIssueId),
visibleIssueCondition(),
),
)
.then((rows) => rows[0] ?? null);
if (!fromContext) return fromActivity;
return [fromContext, ...fromActivity];
},
create: (data: typeof activityLog.$inferInsert) =>
db
.insert(activityLog)
.values(data)
.returning()
.then((rows) => rows[0]),
};
}