mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People open tasks from the task list to see the latest messages and to reply to an agent > - Before the task chat shows its first frame, it waits for about ten separate reads: the issue, recent comments, comment input facts, recent activity, recent runs, interactions, attachments, live runs and the active run > - On a busy instance each read waits for the database, and the slowest read decides when the chat appears. A cold open takes 1.5–4 s on staging > - This pull request adds one read that returns all of that data for up to ten tasks, in the same JSON shapes as the existing endpoints, plus a single-task form > - The benefit is that the task list can preload its top tasks, and a cold open needs one round trip instead of about ten. The UI change is in the stacked pull request ## Linked Issues or Issue Description **Problem or motivation** A task open waits for about ten reads. With the database under load, these reads queue, and a cold open takes seconds. The list cannot preload a task cheaply, because each task costs about ten requests. **Proposed solution** `GET /api/companies/:companyId/issues/chat-bootstrap?ids=` returns the first-frame data for up to 10 tasks. `GET /api/issues/:ref/chat-bootstrap` returns it for one task. Each field equals the matching single endpoint's JSON exactly. **Alternatives considered** - Prefetch the existing endpoints for each list row. For 30 rows this sends about 210 requests to a database that is already the bottleneck. - A new projection with different shapes. This would duplicate client merge logic, and freshness rules could drift. **Roadmap alignment** This is part of the task-loading performance plan in #20276. It follows #20277, #20284 and #20286. ## What Changed - `packages/shared`: - `ISSUE_CHAT_BOOTSTRAP_MAX_IDS` (10). - `issueChatBootstrapQuerySchema`: UUIDs or identifiers, normalized, deduplicated and strict. - The `IssueChatBootstrap` and `IssueChatBootstrapItem` types. - New `server/src/services/issue-chat-bootstrap.ts`: - Resolves the refs inside the company and applies the `issue:read` decision to each task. - Reads the tasks with a concurrency of 2, and returns them in request order. - `server/src/routes/issues.ts`: - The core issue read and the recent-comments read are now shared helpers, so the bootstrap and the single endpoints use the same code. The core view keeps its Server-Timing header. - Adds the batch route and the single-task route. - Excludes the single-task route from the issue ETag middleware, so a fetch without validators always gets a 200 with `Cache-Control: private, no-store`. - New `server/src/services/issue-run-presence.ts`: - The live-runs and active-run payload builders, moved from `agents.ts` without change. - The existing routes call them, and their behavior is unchanged. - `server/src/routes/openapi.ts`: entries for both routes. ## Verification - `cd server && npx vitest run src/__tests__/issue-chat-bootstrap-routes.test.ts` passes 17 tests. They run on embedded Postgres with privacy enforcement on. They cover: - Each field deep-equals its single endpoint's JSON, for an ordinary task and for a conversation task. - The single-task route equals the batch item, by UUID and by identifier. It is never answered with a 304. - Items keep the request order, with UUIDs and identifiers mixed. - Private, other-company, missing and unknown tasks are omitted, and none of their data appears. The single-task route returns 404 for them. - Malformed, extra or repeated parameters return 400. More than 10 ids also return 400. - A task-bridge key gets 403. - A 10-task request writes nothing: no inserts, updates or deletes, and no activity-log or comment changes. - The issue, agent, activity, live-run, cold-open, summaries, ETag and OpenAPI route suites pass: 56 files, 1,771 tests. - `cd server && pnpm typecheck` passes. - Query cost, local embedded Postgres, one 10-task request: about 49 SQL statements per task, 55–130 ms in total. A test fails if it goes above 60 per task. ## Risks - **Database load:** the stacked UI preloads up to 30 tasks, about 1,500 statements at idle time. This is the same work as opening those tasks, but it is done before the person clicks. The work runs in sequential requests of 10, with 2 tasks at a time. Most of the cost is existing run decoration. Live runs and the active run decorate the same running run, so that work runs twice. Sharing it is a follow-up. - **Stricter access:** live runs and the active run now apply the `issue:read` policy inside this endpoint. The existing routes keep their company-only check. - Hidden tasks are returned, as `GET /issues/:id` returns them. - No migration and no persisted-state change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use (shell, browser automation, file editing, subagents). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
711 lines
30 KiB
TypeScript
711 lines
30 KiB
TypeScript
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
|
import { skillAccessService } from "./skill-access.js";
|
|
import { credentialAccessConnectionNameSql } from "./credential-access-visibility.js";
|
|
import { executionProjectionsForRuns } from "./execution-projection.js";
|
|
import { and, asc, desc, eq, inArray, isNull, or, sql, type SQL } from "drizzle-orm";
|
|
import type { Db } from "@paperclipai/db";
|
|
import {
|
|
activityLog,
|
|
agents,
|
|
documentRevisions,
|
|
documents,
|
|
environmentLeases,
|
|
environments,
|
|
heartbeatRunEvents,
|
|
heartbeatRuns,
|
|
workspaceRestoreFailureCodesSql,
|
|
issueComments,
|
|
issueDocuments,
|
|
issues,
|
|
issueWorkProducts,
|
|
workspaceOperations,
|
|
} from "@paperclipai/db";
|
|
import { hasWorkspaceRestoreFailure, safeWorkspaceRestorePath, ISSUE_CONTINUATION_SUMMARY_DOCUMENT_KEY } from "@paperclipai/shared";
|
|
import { logger } from "../middleware/logger.js";
|
|
import { visibleIssueCondition } from "./issue-visibility.js";
|
|
import { classifyRunLiveness } from "./run-liveness.js";
|
|
|
|
export interface ActivityFilters {
|
|
companyId: string;
|
|
agentId?: string;
|
|
entityType?: string;
|
|
entityId?: string;
|
|
limit?: number;
|
|
readCondition?: SQL<boolean>;
|
|
caseReadCondition?: SQL<boolean>;
|
|
}
|
|
|
|
const DEFAULT_ACTIVITY_LIMIT = 100;
|
|
const MAX_ACTIVITY_LIMIT = 500;
|
|
|
|
export function normalizeActivityLimit(limit: number | undefined) {
|
|
if (!Number.isFinite(limit)) return DEFAULT_ACTIVITY_LIMIT;
|
|
return Math.max(1, Math.min(MAX_ACTIVITY_LIMIT, Math.floor(limit ?? DEFAULT_ACTIVITY_LIMIT)));
|
|
}
|
|
|
|
export function activityService(db: Db) {
|
|
const scheduledLivenessBackfills = new Set<string>();
|
|
const issueIdAsText = sql<string>`${issues.id}::text`;
|
|
const summarizedUsageJson = sql<Record<string, unknown> | null>`
|
|
case
|
|
when ${heartbeatRuns.usageJson} is null then null
|
|
else jsonb_strip_nulls(jsonb_build_object(
|
|
'inputTokens', coalesce(${heartbeatRuns.usageJson} -> 'inputTokens', ${heartbeatRuns.usageJson} -> 'input_tokens'),
|
|
'input_tokens', coalesce(${heartbeatRuns.usageJson} -> 'input_tokens', ${heartbeatRuns.usageJson} -> 'inputTokens'),
|
|
'outputTokens', coalesce(${heartbeatRuns.usageJson} -> 'outputTokens', ${heartbeatRuns.usageJson} -> 'output_tokens'),
|
|
'output_tokens', coalesce(${heartbeatRuns.usageJson} -> 'output_tokens', ${heartbeatRuns.usageJson} -> 'outputTokens'),
|
|
'cachedInputTokens', coalesce(
|
|
${heartbeatRuns.usageJson} -> 'cachedInputTokens',
|
|
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
|
|
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens'
|
|
),
|
|
'cached_input_tokens', coalesce(
|
|
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
|
|
${heartbeatRuns.usageJson} -> 'cachedInputTokens',
|
|
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens'
|
|
),
|
|
'cache_read_input_tokens', coalesce(
|
|
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens',
|
|
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
|
|
${heartbeatRuns.usageJson} -> 'cachedInputTokens'
|
|
),
|
|
'billingType', coalesce(${heartbeatRuns.usageJson} -> 'billingType', ${heartbeatRuns.usageJson} -> 'billing_type'),
|
|
'billing_type', coalesce(${heartbeatRuns.usageJson} -> 'billing_type', ${heartbeatRuns.usageJson} -> 'billingType'),
|
|
'costUsd', coalesce(
|
|
${heartbeatRuns.usageJson} -> 'costUsd',
|
|
${heartbeatRuns.usageJson} -> 'cost_usd',
|
|
${heartbeatRuns.usageJson} -> 'total_cost_usd'
|
|
),
|
|
'cost_usd', coalesce(
|
|
${heartbeatRuns.usageJson} -> 'cost_usd',
|
|
${heartbeatRuns.usageJson} -> 'costUsd',
|
|
${heartbeatRuns.usageJson} -> 'total_cost_usd'
|
|
),
|
|
'total_cost_usd', coalesce(
|
|
${heartbeatRuns.usageJson} -> 'total_cost_usd',
|
|
${heartbeatRuns.usageJson} -> 'cost_usd',
|
|
${heartbeatRuns.usageJson} -> 'costUsd'
|
|
)
|
|
))
|
|
end
|
|
`.as("usageJson");
|
|
// A denied user needs the selected connection's name to repair their task.
|
|
// Other readers need the connection's existing human sharing permission.
|
|
// A malformed present Stop receipt must stay unconfirmed, not disappear.
|
|
const summarizedResultJson = (viewerUserId: string | null) => sql<Record<string, unknown> | null>`
|
|
case
|
|
when ${heartbeatRuns.resultJson} is null then null
|
|
else jsonb_strip_nulls(jsonb_build_object(
|
|
'conversationReset', ${heartbeatRuns.resultJson} -> 'conversationReset',
|
|
'executionCancellation', case when ${heartbeatRuns.resultJson} ? 'executionCancellation'
|
|
then jsonb_build_object('state', case
|
|
when jsonb_typeof(${heartbeatRuns.resultJson} -> 'executionCancellation') = 'object'
|
|
and ${heartbeatRuns.resultJson} #>> '{executionCancellation,state}' in ('requested', 'acknowledged')
|
|
then ${heartbeatRuns.resultJson} #>> '{executionCancellation,state}' else 'unknown' end) end,
|
|
'nativeCancellation', case
|
|
when jsonb_typeof(${heartbeatRuns.resultJson} -> 'nativeCancellation') = 'object'
|
|
and ${heartbeatRuns.resultJson} #>> '{nativeCancellation,schema}' = 'paperclip.native-cancellation.v1'
|
|
and ${heartbeatRuns.resultJson} #>> '{nativeCancellation,runId}' = ${heartbeatRuns.id}::text
|
|
and ${heartbeatRuns.resultJson} #>> '{nativeCancellation,companyId}' = ${heartbeatRuns.companyId}::text
|
|
and ${heartbeatRuns.resultJson} #>> '{nativeCancellation,dispatchState}' in ('pending', 'acknowledged')
|
|
then jsonb_build_object('schema', 'paperclip.native-cancellation.v1',
|
|
'runId', ${heartbeatRuns.id}, 'companyId', ${heartbeatRuns.companyId},
|
|
'dispatchState', ${heartbeatRuns.resultJson} #>> '{nativeCancellation,dispatchState}') end,
|
|
'configurationIncomplete', case
|
|
when ${heartbeatRuns.resultJson} -> 'configurationIncomplete' ->> 'reason' = 'workspace_base_ref_unresolved'
|
|
then jsonb_build_object('reason', 'workspace_base_ref_unresolved')
|
|
when ${heartbeatRuns.errorCode} = 'configuration_incomplete'
|
|
and (${heartbeatRuns.resultJson} #>> '{configurationIncomplete,selectionFailure}' = 'ai_connection_credential_not_shared'
|
|
or ${heartbeatRuns.error} = 'This credential is not shared with the responsible user')
|
|
then jsonb_build_object(
|
|
'selectionFailure', case when ${heartbeatRuns.resultJson} #>> '{configurationIncomplete,selectionFailure}' = 'ai_connection_credential_not_shared'
|
|
then 'ai_connection_credential_not_shared' end,
|
|
'credentialAccess', jsonb_build_object(
|
|
'connectionName', ${credentialAccessConnectionNameSql(viewerUserId)}
|
|
)
|
|
) end,
|
|
'workspaceRestoreFailure', case when ${heartbeatRuns.resultJson} ->> 'workspaceRestoreFailure'
|
|
in (${workspaceRestoreFailureCodesSql})
|
|
then ${heartbeatRuns.resultJson} -> 'workspaceRestoreFailure' end,
|
|
'workspaceRestorePath', case when length(${heartbeatRuns.resultJson} ->> 'workspaceRestorePath') <= 180
|
|
then ${heartbeatRuns.resultJson} -> 'workspaceRestorePath' end,
|
|
'finalResponseRecorded', case when jsonb_typeof(${heartbeatRuns.resultJson} -> 'finalResponseRecorded') = 'boolean'
|
|
then ${heartbeatRuns.resultJson} -> 'finalResponseRecorded' end,
|
|
'billingType', coalesce(${heartbeatRuns.resultJson} -> 'billingType', ${heartbeatRuns.resultJson} -> 'billing_type'),
|
|
'billing_type', coalesce(${heartbeatRuns.resultJson} -> 'billing_type', ${heartbeatRuns.resultJson} -> 'billingType'),
|
|
'costUsd', coalesce(
|
|
${heartbeatRuns.resultJson} -> 'costUsd',
|
|
${heartbeatRuns.resultJson} -> 'cost_usd',
|
|
${heartbeatRuns.resultJson} -> 'total_cost_usd'
|
|
),
|
|
'cost_usd', coalesce(
|
|
${heartbeatRuns.resultJson} -> 'cost_usd',
|
|
${heartbeatRuns.resultJson} -> 'costUsd',
|
|
${heartbeatRuns.resultJson} -> 'total_cost_usd'
|
|
),
|
|
'total_cost_usd', coalesce(
|
|
${heartbeatRuns.resultJson} -> 'total_cost_usd',
|
|
${heartbeatRuns.resultJson} -> 'cost_usd',
|
|
${heartbeatRuns.resultJson} -> 'costUsd'
|
|
),
|
|
'stopReason', ${heartbeatRuns.resultJson} -> 'stopReason',
|
|
'effectiveTimeoutSec', ${heartbeatRuns.resultJson} -> 'effectiveTimeoutSec',
|
|
'effectiveTimeoutMs', ${heartbeatRuns.resultJson} -> 'effectiveTimeoutMs',
|
|
'timeoutConfigured', ${heartbeatRuns.resultJson} -> 'timeoutConfigured',
|
|
'timeoutSource', ${heartbeatRuns.resultJson} -> 'timeoutSource',
|
|
'timeoutFired', ${heartbeatRuns.resultJson} -> 'timeoutFired'
|
|
))
|
|
end
|
|
`.as("resultJson");
|
|
|
|
function countValue(value: unknown) {
|
|
const parsed = Number(value ?? 0);
|
|
return Number.isFinite(parsed) ? Math.max(0, Math.floor(parsed)) : 0;
|
|
}
|
|
|
|
function dateValue(value: unknown) {
|
|
if (value instanceof Date) return Number.isNaN(value.getTime()) ? null : value;
|
|
if (typeof value === "string" || typeof value === "number") {
|
|
const parsed = new Date(value);
|
|
return Number.isNaN(parsed.getTime()) ? null : parsed;
|
|
}
|
|
return null;
|
|
}
|
|
|
|
function latestDate(...values: unknown[]) {
|
|
let latest: Date | null = null;
|
|
for (const value of values) {
|
|
const parsed = dateValue(value);
|
|
if (!parsed) continue;
|
|
if (!latest || parsed.getTime() > latest.getTime()) latest = parsed;
|
|
}
|
|
return latest;
|
|
}
|
|
|
|
function asRecord(value: unknown): Record<string, unknown> | null {
|
|
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
|
|
return value as Record<string, unknown>;
|
|
}
|
|
|
|
function readNumber(value: unknown) {
|
|
return typeof value === "number" && Number.isFinite(value) ? value : null;
|
|
}
|
|
|
|
async function backfillMissingRunLivenessForIssue(companyId: string, issueId: string) {
|
|
const runs = await db
|
|
.select({
|
|
id: heartbeatRuns.id,
|
|
companyId: heartbeatRuns.companyId,
|
|
status: heartbeatRuns.status,
|
|
contextSnapshot: heartbeatRuns.contextSnapshot,
|
|
resultJson: heartbeatRuns.resultJson,
|
|
stdoutExcerpt: heartbeatRuns.stdoutExcerpt,
|
|
stderrExcerpt: heartbeatRuns.stderrExcerpt,
|
|
error: heartbeatRuns.error,
|
|
errorCode: heartbeatRuns.errorCode,
|
|
continuationAttempt: heartbeatRuns.continuationAttempt,
|
|
})
|
|
.from(heartbeatRuns)
|
|
.where(
|
|
and(
|
|
eq(heartbeatRuns.companyId, companyId),
|
|
isNull(heartbeatRuns.livenessState),
|
|
sql`${heartbeatRuns.status} not in ('queued', 'running')`,
|
|
or(
|
|
sql`${heartbeatRuns.contextSnapshot} ->> 'issueId' = ${issueId}`,
|
|
sql`exists (
|
|
select 1
|
|
from ${activityLog}
|
|
where ${activityLog.companyId} = ${companyId}
|
|
and ${activityLog.entityType} = 'issue'
|
|
and ${activityLog.entityId} = ${issueId}
|
|
and ${activityLog.runId} = ${heartbeatRuns.id}
|
|
)`,
|
|
),
|
|
),
|
|
)
|
|
.limit(20);
|
|
|
|
if (runs.length === 0) return;
|
|
|
|
const issue = await db
|
|
.select({
|
|
status: issues.status,
|
|
title: issues.title,
|
|
description: issues.description,
|
|
workMode: issues.workMode,
|
|
})
|
|
.from(issues)
|
|
.where(and(eq(issues.companyId, companyId), eq(issues.id, issueId)))
|
|
.then((rows) => rows[0] ?? null);
|
|
|
|
for (const run of runs) {
|
|
const context = asRecord(run.contextSnapshot);
|
|
const continuationAttempt =
|
|
readNumber(context?.continuationAttempt) ??
|
|
readNumber(context?.livenessContinuationAttempt) ??
|
|
run.continuationAttempt ??
|
|
0;
|
|
|
|
const [commentStats] = await db
|
|
.select({
|
|
count: sql<number>`count(*)::int`,
|
|
latestAt: sql<Date | null>`max(${issueComments.createdAt})`,
|
|
})
|
|
.from(issueComments)
|
|
.where(
|
|
and(
|
|
eq(issueComments.companyId, companyId),
|
|
eq(issueComments.issueId, issueId),
|
|
eq(issueComments.createdByRunId, run.id),
|
|
),
|
|
);
|
|
|
|
const [documentStats] = await db
|
|
.select({
|
|
count: sql<number>`count(*)::int`,
|
|
planCount: sql<number>`count(*) filter (where ${issueDocuments.key} = 'plan')::int`,
|
|
latestAt: sql<Date | null>`max(${documentRevisions.createdAt})`,
|
|
})
|
|
.from(documentRevisions)
|
|
.innerJoin(issueDocuments, eq(documentRevisions.documentId, issueDocuments.documentId))
|
|
.where(
|
|
and(
|
|
eq(documentRevisions.companyId, companyId),
|
|
eq(documentRevisions.createdByRunId, run.id),
|
|
eq(issueDocuments.companyId, companyId),
|
|
eq(issueDocuments.issueId, issueId),
|
|
sql`${issueDocuments.key} != ${ISSUE_CONTINUATION_SUMMARY_DOCUMENT_KEY}`,
|
|
),
|
|
);
|
|
|
|
const [workProductStats] = await db
|
|
.select({
|
|
count: sql<number>`count(*)::int`,
|
|
latestAt: sql<Date | null>`max(${issueWorkProducts.createdAt})`,
|
|
})
|
|
.from(issueWorkProducts)
|
|
.where(
|
|
and(
|
|
eq(issueWorkProducts.companyId, companyId),
|
|
eq(issueWorkProducts.issueId, issueId),
|
|
eq(issueWorkProducts.createdByRunId, run.id),
|
|
),
|
|
);
|
|
|
|
const [workspaceOperationStats] = await db
|
|
.select({
|
|
count: sql<number>`count(*)::int`,
|
|
latestAt: sql<Date | null>`max(${workspaceOperations.startedAt})`,
|
|
})
|
|
.from(workspaceOperations)
|
|
.where(and(eq(workspaceOperations.companyId, companyId), eq(workspaceOperations.heartbeatRunId, run.id)));
|
|
|
|
const [activityStats] = await db
|
|
.select({
|
|
count: sql<number>`count(*)::int`,
|
|
latestAt: sql<Date | null>`max(${activityLog.createdAt})`,
|
|
})
|
|
.from(activityLog)
|
|
.where(and(eq(activityLog.companyId, companyId), eq(activityLog.runId, run.id)));
|
|
|
|
const [eventStats] = await db
|
|
.select({
|
|
count: sql<number>`count(*) filter (where ${heartbeatRunEvents.eventType} not in ('lifecycle', 'adapter.invoke', 'error'))::int`,
|
|
latestAt: sql<Date | null>`max(${heartbeatRunEvents.createdAt}) filter (where ${heartbeatRunEvents.eventType} not in ('lifecycle', 'adapter.invoke', 'error'))`,
|
|
})
|
|
.from(heartbeatRunEvents)
|
|
.where(and(eq(heartbeatRunEvents.companyId, companyId), eq(heartbeatRunEvents.runId, run.id)));
|
|
|
|
const classification = classifyRunLiveness({
|
|
runStatus: run.status,
|
|
issue,
|
|
resultJson: asRecord(run.resultJson),
|
|
stdoutExcerpt: run.stdoutExcerpt,
|
|
stderrExcerpt: run.stderrExcerpt,
|
|
error: run.error,
|
|
errorCode: run.errorCode,
|
|
continuationAttempt,
|
|
evidence: {
|
|
issueCommentsCreated: countValue(commentStats?.count),
|
|
documentRevisionsCreated: countValue(documentStats?.count),
|
|
planDocumentRevisionsCreated: countValue(documentStats?.planCount),
|
|
workProductsCreated: countValue(workProductStats?.count),
|
|
workspaceOperationsCreated: countValue(workspaceOperationStats?.count),
|
|
activityEventsCreated: countValue(activityStats?.count),
|
|
toolOrActionEventsCreated: countValue(eventStats?.count),
|
|
latestEvidenceAt: latestDate(
|
|
commentStats?.latestAt,
|
|
documentStats?.latestAt,
|
|
workProductStats?.latestAt,
|
|
workspaceOperationStats?.latestAt,
|
|
activityStats?.latestAt,
|
|
eventStats?.latestAt,
|
|
),
|
|
},
|
|
});
|
|
|
|
await db
|
|
.update(heartbeatRuns)
|
|
.set({
|
|
livenessState: classification.livenessState,
|
|
livenessReason: classification.livenessReason,
|
|
continuationAttempt: classification.continuationAttempt,
|
|
lastUsefulActionAt: classification.lastUsefulActionAt,
|
|
nextAction: classification.nextAction,
|
|
updatedAt: new Date(),
|
|
})
|
|
.where(and(eq(heartbeatRuns.id, run.id), isNull(heartbeatRuns.livenessState)));
|
|
}
|
|
}
|
|
|
|
function scheduleRunLivenessBackfill(companyId: string, issueId: string) {
|
|
const key = `${companyId}:${issueId}`;
|
|
if (scheduledLivenessBackfills.has(key)) return;
|
|
scheduledLivenessBackfills.add(key);
|
|
void backfillMissingRunLivenessForIssue(companyId, issueId)
|
|
.catch((err: unknown) => {
|
|
logger.warn({ err, companyId, issueId }, "run liveness backfill failed");
|
|
})
|
|
.finally(() => {
|
|
scheduledLivenessBackfills.delete(key);
|
|
});
|
|
}
|
|
|
|
return {
|
|
list: async (filters: ActivityFilters) => {
|
|
const conditions = [eq(activityLog.companyId, filters.companyId), await skillAccessService(db).activityCondition(filters.companyId),
|
|
filters.caseReadCondition ?? sql<boolean>`${activityLog.entityType} <> 'case'`];
|
|
const limit = normalizeActivityLimit(filters.limit);
|
|
|
|
if (filters.agentId) {
|
|
conditions.push(eq(activityLog.agentId, filters.agentId));
|
|
}
|
|
if (filters.entityType) {
|
|
conditions.push(eq(activityLog.entityType, filters.entityType));
|
|
}
|
|
if (filters.entityId) {
|
|
conditions.push(eq(activityLog.entityId, filters.entityId));
|
|
}
|
|
|
|
return db
|
|
.select({ activityLog })
|
|
.from(activityLog)
|
|
.leftJoin(
|
|
issues,
|
|
and(
|
|
eq(activityLog.entityType, sql`'issue'`),
|
|
eq(activityLog.entityId, issueIdAsText),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
...conditions,
|
|
or(
|
|
sql`${activityLog.entityType} != 'issue'`,
|
|
and(visibleIssueCondition(), filters.readCondition ?? sql<boolean>`true`),
|
|
),
|
|
),
|
|
)
|
|
.orderBy(desc(activityLog.createdAt))
|
|
.limit(limit)
|
|
.then((rows) => rows.map((r) => r.activityLog));
|
|
},
|
|
|
|
inputFactsForIssue: async (companyId: string, issueId: string, commentIds: string[]) => {
|
|
if (!commentIds.length) return [];
|
|
const commentId = sql<string>`${activityLog.details}->>'commentId'`;
|
|
return db.selectDistinctOn([commentId, activityLog.action])
|
|
.from(activityLog)
|
|
.where(and(
|
|
eq(activityLog.companyId, companyId),
|
|
eq(activityLog.entityType, "issue"),
|
|
eq(activityLog.entityId, issueId),
|
|
inArray(commentId, commentIds),
|
|
inArray(activityLog.action, ["issue.queued_comment_steered", "issue.comment_added"]),
|
|
sql`${activityLog.details}->>'duplicate' is distinct from 'true'`,
|
|
))
|
|
.orderBy(commentId, activityLog.action, asc(activityLog.createdAt), asc(activityLog.id));
|
|
},
|
|
|
|
forIssue: (issueId: string, limit?: number) => {
|
|
const query = db
|
|
.select()
|
|
.from(activityLog)
|
|
.where(
|
|
or(
|
|
and(eq(activityLog.entityType, "issue"), eq(activityLog.entityId, issueId)),
|
|
and(or(eq(activityLog.action, "project.created"), eq(activityLog.action, "company.skill_created")), sql`${activityLog.details}->>'sourceIssueId' = ${issueId}`,
|
|
sql`${activityLog.companyId} = (select company_id from issues where id = ${issueId})`),
|
|
),
|
|
)
|
|
.orderBy(desc(activityLog.createdAt), desc(activityLog.id));
|
|
return limit === undefined ? query : query.limit(limit);
|
|
},
|
|
|
|
runsForIssue: async (
|
|
companyId: string,
|
|
issueId: string,
|
|
viewerUserId: string | null = null,
|
|
limit?: number,
|
|
selectedRunIds?: string[],
|
|
// Read-only callers (the chat bootstrap) skip the background liveness
|
|
// backfill: it writes to runs and outlives the request's own limits.
|
|
// The task page's own `/runs` read still schedules it.
|
|
options: { skipLivenessBackfill?: boolean } = {},
|
|
) => {
|
|
if (selectedRunIds && !selectedRunIds.length) return [];
|
|
if (!options.skipLivenessBackfill) scheduleRunLivenessBackfill(companyId, issueId);
|
|
const runsQuery = db
|
|
.select({
|
|
runId: heartbeatRuns.id,
|
|
runtimeMode: heartbeatRuns.runtimeMode,
|
|
wakeupRequestId: heartbeatRuns.wakeupRequestId,
|
|
status: heartbeatRuns.status,
|
|
agentId: heartbeatRuns.agentId,
|
|
adapterType: agents.adapterType,
|
|
startedAt: heartbeatRuns.startedAt,
|
|
finishedAt: heartbeatRuns.finishedAt,
|
|
createdAt: heartbeatRuns.createdAt,
|
|
invocationSource: heartbeatRuns.invocationSource,
|
|
responsibleUserId: heartbeatRuns.responsibleUserId,
|
|
errorCode: heartbeatRuns.errorCode,
|
|
error: sql<string | null>`case when ${heartbeatRuns.status} = 'failed'
|
|
and ${heartbeatRuns.errorCode} = 'native_provider_model_rejected'
|
|
then left(${heartbeatRuns.error}, 2000)
|
|
when ${heartbeatRuns.status} = 'failed'
|
|
and ${heartbeatRuns.error} = 'This credential is not shared with the responsible user'
|
|
then 'This credential is not shared with the responsible user' else null end`,
|
|
usageJson: summarizedUsageJson,
|
|
resultJson: summarizedResultJson(viewerUserId),
|
|
logBytes: heartbeatRuns.logBytes,
|
|
retryOfRunId: heartbeatRuns.retryOfRunId,
|
|
scheduledRetryAt: heartbeatRuns.scheduledRetryAt,
|
|
scheduledRetryAttempt: heartbeatRuns.scheduledRetryAttempt,
|
|
scheduledRetryReason: heartbeatRuns.scheduledRetryReason,
|
|
livenessState: heartbeatRuns.livenessState,
|
|
livenessReason: heartbeatRuns.livenessReason,
|
|
continuationAttempt: heartbeatRuns.continuationAttempt,
|
|
lastUsefulActionAt: heartbeatRuns.lastUsefulActionAt,
|
|
nextAction: heartbeatRuns.nextAction,
|
|
wakeCommentIds: sql<string[] | null>`${heartbeatRuns.contextSnapshot} -> 'wakeCommentIds'`,
|
|
queuedCommentSourceWakeIds: sql<string[] | null>`${heartbeatRuns.contextSnapshot} -> 'queuedCommentSourceWakeIds'`,
|
|
wakeCommentId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'wakeCommentId'`,
|
|
contextCommentId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'commentId'`,
|
|
contextIssueId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'issueId'`,
|
|
})
|
|
.from(heartbeatRuns)
|
|
.innerJoin(
|
|
agents,
|
|
and(
|
|
eq(agents.id, heartbeatRuns.agentId),
|
|
eq(agents.companyId, heartbeatRuns.companyId),
|
|
),
|
|
)
|
|
.where(
|
|
and(
|
|
eq(heartbeatRuns.companyId, companyId),
|
|
sql`(not exists (select 1 from ${issues} where ${issues.id} = ${issueId}
|
|
and ${issues.companyId} = ${companyId} and ${issues.conversationAgentId} is not null)
|
|
or coalesce(${heartbeatRuns.nativeIssueId}::text, ${heartbeatRuns.issueId}::text,
|
|
${heartbeatRuns.contextSnapshot} ->> 'issueId', ${heartbeatRuns.contextSnapshot} ->> 'taskId') = ${issueId})`,
|
|
...(selectedRunIds ? [inArray(heartbeatRuns.id, selectedRunIds)] : []),
|
|
or(
|
|
eq(heartbeatRuns.issueId, issueId),
|
|
sql`exists (
|
|
select 1
|
|
from ${activityLog}
|
|
where ${activityLog.companyId} = ${companyId}
|
|
and ${activityLog.entityType} = 'issue'
|
|
and ${activityLog.entityId} = ${issueId}
|
|
and ${activityLog.runId} = ${heartbeatRuns.id}
|
|
)`,
|
|
),
|
|
),
|
|
)
|
|
.orderBy(desc(heartbeatRuns.createdAt), desc(heartbeatRuns.id));
|
|
const runs = await (limit === undefined ? runsQuery : runsQuery.limit(limit));
|
|
|
|
if (runs.length === 0) return runs;
|
|
const runIds = runs.map((run) => run.runId);
|
|
if (runIds.length === 0) return runs;
|
|
|
|
const exhaustionRowsQuery = db
|
|
.select({
|
|
runId: heartbeatRunEvents.runId,
|
|
message: heartbeatRunEvents.message,
|
|
})
|
|
.from(heartbeatRunEvents)
|
|
.where(
|
|
and(
|
|
inArray(heartbeatRunEvents.runId, runIds),
|
|
eq(heartbeatRunEvents.eventType, "lifecycle"),
|
|
sql`${heartbeatRunEvents.message} like 'Bounded retry exhausted%'`,
|
|
),
|
|
)
|
|
.orderBy(asc(heartbeatRunEvents.runId), desc(heartbeatRunEvents.id));
|
|
|
|
const leaseRowsQuery = db
|
|
.select({
|
|
lease: environmentLeases,
|
|
environment: {
|
|
id: environments.id,
|
|
name: environments.name,
|
|
driver: environments.driver,
|
|
},
|
|
})
|
|
.from(environmentLeases)
|
|
.innerJoin(environments, eq(environmentLeases.environmentId, environments.id))
|
|
.where(
|
|
and(
|
|
eq(environmentLeases.companyId, companyId),
|
|
inArray(environmentLeases.heartbeatRunId, runIds),
|
|
),
|
|
)
|
|
.orderBy(desc(environmentLeases.lastUsedAt), desc(environmentLeases.createdAt));
|
|
|
|
// Only stored, current plan revisions can support a saved-plan link.
|
|
// Do not trust an adapter's claim that it wrote a document.
|
|
const savedPlanQuery = runs.some((run) => hasWorkspaceRestoreFailure(run.resultJson))
|
|
? db.select({ revisionId: documentRevisions.id, runId: documentRevisions.createdByRunId })
|
|
.from(issueDocuments)
|
|
.innerJoin(documents, and(eq(documents.id, issueDocuments.documentId), eq(documents.companyId, companyId)))
|
|
.innerJoin(documentRevisions, and(eq(documentRevisions.id, documents.latestRevisionId), eq(documentRevisions.documentId, documents.id), eq(documentRevisions.companyId, companyId)))
|
|
.where(and(eq(issueDocuments.companyId, companyId), eq(issueDocuments.issueId, issueId), eq(issueDocuments.key, "plan")))
|
|
.limit(1)
|
|
: Promise.resolve([]);
|
|
const [exhaustionRows, leaseRows, executionByRunId, [savedPlan]] = await Promise.all([
|
|
exhaustionRowsQuery,
|
|
leaseRowsQuery,
|
|
executionProjectionsForRuns(db, companyId, runIds, new Date(), { retryDatabaseReads: true }),
|
|
savedPlanQuery,
|
|
]);
|
|
const retryExhaustedReasonByRunId = new Map<string, string>();
|
|
for (const row of exhaustionRows) {
|
|
if (!row.message || retryExhaustedReasonByRunId.has(row.runId)) continue;
|
|
retryExhaustedReasonByRunId.set(row.runId, row.message);
|
|
}
|
|
|
|
const leaseByRunId = new Map<string, (typeof leaseRows)[number]>();
|
|
for (const row of leaseRows) {
|
|
if (row.lease.heartbeatRunId && !leaseByRunId.has(row.lease.heartbeatRunId)) {
|
|
leaseByRunId.set(row.lease.heartbeatRunId, row);
|
|
}
|
|
}
|
|
|
|
return runs.map((run) => {
|
|
const leaseRow = leaseByRunId.get(run.runId);
|
|
const leaseMetadata = leaseRow?.lease.metadata ?? null;
|
|
const workspacePath =
|
|
typeof leaseMetadata?.remoteCwd === "string" && leaseMetadata.remoteCwd.trim().length > 0
|
|
? leaseMetadata.remoteCwd
|
|
: typeof leaseMetadata?.remoteWorkspacePath === "string" && leaseMetadata.remoteWorkspacePath.trim().length > 0
|
|
? leaseMetadata.remoteWorkspacePath
|
|
: null;
|
|
return {
|
|
...run,
|
|
resultJson: run.resultJson ? {
|
|
...run.resultJson,
|
|
...(Object.hasOwn(run.resultJson, "workspaceRestorePath") ? {
|
|
workspaceRestorePath: safeWorkspaceRestorePath(run.resultJson.workspaceRestorePath),
|
|
} : {}),
|
|
...(hasWorkspaceRestoreFailure(run.resultJson) ? {
|
|
...(savedPlan?.runId === run.runId ? { savedPlanRevisionId: savedPlan.revisionId } : {}),
|
|
} : {}),
|
|
} : null,
|
|
execution: executionByRunId.get(run.runId) ?? null,
|
|
environment: leaseRow
|
|
? {
|
|
id: leaseRow.environment.id,
|
|
name: leaseRow.environment.name,
|
|
driver: leaseRow.environment.driver,
|
|
}
|
|
: null,
|
|
environmentLease: leaseRow
|
|
? {
|
|
id: leaseRow.lease.id,
|
|
status: leaseRow.lease.status,
|
|
leasePolicy: leaseRow.lease.leasePolicy,
|
|
provider: leaseRow.lease.provider,
|
|
providerLeaseId: leaseRow.lease.providerLeaseId,
|
|
executionWorkspaceId: leaseRow.lease.executionWorkspaceId,
|
|
workspacePath,
|
|
failureReason: leaseRow.lease.failureReason,
|
|
cleanupStatus: leaseRow.lease.cleanupStatus,
|
|
acquiredAt: leaseRow.lease.acquiredAt,
|
|
releasedAt: leaseRow.lease.releasedAt,
|
|
}
|
|
: null,
|
|
retryExhaustedReason: retryExhaustedReasonByRunId.get(run.runId) ?? null,
|
|
};
|
|
});
|
|
},
|
|
|
|
issuesForRun: async (runId: string, readCondition: SQL<boolean>) => {
|
|
const run = await db
|
|
.select({
|
|
companyId: heartbeatRuns.companyId,
|
|
issueId: heartbeatRuns.issueId,
|
|
})
|
|
.from(heartbeatRuns)
|
|
.where(eq(heartbeatRuns.id, runId))
|
|
.then((rows) => rows[0] ?? null);
|
|
if (!run) return [];
|
|
|
|
const fromActivity = await db
|
|
.selectDistinctOn([issueIdAsText], {
|
|
issueId: issues.id,
|
|
identifier: issues.identifier,
|
|
title: issues.title,
|
|
status: issues.status,
|
|
priority: issues.priority,
|
|
})
|
|
.from(activityLog)
|
|
.innerJoin(issues, eq(activityLog.entityId, issueIdAsText))
|
|
.where(
|
|
and(
|
|
eq(activityLog.companyId, run.companyId),
|
|
eq(activityLog.runId, runId),
|
|
eq(activityLog.entityType, "issue"),
|
|
eq(issues.companyId, run.companyId),
|
|
visibleIssueCondition(),
|
|
readCondition,
|
|
),
|
|
)
|
|
.orderBy(issueIdAsText);
|
|
|
|
const contextIssueId = run.issueId;
|
|
if (!contextIssueId) return fromActivity;
|
|
if (fromActivity.some((issue) => issue.issueId === contextIssueId)) return fromActivity;
|
|
|
|
const fromContext = await db
|
|
.select({
|
|
issueId: issues.id,
|
|
identifier: issues.identifier,
|
|
title: issues.title,
|
|
status: issues.status,
|
|
priority: issues.priority,
|
|
})
|
|
.from(issues)
|
|
.where(
|
|
and(
|
|
eq(issues.companyId, run.companyId),
|
|
eq(issues.id, contextIssueId),
|
|
visibleIssueCondition(),
|
|
readCondition,
|
|
),
|
|
)
|
|
.then((rows) => rows[0] ?? null);
|
|
|
|
if (!fromContext) return fromActivity;
|
|
return [fromContext, ...fromActivity];
|
|
},
|
|
|
|
create: (data: typeof activityLog.$inferInsert) => db.transaction(async tx => {
|
|
if (data.entityType === "issue") await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpEvents, `task:${data.entityId}`);
|
|
const [row] = await tx.insert(activityLog).values(data).returning();
|
|
return row;
|
|
}),
|
|
};
|
|
}
|