Files
PaperClipAI/server/src/services/activity.ts
T
Devin FoleyandPaperclip 647116dd4b fix(adapter-utils): allow sandbox agents to read run diagnostics (#16575)
## Thinking Path

> - Paperclip manages agent work and its execution history.
> - Sandbox agents use a callback bridge to reach the control plane.
> - The API already provides authorized run diagnostics.
> - The bridge denied those read routes before the API could check
access.
> - Health scans could see task activity but could not inspect failed
runs.
> - This pull request admits exact GET routes on both bridge transports.
> - Linked task results also apply the caller's task-read filter.
> - Agents can inspect failures through their existing run identity.

## Linked Issues or Issue Description

**What happened?**

A sandbox health scan received `403 Route not allowed` for run history,
run details, events, and task recovery records. The same agent could
read task activity and connected provider state. The bridge stopped
these diagnostic requests before they reached the API.

**Expected behavior**

The bridge should admit the existing diagnostic GET routes. The API must
still enforce company and task access and redact run content. Execution
and recovery writes must stay denied by these rules.

**Steps to reproduce**

1. Start an agent in a sandbox with the normal callback bridge.
2. Use its injected API credential to request company run history.
3. Request a run's details or events.
4. Observe the bridge's route denial before the API handles the request.

**Paperclip version or commit**

Reproduced with the callback route list on master at `9f4a956b906`.

**Deployment mode**

Authenticated server with a managed sandbox. Both queue and HTTP/2
transports use this route list.

Related work: Refs #15854 adds separate cost-read routes to this list.
It does not add run diagnostics. Searches found no duplicate
run-diagnostic change.

## What Changed

- Admit nine existing diagnostic GET paths: company history and live
runs; run details, events, logs, workspace operations and task links;
task runs and recovery records.
- Reject other methods, extra path segments, execution control, recovery
resolution, and provider-trace administration.
- Filter linked tasks by the caller's task-read predicate in both
activity and original-task queries. Keep those results in the run's
company. A readable run must not expose private task titles.
- Test both route lists and the HTTP/2 request path. Preserve query
parameters and controller denial responses. Add a database regression
with public and private tasks linked to a readable company run.
- Document bounded diagnostic reads, private-run stubs, access checks,
and credential handling in the agent API reference. Update both affected
run access maps with the sandbox entry points and verification limits.

## Verification

- Passed both bridge suites: 139 tests.
- Passed the server run-route, activity-route, and secret-redaction
suites: 117 tests, including the new task-link filter test. The cold
route-import test needed a 30-second timeout on this host.
- Passed `pnpm -r typecheck`, `pnpm build`, the adapter-utils typecheck,
both generated capability checks, and `node scripts/check-maps.mjs`.
- Attempted `pnpm test:run`. The bundled PostgreSQL binary cannot start
on this host. The native session suite reported 15 database startup
failures; the privacy suite skipped its 15 cases. Stopped that full
local attempt after those failures. No full local-suite pass is claimed.
The final 16-case privacy suite also skipped locally for the same host
limitation; Linux CI ran all 16 cases successfully.
- One unchanged queue-response test hit a file-read race during the
combined local run. A separate rerun of both complete bridge suites
passed all 139 cases. The transient race was not independently isolated;
the original failure is retained in this record.
- All CI gates pass on head `97c65cec459d3fc2787083d05ae6ec51e3fd7b70`.
Linux CI passed all 16 database-backed run privacy cases, including the
new public/private task-link regression. These also cover private-run
denial, authorized owner/reviewer reads, redacted list stubs, credential
diagnostics, and deleted-task handling: [serialized shard
1](https://github.com/paperclipai/paperclip/actions/runs/38925932547/job/117057907621).
- Apex reviewed the final head at 5/5. Both findings are addressed and
their review threads are resolved.
- After deployment, run a new sandbox audit. Confirm it can list and
inspect authorized failed runs without a board credential. This has not
been deployed or live-tested by this PR.

## Risks

- Sandbox agents can reach more diagnostic GET routes. The existing
server access checks and redaction still apply.
- Linked task results now omit tasks that the caller cannot read.
Authorized task links remain visible.
- This change does not grant execution control or recover a failed
workspace.
- Existing bridge sessions may need a new run to load the updated route
list.
- No schema, credential, UI, workflow, or lockfile change.

## Model Used

OpenAI Codex, based on GPT-6, assisted with implementation and
verification. The exact serving model ID and context-window limit are
not exposed in this session. Capabilities used: code editing, shell
execution, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-16 09:04:07 -07:00

658 lines
27 KiB
TypeScript

import { skillAccessService } from "./skill-access.js";
import { credentialAccessConnectionNameSql } from "./credential-access-visibility.js";
import { executionProjectionsForRuns } from "./execution-projection.js";
import { and, asc, desc, eq, inArray, isNull, or, sql, type SQL } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import {
activityLog,
agents,
documentRevisions,
documents,
environmentLeases,
environments,
heartbeatRunEvents,
heartbeatRuns,
issueComments,
issueDocuments,
issues,
issueWorkProducts,
workspaceOperations,
} from "@paperclipai/db";
import { hasWorkspaceRestoreFailure, safeWorkspaceRestorePath, ISSUE_CONTINUATION_SUMMARY_DOCUMENT_KEY } from "@paperclipai/shared";
import { logger } from "../middleware/logger.js";
import { visibleIssueCondition } from "./issue-visibility.js";
import { classifyRunLiveness } from "./run-liveness.js";
export interface ActivityFilters {
companyId: string;
agentId?: string;
entityType?: string;
entityId?: string;
limit?: number;
readCondition?: SQL<boolean>;
}
const DEFAULT_ACTIVITY_LIMIT = 100;
const MAX_ACTIVITY_LIMIT = 500;
export function normalizeActivityLimit(limit: number | undefined) {
if (!Number.isFinite(limit)) return DEFAULT_ACTIVITY_LIMIT;
return Math.max(1, Math.min(MAX_ACTIVITY_LIMIT, Math.floor(limit ?? DEFAULT_ACTIVITY_LIMIT)));
}
export function activityService(db: Db) {
const scheduledLivenessBackfills = new Set<string>();
const issueIdAsText = sql<string>`${issues.id}::text`;
const summarizedUsageJson = sql<Record<string, unknown> | null>`
case
when ${heartbeatRuns.usageJson} is null then null
else jsonb_strip_nulls(jsonb_build_object(
'inputTokens', coalesce(${heartbeatRuns.usageJson} -> 'inputTokens', ${heartbeatRuns.usageJson} -> 'input_tokens'),
'input_tokens', coalesce(${heartbeatRuns.usageJson} -> 'input_tokens', ${heartbeatRuns.usageJson} -> 'inputTokens'),
'outputTokens', coalesce(${heartbeatRuns.usageJson} -> 'outputTokens', ${heartbeatRuns.usageJson} -> 'output_tokens'),
'output_tokens', coalesce(${heartbeatRuns.usageJson} -> 'output_tokens', ${heartbeatRuns.usageJson} -> 'outputTokens'),
'cachedInputTokens', coalesce(
${heartbeatRuns.usageJson} -> 'cachedInputTokens',
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens'
),
'cached_input_tokens', coalesce(
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
${heartbeatRuns.usageJson} -> 'cachedInputTokens',
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens'
),
'cache_read_input_tokens', coalesce(
${heartbeatRuns.usageJson} -> 'cache_read_input_tokens',
${heartbeatRuns.usageJson} -> 'cached_input_tokens',
${heartbeatRuns.usageJson} -> 'cachedInputTokens'
),
'billingType', coalesce(${heartbeatRuns.usageJson} -> 'billingType', ${heartbeatRuns.usageJson} -> 'billing_type'),
'billing_type', coalesce(${heartbeatRuns.usageJson} -> 'billing_type', ${heartbeatRuns.usageJson} -> 'billingType'),
'costUsd', coalesce(
${heartbeatRuns.usageJson} -> 'costUsd',
${heartbeatRuns.usageJson} -> 'cost_usd',
${heartbeatRuns.usageJson} -> 'total_cost_usd'
),
'cost_usd', coalesce(
${heartbeatRuns.usageJson} -> 'cost_usd',
${heartbeatRuns.usageJson} -> 'costUsd',
${heartbeatRuns.usageJson} -> 'total_cost_usd'
),
'total_cost_usd', coalesce(
${heartbeatRuns.usageJson} -> 'total_cost_usd',
${heartbeatRuns.usageJson} -> 'cost_usd',
${heartbeatRuns.usageJson} -> 'costUsd'
)
))
end
`.as("usageJson");
// A denied user needs the selected connection's name to repair their task.
// Other readers need the connection's existing human sharing permission.
const summarizedResultJson = (viewerUserId: string | null) => sql<Record<string, unknown> | null>`
case
when ${heartbeatRuns.resultJson} is null then null
else jsonb_strip_nulls(jsonb_build_object(
'conversationReset', ${heartbeatRuns.resultJson} -> 'conversationReset',
'configurationIncomplete', case
when ${heartbeatRuns.resultJson} -> 'configurationIncomplete' ->> 'reason' = 'workspace_base_ref_unresolved'
then jsonb_build_object('reason', 'workspace_base_ref_unresolved')
when ${heartbeatRuns.errorCode} = 'configuration_incomplete'
and (${heartbeatRuns.resultJson} #>> '{configurationIncomplete,selectionFailure}' = 'ai_connection_credential_not_shared'
or ${heartbeatRuns.error} = 'This credential is not shared with the responsible user')
then jsonb_build_object(
'selectionFailure', case when ${heartbeatRuns.resultJson} #>> '{configurationIncomplete,selectionFailure}' = 'ai_connection_credential_not_shared'
then 'ai_connection_credential_not_shared' end,
'credentialAccess', jsonb_build_object(
'connectionName', ${credentialAccessConnectionNameSql(viewerUserId)}
)
) end,
'workspaceRestoreFailure', case when ${heartbeatRuns.resultJson} ->> 'workspaceRestoreFailure'
in ('restore_permission_denied', 'restore_lock_timeout', 'restore_unsafe_archive', 'restore_failed')
then ${heartbeatRuns.resultJson} -> 'workspaceRestoreFailure' end,
'workspaceRestorePath', case when length(${heartbeatRuns.resultJson} ->> 'workspaceRestorePath') <= 180
then ${heartbeatRuns.resultJson} -> 'workspaceRestorePath' end,
'finalResponseRecorded', case when jsonb_typeof(${heartbeatRuns.resultJson} -> 'finalResponseRecorded') = 'boolean'
then ${heartbeatRuns.resultJson} -> 'finalResponseRecorded' end,
'billingType', coalesce(${heartbeatRuns.resultJson} -> 'billingType', ${heartbeatRuns.resultJson} -> 'billing_type'),
'billing_type', coalesce(${heartbeatRuns.resultJson} -> 'billing_type', ${heartbeatRuns.resultJson} -> 'billingType'),
'costUsd', coalesce(
${heartbeatRuns.resultJson} -> 'costUsd',
${heartbeatRuns.resultJson} -> 'cost_usd',
${heartbeatRuns.resultJson} -> 'total_cost_usd'
),
'cost_usd', coalesce(
${heartbeatRuns.resultJson} -> 'cost_usd',
${heartbeatRuns.resultJson} -> 'costUsd',
${heartbeatRuns.resultJson} -> 'total_cost_usd'
),
'total_cost_usd', coalesce(
${heartbeatRuns.resultJson} -> 'total_cost_usd',
${heartbeatRuns.resultJson} -> 'cost_usd',
${heartbeatRuns.resultJson} -> 'costUsd'
),
'stopReason', ${heartbeatRuns.resultJson} -> 'stopReason',
'effectiveTimeoutSec', ${heartbeatRuns.resultJson} -> 'effectiveTimeoutSec',
'effectiveTimeoutMs', ${heartbeatRuns.resultJson} -> 'effectiveTimeoutMs',
'timeoutConfigured', ${heartbeatRuns.resultJson} -> 'timeoutConfigured',
'timeoutSource', ${heartbeatRuns.resultJson} -> 'timeoutSource',
'timeoutFired', ${heartbeatRuns.resultJson} -> 'timeoutFired'
))
end
`.as("resultJson");
function countValue(value: unknown) {
const parsed = Number(value ?? 0);
return Number.isFinite(parsed) ? Math.max(0, Math.floor(parsed)) : 0;
}
function dateValue(value: unknown) {
if (value instanceof Date) return Number.isNaN(value.getTime()) ? null : value;
if (typeof value === "string" || typeof value === "number") {
const parsed = new Date(value);
return Number.isNaN(parsed.getTime()) ? null : parsed;
}
return null;
}
function latestDate(...values: unknown[]) {
let latest: Date | null = null;
for (const value of values) {
const parsed = dateValue(value);
if (!parsed) continue;
if (!latest || parsed.getTime() > latest.getTime()) latest = parsed;
}
return latest;
}
function asRecord(value: unknown): Record<string, unknown> | null {
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
return value as Record<string, unknown>;
}
function readNumber(value: unknown) {
return typeof value === "number" && Number.isFinite(value) ? value : null;
}
async function backfillMissingRunLivenessForIssue(companyId: string, issueId: string) {
const runs = await db
.select({
id: heartbeatRuns.id,
companyId: heartbeatRuns.companyId,
status: heartbeatRuns.status,
contextSnapshot: heartbeatRuns.contextSnapshot,
resultJson: heartbeatRuns.resultJson,
stdoutExcerpt: heartbeatRuns.stdoutExcerpt,
stderrExcerpt: heartbeatRuns.stderrExcerpt,
error: heartbeatRuns.error,
errorCode: heartbeatRuns.errorCode,
continuationAttempt: heartbeatRuns.continuationAttempt,
})
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.companyId, companyId),
isNull(heartbeatRuns.livenessState),
sql`${heartbeatRuns.status} not in ('queued', 'running')`,
or(
sql`${heartbeatRuns.contextSnapshot} ->> 'issueId' = ${issueId}`,
sql`exists (
select 1
from ${activityLog}
where ${activityLog.companyId} = ${companyId}
and ${activityLog.entityType} = 'issue'
and ${activityLog.entityId} = ${issueId}
and ${activityLog.runId} = ${heartbeatRuns.id}
)`,
),
),
)
.limit(20);
if (runs.length === 0) return;
const issue = await db
.select({
status: issues.status,
title: issues.title,
description: issues.description,
workMode: issues.workMode,
})
.from(issues)
.where(and(eq(issues.companyId, companyId), eq(issues.id, issueId)))
.then((rows) => rows[0] ?? null);
for (const run of runs) {
const context = asRecord(run.contextSnapshot);
const continuationAttempt =
readNumber(context?.continuationAttempt) ??
readNumber(context?.livenessContinuationAttempt) ??
run.continuationAttempt ??
0;
const [commentStats] = await db
.select({
count: sql<number>`count(*)::int`,
latestAt: sql<Date | null>`max(${issueComments.createdAt})`,
})
.from(issueComments)
.where(
and(
eq(issueComments.companyId, companyId),
eq(issueComments.issueId, issueId),
eq(issueComments.createdByRunId, run.id),
),
);
const [documentStats] = await db
.select({
count: sql<number>`count(*)::int`,
planCount: sql<number>`count(*) filter (where ${issueDocuments.key} = 'plan')::int`,
latestAt: sql<Date | null>`max(${documentRevisions.createdAt})`,
})
.from(documentRevisions)
.innerJoin(issueDocuments, eq(documentRevisions.documentId, issueDocuments.documentId))
.where(
and(
eq(documentRevisions.companyId, companyId),
eq(documentRevisions.createdByRunId, run.id),
eq(issueDocuments.companyId, companyId),
eq(issueDocuments.issueId, issueId),
sql`${issueDocuments.key} != ${ISSUE_CONTINUATION_SUMMARY_DOCUMENT_KEY}`,
),
);
const [workProductStats] = await db
.select({
count: sql<number>`count(*)::int`,
latestAt: sql<Date | null>`max(${issueWorkProducts.createdAt})`,
})
.from(issueWorkProducts)
.where(
and(
eq(issueWorkProducts.companyId, companyId),
eq(issueWorkProducts.issueId, issueId),
eq(issueWorkProducts.createdByRunId, run.id),
),
);
const [workspaceOperationStats] = await db
.select({
count: sql<number>`count(*)::int`,
latestAt: sql<Date | null>`max(${workspaceOperations.startedAt})`,
})
.from(workspaceOperations)
.where(and(eq(workspaceOperations.companyId, companyId), eq(workspaceOperations.heartbeatRunId, run.id)));
const [activityStats] = await db
.select({
count: sql<number>`count(*)::int`,
latestAt: sql<Date | null>`max(${activityLog.createdAt})`,
})
.from(activityLog)
.where(and(eq(activityLog.companyId, companyId), eq(activityLog.runId, run.id)));
const [eventStats] = await db
.select({
count: sql<number>`count(*) filter (where ${heartbeatRunEvents.eventType} not in ('lifecycle', 'adapter.invoke', 'error'))::int`,
latestAt: sql<Date | null>`max(${heartbeatRunEvents.createdAt}) filter (where ${heartbeatRunEvents.eventType} not in ('lifecycle', 'adapter.invoke', 'error'))`,
})
.from(heartbeatRunEvents)
.where(and(eq(heartbeatRunEvents.companyId, companyId), eq(heartbeatRunEvents.runId, run.id)));
const classification = classifyRunLiveness({
runStatus: run.status,
issue,
resultJson: asRecord(run.resultJson),
stdoutExcerpt: run.stdoutExcerpt,
stderrExcerpt: run.stderrExcerpt,
error: run.error,
errorCode: run.errorCode,
continuationAttempt,
evidence: {
issueCommentsCreated: countValue(commentStats?.count),
documentRevisionsCreated: countValue(documentStats?.count),
planDocumentRevisionsCreated: countValue(documentStats?.planCount),
workProductsCreated: countValue(workProductStats?.count),
workspaceOperationsCreated: countValue(workspaceOperationStats?.count),
activityEventsCreated: countValue(activityStats?.count),
toolOrActionEventsCreated: countValue(eventStats?.count),
latestEvidenceAt: latestDate(
commentStats?.latestAt,
documentStats?.latestAt,
workProductStats?.latestAt,
workspaceOperationStats?.latestAt,
activityStats?.latestAt,
eventStats?.latestAt,
),
},
});
await db
.update(heartbeatRuns)
.set({
livenessState: classification.livenessState,
livenessReason: classification.livenessReason,
continuationAttempt: classification.continuationAttempt,
lastUsefulActionAt: classification.lastUsefulActionAt,
nextAction: classification.nextAction,
updatedAt: new Date(),
})
.where(and(eq(heartbeatRuns.id, run.id), isNull(heartbeatRuns.livenessState)));
}
}
function scheduleRunLivenessBackfill(companyId: string, issueId: string) {
const key = `${companyId}:${issueId}`;
if (scheduledLivenessBackfills.has(key)) return;
scheduledLivenessBackfills.add(key);
void backfillMissingRunLivenessForIssue(companyId, issueId)
.catch((err: unknown) => {
logger.warn({ err, companyId, issueId }, "run liveness backfill failed");
})
.finally(() => {
scheduledLivenessBackfills.delete(key);
});
}
return {
list: async (filters: ActivityFilters) => {
const conditions = [eq(activityLog.companyId, filters.companyId), await skillAccessService(db).activityCondition(filters.companyId)];
const limit = normalizeActivityLimit(filters.limit);
if (filters.agentId) {
conditions.push(eq(activityLog.agentId, filters.agentId));
}
if (filters.entityType) {
conditions.push(eq(activityLog.entityType, filters.entityType));
}
if (filters.entityId) {
conditions.push(eq(activityLog.entityId, filters.entityId));
}
return db
.select({ activityLog })
.from(activityLog)
.leftJoin(
issues,
and(
eq(activityLog.entityType, sql`'issue'`),
eq(activityLog.entityId, issueIdAsText),
),
)
.where(
and(
...conditions,
or(
sql`${activityLog.entityType} != 'issue'`,
and(visibleIssueCondition(), filters.readCondition ?? sql<boolean>`true`),
),
),
)
.orderBy(desc(activityLog.createdAt))
.limit(limit)
.then((rows) => rows.map((r) => r.activityLog));
},
forIssue: (issueId: string) =>
db
.select()
.from(activityLog)
.where(
or(
and(eq(activityLog.entityType, "issue"), eq(activityLog.entityId, issueId)),
and(or(eq(activityLog.action, "project.created"), eq(activityLog.action, "company.skill_created")), sql`${activityLog.details}->>'sourceIssueId' = ${issueId}`,
sql`${activityLog.companyId} = (select company_id from issues where id = ${issueId})`),
),
)
.orderBy(desc(activityLog.createdAt)),
runsForIssue: async (companyId: string, issueId: string, viewerUserId: string | null = null) => {
scheduleRunLivenessBackfill(companyId, issueId);
const runs = await db
.select({
runId: heartbeatRuns.id,
runtimeMode: heartbeatRuns.runtimeMode,
wakeupRequestId: heartbeatRuns.wakeupRequestId,
status: heartbeatRuns.status,
agentId: heartbeatRuns.agentId,
adapterType: agents.adapterType,
startedAt: heartbeatRuns.startedAt,
finishedAt: heartbeatRuns.finishedAt,
createdAt: heartbeatRuns.createdAt,
invocationSource: heartbeatRuns.invocationSource,
responsibleUserId: heartbeatRuns.responsibleUserId,
errorCode: heartbeatRuns.errorCode,
error: sql<string | null>`case when ${heartbeatRuns.status} = 'failed'
and ${heartbeatRuns.errorCode} = 'native_provider_model_rejected'
then left(${heartbeatRuns.error}, 2000)
when ${heartbeatRuns.status} = 'failed'
and ${heartbeatRuns.error} = 'This credential is not shared with the responsible user'
then 'This credential is not shared with the responsible user' else null end`,
usageJson: summarizedUsageJson,
resultJson: summarizedResultJson(viewerUserId),
logBytes: heartbeatRuns.logBytes,
retryOfRunId: heartbeatRuns.retryOfRunId,
scheduledRetryAt: heartbeatRuns.scheduledRetryAt,
scheduledRetryAttempt: heartbeatRuns.scheduledRetryAttempt,
scheduledRetryReason: heartbeatRuns.scheduledRetryReason,
livenessState: heartbeatRuns.livenessState,
livenessReason: heartbeatRuns.livenessReason,
continuationAttempt: heartbeatRuns.continuationAttempt,
lastUsefulActionAt: heartbeatRuns.lastUsefulActionAt,
nextAction: heartbeatRuns.nextAction,
wakeCommentIds: sql<string[] | null>`${heartbeatRuns.contextSnapshot} -> 'wakeCommentIds'`,
queuedCommentSourceWakeIds: sql<string[] | null>`${heartbeatRuns.contextSnapshot} -> 'queuedCommentSourceWakeIds'`,
wakeCommentId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'wakeCommentId'`,
contextCommentId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'commentId'`,
contextIssueId: sql<string | null>`${heartbeatRuns.contextSnapshot} ->> 'issueId'`,
})
.from(heartbeatRuns)
.innerJoin(
agents,
and(
eq(agents.id, heartbeatRuns.agentId),
eq(agents.companyId, heartbeatRuns.companyId),
),
)
.where(
and(
eq(heartbeatRuns.companyId, companyId),
or(
eq(heartbeatRuns.issueId, issueId),
sql`exists (
select 1
from ${activityLog}
where ${activityLog.companyId} = ${companyId}
and ${activityLog.entityType} = 'issue'
and ${activityLog.entityId} = ${issueId}
and ${activityLog.runId} = ${heartbeatRuns.id}
)`,
),
),
)
.orderBy(desc(heartbeatRuns.createdAt));
if (runs.length === 0) return runs;
const runIds = runs.map((run) => run.runId);
if (runIds.length === 0) return runs;
const exhaustionRowsQuery = db
.select({
runId: heartbeatRunEvents.runId,
message: heartbeatRunEvents.message,
})
.from(heartbeatRunEvents)
.where(
and(
inArray(heartbeatRunEvents.runId, runIds),
eq(heartbeatRunEvents.eventType, "lifecycle"),
sql`${heartbeatRunEvents.message} like 'Bounded retry exhausted%'`,
),
)
.orderBy(asc(heartbeatRunEvents.runId), desc(heartbeatRunEvents.id));
const leaseRowsQuery = db
.select({
lease: environmentLeases,
environment: {
id: environments.id,
name: environments.name,
driver: environments.driver,
},
})
.from(environmentLeases)
.innerJoin(environments, eq(environmentLeases.environmentId, environments.id))
.where(
and(
eq(environmentLeases.companyId, companyId),
inArray(environmentLeases.heartbeatRunId, runIds),
),
)
.orderBy(desc(environmentLeases.lastUsedAt), desc(environmentLeases.createdAt));
// Only stored, current plan revisions can support a saved-plan link.
// Do not trust an adapter's claim that it wrote a document.
const savedPlanQuery = runs.some((run) => hasWorkspaceRestoreFailure(run.resultJson))
? db.select({ revisionId: documentRevisions.id, runId: documentRevisions.createdByRunId })
.from(issueDocuments)
.innerJoin(documents, and(eq(documents.id, issueDocuments.documentId), eq(documents.companyId, companyId)))
.innerJoin(documentRevisions, and(eq(documentRevisions.id, documents.latestRevisionId), eq(documentRevisions.documentId, documents.id), eq(documentRevisions.companyId, companyId)))
.where(and(eq(issueDocuments.companyId, companyId), eq(issueDocuments.issueId, issueId), eq(issueDocuments.key, "plan")))
.limit(1)
: Promise.resolve([]);
const [exhaustionRows, leaseRows, executionByRunId, [savedPlan]] = await Promise.all([
exhaustionRowsQuery,
leaseRowsQuery,
executionProjectionsForRuns(db, companyId, runIds, new Date(), { retryDatabaseReads: true }),
savedPlanQuery,
]);
const retryExhaustedReasonByRunId = new Map<string, string>();
for (const row of exhaustionRows) {
if (!row.message || retryExhaustedReasonByRunId.has(row.runId)) continue;
retryExhaustedReasonByRunId.set(row.runId, row.message);
}
const leaseByRunId = new Map<string, (typeof leaseRows)[number]>();
for (const row of leaseRows) {
if (row.lease.heartbeatRunId && !leaseByRunId.has(row.lease.heartbeatRunId)) {
leaseByRunId.set(row.lease.heartbeatRunId, row);
}
}
return runs.map((run) => {
const leaseRow = leaseByRunId.get(run.runId);
const leaseMetadata = leaseRow?.lease.metadata ?? null;
const workspacePath =
typeof leaseMetadata?.remoteCwd === "string" && leaseMetadata.remoteCwd.trim().length > 0
? leaseMetadata.remoteCwd
: typeof leaseMetadata?.remoteWorkspacePath === "string" && leaseMetadata.remoteWorkspacePath.trim().length > 0
? leaseMetadata.remoteWorkspacePath
: null;
return {
...run,
resultJson: run.resultJson ? {
...run.resultJson,
...(Object.hasOwn(run.resultJson, "workspaceRestorePath") ? {
workspaceRestorePath: safeWorkspaceRestorePath(run.resultJson.workspaceRestorePath),
} : {}),
...(hasWorkspaceRestoreFailure(run.resultJson) ? {
...(savedPlan?.runId === run.runId ? { savedPlanRevisionId: savedPlan.revisionId } : {}),
} : {}),
} : null,
execution: executionByRunId.get(run.runId) ?? null,
environment: leaseRow
? {
id: leaseRow.environment.id,
name: leaseRow.environment.name,
driver: leaseRow.environment.driver,
}
: null,
environmentLease: leaseRow
? {
id: leaseRow.lease.id,
status: leaseRow.lease.status,
leasePolicy: leaseRow.lease.leasePolicy,
provider: leaseRow.lease.provider,
providerLeaseId: leaseRow.lease.providerLeaseId,
executionWorkspaceId: leaseRow.lease.executionWorkspaceId,
workspacePath,
failureReason: leaseRow.lease.failureReason,
cleanupStatus: leaseRow.lease.cleanupStatus,
acquiredAt: leaseRow.lease.acquiredAt,
releasedAt: leaseRow.lease.releasedAt,
}
: null,
retryExhaustedReason: retryExhaustedReasonByRunId.get(run.runId) ?? null,
};
});
},
issuesForRun: async (runId: string, readCondition: SQL<boolean>) => {
const run = await db
.select({
companyId: heartbeatRuns.companyId,
issueId: heartbeatRuns.issueId,
})
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, runId))
.then((rows) => rows[0] ?? null);
if (!run) return [];
const fromActivity = await db
.selectDistinctOn([issueIdAsText], {
issueId: issues.id,
identifier: issues.identifier,
title: issues.title,
status: issues.status,
priority: issues.priority,
})
.from(activityLog)
.innerJoin(issues, eq(activityLog.entityId, issueIdAsText))
.where(
and(
eq(activityLog.companyId, run.companyId),
eq(activityLog.runId, runId),
eq(activityLog.entityType, "issue"),
eq(issues.companyId, run.companyId),
visibleIssueCondition(),
readCondition,
),
)
.orderBy(issueIdAsText);
const contextIssueId = run.issueId;
if (!contextIssueId) return fromActivity;
if (fromActivity.some((issue) => issue.issueId === contextIssueId)) return fromActivity;
const fromContext = await db
.select({
issueId: issues.id,
identifier: issues.identifier,
title: issues.title,
status: issues.status,
priority: issues.priority,
})
.from(issues)
.where(
and(
eq(issues.companyId, run.companyId),
eq(issues.id, contextIssueId),
visibleIssueCondition(),
readCondition,
),
)
.then((rows) => rows[0] ?? null);
if (!fromContext) return fromActivity;
return [fromContext, ...fromActivity];
},
create: (data: typeof activityLog.$inferInsert) =>
db
.insert(activityLog)
.values(data)
.returning()
.then((rows) => rows[0]),
};
}