Files
PaperClipAI/server/src/services/heartbeat-run-runtime-status.test.ts
T
Devin FoleyandPaperclip c48feee190 Improve live agent feedback during sandboxed runs (#8915)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A core part of that experience is watching active agent runs without
dropping into raw logs first
> - Local and sandbox-backed adapters already record useful run output,
progress, and tool activity
> - But active issue threads could sit visually stale while the agent
was syncing workspaces, tailing sandbox output, or emitting incremental
tool-call updates
> - Operators need timely, human-readable progress while preserving the
raw transcript underneath
> - This pull request streams sandbox run-log progress into runtime
status, keeps visible issue threads refreshed, and folds repeated ACPX
tool updates into stable transcript cards
> - The benefit is that long-running agent work becomes easier to
supervise without changing the task/comment control-plane model

## Linked Issues or Issue Description

No public GitHub issue exists for this exact change.

Problem/motivation:

- During long-running sandboxed agent work, the issue UI can appear idle
even though the agent is actively syncing, running tools, or producing
incremental output.
- Operators need realtime feedback at the issue-thread layer, not only
after opening raw logs or waiting for the final heartbeat result.
- Related public context: #1808 previously added live-run status dots to
Projects; #4362 touches heartbeat wakeup behavior but is not a duplicate
of this runtime/UI feedback change.

## What Changed

- Added sandbox run-log streaming support and defaulted sandbox-capable
local adapters into the richer live-feedback path.
- Surfaced environment/sandbox sync progress through heartbeat runtime
status with bounded, redacted snippets.
- Added live issue-thread cache patching so visible active runs update
as progress events arrive.
- Folded repeated ACPX `tool_call` updates into one transcript card
instead of stacking duplicate cards.
- Updated adapter docs and added focused regression coverage for sandbox
log streaming, runtime status, ACPX parsing, live updates, transcript
rendering, and issue chat messages.

## Verification

- `pnpm install --frozen-lockfile`
- `pnpm exec vitest run ui/src/context/LiveUpdatesProvider.test.ts`
- `pnpm exec vitest run
server/src/services/heartbeat-run-runtime-status.test.ts
server/src/__tests__/heartbeat-runtime-state.test.ts
ui/src/context/LiveUpdatesProvider.test.ts`
- `pnpm exec vitest run
packages/adapter-utils/src/execution-target-sandbox.test.ts
packages/adapter-utils/src/sandbox-managed-runtime.test.ts
server/src/services/heartbeat-run-runtime-status.test.ts
server/src/__tests__/agent-live-run-routes.test.ts
server/src/__tests__/heartbeat-runtime-state.test.ts
packages/adapters/acpx-local/src/ui/parse-stdout.test.ts
ui/src/context/LiveUpdatesProvider.test.ts
ui/src/components/transcript/RunTranscriptView.test.tsx
ui/src/lib/issue-chat-messages.test.ts
ui/src/components/IssueChatThread.test.tsx`
- GitHub PR workflow on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`:
`verify`, build, typecheck/release-registry, e2e, general shards,
serialized server shards, and canary dry run passed.
- Greptile Review on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`:
Confidence Score 5/5, no unresolved review threads.

## Risks

- Live issue-thread cache patching could miss an edge case for a route
shape not covered by tests.
- Surfacing active-run snippets needs continued care around redaction;
this PR keeps snippets bounded and adds redaction-focused coverage.
- More frequent active-run UI refreshes could expose performance issues
on very large issue threads, though updates are scoped to visible
run/query caches.

## Model Used

OpenAI GPT-5 via Codex, operating as a tool-enabled coding agent with
shell, git, and repository-editing capabilities. Context window size is
not exposed in this runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-02 22:21:56 -07:00

195 lines
6.5 KiB
TypeScript

import { afterEach, describe, expect, it } from "vitest";
import {
clearAllHeartbeatRunRuntimeStatuses,
clearHeartbeatRunRuntimeStatus,
getHeartbeatRunRuntimeStatus,
MAX_HEARTBEAT_RUN_RUNTIME_STATUS_MESSAGE_CHARS,
setHeartbeatRunRuntimeStatus,
sweepExpiredHeartbeatRunRuntimeStatuses,
touchHeartbeatRunRuntimeStatus,
} from "./heartbeat-run-runtime-status.js";
describe("heartbeat run runtime status store", () => {
afterEach(() => {
clearAllHeartbeatRunRuntimeStatuses();
});
it("stores scoped ephemeral status and expires stale entries", () => {
const updatedAt = new Date("2026-06-24T00:00:00.000Z");
const status = setHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: "issue-1",
agentId: "agent-1",
runId: "run-1",
phase: "config_sync",
message: `Syncing workspace with apiKey: "sk-test-secret" ${"x".repeat(300)}`,
currentToolName: `bash apiKey: "sk-tool-secret" ${"x".repeat(120)}`,
lastAssistantSnippet: `Reading apiKey: "sk-snippet-secret" ${"x".repeat(300)}`,
lastEventAt: new Date("2026-06-24T00:00:05.000Z"),
updatedAt,
});
expect(status?.message).toContain("***REDACTED***");
expect(status?.message.length).toBeLessThanOrEqual(MAX_HEARTBEAT_RUN_RUNTIME_STATUS_MESSAGE_CHARS);
expect(status?.currentToolName).toContain("***REDACTED***");
expect(status?.lastAssistantSnippet).toContain("***REDACTED***");
expect(status?.lastEventAt).toEqual(new Date("2026-06-24T00:00:05.000Z"));
expect(getHeartbeatRunRuntimeStatus("run-1", {
companyId: "company-1",
issueId: "issue-1",
agentId: "agent-1",
now: new Date("2026-06-24T00:00:30.000Z"),
})).toMatchObject({
companyId: "company-1",
issueId: "issue-1",
agentId: "agent-1",
runId: "run-1",
phase: "config_sync",
currentToolName: expect.stringContaining("***REDACTED***"),
lastAssistantSnippet: expect.stringContaining("***REDACTED***"),
lastEventAt: new Date("2026-06-24T00:00:05.000Z"),
});
expect(getHeartbeatRunRuntimeStatus("run-1", { companyId: "other-company" })).toBeNull();
expect(getHeartbeatRunRuntimeStatus("run-1", {
companyId: "company-1",
now: new Date("2026-06-24T00:02:00.001Z"),
})).toBeNull();
expect(getHeartbeatRunRuntimeStatus("run-1")).toBeNull();
});
it("clears status explicitly", () => {
setHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: null,
agentId: "agent-1",
runId: "run-1",
phase: "finalize",
message: "Finalizing sandbox workspace",
});
expect(clearHeartbeatRunRuntimeStatus("run-1")).toBe(true);
expect(getHeartbeatRunRuntimeStatus("run-1")).toBeNull();
});
it("touch refreshes timestamps while preserving the existing status context", () => {
setHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: "issue-1",
agentId: "agent-1",
runId: "run-1",
phase: "run_activity",
message: "Using Bash",
currentToolName: "Bash",
lastAssistantSnippet: "Running the tests",
updatedAt: new Date("2026-06-24T00:00:00.000Z"),
lastEventAt: new Date("2026-06-24T00:00:00.000Z"),
});
const touched = touchHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: "issue-1",
agentId: "agent-1",
runId: "run-1",
at: new Date("2026-06-24T00:00:45.000Z"),
});
expect(touched).toMatchObject({
runId: "run-1",
phase: "run_activity",
message: "Using Bash",
currentToolName: "Bash",
lastAssistantSnippet: "Running the tests",
updatedAt: new Date("2026-06-24T00:00:45.000Z"),
lastEventAt: new Date("2026-06-24T00:00:45.000Z"),
});
expect(getHeartbeatRunRuntimeStatus("run-1", {
companyId: "company-1",
now: new Date("2026-06-24T00:02:00.000Z"),
})).toMatchObject({ message: "Using Bash" });
});
it("touch does not move timestamps backwards", () => {
setHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: null,
agentId: "agent-1",
runId: "run-1",
phase: "run_activity",
message: "Using Bash",
updatedAt: new Date("2026-06-24T00:01:00.000Z"),
lastEventAt: new Date("2026-06-24T00:01:00.000Z"),
});
const touched = touchHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: null,
agentId: "agent-1",
runId: "run-1",
at: new Date("2026-06-24T00:00:30.000Z"),
});
expect(touched).toMatchObject({
updatedAt: new Date("2026-06-24T00:01:00.000Z"),
lastEventAt: new Date("2026-06-24T00:01:00.000Z"),
});
});
it("touch creates a fallback run_activity status when none is live", () => {
const created = touchHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: "issue-1",
agentId: "agent-1",
runId: "run-1",
at: new Date("2026-06-24T00:00:00.000Z"),
});
expect(created).toMatchObject({
runId: "run-1",
phase: "run_activity",
message: "Receiving agent output",
updatedAt: new Date("2026-06-24T00:00:00.000Z"),
lastEventAt: new Date("2026-06-24T00:00:00.000Z"),
});
// An expired entry is replaced with a fresh fallback rather than revived.
const expiredTouch = touchHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: "issue-1",
agentId: "agent-1",
runId: "run-1",
at: new Date("2026-06-24T00:05:00.000Z"),
});
expect(expiredTouch).toMatchObject({
phase: "run_activity",
message: "Receiving agent output",
updatedAt: new Date("2026-06-24T00:05:00.000Z"),
});
});
it("sweeps expired statuses without touching fresh entries", () => {
setHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: null,
agentId: "agent-1",
runId: "stale-run",
phase: "git_sync",
message: "Syncing stale workspace",
updatedAt: new Date("2026-06-24T00:00:00.000Z"),
});
setHeartbeatRunRuntimeStatus({
companyId: "company-1",
issueId: null,
agentId: "agent-1",
runId: "fresh-run",
phase: "git_sync",
message: "Syncing fresh workspace",
updatedAt: new Date("2026-06-24T00:01:00.000Z"),
});
const now = new Date("2026-06-24T00:01:31.000Z");
expect(sweepExpiredHeartbeatRunRuntimeStatuses(now)).toBe(1);
expect(getHeartbeatRunRuntimeStatus("stale-run")).toBeNull();
expect(getHeartbeatRunRuntimeStatus("fresh-run", { now })).toMatchObject({ runId: "fresh-run" });
});
});