Files
PaperClipAI/server/src/__tests__/issue-watchdogs-routes.test.ts
T
DottaandPaperclip b163c6c473 fix(server): preserve hot restart intent across path upgrade (#10593)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server can preserve eligible agent runs during a controlled hot
restart.
> - A path change moved restart state from the Paperclip home root to
the instance root.
> - A staged update can therefore make the old server and the new server
read different intent files.
> - The old server then drains live runs, while the new server can start
without a shutdown snapshot.
> - This pull request adds a correlated compatibility handoff and
records the live preflight set.
> - It also verifies the target process instance on Linux, macOS, and
Windows.
> - The benefit is complete and safe run classification across the path
upgrade.

## Linked Issues or Issue Description

No public GitHub issue covers this defect.

**What happened?**

A staged hot restart can run an older server that reads
`hot-restart-intent.json` from the Paperclip home root and a new server
that writes the file under the instance root. The old server misses the
request and uses graceful drain. The new server later finds its marker
without a shutdown snapshot. Before this change, that state could
produce an empty loss list even when live runs existed before restart.

**Expected behavior**

The old server must receive the PID-targeted restart request at its
legacy path. The new server must correlate the legacy shutdown snapshot
with its instance-scoped request. Every run that was live during
preflight must appear as adopted, finalized while down, or lost. A
reused PID must not let a stale marker claim a different process
instance.

**Steps to reproduce**

1. Start a server version from before the instance-root marker change.
2. Keep one or more local-agent heartbeat runs active.
3. Stage a current build and request a hot restart from that build.
4. Observe that the old server reads only the home-root path while the
staged build writes only the instance-root path.
5. Observe graceful drain and a new-server intent that has no shutdown
snapshot.

**Paperclip version or commit**

The path transition entered `master` in #10045. The hot-restart adoption
flow came from #9647. This fix targets current `master` and
compatibility with the immediately preceding home-root behavior.

**Deployment mode**

Self-hosted server built from source with controlled service hot
restarts.

Related work: #9628 is the original broader hot-restart feature PR.
#10556 addresses embedded PostgreSQL lifecycle behavior and does not
address marker-path compatibility.

## What Changed

- Write an authoritative instance-scoped intent and a correlated legacy
home-root handoff marker.
- Merge a legacy shutdown snapshot only when immutable request identity
fields match.
- Prevent a non-default instance from consuming an uncorrelated
legacy-only marker.
- Record preflight running heartbeat IDs and reconcile snapshot
omissions from current database state.
- Serialize marker claims, snapshot writes, stale recovery, and matching
cleanup with recoverable per-path filesystem leases.
- Read process start identity on Linux, macOS, and Windows to
distinguish a reused PID from the original server.
- Require identity for new restart requests and fail closed when a
supported platform cannot provide it.
- Classify older markers by comparing the replacement server boot time
or operating-system process start time with the request time.
- Close the preflight database client explicitly and use a root-safe SQL
query.
- Add focused unit, platform-branch, database-backed, and CI regression
coverage.
- Document the compatibility handoff, process identity probes, and
instance-scoped report path.

## Verification

- `pnpm exec vitest run server/src/services/hot-restart.test.ts
server/src/__tests__/heartbeat-process-recovery.test.ts -t
"hot-restart|old-server legacy|preflight live|preflight run|spawn
identity before hot restart"` — 24 tests passed and 90 tests were
skipped across 2 files.
- `pnpm exec vitest run server/src/services/hot-restart.test.ts` — 17
tests passed.
- `pnpm exec vitest run
server/src/__tests__/issue-watchdogs-routes.test.ts -t "restarts a
stalled claimed run"` — 1 test passed and 10 tests were skipped.
- `pnpm exec vitest run
server/src/__tests__/agent-action-audit-routes.test.ts -t "allows an
agent with issue:delegate"` — 1 test passed and 7 tests were skipped.
- `pnpm --filter @paperclipai/server typecheck` — passed.
- `git diff --check` — passed.
- GitHub Actions — 26 of 26 checks passed at
`55a79cb029be8b1dc89926d9d89ccd2181266d5c`.
- Greptile — 5/5 at the same head with no unresolved current-head review
threads.

## Risks

- The legacy handoff path is shared across instances. Exclusive claims
and per-path leases prevent overwrite and match-before-delete races.
- Process identity uses platform commands as a fallback when the health
endpoint has no identity. Linux reads `/proc`, macOS and BSD use `ps`,
and Windows uses PowerShell.
- A supported-platform identity probe failure aborts the restart. This
fails closed instead of replacing an unknown live process.
- Older intent files do not contain process identity. The server
compares the replacement boot or process start time with the request
time when those values are available.
- A preflight database read can fail before the marker is written. The
command fails closed instead of claiming a restart whose live-run set is
unknown.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment model ID and
context-window size were not exposed by this runtime. Reasoning,
repository editing, shell execution, test execution, GitHub CLI, and
Paperclip API capabilities were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-31 21:55:59 -07:00

668 lines
25 KiB
TypeScript

import { randomUUID } from "node:crypto";
import express from "express";
import request from "supertest";
import { and, eq } from "drizzle-orm";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import {
activityLog,
agentRuntimeState,
agentWakeupRequests,
agents,
companies,
companyMemberships,
companySkills,
createDb,
heartbeatRunEvents,
heartbeatRuns,
issueComments,
issueRelations,
issueWatchdogs,
issues,
principalPermissionGrants,
} from "@paperclipai/db";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
import { errorHandler } from "../middleware/index.js";
import { runningProcesses } from "../adapters/index.ts";
import { issueRoutes } from "../routes/issues.js";
import { heartbeatService } from "../services/heartbeat.js";
import { ensureHumanRoleDefaultGrants } from "../services/principal-access-compatibility.js";
import { taskWatchdogService } from "../services/task-watchdogs.js";
import { drainHeartbeatRunsToQuiescence } from "./helpers/drain-heartbeat-runs.js";
const mockAdapterExecute = vi.hoisted(() =>
vi.fn(async () => ({
exitCode: 0,
signal: null,
timedOut: false,
errorMessage: null,
summary: "Issue watchdog route test run.",
provider: "test",
model: "test-model",
})),
);
vi.mock("../adapters/index.ts", async () => {
const actual = await vi.importActual<typeof import("../adapters/index.ts")>("../adapters/index.ts");
return {
...actual,
getServerAdapter: vi.fn(() => ({
supportsLocalAgentJwt: false,
execute: mockAdapterExecute,
})),
};
});
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
if (!embeddedPostgresSupport.supported) {
console.warn(
`Skipping embedded Postgres issue watchdog route tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
);
}
describeEmbeddedPostgres("issue watchdog routes", () => {
let db!: ReturnType<typeof createDb>;
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
beforeAll(async () => {
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-issue-watchdogs-routes-");
db = createDb(tempDb.connectionString);
}, 20_000);
afterEach(async () => {
mockAdapterExecute.mockClear();
runningProcesses.clear();
await drainHeartbeatRunsToQuiescence(db, heartbeatService(db));
await db.delete(activityLog);
await db.delete(issueComments);
await db.delete(heartbeatRunEvents);
await db.delete(heartbeatRuns);
await db.delete(agentWakeupRequests);
await db.delete(agentRuntimeState);
await db.delete(issueRelations);
await db.delete(issueWatchdogs);
await db.delete(issues);
await db.delete(agents);
await db.delete(companySkills);
await db.delete(principalPermissionGrants);
await db.delete(companyMemberships);
await db.delete(companies);
});
afterAll(async () => {
await tempDb?.cleanup();
});
function createApp(companyId: string, actor?: Record<string, unknown>) {
const app = express();
app.use(express.json());
app.use((req, _res, next) => {
(req as any).actor = actor ?? {
type: "board",
userId: "cloud-user-1",
companyIds: [companyId],
memberships: [{ companyId, membershipRole: "owner", status: "active" }],
source: "cloud_tenant",
isInstanceAdmin: false,
};
next();
});
app.use("/api", issueRoutes(db, {} as any, { taskWatchdogEnqueueWakeup: null }));
app.use(errorHandler);
return app;
}
function uniqueIssuePrefix() {
return `W${randomUUID().replace(/-/g, "").slice(0, 5).toUpperCase()}`;
}
async function seedCloudTenantMember(companyId: string) {
await db.insert(companyMemberships).values({
companyId,
principalType: "user",
principalId: "cloud-user-1",
status: "active",
membershipRole: "owner",
updatedAt: new Date(),
});
await ensureHumanRoleDefaultGrants(db, {
companyId,
principalId: "cloud-user-1",
membershipRole: "owner",
grantedByUserId: null,
});
}
async function seedCompany(name = "Paperclip") {
const companyId = randomUUID();
await db.insert(companies).values({
id: companyId,
name,
issuePrefix: uniqueIssuePrefix(),
requireBoardApprovalForNewAgents: false,
});
await seedCloudTenantMember(companyId);
return companyId;
}
async function seedAgent(companyId: string, overrides: Partial<typeof agents.$inferInsert> = {}) {
const id = overrides.id ?? randomUUID();
await db.insert(agents).values({
id,
companyId,
name: overrides.name ?? "Watchdog Agent",
role: overrides.role ?? "engineer",
status: overrides.status ?? "active",
adapterType: overrides.adapterType ?? "codex_local",
adapterConfig: overrides.adapterConfig ?? {},
runtimeConfig: overrides.runtimeConfig ?? {},
permissions: overrides.permissions ?? {},
reportsTo: overrides.reportsTo,
});
return id;
}
async function seedIssue(companyId: string, overrides: Partial<typeof issues.$inferInsert> = {}) {
const id = overrides.id ?? randomUUID();
await db.insert(issues).values({
id,
companyId,
title: overrides.title ?? "Watched task",
status: overrides.status ?? "todo",
priority: overrides.priority ?? "medium",
identifier: overrides.identifier,
issueNumber: overrides.issueNumber,
assigneeAgentId: overrides.assigneeAgentId,
parentId: overrides.parentId,
projectId: overrides.projectId,
goalId: overrides.goalId,
originKind: overrides.originKind,
originId: overrides.originId,
// Default to an "established" issue (created before the first-run grace
// window) so attaching a watchdog evaluates immediately instead of being
// deferred by the pending-first-run guard.
createdAt: overrides.createdAt ?? new Date(Date.now() - 60 * 60 * 1000),
});
return id;
}
async function seedWatchdogRun(input: {
companyId: string;
watchdogAgentId: string;
watchedIssueId: string;
watchdogIssueId: string;
}) {
await db.insert(issueWatchdogs).values({
companyId: input.companyId,
issueId: input.watchedIssueId,
watchdogAgentId: input.watchdogAgentId,
watchdogIssueId: input.watchdogIssueId,
status: "active",
});
await taskWatchdogService(db).reconcileTaskWatchdogs({ companyId: input.companyId });
const [watchdog] = await db
.select({ lastObservedFingerprint: issueWatchdogs.lastObservedFingerprint })
.from(issueWatchdogs)
.where(and(eq(issueWatchdogs.companyId, input.companyId), eq(issueWatchdogs.issueId, input.watchedIssueId)));
const runId = randomUUID();
await db.insert(heartbeatRuns).values({
id: runId,
companyId: input.companyId,
agentId: input.watchdogAgentId,
status: "running",
contextSnapshot: {
issueId: input.watchdogIssueId,
taskWatchdog: {
watchedIssueId: input.watchedIssueId,
watchedIssueIdentifier: "WDOG-ROOT",
watchedIssueTitle: "Watched root",
stopFingerprint: watchdog?.lastObservedFingerprint,
},
},
});
return runId;
}
async function waitForAssignmentWakeup(companyId: string) {
for (let attempt = 0; attempt < 20; attempt += 1) {
const rows = await db
.select({ id: heartbeatRuns.id })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.companyId, companyId))
.limit(1);
if (rows.length > 0) return;
await new Promise((resolve) => setTimeout(resolve, 10));
}
}
it("creates, updates, reads, lists, and removes an issue watchdog with activity logs", async () => {
const companyId = await seedCompany();
const issueId = await seedIssue(companyId, { identifier: "WDOG-1", issueNumber: 1 });
const firstAgentId = await seedAgent(companyId, { name: "First Watchdog" });
const secondAgentId = await seedAgent(companyId, { name: "Second Watchdog" });
const app = createApp(companyId);
const created = await request(app)
.put(`/api/issues/${issueId}/watchdog`)
.send({ agentId: firstAgentId, instructions: "Check screenshots and tests." });
expect(created.status, JSON.stringify(created.body)).toBe(200);
expect(created.body).toMatchObject({
issueId,
watchdogAgentId: firstAgentId,
instructions: "Check screenshots and tests.",
status: "active",
});
const updated = await request(app)
.put(`/api/issues/${issueId}/watchdog`)
.send({ agentId: secondAgentId, instructions: "Be skeptical." });
expect(updated.status, JSON.stringify(updated.body)).toBe(200);
expect(updated.body.id).toBe(created.body.id);
expect(updated.body).toMatchObject({
issueId,
watchdogAgentId: secondAgentId,
instructions: "Be skeptical.",
status: "active",
});
const read = await request(app).get(`/api/issues/${issueId}/watchdog`);
expect(read.status, JSON.stringify(read.body)).toBe(200);
expect(read.body).toMatchObject({ id: created.body.id, watchdogAgentId: secondAgentId });
const detail = await request(app).get(`/api/issues/${issueId}`);
expect(detail.status, JSON.stringify(detail.body)).toBe(200);
expect(detail.body.watchdog).toMatchObject({ id: created.body.id, watchdogAgentId: secondAgentId });
const list = await request(app).get(`/api/companies/${companyId}/issues`);
expect(list.status, JSON.stringify(list.body)).toBe(200);
expect(list.body.find((issue: { id: string }) => issue.id === issueId)?.watchdog)
.toMatchObject({ id: created.body.id, watchdogAgentId: secondAgentId });
const removed = await request(app).delete(`/api/issues/${issueId}/watchdog`);
expect(removed.status, JSON.stringify(removed.body)).toBe(200);
expect(removed.body).toEqual({ ok: true });
const afterDelete = await request(app).get(`/api/issues/${issueId}/watchdog`);
expect(afterDelete.status, JSON.stringify(afterDelete.body)).toBe(200);
expect(afterDelete.body).toBeNull();
const stored = await db
.select()
.from(issueWatchdogs)
.where(and(eq(issueWatchdogs.companyId, companyId), eq(issueWatchdogs.issueId, issueId)))
.then((rows) => rows[0] ?? null);
expect(stored).toMatchObject({
id: created.body.id,
status: "disabled",
watchdogAgentId: secondAgentId,
});
const actions = await db
.select({ action: activityLog.action })
.from(activityLog)
.where(eq(activityLog.entityId, issueId));
const actionNames = actions.map((row) => row.action);
expect(actionNames.filter((action) => action.startsWith("issue.watchdog_"))).toEqual([
"issue.watchdog_created",
"issue.watchdog_updated",
"issue.watchdog_removed",
]);
expect(actionNames).toContain("issue.task_watchdog_triggered");
});
it("handles concurrent first-time watchdog upserts without duplicate-key failures", async () => {
const companyId = await seedCompany();
const issueId = await seedIssue(companyId, { identifier: "WDOG-RACE", issueNumber: 99 });
const agentId = await seedAgent(companyId, { name: "Race Watchdog" });
const app = createApp(companyId);
const responses = await Promise.all(
Array.from({ length: 12 }, (_, index) =>
request(app)
.put(`/api/issues/${issueId}/watchdog`)
.send({ agentId, instructions: `Concurrent instructions ${index}` }),
),
);
expect(responses.map((res) => res.status), JSON.stringify(responses.map((res) => res.body)))
.toEqual(Array(12).fill(200));
const stored = await db
.select()
.from(issueWatchdogs)
.where(and(eq(issueWatchdogs.companyId, companyId), eq(issueWatchdogs.issueId, issueId)));
expect(stored).toHaveLength(1);
expect(stored[0]).toMatchObject({ status: "active", watchdogAgentId: agentId });
});
it("creates an issue and watchdog atomically from the create issue route", async () => {
const companyId = await seedCompany();
const agentId = await seedAgent(companyId);
const app = createApp(companyId);
const res = await request(app)
.post(`/api/companies/${companyId}/issues`)
.send({
title: "Create with watchdog",
watchdog: {
agentId,
instructions: "Confirm the final state.",
},
});
expect(res.status, JSON.stringify(res.body)).toBe(201);
expect(res.body.watchdog).toMatchObject({
issueId: res.body.id,
watchdogAgentId: agentId,
instructions: "Confirm the final state.",
status: "active",
});
const rows = await db
.select()
.from(issueWatchdogs)
.where(eq(issueWatchdogs.issueId, res.body.id));
expect(rows).toHaveLength(1);
const activityRows = await db
.select({ action: activityLog.action })
.from(activityLog)
.where(eq(activityLog.entityId, res.body.id));
expect(activityRows.map((row) => row.action)).toContain("issue.watchdog_created");
});
it("does not create an immediate watchdog review for a newly assigned issue", async () => {
const companyId = await seedCompany();
const workerAgentId = await seedAgent(companyId, { name: "Worker" });
const watchdogAgentId = await seedAgent(companyId, { name: "Watchdog" });
const app = createApp(companyId);
const res = await request(app)
.post(`/api/companies/${companyId}/issues`)
.send({
title: "Assigned issue with watchdog",
assigneeAgentId: workerAgentId,
watchdog: {
agentId: watchdogAgentId,
instructions: "Confirm whether the worker got started.",
},
});
expect(res.status, JSON.stringify(res.body)).toBe(201);
await waitForAssignmentWakeup(companyId);
expect(res.body).toMatchObject({
assigneeAgentId: workerAgentId,
watchdog: {
issueId: res.body.id,
watchdogAgentId,
status: "active",
},
});
const watchdogReviewIssues = await db
.select()
.from(issues)
.where(and(eq(issues.companyId, companyId), eq(issues.originKind, "task_watchdog")));
expect(watchdogReviewIssues).toHaveLength(0);
const [watchdog] = await db
.select()
.from(issueWatchdogs)
.where(and(eq(issueWatchdogs.companyId, companyId), eq(issueWatchdogs.issueId, res.body.id)));
expect(watchdog?.triggerCount).toBe(0);
const taskWatchdogActivity = await db
.select({ action: activityLog.action })
.from(activityLog)
.where(and(eq(activityLog.entityId, res.body.id), eq(activityLog.action, "issue.task_watchdog_triggered")));
expect(taskWatchdogActivity).toHaveLength(0);
});
it("enforces persisted watchdog scope for issue mutations and child creation", async () => {
const companyId = await seedCompany();
const watchdogAgentId = await seedAgent(companyId, { name: "Scoped Watchdog" });
const watchedRootId = await seedIssue(companyId, { title: "Watched root", identifier: "WDOG-ROOT" });
const watchedChildId = await seedIssue(companyId, { title: "Watched child", parentId: watchedRootId });
const unrelatedRootId = await seedIssue(companyId, { title: "Unrelated root" });
const watchdogIssueId = await seedIssue(companyId, {
title: "Reusable watchdog issue",
parentId: watchedRootId,
assigneeAgentId: watchdogAgentId,
originKind: "task_watchdog",
originId: watchedRootId,
});
const watchdogIssueChildId = await seedIssue(companyId, {
title: "Watchdog issue child",
parentId: watchdogIssueId,
});
const runId = await seedWatchdogRun({
companyId,
watchdogAgentId,
watchedIssueId: watchedRootId,
watchdogIssueId,
});
const app = createApp(companyId, {
type: "agent",
agentId: watchdogAgentId,
companyId,
runId,
source: "agent_jwt",
});
const watchdogIssuePatch = await request(app)
.patch(`/api/issues/${watchdogIssueId}`)
.send({ title: "Reusable watchdog issue completed" });
expect(watchdogIssuePatch.status, JSON.stringify(watchdogIssuePatch.body)).toBe(200);
const deniedWatchdogDescendantPatch = await request(app)
.patch(`/api/issues/${watchdogIssueChildId}`)
.send({ title: "Denied watchdog descendant mutation" });
expect(deniedWatchdogDescendantPatch.status, JSON.stringify(deniedWatchdogDescendantPatch.body)).toBe(403);
expect(deniedWatchdogDescendantPatch.body.error).toBe(
"Task-watchdog runs can only mutate the watched issue subtree.",
);
const deniedPatch = await request(app)
.patch(`/api/issues/${unrelatedRootId}`)
.send({ title: "Out-of-scope mutation" });
expect(deniedPatch.status, JSON.stringify(deniedPatch.body)).toBe(403);
expect(deniedPatch.body.error).toBe("Task-watchdog runs can only mutate the watched issue subtree.");
const deniedChild = await request(app)
.post(`/api/issues/${unrelatedRootId}/children`)
.send({ title: "Denied unrelated child" });
expect(deniedChild.status, JSON.stringify(deniedChild.body)).toBe(403);
expect(deniedChild.body.error).toBe("Task-watchdog runs can only mutate the watched issue subtree.");
const deniedWatchdogIssueChild = await request(app)
.post(`/api/issues/${watchdogIssueId}/children`)
.send({ title: "Denied watchdog issue child" });
expect(deniedWatchdogIssueChild.status, JSON.stringify(deniedWatchdogIssueChild.body)).toBe(403);
const deniedVisibleProbeIssues = await db
.select({ id: issues.id })
.from(issues)
.where(and(eq(issues.companyId, companyId), eq(issues.title, "Denied watchdog issue child")));
expect(deniedVisibleProbeIssues).toHaveLength(0);
const deniedParentCreate = await request(app)
.post(`/api/companies/${companyId}/issues`)
.send({ title: "Denied parent create", parentId: unrelatedRootId });
expect(deniedParentCreate.status, JSON.stringify(deniedParentCreate.body)).toBe(403);
expect(deniedParentCreate.body.error).toBe("Task-watchdog runs can only mutate the watched issue subtree.");
const deniedNestedWatchdog = await request(app)
.put(`/api/issues/${watchedChildId}/watchdog`)
.send({ agentId: watchdogAgentId, instructions: "Create a nested watchdog" });
expect(deniedNestedWatchdog.status, JSON.stringify(deniedNestedWatchdog.body)).toBe(403);
expect(deniedNestedWatchdog.body.error).toBe("Task-watchdog runs cannot change watchdog configuration.");
const deniedWatchdogRemoval = await request(app).delete(`/api/issues/${watchedRootId}/watchdog`);
expect(deniedWatchdogRemoval.status, JSON.stringify(deniedWatchdogRemoval.body)).toBe(403);
expect(deniedWatchdogRemoval.body.error).toBe("Task-watchdog runs cannot change watchdog configuration.");
const nestedWatchdogs = await db
.select({ id: issueWatchdogs.id })
.from(issueWatchdogs)
.where(and(eq(issueWatchdogs.companyId, companyId), eq(issueWatchdogs.issueId, watchedChildId)));
expect(nestedWatchdogs).toHaveLength(0);
const allowedChild = await request(app)
.post(`/api/issues/${watchedChildId}/children`)
.send({ title: "Allowed watched child" });
expect(allowedChild.status, JSON.stringify(allowedChild.body)).toBe(201);
expect(allowedChild.body.parentId).toBe(watchedChildId);
});
it("routes watchdog-discovered product bugs outside the watched source tree with evidence links", async () => {
const companyId = await seedCompany();
const watchdogAgentId = await seedAgent(companyId, { name: "Product Bug Watchdog" });
const watchedRootId = await seedIssue(companyId, {
title: "Watched root",
identifier: "PAP-100",
issueNumber: 100,
});
const watchedChildId = await seedIssue(companyId, {
title: "Watched child",
identifier: "PAP-101",
issueNumber: 101,
parentId: watchedRootId,
});
const watchdogIssueId = await seedIssue(companyId, {
title: "Reusable watchdog issue",
identifier: "PAP-102",
issueNumber: 102,
parentId: watchedRootId,
assigneeAgentId: watchdogAgentId,
originKind: "task_watchdog",
originId: watchedRootId,
});
const runId = await seedWatchdogRun({
companyId,
watchdogAgentId,
watchedIssueId: watchedRootId,
watchdogIssueId,
});
const app = createApp(companyId, {
type: "agent",
agentId: watchdogAgentId,
companyId,
runId,
source: "agent_jwt",
});
const res = await request(app)
.post(`/api/companies/${companyId}/issues`)
.send({
title: "Fix watchdog source-tree pollution",
description: "Watchdog found a Paperclip follow-up routing bug.",
parentId: watchedChildId,
watchdogDiscovery: {
kind: "product_bug",
evidenceMarkdown: "The watchdog would otherwise create this under the watched child.",
},
});
expect(res.status, JSON.stringify(res.body)).toBe(201);
expect(res.body).toMatchObject({
title: "Fix watchdog source-tree pollution",
parentId: null,
originKind: "task_watchdog_product_bug",
originId: watchedRootId,
originRunId: runId,
});
expect(res.body.description).toContain("## Watchdog Discovery");
expect(res.body.description).toContain("Watched source issue: [PAP-100](/PAP/issues/PAP-100)");
expect(res.body.description).toContain("Watchdog issue: [PAP-102](/PAP/issues/PAP-102)");
expect(res.body.referencedIssueIdentifiers).toEqual(expect.arrayContaining(["PAP-100", "PAP-102"]));
const watchedSourceChildren = await db
.select({ id: issues.id, title: issues.title })
.from(issues)
.where(and(eq(issues.companyId, companyId), eq(issues.parentId, watchedChildId)));
expect(watchedSourceChildren).toHaveLength(0);
const [createdActivity] = await db
.select({ details: activityLog.details })
.from(activityLog)
.where(and(eq(activityLog.companyId, companyId), eq(activityLog.entityId, res.body.id)));
expect(createdActivity?.details).toMatchObject({
watchdogDiscovery: {
kind: "product_bug",
sourceIssueId: watchedRootId,
watchdogIssueId,
},
});
});
it("rejects watchdog interaction-resolution attempts outside the persisted watched subtree", async () => {
const companyId = await seedCompany();
const watchdogAgentId = await seedAgent(companyId, { name: "Interaction Watchdog" });
const watchedRootId = await seedIssue(companyId, { title: "Watched root" });
const unrelatedRootId = await seedIssue(companyId, { title: "Unrelated root" });
const watchdogIssueId = await seedIssue(companyId, { title: "Reusable watchdog issue" });
const runId = await seedWatchdogRun({
companyId,
watchdogAgentId,
watchedIssueId: watchedRootId,
watchdogIssueId,
});
const app = createApp(companyId, {
type: "agent",
agentId: watchdogAgentId,
companyId,
runId,
source: "agent_jwt",
});
const res = await request(app)
.post(`/api/issues/${unrelatedRootId}/interactions/${randomUUID()}/accept`)
.send({});
expect(res.status, JSON.stringify(res.body)).toBe(403);
expect(res.body.error).toBe("Task-watchdog runs can only mutate the watched issue subtree.");
});
it("rejects cross-company watched issues and watchdog agents", async () => {
const companyId = await seedCompany("Allowed company");
const otherCompanyId = await seedCompany("Other company");
const issueId = await seedIssue(companyId);
const otherIssueId = await seedIssue(otherCompanyId);
const otherAgentId = await seedAgent(otherCompanyId);
const app = createApp(companyId);
const foreignIssue = await request(app)
.put(`/api/issues/${otherIssueId}/watchdog`)
.send({ agentId: otherAgentId });
// Uniform 404 so cross-tenant ids are indistinguishable from missing ones.
expect(foreignIssue.status, JSON.stringify(foreignIssue.body)).toBe(404);
expect(foreignIssue.body.error).toBe("Issue not found");
const foreignAgent = await request(app)
.put(`/api/issues/${issueId}/watchdog`)
.send({ agentId: otherAgentId });
expect(foreignAgent.status, JSON.stringify(foreignAgent.body)).toBe(404);
});
it.each(["paused", "terminated", "pending_approval"])(
"rejects %s watchdog agents",
async (status) => {
const companyId = await seedCompany();
const issueId = await seedIssue(companyId);
const agentId = await seedAgent(companyId, { status });
const app = createApp(companyId);
const res = await request(app)
.put(`/api/issues/${issueId}/watchdog`)
.send({ agentId });
expect(res.status, JSON.stringify(res.body)).toBe(409);
expect(res.body.error).toBe("Cannot assign watchdog to an agent that is not invokable");
},
);
});