mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 20:29:18 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat sends each person's message to the conversation agent through a durable outbox (`deliverConversationComments`). The outbox keys each wake receipt as `conversation-comment:<commentId>`. > - The outbox treats any receipt with that key as delivered. Queue admission writes a `skipped` receipt when the agent reached its daily run or cost limit. > - So a chat message sent after the agent used its daily allowance was consumed silently. No run started, the chat showed nothing, and nothing resumed it. > - A person's message must never be silently dropped. The daily limit is an operator control, so the message must wait honestly, not bypass the limit. > - This pull request makes the outbox hold the message at the daily limit, like it already does for a paused agent, and resume it when the limit resets. > - The benefit is that a chat message at the daily limit is saved, shown with its reason, and sent automatically at 00:00 UTC. ## Linked Issues or Issue Description No public issue exists. Inline bug report: **What happened** In a chat eval, the conversation agent (CEO) had `maxDailyRuns: 3`. After three chat turns and a separate failed-model phase on another agent, the person sent a fourth message. The comment was saved (POST 201), but no CEO run started. The conversation stayed `active` with no `executionRunId`. The ownership reconciler later reported `I7_visible_next_action`. The server log shows no wake error. Queue admission had written a `skipped` receipt (`heartbeat.daily_run_limit`) under the outbox key, so the outbox counted the message as delivered. **Expected behavior** A chat message that reaches the agent's daily limit is not consumed. It stays saved, the chat shows why it waits, and it is sent automatically when the limit resets. The daily limit still applies: the message does not start a run early. **Steps to reproduce** 1. Set a conversation agent's `runtimeConfig.heartbeat.maxDailyRuns` to N. 2. Send N chat messages and let each turn finish. 3. Send one more chat message. 4. See that no run starts, the chat shows no waiting state, and the message is never answered (a `skipped` wake receipt with reason `heartbeat.daily_run_limit` exists for `conversation-comment:<id>`). ## What Changed - `server/src/services/agent-conversations.ts`: add `conversationDeliveryHold`, which applies the same daily run/cost limit check as queue admission. The outbox returns `deferred` at the limit and writes no receipt, so the message stays owed. It reports the next UTC day start through a new optional `onDeferredDeadline` option. - `server/src/services/recovery/service.ts`: stranded-work recovery passes its `onDeferredDeadline` to the outbox, so the held message is retried when the limit resets. - `server/src/services/heartbeat/queue.ts`: if the limit is reached while an outbox wake is in admission (a race), admission throws a 409 with `reason: heartbeat.daily_*` for `conversation-comment:` keys instead of writing a `skipped` receipt. The outbox already maps a matching 409 to `deferred`. Other wakes keep the current skip behavior. - `server/src/routes/issues.ts` (`GET /issues/:id/queued-comments`): when a chat has no queue receipt but has messages held by the daily limit, return them as a `deferred` queue with `executionWait` (reason and message). Edit, send and discard stay off because the outbox owns delivery. - `ui/src/components/task-chat/TaskChatQueuedMessages.tsx`: show the `executionWait` message for such an outbox-held queue (no queue id and no target run). Receipt-backed queues keep the current behavior and do not duplicate the task status. - Tests: three real-database tests in `agent-conversation-delivery.test.ts` (hold and resume after the limit resets, recovery deadline, and a limit reached during real `heartbeatService.wakeup` admission) and one UI test. ## Verification - `cd server && pnpm exec vitest run src/__tests__/agent-conversation-delivery.test.ts`: 10 passed. Without the source change, the three new tests fail (`expected 'delivered' to be 'deferred'`; the real queue path writes the skipped receipt). - `cd server && pnpm exec vitest run src/__tests__/issue-recovery-conversations.test.ts src/__tests__/agent-conversations.test.ts src/__tests__/heartbeat-undispatched-execution-wait.test.ts src/services/heartbeat/queue.test.ts src/__tests__/agent-chat-messages.test.ts`: 215 passed. - `cd ui && pnpm exec vitest run src/components/task-chat/TaskChatQueuedMessages.test.tsx`: 36 passed. - `pnpm exec tsc --noEmit -p server/tsconfig.json`: clean. `node scripts/check-token-gates.mjs`: clean. - Manual: give a chat agent `maxDailyRuns: 1`, send two messages. The second message shows in the queued messages bar with "<agent> reached its daily run limit. Your message is saved and sends automatically when the limit resets at 00:00 UTC." ## Risks - Behavior change for chat only: a chat message at the daily limit now waits in the outbox instead of being consumed. It is sent after 00:00 UTC. If an operator raises the limit, the message is sent on the next recovery pass for that chat; the UI message does not promise an earlier send. - Saved state from the previous release: no persisted shape changes. Messages that an earlier release already consumed with a `skipped` receipt stay consumed. They are not replayed later, which avoids sending old messages without notice. - The queued-comments read adds one agent read and one daily-limit count for a chat with no queue receipt. This is the same query that admission runs. - The recovery deadline is in-process; instance-sleep scheduling does not get a new deadline from this change. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
180 lines
11 KiB
TypeScript
180 lines
11 KiB
TypeScript
import { randomUUID } from "node:crypto";
|
|
import { eq } from "drizzle-orm";
|
|
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
|
import { agents, agentWakeupRequests, companies, createDb, heartbeatRuns, issueComments, issues } from "@paperclipai/db";
|
|
import { conflict } from "../errors.js";
|
|
import { deliverConversationComments, heldConversationComments, undeliveredConversationComments } from "../services/agent-conversations.js";
|
|
import { heartbeatService } from "../services/heartbeat.js";
|
|
import { instanceSettingsService } from "../services/instance-settings.js";
|
|
import { recoveryService } from "../services/recovery/service.js";
|
|
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
|
|
|
const support = await getEmbeddedPostgresTestSupport();
|
|
(support.supported ? describe : describe.skip)("conversation outbox eligibility", () => {
|
|
let db: ReturnType<typeof createDb>;
|
|
let temporary: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
|
beforeAll(async () => {
|
|
temporary = await startEmbeddedPostgresTestDatabase("conversation-delivery-");
|
|
db = createDb(temporary.connectionString);
|
|
await instanceSettingsService(db).updateExperimental({ enableAgentChat: true });
|
|
}, 90_000);
|
|
afterAll(async () => { await db?.$client.end({ timeout: 0 }); await temporary?.cleanup(); });
|
|
|
|
async function fixture(status = "paused") {
|
|
const companyId = randomUUID(), agentId = randomUUID();
|
|
await db.insert(companies).values({ id: companyId, name: "Delivery", issuePrefix: companyId.slice(0, 8) });
|
|
await db.insert(agents).values({ id: agentId, companyId, name: "Coordinator", status, adapterType: "process" });
|
|
const [chat] = await db.insert(issues).values({ companyId, title: "Standing chat", status: "in_review",
|
|
conversationAgentId: agentId, conversationUserId: "local-board", conversationState: "waiting", assigneeAgentId: agentId }).returning();
|
|
const [comment] = await db.insert(issueComments).values({ companyId, issueId: chat.id, authorUserId: "local-board",
|
|
body: "Keep this request until the agent resumes.", clientRequestId: randomUUID() }).returning();
|
|
const pending = () => undeliveredConversationComments(db, companyId, chat.id);
|
|
return { companyId, agentId, chat, comment, pending };
|
|
}
|
|
|
|
// Queue persistence only: these tests never start an agent or provider.
|
|
const enqueue = () => vi.fn<Parameters<typeof recoveryService>[1]["enqueueWakeup"]>(async (agentId, options) => {
|
|
const [agent] = await db.select().from(agents).where(eq(agents.id, agentId));
|
|
await db.insert(agentWakeupRequests).values({ companyId: agent.companyId, agentId,
|
|
source: options?.source ?? "on_demand", status: "queued", idempotencyKey: options?.idempotencyKey,
|
|
payload: options?.payload, requestedByActorType: options?.requestedByActorType,
|
|
requestedByActorId: options?.requestedByActorId });
|
|
return null;
|
|
});
|
|
|
|
it.each(["paused", "terminated", "pending_approval"])("defers %s agents without consuming their comment", async (status) => {
|
|
const f = await fixture(status);
|
|
const wake = enqueue();
|
|
expect(await deliverConversationComments(db, f.chat, wake)).toBe("deferred");
|
|
expect(wake).not.toHaveBeenCalled();
|
|
expect((await f.pending()).map(row => row.id)).toEqual([f.comment.id]);
|
|
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, f.agentId))).toEqual([]);
|
|
});
|
|
|
|
it("reconciles other chats while paused comments remain pending, then delivers once after unpause", async () => {
|
|
const paused = await fixture(), ready = await fixture("idle");
|
|
const wake = enqueue();
|
|
const recovery = recoveryService(db, { enqueueWakeup: wake });
|
|
await expect(recovery.reconcileStrandedAssignedIssues()).resolves.toBeDefined();
|
|
expect(wake.mock.calls.map(([agentId]) => agentId)).toEqual([ready.agentId]);
|
|
expect(await paused.pending()).toHaveLength(1);
|
|
expect(await ready.pending()).toHaveLength(0);
|
|
await db.update(agents).set({ status: "idle" }).where(eq(agents.id, paused.agentId));
|
|
await recovery.reconcileStrandedAssignedIssues();
|
|
await recovery.reconcileStrandedAssignedIssues();
|
|
expect(wake.mock.calls.map(([agentId]) => agentId)).toEqual([ready.agentId, paused.agentId]);
|
|
expect(await paused.pending()).toHaveLength(0);
|
|
});
|
|
|
|
it("defers a pause racing queue admission without losing the pending comment", async () => {
|
|
const f = await fixture("idle");
|
|
const wake = vi.fn(async () => {
|
|
await db.update(agents).set({ status: "paused" }).where(eq(agents.id, f.agentId));
|
|
throw conflict("Agent is not invokable in its current state", { reason: "paused" });
|
|
});
|
|
expect(await deliverConversationComments(db, f.chat, wake)).toBe("deferred");
|
|
expect(wake).toHaveBeenCalledTimes(1);
|
|
expect(await f.pending()).toHaveLength(1);
|
|
await db.update(agents).set({ status: "idle" }).where(eq(agents.id, f.agentId));
|
|
expect(await deliverConversationComments(db, f.chat, enqueue())).toBe("delivered");
|
|
expect(await f.pending()).toHaveLength(0);
|
|
});
|
|
|
|
// recovery.browser-reconnect: the fourth chat message after a failed-model
|
|
// phase hit the agent's daily run limit; admission wrote a skipped receipt
|
|
// under the outbox key, so the message counted as delivered and never ran.
|
|
async function capReached(f: Awaited<ReturnType<typeof fixture>>, startedAt = new Date()) {
|
|
await db.update(agents).set({ runtimeConfig: { heartbeat: { maxDailyRuns: 1 } } }).where(eq(agents.id, f.agentId));
|
|
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId,
|
|
status: "succeeded", startedAt, finishedAt: startedAt }).returning();
|
|
return run;
|
|
}
|
|
const nextUtcDay = () => { const now = new Date(); return Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), now.getUTCDate() + 1); };
|
|
|
|
it("holds a message at the agent's daily limit, shows why, and sends it once the limit resets", async () => {
|
|
const f = await fixture("idle");
|
|
const run = await capReached(f);
|
|
const wake = enqueue(), deadlines: number[] = [];
|
|
expect(await deliverConversationComments(db, f.chat, wake, { onDeferredDeadline: at => deadlines.push(at) })).toBe("deferred");
|
|
expect(wake).not.toHaveBeenCalled();
|
|
expect(deadlines).toEqual([nextUtcDay()]);
|
|
expect((await f.pending()).map(row => row.id)).toEqual([f.comment.id]);
|
|
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, f.agentId))).toEqual([]);
|
|
const held = await heldConversationComments(db, f.chat);
|
|
expect(held?.hold).toMatchObject({ reason: "heartbeat.daily_run_limit", retryAt: nextUtcDay() });
|
|
expect(held?.hold.message).toContain("Your message is saved");
|
|
expect(held?.comments.map(row => row.id)).toEqual([f.comment.id]);
|
|
|
|
// The limit counts the current UTC day; yesterday's run no longer holds it.
|
|
await db.update(heartbeatRuns).set({ startedAt: new Date(Date.now() - 86_400_000) }).where(eq(heartbeatRuns.id, run.id));
|
|
expect(await heldConversationComments(db, f.chat)).toBeNull();
|
|
expect(await deliverConversationComments(db, f.chat, wake)).toBe("delivered");
|
|
expect(wake).toHaveBeenCalledTimes(1);
|
|
expect(await f.pending()).toHaveLength(0);
|
|
});
|
|
|
|
it("schedules stranded-work recovery for the limit reset instead of consuming the message", async () => {
|
|
const f = await fixture("idle");
|
|
await capReached(f);
|
|
const wake = enqueue(), deadlines: number[] = [];
|
|
await recoveryService(db, { enqueueWakeup: wake }).reconcileStrandedAssignedIssues({
|
|
companyId: f.companyId, issueId: f.chat.id, onDeferredDeadline: at => deadlines.push(at) });
|
|
expect(wake).not.toHaveBeenCalled();
|
|
expect(deadlines).toContain(nextUtcDay());
|
|
expect(await f.pending()).toHaveLength(1);
|
|
});
|
|
|
|
it("keeps the message owed when the limit is reached while real queue admission runs", async () => {
|
|
const f = await fixture("idle");
|
|
const heartbeat = heartbeatService(db);
|
|
const wake = vi.fn<Parameters<typeof deliverConversationComments>[2]>(async (agentId, options) => {
|
|
await capReached(f);
|
|
return heartbeat.wakeup(agentId, options);
|
|
});
|
|
expect(await deliverConversationComments(db, f.chat, wake)).toBe("deferred");
|
|
expect(wake).toHaveBeenCalledTimes(1);
|
|
expect((await f.pending()).map(row => row.id)).toEqual([f.comment.id]);
|
|
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, f.agentId))).toEqual([]);
|
|
});
|
|
|
|
it("makes a message owed again when dispatch cancels its accepted run at the daily limit", async () => {
|
|
const f = await fixture("idle");
|
|
await db.update(agents).set({ runtimeConfig: { heartbeat: { maxDailyRuns: 1 } } }).where(eq(agents.id, f.agentId));
|
|
const [receipt] = await db.insert(agentWakeupRequests).values({ companyId: f.companyId, agentId: f.agentId,
|
|
source: "on_demand", triggerDetail: "manual", reason: "issue_commented", status: "queued",
|
|
idempotencyKey: `conversation-comment:${f.comment.id}`, requestedByActorType: "user", requestedByActorId: "local-board",
|
|
payload: { issueId: f.chat.id, commentId: f.comment.id } }).returning();
|
|
const [queued] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: f.agentId, status: "queued",
|
|
wakeupRequestId: receipt.id, invocationSource: "on_demand",
|
|
contextSnapshot: { issueId: f.chat.id, wakeCommentId: f.comment.id, wakeCommentIds: [f.comment.id] } }).returning();
|
|
await db.update(agentWakeupRequests).set({ runId: queued.id }).where(eq(agentWakeupRequests.id, receipt.id));
|
|
expect(await f.pending()).toHaveLength(0);
|
|
// Another turn reaches the limit after the message was accepted.
|
|
await capReached(f);
|
|
await heartbeatService(db).resumeQueuedRunsForAgent(f.companyId, f.agentId);
|
|
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, queued.id)))[0])
|
|
.toMatchObject({ status: "cancelled", errorCode: "heartbeat.daily_run_limit" });
|
|
expect((await f.pending()).map(row => row.id)).toEqual([f.comment.id]);
|
|
const wake = enqueue();
|
|
expect(await deliverConversationComments(db, f.chat, wake)).toBe("deferred");
|
|
expect(wake).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("keeps unrelated admission errors visible and comments pending", async () => {
|
|
const f = await fixture("idle");
|
|
await expect(deliverConversationComments(db, f.chat, async () => {
|
|
throw conflict("Budget exhausted", { scopeType: "company" });
|
|
})).rejects.toThrow("Budget exhausted");
|
|
expect(await f.pending()).toHaveLength(1);
|
|
});
|
|
|
|
it("preserves the queue's setup exception for retaining accepted work", async () => {
|
|
const f = await fixture();
|
|
await db.update(agents).set({ lifecycleState: "preparing", lifecycleHolds: [] }).where(eq(agents.id, f.agentId));
|
|
const wake = enqueue();
|
|
expect(await deliverConversationComments(db, f.chat, wake)).toBe("delivered");
|
|
expect(wake).toHaveBeenCalledTimes(1);
|
|
expect(await f.pending()).toHaveLength(0);
|
|
});
|
|
});
|