mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connected chats show safe notices when a run starts, waits, finishes, or fails. > - A recurring scan projected these notices even when no eligible work existed. > - Eligibility depends on committed run state, conversation lineage, and current authority. > - Existing database rows can reconstruct that work after restart. > - This PR wakes a scoped worker from those writes and removes lifecycle polling. > - Native progress events keep their separate existing owner until the next migration. ## Linked Issues or Issue Description **What happened?** The chat maintenance timer repeatedly scanned run lifecycle state. Empty and settled instances still queried the database. A run-status event alone cannot replace the scan: prompt receipts, continuation receipts, identity changes, and restored authority can make a run eligible later. **What did you expect?** Committed changes wake projection. Startup reconstructs eligible work. A completed pass stops until another relevant change arrives. Failed work retains recovery without scanning settled history forever. **Steps to reproduce** Configure an automatic chat endpoint. Finish its work, or leave it unused. Observe recurring lifecycle projection queries from the chat maintenance callback. Stacked on #17592. Related foundation: #15625. This PR owns lifecycle milestones only. ## What Changed - Preserve upstream coalesced Cloud membership writes together with locked membership restoration and its committed chat notification. - Signal milestone eligibility from the real Slack account-link transaction. - Register the lifecycle worker with the existing delivery coordinator. Remove lifecycle projection from the periodic chat callback. Keep native progress in a separately named lane and preserve the explicit combined API. - Reconstruct company/issue scopes from the existing selector. Drain bounded batches with company fairness. Retry failures, retain writes through drain/resume, and disarm settled work. - Signal from supported run, presentation, interaction, receipt, lineage, conversation, recovery, and authority writers. Preserve their transaction ownership, including nested savepoints and uncertain settlement. No new table is added. - Preserve provider authorization, authored-final suppression, responsible-user continuity, deduplication, the 24-hour selection window, and raw-output restrictions. Keep provider interpretation in its owner. - Preserve PostgreSQL timestamp precision in lifecycle pagination. Rejected submillisecond candidates must not repeat the same page and starve later notices. - Document producer coverage and alternate-writer dispositions in `doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`. ## Verification Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting master after #17592 merged at `7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates the actual landed parent. The entire 77-path milestone patch is binary-identical to the locally verified preparation across that ancestry reconciliation; both unrelated master paths are preserved. The four earlier producer/test integration conflicts retain scoped milestone and voice notices, current onboarding completion admission, and expanded identity query/read-scope coverage. - **330 distinct selected cases pass on the verified unchanged patch:** 305 across 14 complete suites (all five milestone worker suites, run identity, board claim, Cloud provisioning, auth session routes, Cloud actor authentication, stale-lock recovery, GitHub credentials, project Git authorization, and app admission wiring), plus 25 selected real chat producer cases. The remaining 1,590 chat integration cases were skipped, not passed. - The real producer cases cover saved milestone discovery, GitHub and Slack identity changes, account-link rollback/reconfirmation, ordinary ingress reactivation, post-close action/delivery receipt settlement, all ten authority-restoration paths including Cloud synchronization, and changed native review metadata. - Worker coverage verifies outer commit, rollback, uncertain-write recovery, restart, bounded batching/fairness, timestamp precision, retries, drain/resume, and settled SQL quiet. Authorization and existing publication deduplication remain with their owning services. - Server production typechecking, build, and module-boundary checks pass. The tests above were executed separately. Production typechecking excludes `src/__tests__` and does not replace those runs. Each database owner uses its own newly created and app-migrated PostgreSQL fixture. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28) records commands, counts, producer scope, conflict dispositions, and limits. Earlier combined-stack counts and failures keep their original revision scope; local acceptance does not erase old CI failures. - No new migration, queue table, dependency, workflow, or shared scheduler contract is introduced. Parent migration 0397, bounded consent recovery, voice cleanup/retention repairs, and runner checkpoint restoration remain present. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing review is waived. Older reviews and parent checks are not current passes. Actual failing Ponytail findings must be fixed; an absent/stopped/paused turn without a verdict is an authorized nonblocker. Active unfinished Ponytail uses its original request/start timer, with a 25-minute monitor cutoff respecting the user's 30-minute maximum. - No complete local repository matrix, live provider tenant, external alarm, physical host/database suspension, or fleet savings claim is made. Native progress remains a separate following slice. ## Risks - A future eligibility writer must notify the existing projection boundary. Independent SQL writers or another active server process are outside this process-local signal contract. - Authority and lineage remain sensitive. The worker uses the existing selectors and publication policy; it does not grant execution or disclosure authority. - Native progress and other publication maintenance still have recurring work. This PR removes one reason, not the entire chat maintenance timer. - Worker quiet does not prove database suspension, host sleep, or external alarm coverage. Cloud sleep blockers remain unchanged. - Merge eligibility requires passing current-head CI and Greptile 5/5 after this actual-master reconciliation. Subsequent stack slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window size are not exposed. Used repository inspection, reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the documented focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
340 lines
15 KiB
TypeScript
340 lines
15 KiB
TypeScript
import { randomUUID } from "node:crypto";
|
|
import { eq } from "drizzle-orm";
|
|
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
|
import type { Db } from "@paperclipai/db";
|
|
import { activityLog, agentWakeupRequests, agents, companies, createDb, heartbeatRuns, issueComments, issues } from "@paperclipai/db";
|
|
import {
|
|
getEmbeddedPostgresTestSupport,
|
|
startEmbeddedPostgresTestDatabase,
|
|
} from "../../../__tests__/helpers/embedded-postgres.js";
|
|
import { resolveExternalTestDatabaseUrl } from "../../../__tests__/helpers/external-test-database.js";
|
|
import { createQueuedCommentIssueLockWriter } from "./queued-comment-postgres.js";
|
|
import type { QueuedCommentQueuePostgresAdapterDeps } from "./queued-comment-postgres.js";
|
|
import { QueuedCommentMutationError } from "../application/queued-comment-use-cases.js";
|
|
|
|
// Proves the same two properties the release-half adapter test proves for
|
|
// this module's other transaction: the one company the caller names in
|
|
// `issue.companyId` binds every read and write for the whole transaction, so
|
|
// a caller-supplied `issue`/`wake` for the wrong company sees nothing, and a
|
|
// guarded write that affects no row rolls the transaction back instead of
|
|
// leaving a partial write. The decision branching itself is proven against
|
|
// plain facts in `domain/policy.test.ts`; the use-case orchestration is
|
|
// proven against a mocked port in `application/queued-comment-use-cases.test.ts`.
|
|
const external = resolveExternalTestDatabaseUrl("PAPERCLIP_QUEUED_COMMENT_TEST_DATABASE_URL");
|
|
const embeddedPostgresSupport = external ? { supported: true, reason: undefined } : await getEmbeddedPostgresTestSupport();
|
|
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
|
|
|
if (!embeddedPostgresSupport.supported) {
|
|
console.warn(
|
|
`Skipping embedded Postgres queued-comment adapter tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
|
|
);
|
|
}
|
|
|
|
describeEmbeddedPostgres("queued-comment postgres adapter", () => {
|
|
let db!: ReturnType<typeof createDb>;
|
|
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
|
|
|
const noopDeps: QueuedCommentQueuePostgresAdapterDeps = {
|
|
syncCommentReferences: async () => {},
|
|
deleteCommentReferenceSource: async () => {},
|
|
syncCommentExternalObjectsSafely: async () => {},
|
|
};
|
|
|
|
beforeAll(async () => {
|
|
if (!external) tempDb = await startEmbeddedPostgresTestDatabase("paperclip-queued-comment-postgres-adapter-");
|
|
db = createDb(external ?? tempDb!.connectionString);
|
|
}, 20_000);
|
|
|
|
afterEach(async () => {
|
|
// Deleted first: activity_log rows reference companies, agents, and
|
|
// heartbeat_runs, and none of those foreign keys cascade.
|
|
await db.delete(activityLog);
|
|
await db.delete(issueComments);
|
|
await db.delete(heartbeatRuns);
|
|
await db.delete(agentWakeupRequests);
|
|
await db.delete(issues);
|
|
await db.delete(agents);
|
|
await db.delete(companies);
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await db?.$client.end();
|
|
await tempDb?.cleanup();
|
|
});
|
|
|
|
async function seedCompany(): Promise<string> {
|
|
const companyId = randomUUID();
|
|
await db.insert(companies).values({
|
|
id: companyId,
|
|
name: "Paperclip",
|
|
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
|
requireBoardApprovalForNewAgents: false,
|
|
defaultResponsibleUserId: "responsible-user",
|
|
});
|
|
return companyId;
|
|
}
|
|
|
|
async function seedAgent(input: { companyId: string; adapterType?: string }): Promise<string> {
|
|
const agentId = randomUUID();
|
|
await db.insert(agents).values({
|
|
id: agentId,
|
|
companyId: input.companyId,
|
|
name: "CodexCoder",
|
|
role: "engineer",
|
|
status: "active",
|
|
adapterType: input.adapterType ?? "codex_local",
|
|
adapterConfig: {},
|
|
runtimeConfig: { heartbeat: { wakeOnDemand: true, maxConcurrentRuns: 1 } },
|
|
permissions: {},
|
|
});
|
|
return agentId;
|
|
}
|
|
|
|
async function seedIssue(input: { companyId: string; assigneeAgentId: string | null }): Promise<string> {
|
|
const issueId = randomUUID();
|
|
await db.insert(issues).values({
|
|
id: issueId,
|
|
companyId: input.companyId,
|
|
title: "Queued-comment adapter fixture issue",
|
|
status: "in_progress",
|
|
priority: "medium",
|
|
assigneeAgentId: input.assigneeAgentId,
|
|
});
|
|
return issueId;
|
|
}
|
|
|
|
async function seedComment(input: { companyId: string; issueId: string; authorUserId: string }): Promise<string> {
|
|
const commentId = randomUUID();
|
|
await db.insert(issueComments).values({
|
|
id: commentId,
|
|
companyId: input.companyId,
|
|
issueId: input.issueId,
|
|
authorUserId: input.authorUserId,
|
|
body: "queued message",
|
|
});
|
|
return commentId;
|
|
}
|
|
|
|
async function seedDeferredWake(input: {
|
|
companyId: string;
|
|
agentId: string;
|
|
issueId: string;
|
|
commentIds: string[];
|
|
}): Promise<string> {
|
|
const id = randomUUID();
|
|
await db.insert(agentWakeupRequests).values({
|
|
id,
|
|
companyId: input.companyId,
|
|
agentId: input.agentId,
|
|
source: "automation",
|
|
reason: "issue_commented",
|
|
status: "deferred_issue_execution",
|
|
requestedByActorType: "user",
|
|
payload: {
|
|
issueId: input.issueId,
|
|
_paperclipWakeContext: { wakeCommentIds: input.commentIds },
|
|
},
|
|
});
|
|
return id;
|
|
}
|
|
|
|
it("scopes the wake lookup to its own company: a foreign-company issue context resolves not_pending and deletes nothing", async () => {
|
|
const companyId = await seedCompany();
|
|
const otherCompanyId = await seedCompany();
|
|
const agentId = await seedAgent({ companyId });
|
|
const issueId = await seedIssue({ companyId, assigneeAgentId: agentId });
|
|
const commentId = await seedComment({ companyId, issueId, authorUserId: "user-1" });
|
|
const wakeId = await seedDeferredWake({ companyId, agentId, issueId, commentIds: [commentId] });
|
|
|
|
const issueLock = createQueuedCommentIssueLockWriter(db, noopDeps);
|
|
await expect(
|
|
issueLock.withLockedQueue(
|
|
{
|
|
// The caller mistakenly names the *other* company on the issue
|
|
// context; that single value binds every read and write for the
|
|
// whole transaction, so it alone must decide what is visible.
|
|
issue: { id: issueId, companyId: otherCompanyId, assigneeAgentId: agentId, executionRunId: null },
|
|
actor: { actorType: "user", actorId: "user-1", agentId: null, runId: null, agentApiKeyId: null },
|
|
queueId: wakeId,
|
|
},
|
|
async () => {
|
|
throw new Error("fn must not run when the wake is invisible to the caller's company");
|
|
},
|
|
),
|
|
).rejects.toMatchObject({ code: "queued_comment_not_pending" });
|
|
|
|
const commentRow = (await db.select().from(issueComments).where(eq(issueComments.id, commentId)))[0];
|
|
expect(commentRow?.deletedAt ?? null).toBeNull();
|
|
const wakeRow = (await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, wakeId)))[0];
|
|
expect(wakeRow?.status).toBe("deferred_issue_execution");
|
|
});
|
|
|
|
it("rolls back a discard when the comment id given belongs to a different issue than the one this transaction locked", async () => {
|
|
const companyId = await seedCompany();
|
|
const agentId = await seedAgent({ companyId });
|
|
const issueId = await seedIssue({ companyId, assigneeAgentId: agentId });
|
|
const otherIssueId = await seedIssue({ companyId, assigneeAgentId: agentId });
|
|
const commentId = await seedComment({ companyId, issueId: otherIssueId, authorUserId: "user-1" });
|
|
const wakeId = await seedDeferredWake({ companyId, agentId, issueId, commentIds: [commentId] });
|
|
|
|
const issueLock = createQueuedCommentIssueLockWriter(db, noopDeps);
|
|
await expect(
|
|
issueLock.withLockedQueue(
|
|
{
|
|
issue: { id: issueId, companyId, assigneeAgentId: agentId, executionRunId: null },
|
|
actor: { actorType: "user", actorId: "user-1", agentId: null, runId: null, agentApiKeyId: null },
|
|
queueId: wakeId,
|
|
},
|
|
async (_locked, transaction) => {
|
|
// The comment belongs to a different issue than the one this
|
|
// transaction locked. The write's own `issueId` predicate, not
|
|
// just the bound company, must decide what is visible.
|
|
const deleted = await transaction.deleteComment({ issueId, commentId });
|
|
if (!deleted) {
|
|
throw new QueuedCommentMutationError("queued_comment_not_pending", "The queued message is no longer pending");
|
|
}
|
|
return deleted;
|
|
},
|
|
),
|
|
).rejects.toMatchObject({ code: "queued_comment_not_pending" });
|
|
|
|
const commentRow = (await db.select().from(issueComments).where(eq(issueComments.id, commentId)))[0];
|
|
expect(commentRow).toBeDefined();
|
|
expect(commentRow?.deletedAt ?? null).toBeNull();
|
|
});
|
|
|
|
it("edits the comment body, syncs references, and rebuilds the queue snapshot inside one company-scoped transaction", async () => {
|
|
const companyId = await seedCompany();
|
|
const agentId = await seedAgent({ companyId });
|
|
const issueId = await seedIssue({ companyId, assigneeAgentId: agentId });
|
|
const commentId = await seedComment({ companyId, issueId, authorUserId: "user-1" });
|
|
const wakeId = await seedDeferredWake({ companyId, agentId, issueId, commentIds: [commentId] });
|
|
|
|
let syncedCommentId: string | null = null;
|
|
const deps: QueuedCommentQueuePostgresAdapterDeps = {
|
|
...noopDeps,
|
|
syncCommentReferences: async (id) => {
|
|
syncedCommentId = id;
|
|
},
|
|
};
|
|
const issueLock = createQueuedCommentIssueLockWriter(db, deps);
|
|
|
|
const queue = await issueLock.withLockedQueue(
|
|
{
|
|
issue: { id: issueId, companyId, assigneeAgentId: agentId, executionRunId: null },
|
|
actor: { actorType: "user", actorId: "user-1", agentId: null, runId: null, agentApiKeyId: null },
|
|
queueId: wakeId,
|
|
},
|
|
async (locked, transaction) => {
|
|
expect(locked.state).toBe("deferred");
|
|
const updated = await transaction.updateCommentBody({
|
|
issueId,
|
|
commentId,
|
|
body: "edited body",
|
|
updatedAt: new Date(),
|
|
});
|
|
expect(updated).toBe(true);
|
|
await transaction.syncCommentReferences(commentId);
|
|
return transaction.buildQueueSnapshot({
|
|
issue: { id: issueId, companyId, assigneeAgentId: agentId, executionRunId: null },
|
|
actor: { actorType: "user", actorId: "user-1", agentId: null, runId: null, agentApiKeyId: null },
|
|
wake: locked.wake,
|
|
state: locked.state,
|
|
queueRun: locked.queueRun,
|
|
activeRun: locked.activeRun,
|
|
});
|
|
},
|
|
);
|
|
|
|
expect(syncedCommentId).toBe(commentId);
|
|
expect(queue.entries).toHaveLength(1);
|
|
expect((queue.entries[0]!.comment as { body: string }).body).toBe("edited body");
|
|
const commentRow = (await db.select().from(issueComments).where(eq(issueComments.id, commentId)))[0];
|
|
expect(commentRow?.body).toBe("edited body");
|
|
});
|
|
|
|
it("rolls back an already-applied comment edit when its own activity insert fails", async () => {
|
|
const companyId = await seedCompany();
|
|
const agentId = await seedAgent({ companyId });
|
|
const issueId = await seedIssue({ companyId, assigneeAgentId: agentId });
|
|
const commentId = await seedComment({ companyId, issueId, authorUserId: "user-1" });
|
|
const wakeId = await seedDeferredWake({ companyId, agentId, issueId, commentIds: [commentId] });
|
|
|
|
const issueLock = createQueuedCommentIssueLockWriter(db, noopDeps);
|
|
const missingAgentId = randomUUID();
|
|
|
|
await expect(
|
|
issueLock.withLockedQueue(
|
|
{
|
|
issue: { id: issueId, companyId, assigneeAgentId: agentId, executionRunId: null },
|
|
actor: { actorType: "user", actorId: "user-1", agentId: null, runId: null, agentApiKeyId: null },
|
|
queueId: wakeId,
|
|
},
|
|
async (_locked, transaction) => {
|
|
await transaction.updateCommentBody({
|
|
issueId,
|
|
commentId,
|
|
body: "edited body",
|
|
updatedAt: new Date(),
|
|
});
|
|
// `agentId` carries a foreign key to `agents.id`; naming an agent
|
|
// that was never seeded forces the activity insert to fail, which
|
|
// must roll back the comment edit issued moments earlier on the
|
|
// same transaction.
|
|
await transaction.logActivity({
|
|
actorType: "agent",
|
|
actorId: missingAgentId,
|
|
agentId: missingAgentId,
|
|
runId: null,
|
|
agentApiKeyId: null,
|
|
action: "issue.queued_comment_edited",
|
|
entityId: issueId,
|
|
details: {},
|
|
});
|
|
},
|
|
),
|
|
).rejects.toMatchObject({ cause: { code: "23503" } });
|
|
|
|
const commentRow = (await db.select().from(issueComments).where(eq(issueComments.id, commentId)))[0];
|
|
expect(commentRow?.body).toBe("queued message");
|
|
const activityRows = await db.select().from(activityLog).where(eq(activityLog.entityId, issueId));
|
|
expect(activityRows).toHaveLength(0);
|
|
});
|
|
|
|
// Pins a fact the database itself cannot persist today: `runtime_mode` is
|
|
// a NOT NULL column, so a real active run's own field is never null. The
|
|
// port type allows it (`runtimeMode: string | null`), so this test builds
|
|
// the fact directly instead of through a database row, to keep this
|
|
// branch of the shared steering rule under a regression test.
|
|
it("answers the steering question for a deferred paperclip_runner queue whose active run has no persisted runtime mode yet", async () => {
|
|
const companyId = await seedCompany();
|
|
const agentId = await seedAgent({ companyId, adapterType: "paperclip_runner" });
|
|
const issueId = await seedIssue({ companyId, assigneeAgentId: agentId });
|
|
const commentId = await seedComment({ companyId, issueId, authorUserId: "user-1" });
|
|
const wakeId = await seedDeferredWake({ companyId, agentId, issueId, commentIds: [commentId] });
|
|
|
|
const issueLock = createQueuedCommentIssueLockWriter(db, noopDeps);
|
|
const queue = await issueLock.withLockedQueue(
|
|
{
|
|
issue: { id: issueId, companyId, assigneeAgentId: agentId, executionRunId: null },
|
|
actor: { actorType: "user", actorId: "user-1", agentId: null, runId: null, agentApiKeyId: null },
|
|
queueId: wakeId,
|
|
},
|
|
async (locked, transaction) => {
|
|
expect(locked.state).toBe("deferred");
|
|
return transaction.buildQueueSnapshot({
|
|
issue: { id: issueId, companyId, assigneeAgentId: agentId, executionRunId: null },
|
|
actor: { actorType: "user", actorId: "user-1", agentId: null, runId: null, agentApiKeyId: null },
|
|
wake: locked.wake,
|
|
state: "deferred",
|
|
queueRun: null,
|
|
activeRun: { id: randomUUID(), status: "running", runtimeMode: null, contextSnapshot: {} },
|
|
});
|
|
},
|
|
);
|
|
|
|
expect(queue.protocol).toBe("paperclip_runner_v1");
|
|
expect(queue.steeringDisposition).toBe("temporarily_unavailable");
|
|
});
|
|
});
|