mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
fix(server): wake approval queue seeding from committed changes (#17510)
## Thinking Path > - Paperclip manages work and human decisions for AI-agent companies. > - Automatic decision queues store membership for pending attention sources. > - The old attention maintenance scan created that membership as a side effect. > - Moving approval retention out of the scan also removed approval queue discovery. > - Queue API consumers do not build the attention feed first, so they can miss pending PR approvals. > - This pull request gives approval queue discovery its own worker and commit signals. > - The worker becomes quiet after it saves every missing membership. ## Linked Issues or Issue Description Refs #17497. Stacked on #17508, following #17505. This is a correctness dependency for the approval retention migration in #17497. Do not treat that migration as complete without this queue-population path. A real PostgreSQL regression found no PR queue when a pending approval had a linked PR work product and the maintenance feed excluded approvals. Related: #13364 proposes pruning stale seeded memberships. This PR preserves existing membership retention and does not implement pruning. #15149 documents the decision-queue APIs. ## What Changed - Add an approval queue worker that selects missing PR-queue memberships from existing durable records. Bound each pass to 100 rows. Continue past failed rows and revisit them on retry. - Reuse the queue service's seed writer through a normalized source contract. Keep queue rules, uniqueness, and audit writes in that service. Keep feed readers compatible. - Wake from committed approval and link changes, issue/company visibility, PR work-product changes and imports, and queue-rule changes. Reuse the existing transaction boundaries. - Recheck approval status under a row lock. Serialize queue-rule changes with the seed writer. Preserve required recovery during drain and shutdown. - Register startup reconstruction under the existing maintenance switch. Leave other attention-source seeding with its current owner until its own migration. - Document the complete producer audit and stack dependency. Add a validated external PostgreSQL option to the existing queue route suite so it runs on hosts without embedded database support. ## Verification Current tested revision: `7948136ef8`. This PR targets master after #17508 merged as `8406047b4e`. Normal ancestry reconciliation preserved the complete tested integration tree, before the review repairs below. Only the original 25 seeding paths differ from master, and database schema/migration metadata is identical. - **56 cases pass in the three complete affected suites:** 28 approval seeding, 6 queue routes and 22 task-watchdog cases. Server typecheck and production build pass. - Three real PostgreSQL retry regressions failed then passed: new matching arrivals cannot postpone a recovered failure indefinitely; successful partial progress resets old failure penalties; an empty cursor rollover does not reset outage backoff. Exact one- and two-second retry assertions, eventual complete recovery and existing capped-backoff/company isolation tests pass. - Three genuinely empty external databases failed before setup repair because tables were missing. Each suite now applies migrations itself. All 55 cases passed on a second set of fresh databases; adding the final rollover guard then passed all 56 cases using those now-migrated fixtures. No production migration or table is added. - The initial expanded fresh run passed 54 cases but failed a cleanup expectation that omitted a required cursor rollover pass. The fixture now permits that explicit continuation while preserving exact deadline and complete-membership assertions. All failure records remain in `doc/idle-approval-queue-seeding.md`. - The prior integrated revision `4e725e2055` passed **522 cases in fifteen complete producer/consumer suites**, plus types/build. Its unchanged consumer code and historical deadlock, cross-company amplification, backoff, gateway integration and setup evidence remain documented; those counts are not relabeled as current acceptance. Commands: server Vitest for approval queue seeding, queue routes and task watchdogs; server typecheck/build. Fresh exact-head Greptile/Ponytail reviews and all CI checks remain required. No full local repository-suite, deployed suspension or cost claim is made. ## Risks - Signals cover the supported in-process service writers. Independent SQL writers and multiple server replicas need a separate notification contract. - The source predicate must stay aligned with unscoped attention maintenance. The first linked issue, company scope, pending status, and execution visibility are tested. A pending approval itself keeps its linked external conversation active. - Existing enabled seed rules can re-add manually removed matching memberships. This preserves the old scan's behavior. Completed or archived membership is not pruned. - Queue seeding and retention are separate owners. Other attention sources still need their own event-driven discovery before the aggregate scan can be retired. - This PR does not extend Cloud wake coverage, relax sleep admission, or claim measured host/database suspension. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The exact serving model identifier and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
8406047b4e
commit
4e27a82a42
25 files changed
+1218
-165
No files matched your search
@@ -0,0 +1,106 @@
|
||||
import { and, asc, desc, eq, gt, lte, sql, type SQL } from "drizzle-orm";
|
||||
import { approvals, companies, issues, issueWorkProducts, type Db } from "@paperclipai/db";
|
||||
import { firstApprovalIssueId } from "./approval-visibility.js";
|
||||
import { executionIssueCondition } from "./issue-visibility.js";
|
||||
import { pullRequestSeedProductCondition } from "./decision-queue-seed-rules.js";
|
||||
import { decisionQueueService } from "./decision-queues.js";
|
||||
import type { createDeliveryWorkCoordinator } from "./delivery-work-coordinator.js";
|
||||
import { DELIVERY_QUEUES } from "./delivery-work-notifications.js";
|
||||
|
||||
const linkedIssue = firstApprovalIssueId();
|
||||
|
||||
export function approvalQueueSeedingService(db: Db) {
|
||||
const pending = new Map<string | undefined, { cursor?: string; end?: string; generation: number; failures: number; retryAt: number }>([[undefined, { generation: 0, failures: 0, retryAt: 0 }]]);
|
||||
function request(companyIds: ReadonlySet<string> | undefined) {
|
||||
for (const companyId of companyIds ?? []) {
|
||||
const existing = pending.get(companyId);
|
||||
if (existing) existing.generation++;
|
||||
else pending.set(companyId, { generation: 0, failures: 0, retryAt: 0 });
|
||||
}
|
||||
}
|
||||
function candidates(database: Db, companyId: string | undefined, condition?: SQL, order = asc) {
|
||||
return database.select({ id: approvals.id, companyId: approvals.companyId, issueId: linkedIssue }).from(approvals)
|
||||
.innerJoin(companies, eq(companies.id, approvals.companyId))
|
||||
.where(and(eq(approvals.status, "pending"), eq(companies.status, "active"), companyId === undefined ? undefined : eq(approvals.companyId, companyId), condition,
|
||||
sql`exists (select 1 from ${issues} where ${issues.id} = ${linkedIssue}
|
||||
and ${issues.companyId} = ${approvals.companyId} and ${executionIssueCondition()}
|
||||
and exists (select 1 from ${issueWorkProducts} where ${pullRequestSeedProductCondition(approvals.companyId)}
|
||||
and ${issueWorkProducts.issueId} = ${issues.id}))`,
|
||||
sql`not exists (select 1 from decision_queues queue where queue.company_id = ${approvals.companyId} and queue.key = 'prs'
|
||||
and (not queue.seed_rules_enabled or exists (select 1 from decision_queue_items item where item.queue_id = queue.id
|
||||
and item.company_id = ${approvals.companyId} and item.source_kind = 'approval' and item.source_id = ${approvals.id}::text)))`,
|
||||
)).orderBy(order(approvals.id));
|
||||
}
|
||||
async function sweep(signal?: AbortSignal) {
|
||||
const scope = [...pending.entries()].find(([, state]) => state.retryAt <= Date.now());
|
||||
if (!scope) return;
|
||||
const [companyId, state] = scope;
|
||||
try {
|
||||
const generation = state.generation;
|
||||
state.end ??= (await candidates(db, companyId, undefined, desc).limit(1))[0]?.id;
|
||||
const rows = state.end ? await candidates(db, companyId, and(lte(approvals.id, state.end),
|
||||
state.cursor ? gt(approvals.id, state.cursor) : undefined)).limit(100) : [];
|
||||
const errors: unknown[] = [];
|
||||
for (const row of rows) {
|
||||
if (signal?.aborted) break;
|
||||
try {
|
||||
await db.transaction(async tx => {
|
||||
const database = tx as unknown as Db;
|
||||
const [current] = await candidates(database, row.companyId, eq(approvals.id, row.id)).for("update", { of: approvals });
|
||||
if (!current) return;
|
||||
await decisionQueueService(database).materializeSeedSources(current.companyId, [{
|
||||
sourceKind: "approval", sourceId: current.id, issueId: current.issueId,
|
||||
}]);
|
||||
});
|
||||
} catch (error) { errors.push(error); }
|
||||
state.cursor = row.id;
|
||||
}
|
||||
if (rows.length < 100) { state.cursor = undefined; state.end = undefined; }
|
||||
const remaining = await candidates(db, companyId).limit(1);
|
||||
if (!signal?.aborted && generation === state.generation && remaining.length === 0) pending.delete(companyId);
|
||||
if (errors.length === 1) throw errors[0];
|
||||
if (errors.length) throw new AggregateError(errors, "Approval queue seeding failed");
|
||||
if (rows.length > 0 || remaining.length === 0) state.failures = 0;
|
||||
state.retryAt = 0;
|
||||
} catch (error) {
|
||||
state.failures = Math.min(state.failures + 1, 7);
|
||||
state.retryAt = Date.now() + Math.min(60_000, 1_000 * 2 ** (state.failures - 1));
|
||||
throw error;
|
||||
} finally {
|
||||
if (pending.has(companyId)) {
|
||||
pending.delete(companyId);
|
||||
pending.set(companyId, state);
|
||||
}
|
||||
}
|
||||
}
|
||||
function next() {
|
||||
let earliest: number | null = null;
|
||||
for (const state of pending.values()) earliest = earliest === null ? state.retryAt : Math.min(earliest, state.retryAt);
|
||||
return earliest;
|
||||
}
|
||||
return { sweep, next, request };
|
||||
}
|
||||
|
||||
export function registerApprovalQueueSeedingWork(
|
||||
coordinator: ReturnType<typeof createDeliveryWorkCoordinator>,
|
||||
service: ReturnType<typeof approvalQueueSeedingService>,
|
||||
canRun: () => boolean,
|
||||
) {
|
||||
let earliest = 0;
|
||||
return coordinator.register(DELIVERY_QUEUES.approvalQueueSeeding, {
|
||||
retryMs: 1_000,
|
||||
pausePassiveMaintenance: true,
|
||||
onSettled: service.request,
|
||||
run: async signal => {
|
||||
if (!canRun()) return;
|
||||
if (Date.now() < earliest) return;
|
||||
earliest = Date.now() + 1_000;
|
||||
await service.sweep(signal);
|
||||
},
|
||||
hasPending: async () => (service.next() ?? Infinity) <= Date.now(),
|
||||
nextRunAt: () => {
|
||||
const next = service.next();
|
||||
return next === null ? null : Math.max(earliest, next);
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
|
||||
export const notifyApprovalQueueWork = (transaction: object, companyId: string) => notifyDeliveryWork(transaction, DELIVERY_QUEUES.approvalQueueSeeding, companyId);
|
||||
|
||||
export async function notifyApprovalWork(transaction: object, companyId: string) {
|
||||
await notifyDeliveryWork(transaction, DELIVERY_QUEUES.approvalRetention);
|
||||
await notifyApprovalQueueWork(transaction, companyId);
|
||||
}
|
||||
|
||||
export function writeApprovalChange<T>(db: Db, companyId: string, write: (transaction: Db) => Promise<T>): Promise<T> {
|
||||
return db.transaction(async tx => {
|
||||
await notifyApprovalWork(tx, companyId);
|
||||
return write(tx as unknown as Db);
|
||||
});
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { notifyToolReviewChange } from "./tool-review-work-notifications.js";
|
||||
import { writeApprovalChange } from "./retention-work-notifications.js";
|
||||
import { writeApprovalChange } from "./approval-work-notifications.js";
|
||||
import { approvalResolutionRecords, resolvableApprovalStatuses as resolvableStatuses } from "../lib/approval-records.js";
|
||||
import { notifyHireApproved } from "./hire-hook.js";
|
||||
|
||||
@@ -55,7 +55,7 @@ export function approvalRecords(db: Db) {
|
||||
},
|
||||
|
||||
create: (companyId: string, data: Omit<typeof approvals.$inferInsert, "companyId">) =>
|
||||
writeApprovalChange(db, tx => tx
|
||||
writeApprovalChange(db, companyId, tx => tx
|
||||
.insert(approvals)
|
||||
.values({ ...data, companyId })
|
||||
.returning()
|
||||
@@ -68,7 +68,7 @@ export function approvalRecords(db: Db) {
|
||||
const now = new Date();
|
||||
const [existing] = await db.select({ companyId: approvals.companyId }).from(approvals).where(eq(approvals.id, id));
|
||||
if (!existing) return null;
|
||||
const updated = await writeApprovalChange(db, async tx => {
|
||||
const updated = await writeApprovalChange(db, existing.companyId, async tx => {
|
||||
await notifyToolReviewChange(tx, existing.companyId);
|
||||
return tx.update(approvals)
|
||||
.set({
|
||||
@@ -91,7 +91,7 @@ export function approvalRecords(db: Db) {
|
||||
}
|
||||
|
||||
const now = new Date();
|
||||
return writeApprovalChange(db, tx => tx
|
||||
return writeApprovalChange(db, existing.companyId, tx => tx
|
||||
.update(approvals)
|
||||
.set({
|
||||
status: "revision_requested",
|
||||
@@ -112,7 +112,7 @@ export function approvalRecords(db: Db) {
|
||||
}
|
||||
|
||||
const now = new Date();
|
||||
return writeApprovalChange(db, tx => tx
|
||||
return writeApprovalChange(db, existing.companyId, tx => tx
|
||||
.update(approvals)
|
||||
.set({
|
||||
status: "pending",
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { notifyCompanyRetentionWork, notifyRetentionWork } from "./retention-work-notifications.js";
|
||||
import { notifyApprovalWork } from "./approval-work-notifications.js";
|
||||
import { notifyCompanyRetentionWork } from "./retention-work-notifications.js";
|
||||
import { addCents, compareCents, normalizeCents } from "@paperclipai/shared";
|
||||
import { and, desc, eq, inArray, sql, or, isNull } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
@@ -259,7 +260,9 @@ async function markApprovalStatus(
|
||||
decidedByUserId: string,
|
||||
) {
|
||||
if (!approvalId) return;
|
||||
await notifyRetentionWork(db, "approval");
|
||||
const [approval] = await db.select({ companyId: approvals.companyId }).from(approvals).where(eq(approvals.id, approvalId));
|
||||
if (!approval) return;
|
||||
await notifyApprovalWork(db, approval.companyId);
|
||||
await db
|
||||
.update(approvals)
|
||||
.set({
|
||||
@@ -297,7 +300,7 @@ export function budgetServiceInTransaction(db: Db, publications: ActivityPublica
|
||||
|
||||
const scope = await resolveScopeRecord(db, "company", policy.scopeId);
|
||||
if (scope.paused) return;
|
||||
await notifyCompanyRetentionWork(db);
|
||||
await notifyCompanyRetentionWork(db, policy.companyId);
|
||||
await notifyDeliveryWork(db, DELIVERY_QUEUES.agentLifecycle, policy.companyId);
|
||||
await notifyDeliveryWork(db, DELIVERY_QUEUES.skillSourceSync);
|
||||
await db
|
||||
@@ -354,7 +357,7 @@ export function budgetServiceInTransaction(db: Db, publications: ActivityPublica
|
||||
await notifyDeliveryWork(db, DELIVERY_QUEUES.skillSourceSync);
|
||||
const scope = await resolveScopeRecord(db, "company", policy.scopeId);
|
||||
if (scope.pauseReason !== "budget") return;
|
||||
await notifyCompanyRetentionWork(db);
|
||||
await notifyCompanyRetentionWork(db, policy.companyId);
|
||||
await notifyDeliveryWork(db, DELIVERY_QUEUES.agentLifecycle, policy.companyId);
|
||||
await db
|
||||
.update(companies)
|
||||
@@ -457,7 +460,7 @@ export function budgetServiceInTransaction(db: Db, publications: ActivityPublica
|
||||
windowEnd: end,
|
||||
});
|
||||
|
||||
if (thresholdType === "hard") await notifyRetentionWork(db, "approval");
|
||||
if (thresholdType === "hard") await notifyApprovalWork(db, policy.companyId);
|
||||
const approval = thresholdType === "hard"
|
||||
? await db
|
||||
.insert(approvals)
|
||||
|
||||
@@ -331,7 +331,7 @@ export function companyService(db: Db, _budgetHooks: BudgetServiceHooks = {}) {
|
||||
|
||||
if (companyPatch.status !== undefined && companyPatch.status !== existing.status) {
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.skillSourceSync);
|
||||
await notifyCompanyRetentionWork(tx);
|
||||
await notifyCompanyRetentionWork(tx, id);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.agentLifecycle, id);
|
||||
}
|
||||
const updated = await tx
|
||||
@@ -483,7 +483,7 @@ export function companyService(db: Db, _budgetHooks: BudgetServiceHooks = {}) {
|
||||
|
||||
if (!wasAlreadyArchived) {
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.skillSourceSync);
|
||||
await notifyCompanyRetentionWork(tx);
|
||||
await notifyCompanyRetentionWork(tx, id);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.agentLifecycle, id);
|
||||
await tx
|
||||
.update(companies)
|
||||
|
||||
@@ -97,7 +97,7 @@ export async function deleteCompany(db: Db, id: string) {
|
||||
await tx.delete(agentRuntimeState).where(eq(agentRuntimeState.companyId, id));
|
||||
await tx.delete(issueComments).where(eq(issueComments.companyId, id));
|
||||
await tx.delete(approvalComments).where(eq(approvalComments.companyId, id));
|
||||
await notifyCompanyRetentionWork(tx);
|
||||
await notifyCompanyRetentionWork(tx, id);
|
||||
await tx.delete(approvals).where(eq(approvals.companyId, id));
|
||||
await tx.delete(companySecrets).where(eq(companySecrets.companyId, id));
|
||||
await tx.delete(joinRequests).where(eq(joinRequests.companyId, id));
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
import { and, eq, type SQLWrapper } from "drizzle-orm";
|
||||
import { issueWorkProducts } from "@paperclipai/db";
|
||||
|
||||
export function pullRequestSeedProductCondition(companyId: string | SQLWrapper) {
|
||||
return and(eq(issueWorkProducts.companyId, companyId), eq(issueWorkProducts.type, "pull_request"));
|
||||
}
|
||||
@@ -1,3 +1,5 @@
|
||||
import { pullRequestSeedProductCondition } from "./decision-queue-seed-rules.js";
|
||||
import { notifyApprovalQueueWork } from "./approval-work-notifications.js";
|
||||
import { and, desc, eq, inArray, isNull, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import {
|
||||
@@ -43,6 +45,14 @@ export type DecisionMutationActor = {
|
||||
responsibleUserId: string | null;
|
||||
};
|
||||
|
||||
export type DecisionQueueSeedSource = {
|
||||
sourceKind: AttentionSourceKind;
|
||||
sourceId: string;
|
||||
issueId: string | null;
|
||||
isPlanTarget?: boolean;
|
||||
interactionKind?: string;
|
||||
};
|
||||
|
||||
type SeedDefinition = {
|
||||
key: string;
|
||||
title: string;
|
||||
@@ -367,6 +377,12 @@ export function decisionQueueService(db: Db) {
|
||||
.then((rows) => rows[0] ?? null);
|
||||
}
|
||||
|
||||
async function lockQueue(database: Db, companyId: string, id: string) {
|
||||
const [queue] = await database.select({ id: decisionQueues.id }).from(decisionQueues)
|
||||
.where(and(eq(decisionQueues.companyId, companyId), eq(decisionQueues.id, id))).for("update");
|
||||
if (!queue) throw notFound("Decision queue not found");
|
||||
}
|
||||
|
||||
async function visibleItems(companyId: string, queueId: string, authActor: AuthorizationActor) {
|
||||
const rows = await db.select().from(decisionQueueItems)
|
||||
.where(and(eq(decisionQueueItems.companyId, companyId), eq(decisionQueueItems.queueId, queueId)))
|
||||
@@ -380,6 +396,103 @@ export function decisionQueueService(db: Db) {
|
||||
return visible;
|
||||
}
|
||||
|
||||
async function materializeSeedSources(companyId: string, items: DecisionQueueSeedSource[]) {
|
||||
if (items.length === 0) return;
|
||||
const issueIds = [...new Set(items.map(item => item.issueId).filter((id): id is string => Boolean(id)))];
|
||||
const prIssueIds = new Set(issueIds.length === 0 ? [] : await db
|
||||
.select({ issueId: issueWorkProducts.issueId })
|
||||
.from(issueWorkProducts)
|
||||
.where(and(
|
||||
pullRequestSeedProductCondition(companyId),
|
||||
inArray(issueWorkProducts.issueId, issueIds),
|
||||
)).then((rows) => rows.map((row) => row.issueId)));
|
||||
|
||||
const matches = new Map<string, DecisionQueueSeedSource[]>();
|
||||
for (const item of items) {
|
||||
if (prIssueIds.has(item.issueId ?? "")) {
|
||||
matches.set("prs", [...(matches.get("prs") ?? []), item]);
|
||||
}
|
||||
if (item.sourceKind === "issue_thread_interaction" && item.isPlanTarget === true) {
|
||||
matches.set("plans", [...(matches.get("plans") ?? []), item]);
|
||||
}
|
||||
if (item.sourceKind === "issue_thread_interaction" && item.interactionKind === "ask_user_questions") {
|
||||
matches.set("questions", [...(matches.get("questions") ?? []), item]);
|
||||
}
|
||||
}
|
||||
|
||||
for (const seed of DECISION_QUEUE_SEEDS) {
|
||||
const matchingItems = matches.get(seed.key) ?? [];
|
||||
if (matchingItems.length === 0) continue;
|
||||
await db.transaction(async (tx) => {
|
||||
const txDb = tx as unknown as Db;
|
||||
const insertedQueue = await txDb.insert(decisionQueues).values({
|
||||
companyId,
|
||||
key: seed.key,
|
||||
title: seed.title,
|
||||
description: seed.description,
|
||||
createdByType: "system",
|
||||
seedRules: seed.rules,
|
||||
seedRulesEnabled: true,
|
||||
}).onConflictDoNothing({ target: [decisionQueues.companyId, decisionQueues.key] }).returning();
|
||||
const queue = insertedQueue[0] ?? await txDb.select().from(decisionQueues)
|
||||
.where(and(eq(decisionQueues.companyId, companyId), eq(decisionQueues.key, seed.key)))
|
||||
.for("update")
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!queue || !queue.seedRulesEnabled) return;
|
||||
if (insertedQueue[0]) {
|
||||
await txDb.insert(decisionTriageEvents).values({
|
||||
companyId,
|
||||
queueId: queue.id,
|
||||
action: "queue.seeded",
|
||||
...eventActorColumns(SYSTEM_ACTOR),
|
||||
details: { key: seed.key, rules: seed.rules.map((rule) => rule.key) },
|
||||
});
|
||||
await recordActivity(txDb, SYSTEM_ACTOR, {
|
||||
companyId,
|
||||
action: "decision_queue.seeded",
|
||||
entityType: "decision_queue",
|
||||
entityId: queue.id,
|
||||
details: { key: seed.key },
|
||||
});
|
||||
}
|
||||
let insertedAnyItem = false;
|
||||
for (const item of matchingItems) {
|
||||
const inserted = await txDb.insert(decisionQueueItems).values({
|
||||
companyId,
|
||||
queueId: queue.id,
|
||||
sourceKind: item.sourceKind,
|
||||
sourceId: item.sourceId,
|
||||
addedByType: "system",
|
||||
}).onConflictDoNothing({
|
||||
target: [decisionQueueItems.queueId, decisionQueueItems.sourceKind, decisionQueueItems.sourceId],
|
||||
}).returning();
|
||||
if (!inserted[0]) continue;
|
||||
insertedAnyItem = true;
|
||||
await txDb.insert(decisionTriageEvents).values({
|
||||
companyId,
|
||||
queueId: queue.id,
|
||||
sourceKind: item.sourceKind,
|
||||
sourceId: item.sourceId,
|
||||
action: "queue_item.seeded",
|
||||
...eventActorColumns(SYSTEM_ACTOR),
|
||||
details: { seedKey: seed.key },
|
||||
});
|
||||
await recordActivity(txDb, SYSTEM_ACTOR, {
|
||||
companyId,
|
||||
action: "decision_queue_item.seeded",
|
||||
entityType: "decision_queue",
|
||||
entityId: queue.id,
|
||||
details: { sourceKind: item.sourceKind, sourceId: item.sourceId, seedKey: seed.key },
|
||||
});
|
||||
}
|
||||
if (insertedAnyItem) {
|
||||
await txDb.update(decisionQueues).set({ updatedAt: new Date() })
|
||||
.where(and(eq(decisionQueues.companyId, companyId), eq(decisionQueues.id, queue.id)));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
create: async (input: {
|
||||
companyId: string;
|
||||
@@ -392,6 +505,7 @@ export function decisionQueueService(db: Db) {
|
||||
}) => {
|
||||
const result = await db.transaction(async (tx) => {
|
||||
const txDb = tx as unknown as Db;
|
||||
if (input.key === "prs") await notifyApprovalQueueWork(txDb, input.companyId);
|
||||
const inserted = await txDb.insert(decisionQueues).values({
|
||||
companyId: input.companyId,
|
||||
key: input.key,
|
||||
@@ -449,6 +563,7 @@ export function decisionQueueService(db: Db) {
|
||||
}) => {
|
||||
const row = await db.transaction(async (tx) => {
|
||||
const txDb = tx as unknown as Db;
|
||||
if (input.key === "prs") await notifyApprovalQueueWork(txDb, input.companyId);
|
||||
const updated = await txDb.update(decisionQueues).set({ ...input.patch, updatedAt: new Date() })
|
||||
.where(and(eq(decisionQueues.companyId, input.companyId), eq(decisionQueues.key, input.key)))
|
||||
.returning().then((rows) => rows[0] ?? null);
|
||||
@@ -491,6 +606,7 @@ export function decisionQueueService(db: Db) {
|
||||
await requireSourceRead(db, input.authActor, input.companyId, input.sourceKind, input.sourceId);
|
||||
return db.transaction(async (tx) => {
|
||||
const txDb = tx as unknown as Db;
|
||||
await lockQueue(txDb, input.companyId, queue.id);
|
||||
const inserted = await txDb.insert(decisionQueueItems).values({
|
||||
companyId: input.companyId,
|
||||
queueId: queue.id,
|
||||
@@ -556,6 +672,8 @@ export function decisionQueueService(db: Db) {
|
||||
}
|
||||
return db.transaction(async (tx) => {
|
||||
const txDb = tx as unknown as Db;
|
||||
await lockQueue(txDb, input.companyId, queue.id);
|
||||
if (input.key === "prs" && input.sourceKind === "approval") await notifyApprovalQueueWork(txDb, input.companyId);
|
||||
const removed = await txDb.delete(decisionQueueItems).where(and(
|
||||
eq(decisionQueueItems.companyId, input.companyId),
|
||||
eq(decisionQueueItems.queueId, queue.id),
|
||||
@@ -682,101 +800,13 @@ export function decisionQueueService(db: Db) {
|
||||
});
|
||||
},
|
||||
|
||||
materializeSeededQueues: async (companyId: string, items: AttentionItem[]) => {
|
||||
if (items.length === 0) return;
|
||||
const issueIds = [...new Set(items.map(itemIssueId).filter((id): id is string => Boolean(id)))];
|
||||
const prIssueIds = new Set(issueIds.length === 0 ? [] : await db
|
||||
.select({ issueId: issueWorkProducts.issueId })
|
||||
.from(issueWorkProducts)
|
||||
.where(and(
|
||||
eq(issueWorkProducts.companyId, companyId),
|
||||
eq(issueWorkProducts.type, "pull_request"),
|
||||
inArray(issueWorkProducts.issueId, issueIds),
|
||||
)).then((rows) => rows.map((row) => row.issueId)));
|
||||
|
||||
const matches = new Map<string, AttentionItem[]>();
|
||||
for (const item of items) {
|
||||
if (prIssueIds.has(itemIssueId(item) ?? "")) {
|
||||
matches.set("prs", [...(matches.get("prs") ?? []), item]);
|
||||
}
|
||||
if (item.sourceKind === "issue_thread_interaction" && item.subject.metadata?.isPlanTarget === true) {
|
||||
matches.set("plans", [...(matches.get("plans") ?? []), item]);
|
||||
}
|
||||
if (item.sourceKind === "issue_thread_interaction" && item.subject.metadata?.kind === "ask_user_questions") {
|
||||
matches.set("questions", [...(matches.get("questions") ?? []), item]);
|
||||
}
|
||||
}
|
||||
|
||||
for (const seed of DECISION_QUEUE_SEEDS) {
|
||||
const matchingItems = matches.get(seed.key) ?? [];
|
||||
if (matchingItems.length === 0) continue;
|
||||
await db.transaction(async (tx) => {
|
||||
const txDb = tx as unknown as Db;
|
||||
const insertedQueue = await txDb.insert(decisionQueues).values({
|
||||
companyId,
|
||||
key: seed.key,
|
||||
title: seed.title,
|
||||
description: seed.description,
|
||||
createdByType: "system",
|
||||
seedRules: seed.rules,
|
||||
seedRulesEnabled: true,
|
||||
}).onConflictDoNothing({ target: [decisionQueues.companyId, decisionQueues.key] }).returning();
|
||||
const queue = insertedQueue[0] ?? await txDb.select().from(decisionQueues)
|
||||
.where(and(eq(decisionQueues.companyId, companyId), eq(decisionQueues.key, seed.key)))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!queue || !queue.seedRulesEnabled) return;
|
||||
if (insertedQueue[0]) {
|
||||
await txDb.insert(decisionTriageEvents).values({
|
||||
companyId,
|
||||
queueId: queue.id,
|
||||
action: "queue.seeded",
|
||||
...eventActorColumns(SYSTEM_ACTOR),
|
||||
details: { key: seed.key, rules: seed.rules.map((rule) => rule.key) },
|
||||
});
|
||||
await recordActivity(txDb, SYSTEM_ACTOR, {
|
||||
companyId,
|
||||
action: "decision_queue.seeded",
|
||||
entityType: "decision_queue",
|
||||
entityId: queue.id,
|
||||
details: { key: seed.key },
|
||||
});
|
||||
}
|
||||
let insertedAnyItem = false;
|
||||
for (const item of matchingItems) {
|
||||
const inserted = await txDb.insert(decisionQueueItems).values({
|
||||
companyId,
|
||||
queueId: queue.id,
|
||||
sourceKind: item.sourceKind,
|
||||
sourceId: item.subject.id,
|
||||
addedByType: "system",
|
||||
}).onConflictDoNothing({
|
||||
target: [decisionQueueItems.queueId, decisionQueueItems.sourceKind, decisionQueueItems.sourceId],
|
||||
}).returning();
|
||||
if (!inserted[0]) continue;
|
||||
insertedAnyItem = true;
|
||||
await txDb.insert(decisionTriageEvents).values({
|
||||
companyId,
|
||||
queueId: queue.id,
|
||||
sourceKind: item.sourceKind,
|
||||
sourceId: item.subject.id,
|
||||
action: "queue_item.seeded",
|
||||
...eventActorColumns(SYSTEM_ACTOR),
|
||||
details: { seedKey: seed.key },
|
||||
});
|
||||
await recordActivity(txDb, SYSTEM_ACTOR, {
|
||||
companyId,
|
||||
action: "decision_queue_item.seeded",
|
||||
entityType: "decision_queue",
|
||||
entityId: queue.id,
|
||||
details: { sourceKind: item.sourceKind, sourceId: item.subject.id, seedKey: seed.key },
|
||||
});
|
||||
}
|
||||
if (insertedAnyItem) {
|
||||
await txDb.update(decisionQueues).set({ updatedAt: new Date() })
|
||||
.where(and(eq(decisionQueues.companyId, companyId), eq(decisionQueues.id, queue.id)));
|
||||
}
|
||||
});
|
||||
}
|
||||
},
|
||||
materializeSeedSources,
|
||||
materializeSeededQueues: (companyId: string, items: AttentionItem[]) => materializeSeedSources(companyId, items.map(item => ({
|
||||
sourceKind: item.sourceKind,
|
||||
sourceId: item.subject.id,
|
||||
issueId: itemIssueId(item),
|
||||
isPlanTarget: item.subject.metadata?.isPlanTarget === true,
|
||||
interactionKind: typeof item.subject.metadata?.kind === "string" ? item.subject.metadata.kind : undefined,
|
||||
}))),
|
||||
};
|
||||
}
|
||||
@@ -9,6 +9,7 @@ export const DELIVERY_QUEUES = {
|
||||
executionStatus: "execution-status",
|
||||
executionDisposition: "execution-disposition",
|
||||
environmentCleanup: "environment-cleanup",
|
||||
approvalQueueSeeding: "approval-queue-seeding",
|
||||
approvalRetention: "approval-retention",
|
||||
joinRequestRetention: "join-request-retention",
|
||||
decisionTargets: "decision-targets",
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { writeApprovalChange } from "./retention-work-notifications.js";
|
||||
import { writeApprovalChange } from "./approval-work-notifications.js";
|
||||
import { and, desc, eq, inArray } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { approvals, issueApprovals, issues } from "@paperclipai/db";
|
||||
@@ -108,7 +108,7 @@ export function issueApprovalService(db: Db) {
|
||||
link: async (issueId: string, approvalId: string, actor?: LinkActor) => {
|
||||
const { issue } = await assertIssueAndApprovalSameCompany(issueId, approvalId);
|
||||
|
||||
await writeApprovalChange(db, tx => tx
|
||||
await writeApprovalChange(db, issue.companyId, tx => tx
|
||||
.insert(issueApprovals)
|
||||
.values({
|
||||
companyId: issue.companyId,
|
||||
@@ -128,7 +128,7 @@ export function issueApprovalService(db: Db) {
|
||||
|
||||
unlink: async (issueId: string, approvalId: string) => {
|
||||
const { issue } = await assertIssueAndApprovalSameCompany(issueId, approvalId);
|
||||
await writeApprovalChange(db, tx => tx
|
||||
await writeApprovalChange(db, issue.companyId, tx => tx
|
||||
.delete(issueApprovals)
|
||||
.where(and(eq(issueApprovals.issueId, issueId), eq(issueApprovals.approvalId, approvalId))));
|
||||
},
|
||||
@@ -158,7 +158,7 @@ export function issueApprovalService(db: Db) {
|
||||
}
|
||||
}
|
||||
|
||||
await writeApprovalChange(db, tx => tx
|
||||
await writeApprovalChange(db, approval.companyId, tx => tx
|
||||
.insert(issueApprovals)
|
||||
.values(
|
||||
uniqueIssueIds.map((issueId) => ({
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { notifyRetentionWork } from "./retention-work-notifications.js";
|
||||
import { notifyApprovalWork } from "./approval-work-notifications.js";
|
||||
import { notifyDecisionTargetWork, notifyDecisionTargetIssueRemoval } from "./decision-targets.js";
|
||||
import { assertOnboardingFirstTaskCreationAllowed } from "./onboarding-first-task-admission.js";
|
||||
|
||||
@@ -11391,7 +11391,7 @@ export function issueService(db: Db) {
|
||||
await notifyDecisionTargetWork(tx, `issue:${id}`);
|
||||
}
|
||||
if (patch.hiddenAt !== undefined || patch.harnessKind !== undefined || patch.conversationAgentId !== undefined) {
|
||||
await notifyRetentionWork(tx, "approval");
|
||||
await notifyApprovalWork(tx, receiptExisting.companyId);
|
||||
}
|
||||
const updated = await tx
|
||||
.update(issues)
|
||||
@@ -11800,7 +11800,8 @@ export function issueService(db: Db) {
|
||||
try {
|
||||
await notifyTerminalWorkspaceWork(tx, `issue:${id}`);
|
||||
await requestRestorePointDeletion(tx, { sourceIssueId: id });
|
||||
await notifyRetentionWork(tx, "approval");
|
||||
const [owner] = await tx.select({ companyId: issues.companyId }).from(issues).where(eq(issues.id, id));
|
||||
if (owner) await notifyApprovalWork(tx, owner.companyId);
|
||||
await notifyDecisionTargetIssueRemoval(tx, id);
|
||||
removedIssue = await tx
|
||||
.delete(issues)
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyApprovalWork } from "./approval-work-notifications.js";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import type { AttentionSourceKind } from "@paperclipai/shared";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
@@ -7,15 +8,8 @@ export async function notifyRetentionWork(transaction: object, kind: AttentionSo
|
||||
if (kind === "join_request") await notifyDeliveryWork(transaction, DELIVERY_QUEUES.joinRequestRetention);
|
||||
}
|
||||
|
||||
export function writeApprovalChange<T>(db: Db, write: (transaction: Db) => Promise<T>): Promise<T> {
|
||||
return db.transaction(async tx => {
|
||||
await notifyRetentionWork(tx, "approval");
|
||||
return write(tx as unknown as Db);
|
||||
});
|
||||
}
|
||||
|
||||
export async function notifyCompanyRetentionWork(transaction: object) {
|
||||
await notifyRetentionWork(transaction, "approval");
|
||||
export async function notifyCompanyRetentionWork(transaction: object, companyId: string) {
|
||||
await notifyApprovalWork(transaction, companyId);
|
||||
await notifyRetentionWork(transaction, "join_request");
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { notifyRetentionWork } from "./retention-work-notifications.js";
|
||||
import { notifyApprovalWork } from "./approval-work-notifications.js";
|
||||
import { notifyDecisionTargetIssueRemoval } from "./decision-targets.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
|
||||
import { verifyAppWebhook } from "./app-webhook.js";
|
||||
@@ -1966,7 +1966,7 @@ export function routineService(
|
||||
return updated ?? createdRun;
|
||||
} catch (error) {
|
||||
if (createdIssue) {
|
||||
await notifyRetentionWork(txDb, "approval");
|
||||
await notifyApprovalWork(txDb, createdIssue.companyId);
|
||||
await notifyDecisionTargetIssueRemoval(txDb, createdIssue.id);
|
||||
await txDb.delete(issues).where(eq(issues.id, createdIssue.id));
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { writeApprovalChange } from "./retention-work-notifications.js";
|
||||
import { writeApprovalChange } from "./approval-work-notifications.js";
|
||||
import { toolReviewProjectionService } from "./tool-review-projection.js";
|
||||
import { toolReviewInterruptionService } from "./tool-review-interruption.js";
|
||||
import { bindVerifiedToolApprovalArguments, claimToolTestApproval, releaseToolApproval, persistToolTestOutcome, startToolInvocation, type ToolApprovalClaim } from "./tool-approval-execution.js";
|
||||
@@ -2650,7 +2650,7 @@ export function createToolGatewayService(
|
||||
|
||||
let formalApprovalId: string | null = null;
|
||||
if (toolRequiresFormalApproval(input.tool)) {
|
||||
const [approval] = await writeApprovalChange(db, tx => tx
|
||||
const [approval] = await writeApprovalChange(db, input.session.companyId, tx => tx
|
||||
.insert(approvals)
|
||||
.values({
|
||||
companyId: input.session.companyId,
|
||||
@@ -2676,7 +2676,7 @@ export function createToolGatewayService(
|
||||
.returning());
|
||||
formalApprovalId = approval.id;
|
||||
const approvalIssueId = input.session.issueId;
|
||||
await writeApprovalChange(db, tx => tx
|
||||
await writeApprovalChange(db, input.session.companyId, tx => tx
|
||||
.insert(issueApprovals)
|
||||
.values({
|
||||
companyId: input.session.companyId,
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { notifyApprovalQueueWork } from "./approval-work-notifications.js";
|
||||
import { and, desc, eq, inArray } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { executionWorkspaces, heartbeatRunEvents, issueWorkProducts, workspaceRuntimeServices } from "@paperclipai/db";
|
||||
@@ -273,7 +274,10 @@ export function workProductService(
|
||||
|
||||
createForIssue: async (issueId: string, companyId: string, data: Omit<typeof issueWorkProducts.$inferInsert, "issueId" | "companyId">) => {
|
||||
const row = await db.transaction(async (tx) => {
|
||||
if (data.type === "pull_request") await notifyTerminalWorkspaceWork(tx, `issue:${issueId}`);
|
||||
if (data.type === "pull_request") {
|
||||
await notifyTerminalWorkspaceWork(tx, `issue:${issueId}`);
|
||||
await notifyApprovalQueueWork(tx, companyId);
|
||||
}
|
||||
await validateExecutionWorkspace(tx, companyId, data.executionWorkspaceId);
|
||||
if (data.isPrimary) {
|
||||
await tx
|
||||
@@ -309,7 +313,10 @@ export function workProductService(
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!existing) return null;
|
||||
|
||||
if (existing.type === "pull_request" || patch.type === "pull_request") await notifyTerminalWorkspaceWork(tx, `issue:${existing.issueId}`);
|
||||
if (existing.type === "pull_request" || patch.type === "pull_request") {
|
||||
await notifyTerminalWorkspaceWork(tx, `issue:${existing.issueId}`);
|
||||
await notifyApprovalQueueWork(tx, existing.companyId);
|
||||
}
|
||||
|
||||
await validateExecutionWorkspace(tx, existing.companyId, patch.executionWorkspaceId);
|
||||
|
||||
@@ -378,6 +385,9 @@ export function workProductService(
|
||||
// prefix behind (which a retry would then duplicate). Mirrors the
|
||||
// per-writer transaction the batched issue/document writers use.
|
||||
await db.transaction(async (tx) => {
|
||||
for (const companyId of new Set(values.filter(value => value.type === "pull_request").map(value => value.companyId))) {
|
||||
await notifyApprovalQueueWork(tx, companyId);
|
||||
}
|
||||
for (const value of values) {
|
||||
if (value.type === "pull_request") await notifyTerminalWorkspaceWork(tx, `issue:${value.issueId}`);
|
||||
}
|
||||
@@ -387,9 +397,12 @@ export function workProductService(
|
||||
|
||||
remove: async (id: string) => {
|
||||
const row = await db.transaction(async tx => {
|
||||
const [existing] = await tx.select({ type: issueWorkProducts.type, issueId: issueWorkProducts.issueId })
|
||||
const [existing] = await tx.select({ type: issueWorkProducts.type, issueId: issueWorkProducts.issueId, companyId: issueWorkProducts.companyId })
|
||||
.from(issueWorkProducts).where(eq(issueWorkProducts.id, id));
|
||||
if (existing?.type === "pull_request") await notifyTerminalWorkspaceWork(tx, `issue:${existing.issueId}`);
|
||||
if (existing?.type === "pull_request") {
|
||||
await notifyTerminalWorkspaceWork(tx, `issue:${existing.issueId}`);
|
||||
await notifyApprovalQueueWork(tx, existing.companyId);
|
||||
}
|
||||
return tx.delete(issueWorkProducts).where(eq(issueWorkProducts.id, id))
|
||||
.returning().then(rows => rows[0] ?? null);
|
||||
});
|
||||
|
||||
Reference in new issue
Block a user