mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 17:51:58 +01:00
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/206/changes/paperclip/206 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip coordinates agent work and presents its progress in chat. > - GitHub deliveries save acknowledgement actions and provider receipts. > - Periodic maintenance currently searches for unfinished acknowledgements even when idle. > - Committed delivery, lease, response and presentation changes identify when work can advance. > - The existing owner should wake on those changes and recover only at saved deadlines. > - Completed history should become quiet without losing unfinished provider effects. ## Linked Issues or Issue Description Refs #17653, which is merged. This existing GitHub acknowledgement slice targets master. **Problem:** A saved GitHub acknowledgement can wait for a publication lease, fast-response result or native presentation decision. Polling empty or unchanged work keeps the tenant active. Removing maintenance without covering these producers can strand the acknowledgement. **Expected behavior:** Committed changes wake the scoped owner. It reconstructs existing actions on startup, retries at saved deadlines, parks unchanged presentation waits and becomes quiet when settled. ## What Changed - Register GitHub acknowledgement work with the shared scoped deadline coordinator. Reuse existing action, delivery, lease and fast-response rows. - Add transaction-aware response-availability notifications at supported writes. Keep GitHub protocol details in its platform owner and shared notifications limited to IDs. - Share the bounded latest-request/run projection between scheduling and direct response authorization. An absent native presentation decision parks the action until its real writer commits. - Retain publication leases, source/principal authority, version fencing, response markers and final receipt precedence. Check admission before each new provider request; let in-flight work settle. Stop does not consume the provider retry budget. - Remove only recurring acknowledgement recovery and keep its manual path. Add no work table or scheduler service. - Regenerate the three original indexes as `0400_absent_phantom_reporter.sql`: pending acknowledgement IDs, fast-response delivery lookup and publication comment/destination lookup. Each uses concurrent/idempotent creation and the existing top-level migration runner. Preserve all landed SQL/journal entries and the five newest snapshots. ## Verification Published head: `09ca00b9f029838434ec042cda691345d67b493e`, on landed continuation parent `063cfd43813cabf18417605fb9eefb9aacf95c8d`. - **87 distinct current cases pass:** 18 real PostgreSQL acknowledgement lifecycle/presentation cases, all 26 fast-response service cases, all 30 heartbeat completion cases and all 13 startup cases. Database build, server typechecking/build, module boundaries and three fresh migrations through 0400 pass. The current Runner bundle rebuild also passes after upstream provider-event changes. - The lifecycle matrix covers empty/history quiet, lease release, restart, drain/standby, fast-response failure/expiry/publication/unknown outcome, streaming quarantine, outer rollback/commit, saved provider retry, stop between requests and receipt state. The real presentation writer proves quiet while pending, rollback, committed reactivation and final quiet. - The earlier **305-case** matrix (236 full GitHub workflow cases, 26 fast-response service cases, 30 completion cases and 13 startup cases) passes at `337f272853d4f8a51b5b9300900a46eb4e13f698`. It retains its prepared-parent scope. Repeated cases are not added to current 87. - A populated 0399 database upgrades to 0400 with all 398 prior migration receipts unchanged, one new receipt and no additional receipt on replay. All three indexes are valid/ready on fresh and upgraded databases. Their SQL definitions match the original draft after whitespace normalization; migration history and retained snapshot bytes are preserved. - Normal integration `9b3607e159153e00ccdfe538d52b591bd38d6499` preserves the full 28-path owned binary patch and all 20 unrelated master paths, including native retry, provider-event and evaluation workflow changes. Both normal merge parents and the exact tree are checked. No force push or rebase occurs. - [The owner audit](https://github.com/paperclipai/paperclip/blob/09ca00b9f029838434ec042cda691345d67b493e/doc/plans/2026-10-25-github-chat-maintenance-sleep-audit.md#acceptance-on-the-actual-landed-continuation-parent) retains the original missing-notification, JSON precedence, receipt lifecycle, external-database fixture and migration-registry failures. Earlier 69/228/broad counts keep their original scope. Skips and failed revisions are not relabelled as current passes. - The final audit-only edit leaves all owned source/test/migration hashes unchanged. **Fresh exact-head CI and Greptile 5/5 with no actionable findings remain required.** No full local repository, live GitHub tenant or deployed sleep result is claimed. ## Risks - Process-local transaction hints cover supported application writers and shared pools. Independent SQL writers and other replicas do not gain a new wake guarantee. - Provider effects remain guarded by the existing publication lease and current source/principal authority. Unknown outcomes retain their existing recovery semantics. - Presentation waits depend on their supported milestone/transaction producers. A missing producer can defer recovery until another supported change or restart. - Index creation can take time on large histories and runs concurrently outside a transaction. Existing migration recovery handles replay; prior SQL is unchanged. - Worker quiet is distinct from host/database suspension. Cloud wake coverage and sleep admission remain unchanged; later maintenance duties are separate slices. ## Model Used OpenAI Codex, using reasoning, repository inspection, code execution and real PostgreSQL tests. The exact model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
113 lines
7.9 KiB
TypeScript
113 lines
7.9 KiB
TypeScript
import { readGitHubResponseRun } from "./response-run-state.js";
|
|
import { githubRunReplyState } from "../../chat-run-publications.js";
|
|
import { and, asc, eq, gt, isNotNull, or, sql } from "drizzle-orm";
|
|
import { chatActions, chatConversations, chatEndpointLeases, chatEndpointResources,
|
|
chatPublications, fastResponseRequests, chatEndpoints, chatMessageLinks, type Db } from "@paperclipai/db";
|
|
import type { createDeliveryWorkCoordinator } from "../../delivery-work-coordinator.js";
|
|
import { DELIVERY_QUEUES } from "../../delivery-work-notifications.js";
|
|
import type { ChatResponseScope } from "../../chat-response-availability.js";
|
|
import { registerScopedDeadlineWork } from "../../scoped-deadline-work.js";
|
|
|
|
type Scope = ChatResponseScope & { endpointId?: string; repositoryId?: string; number?: number };
|
|
type Entry = { id: string; companyId: string; endpointId: string; deliveryId: string;
|
|
issueId: string | null; conversationId: string | null; repositoryId: string | null; number: number | null; dueAt: number };
|
|
type FastResponse = { status: string; expiresAt: string; publicationState: string | null };
|
|
|
|
export function registerGitHubAcknowledgementWork(
|
|
db: Db,
|
|
coordinator: ReturnType<typeof createDeliveryWorkCoordinator>,
|
|
acknowledge: (deliveryId: string, canContinue: () => boolean) => Promise<unknown>,
|
|
canRun: () => boolean,
|
|
) {
|
|
const repositoryId = sql<string | null>`${chatEndpointResources.metadata}->>'providerRepositoryId'`;
|
|
const number = sql<string | null>`substring(${chatConversations.externalThreadId}
|
|
from '^github:[^:]+:(?:issue:)?([1-9][0-9]*)(?::rc:[1-9][0-9]*)?$')`;
|
|
const selection = {
|
|
id: chatActions.id, companyId: chatActions.companyId, endpointId: chatActions.endpointId,
|
|
deliveryId: chatActions.deliveryId, conversationId: chatActions.conversationId, issueId: chatConversations.issueId,
|
|
agentId: chatEndpoints.assignedAgentId,
|
|
sourceMessageId: sql<string | null>`(select m.comment_id from ${chatMessageLinks} m
|
|
where m.company_id = ${chatActions.companyId} and m.endpoint_id = ${chatActions.endpointId}
|
|
and m.delivery_id = ${chatActions.deliveryId} and m.direction = 'inbound' limit 1)`,
|
|
repositoryId, number, status: chatActions.status, result: chatActions.result,
|
|
leaseExpiresAt: sql<string | null>`(select max(l.expires_at) from ${chatEndpointLeases} l
|
|
where l.company_id = ${chatActions.companyId} and l.endpoint_id = ${chatActions.endpointId}
|
|
and l.lease_key = 'github-publication:' || (${repositoryId}) || ':' || ${number})`,
|
|
fast: sql<FastResponse | null>`(select jsonb_build_object('status', f.status, 'expiresAt', f.expires_at,
|
|
'publicationState', p.state) from ${fastResponseRequests} f left join ${chatPublications} p
|
|
on p.company_id = f.company_id and p.comment_id = f.comment_id
|
|
and p.endpoint_id = f.endpoint_id and p.conversation_id = f.conversation_id
|
|
where f.company_id = ${chatActions.companyId} and f.endpoint_id = ${chatActions.endpointId}
|
|
and f.delivery_id = ${chatActions.deliveryId} limit 1)`,
|
|
};
|
|
async function page(scope?: Scope, cursor?: string, actionId?: string) {
|
|
return db.select(selection).from(chatActions)
|
|
.innerJoin(chatEndpoints, and(eq(chatEndpoints.id, chatActions.endpointId), eq(chatEndpoints.companyId, chatActions.companyId)))
|
|
.leftJoin(chatConversations, and(eq(chatConversations.id, chatActions.conversationId), eq(chatConversations.companyId, chatActions.companyId)))
|
|
.leftJoin(chatEndpointResources, and(eq(chatEndpointResources.id, chatConversations.resourceId), eq(chatEndpointResources.companyId, chatActions.companyId)))
|
|
.where(and(eq(chatActions.kind, "github_response_comment"), isNotNull(chatActions.deliveryId),
|
|
sql`coalesce(${chatActions.result}->>'final', 'false') <> 'true'`,
|
|
or(eq(chatActions.status, "received"), and(eq(chatActions.status, "failed"), sql`${chatActions.result}->>'retryable' = 'true'`)),
|
|
scope ? eq(chatActions.companyId, scope.companyId) : undefined,
|
|
scope?.issueId ? eq(chatConversations.issueId, scope.issueId) : undefined,
|
|
scope?.conversationId ? eq(chatActions.conversationId, scope.conversationId) : undefined,
|
|
scope?.deliveryId ? eq(chatActions.deliveryId, scope.deliveryId) : undefined,
|
|
scope?.endpointId ? eq(chatActions.endpointId, scope.endpointId) : undefined,
|
|
scope?.repositoryId ? eq(repositoryId, scope.repositoryId) : undefined,
|
|
scope?.number ? eq(number, String(scope.number)) : undefined,
|
|
cursor ? gt(chatActions.id, cursor) : undefined,
|
|
actionId ? eq(chatActions.id, actionId) : undefined,
|
|
)).orderBy(asc(chatActions.id)).limit(100);
|
|
}
|
|
async function entry(row: Awaited<ReturnType<typeof page>>[number]): Promise<Entry | null> {
|
|
const { latestRun, ownsLatestRequest } = await readGitHubResponseRun(db, {
|
|
companyId: row.companyId, endpointId: row.endpointId, agentId: row.agentId,
|
|
issueId: row.issueId, deliveryId: row.deliveryId!, sourceMessageId: row.sourceMessageId,
|
|
});
|
|
// Execution success alone does not decide whether a final response exists.
|
|
// Presentation/publication commits invalidate this parked request.
|
|
if (row.issueId && ownsLatestRequest && latestRun?.nativePresentationPending &&
|
|
await githubRunReplyState(db, { companyId: row.companyId, endpointId: row.endpointId,
|
|
issueId: row.issueId, runId: latestRun.id }) === "none") return null;
|
|
let dueAt = row.status === "failed" ? Date.parse(String(row.result?.retryAt)) : 0;
|
|
if (!Number.isFinite(dueAt)) throw new Error("GitHub acknowledgement has an invalid retry deadline");
|
|
const fast = row.fast;
|
|
if (fast?.publicationState === "streaming") return null;
|
|
if (fast && !["published", "delivery_unknown"].includes(fast.publicationState ?? "") &&
|
|
(["pending", "running"].includes(fast.status) || ["pending", "retry"].includes(fast.publicationState ?? ""))) {
|
|
dueAt = Math.max(dueAt, Date.parse(fast.expiresAt));
|
|
}
|
|
if (row.leaseExpiresAt) dueAt = Math.max(dueAt, Date.parse(row.leaseExpiresAt));
|
|
return { id: row.id, companyId: row.companyId, endpointId: row.endpointId, deliveryId: row.deliveryId!,
|
|
issueId: row.issueId, conversationId: row.conversationId, repositoryId: row.repositoryId, number: row.number ? Number(row.number) : null, dueAt };
|
|
}
|
|
return registerScopedDeadlineWork(coordinator, {
|
|
queue: DELIVERY_QUEUES.chatGitHubAcknowledgements,
|
|
decodeScope: (hint: string): Scope | undefined => hint === "*" ? undefined : JSON.parse(hint),
|
|
matches: (value: Entry, scope?: Scope) => !scope || (value.companyId === scope.companyId &&
|
|
(!scope.issueId || value.issueId === scope.issueId) && (!scope.deliveryId || value.deliveryId === scope.deliveryId) &&
|
|
(!scope.conversationId || value.conversationId === scope.conversationId) &&
|
|
(!scope.endpointId || value.endpointId === scope.endpointId) && (!scope.repositoryId || value.repositoryId === scope.repositoryId) &&
|
|
(!scope.number || value.number === scope.number)),
|
|
load: async (scope?: Scope, signal?: AbortSignal) => {
|
|
const entries: Entry[] = [];
|
|
let cursor: string | undefined;
|
|
while (canRun() && !signal?.aborted) {
|
|
const rows = await page(scope, cursor);
|
|
for (const row of rows) { const value = await entry(row); if (value) entries.push(value); }
|
|
if (rows.length < 100) return entries;
|
|
cursor = rows[rows.length - 1].id;
|
|
}
|
|
throw signal?.reason ?? new Error("GitHub acknowledgement recovery stopped");
|
|
},
|
|
check: async (value: Entry, signal?: AbortSignal) => {
|
|
const [row] = await page(undefined, undefined, value.id);
|
|
const current = row ? await entry(row) : null;
|
|
if (!current || current.dueAt > Date.now() || !canRun() || signal?.aborted) return current;
|
|
await acknowledge(current.deliveryId, () => canRun() && !signal?.aborted);
|
|
const [remaining] = await page(undefined, undefined, value.id);
|
|
return remaining ? await entry(remaining) : null;
|
|
},
|
|
}, canRun);
|
|
}
|