mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 16:27:18 +02:00
## Thinking Path > - Paperclip manages agents and their runtime configuration. > - Plugins can maintain external agent computers. > - Configuration and file saves currently miss durable plugin delivery. > - A transient activity notification cannot recover after interruption. > - This pull request adds an opt-in agent runtime resource stream. > - The existing dispatcher delivers and acknowledges its committed work. > - Plugins can prepare computers when configuration changes. ## Linked Issues or Issue Description Related: #15631 and #16203 establish agent lifecycle and native durable delivery. **Problem** Plugins cannot reliably react to agent configuration and instruction changes without polling or waiting for execution. **Proposed solution** Add an opt-in `agent_runtime` stream to the existing resource journal. Record changes at the database mutation and managed file audit boundaries. Wake the dispatcher with content-free commit notifications. Provide scoped runtime inputs to the selected lifecycle plugin. **Alternatives considered** Route-only activity events miss service and background writers. Scheduled scans delay changes and repeatedly query unchanged state. **Roadmap alignment** This extends the existing plugin and agent configuration work. ## What Changed - Add durable update/delete events for agent runtime state, including an existing-agent baseline. - Capture configuration changes, instruction edits, deletion, restore, and managed run writeback. - Add commit notifications and a reconnect scan. Register one listener for configured, enabled runtime subscribers. Release it when none remain. - Keep HTTP startup independent of queued plugin callbacks. - Add `ctx.agents.getRuntime`. Require lifecycle capability, invocation company, enabled plugin, and owning lifecycle selection. - Keep legacy agent/project subscriptions isolated. Manual inbox reads retain both legacy streams. - Resolve recovered instruction paths without reading or hashing bundle contents. Use the company reporting graph for runtime health. - Repair missed file events on identical save/deletion retries after a journal failure; preserve content tokens and conditional-write checks. - Correlate file audits with runtime revisions to deduplicate recorded retries without a new table. Bound the lookup with the existing company/timestamp index. - Document the contract, migration, access boundaries, and session-connection requirement. ## Verification - Rebased the full stack onto origin/master after #16507. - Final replayed stack: `pnpm -r typecheck` and `pnpm build` passed. - All 132 affected runtime, skill, connector, publication, instruction, managed-agent, and delivery tests passed after the latest-master rebase. Workspace typecheck and build passed. - Journal failure injection verifies recovery on identical file writes, deletions, and instruction retries from an acknowledged inbox. - Deduplication follow-up: 80 runtime/instruction/delivery tests and server typecheck passed. Repeated retries create no audit/events; old hashes returning after an intervening commit still repair delivery. Built-in/native instruction writer regressions passed all 56 tests. - PR 1 follow-up tests cover valid/cyclic reporting chains, listener registration failure, multiple subscribers, global/company disable, re-enable, unload, and startup with blocked callbacks. - Database coverage includes rollback, journal failure, no-op suppression, file saves/deletion, ordering/replay, owner/company denial, independent-client commits, and real listener loss/reconnect. - SDK/shared suites passed 1,059 tests during implementation. CLI/delivery regressions also passed. - Migration safety, map checks, diff whitespace, and local secrets/PII scan passed. - Hosted CI and security checks are fully green at `b296405e2`. Standard Greptile and Apex both returned 5/5 with no unresolved findings. - The broad local suite encountered an unchanged diagnostic test that rejects the word "private" from an error stack. This worktree lives under /private/tmp. The same unchanged diagnostic suite passed all 108 tests in the original checkout. Hosted full-suite checks provide exact-head verification. ## Risks - Apply migration 0347 before enabling a runtime subscriber. Runtime notifications require session-level PostgreSQL LISTEN. Transaction-mode poolers are unsupported for this opt-in feature. - Database triggers share the mutation transaction. Journal failure prevents configuration commits. Trigger work adds a journal insert to relevant changes. - Filesystem publication and SQL commit are separate boundaries. Direct filesystem edits do not emit hooks. External clients must fence stale revisions and retain conditional-write checks. - The reader returns only that plugin's own instance secrets. Keep them out of UI data, logs, plugin state, and agent files. - No private transport integration or live remote computer qualification is included. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code execution, and testing. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass (focused suites; broad local limitation documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
59 lines
4.1 KiB
TypeScript
59 lines
4.1 KiB
TypeScript
import { pluginCompanySettings, pluginLifecycleAcknowledgments, plugins, resourceLifecycleEvents, type Db } from "@paperclipai/db";
|
|
import type { ResourceLifecycleEvent } from "@paperclipai/plugin-sdk";
|
|
import { z } from "zod";
|
|
import { and, asc, eq, gt, inArray, sql } from "drizzle-orm";
|
|
import { badRequest, conflict, forbidden, notFound } from "../errors.js";
|
|
|
|
/** Durable delivery inbox. Provider work stays outside tenant transactions. */
|
|
export function pluginLifecycleInbox(db: Db, pluginId: string) {
|
|
const unacknowledged = sql`NOT EXISTS (
|
|
SELECT 1 FROM plugin_lifecycle_acknowledgments ack
|
|
WHERE ack.plugin_id = ${pluginId} AND ack.event_id = ${resourceLifecycleEvents.id}
|
|
)`;
|
|
const firstForResource = sql`NOT EXISTS (
|
|
SELECT 1 FROM resource_lifecycle_events older
|
|
WHERE older.company_id = ${resourceLifecycleEvents.companyId}
|
|
AND older.resource_type = ${resourceLifecycleEvents.resourceType}
|
|
AND older.resource_id = ${resourceLifecycleEvents.resourceId}
|
|
-- Backfilled creation must precede transitions captured before this baseline.
|
|
AND ${resourceLifecycleEvents.action} <> 'create'
|
|
AND (older.action = 'create' OR older.id < ${resourceLifecycleEvents.id})
|
|
AND NOT EXISTS (SELECT 1 FROM plugin_lifecycle_acknowledgments ack
|
|
WHERE ack.plugin_id = ${pluginId} AND ack.event_id = older.id)
|
|
)`;
|
|
|
|
async function assertAvailable(database: Db, companyId: string) {
|
|
if (!z.string().uuid().safeParse(companyId).success) throw badRequest("companyId must be a UUID");
|
|
const [plugin] = await database.select({ status: plugins.status }).from(plugins).where(eq(plugins.id, pluginId));
|
|
const [settings] = await database.select({ enabled: pluginCompanySettings.enabled }).from(pluginCompanySettings)
|
|
.where(and(eq(pluginCompanySettings.pluginId, pluginId), eq(pluginCompanySettings.companyId, companyId)));
|
|
if (plugin?.status !== "ready" || settings?.enabled === false) throw forbidden("Plugin is not enabled for this company");
|
|
}
|
|
|
|
return {
|
|
async list(companyId: string, limit = 50, afterId?: string, resourceTypes?: ResourceLifecycleEvent["resourceType"][]): Promise<ResourceLifecycleEvent[]> {
|
|
if (!Number.isInteger(limit) || limit < 1 || limit > 100) throw badRequest("limit must be an integer from 1 to 100");
|
|
if (afterId !== undefined && (typeof afterId !== "string" || !/^[1-9]\d*$/.test(afterId) || !Number.isSafeInteger(Number(afterId)))) throw badRequest("Invalid lifecycle page id");
|
|
await assertAvailable(db, companyId);
|
|
const rows = await db.select().from(resourceLifecycleEvents)
|
|
.where(and(eq(resourceLifecycleEvents.companyId, companyId), unacknowledged, firstForResource, inArray(resourceLifecycleEvents.resourceType, resourceTypes ?? ["agent", "project"]), afterId === undefined ? undefined : gt(resourceLifecycleEvents.id, Number(afterId))))
|
|
.orderBy(asc(resourceLifecycleEvents.id)).limit(limit);
|
|
return rows.map(row => ({ ...row, id: String(row.id), createdAt: row.createdAt.toISOString() })) as ResourceLifecycleEvent[];
|
|
},
|
|
async acknowledge(companyId: string, eventId: string): Promise<void> {
|
|
if (typeof eventId !== "string" || !/^[1-9]\d*$/.test(eventId) || !Number.isSafeInteger(Number(eventId))) throw badRequest("Invalid lifecycle event id");
|
|
await db.transaction(async tx => {
|
|
const txDb = tx as unknown as Db;
|
|
await assertAvailable(txDb, companyId);
|
|
const [event] = await tx.select().from(resourceLifecycleEvents)
|
|
.where(and(eq(resourceLifecycleEvents.companyId, companyId), eq(resourceLifecycleEvents.id, Number(eventId)))).for("update");
|
|
if (!event) throw notFound("Lifecycle event not found");
|
|
const [eligible] = await tx.select({ id: resourceLifecycleEvents.id }).from(resourceLifecycleEvents)
|
|
.where(and(eq(resourceLifecycleEvents.id, event.id), firstForResource));
|
|
if (!eligible) throw conflict("Acknowledge earlier lifecycle events for this resource first");
|
|
await tx.insert(pluginLifecycleAcknowledgments).values({ pluginId, eventId: event.id }).onConflictDoNothing();
|
|
});
|
|
},
|
|
};
|
|
}
|