mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-09 16:35:27 +02:00
Nicky LeachandPaperclip
1cebdd4c4f
## Thinking Path > - Paperclip manages agents and projects for work. > - Their lifecycle changes already commit to a durable journal. > - Plugins need to read these records and track completed work. > - Existing resources need a one-time baseline before delivery is enabled. > - Worker crashes must leave unfinished records available for retry. > - Each plugin needs its own progress and company access checks. > - This PR adds a pull inbox through the existing plugin SDK and job system. ## Linked Issues or Issue Description **Problem or motivation** Plugins cannot consume the durable resource lifecycle journal. In-process notifications can disappear during a restart and cannot record successful completion. Resource plugins need retries, company boundaries, and ordered transitions. **Proposed solution** Add `ctx.events.listLifecycle(companyId, limit?, afterId?)` and `ctx.events.acknowledgeLifecycle(companyId, eventId)` under `events.subscribe`. Seed a one-time baseline from current resource state. Deliver creation first, then the remaining transitions in ID order. Store acknowledgments for each plugin. Use existing plugin jobs to poll configured companies. **Alternatives considered** A global sequence cursor can skip lower IDs that commit later. Fire-and-forget subscriptions cannot record completion. A new dispatcher is unnecessary because plugin jobs support polling. Consumers serialize polling and use provider idempotency keys. **Roadmap alignment** This extends the existing plugin system and builds on #15280 and #15306. Searches found no duplicate resource inbox work. Related #13306 exposes decision events on the in-process bus. This PR includes the initial journal baseline. Provider provisioning remains separate work. ## What Changed - Add company-scoped lifecycle reads and acknowledgments to the SDK and worker RPC host. - Gate both methods by capability, invocation or proactive company scope, plugin readiness, and company enablement. - Seed hired agents and all projects once. Preserve paused/terminated agent state and archived project state. Keep pending hires behind approval. - Preserve existing history and deliver backfilled creation before partial transition histories. - Deliver project archive events from #15371 through the SDK. Seed archive intents for archived projects and update intents for active projects with a partial archive history. - Store acknowledgments per plugin and reject acknowledgments that skip earlier resource events. - Page past failed resources while retaining their pending records. Reset the page cursor each sweep to include late commits. - Add acknowledgment storage, an index for resource ordering, tests, and authoring guidance. - Preserve native identity definitions and sequence progress in JavaScript backups. Repair journal ID generators lost by older backups before seeding the baseline, without changing existing IDs. ## Verification - `pnpm -r typecheck` passed after the final origin/master rebase. - `pnpm build` passed before the final metadata rebase. The final CI build also passed. - All 46 focused lifecycle, SDK harness/RPC, migration snapshot, and legacy restore checks passed again with migration 0309. - Checks cover retries, per-plugin progress, resource order, capability/company boundaries, baseline idempotency, approval gates, archive delivery, restored projects with partial histories, and atomic migration rollback. - Full CI passed on final commit `0e674b87fc`. One unrelated Cursor fixture hit a 10-second timeout; it passed locally in under one second, and the failed server shard passed on retry. - Greptile scored the final commit 5/5 with no unresolved review threads. - The branch is rebased onto origin/master and is conflict-free. - Earlier full local runs were stopped as scope changed. CI ran the complete repository suite. - `git diff --check` and a local secret/PII scan passed. ## Risks - Apply migration `0309_loving_the_hood.sql` before starting the new server. It creates acknowledgment storage and an index, then seeds the baseline in the same transaction. Agent/project writes wait for the migration to commit. Keep these writes quiesced through the migration and activation of the new capture-capable server; do not resume an older runtime that lacks capture after the baseline. - The baseline records current desired state, not historical transitions. It includes archived projects and terminated-agent cleanup intents. It runs once with the delivery migration; no later or runtime journal backfill is planned. - Delivery is at least once. Concurrent reads can repeat an event. Consumers must serialize polling, use stable company/event idempotency keys, and acknowledge successful operations only. - Reset `afterId` at the start of every polling sweep. It is a page cursor, not a persisted high-water mark. - Deleting a plugin removes its acknowledgment records. A new installation may replay existing journal events. - Records contain identity and action. Consumers must load current authorized data before acting. Cleanup and retention policy belong to the provider plugin. - Provider calls, VM/volume provisioning, and journal retention are outside this PR. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository inspection, code execution, and tool use. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
53 lines
3.7 KiB
TypeScript
53 lines
3.7 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import { createTestHarness } from "../src/testing.js";
|
|
import type { PaperclipPluginManifestV1, ResourceLifecycleEvent } from "../src/types.js";
|
|
|
|
const manifest: PaperclipPluginManifestV1 = {
|
|
id: "fixture.lifecycle", apiVersion: 1, version: "1.0.0", displayName: "Lifecycle", description: "Fixture",
|
|
author: "Tests", categories: ["automation"], capabilities: ["events.subscribe"], entrypoints: {},
|
|
};
|
|
|
|
describe("Lifecycle inbox test harness", () => {
|
|
it("delivers project creation, archive, and restore in order", async () => {
|
|
const harness = createTestHarness({ manifest });
|
|
const event: ResourceLifecycleEvent = { id: "1", companyId: "a", resourceType: "project", resourceId: "project", action: "create", createdAt: new Date(0).toISOString() };
|
|
harness.seed({ lifecycleEvents: [event, { ...event, id: "2", action: "archive" }, { ...event, id: "3", action: "update" }] });
|
|
for (const action of ["create", "archive", "update"]) {
|
|
const [next] = await harness.ctx.events.listLifecycle("a");
|
|
expect(next.action).toBe(action);
|
|
await harness.ctx.events.acknowledgeLifecycle("a", next.id);
|
|
}
|
|
expect(await harness.ctx.events.listLifecycle("a")).toEqual([]);
|
|
});
|
|
it("lets a plugin exercise retries, acknowledgment order, and company scope", async () => {
|
|
const harness = createTestHarness({ manifest });
|
|
const event: ResourceLifecycleEvent = { id: "1", companyId: "a", resourceType: "agent", resourceId: "agent", action: "create", createdAt: new Date(0).toISOString() };
|
|
harness.seed({ lifecycleEvents: [event, { ...event, id: "2", action: "pause" }, { ...event, id: "3", companyId: "b" }] });
|
|
expect(await harness.ctx.events.listLifecycle("a")).toEqual([event]);
|
|
expect(await harness.ctx.events.listLifecycle("a", 100, "1")).toEqual([]);
|
|
expect(await harness.ctx.events.listLifecycle("a")).toEqual([event]);
|
|
await expect(harness.ctx.events.acknowledgeLifecycle("a", "2")).rejects.toThrow("earlier lifecycle events");
|
|
await expect(harness.ctx.events.acknowledgeLifecycle("a", "3")).rejects.toThrow("not found");
|
|
await harness.ctx.events.acknowledgeLifecycle("a", "1");
|
|
await harness.ctx.events.acknowledgeLifecycle("a", "1");
|
|
expect(await harness.ctx.events.listLifecycle("a")).toEqual([expect.objectContaining({ action: "pause" })]);
|
|
expect(await harness.ctx.events.listLifecycle("b")).toEqual([expect.objectContaining({ id: "3" })]);
|
|
const denied = createTestHarness({ manifest, capabilities: [] });
|
|
await expect(denied.ctx.events.listLifecycle("a")).rejects.toThrow("events.subscribe");
|
|
await expect(denied.ctx.events.acknowledgeLifecycle("a", "1")).rejects.toThrow("events.subscribe");
|
|
});
|
|
it("delivers a backfilled creation before earlier transitions", async () => {
|
|
const harness = createTestHarness({ manifest });
|
|
const event: ResourceLifecycleEvent = { id: "1", companyId: "a", resourceType: "agent", resourceId: "agent", action: "pause", createdAt: new Date(0).toISOString() };
|
|
harness.seed({ lifecycleEvents: [event, { ...event, id: "2", action: "resume" }, { ...event, id: "3", action: "create" }] });
|
|
expect(await harness.ctx.events.listLifecycle("a")).toEqual([expect.objectContaining({ id: "3", action: "create" })]);
|
|
await expect(harness.ctx.events.acknowledgeLifecycle("a", "1")).rejects.toThrow("earlier lifecycle events");
|
|
await harness.ctx.events.acknowledgeLifecycle("a", "3");
|
|
expect(await harness.ctx.events.listLifecycle("a", 100, "3")).toEqual([]);
|
|
expect(await harness.ctx.events.listLifecycle("a")).toEqual([event]);
|
|
await harness.ctx.events.acknowledgeLifecycle("a", "1");
|
|
expect(await harness.ctx.events.listLifecycle("a")).toEqual([expect.objectContaining({ id: "2" })]);
|
|
});
|
|
|
|
});
|