mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-31 14:16:39 +01:00
## Thinking Path > - Paperclip runs autonomous work through agents, schedules, and plugins. > - Automatic idle sleep must preserve accepted work and unfinished cleanup. > - Every enabled plugin currently blocks sleep, including unused providers. > - An enabled flag or package version cannot prove that a worker is idle. > - This change adds a live worker drain under the existing owned hold. > - Idle workers can allow sleep while unknown work stays protected. ## Linked Issues or Issue Description Refs #15522. Related: #15391 adds plugin readiness for agent admission; this change concerns instance sleep and does not replace that contract. **Subsystem affected** Plugin worker lifecycle and automatic idle sleep. **Problem or motivation** A workspace with no pending work cannot sleep when any plugin is enabled. Removing that check alone would lose accepted RPCs, background tasks, or cleanup after a caller timeout. **Proposed solution** Require a live `onIdleDrain` handshake from each worker. Close admission in both processes for the exact owner and expiry. Count accepted work until completion. Continue checking durable work separately. ## What Changed - Add bounded worker holds, exact-owner release, automatic expiry, and an abort signal for plugin-owned background work. - Count host and worker requests through their real completion receipts. Keep timed-out work counted. Check active notifications and terminal routes. - Accept enabled plugins only when their current workers provide matching runtime receipts. Missing workers, old SDKs, crashes, invalid replies, and unknown cleanup still prevent sleep. - Let unused Daytona workers opt in. Once a worker contacts the provider, it remains a blocker for that process lifetime. This restriction avoids treating its existing timeout and terminal-close behavior as a cleanup receipt. - Document the plugin author contract. No schema or user-facing API is added. ## Verification - All hosted CI checks passed on6ffb682894, including all server, runner, workspace, build, typecheck, and end-to-end checks. One unrelated OpenCode transport test hit a five-second timeout and passed on rerun. - `pnpm -r typecheck` passed. - `pnpm build` passed. - Focused Vitest: 293 tests passed across worker admission, real child-process RPC, durable idle checks, and Daytona. - Adjacent worker manager and duplex tests: 119 passed. - Real child-process tests cover late worker completion, a host write after caller timeout, stale owners, closed admission, and release. - SDK tests cover hook failures, busy workers, expiry without another request, and late acknowledgements after expiry. - Initial long local run: 17,202 passed, 10 failed. Five failures involved parent-directory skill fixtures; four of those were already reproduced and passed in an isolated worktree. One HTTP test had a socket hang-up; all 66 tests in that file passed on rerun. The other four used new tests with runtime modules loaded before the follow-up edits; all pass in the fresh focused run above. A clean build and full-suite rerun at the final commit are running in an isolated worktree. No full-suite pass is claimed. - Follow-up: 44 focused tests passed, including repeated owned holds without an intervening normal request. Worker status changes now invalidate an in-progress scan. - Apex follow-up: 13 idle tests, 119 adjacent worker tests, 71 instance-settings route tests, and server typecheck passed. A real queued notification completes its write while preparation is in flight; synchronous and async session callback failures are logged. - Diff whitespace and added-line secret/private-reference scans passed. ## Risks - Plugins are trusted code. A hook must account for work outside SDK RPCs and keep it quiescent until its signal aborts. There is no package-name or manifest-only exemption. - This first change targets unused workers. Used Daytona workers remain awake until remote cleanup receipts are complete. Restarting does not bypass durable lease and recovery checks. - Unknown completion remains a blocker, including after a worker crash. This can retain cost but cannot authorize sleep from a timeout alone. - Enabled jobs, schedules, credentials, integrations, and other durable work remain blockers. Durable wake ownership is separate follow-up work. - Rollback restores the blanket plugin blocker. No data migration is required. ## Model Used OpenAI Codex, GPT-6. Used repository inspection, reasoning, code editing, and command execution. The runtime did not report an exact model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing public issues or described the issue in this PR - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change and contains no internal identifiers - [x] Focused local tests pass; full CI is green (long local rerun status is recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile Apex is 5/5 on6ffb682894with no open findings - [x] I will address all review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
90 lines
4.7 KiB
TypeScript
90 lines
4.7 KiB
TypeScript
import { PassThrough } from "node:stream";
|
|
import { describe, expect, it } from "vitest";
|
|
import { definePlugin, type PluginDefinition } from "../../../packages/plugins/sdk/src/define-plugin.js";
|
|
import { startWorkerRpcHost } from "../../../packages/plugins/sdk/src/worker-rpc-host.js";
|
|
|
|
function fixture(definition: PluginDefinition, rpcTimeoutMs = 30_000) {
|
|
const stdin = new PassThrough();
|
|
const stdout = new PassThrough();
|
|
const worker = startWorkerRpcHost({ plugin: definePlugin(definition), stdin, stdout, rpcTimeoutMs });
|
|
const messages: any[] = [];
|
|
stdout.on("data", (chunk) => { for (const line of String(chunk).trim().split("\n")) messages.push(JSON.parse(line)); });
|
|
let id = 0;
|
|
async function call(method: string, params: unknown = {}) {
|
|
const callId = ++id;
|
|
stdin.write(JSON.stringify({ jsonrpc: "2.0", id: callId, method, params }) + "\n");
|
|
await expect.poll(() => messages.find((message) => message.id === callId && !message.method)).toBeDefined();
|
|
return messages.find((message) => message.id === callId && !message.method);
|
|
}
|
|
const initialize = () => call("initialize", { manifest: { id: "test.idle", apiVersion: 1 }, config: {}, apiVersion: 1 });
|
|
return { call, initialize, messages, stdin, close: () => worker.stop() };
|
|
}
|
|
|
|
describe("plugin idle RPC", () => {
|
|
const hold = () => ({ ownerId: "idle-owner", expiresAt: Date.now() + 30_000 });
|
|
|
|
it("seals an idle worker, fences stale releases, and restores normal calls", async () => {
|
|
const f = fixture({ async setup() {}, async onIdleDrain() { return "none"; } });
|
|
try {
|
|
expect((await f.initialize()).result.supportedMethods).toContain("prepareIdleSleep");
|
|
const lease = hold();
|
|
expect((await f.call("prepareIdleSleep", lease)).result).toEqual({ ...lease, backgroundWork: "none" });
|
|
expect((await f.call("health")).error).toBeDefined();
|
|
await f.call("releaseIdleSleep", { ownerId: "stale" });
|
|
expect((await f.call("health")).error).toBeDefined();
|
|
await f.call("releaseIdleSleep", lease);
|
|
expect((await f.call("health")).result.status).toBe("ok");
|
|
} finally { f.close(); }
|
|
});
|
|
|
|
it("does not mistake a caller timeout for a completed worker handler", async () => {
|
|
let finish!: () => void;
|
|
const f = fixture({ async setup() {}, async onIdleDrain() { return "none"; },
|
|
async onHealth() { await new Promise<void>((resolve) => { finish = resolve; }); return { status: "ok" }; } });
|
|
try {
|
|
await f.initialize();
|
|
const accepted = f.call("health");
|
|
await expect.poll(() => finish).toBeDefined();
|
|
expect((await f.call("prepareIdleSleep", hold())).result.backgroundWork).toBe("present");
|
|
finish(); await accepted;
|
|
expect((await f.call("prepareIdleSleep", hold())).result.backgroundWork).toBe("none");
|
|
} finally { f.close(); }
|
|
});
|
|
|
|
it("retains timed-out worker-to-host writes until the host's late receipt", async () => {
|
|
let write!: () => Promise<unknown>;
|
|
const f = fixture({ async setup(ctx) { write = () => ctx.state.set({ scopeKind: "instance", stateKey: "fixture" }, { value: 1 }); },
|
|
async onIdleDrain() { return "none"; } }, 10);
|
|
try {
|
|
await f.initialize();
|
|
await expect(write()).rejects.toThrow("timed out");
|
|
const request = f.messages.find((message) => message.method === "state.set");
|
|
expect((await f.call("prepareIdleSleep", hold())).result.backgroundWork).toBe("present");
|
|
f.stdin.write(JSON.stringify({ jsonrpc: "2.0", id: request.id, result: null }) + "\n");
|
|
expect((await f.call("prepareIdleSleep", hold())).result.backgroundWork).toBe("none");
|
|
} finally { f.close(); }
|
|
});
|
|
|
|
it.each([false, true])("logs session callback failures (async: %s)", async (asynchronous) => {
|
|
let subscribe!: () => Promise<unknown>;
|
|
const f = fixture({ async setup(ctx) {
|
|
subscribe = () => ctx.agents.sessions.sendMessage("fixture-session", "fixture-company", {
|
|
prompt: "fixture",
|
|
onEvent: asynchronous
|
|
? async () => { throw new Error("fixture callback failure"); }
|
|
: () => { throw new Error("fixture callback failure"); },
|
|
});
|
|
} });
|
|
try {
|
|
await f.initialize();
|
|
const subscription = subscribe();
|
|
const request = f.messages.find((message) => message.method === "agents.sessions.sendMessage");
|
|
f.stdin.write(JSON.stringify({ jsonrpc: "2.0", id: request.id, result: {} }) + "\n");
|
|
await subscription;
|
|
f.stdin.write(JSON.stringify({ jsonrpc: "2.0", method: "agents.sessions.event", params: { sessionId: "fixture-session" } }) + "\n");
|
|
await expect.poll(() => f.messages.find((message) => message.method === "log" &&
|
|
message.params.level === "error" && message.params.message.includes("fixture callback failure"))).toBeDefined();
|
|
} finally { f.close(); }
|
|
});
|
|
});
|