Files
PaperClipAI/server/src/__tests__/plugin-idle-rpc.test.ts
T
Devin FoleyandPaperclip 3d1d5294d9 Drain idle plugin workers before automatic sleep (#15599)
## Thinking Path

> - Paperclip runs autonomous work through agents, schedules, and
plugins.
> - Automatic idle sleep must preserve accepted work and unfinished
cleanup.
> - Every enabled plugin currently blocks sleep, including unused
providers.
> - An enabled flag or package version cannot prove that a worker is
idle.
> - This change adds a live worker drain under the existing owned hold.
> - Idle workers can allow sleep while unknown work stays protected.

## Linked Issues or Issue Description

Refs #15522. Related: #15391 adds plugin readiness for agent admission;
this change concerns instance sleep and does not replace that contract.

**Subsystem affected**

Plugin worker lifecycle and automatic idle sleep.

**Problem or motivation**

A workspace with no pending work cannot sleep when any plugin is
enabled. Removing that check alone would lose accepted RPCs, background
tasks, or cleanup after a caller timeout.

**Proposed solution**

Require a live `onIdleDrain` handshake from each worker. Close admission
in both processes for the exact owner and expiry. Count accepted work
until completion. Continue checking durable work separately.

## What Changed

- Add bounded worker holds, exact-owner release, automatic expiry, and
an abort signal for plugin-owned background work.
- Count host and worker requests through their real completion receipts.
Keep timed-out work counted. Check active notifications and terminal
routes.
- Accept enabled plugins only when their current workers provide
matching runtime receipts. Missing workers, old SDKs, crashes, invalid
replies, and unknown cleanup still prevent sleep.
- Let unused Daytona workers opt in. Once a worker contacts the
provider, it remains a blocker for that process lifetime. This
restriction avoids treating its existing timeout and terminal-close
behavior as a cleanup receipt.
- Document the plugin author contract. No schema or user-facing API is
added.

## Verification

- All hosted CI checks passed on 6ffb682894, including all server,
runner, workspace, build, typecheck, and end-to-end checks. One
unrelated OpenCode transport test hit a five-second timeout and passed
on rerun.
- `pnpm -r typecheck` passed.
- `pnpm build` passed.
- Focused Vitest: 293 tests passed across worker admission, real
child-process RPC, durable idle checks, and Daytona.
- Adjacent worker manager and duplex tests: 119 passed.
- Real child-process tests cover late worker completion, a host write
after caller timeout, stale owners, closed admission, and release.
- SDK tests cover hook failures, busy workers, expiry without another
request, and late acknowledgements after expiry.
- Initial long local run: 17,202 passed, 10 failed. Five failures
involved parent-directory skill fixtures; four of those were already
reproduced and passed in an isolated worktree. One HTTP test had a
socket hang-up; all 66 tests in that file passed on rerun. The other
four used new tests with runtime modules loaded before the follow-up
edits; all pass in the fresh focused run above. A clean build and
full-suite rerun at the final commit are running in an isolated
worktree. No full-suite pass is claimed.
- Follow-up: 44 focused tests passed, including repeated owned holds
without an intervening normal request. Worker status changes now
invalidate an in-progress scan.
- Apex follow-up: 13 idle tests, 119 adjacent worker tests, 71
instance-settings route tests, and server typecheck passed. A real
queued notification completes its write while preparation is in flight;
synchronous and async session callback failures are logged.
- Diff whitespace and added-line secret/private-reference scans passed.

## Risks

- Plugins are trusted code. A hook must account for work outside SDK
RPCs and keep it quiescent until its signal aborts. There is no
package-name or manifest-only exemption.
- This first change targets unused workers. Used Daytona workers remain
awake until remote cleanup receipts are complete. Restarting does not
bypass durable lease and recovery checks.
- Unknown completion remains a blocker, including after a worker crash.
This can retain cost but cannot authorize sleep from a timeout alone.
- Enabled jobs, schedules, credentials, integrations, and other durable
work remain blockers. Durable wake ownership is separate follow-up work.
- Rollback restores the blanket plugin blocker. No data migration is
required.

## Model Used

OpenAI Codex, GPT-6. Used repository inspection, reasoning, code
editing, and command execution. The runtime did not report an exact
model revision or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing public issues or described the issue
in this PR
- [x] I have not referenced internal or instance-local issues or links
- [x] My branch name describes the change and contains no internal
identifiers
- [x] Focused local tests pass; full CI is green (long local rerun
status is recorded above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile Apex is 5/5 on 6ffb682894 with no open findings
- [x] I will address all review comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-08 12:57:32 -07:00

90 lines
4.7 KiB
TypeScript

import { PassThrough } from "node:stream";
import { describe, expect, it } from "vitest";
import { definePlugin, type PluginDefinition } from "../../../packages/plugins/sdk/src/define-plugin.js";
import { startWorkerRpcHost } from "../../../packages/plugins/sdk/src/worker-rpc-host.js";
function fixture(definition: PluginDefinition, rpcTimeoutMs = 30_000) {
const stdin = new PassThrough();
const stdout = new PassThrough();
const worker = startWorkerRpcHost({ plugin: definePlugin(definition), stdin, stdout, rpcTimeoutMs });
const messages: any[] = [];
stdout.on("data", (chunk) => { for (const line of String(chunk).trim().split("\n")) messages.push(JSON.parse(line)); });
let id = 0;
async function call(method: string, params: unknown = {}) {
const callId = ++id;
stdin.write(JSON.stringify({ jsonrpc: "2.0", id: callId, method, params }) + "\n");
await expect.poll(() => messages.find((message) => message.id === callId && !message.method)).toBeDefined();
return messages.find((message) => message.id === callId && !message.method);
}
const initialize = () => call("initialize", { manifest: { id: "test.idle", apiVersion: 1 }, config: {}, apiVersion: 1 });
return { call, initialize, messages, stdin, close: () => worker.stop() };
}
describe("plugin idle RPC", () => {
const hold = () => ({ ownerId: "idle-owner", expiresAt: Date.now() + 30_000 });
it("seals an idle worker, fences stale releases, and restores normal calls", async () => {
const f = fixture({ async setup() {}, async onIdleDrain() { return "none"; } });
try {
expect((await f.initialize()).result.supportedMethods).toContain("prepareIdleSleep");
const lease = hold();
expect((await f.call("prepareIdleSleep", lease)).result).toEqual({ ...lease, backgroundWork: "none" });
expect((await f.call("health")).error).toBeDefined();
await f.call("releaseIdleSleep", { ownerId: "stale" });
expect((await f.call("health")).error).toBeDefined();
await f.call("releaseIdleSleep", lease);
expect((await f.call("health")).result.status).toBe("ok");
} finally { f.close(); }
});
it("does not mistake a caller timeout for a completed worker handler", async () => {
let finish!: () => void;
const f = fixture({ async setup() {}, async onIdleDrain() { return "none"; },
async onHealth() { await new Promise<void>((resolve) => { finish = resolve; }); return { status: "ok" }; } });
try {
await f.initialize();
const accepted = f.call("health");
await expect.poll(() => finish).toBeDefined();
expect((await f.call("prepareIdleSleep", hold())).result.backgroundWork).toBe("present");
finish(); await accepted;
expect((await f.call("prepareIdleSleep", hold())).result.backgroundWork).toBe("none");
} finally { f.close(); }
});
it("retains timed-out worker-to-host writes until the host's late receipt", async () => {
let write!: () => Promise<unknown>;
const f = fixture({ async setup(ctx) { write = () => ctx.state.set({ scopeKind: "instance", stateKey: "fixture" }, { value: 1 }); },
async onIdleDrain() { return "none"; } }, 10);
try {
await f.initialize();
await expect(write()).rejects.toThrow("timed out");
const request = f.messages.find((message) => message.method === "state.set");
expect((await f.call("prepareIdleSleep", hold())).result.backgroundWork).toBe("present");
f.stdin.write(JSON.stringify({ jsonrpc: "2.0", id: request.id, result: null }) + "\n");
expect((await f.call("prepareIdleSleep", hold())).result.backgroundWork).toBe("none");
} finally { f.close(); }
});
it.each([false, true])("logs session callback failures (async: %s)", async (asynchronous) => {
let subscribe!: () => Promise<unknown>;
const f = fixture({ async setup(ctx) {
subscribe = () => ctx.agents.sessions.sendMessage("fixture-session", "fixture-company", {
prompt: "fixture",
onEvent: asynchronous
? async () => { throw new Error("fixture callback failure"); }
: () => { throw new Error("fixture callback failure"); },
});
} });
try {
await f.initialize();
const subscription = subscribe();
const request = f.messages.find((message) => message.method === "agents.sessions.sendMessage");
f.stdin.write(JSON.stringify({ jsonrpc: "2.0", id: request.id, result: {} }) + "\n");
await subscription;
f.stdin.write(JSON.stringify({ jsonrpc: "2.0", method: "agents.sessions.event", params: { sessionId: "fixture-session" } }) + "\n");
await expect.poll(() => f.messages.find((message) => message.method === "log" &&
message.params.level === "error" && message.params.message.includes("fixture callback failure"))).toBeDefined();
} finally { f.close(); }
});
});