Files
PaperClipAI/server/src/services/idle-local-work.ts
T
Devin FoleyandPaperclip 4b59e2a2ec fix(server): allow idle sleep with saved AI auth and verified lease history (#15897)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Automatic idle sleep must preserve accepted work, cleanup and
recovery.
> - Saved AI login connections are configuration without independent
workers.
> - Failed ephemeral leases can remain after their resources have
stopped.
> - Detached subscription observations and quota refreshes can outlive a
response.
> - This pull request distinguishes inert configuration and verified
history from unfinished work.
> - The benefit is safe idle eligibility while keeping credentials and
audit history.

## Linked Issues or Issue Description

**What happened?**

An active saved AI login blocks idle sleep after its login and agent
runs finish. A failed ephemeral lease also blocks after successful
cleanup and provider termination. The connection guard currently hides
detached quota and subscription work that can outlive the request that
started it.

**Expected behavior**

An idle instance can sleep with saved AI credentials and verified
failed-lease history. Accepted refresh work, recovery journals,
unfinished login sessions, pending cleanup and run accounting must still
block.

**Steps to reproduce**

1. Save an AI connection with purpose `ai`, transport `runtime_auth` and
status `active`.
2. Finish login and all agent work, then acquire an owned idle drain.
3. Read the idle safety report. The saved connection alone reports
background work.
4. Repeat with a failed ephemeral lease whose exact provider termination
and finished run are recorded. Its history also reports background work.

**Paperclip version or commit**

Based on `0bbd586a2f6`, after updates from master.

**Deployment mode**

Single-process managed or self-hosted server using the owned idle-drain
protocol.

Related work: #15522, #15761 and #15851. Searched open and closed pull
requests for related idle-sleep changes. The open accounting recovery
work in #15879 is separate; this change does not settle or discard
accounting debt.

## What Changed

- Exempt active saved AI runtime-auth configuration from the connection
sleep check. Other connection types and unknown states keep their
existing checks.
- Track detached subscription discovery and provider observations until
their actual promises settle.
- Track underlying quota work through an HTTP timeout, including an
OAuth exchange and the save of its replacement credential.
- Inspect the credential-recovery directory without reading credential
bytes. Journals, probes, temporary files, unexpected entries and read
failures keep sleep blocked. Empty company directories left by
successful recovery permit sleep.
- Exempt a failed ephemeral lease only after successful cleanup, a
release timestamp, an exact company/run/lease/resource termination
receipt, and a finished run without retry or accounting debt. Export and
cleanup intents still block.
- Use the same failed-lease proof when checking finished login-session
cleanup.
- Add PostgreSQL and filesystem regression tests and document the
behavior.

## Verification

- Four focused suites passed: 186 tests covering durable guard
decisions, local recovery, detached subscription work and quota
timeouts.
- `pnpm build` passed.
- `pnpm -r typecheck` passed.
- `pnpm --filter @paperclipai/server build` and a final server
TypeScript check passed after the directory-scanner cleanup.
- The final directory-scan recheck passed all 18 tests.
- The original-base `pnpm test:run` attempt found three local failures.
It was stopped before completion to update the branch after two master
changes. All three also reproduce on unchanged master `e733bddc45e`: two
Git default-branch fixtures in `workspace-runtime.test.ts` and one
native runner recovery fixture in `native-session-resume.test.ts`.
- Both Git fixtures pass with a command-scoped
`init.defaultBranch=master` setting. The native fixture fails with the
same result on both branches. No global Git setting was changed.
- All CI gates passed on the final head
`ff63497d9bef40f5f00567bc729fb17536cfe136`: [complete CI
run](https://github.com/paperclipai/paperclip/actions/runs/38156312722).
The only skipped checks are the two path-gated Storybook jobs.
- Greptile Apex scored the final head 5/5 with no actionable findings
and no unresolved review threads. The branch has no merge conflicts with
master.
- After the master update, `pnpm build` and `pnpm -r typecheck` passed
again. All four focused suites passed again: 186 tests.
- Staging deployment and live sleep/wake verification are pending the
Core merge and the verified private Cloud build. No production rollout
has been started.

## Risks

- The failed-lease exception is deliberately narrow. Older records
without exact termination evidence or a complete run receipt continue to
block.
- Pending OAuth recovery blocks sleep until it is recovered. No
credential, history, cleanup obligation or accounting receipt is
deleted.
- The existing owned drain, ingress fence, local-work generation checks,
plugin drain and backup checkpoint remain required.
- No schema migration or provider mutation is introduced. Roll back to
the previous application release to restore the conservative guard
behavior.

## Model Used

OpenAI GPT-6 (Codex), with repository inspection, code editing, test
execution and read-only provider analysis. No additional model version
or context-window size was supplied by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
following the relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
ticket id or instance-derived details
- [x] I have run focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no unresolved comments
- [x] I will address all reviewer comments before requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-11 10:01:57 -07:00

72 lines
3.6 KiB
TypeScript

import { promises as fs } from "node:fs";
import path from "node:path";
import { resolvePaperclipInstanceRoot } from "../home-paths.js";
import { idleWorkSnapshot } from "./task-admission.js";
export type IdleLocalWork = "none" | "present" | "unknown";
const spoolDirectories = new Set<string>();
export function registerIdleSpoolDirectory(directory: string): void { spoolDirectories.add(path.resolve(directory)); }
export const idleAccountingSpoolPath = () => path.join(resolvePaperclipInstanceRoot(), "accounting-receipts");
export const idleOrphanSpoolPath = () => process.env.SANDBOX_ORPHAN_CLEANUP_SPOOL_DIR ??
path.join(resolvePaperclipInstanceRoot(), "data", "sandbox-orphan-cleanup");
export const idleQuotaRecoveryPath = () => path.join(resolvePaperclipInstanceRoot(), "quota-credential-recovery");
let startupComplete = false;
let ingressTracked = false;
let scheduledBackups = false;
export function markIdleIngressTracked() { ingressTracked = true; }
export function markIdleStartupComplete(options: { scheduledBackups: boolean }) {
scheduledBackups = options.scheduledBackups;
startupComplete = true;
}
/** Any entry, including malformed JSON, a temp file or a failed probe, is
* evidence to retain. Recovery readers intentionally skip some such files;
* they cannot certify that stopping the process is safe. */
export async function inspectIdleSpool(directory: string): Promise<IdleLocalWork> {
try {
const info = await fs.lstat(directory);
if (!info.isDirectory() || info.isSymbolicLink()) return "unknown";
const dir = await fs.opendir(directory);
try { return (await dir.read()) === null ? "none" : "present"; }
finally { await dir.close(); }
} catch (error) {
return (error as NodeJS.ErrnoException).code === "ENOENT" ? "none" : "unknown";
}
}
/** Credential journals are grouped by company. Successful recovery leaves its
* empty company directory behind; every file, probe or unexpected entry still
* blocks. Never decrypt journals or follow directory symlinks during a scan. */
export async function inspectIdleQuotaRecovery(directory: string): Promise<IdleLocalWork> {
try {
const info = await fs.lstat(directory);
if (!info.isDirectory() || info.isSymbolicLink()) return "unknown";
const dir = await fs.opendir(directory);
try {
for (let entry = await dir.read(); entry !== null; entry = await dir.read()) {
if (!entry.isDirectory() || entry.isSymbolicLink()
|| !/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(entry.name)) return "present";
const state = await inspectIdleSpool(path.join(directory, entry.name));
if (state !== "none") return state;
}
return "none";
} finally { await dir.close(); }
} catch (error) {
return (error as NodeJS.ErrnoException).code === "ENOENT" ? "none" : "unknown";
}
}
export async function readIdleLocalWork(options: { backupCheckpoint?: boolean } = {}): Promise<IdleLocalWork> {
if (!startupComplete || !ingressTracked) return "unknown";
// This exemption belongs only to a scan which will finish a new backup
// under the same admission hold before it can return an empty report.
if ((scheduledBackups && !options.backupCheckpoint) || idleWorkSnapshot().active !== 0) return "present";
const directories = new Set([...spoolDirectories, idleAccountingSpoolPath(), idleOrphanSpoolPath()]);
const results = await Promise.all([
...[...directories].map(inspectIdleSpool),
inspectIdleQuotaRecovery(idleQuotaRecoveryPath()),
]);
if (results.includes("unknown")) return "unknown";
return results.includes("present") ? "present" : "none";
}