mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 08:22:20 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Automatic idle sleep must preserve accepted work, cleanup and recovery. > - Saved AI login connections are configuration without independent workers. > - Failed ephemeral leases can remain after their resources have stopped. > - Detached subscription observations and quota refreshes can outlive a response. > - This pull request distinguishes inert configuration and verified history from unfinished work. > - The benefit is safe idle eligibility while keeping credentials and audit history. ## Linked Issues or Issue Description **What happened?** An active saved AI login blocks idle sleep after its login and agent runs finish. A failed ephemeral lease also blocks after successful cleanup and provider termination. The connection guard currently hides detached quota and subscription work that can outlive the request that started it. **Expected behavior** An idle instance can sleep with saved AI credentials and verified failed-lease history. Accepted refresh work, recovery journals, unfinished login sessions, pending cleanup and run accounting must still block. **Steps to reproduce** 1. Save an AI connection with purpose `ai`, transport `runtime_auth` and status `active`. 2. Finish login and all agent work, then acquire an owned idle drain. 3. Read the idle safety report. The saved connection alone reports background work. 4. Repeat with a failed ephemeral lease whose exact provider termination and finished run are recorded. Its history also reports background work. **Paperclip version or commit** Based on `0bbd586a2f6`, after updates from master. **Deployment mode** Single-process managed or self-hosted server using the owned idle-drain protocol. Related work: #15522, #15761 and #15851. Searched open and closed pull requests for related idle-sleep changes. The open accounting recovery work in #15879 is separate; this change does not settle or discard accounting debt. ## What Changed - Exempt active saved AI runtime-auth configuration from the connection sleep check. Other connection types and unknown states keep their existing checks. - Track detached subscription discovery and provider observations until their actual promises settle. - Track underlying quota work through an HTTP timeout, including an OAuth exchange and the save of its replacement credential. - Inspect the credential-recovery directory without reading credential bytes. Journals, probes, temporary files, unexpected entries and read failures keep sleep blocked. Empty company directories left by successful recovery permit sleep. - Exempt a failed ephemeral lease only after successful cleanup, a release timestamp, an exact company/run/lease/resource termination receipt, and a finished run without retry or accounting debt. Export and cleanup intents still block. - Use the same failed-lease proof when checking finished login-session cleanup. - Add PostgreSQL and filesystem regression tests and document the behavior. ## Verification - Four focused suites passed: 186 tests covering durable guard decisions, local recovery, detached subscription work and quota timeouts. - `pnpm build` passed. - `pnpm -r typecheck` passed. - `pnpm --filter @paperclipai/server build` and a final server TypeScript check passed after the directory-scanner cleanup. - The final directory-scan recheck passed all 18 tests. - The original-base `pnpm test:run` attempt found three local failures. It was stopped before completion to update the branch after two master changes. All three also reproduce on unchanged master `e733bddc45e`: two Git default-branch fixtures in `workspace-runtime.test.ts` and one native runner recovery fixture in `native-session-resume.test.ts`. - Both Git fixtures pass with a command-scoped `init.defaultBranch=master` setting. The native fixture fails with the same result on both branches. No global Git setting was changed. - All CI gates passed on the final head `ff63497d9bef40f5f00567bc729fb17536cfe136`: [complete CI run](https://github.com/paperclipai/paperclip/actions/runs/38156312722). The only skipped checks are the two path-gated Storybook jobs. - Greptile Apex scored the final head 5/5 with no actionable findings and no unresolved review threads. The branch has no merge conflicts with master. - After the master update, `pnpm build` and `pnpm -r typecheck` passed again. All four focused suites passed again: 186 tests. - Staging deployment and live sleep/wake verification are pending the Core merge and the verified private Cloud build. No production rollout has been started. ## Risks - The failed-lease exception is deliberately narrow. Older records without exact termination evidence or a complete run receipt continue to block. - Pending OAuth recovery blocks sleep until it is recovered. No credential, history, cleanup obligation or accounting receipt is deleted. - The existing owned drain, ingress fence, local-work generation checks, plugin drain and backup checkpoint remain required. - No schema migration or provider mutation is introduced. Roll back to the previous application release to restore the conservative guard behavior. ## Model Used OpenAI GPT-6 (Codex), with repository inspection, code editing, test execution and read-only provider analysis. No additional model version or context-window size was supplied by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id or instance-derived details - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no unresolved comments - [x] I will address all reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
83 lines
4.7 KiB
TypeScript
83 lines
4.7 KiB
TypeScript
import { and, eq, lt, or, isNull } from "drizzle-orm";
|
|
import { aiSubscriptions, aiSubscriptionConnections, type Db } from "@paperclipai/db";
|
|
import { aiConnectionService } from "./ai-connections.js";
|
|
import { subscriptionService, type SubscriptionConnection } from "./subscriptions.js";
|
|
import { probeSubscriptionIdentity } from "./subscription-identity.js";
|
|
import { logger } from "../middleware/logger.js";
|
|
import { tooManyRequests } from "../errors.js";
|
|
import { trackIdleWork } from "./task-admission.js";
|
|
|
|
const jobs = new WeakMap<Db, Map<string, Promise<void>>>();
|
|
const discovery = new WeakMap<Db, Map<string, Promise<void>>>();
|
|
const REFRESH_MS = 6 * 60 * 60 * 1000;
|
|
|
|
/** Cached metadata refresh only. Never exchanges tokens or blocks a run/page on a provider. */
|
|
export function refreshSubscriptionConnection(db: Db, input: SubscriptionConnection, id: string, request: typeof fetch = fetch): Promise<void> {
|
|
let active = jobs.get(db);
|
|
if (!active) jobs.set(db, active = new Map());
|
|
const existing = active.get(id);
|
|
if (existing) return existing;
|
|
// Bound provider concurrency across all companies on this instance. Dashboard
|
|
// discovery waits below; opportunistic run observations remain due when busy.
|
|
if (active.size >= 4) return Promise.resolve();
|
|
const task = trackIdleWork((async () => {
|
|
const [claimed] = await db.update(aiSubscriptions).set({ lastCheckedAt: new Date() }).where(and(
|
|
eq(aiSubscriptions.companyId, input.companyId), eq(aiSubscriptions.id, id),
|
|
or(isNull(aiSubscriptions.lastCheckedAt), lt(aiSubscriptions.lastCheckedAt, new Date(Date.now() - REFRESH_MS))),
|
|
)).returning({ id: aiSubscriptions.id });
|
|
if (!claimed) return;
|
|
try {
|
|
const observation = await probeSubscriptionIdentity(input.companyId, input.provider, input.credential, request);
|
|
// A later execution/reconnect may now use another account. Never replace
|
|
// that binding with a slow response for the previous credential.
|
|
const [binding] = await db.select().from(aiSubscriptionConnections).where(and(eq(aiSubscriptionConnections.companyId, input.companyId), eq(aiSubscriptionConnections.grantId, input.grantId)));
|
|
if (binding?.credentialKey !== observation.credentialKey) return;
|
|
await subscriptionService(db).register(input, observation);
|
|
} catch {
|
|
await db.update(aiSubscriptions).set({ refreshStatus: "unavailable" }).where(and(eq(aiSubscriptions.companyId, input.companyId), eq(aiSubscriptions.id, id)));
|
|
}
|
|
})().catch(() => {
|
|
// No raw provider/credential errors are logged or returned to a user.
|
|
logger.warn({ companyId: input.companyId, subscriptionId: id }, "Subscription observation could not be saved");
|
|
}).finally(() => active!.delete(id)));
|
|
active.set(id, task);
|
|
return task;
|
|
}
|
|
|
|
export function refreshCompanySubscriptions(db: Db, companyId: string, userId: string): Promise<void> {
|
|
let active = discovery.get(db);
|
|
if (!active) discovery.set(db, active = new Map());
|
|
const key = `${companyId}:${userId}`;
|
|
const existing = active.get(key);
|
|
if (existing) return existing;
|
|
if (active.size >= 20) throw tooManyRequests("Subscription account checks are busy. Try again shortly.");
|
|
const task = trackIdleWork((async () => {
|
|
const service = aiConnectionService(db);
|
|
// Uses the existing credential audience. Being a company admin does not
|
|
// authorize probing another member's personal credentials.
|
|
for (const row of await service.subscriptionAccounts(companyId, userId)) {
|
|
if (row.summary.status !== "connected") continue;
|
|
try {
|
|
const credential = await service.credential(row);
|
|
const input: SubscriptionConnection = { companyId, connectionId: row.connection.id, grantId: row.grant.id,
|
|
provider: row.summary.provider, name: row.connection.name, ownerUserId: row.grant.subjectUserId, credential };
|
|
const id = await subscriptionService(db).register(input);
|
|
// Accepted discovery must not silently skip an account. At most twenty
|
|
// discovery tasks can wait here; never create an unbounded provider queue.
|
|
// Recheck after each completion because another discovery may take a slot.
|
|
let providers = jobs.get(db);
|
|
while (providers && providers.size >= 4 && !providers.has(id)) {
|
|
await Promise.race(providers.values());
|
|
providers = jobs.get(db);
|
|
}
|
|
await refreshSubscriptionConnection(db, input, id);
|
|
} catch {
|
|
logger.warn({ companyId, connectionId: row.connection.id }, "Subscription discovery unavailable for a connection");
|
|
}
|
|
}
|
|
})().catch(() => logger.warn({ companyId }, "Subscription discovery unavailable"))
|
|
.finally(() => active!.delete(key)));
|
|
active.set(key, task);
|
|
return task;
|
|
}
|