Files
PaperClipAI/server/src/services/agent-hiring.ts
T
DottaandPaperclip 006d5691a6 feat(server): support volume-free web server handoff (#18518)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its web server also owns controller admission and user commands.
> - A rolling deployment starts a replacement before the old server
exits.
> - Shared database state alone does not preserve local agent files,
browser chunks, or in-process commands.
> - This pull request adds an opt-in ephemeral server contract and a
local overlap test.
> - The benefit is a web server that can relinquish traffic and local
files without losing acknowledged work.

## Linked Issues or Issue Description

Refs #18403, #18327, #18335, #18341.

**Current behavior**

The control-plane web server still has local agent-file authority and
local command targets. A new container can miss saved bytes or send a
command to the wrong process. An existing browser tab also needs chunks
from its original build.

**Proposed behavior**

Eligible servers share only PostgreSQL, object storage and stable
secrets. A replacement waits for readiness. The old server drains
admitted requests, keeps owner commands available through handoff, and
releases controller leases. Explicit maintenance adopts existing files
before the volume is removed.

**Reason and benefit**

This is the application prerequisite for rolling app-only deployments.
Schema migrations and permanent multi-replica operation are outside this
change.

## What Changed

- Store agent-file manifests in PostgreSQL and immutable bytes in S3.
Treat local roots as disposable projections. Preserve existing
permission checks and edit conflicts.
- Add an explicit migration, eligibility and object-retention command to
the production runtime bundle.
- Add `/api/ready`, admitted-request drain, browser restart signaling
and retained immutable UI chunks.
- Carry transaction-aware work hints across controllers during handoff
and active execution. Keep listeners closed when idle.
- Deliver durable Stop and runtime responses to the execution owner.
Recheck request lifecycle and access before response dispatch.
- Reject local-only configuration in ephemeral mode. Keep the feature
disabled for existing installations by default.
- Add an isolated PostgreSQL/S3/proxy/browser deployment harness with
independent server homes and retained failure evidence.

## Verification

- All CI checks pass on `ce17b95019d3cf1cabd357d19b6d37f96782cc5c`:
workspace typecheck, build, Runner checks, server/general tests and
browser shards. Greptile reviewed the runtime changes at 5/5 with all
findings resolved; the final commit updates maps and the plan.
- Local targeted checks passed: coordinator 27; Stop/recovery 33;
PostgreSQL command delivery 6; route/workspace regressions 8; bootstrap
Stop 1; restart recovery 75. PostgreSQL command tests simulate the
provider edge.
- Local browser proof passed 91 reads and 19 saves; reconnect took
1,013ms. The tab opened Audit from S3 after deletion of A's whole
home/build. This uses compiled workspace code and local-trusted auth,
not released containers or live agents.
- Local workspace typecheck/build passed; current server and runtime
bundle passed. The earlier broad local suite was stopped after three
hours and is not a passing receipt. Its retained failures and isolated
reruns are documented in the plan.
- Stacked #18526 adds stricter graceful-exit proof, an unready
replacement control, live Runner qualification and a Stop handoff
correction. Hosted enablement remains gated on those results and Cloud
#1096. No hosted stack has been enabled.

## Risks

- Apply migration 0422 before enabling the feature. Existing data must
pass each owner's migration and eligibility checks. Do not discard a
volume based on a single migration page.
- Durable manifests remain authoritative after the opt-in is disabled.
Rolling back to a binary that ignores them is unsupported.
- Cold directory restoration and complete shutdown timing still need
qualification before enabling a hosted stack.
- Retained UI assets must cover the supported browser and rollback
window. Cleanup is explicit; no new idle maintenance polling is
introduced.
- This change does not claim production zero downtime or authorize a
fleet-wide enablement.

## Model Used

OpenAI Codex, GPT-6 family. The exact serving model ID and
context-window limit are not exposed in this session. Used reasoning,
code editing, shell execution and browser automation.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-01 01:58:54 -06:00

159 lines
8.9 KiB
TypeScript

import { assertEphemeralAgentInstructions } from "./agent-instructions.js";
import { initializeDurableAgentFiles } from "./agent-file-durability.js";
import { notifyAttentionRetentionWork } from "./retention-work-notifications.js";
import { notifySkillVersionRetention, notifySkillVersionPinChange } from "./skill-version-retention-notifications.js";
import { agentAppearanceSchema, randomAgentAppearance } from "@paperclipai/shared";
import { normalizePaperclipRunnerAdapterConfig } from "@paperclipai/adapter-utils/server-utils";
import { agents, toolConnectionInstalls, principalPermissionGrants, type Db } from "@paperclipai/db";
import { eq } from "drizzle-orm";
import { unprocessable } from "../errors.js";
import { agentRecordQueries, isPlainRecord, configPatchFromApprovalPayload, normalizeRuntimeConfigForNewAgent,
deduplicateAgentName, type CreateAgentData, type CreateAgentOptions, type AgentConfigurationPatch, type AgentHireRecord } from "../lib/agent-records.js";
import { normalizeAgentPermissions, permissionsImplyLowTrust, NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS,
newStandardAgentGrantScope } from "../lib/agent-permissions.js";
import { readBuiltInAgentMarker } from "../lib/built-in-agent-metadata.js";
import { secretService, assertClaudeOAuthBindingInvariant } from "./secrets.js";
import { agentCredentialService } from "./agent-credentials.js";
import { agentIdentityService } from "./agent-identity.js";
import { initializePrimaryAgent } from "./primary-agent.js";
import { resolveAgentRunnerTaskDriverForCompany, resolveNewAgentRunnerForCompany } from "./agent-runner-selection.js";
export async function prepareAgentHire(db: Db, companyId: string, input: CreateAgentData, options?: CreateAgentOptions): Promise<CreateAgentData> {
await notifyAttentionRetentionWork(db, { companyId });
const { runner, ...data } = input;
const selection = options?.runnerResolved
? { adapterType: data.adapterType ?? "process", adapterConfig: isPlainRecord(data.adapterConfig) ? data.adapterConfig : {} }
: await resolveNewAgentRunnerForCompany(db, companyId, {
...data, adapterConfig: isPlainRecord(data.adapterConfig) ? data.adapterConfig : {}, runner,
});
data.adapterType = selection.adapterType;
data.adapterConfig = selection.adapterConfig;
const { ensureManager, assertBuiltInAgentMetadataMutationAllowed } = agentRecordQueries(db);
if (data.appearance?.customAvatarAssetId) throw unprocessable("Create the agent before uploading its avatar");
assertBuiltInAgentMetadataMutationAllowed(null, data.metadata, options);
if (data.reportsTo) {
await ensureManager(companyId, data.reportsTo);
}
const existingAgents = await db
.select({ id: agents.id, name: agents.name, status: agents.status })
.from(agents)
.where(eq(agents.companyId, companyId));
const uniqueName = deduplicateAgentName(data.name, existingAgents.filter(agent => agent.id !== options?.setupAgentId));
const role = data.role ?? "general";
const normalizedPermissions = normalizeAgentPermissions(data.permissions, { context: "create" });
const runtimeConfig = normalizeRuntimeConfigForNewAgent(data.runtimeConfig);
const adapterType = data.adapterType ?? "process";
const rawAdapterConfig = isPlainRecord(data.adapterConfig)
? await secretService(db).normalizeAdapterConfigForPersistence(companyId, data.adapterConfig, { adapterType })
: {};
const adapterConfig = normalizePaperclipRunnerAdapterConfig(adapterType, rawAdapterConfig);
if (adapterType === "paperclip_runner" && !["openai_dot", "grok_bot", "muse"].includes(String(adapterConfig.provider)) &&
(await resolveAgentRunnerTaskDriverForCompany(db, companyId, data.defaultEnvironmentId))?.warmTaskVersion === 1) {
adapterConfig.lifecycleMode = "warm";
}
await agentCredentialService(db).enforceClaudeOAuthBindingClaim(db, {
companyId, consume: true, environmentId: data.defaultEnvironmentId ?? null,
childAdapterConfig: adapterConfig, claudeLogin: options?.claudeLogin,
decision: assertClaudeOAuthBindingInvariant({ adapterType, nextConfig: adapterConfig }),
});
if (options?.setupAgentId) await notifySkillVersionRetention(db, companyId);
return { ...data, name: uniqueName,
appearance: data.appearance == null ? randomAgentAppearance() : agentAppearanceSchema.parse(data.appearance),
role, adapterType, adapterConfig, permissions: normalizedPermissions, runtimeConfig };
}
export async function initializeAgentHire(txDb: Db, created: AgentHireRecord, options?: CreateAgentOptions) {
const tx = txDb;
const companyId = created.companyId;
assertEphemeralAgentInstructions({ ...created, name: "" });
await initializeDurableAgentFiles(txDb, created);
await agentIdentityService(txDb).ensureAgentIdentity(companyId, created.id);
// New standard agents receive the standard direct grants at activation.
// Low-trust and bundled agents keep their explicit, narrower grants.
if (!options?.setupOnly && created.status !== "pending_approval" && !permissionsImplyLowTrust(created.permissions) &&
!readBuiltInAgentMarker(created.metadata)) {
await tx.insert(principalPermissionGrants).values(
NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS.map((permissionKey) => ({
companyId,
principalType: "agent" as const,
principalId: created.id,
permissionKey,
scope: newStandardAgentGrantScope(permissionKey, created.id),
})),
).onConflictDoNothing();
}
if (options?.aiConnectionInstall) {
const install = options.aiConnectionInstall;
const connectionIds = [...new Set([install.connectionId, ...(install.memberConnectionIds ?? [])])];
await tx.insert(toolConnectionInstalls).values(connectionIds.map(connectionId => ({
companyId, connectionId,
targetType: "agent" as const, targetId: created.id,
createdByUserId: install.createdByUserId,
}))).onConflictDoNothing();
}
await agentCredentialService(txDb).syncAgentSecretBindings(created, txDb);
if (!options?.setupOnly && options?.createdByUserId && !readBuiltInAgentMarker(created.metadata)) {
await initializePrimaryAgent(txDb, companyId, options.createdByUserId, created.id);
}
}
export async function prepareAgentHireApproval(txDb: Db, existing: AgentHireRecord, approvedPayload?: Record<string, unknown> | null): Promise<AgentConfigurationPatch> {
const approvedPatch = approvedPayload ? configPatchFromApprovalPayload(approvedPayload) : {};
const patch = { ...approvedPatch } as AgentConfigurationPatch;
const hasApprovedAdapterConfig = Object.prototype.hasOwnProperty.call(patch, "adapterConfig") && isPlainRecord(patch.adapterConfig);
if (
Object.prototype.hasOwnProperty.call(patch, "adapterConfig") &&
isPlainRecord(patch.adapterConfig)
) {
const normalizedAdapterConfig = await secretService(txDb).normalizeAdapterConfigForPersistence(
existing.companyId, patch.adapterConfig, { adapterType: (patch.adapterType ?? existing.adapterType) as string },
);
patch.adapterConfig = normalizePaperclipRunnerAdapterConfig(
(patch.adapterType ?? existing.adapterType) as string,
normalizedAdapterConfig,
);
} else if (
Object.prototype.hasOwnProperty.call(patch, "adapterType")
&& isPlainRecord(existing.adapterConfig)
) {
patch.adapterConfig = normalizePaperclipRunnerAdapterConfig(
patch.adapterType as string,
existing.adapterConfig,
);
}
if (patch.permissions !== undefined) {
// The pending-approval activation replays the original hire
// request, so the new-agent creation default applies.
patch.permissions = normalizeAgentPermissions(patch.permissions, { context: "create" });
}
if (hasApprovedAdapterConfig) {
await agentCredentialService(txDb).enforceClaudeOAuthBindingClaim(txDb, {
companyId: existing.companyId, consume: false, environmentId: null,
childAdapterConfig: patch.adapterConfig,
decision: assertClaudeOAuthBindingInvariant({ adapterType: patch.adapterType ?? existing.adapterType,
nextConfig: patch.adapterConfig, priorConfig: existing.adapterConfig }),
});
}
if (patch.adapterConfig !== undefined) await notifySkillVersionPinChange(txDb, existing.companyId, existing.adapterConfig ?? {}, patch.adapterConfig);
assertEphemeralAgentInstructions({ ...existing, ...patch, name: "" });
return patch;
}
export async function completeAgentHireApproval(txDb: Db, updated: AgentHireRecord, existing: AgentHireRecord) {
const tx = txDb;
await agentCredentialService(txDb).syncAgentSecretBindings(updated, txDb, existing.adapterConfig);
if (!permissionsImplyLowTrust(updated.permissions) && !readBuiltInAgentMarker(updated.metadata)) {
await tx.insert(principalPermissionGrants).values(
NEW_STANDARD_AGENT_DEFAULT_GRANT_KEYS.map((permissionKey) => ({
companyId: updated.companyId,
principalType: "agent" as const,
principalId: updated.id,
permissionKey,
scope: newStandardAgentGrantScope(permissionKey, updated.id),
})),
).onConflictDoNothing();
}
}