Files
PaperClipAI/server/src/services/onboarding-seed.ts
T
Nicky LeachandPaperclip 2dd9811e8b Own agent hiring and termination in a lifecycle module (#15631)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - Agent creation, approval, pause, and termination change whether an
agent can work.
> - These changes currently use separate service and database paths.
> - Some agents need setup after their record exists and before their
harness can run.
> - This PR adds one module that owns those state changes and their
transactions.
> - The module records progress and retries incomplete work after a
server restart.

## Linked Issues or Issue Description

**Problem or motivation**

A hire can become active before its execution environment is ready.
Direct agent state writes also bypass one common lifecycle boundary.
Termination does not have a durable step for resource cleanup.

**Proposed solution**

Use agent lifecycle commands for hiring, approval, pause, resume, and
termination. Keep proposed hires pending until approval. Prepare each
approved hire, test its saved harness configuration, and then permit
work. Wait for required plugin work before completing a step.

**Alternatives considered**

A general database commit hook would expose transaction details to
callers. A timer alone would delay each new hire. This module owns each
command transaction and starts its worker after commit. A periodic scan
recovers work after a process stops.

**Roadmap alignment**

This change adds the lifecycle boundary for the existing agent and
plugin systems. It does not add a user interface or a provider
implementation. It replaces the lifecycle delivery approach in #15391.
Related proposals include #11475 and #13099; this change does not add
their external executor protocols.

## What Changed

- Add durable approval, preparation, verification, pause, resume, stop,
and cleanup states.
- Keep agent creation, lifecycle state, configuration writes, and agent
record deletion behind `modules/agent-lifecycle`. Keep validation,
credential rules, reads, onboarding, invitations, general approvals, and
company data deletion in their services.
- Commit a hire before the remaining onboarding or invitation work.
Serialize requests with the existing dedicated database connection
helper. Reuse the hire after a failed attempt.
- Keep accepted work in the run queue during setup. Require execution
and cleanup evidence before completing pause or termination.
- Retain the original credential owner when approving migrated hires.
- Start lifecycle work after each command commits. Use database leases
and revision checks for retries and late results.
- Add a generic required plugin method and capability. Store the
required plugin IDs for each agent. Track host and plugin completion
separately.
- Keep harness verification and credential rules in services. Supply
their results to lifecycle through explicit ports. Reject failed logins
and incomplete probes.
- Remove reverse imports from lifecycle adapters into services. Share
record queries and validation below both layers.
- Use one hire decision path for agent IDs and approval IDs. Resolve the
approval and agent change in one transaction. Expose policy
reconciliation on the command object.
- Save configuration and verification invalidation through one module
operation. Ordinary commands own their transaction. Restrict the shared
transaction integration to its listed callers. Reject identity, company,
accounting, timestamp, and unknown fields.
- Require completed termination before deleting agent records. The
company deletion service calls the module through a separate deletion
interface.
- Save an approved secret before applying its agent binding. A failed
binding leaves the secret approved. A retry reuses it.
- Remove old permission and metadata export files. Move hire
configuration rules out of lifecycle persistence.
- Report failed worker steps and scans with operation context and known
error codes. Omit provider messages, stacks, and configuration values.
Document ordinary application use and extension rules.
- Recover expired native owners only after execution stops. Retry
cleanup of failed retained leases. Wait for setup in the live smoke
script.
- Keep manual, budget, and company pause reasons separate. Guard
execution status writes with the ready state.
- Add a migration, boundary checks, regression tests, and a lifecycle
guide. Keep the current UI.
- Limit periodic policy checks to 100 eligible agents per minute.
Continue from the last agent ID. Exclude pending hires and agents in
termination or a final state. Keep explicit company and budget updates
immediate.

## Verification

- Latest head `ccc3c86b0` aligns the shared status update schema with
the PATCH route. All 15 shared tests, 104 server route/OpenAPI tests, 2
CLI tests, and shared/server/CLI TypeScript checks pass. The Greptile
comment is fixed. CI is running; no new Apex review was requested for
this small fix.

- Rebase `75708ac94` rebases onto master at `2eb5187e4`. The backup test
uses the shared 90-second timeout from master; the older timeout commit
was removed. The affected backup test passes.
- Agent budget edits now pass the target agent ID through enforcement.
All 50 affected tests, server TypeScript, and 11 boundary tests pass.
Both configuration update paths have regression coverage. CI is running
on the latest head. No new Apex review was requested for these small
fixes.

- Rebase `9b568666a` preserves the run-control extraction and
delivery-work startup changes from master. All 559 affected tests,
server TypeScript, and 11 boundary tests pass. CI and Apex review are
running on the final head.
- The policy scan fix passes 56 lifecycle and diagnostic tests, 24
policy and caller tests, and 11 boundary tests. A regression test checks
the page limit, interval, excluded states, and cursor wraparound.

- Rebase `fd4698750` passes all 611 tests in eight affected lifecycle,
route, and heartbeat suites. Server TypeScript and all 11 module
boundary tests pass. The company deletion rollback and concurrent hire
tests are included.

- Follow-up `00aac7646` fixes the failed decision deletion test and the
public update schema. All 126 affected server tests, 8 shared tests, 2
CLI tests, and 11 boundary tests pass. Shared, server, CLI, and UI
TypeScript checks pass.

- Server TypeScript passes: `pnpm --filter @paperclipai/server exec tsc
--noEmit`.
- All 11 boundary tests pass: `node --test
scripts/check-module-boundaries.test.mjs`. The repository boundary scan
also passes.
- All 68 lifecycle, failure diagnostic, and harness verification tests
pass. Tests reject forbidden fields through the root command, combined
transition command, and restricted transaction integration. They also
check invalid fields returned by configuration preparation.
- Failure tests cover host work, plugin work, claims, renewals,
deferrals, and scans. Log tests check known error codes and exclusion of
private error contents.
- All 355 tests in 10 existing caller suites pass. These tests cover
secret approval, pairing, pending hires, credentials, company import,
built-in agents, managed agents, instructions, and skills.
- The branch is rebased onto `origin/master` at `2eb5187e4`. The queue
guards and cancellation lock fix use the extracted heartbeat queue.
Agent cancellation uses the extracted run-control file. The source diff
scan found no secrets or private URLs.
- Earlier checks cover approval retries, rollback, late verification
results, policy holds, deletion rollback, concurrent company deletion,
and queue admission during setup.
- Before the rebase, CI passed on `00aac7646` and Greptile Apex rated it
5/5 with no open comments. CI and a new Apex review are pending for the
rebased commit. The branch has no merge conflicts.
- The full local build and native Dot suite were not run. The host has
no Cargo. No live provider test was run.

## Risks

- Background policy recovery takes more than one minute when more than
100 agents are eligible. Explicit company and budget updates still
reconcile immediately.

- The restricted configuration integration keeps an agent reference and
its dependent records in one transaction. It cannot change a budget or
invoke a lifecycle transition. A periodic scan starts pending setup
after that transaction commits.
- Secret approval and binding use separate transactions. A failed
binding can leave an approved secret with no agent binding.
- New approved hires remain unavailable until setup and verification
complete. Existing clients see `paused` during these steps. This PR does
not change the UI.
- Migration `0320_daily_onslaught.sql` maps existing statuses and pause
reasons. It does not create external resources or claim that an existing
harness passed verification.
- Use a coordinated server update for this migration. Older server
versions can write legacy status without the new lifecycle checks.
- A failed onboarding or invitation request can leave a committed hire.
A retry reuses that hire and completes the remaining records.
- Deletion requires completed termination. Company deletion can require
another request while cleanup is pending.
- Required plugins must make repeated calls safe and reject older
revisions. A disabled or removed required plugin blocks completion until
it is restored.
- A new hire can run a second harness test after the existing UI test.
Some adapters can charge for this test.
- No live provider test was run. Provider implementation and resource
backfill remain separate work.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, code tools, and local test
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in this PR with the feature request
field labels
- [x] I have not referenced internal or instance-local issues or links
- [x] My branch name describes the change
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] Targeted local tests pass
- [x] All Paperclip CI gates are green on the new commit
- [x] Greptile Apex is 5/5 with no open comments on the new commit
- [x] I will address review comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-09 09:05:12 -07:00

406 lines
15 KiB
TypeScript

import { and, eq, ne, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { agents, companyOnboardingSeeds, goals, issues, projects } from "@paperclipai/db";
import type { ApplyOnboardingSeed } from "@paperclipai/shared";
import { writePaperclipSkillSyncPreference } from "@paperclipai/adapter-utils/server-utils";
import { findActiveServerAdapter } from "../adapters/registry.js";
import { agentService } from "./agents.js";
import { createAgentLifecycle, scheduleAgentLifecycle } from "./agent-lifecycle.js";
import { withDedicatedDbConnection } from "@paperclipai/db";
import { PAPERCLIP_CORE_SKILL_KEYS } from "./company-skills.js";
import { goalService } from "./goals.js";
import { projectService } from "./projects.js";
import { issueService } from "./issues.js";
import { readBuiltInAgentMarker } from "../lib/built-in-agent-metadata.js";
import { logActivity, publishActivity, type ActivityPublication } from "./activity-log.js";
/**
* The project the seeded first task lands in, matching the name the tenant's
* own first-run wizard uses so a later manual run reuses it instead of
* creating a second "Onboarding" project.
*/
export const ONBOARDING_SEED_PROJECT_NAME = "Onboarding";
/**
* Role assigned to the seeded lead agent. The seed's own `agent.role` is
* customer free text ("Chief of Staff") and lands on `title`; `role` stays the
* structural `ceo` key the org chart and default-instructions lookup read.
*/
const SEEDED_AGENT_ROLE = "ceo";
/**
* Adapter the seeded agent is created with. Mirrors the teams-catalog default
* (`claude_local`), which is the safe adapter for agents created server-side
* without a human running an environment test first.
*/
const FALLBACK_SEEDED_AGENT_ADAPTER_TYPE = "claude_local";
function seededAgentAdapterType() {
const configured = process.env.PAPERCLIP_ONBOARDING_SEED_ADAPTER_TYPE?.trim()
|| process.env.PAPERCLIP_TEAMS_CATALOG_DEFAULT_ADAPTER_TYPE?.trim()
|| FALLBACK_SEEDED_AGENT_ADAPTER_TYPE;
// Server-seeded onboarding deliberately stays on a direct adapter. Native
// runner rollout is an explicit post-onboarding configuration choice.
return configured === "paperclip_runner"
? FALLBACK_SEEDED_AGENT_ADAPTER_TYPE
: configured;
}
/**
* Adapter config for the seeded CEO. The default CEO instructions tell the
* agent to use the core paperclip skills (hiring, memory, coordination), and
* an agent's runtime only receives skills listed in its own desired set — so
* a seeded CEO with an empty adapter config arrives with zero skills and
* truthfully reports its own toolkit as not installed. Enable the core set
* whenever the seeded adapter supports skill sync.
*/
function seededAgentAdapterConfig(adapterType: string): Record<string, unknown> {
const adapter = findActiveServerAdapter(adapterType);
if (!adapter?.listSkills && !adapter?.syncSkills) return {};
return writePaperclipSkillSyncPreference(
{},
PAPERCLIP_CORE_SKILL_KEYS.map((key) => ({ key, versionId: null })),
);
}
/**
* Split a free-text mission into a goal title + description the same way the
* first-run wizard's `parseOnboardingGoalInput` does: first line is the title,
* the remainder is the description.
*/
export function parseSeedMission(raw: string): { title: string; description: string | null } {
const trimmed = raw.trim();
if (!trimmed) return { title: "", description: null };
const [firstLine, ...restLines] = trimmed.split(/\r?\n/);
const description = restLines.join("\n").trim();
return {
title: (firstLine ?? "").trim(),
description: description.length > 0 ? description : null,
};
}
export type OnboardingSeedApplication = {
revision: string;
/** False when the stored revision already matched and nothing was re-applied. */
changed: boolean;
goalId: string | null;
agentId: string | null;
issueId: string | null;
};
/**
* The actor fields the audit entry needs, as `getActorInfo` produces them.
* Narrowed to what {@link LogActivityInput} reads so the route can hand its
* actor straight through without the service depending on Express.
*/
export type OnboardingSeedAuditActor = {
actorType: "agent" | "user" | "system" | "plugin";
actorId: string;
agentId?: string | null;
runId?: string | null;
agentApiKeyId?: string | null;
};
export function onboardingSeedService(db: Db) {
async function readRecord(dbx: Db, companyId: string) {
return dbx
.select()
.from(companyOnboardingSeeds)
.where(eq(companyOnboardingSeeds.companyId, companyId))
.then((rows) => rows[0] ?? null);
}
async function goalStillExists(dbx: Db, companyId: string, goalId: string | null) {
if (!goalId) return false;
return dbx
.select({ id: goals.id })
.from(goals)
.where(and(eq(goals.id, goalId), eq(goals.companyId, companyId)))
.then((rows) => rows.length > 0);
}
/**
* The agent a re-push should update rather than duplicate: the one this
* seed created if it is still around, else a pre-existing lead the tenant
* already has. Built-in agents are excluded — they are provisioned by the
* platform and are not the customer's first hire.
*/
async function resolveTargetAgentId(dbx: Db, companyId: string, recordedAgentId: string | null) {
if (recordedAgentId) {
const recorded = await dbx
.select({ id: agents.id })
.from(agents)
.where(and(eq(agents.id, recordedAgentId), eq(agents.companyId, companyId)))
.then((rows) => rows[0] ?? null);
if (recorded) return recorded.id;
}
const candidates = await dbx
.select({ id: agents.id, metadata: agents.metadata })
.from(agents)
.where(and(
eq(agents.companyId, companyId),
eq(agents.role, SEEDED_AGENT_ROLE),
ne(agents.status, "terminated"),
));
return candidates.find((row) => !readBuiltInAgentMarker(row.metadata))?.id ?? null;
}
async function issueStillExists(dbx: Db, companyId: string, issueId: string | null) {
if (!issueId) return false;
return dbx
.select({ id: issues.id })
.from(issues)
.where(and(eq(issues.id, issueId), eq(issues.companyId, companyId)))
.then((rows) => rows.length > 0);
}
async function resolveOnboardingProjectId(
dbx: Db,
projectSvc: ReturnType<typeof projectService>,
companyId: string,
goalId: string | null,
) {
const existing = await dbx
.select({ id: projects.id, name: projects.name, status: projects.status })
.from(projects)
.where(eq(projects.companyId, companyId));
const reusable = existing.find(
(project) =>
project.status !== "cancelled"
&& project.name.trim().toLowerCase() === ONBOARDING_SEED_PROJECT_NAME.toLowerCase(),
);
if (reusable) return reusable.id;
const created = await projectSvc.create(companyId, {
name: ONBOARDING_SEED_PROJECT_NAME,
status: "in_progress",
...(goalId ? { goalIds: [goalId] } : {}),
});
return created.id;
}
async function prepareAgent(dbx: Db, companyId: string, seed: ApplyOnboardingSeed, audit?: OnboardingSeedAuditActor) {
const existing = await readRecord(dbx, companyId);
if (existing?.revision === seed.revision) return existing.agentId;
const agentSvc = agentService(dbx);
const agentName = seed.agent?.name.trim() || null;
const agentRole = seed.agent?.role?.trim() || null;
// 2. Agent → the customer's first hire, the lead the first task is
// assigned to.
let agentId = await resolveTargetAgentId(dbx, companyId, existing?.agentId ?? null);
if (agentName) {
if (agentId) {
await agentSvc.update(agentId, { name: agentName, title: agentRole });
} else {
const adapterType = seededAgentAdapterType();
const created = await createAgentLifecycle(dbx).requestHire(companyId, {
name: agentName,
role: SEEDED_AGENT_ROLE,
title: agentRole,
adapterType,
adapterConfig: seededAgentAdapterConfig(adapterType),
runtimeConfig: {},
permissions: {},
status: "idle",
spentMonthlyCents: 0,
lastHeartbeatAt: null,
}, { createdByUserId: audit?.actorType === "user" ? audit.actorId : null });
agentId = created.id;
}
}
return agentId;
}
async function applyWithin(
dbx: Db,
companyId: string,
seed: ApplyOnboardingSeed,
preparedAgentId: string | null,
): Promise<OnboardingSeedApplication> {
const goalSvc = goalService(dbx);
const projectSvc = projectService(dbx);
const issueSvc = issueService(dbx);
const existing = await readRecord(dbx, companyId);
if (existing && existing.revision === seed.revision) {
return {
revision: existing.revision,
changed: false,
goalId: existing.goalId,
agentId: existing.agentId,
issueId: existing.issueId,
};
}
const mission = seed.mission?.trim() || null;
const agentName = seed.agent?.name.trim() || null;
const agentRole = seed.agent?.role?.trim() || null;
const firstTaskTitle = seed.firstTask?.title.trim() || null;
const firstTaskDetails = seed.firstTask?.details?.trim() || null;
// 1. Mission → the company-level goal the dashboard reads.
let goalId = existing?.goalId ?? null;
if (mission) {
const parsed = parseSeedMission(mission);
const target = (await goalStillExists(dbx, companyId, goalId))
? goalId
: (await goalSvc.getDefaultCompanyGoal(companyId))?.id ?? null;
if (target) {
await goalSvc.update(target, {
title: parsed.title,
description: parsed.description,
});
goalId = target;
} else {
const created = await goalSvc.create(companyId, {
title: parsed.title,
description: parsed.description,
level: "company",
status: "active",
});
goalId = created.id;
}
}
const agentId = preparedAgentId;
// A mission-only seed leaves the first task to the onboarding route.
let issueId = existing?.issueId ?? null;
if (firstTaskTitle) {
if (await issueStillExists(dbx, companyId, issueId)) {
await issueSvc.update(
issueId as string,
{
title: firstTaskTitle,
description: firstTaskDetails,
// Keep the task's relationships in step with a later revision that
// supplied the agent or goal after the task already existed —
// otherwise the record would report an assignee/goal the issue row
// does not actually carry. Only set them when resolved, so an
// absent value never clears an assignment the tenant made.
...(agentId ? { assigneeAgentId: agentId } : {}),
...(goalId ? { goalId } : {}),
},
dbx,
);
} else {
const projectId = await resolveOnboardingProjectId(dbx, projectSvc, companyId, goalId);
// The idempotency key is what protects two pushes that arrive at once
// — Cloud's reconcile runs off portfolio fetches, which can overlap.
// It is deliberately not revision-scoped: if the recorded issue is
// lost, a later revision should still dedupe against whatever the
// first push created.
const created = await issueSvc.create(companyId, {
title: firstTaskTitle,
...(firstTaskDetails ? { description: firstTaskDetails } : {}),
...(agentId ? { assigneeAgentId: agentId } : {}),
projectId,
...(goalId ? { goalId } : {}),
status: "todo",
idempotencyKey: `onboarding-seed:${companyId}`,
});
issueId = created.id;
}
}
// 4. Record the revision last. Everything above has to have landed before
// this row claims the seed is applied.
const now = new Date();
const values = {
companyId,
revision: seed.revision,
mission,
agentName,
agentRole,
firstTaskTitle,
firstTaskDetails,
goalId,
agentId,
issueId,
appliedAt: now,
updatedAt: now,
};
await dbx
.insert(companyOnboardingSeeds)
.values(values)
.onConflictDoUpdate({
target: companyOnboardingSeeds.companyId,
set: {
revision: values.revision,
mission: values.mission,
agentName: values.agentName,
agentRole: values.agentRole,
firstTaskTitle: values.firstTaskTitle,
firstTaskDetails: values.firstTaskDetails,
goalId: values.goalId,
agentId: values.agentId,
issueId: values.issueId,
appliedAt: values.appliedAt,
updatedAt: values.updatedAt,
},
});
return { revision: seed.revision, changed: true, goalId, agentId, issueId };
}
// Hire first. If the content transaction fails, a retry reuses the committed agent.
// The dedicated transaction holds only the workflow lock, not the hire writes.
async function apply(
companyId: string,
seed: ApplyOnboardingSeed,
audit?: OnboardingSeedAuditActor,
): Promise<OnboardingSeedApplication> {
// Collected inside the transaction, published only after it commits: the
// activity row is transactional but its realtime/plugin fan-out is not, and
// announcing a seed that then rolled back would be worse than announcing it
// late.
const publications: ActivityPublication[] = [];
const database = db;
const lockKey = `paperclip:onboarding-seed:${companyId}`;
return withDedicatedDbConnection(db, lockDb => lockDb.transaction(async lock => {
await lock.execute(sql`select pg_advisory_xact_lock(hashtextextended(${lockKey}, 0))`);
const preparedAgentId = await prepareAgent(database, companyId, seed, audit);
const result = await database.transaction(async (tx) => {
const dbx = tx as unknown as Db;
const applied = await applyWithin(dbx, companyId, seed, preparedAgentId);
if (applied.changed && audit) {
await logActivity(
dbx,
{
companyId,
actorType: audit.actorType,
actorId: audit.actorId,
agentId: audit.agentId,
runId: audit.runId,
agentApiKeyId: audit.agentApiKeyId,
action: "company.onboarding_seed_applied",
entityType: "company",
entityId: companyId,
details: {
revision: applied.revision,
goalId: applied.goalId,
agentId: applied.agentId,
issueId: applied.issueId,
},
},
publications,
);
}
return applied;
});
for (const publication of publications) publishActivity(publication);
if (result.agentId) scheduleAgentLifecycle(db, result.agentId);
return result;
}));
}
return { apply, get: (companyId: string) => readRecord(db, companyId) };
}