mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 09:37:19 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connected chats show safe notices when a run starts, waits, finishes, or fails. > - A recurring scan projected these notices even when no eligible work existed. > - Eligibility depends on committed run state, conversation lineage, and current authority. > - Existing database rows can reconstruct that work after restart. > - This PR wakes a scoped worker from those writes and removes lifecycle polling. > - Native progress events keep their separate existing owner until the next migration. ## Linked Issues or Issue Description **What happened?** The chat maintenance timer repeatedly scanned run lifecycle state. Empty and settled instances still queried the database. A run-status event alone cannot replace the scan: prompt receipts, continuation receipts, identity changes, and restored authority can make a run eligible later. **What did you expect?** Committed changes wake projection. Startup reconstructs eligible work. A completed pass stops until another relevant change arrives. Failed work retains recovery without scanning settled history forever. **Steps to reproduce** Configure an automatic chat endpoint. Finish its work, or leave it unused. Observe recurring lifecycle projection queries from the chat maintenance callback. Stacked on #17592. Related foundation: #15625. This PR owns lifecycle milestones only. ## What Changed - Preserve upstream coalesced Cloud membership writes together with locked membership restoration and its committed chat notification. - Signal milestone eligibility from the real Slack account-link transaction. - Register the lifecycle worker with the existing delivery coordinator. Remove lifecycle projection from the periodic chat callback. Keep native progress in a separately named lane and preserve the explicit combined API. - Reconstruct company/issue scopes from the existing selector. Drain bounded batches with company fairness. Retry failures, retain writes through drain/resume, and disarm settled work. - Signal from supported run, presentation, interaction, receipt, lineage, conversation, recovery, and authority writers. Preserve their transaction ownership, including nested savepoints and uncertain settlement. No new table is added. - Preserve provider authorization, authored-final suppression, responsible-user continuity, deduplication, the 24-hour selection window, and raw-output restrictions. Keep provider interpretation in its owner. - Preserve PostgreSQL timestamp precision in lifecycle pagination. Rejected submillisecond candidates must not repeat the same page and starve later notices. - Document producer coverage and alternate-writer dispositions in `doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`. ## Verification Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting master after #17592 merged at `7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates the actual landed parent. The entire 77-path milestone patch is binary-identical to the locally verified preparation across that ancestry reconciliation; both unrelated master paths are preserved. The four earlier producer/test integration conflicts retain scoped milestone and voice notices, current onboarding completion admission, and expanded identity query/read-scope coverage. - **330 distinct selected cases pass on the verified unchanged patch:** 305 across 14 complete suites (all five milestone worker suites, run identity, board claim, Cloud provisioning, auth session routes, Cloud actor authentication, stale-lock recovery, GitHub credentials, project Git authorization, and app admission wiring), plus 25 selected real chat producer cases. The remaining 1,590 chat integration cases were skipped, not passed. - The real producer cases cover saved milestone discovery, GitHub and Slack identity changes, account-link rollback/reconfirmation, ordinary ingress reactivation, post-close action/delivery receipt settlement, all ten authority-restoration paths including Cloud synchronization, and changed native review metadata. - Worker coverage verifies outer commit, rollback, uncertain-write recovery, restart, bounded batching/fairness, timestamp precision, retries, drain/resume, and settled SQL quiet. Authorization and existing publication deduplication remain with their owning services. - Server production typechecking, build, and module-boundary checks pass. The tests above were executed separately. Production typechecking excludes `src/__tests__` and does not replace those runs. Each database owner uses its own newly created and app-migrated PostgreSQL fixture. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28) records commands, counts, producer scope, conflict dispositions, and limits. Earlier combined-stack counts and failures keep their original revision scope; local acceptance does not erase old CI failures. - No new migration, queue table, dependency, workflow, or shared scheduler contract is introduced. Parent migration 0397, bounded consent recovery, voice cleanup/retention repairs, and runner checkpoint restoration remain present. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing review is waived. Older reviews and parent checks are not current passes. Actual failing Ponytail findings must be fixed; an absent/stopped/paused turn without a verdict is an authorized nonblocker. Active unfinished Ponytail uses its original request/start timer, with a 25-minute monitor cutoff respecting the user's 30-minute maximum. - No complete local repository matrix, live provider tenant, external alarm, physical host/database suspension, or fleet savings claim is made. Native progress remains a separate following slice. ## Risks - A future eligibility writer must notify the existing projection boundary. Independent SQL writers or another active server process are outside this process-local signal contract. - Authority and lineage remain sensitive. The worker uses the existing selectors and publication policy; it does not grant execution or disclosure authority. - Native progress and other publication maintenance still have recurring work. This PR removes one reason, not the entire chat maintenance timer. - Worker quiet does not prove database suspension, host sleep, or external alarm coverage. Cloud sleep blockers remain unchanged. - Merge eligibility requires passing current-head CI and Greptile 5/5 after this actual-master reconciliation. Subsequent stack slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window size are not exposed. Used repository inspection, reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the documented focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
339 lines
11 KiB
TypeScript
339 lines
11 KiB
TypeScript
import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js";
|
|
import { createHash } from "node:crypto";
|
|
import { afterAll, afterEach, beforeEach, beforeAll, describe, expect, it } from "vitest";
|
|
import { and, eq } from "drizzle-orm";
|
|
import {
|
|
activityLog,
|
|
authUsers,
|
|
cases,
|
|
companies,
|
|
companyMemberships,
|
|
createDb,
|
|
instanceUserRoles,
|
|
issues,
|
|
principalPermissionGrants,
|
|
} from "@paperclipai/db";
|
|
import {
|
|
getEmbeddedPostgresTestSupport,
|
|
startEmbeddedPostgresTestDatabase,
|
|
} from "./helpers/embedded-postgres.js";
|
|
import { cloudActorHeaderSourceFromHeaders, resolveCloudTenantActor } from "../middleware/auth.js";
|
|
|
|
const external = resolveExternalTestDatabaseUrl("PAPERCLIP_CLOUD_PROVISIONING_TEST_DATABASE_URL");
|
|
const embeddedPostgresSupport = external ? { supported: true, reason: undefined } : await getEmbeddedPostgresTestSupport();
|
|
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
|
|
|
if (!embeddedPostgresSupport.supported) {
|
|
console.warn(
|
|
`Skipping embedded Postgres tenant provisioning tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
|
|
);
|
|
}
|
|
|
|
const SERVER_TOKEN = "test-server-token";
|
|
|
|
function tenantHeaders(input: { stackId: string; userId: string; companyName?: string }) {
|
|
const headers: Record<string, string> = {
|
|
"x-paperclip-cloud-tenant-token": SERVER_TOKEN,
|
|
"x-paperclip-cloud-user-id": input.userId,
|
|
"x-paperclip-cloud-user-email": `${input.userId}@example.com`,
|
|
"x-paperclip-cloud-stack-id": input.stackId,
|
|
"x-paperclip-cloud-stack-role": "owner",
|
|
};
|
|
if (input.companyName) headers["x-paperclip-cloud-paperclip-company-name"] = input.companyName;
|
|
return cloudActorHeaderSourceFromHeaders(headers);
|
|
}
|
|
|
|
/**
|
|
* The prefix a pre-name-derivation build wrote. Pinned here on purpose: the
|
|
* repair detects legacy rows by this exact value, so it must keep matching
|
|
* what those builds produced.
|
|
*/
|
|
function legacyProvisionedPrefix(stackId: string) {
|
|
return `PC${createHash("sha256").update(stackId).digest("hex").slice(0, 4).toUpperCase()}`;
|
|
}
|
|
|
|
function legacyProvisionedDescription(stackId: string) {
|
|
return `Provisioned by Paperclip Cloud for stack ${stackId}.`;
|
|
}
|
|
|
|
describeEmbeddedPostgres("cloud tenant company provisioning", () => {
|
|
let db!: ReturnType<typeof createDb>;
|
|
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
|
|
|
beforeAll(async () => {
|
|
if (!external) tempDb = await startEmbeddedPostgresTestDatabase("paperclip-tenant-provisioning-");
|
|
db = createDb(external ?? tempDb!.connectionString);
|
|
}, 20_000);
|
|
|
|
beforeEach(() => {
|
|
process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN = SERVER_TOKEN;
|
|
});
|
|
|
|
afterEach(async () => {
|
|
delete process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN;
|
|
await db.delete(activityLog);
|
|
await db.delete(principalPermissionGrants);
|
|
await db.delete(companyMemberships);
|
|
await db.delete(cases);
|
|
await db.delete(issues);
|
|
await db.delete(companies);
|
|
await db.delete(instanceUserRoles);
|
|
await db.delete(authUsers);
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await db?.$client.end({ timeout: 1 });
|
|
await tempDb?.cleanup();
|
|
});
|
|
|
|
async function readCompany(companyId: string) {
|
|
return db
|
|
.select({
|
|
name: companies.name,
|
|
issuePrefix: companies.issuePrefix,
|
|
description: companies.description,
|
|
})
|
|
.from(companies)
|
|
.where(eq(companies.id, companyId))
|
|
.then((rows) => rows[0] ?? null);
|
|
}
|
|
|
|
async function readIdentifiers(companyId: string) {
|
|
const issueRows = await db
|
|
.select({ identifier: issues.identifier })
|
|
.from(issues)
|
|
.where(eq(issues.companyId, companyId));
|
|
const caseRows = await db
|
|
.select({ identifier: cases.identifier })
|
|
.from(cases)
|
|
.where(eq(cases.companyId, companyId));
|
|
return {
|
|
issues: issueRows.map((row) => row.identifier).sort(),
|
|
cases: caseRows.map((row) => row.identifier).sort(),
|
|
};
|
|
}
|
|
|
|
describe("claim time", () => {
|
|
it("derives the issue prefix from the company name and writes no placeholder description", async () => {
|
|
const actor = await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({ stackId: "stack-claim-1", userId: "user-claim-1", companyName: "Acme Robotics" }),
|
|
);
|
|
|
|
const companyId = actor!.companyIds![0]!;
|
|
await expect(readCompany(companyId)).resolves.toEqual({
|
|
name: "Acme Robotics",
|
|
issuePrefix: "ACM",
|
|
description: null,
|
|
});
|
|
});
|
|
|
|
it("derives the prefix from the humanized stack slug when no company name header is sent", async () => {
|
|
const actor = await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({ stackId: "paperclip-stack-borealis", userId: "user-claim-2" }),
|
|
);
|
|
|
|
const companyId = actor!.companyIds![0]!;
|
|
await expect(readCompany(companyId)).resolves.toMatchObject({
|
|
name: "Borealis",
|
|
issuePrefix: "BOR",
|
|
description: null,
|
|
});
|
|
});
|
|
|
|
it("suffixes the derived prefix when another company already holds it", async () => {
|
|
await db.insert(companies).values({ name: "Acme Holdings", issuePrefix: "ACM" });
|
|
|
|
const actor = await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({ stackId: "stack-claim-3", userId: "user-claim-3", companyName: "Acme Robotics" }),
|
|
);
|
|
|
|
const companyId = actor!.companyIds![0]!;
|
|
await expect(readCompany(companyId)).resolves.toMatchObject({
|
|
name: "Acme Robotics",
|
|
issuePrefix: "ACMA",
|
|
});
|
|
});
|
|
});
|
|
|
|
describe("legacy provisioning repair", () => {
|
|
/**
|
|
* Claims the company through the normal path, then rewrites it into the
|
|
* shape a pre-name-derivation build left behind.
|
|
*/
|
|
async function seedLegacyCompany(input: {
|
|
stackId: string;
|
|
claimUserId: string;
|
|
companyName: string;
|
|
issuePrefix?: string;
|
|
description?: string | null;
|
|
}) {
|
|
const actor = await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({
|
|
stackId: input.stackId,
|
|
userId: input.claimUserId,
|
|
companyName: input.companyName,
|
|
}),
|
|
);
|
|
const companyId = actor!.companyIds![0]!;
|
|
const issuePrefix = input.issuePrefix ?? legacyProvisionedPrefix(input.stackId);
|
|
await db
|
|
.update(companies)
|
|
.set({
|
|
issuePrefix,
|
|
description: input.description === undefined
|
|
? legacyProvisionedDescription(input.stackId)
|
|
: input.description,
|
|
})
|
|
.where(eq(companies.id, companyId));
|
|
await db.insert(issues).values([
|
|
{ companyId, title: "First", issueNumber: 1, identifier: `${issuePrefix}-1` },
|
|
{ companyId, title: "Second", issueNumber: 20, identifier: `${issuePrefix}-20` },
|
|
]);
|
|
await db.insert(cases).values({
|
|
companyId,
|
|
caseNumber: 4,
|
|
identifier: `${issuePrefix}-C4`,
|
|
caseType: "decision",
|
|
title: "A case",
|
|
});
|
|
return { companyId, issuePrefix };
|
|
}
|
|
|
|
it("re-derives the prefix, clears the placeholder, and re-keys both identifier tables", async () => {
|
|
const stackId = "stack-repair-1";
|
|
const { companyId, issuePrefix: legacyPrefix } = await seedLegacyCompany({
|
|
stackId,
|
|
claimUserId: "user-repair-seed-1",
|
|
companyName: "Acme Robotics",
|
|
});
|
|
|
|
await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({ stackId, userId: "user-repair-1", companyName: "Acme Robotics" }),
|
|
);
|
|
|
|
await expect(readCompany(companyId)).resolves.toEqual({
|
|
name: "Acme Robotics",
|
|
issuePrefix: "ACM",
|
|
description: null,
|
|
});
|
|
await expect(readIdentifiers(companyId)).resolves.toEqual({
|
|
issues: ["ACM-1", "ACM-20"],
|
|
cases: ["ACM-C4"],
|
|
});
|
|
|
|
const logged = await db
|
|
.select({ details: activityLog.details })
|
|
.from(activityLog)
|
|
.where(and(
|
|
eq(activityLog.companyId, companyId),
|
|
eq(activityLog.action, "company.updated"),
|
|
));
|
|
expect(logged).toHaveLength(1);
|
|
expect(logged[0]).toMatchObject({
|
|
details: {
|
|
source: "cloud_tenant_auth",
|
|
reason: "legacy_provision_defaults_repair",
|
|
previousIssuePrefix: legacyPrefix,
|
|
issuePrefix: "ACM",
|
|
descriptionCleared: true,
|
|
issuesRekeyed: 2,
|
|
casesRekeyed: 1,
|
|
},
|
|
});
|
|
});
|
|
|
|
it("is a no-op on a second pass", async () => {
|
|
const stackId = "stack-repair-2";
|
|
const { companyId } = await seedLegacyCompany({
|
|
stackId,
|
|
claimUserId: "user-repair-seed-2",
|
|
companyName: "Acme Robotics",
|
|
});
|
|
|
|
await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({ stackId, userId: "user-repair-2a", companyName: "Acme Robotics" }),
|
|
);
|
|
const afterFirstPass = await readCompany(companyId);
|
|
|
|
await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({ stackId, userId: "user-repair-2b", companyName: "Acme Robotics" }),
|
|
);
|
|
|
|
await expect(readCompany(companyId)).resolves.toEqual(afterFirstPass);
|
|
await expect(readIdentifiers(companyId)).resolves.toEqual({
|
|
issues: ["ACM-1", "ACM-20"],
|
|
cases: ["ACM-C4"],
|
|
});
|
|
// The repair logged once, on the pass that actually changed the row.
|
|
const logged = await db
|
|
.select({ id: activityLog.id })
|
|
.from(activityLog)
|
|
.where(and(
|
|
eq(activityLog.companyId, companyId),
|
|
eq(activityLog.action, "company.updated"),
|
|
));
|
|
expect(logged).toHaveLength(1);
|
|
});
|
|
|
|
it("clears only the placeholder description when the prefix is not the legacy hash", async () => {
|
|
const stackId = "stack-repair-3";
|
|
const { companyId } = await seedLegacyCompany({
|
|
stackId,
|
|
claimUserId: "user-repair-seed-3",
|
|
companyName: "Acme Robotics",
|
|
// A prefix the operator already re-derived or chose.
|
|
issuePrefix: "ZEN",
|
|
});
|
|
|
|
await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({ stackId, userId: "user-repair-3", companyName: "Acme Robotics" }),
|
|
);
|
|
|
|
await expect(readCompany(companyId)).resolves.toEqual({
|
|
name: "Acme Robotics",
|
|
issuePrefix: "ZEN",
|
|
description: null,
|
|
});
|
|
// Nothing was re-keyed, so the identifiers keep the prefix they had.
|
|
await expect(readIdentifiers(companyId)).resolves.toEqual({
|
|
issues: ["ZEN-1", "ZEN-20"],
|
|
cases: ["ZEN-C4"],
|
|
});
|
|
const logged = await db
|
|
.select({ id: activityLog.id })
|
|
.from(activityLog)
|
|
.where(eq(activityLog.companyId, companyId));
|
|
expect(logged).toHaveLength(0);
|
|
});
|
|
|
|
it("leaves an operator-written description alone while it re-derives the prefix", async () => {
|
|
const stackId = "stack-repair-4";
|
|
const { companyId } = await seedLegacyCompany({
|
|
stackId,
|
|
claimUserId: "user-repair-seed-4",
|
|
companyName: "Acme Robotics",
|
|
description: "We build robots.",
|
|
});
|
|
|
|
await resolveCloudTenantActor(
|
|
db,
|
|
tenantHeaders({ stackId, userId: "user-repair-4", companyName: "Acme Robotics" }),
|
|
);
|
|
|
|
await expect(readCompany(companyId)).resolves.toEqual({
|
|
name: "Acme Robotics",
|
|
issuePrefix: "ACM",
|
|
description: "We build robots.",
|
|
});
|
|
});
|
|
});
|
|
});
|