mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 13:20:54 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connected chats show safe notices when a run starts, waits, finishes, or fails. > - A recurring scan projected these notices even when no eligible work existed. > - Eligibility depends on committed run state, conversation lineage, and current authority. > - Existing database rows can reconstruct that work after restart. > - This PR wakes a scoped worker from those writes and removes lifecycle polling. > - Native progress events keep their separate existing owner until the next migration. ## Linked Issues or Issue Description **What happened?** The chat maintenance timer repeatedly scanned run lifecycle state. Empty and settled instances still queried the database. A run-status event alone cannot replace the scan: prompt receipts, continuation receipts, identity changes, and restored authority can make a run eligible later. **What did you expect?** Committed changes wake projection. Startup reconstructs eligible work. A completed pass stops until another relevant change arrives. Failed work retains recovery without scanning settled history forever. **Steps to reproduce** Configure an automatic chat endpoint. Finish its work, or leave it unused. Observe recurring lifecycle projection queries from the chat maintenance callback. Stacked on #17592. Related foundation: #15625. This PR owns lifecycle milestones only. ## What Changed - Preserve upstream coalesced Cloud membership writes together with locked membership restoration and its committed chat notification. - Signal milestone eligibility from the real Slack account-link transaction. - Register the lifecycle worker with the existing delivery coordinator. Remove lifecycle projection from the periodic chat callback. Keep native progress in a separately named lane and preserve the explicit combined API. - Reconstruct company/issue scopes from the existing selector. Drain bounded batches with company fairness. Retry failures, retain writes through drain/resume, and disarm settled work. - Signal from supported run, presentation, interaction, receipt, lineage, conversation, recovery, and authority writers. Preserve their transaction ownership, including nested savepoints and uncertain settlement. No new table is added. - Preserve provider authorization, authored-final suppression, responsible-user continuity, deduplication, the 24-hour selection window, and raw-output restrictions. Keep provider interpretation in its owner. - Preserve PostgreSQL timestamp precision in lifecycle pagination. Rejected submillisecond candidates must not repeat the same page and starve later notices. - Document producer coverage and alternate-writer dispositions in `doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md`. ## Verification Published head: `1d056f68c5b1005933b9e12de8745bf1c4cbeeca`, targeting master after #17592 merged at `7d75ae0569e41786640cf09647ae8849411d2f22`. A normal merge integrates the actual landed parent. The entire 77-path milestone patch is binary-identical to the locally verified preparation across that ancestry reconciliation; both unrelated master paths are preserved. The four earlier producer/test integration conflicts retain scoped milestone and voice notices, current onboarding completion admission, and expanded identity query/read-scope coverage. - **330 distinct selected cases pass on the verified unchanged patch:** 305 across 14 complete suites (all five milestone worker suites, run identity, board claim, Cloud provisioning, auth session routes, Cloud actor authentication, stale-lock recovery, GitHub credentials, project Git authorization, and app admission wiring), plus 25 selected real chat producer cases. The remaining 1,590 chat integration cases were skipped, not passed. - The real producer cases cover saved milestone discovery, GitHub and Slack identity changes, account-link rollback/reconfirmation, ordinary ingress reactivation, post-close action/delivery receipt settlement, all ten authority-restoration paths including Cloud synchronization, and changed native review metadata. - Worker coverage verifies outer commit, rollback, uncertain-write recovery, restart, bounded batching/fairness, timestamp precision, retries, drain/resume, and settled SQL quiet. Authorization and existing publication deduplication remain with their owning services. - Server production typechecking, build, and module-boundary checks pass. The tests above were executed separately. Production typechecking excludes `src/__tests__` and does not replace those runs. Each database owner uses its own newly created and app-migrated PostgreSQL fixture. - [Current-parent acceptance and preserved failure ledger](https://github.com/paperclipai/paperclip/blob/1d056f68c5b1005933b9e12de8745bf1c4cbeeca/doc/plans/2026-10-25-chat-run-milestone-sleep-audit.md#current-parent-integration-and-acceptance-october-28) records commands, counts, producer scope, conflict dispositions, and limits. Earlier combined-stack counts and failures keep their original revision scope; local acceptance does not erase old CI failures. - No new migration, queue table, dependency, workflow, or shared scheduler contract is introduced. Parent migration 0397, bounded consent recovery, voice cleanup/retention repairs, and runner checkpoint restoration remain present. - **Exact-head Greptile is 5/5 and all 55 current checks are clean.** No Ponytail review is present, which is an explicitly authorized nonblocker; no failing review is waived. Older reviews and parent checks are not current passes. Actual failing Ponytail findings must be fixed; an absent/stopped/paused turn without a verdict is an authorized nonblocker. Active unfinished Ponytail uses its original request/start timer, with a 25-minute monitor cutoff respecting the user's 30-minute maximum. - No complete local repository matrix, live provider tenant, external alarm, physical host/database suspension, or fleet savings claim is made. Native progress remains a separate following slice. ## Risks - A future eligibility writer must notify the existing projection boundary. Independent SQL writers or another active server process are outside this process-local signal contract. - Authority and lineage remain sensitive. The worker uses the existing selectors and publication policy; it does not grant execution or disclosure authority. - Native progress and other publication maintenance still have recurring work. This PR removes one reason, not the entire chat maintenance timer. - Worker quiet does not prove database suspension, host sleep, or external alarm coverage. Cloud sleep blockers remain unchanged. - Merge eligibility requires passing current-head CI and Greptile 5/5 after this actual-master reconciliation. Subsequent stack slices retain their own gates. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window size are not exposed. Used repository inspection, reasoning, shell execution, PostgreSQL tests, browser E2Es and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the documented focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
440 lines
15 KiB
TypeScript
440 lines
15 KiB
TypeScript
import express from "express";
|
|
import request from "supertest";
|
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
|
import { activityLog, companies, companyMemberships, instanceUserRoles } from "@paperclipai/db";
|
|
import {
|
|
actorMiddleware,
|
|
humanizeCloudStackSlug,
|
|
isKnownBadCloudCompanyName,
|
|
} from "../middleware/auth.js";
|
|
import { errorHandler } from "../middleware/error-handler.js";
|
|
import { assertCompanyAccess } from "../routes/authz.js";
|
|
|
|
function queryResult<T>(rows: T) {
|
|
return Object.assign(Promise.resolve(rows), { for: () => Promise.resolve(rows) });
|
|
}
|
|
|
|
function createSelectChain(rows: unknown[]) {
|
|
return {
|
|
from() {
|
|
return {
|
|
where() {
|
|
return queryResult(rows);
|
|
},
|
|
};
|
|
},
|
|
};
|
|
}
|
|
|
|
function createDb() {
|
|
return {
|
|
select: vi
|
|
.fn()
|
|
.mockImplementationOnce(() => createSelectChain([]))
|
|
.mockImplementationOnce(() => createSelectChain([])),
|
|
} as any;
|
|
}
|
|
|
|
describe("actorMiddleware authenticated session profile", () => {
|
|
const originalCloudTenantToken = process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN;
|
|
|
|
afterEach(() => {
|
|
if (originalCloudTenantToken === undefined) delete process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN;
|
|
else process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN = originalCloudTenantToken;
|
|
});
|
|
|
|
it("preserves the signed-in user name and email on the board actor", async () => {
|
|
const app = express();
|
|
app.use(
|
|
actorMiddleware(createDb(), {
|
|
deploymentMode: "authenticated",
|
|
resolveSession: async () => ({
|
|
session: { id: "session-1", userId: "user-1" },
|
|
user: {
|
|
id: "user-1",
|
|
name: "User One",
|
|
email: "user@example.com",
|
|
},
|
|
}),
|
|
}),
|
|
);
|
|
app.get("/actor", (req, res) => {
|
|
res.json(req.actor);
|
|
});
|
|
|
|
const res = await request(app).get("/actor");
|
|
|
|
expect(res.status).toBe(200);
|
|
expect(res.body).toMatchObject({
|
|
type: "board",
|
|
userId: "user-1",
|
|
userName: "User One",
|
|
userEmail: "user@example.com",
|
|
source: "session",
|
|
companyIds: [],
|
|
memberships: [],
|
|
isInstanceAdmin: false,
|
|
});
|
|
});
|
|
|
|
it("trusts Cloud tenant identity headers and seeds board access", async () => {
|
|
process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN = "tenant-token";
|
|
const inserts: Array<{ values: Record<string, unknown> }> = [];
|
|
const db = {
|
|
insert: vi.fn(() => {
|
|
const chain = {
|
|
values(values: Record<string, unknown>) {
|
|
inserts.push({ values });
|
|
return chain;
|
|
},
|
|
onConflictDoUpdate() {
|
|
return chain;
|
|
},
|
|
onConflictDoNothing() {
|
|
return chain;
|
|
},
|
|
returning() {
|
|
return Promise.resolve([{
|
|
companyId: inserts.at(-1)?.values.companyId,
|
|
membershipRole: inserts.at(-1)?.values.membershipRole,
|
|
status: inserts.at(-1)?.values.status,
|
|
}]);
|
|
},
|
|
};
|
|
return chain;
|
|
}),
|
|
delete: vi.fn(() => ({ where: () => Promise.resolve(undefined) })),
|
|
select: vi.fn(() => createSelectChain([])),
|
|
} as any;
|
|
db.transaction = vi.fn(async (run: (tx: typeof db) => Promise<unknown>) => run(db));
|
|
const app = express();
|
|
app.use(
|
|
actorMiddleware(db, {
|
|
deploymentMode: "authenticated",
|
|
resolveSession: async () => null,
|
|
}),
|
|
);
|
|
app.get("/actor", (req, res) => {
|
|
res.json(req.actor);
|
|
});
|
|
|
|
const res = await request(app)
|
|
.get("/actor")
|
|
.set("x-paperclip-cloud-tenant-token", "tenant-token")
|
|
.set("x-paperclip-cloud-user-id", "global-user-1")
|
|
.set("x-paperclip-cloud-user-email", "owner@example.com")
|
|
.set("x-paperclip-cloud-user-name", "Stack Owner")
|
|
.set("x-paperclip-cloud-stack-id", "stack-alpha")
|
|
.set("x-paperclip-cloud-paperclip-company-id", "paperclip-stack-alpha")
|
|
.set("x-paperclip-cloud-paperclip-company-name", "Purple Rain")
|
|
.set("x-paperclip-cloud-stack-role", "owner");
|
|
|
|
expect(res.status).toBe(200);
|
|
expect(res.body).toMatchObject({
|
|
type: "board",
|
|
userId: "global-user-1",
|
|
userName: "Stack Owner",
|
|
userEmail: "owner@example.com",
|
|
source: "cloud_tenant",
|
|
isInstanceAdmin: false,
|
|
memberships: [expect.objectContaining({ membershipRole: "owner", status: "active" })],
|
|
});
|
|
expect(res.body.companyIds[0]).toMatch(/^[0-9a-f-]{36}$/);
|
|
// authUsers, companies, companyMemberships, the role-default
|
|
// principalPermissionGrants, and the lazily initialized instance setting.
|
|
expect(inserts).toHaveLength(5);
|
|
expect(inserts[0]?.values).toMatchObject({
|
|
id: "global-user-1",
|
|
email: "owner@example.com",
|
|
emailVerified: true,
|
|
});
|
|
expect(inserts[1]?.values).toMatchObject({
|
|
name: "Purple Rain",
|
|
});
|
|
});
|
|
|
|
it("lets the cloud tenant actor through assertCompanyAccess for a company it holds a membership row in", async () => {
|
|
process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN = "tenant-token";
|
|
// A company created on the instance after provisioning (e.g. by a company
|
|
// import) — the user has a real membership row, but it is not the stack's
|
|
// seeded primary company.
|
|
const importedCompanyId = "33333333-3333-4333-8333-333333333333";
|
|
const unrelatedCompanyId = "44444444-4444-4444-8444-444444444444";
|
|
const insertChain = {
|
|
values() {
|
|
return insertChain;
|
|
},
|
|
onConflictDoUpdate() {
|
|
return insertChain;
|
|
},
|
|
onConflictDoNothing() {
|
|
return insertChain;
|
|
},
|
|
returning() {
|
|
return Promise.resolve([{ companyId: "company-1", membershipRole: "member", status: "active" }]);
|
|
},
|
|
then(resolve: (value: unknown) => unknown) {
|
|
return Promise.resolve(undefined).then(resolve);
|
|
},
|
|
};
|
|
const db = {
|
|
select: vi.fn(() => ({
|
|
from: (table: unknown) => ({
|
|
where: () =>
|
|
queryResult(
|
|
table === companyMemberships
|
|
? [{ companyId: importedCompanyId, membershipRole: "member", status: "active" }]
|
|
: [],
|
|
),
|
|
}),
|
|
})),
|
|
insert: vi.fn(() => insertChain),
|
|
delete: vi.fn(() => ({ where: () => Promise.resolve(undefined) })),
|
|
} as any;
|
|
db.transaction = vi.fn(async (run: (tx: typeof db) => Promise<unknown>) => run(db));
|
|
const app = express();
|
|
app.use(
|
|
actorMiddleware(db, {
|
|
deploymentMode: "authenticated",
|
|
resolveSession: async () => null,
|
|
}),
|
|
);
|
|
app.get("/companies/:companyId/resource", (req, res) => {
|
|
assertCompanyAccess(req, req.params.companyId);
|
|
res.json({ ok: true, companyIds: req.actor.companyIds });
|
|
});
|
|
app.post("/companies/:companyId/resource", (req, res) => {
|
|
assertCompanyAccess(req, req.params.companyId);
|
|
res.json({ ok: true });
|
|
});
|
|
app.use(errorHandler);
|
|
|
|
const cloudHeaders = {
|
|
"x-paperclip-cloud-tenant-token": "tenant-token",
|
|
"x-paperclip-cloud-user-id": "global-user-1",
|
|
"x-paperclip-cloud-user-email": "owner@example.com",
|
|
"x-paperclip-cloud-stack-id": "stack-alpha",
|
|
"x-paperclip-cloud-stack-role": "member",
|
|
};
|
|
|
|
// Reads and writes both reach the imported company through the real
|
|
// membership row (write access also consults actor.memberships).
|
|
const read = await request(app).get(`/companies/${importedCompanyId}/resource`).set(cloudHeaders);
|
|
expect(read.status).toBe(200);
|
|
expect(read.body.companyIds).toContain(importedCompanyId);
|
|
|
|
const write = await request(app).post(`/companies/${importedCompanyId}/resource`).set(cloudHeaders);
|
|
expect(write.status).toBe(200);
|
|
|
|
// Companies the user holds no membership row in stay unreachable.
|
|
const denied = await request(app).get(`/companies/${unrelatedCompanyId}/resource`).set(cloudHeaders);
|
|
expect(denied.status).toBe(403);
|
|
});
|
|
|
|
it("repairs a legacy machine company name from the trusted human-name header", async () => {
|
|
process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN = "tenant-token";
|
|
const updates: Array<Record<string, unknown>> = [];
|
|
const activities: Array<Record<string, unknown>> = [];
|
|
const insertChain = {
|
|
values() {
|
|
return insertChain;
|
|
},
|
|
onConflictDoUpdate() {
|
|
return insertChain;
|
|
},
|
|
onConflictDoNothing() {
|
|
return insertChain;
|
|
},
|
|
returning() {
|
|
return Promise.resolve([
|
|
{ companyId: "company-1", membershipRole: "owner", status: "active" },
|
|
]);
|
|
},
|
|
then(resolve: (value: unknown) => unknown) {
|
|
return Promise.resolve(undefined).then(resolve);
|
|
},
|
|
};
|
|
const db = {
|
|
select: vi.fn(() => ({
|
|
from: (table: unknown) => ({
|
|
where: () =>
|
|
queryResult(
|
|
table === companies
|
|
? [{ name: "paperclip-stack-purple-rain" }]
|
|
: [],
|
|
),
|
|
}),
|
|
})),
|
|
insert: vi.fn((table: unknown) => {
|
|
if (table !== activityLog) return insertChain;
|
|
return {
|
|
values(values: Record<string, unknown>) {
|
|
activities.push(values);
|
|
return Promise.resolve(undefined);
|
|
},
|
|
};
|
|
}),
|
|
update: vi.fn(() => ({
|
|
set(values: Record<string, unknown>) {
|
|
updates.push(values);
|
|
return {
|
|
where: () => ({
|
|
returning: () => Promise.resolve([{ id: "company-1" }]),
|
|
}),
|
|
};
|
|
},
|
|
})),
|
|
delete: vi.fn(() => ({ where: () => Promise.resolve(undefined) })),
|
|
} as any;
|
|
db.transaction = vi.fn(async (run: (tx: typeof db) => Promise<unknown>) => run(db));
|
|
const app = express();
|
|
app.use(
|
|
actorMiddleware(db, {
|
|
deploymentMode: "authenticated",
|
|
resolveSession: async () => null,
|
|
}),
|
|
);
|
|
app.get("/actor", (req, res) => res.json(req.actor));
|
|
|
|
const res = await request(app)
|
|
.get("/actor")
|
|
.set("x-paperclip-cloud-tenant-token", "tenant-token")
|
|
.set("x-paperclip-cloud-user-id", "global-user-1")
|
|
.set("x-paperclip-cloud-user-email", "owner@example.com")
|
|
.set("x-paperclip-cloud-stack-id", "stack-purple-rain")
|
|
.set(
|
|
"x-paperclip-cloud-paperclip-company-id",
|
|
"paperclip-stack-purple-rain",
|
|
)
|
|
.set("x-paperclip-cloud-paperclip-company-name", "Purple Rain")
|
|
.set("x-paperclip-cloud-stack-role", "owner");
|
|
|
|
expect(res.status).toBe(200);
|
|
expect(updates).toHaveLength(1);
|
|
expect(updates[0]).toMatchObject({ name: "Purple Rain" });
|
|
expect(activities).toEqual([
|
|
expect.objectContaining({
|
|
companyId: expect.any(String),
|
|
actorType: "system",
|
|
actorId: "cloud-tenant-auth",
|
|
action: "company.updated",
|
|
entityType: "company",
|
|
details: expect.objectContaining({
|
|
reason: "legacy_machine_name_repair",
|
|
previousName: "paperclip-stack-purple-rain",
|
|
name: "Purple Rain",
|
|
}),
|
|
}),
|
|
]);
|
|
});
|
|
|
|
it("purges a stale instance_admin row so the session path stops elevating the cloud-tenant user", async () => {
|
|
process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN = "tenant-token";
|
|
// Simulates a deployment that previously ran the pre-hardening cloud_tenant
|
|
// path: instance_user_roles still holds an instance_admin row for the
|
|
// tenant user, who can also resolve a BetterAuth session for the same id.
|
|
const state = { staleInstanceAdminRow: true };
|
|
const insertChain = {
|
|
values() {
|
|
return insertChain;
|
|
},
|
|
onConflictDoUpdate() {
|
|
return insertChain;
|
|
},
|
|
onConflictDoNothing() {
|
|
return insertChain;
|
|
},
|
|
returning() {
|
|
return Promise.resolve([{ companyId: "company-1", membershipRole: "owner", status: "active" }]);
|
|
},
|
|
then(resolve: (value: unknown) => unknown) {
|
|
return Promise.resolve(undefined).then(resolve);
|
|
},
|
|
};
|
|
const db = {
|
|
select: vi.fn(() => ({
|
|
from: (table: unknown) => ({
|
|
where: () =>
|
|
queryResult(
|
|
table === instanceUserRoles && state.staleInstanceAdminRow ? [{ id: "stale-role-row" }] : [],
|
|
),
|
|
}),
|
|
})),
|
|
insert: vi.fn(() => insertChain),
|
|
delete: vi.fn((table: unknown) => ({
|
|
where: () => {
|
|
if (table === instanceUserRoles) state.staleInstanceAdminRow = false;
|
|
return Promise.resolve(undefined);
|
|
},
|
|
})),
|
|
} as any;
|
|
db.transaction = vi.fn(async (run: (tx: typeof db) => Promise<unknown>) => run(db));
|
|
const app = express();
|
|
app.use(
|
|
actorMiddleware(db, {
|
|
deploymentMode: "authenticated",
|
|
resolveSession: async () => ({
|
|
session: { id: "session-1", userId: "global-user-1" },
|
|
user: { id: "global-user-1", name: "Stack Owner", email: "owner@example.com" },
|
|
}),
|
|
}),
|
|
);
|
|
app.get("/actor", (req, res) => {
|
|
res.json(req.actor);
|
|
});
|
|
|
|
// Control: while the stale row exists, the session path still elevates.
|
|
const before = await request(app).get("/actor");
|
|
expect(before.body).toMatchObject({ source: "session", isInstanceAdmin: true });
|
|
|
|
// One trusted-header authentication purges the stale grant.
|
|
const cloud = await request(app)
|
|
.get("/actor")
|
|
.set("x-paperclip-cloud-tenant-token", "tenant-token")
|
|
.set("x-paperclip-cloud-user-id", "global-user-1")
|
|
.set("x-paperclip-cloud-user-email", "owner@example.com")
|
|
.set("x-paperclip-cloud-stack-id", "stack-alpha")
|
|
.set("x-paperclip-cloud-stack-role", "owner");
|
|
expect(cloud.body).toMatchObject({ source: "cloud_tenant", isInstanceAdmin: false });
|
|
expect(state.staleInstanceAdminRow).toBe(false);
|
|
|
|
// The same user no longer gets instance admin via the session path.
|
|
const after = await request(app).get("/actor");
|
|
expect(after.body).toMatchObject({
|
|
source: "session",
|
|
userId: "global-user-1",
|
|
isInstanceAdmin: false,
|
|
});
|
|
});
|
|
});
|
|
|
|
describe("Cloud tenant company naming", () => {
|
|
const ids = {
|
|
companyId: "11111111-1111-4111-8111-111111111111",
|
|
paperclipCompanyId: "paperclip-stack-purple-rain",
|
|
};
|
|
|
|
it.each([
|
|
"paperclip-stack-purple-rain",
|
|
"stack-purple-rain Paperclip",
|
|
ids.companyId,
|
|
])("repairs the known-bad machine name %s", (name) => {
|
|
expect(isKnownBadCloudCompanyName(name, ids)).toBe(true);
|
|
});
|
|
|
|
it.each([
|
|
"Purple Rain",
|
|
"Paperclip Stack Purple Rain",
|
|
"The Purple Rain Paperclip",
|
|
])("preserves the genuine company name %s", (name) => {
|
|
expect(isKnownBadCloudCompanyName(name, ids)).toBe(false);
|
|
});
|
|
|
|
it("humanizes the stack slug for old harnesses without a name header", () => {
|
|
expect(humanizeCloudStackSlug("stack-purple-rain")).toBe("Purple Rain");
|
|
expect(humanizeCloudStackSlug("paperclip-stack-purple-rain")).toBe(
|
|
"Purple Rain",
|
|
);
|
|
});
|
|
});
|