Files
PaperClipAI/server/src/__tests__/app-private-hostname-gate.test.ts
T
DottaandPaperclip e733bddc45 perf(chat): stop polling provider action and session sync queues (#15822)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Chat stores pending provider actions in durable database rows.
> - Three action queues still scan every second when empty.
> - These scans keep idle databases active.
> - This pull request wakes those queues after a commit or at their next
deadline.
> - Empty queues stop querying while existing recovery rules remain in
place.

## Linked Issues or Issue Description

Refs #15808, #15802, #15625. Related sleep work: #15750, #15757.

**What existing behavior does this improve?**

Provider messages and notices, accepted GitHub webhook receipts, and
Slack session status synchronization.

**Current behavior**

Chat maintenance scans these three queues every second. It also rechecks
paused Slack sync rows every 30 seconds.

**Proposed behavior**

Reuse the transaction-aware action workers and existing chat_actions
rows. Wake after commits. Restore unfinished work on startup. Use stored
retry and stale-claim deadlines. Stop scanning when empty. Park paused
Slack sync rows until an endpoint change.

## What Changed

- Add three topics to the existing action worker registration. No
tables, migrations, triggers, or external services.
- Signal producers, claims, retry updates, terminal settlements, and
operator resolution inside their owning transaction. Provider I/O
remains outside transactions.
- Share pending-state and deadline predicates between dispatch and
scheduling. Exclude terminal history and uncertain provider sends.
- Preserve GitHub receipt attempt fences and provider-send quarantine.
Preserve Slack's 30-minute refresh while a session is processing.
- Remove these workers from periodic app maintenance. Update the sleep
inventory and add regression coverage.

## Verification

- Passed: 13 focused tests across scheduler, app wiring, and real
PostgreSQL integration. Covers actual commit wakes with deferred fast
paths disabled, GitHub retries and startup recovery, stale-send
quarantine, Slack publication-to-sync delivery, paused deadlines, and
idle queues.
- Passed: pnpm typecheck.
- Passed: full chat integration and related coordinator suites, **1,274
tests across five files**.
- Passed: pnpm build.
- Greptile: **5/5**, no actionable findings, completed on
`dc284cd275e5f6df624d60b27954421f2bcdbee2`.
- Hosted CI: **54 checks passed**, two intentionally skipped Storybook
checks, no failures or pending checks on
`dc284cd275e5f6df624d60b27954421f2bcdbee2`.

Provider responses are simulated. PostgreSQL, transactions, service
dispatch, and scheduling are real. No live Slack or GitHub App was used.

## Risks

- Writers must emit the existing process-local transaction signal.
Independent database clients and direct SQL still need explicit
integration or startup recovery.
- Accepted GitHub receipts are covered. Finding callbacks GitHub never
delivered still uses the separate provider recovery lane.
- Other chat maintenance and persistent transports still run. This does
not establish whole-instance sleep.
- Existing claim, revision, credential, and authorization checks remain
authoritative during concurrent foreground and scheduled dispatch.

## Model Used

OpenAI Codex (GPT-6), with repository inspection, code execution, and
automated tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-11 06:08:05 -05:00

399 lines
15 KiB
TypeScript

import { readFileSync } from "node:fs";
import { describe, expect, it, vi } from "vitest";
import {
createChatReconciliationCoordinator,
shouldEnablePrivateHostnameGuard,
} from "../app.ts";
describe("createChatReconciliationCoordinator", () => {
it.each([false, true])(
"keeps later inbound sweeps independent of GitHub recovery and joins shutdown (rejects: %s)",
async (rejects) => {
let release!: () => void;
const held = new Promise<void>((resolve) => {
release = resolve;
});
const failure = new Error("GitHub recovery unavailable");
const processFailedGitHubWebhookDeliveries = vi.fn(async () => {
await held;
if (rejects) throw failure;
});
const processPendingDeliveries = vi.fn(async () => undefined);
const flushPublications = vi.fn(async () => undefined);
const onError = vi.fn();
const input = {
processFailedGitHubWebhookDeliveries,
processPendingDeliveries,
flushPublications,
reconcileProviderRuntimes: async () => undefined,
projectRunMilestones: async () => 0,
processPendingSlackFileUploadReceipts: async () => undefined,
processPendingSlackSessionSyncs: async () => undefined,
onError,
};
const coordinator = createChatReconciliationCoordinator(input);
let draining: Promise<void> | undefined;
try {
coordinator.reconcile();
await vi.waitFor(() => {
expect(processFailedGitHubWebhookDeliveries).toHaveBeenCalledOnce();
expect(processPendingDeliveries).toHaveBeenCalledTimes(1);
});
// A second sweep must admit work arriving AFTER the first sweep while
// remote GitHub recovery remains held. Same-sweep parallelism is not enough.
coordinator.reconcile();
await vi.waitFor(() => {
expect(processPendingDeliveries).toHaveBeenCalledTimes(2);
expect(flushPublications).toHaveBeenCalledTimes(2);
});
expect(processFailedGitHubWebhookDeliveries).toHaveBeenCalledOnce();
coordinator.stop();
let drained = false;
draining = coordinator.drain().then(() => {
drained = true;
});
await new Promise<void>((resolve) => setImmediate(resolve));
expect(drained).toBe(false);
coordinator.reconcile();
release();
await draining;
expect(drained).toBe(true);
expect(processFailedGitHubWebhookDeliveries).toHaveBeenCalledOnce();
expect(processPendingDeliveries).toHaveBeenCalledTimes(2);
expect(onError).toHaveBeenCalledTimes(rejects ? 1 : 0);
if (rejects)
expect(onError).toHaveBeenCalledWith(
"GitHub webhook recovery",
failure,
);
} finally {
release();
coordinator.stop();
await (draining ?? coordinator.drain());
}
},
);
it("wires GitHub recovery into its independent lane", () => {
const source = readFileSync(new URL("../app.ts", import.meta.url), "utf8");
expect(source).toContain(
"chatChannels.processFailedGitHubWebhookDeliveries()",
);
const service = readFileSync(
new URL("../services/chat-channels.ts", import.meta.url),
"utf8",
);
const ordinary = service.slice(
service.indexOf("async function processPendingDeliveries("),
service.indexOf("async function listResources("),
);
expect(ordinary).not.toContain("processFailedGitHubWebhookDeliveries()");
});
it("wires bounded publication refill to commit workers and isolates periodic maintenance", () => {
const source = readFileSync(new URL("../app.ts", import.meta.url), "utf8");
expect(source).toContain("registerChatDeliveryWork(deliveryWork, chatChannels,");
expect(source).toContain("registerChatActionWork(deliveryWork, chatChannels,");
const maintenance = source.slice(
source.indexOf("const reconcileChatPublicationMaintenance ="),
source.indexOf("const chatReconciliation ="),
);
expect(maintenance).toContain("chatChannels.processPublicationMaintenance()");
expect(maintenance).not.toContain("chatChannels.schedulePendingPublications()");
expect(maintenance).not.toContain("chatChannels.processPendingPublications()");
const workers = readFileSync(new URL("../services/chat-delivery-work.ts", import.meta.url), "utf8");
expect(workers).toContain("run: service.scheduleQueuedPublications");
// The service integration tests hold real publication workers while this
// scheduled method returns; app shutdown must also join those workers.
expect(source).toContain("await chatChannels.shutdown()");
expect(source).not.toContain("chatChannels.processPendingSlackSessionSyncs()");
const service = readFileSync(new URL("../services/chat-channels.ts", import.meta.url), "utf8");
const actionMaintenance = service.slice(service.indexOf("async function processPendingChatMaintenance("),
service.indexOf("async function processPendingDeliveries("));
expect(actionMaintenance).not.toContain("processPendingProviderEffects(");
expect(actionMaintenance).not.toContain("processPendingGitHubWebhookIngress(");
});
it.each([false, true])(
"isolates a blocked milestone projector and joins its completion (rejects: %s)",
async (rejects) => {
let release!: () => void;
const held = new Promise<void>((resolve) => {
release = resolve;
});
const failure = new Error("milestone projection failed");
const projectRunMilestones = vi
.fn(async () => 0)
.mockImplementationOnce(async () => {
await held;
if (rejects) throw failure;
return 1;
});
const flushPublications = vi.fn(async () => undefined);
const onError = vi.fn();
const coordinator = createChatReconciliationCoordinator({
projectRunMilestones,
reconcileProviderRuntimes: async () => undefined,
processPendingDeliveries: async () => undefined,
flushPublications,
processPendingSlackFileUploadReceipts: async () => undefined,
processPendingSlackSessionSyncs: async () => undefined,
onError,
});
try {
coordinator.reconcile();
await vi.waitFor(() => {
expect(projectRunMilestones).toHaveBeenCalledTimes(1);
expect(flushPublications).toHaveBeenCalledTimes(1);
});
// A newly committed final/question must dispatch even while a different
// issue's projection is blocked. Repeated signals leave one dirty retry.
coordinator.notifyPublications();
coordinator.notifyPublications();
await vi.waitFor(() =>
expect(flushPublications).toHaveBeenCalledTimes(2),
);
expect(projectRunMilestones).toHaveBeenCalledTimes(1);
let drained = false;
const draining = coordinator.drain().then(() => {
drained = true;
});
await new Promise<void>((resolve) => setImmediate(resolve));
expect(drained).toBe(false);
release();
await draining;
expect(projectRunMilestones).toHaveBeenCalledTimes(2);
expect(onError).toHaveBeenCalledTimes(rejects ? 1 : 0);
if (rejects)
expect(onError).toHaveBeenCalledWith("run milestones", failure);
const completedCounts = [
projectRunMilestones.mock.calls.length,
flushPublications.mock.calls.length,
];
await new Promise<void>((resolve) => setImmediate(resolve));
expect([
projectRunMilestones.mock.calls.length,
flushPublications.mock.calls.length,
]).toEqual(completedCounts);
} finally {
release();
coordinator.stop();
await coordinator.drain();
}
},
);
it("joins a blocked milestone projector at shutdown without starting dirty follow-ups", async () => {
let release!: () => void;
const held = new Promise<void>((resolve) => {
release = resolve;
});
const projectRunMilestones = vi.fn(async () => {
await held;
return 1;
});
const flushPublications = vi.fn(async () => undefined);
const coordinator = createChatReconciliationCoordinator({
projectRunMilestones,
flushPublications,
reconcileProviderRuntimes: async () => undefined,
processPendingDeliveries: async () => undefined,
processPendingSlackFileUploadReceipts: async () => undefined,
processPendingSlackSessionSyncs: async () => undefined,
onError: vi.fn(),
});
try {
coordinator.reconcile();
await vi.waitFor(() => {
expect(projectRunMilestones).toHaveBeenCalledOnce();
expect(flushPublications).toHaveBeenCalledOnce();
});
coordinator.notifyPublications();
coordinator.stop();
let drained = false;
const draining = coordinator.drain().then(() => {
drained = true;
});
await new Promise<void>((resolve) => setImmediate(resolve));
expect(drained).toBe(false);
release();
await draining;
expect(projectRunMilestones).toHaveBeenCalledOnce();
expect(flushPublications).toHaveBeenCalledOnce();
} finally {
release();
coordinator.stop();
await coordinator.drain();
}
});
it("keeps slow optional recovery from suppressing later publication sweeps", async () => {
let releaseDelivery!: () => void;
let releaseSlackStatus!: () => void;
const deliveryReleased = new Promise<void>((resolve) => {
releaseDelivery = resolve;
});
const slackStatusReleased = new Promise<void>((resolve) => {
releaseSlackStatus = resolve;
});
const reconcileProviderRuntimes = vi.fn(async () => undefined);
const processPendingDeliveries = vi.fn(async () => deliveryReleased);
const flushPublications = vi.fn(async () => undefined);
const processPendingSlackFileUploadReceipts = vi.fn(async () => undefined);
const processPendingSlackSessionSyncs = vi.fn(
async () => slackStatusReleased,
);
const onError = vi.fn();
const coordinator = createChatReconciliationCoordinator({
projectRunMilestones: async () => 0,
reconcileProviderRuntimes,
processPendingDeliveries,
flushPublications,
processPendingSlackFileUploadReceipts,
processPendingSlackSessionSyncs,
onError,
});
coordinator.reconcile();
await vi.waitFor(() => {
expect(reconcileProviderRuntimes).toHaveBeenCalledTimes(1);
expect(processPendingDeliveries).toHaveBeenCalledTimes(1);
expect(flushPublications).toHaveBeenCalledTimes(1);
expect(processPendingSlackFileUploadReceipts).toHaveBeenCalledTimes(1);
expect(processPendingSlackSessionSyncs).toHaveBeenCalledTimes(1);
});
coordinator.reconcile();
await vi.waitFor(() => {
expect(reconcileProviderRuntimes).toHaveBeenCalledTimes(2);
expect(flushPublications).toHaveBeenCalledTimes(2);
expect(processPendingSlackFileUploadReceipts).toHaveBeenCalledTimes(2);
});
expect(processPendingDeliveries).toHaveBeenCalledTimes(1);
expect(processPendingSlackSessionSyncs).toHaveBeenCalledTimes(1);
releaseDelivery();
releaseSlackStatus();
await coordinator.drain();
expect(onError).not.toHaveBeenCalled();
});
it.each([false, true])(
"joins the independent Slack file receipt lane at shutdown (rejects: %s)",
async (rejects) => {
let releaseReceipt!: () => void;
const receiptReleased = new Promise<void>((resolve) => {
releaseReceipt = resolve;
});
const lookupError = new Error("receipt lookup failed");
const processPendingSlackFileUploadReceipts = vi
.fn(async () => undefined)
.mockImplementationOnce(async () => {
await receiptReleased;
if (rejects) throw lookupError;
});
const reconcileProviderRuntimes = vi.fn(async () => undefined);
const processPendingDeliveries = vi.fn(async () => undefined);
const flushPublications = vi.fn(async () => undefined);
const processPendingSlackSessionSyncs = vi.fn(async () => undefined);
const onError = vi.fn();
const coordinator = createChatReconciliationCoordinator({
projectRunMilestones: async () => 0,
reconcileProviderRuntimes,
processPendingDeliveries,
flushPublications,
processPendingSlackFileUploadReceipts,
processPendingSlackSessionSyncs,
onError,
});
let draining: Promise<void> | undefined;
try {
coordinator.reconcile();
await vi.waitFor(() => {
expect(processPendingSlackFileUploadReceipts).toHaveBeenCalledTimes(
1,
);
expect(flushPublications).toHaveBeenCalledTimes(1);
});
coordinator.reconcile();
await vi.waitFor(() => {
expect(reconcileProviderRuntimes).toHaveBeenCalledTimes(2);
expect(processPendingDeliveries).toHaveBeenCalledTimes(2);
expect(flushPublications).toHaveBeenCalledTimes(2);
expect(processPendingSlackSessionSyncs).toHaveBeenCalledTimes(2);
});
expect(processPendingSlackFileUploadReceipts).toHaveBeenCalledTimes(1);
let drained = false;
draining = coordinator.drain().then(() => {
drained = true;
});
// Flush the promise queue, not a wall-clock delay: the held receipt
// must still be part of shutdown's joined work after other lanes end.
await new Promise<void>((resolve) => setImmediate(resolve));
expect(drained).toBe(false);
expect(onError).not.toHaveBeenCalled();
releaseReceipt();
await draining;
expect(drained).toBe(true);
expect(onError).toHaveBeenCalledTimes(rejects ? 1 : 0);
if (rejects) {
expect(onError).toHaveBeenCalledWith(
"Slack file receipts",
lookupError,
);
}
// A completed or failed lookup releases only its own single-flight
// slot; the next ordinary reconciliation can recover another receipt.
coordinator.reconcile();
await coordinator.drain();
expect(processPendingSlackFileUploadReceipts).toHaveBeenCalledTimes(2);
expect(onError).toHaveBeenCalledTimes(rejects ? 1 : 0);
} finally {
releaseReceipt();
await draining;
await coordinator.drain();
}
},
);
});
describe("shouldEnablePrivateHostnameGuard", () => {
it("enables the hostname guard for local_trusted private deployments", () => {
expect(
shouldEnablePrivateHostnameGuard({
deploymentMode: "local_trusted",
deploymentExposure: "private",
}),
).toBe(true);
});
it("does not enable the hostname guard for local_trusted public deployments", () => {
expect(
shouldEnablePrivateHostnameGuard({
deploymentMode: "local_trusted",
deploymentExposure: "public",
}),
).toBe(false);
});
it("enables the hostname guard for authenticated private deployments", () => {
expect(
shouldEnablePrivateHostnameGuard({
deploymentMode: "authenticated",
deploymentExposure: "private",
}),
).toBe(true);
});
it("does not enable the hostname guard for authenticated public deployments", () => {
expect(
shouldEnablePrivateHostnameGuard({
deploymentMode: "authenticated",
deploymentExposure: "public",
}),
).toBe(false);
});
});