mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 16:58:52 +01:00
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/85/changes/paperclip/85 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Retired native sessions leave continuity and quarantine evidence on disk. > - The old hourly loop mixes evidence retention with backup expiry, which has different database eligibility rules. > - Evidence already has saved creation timestamps and local publication signals. > - This pull request gives evidence its own registered worker and actual next deadline. > - The worker stops filesystem and SQL activity when its evidence obligations settle. ## Linked Issues or Issue Description Stack slice 65. Parent: #17923 (`codex/sleep-64-skill-version-retention`). Related foundation: #15625. **What happened?** At parent `7b94aa185c29673372646bbf2e07a2fbd16bc394`, an hourly backup/evidence tick still inspects native state roots. Evidence's private due time is checked by that recurring tick instead of the shared scheduler. Empty evidence and future retention windows have no reason to be rediscovered every hour. **Expected behavior** Reconstruct evidence once. Wake from the existing publication signal. Register the earliest saved retention deadline. Keep busy, failed, or incomplete evidence recoverable. Preserve backup eligibility as a separate duty. **Steps to reproduce** Start retention with an empty evidence root or archives inside the seven-day window. The old timer still visits the combined owner. The new evidence worker disarms when empty, or registers the saved expiry with no evidence filesystem or SQL calls between relevant changes and that deadline. ## What Changed - Add an evidence worker through the delivery coordinator and shared scheduler. Track startup reconstruction without delaying HTTP readiness. Unsubscribe local publication before delivery shutdown. - Select `backups` for the existing backup timer and `evidence` for the new worker. Existing callers that omit the selection retain combined behavior. Backup expiration remains the next separate migration. - Use hourly retries for concrete busy or failed evidence, with earlier saved expiries and one-second continuations independent. Restore candidate/removal/byte summaries. Schedule publication at the first removable instant; preserve legacy manual-tick semantics. - Expose the existing earliest evidence deadline. Preserve one-second bounded continuation, concrete failure/busy retries, cleanup-source archive exclusions, symlink protection, late activity checks, atomic rename, and interrupted removal recovery. - Preserve non-ENOENT inspection errors as recovery obligations. Retry a failed session root in a new evidence pass instead of continuing past it and forgetting it. - Register the first removable instant after the inclusive retention boundary. Keep the existing manual dry-run publication semantics. - Add real filesystem/isolated PostgreSQL lifecycle tests and disabled/held-startup wiring cases. Add no table or migration. The archive and quarantine writers already call `noteNativeRunnerEvidence`; this PR connects that existing signal to the scheduler. Shared code owns scheduling, while evidence rules stay with the retention service. See [`doc/plans/2026-10-27-runner-evidence-retention-sleep-audit.md`](doc/plans/2026-10-27-runner-evidence-retention-sleep-audit.md). ## Verification - Adopted actual prerequisite squash `b06ea111a254c62e0a55199b010d82e0adb33bf0` from #17923 through normal merges. Qualified runtime/package/UI/map trees remain unchanged. The final nine-path diff includes one browser-spec readiness repair. - Runtime qualification remains **652 passing TypeScript cases across 22 complete files**, plus **29 passing Rust cases** in native-backend, runtime-notification and configured-environment targets. One actual closed Pi snapshot launch is intentionally skipped because its external distribution root is not provided. Accepted source is `b80eee32f344ae0218d8b791c14f2ea174da09cc`; all runtime and earlier sleep-test bytes are preserved. - The complete outage-recovery browser file passes **all five journeys** on a fresh isolated database after four additive rendered-history readiness assertions. Every original assertion, outage control and timeout remains byte-identical when those new lines are removed. The intentionally unresolved first-session scenario keeps its pre-auth interaction and delayed-owner coverage. The prior CI/local timeout evidence and one rejected broader precondition remain in the audit. - Adapter/Runner/database builds, server/Runner typechecking, Rust formatting/workspace checks, server build, module boundaries, migration order and generated TypeScript/Rust profiles pass. PostgreSQL 17 fixtures apply all 413 journal entries through index 414; source hashes and temporary fixture imports restore exactly. - **Inherited validation exception:** Copilot profile 81's separate source-identity check fails on the merged Rust configured-environment hash. Its inputs and attestations equal upstream and stay unchanged here. Grok's identity check passes. No release attestation is regenerated or new live-provider qualification claimed. Required exact-head CI/security and Greptile 5/5 still must pass. - [The evidence audit](doc/plans/2026-10-27-runner-evidence-retention-sleep-audit.md) retains historical failures, counts and limitations. No full local repository-suite, live provider, external-alarm, measured suspension or fleet-savings claim. Fresh final-head CI/security, exact-head Greptile 5/5 and actionable Ponytail findings remain separate merge gates. ## Risks - Live publication is the existing server-owned executor in one process. Independent filesystem writers need an explicit signal contract. Startup reconstructs restart state. - Busy sessions and concrete inspection/removal failures retain retries. Unknown cleanup-source archives remain protected. - The existing scan enumerates directory names before applying its entry/removal budget. This PR preserves that behavior and does not claim a bounded total directory-listing allocation. - An in-process deadline cannot wake stopped compute. Cloud sleep-safety and external-alarm coverage stay unchanged. Physical evidence reclamation can wait for a supported host wake. - Backup expiration still has an hourly database eligibility scan. This slice removes only the archived-evidence duty from that loop and does not claim full instance suspension. ## Model Used OpenAI Codex (GPT-6-based coding agent), using repository inspection, normal Git merges, composition proofs and complete PostgreSQL/filesystem/startup suites. No subagents were used in this reconciliation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com>
211 lines
13 KiB
TypeScript
211 lines
13 KiB
TypeScript
import { test, expect, type APIResponse } from "@playwright/test";
|
|
|
|
async function json(response: APIResponse) {
|
|
const body = await response.text();
|
|
expect(response.ok(), `${response.url()}: ${response.status()} ${body}`).toBe(true);
|
|
return JSON.parse(body);
|
|
}
|
|
|
|
test("a task title survives an outage and saves after server recovery", async ({ page, request }) => {
|
|
const company = await json(await request.post("/api/companies", { data: { name: `Write recovery ${Date.now()}` } }));
|
|
try {
|
|
const issue = await json(await request.post(`/api/companies/${company.id}/issues`, {
|
|
data: { title: "Original task title", status: "backlog" },
|
|
}));
|
|
const other = await json(await request.post(`/api/companies/${company.id}/issues`, {
|
|
data: { title: "Another task stays unchanged", status: "backlog" },
|
|
}));
|
|
await page.goto(`/${company.issuePrefix}/issues/${issue.identifier}`);
|
|
await expect(page.getByTestId("task-chat-history-content")).toBeVisible({ timeout: 30_000 });
|
|
const header = page.getByTestId("issue-detail-header");
|
|
await expect(header.getByRole("heading", { name: issue.title, exact: true })).toBeVisible();
|
|
|
|
let unavailable = true;
|
|
let failedWrites = 0;
|
|
await page.route("**/api/**", async route => {
|
|
if (!unavailable) return route.continue();
|
|
if (route.request().method() === "PATCH" && [`/api/issues/${issue.id}`, `/api/issues/${issue.identifier}`].includes(new URL(route.request().url()).pathname)) {
|
|
failedWrites += 1;
|
|
}
|
|
if (!failedWrites) return route.continue();
|
|
await route.fulfill({ status: 503, contentType: "application/json", body: JSON.stringify({ error: "tenant_app_unavailable" }) });
|
|
});
|
|
await header.getByRole("heading", { name: issue.title, exact: true }).click();
|
|
const title = header.locator("textarea");
|
|
await title.fill("A title retained through the outage");
|
|
await title.press("Enter");
|
|
await expect.poll(() => failedWrites).toBeGreaterThan(0);
|
|
await expect(page.locator('[data-connection-status="reconnecting"]')).toBeVisible();
|
|
expect((await json(await request.get(`/api/issues/${issue.id}`))).title).toBe(issue.title);
|
|
await expect(header).toBeVisible();
|
|
|
|
unavailable = false;
|
|
await expect.poll(async () => (await json(await request.get(`/api/issues/${issue.id}`))).title,
|
|
{ timeout: 30_000 }).toBe("A title retained through the outage");
|
|
expect((await json(await request.get(`/api/issues/${other.id}`))).title).toBe(other.title);
|
|
await page.reload();
|
|
await expect(header.getByRole("heading", { name: "A title retained through the outage", exact: true })).toBeVisible();
|
|
expect((await json(await request.get(`/api/issues/${issue.id}`))).status).toBe("backlog");
|
|
} finally {
|
|
await request.patch(`/api/companies/${company.id}`, { data: { status: "archived" } });
|
|
}
|
|
});
|
|
|
|
test("two cached tasks keep separate title saves while the browser is offline", async ({ page, request }) => {
|
|
test.setTimeout(120_000);
|
|
const company = await json(await request.post("/api/companies", { data: { name: `Queued writes ${Date.now()}` } }));
|
|
try {
|
|
const first = await json(await request.post(`/api/companies/${company.id}/issues`, { data: { title: "First queued task", status: "todo" } }));
|
|
const second = await json(await request.post(`/api/companies/${company.id}/issues`, { data: { title: "Second queued task", status: "todo" } }));
|
|
await page.goto(`/${company.issuePrefix}/issues/${second.id}`);
|
|
await expect(page.getByRole("heading", { name: second.title, exact: true })).toBeVisible();
|
|
await page.getByRole("link", { name: "Tasks", exact: true }).first().click();
|
|
await page.getByRole("link").filter({ hasText: first.title }).first().click();
|
|
await expect(page.getByTestId("task-chat-history-content")).toBeVisible({ timeout: 30_000 });
|
|
const header = page.getByTestId("issue-detail-header");
|
|
await expect(header.getByRole("heading", { name: first.title, exact: true })).toBeVisible();
|
|
const dismissAnnouncement = page.getByRole("button", { name: "Dismiss announcement" });
|
|
if (await dismissAnnouncement.isVisible()) await dismissAnnouncement.click();
|
|
const secondLink = page.getByRole("link").filter({ hasText: second.title }).first();
|
|
await expect(secondLink).toBeVisible();
|
|
await page.context().setOffline(true);
|
|
await expect(page.locator('[data-connection-status="offline"]')).toBeVisible();
|
|
await header.getByRole("heading", { name: first.title, exact: true }).click();
|
|
await header.locator("textarea").fill("First title retained offline");
|
|
await header.locator("textarea").press("Enter");
|
|
await secondLink.focus();
|
|
await secondLink.press("Enter", { timeout: 5_000 });
|
|
await expect(page).toHaveURL(new RegExp(`${second.id}$`));
|
|
await expect(page.getByTestId("task-chat-history-content")).toBeVisible({ timeout: 30_000 });
|
|
await expect(header.getByRole("heading", { name: second.title, exact: true })).toBeVisible();
|
|
await header.getByRole("heading", { name: second.title, exact: true }).click();
|
|
await expect(header.locator("textarea")).toHaveValue(second.title);
|
|
await header.locator("textarea").fill("Second title retained offline");
|
|
await header.locator("textarea").press("Enter");
|
|
expect((await json(await request.get(`/api/issues/${first.id}`))).title).toBe(first.title);
|
|
expect((await json(await request.get(`/api/issues/${second.id}`))).title).toBe(second.title);
|
|
await page.context().setOffline(false);
|
|
await expect.poll(async () => (await json(await request.get(`/api/issues/${first.id}`))).title, { timeout: 30_000 }).toBe("First title retained offline");
|
|
await expect.poll(async () => (await json(await request.get(`/api/issues/${second.id}`))).title, { timeout: 30_000 }).toBe("Second title retained offline");
|
|
await page.reload();
|
|
await expect(header.getByRole("heading", { name: "Second title retained offline", exact: true })).toBeVisible();
|
|
} finally {
|
|
await page.context().setOffline(false).catch(() => {});
|
|
await request.patch(`/api/companies/${company.id}`, { data: { status: "archived" } });
|
|
}
|
|
});
|
|
|
|
test("a queued startup save survives the first session lookup during an API outage", async ({ page, request }) => {
|
|
const company = await json(await request.post("/api/companies", { data: { name: `Startup recovery ${Date.now()}` } }));
|
|
const session = await json(await request.get("/api/auth/get-session"));
|
|
let releaseSession!: () => void;
|
|
const sessionReady = new Promise<void>(resolve => { releaseSession = resolve; });
|
|
const savedTitle = "Saved through first session lookup";
|
|
let apiUnavailable = false;
|
|
try {
|
|
const issue = await json(await request.post(`/api/companies/${company.id}/issues`, {
|
|
data: { title: "Startup original title", status: "backlog" },
|
|
}));
|
|
await page.route("**/api/**", async route => {
|
|
const path = new URL(route.request().url()).pathname;
|
|
if (apiUnavailable && (path === "/api/health" || (route.request().method() === "PATCH" && path.startsWith("/api/issues/")))) {
|
|
return route.fulfill({ status: 503, contentType: "application/json", body: JSON.stringify({ error: "tenant_app_unavailable" }) });
|
|
}
|
|
await route.continue();
|
|
});
|
|
await page.route("**/api/auth/get-session", async route => {
|
|
await sessionReady;
|
|
await route.fulfill({ status: 200, contentType: "application/json", body: JSON.stringify(session) });
|
|
});
|
|
await page.goto(`/${company.issuePrefix}/issues/${issue.identifier}`);
|
|
const header = page.getByTestId("issue-detail-header");
|
|
await expect(header.getByRole("heading", { name: issue.title, exact: true })).toBeVisible();
|
|
apiUnavailable = true;
|
|
await page.evaluate(() => window.dispatchEvent(new Event("offline")));
|
|
await expect(page.locator('[data-connection-status="offline"]')).toBeVisible();
|
|
await header.getByRole("heading", { name: issue.title, exact: true }).click();
|
|
await header.locator("textarea").fill(savedTitle);
|
|
await header.locator("textarea").press("Enter");
|
|
expect((await json(await request.get(`/api/issues/${issue.id}`))).title).toBe(issue.title);
|
|
releaseSession();
|
|
await expect.poll(() => page.evaluate(({ owner, title }) => Object.keys(localStorage).some(key =>
|
|
key.startsWith(`paperclip:inline-draft:v2:${owner}:`) && JSON.parse(localStorage.getItem(key) ?? "null")?.draft === title),
|
|
{ owner: session.user.id, title: savedTitle })).toBe(true);
|
|
apiUnavailable = false;
|
|
await page.evaluate(() => window.dispatchEvent(new Event("online")));
|
|
await expect.poll(async () => (await json(await request.get(`/api/issues/${issue.id}`))).title,
|
|
{ timeout: 15_000 }).toBe(savedTitle);
|
|
await page.reload();
|
|
await expect(header.getByRole("heading", { name: savedTitle, exact: true })).toBeVisible();
|
|
} finally {
|
|
releaseSession();
|
|
apiUnavailable = false;
|
|
await page.evaluate(() => window.dispatchEvent(new Event("online"))).catch(() => {});
|
|
await request.patch(`/api/companies/${company.id}`, { data: { status: "archived" } });
|
|
}
|
|
});
|
|
|
|
test("replaying an accepted comment does not reopen completed work", async ({ page, request }) => {
|
|
const company = await json(await request.post("/api/companies", { data: { name: `Comment recovery ${Date.now()}` } }));
|
|
try {
|
|
const issue = await json(await request.post(`/api/companies/${company.id}/issues`, {
|
|
data: { title: "Completed work stays completed", status: "backlog", assigneeAgentId: null, assigneeUserId: null },
|
|
}));
|
|
const body = "A comment whose response was lost";
|
|
const clientRequestId = crypto.randomUUID();
|
|
const first = await json(await request.post(`/api/issues/${issue.id}/comments`, { data: { body, clientRequestId } }));
|
|
const agent = await json(await request.post(`/api/companies/${company.id}/agents`, {
|
|
data: { name: "Completed comment worker", role: "engineer", adapterType: "process", runner: "legacy",
|
|
adapterConfig: { command: "/usr/bin/true" }, runtimeConfig: { heartbeat: { enabled: false } } },
|
|
}));
|
|
await json(await request.patch(`/api/issues/${issue.id}`, { data: { status: "done", assigneeAgentId: agent.id } }));
|
|
const receipt = await json(await request.post(`/api/issues/${issue.id}/comments`, { data: { body, clientRequestId } }));
|
|
expect(receipt.id).toBe(first.id);
|
|
expect((await json(await request.get(`/api/issues/${issue.id}`))).status).toBe("done");
|
|
const conflict = await request.post(`/api/issues/${issue.id}/comments`, { data: { body: "Different request", clientRequestId } });
|
|
expect(conflict.status()).toBe(409);
|
|
expect((await json(await request.get(`/api/issues/${issue.id}`))).status).toBe("done");
|
|
const comments = await json(await request.get(`/api/issues/${issue.id}/comments`));
|
|
expect(comments.filter((comment: { clientRequestId?: string }) => comment.clientRequestId === clientRequestId)).toHaveLength(1);
|
|
const activity = await json(await request.get(`/api/issues/${issue.id}/activity`));
|
|
expect(activity.filter((entry: { action: string; details?: { commentId?: string } }) =>
|
|
entry.action === "issue.comment_added" && entry.details?.commentId === first.id)).toHaveLength(1);
|
|
await page.goto(`/${company.issuePrefix}/issues/${issue.id}`);
|
|
await expect(page.getByRole("heading", { name: issue.title, exact: true })).toBeVisible();
|
|
await expect(page.getByText(body, { exact: true })).toBeVisible();
|
|
} finally {
|
|
await request.patch(`/api/companies/${company.id}`, { data: { status: "archived" } });
|
|
}
|
|
});
|
|
|
|
test("startup work retries a failed session lookup while other APIs stay healthy", async ({ page, request }) => {
|
|
const company = await json(await request.post("/api/companies", { data: { name: `Session recovery ${Date.now()}` } }));
|
|
const session = await json(await request.get("/api/auth/get-session"));
|
|
let healthy = false, saving = false, failedAfterSave = 0;
|
|
try {
|
|
const issue = await json(await request.post(`/api/companies/${company.id}/issues`, {
|
|
data: { title: "Before session recovery", status: "backlog" },
|
|
}));
|
|
await page.route("**/api/auth/get-session", async route => {
|
|
if (healthy) return route.fulfill({ status: 200, json: session });
|
|
if (saving) failedAfterSave += 1;
|
|
await route.fulfill({ status: 500, json: { error: "session_lookup_failed" } });
|
|
});
|
|
await page.goto(`/${company.issuePrefix}/issues/${issue.id}`);
|
|
await expect(page.getByTestId("task-chat-history-content")).toBeVisible({ timeout: 30_000 });
|
|
const header = page.getByTestId("issue-detail-header");
|
|
await header.getByRole("heading", { name: issue.title, exact: true }).click();
|
|
await header.locator("textarea").fill("Saved after session recovery");
|
|
saving = true;
|
|
await header.locator("textarea").press("Enter");
|
|
await expect.poll(() => failedAfterSave).toBeGreaterThan(0);
|
|
healthy = true;
|
|
await expect.poll(async () => (await json(await request.get(`/api/issues/${issue.id}`))).title,
|
|
{ timeout: 15_000 }).toBe("Saved after session recovery");
|
|
await page.reload();
|
|
await expect(header.getByRole("heading", { name: "Saved after session recovery", exact: true })).toBeVisible();
|
|
} finally {
|
|
await request.patch(`/api/companies/${company.id}`, { data: { status: "archived" } });
|
|
}
|
|
});
|