mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 16:58:52 +01:00
test(server): avoid deep traversal in durable file assertions (#18533)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Cloud web servers restore agent files from shared storage after replacement. > - The durability test checks restoration, corruption, cancellation and retry. > - Its generic equality assertions traverse every byte in many large buffers. > - The post-merge check for #18531 exceeded the 15-second test timeout. > - This change uses native buffer equality and keeps every byte check. ## Linked Issues or Issue Description Refs #18531. The post-merge Cloud-source verification run timed out in the new cold-folder test. Other shards passed. The unchanged test passed locally. This change removes avoidable assertion cost; it does not increase test or production timeouts. ## What Changed - Use `Buffer.equals` for the cold-folder content assertions. - Keep the concurrency, cancellation, cleanup, old-projection and retry assertions. - Record packaged-image proof, the hosted deployment state and the original CI failure in the existing plan. ## Verification - Eight durability tests passed against a real local PostgreSQL fixture. Object storage uses controlled asynchronous streams. - The same test file took 10.03 seconds before and 5.90 seconds after this change. The changed cold-folder case took 561 ms after the change. - The first local attempt could not start embedded PostgreSQL and ran no tests. The existing external test database enabled the comparison. - Original failed CI: https://github.com/paperclipai/paperclip/actions/runs/41296253723 . That failure remains in the qualification record. - Server typecheck and build passed. All latest-head PR checks passed, including the full test, typecheck, build and browser gates. One cancelled runner job was retried after its log showed an infrastructure shutdown; the retry passed. This PR changes no production code or database schema. ## Risks - A failed content check now reports a boolean mismatch instead of a large byte diff. Equality still covers buffer length and every byte. - Local timing is evidence for the fixture, not a general CI performance guarantee. - Hosted active-agent and rollout acceptance remain incomplete as recorded in the plan. ## Model Used OpenAI Codex, based on GPT-6. The exact runtime snapshot and context-window size are not exposed. Used code editing, shell execution and test analysis. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
1d348bbef0
commit
247552fe07
2 files changed
+90
-4
No files matched your search
@@ -442,3 +442,89 @@ the new launcher. Existing live failure/cleanup receipts remain unchanged.
|
||||
Still required: active execution and continuation, cross-process follow-up,
|
||||
migrated packaged overlap, ten staging replacements including rollback, browser
|
||||
and sleep/wake/idle verification there, and one internal production canary.
|
||||
|
||||
### 2026-11-01 10:12 UTC hosted qualification preparation
|
||||
|
||||
Core maintenance #18529 merged as `d941ed3879`; bounded cold restore #18531
|
||||
merged as `1d348bbef0`. Both passed all current-head CI and Greptile 5/5 with
|
||||
no unresolved review threads. This checkout now starts from the latter merge.
|
||||
Published image and Cloud composition qualification are still pending.
|
||||
|
||||
The earlier staging target was superseded: its historical local execution
|
||||
workspaces require their own migration review. It was unpinned at 09:59 UTC
|
||||
without a volume or deployment mutation by this chat. The superseded browser
|
||||
fixture auto-assigned an agent despite the composer showing no assignee;
|
||||
that attempted run failed. The task was then unassigned, paused and hidden.
|
||||
Do not count this as successful agent execution or assume it incurred no cost.
|
||||
Private setup and restoration receipts remain under `/private/tmp/paperclip-zdt-`.
|
||||
|
||||
A dedicated staging fixture was provisioned through the normal signup API.
|
||||
Its concrete host and stack identity remain in the private qualification receipt.
|
||||
It is pinned through the audited user-managed API, and the existing operator
|
||||
was added through the audited membership API. Normal organization-launcher
|
||||
entry reached its dashboard. A separate synthetic fixture owner authenticated
|
||||
through the normal Cloud entry-code/session flow. The tenant serves Core
|
||||
`16787e4167` in Cloud distribution `cloud-41292158258-1`.
|
||||
|
||||
The fixture contains a paused, wake-disabled agent with 34 synthetic files
|
||||
(524,936 bytes), all read back through the API, and unassigned backlog task
|
||||
with one saved comment. The data will exercise volume enrollment and
|
||||
subsequent replacement. Receipt: `/private/tmp/paperclip-zdt-hosted-fixture.json`.
|
||||
No hosted enrollment or overlap has yet run; the original volume is attached.
|
||||
This controlled fixture does not qualify arbitrary historic local workspaces.
|
||||
The saved E2E model credential remains an external blocker for paid-agent proof.
|
||||
|
||||
The published Linux/amd64 maintenance image for `d941ed3879`, digest
|
||||
`sha256:5370ef6c938a639d5994bb20fb85e0a07b9210b264389b2076cc5890f66ea686`,
|
||||
passed the real-volume migration rehearsal. A legacy container wrote two files
|
||||
and generated its decision-signing key on its volume. After confirmed shutdown,
|
||||
the packaged maintenance entry migrated the files, verified shared objects and
|
||||
exported that exact original key; unauthenticated status was denied and ordinary
|
||||
app routes were absent. Both predecessor containers and the entire volume were
|
||||
then deleted. A new empty-home container retained the original session and read
|
||||
the migrated files. Two further replacements served 111 reads and 23 saves with
|
||||
zero failures; unready replacement refusal and clean exits passed. Owned test
|
||||
resources were removed. Receipt: `/private/tmp/paperclip-zdt-container-7r1Nwv/evidence.json`.
|
||||
This small-file Core image rehearsal complements the multi-page migration tests;
|
||||
it does not qualify Cloud routing, large cold folders or active agents.
|
||||
|
||||
### 2026-11-01 10:29 UTC packaged Cloud proof
|
||||
|
||||
The exact Cloud composition `cloud-41296664811-1`, Core `d941ed3879`,
|
||||
digest `sha256:bce104e05623bd365740013c8fbb87680da7e152f9c94bd099884a9a5f6367cc`,
|
||||
passed the same real-volume maintenance, key preservation, full volume removal
|
||||
and empty-home replacement rehearsal: 121 reads, 25 saves, zero failures.
|
||||
Receipt: `/private/tmp/paperclip-zdt-container-JBK8C7/evidence.json`.
|
||||
This verifies the private Cloud packaging contains the working maintenance entry;
|
||||
it does not substitute for Railway routing or active-agent acceptance.
|
||||
|
||||
The dedicated hosted fixture passed a ten-second authenticated baseline with
|
||||
19 reads, three saves, one WebSocket connection and all 34 file hashes intact.
|
||||
Normal browser entry and an issue-title edit persisted. Receipt:
|
||||
`/private/tmp/paperclip-zdt-hosted-smoke-s1Q33a/evidence.json`.
|
||||
The first named-stack deployment was refused before creating a campaign because
|
||||
an existing fleet campaign still reserved the fixture. That campaign completed
|
||||
at 10:24:40 UTC and skipped the now-pinned fixture. Its operation lease was not
|
||||
held; the campaign reservation was the actual conflict. The refused workflow
|
||||
was explicitly retried, and the named-stack request
|
||||
is deploying the qualified maintenance composition. No hosted enrollment has
|
||||
started and no controller allowlist has changed yet.
|
||||
|
||||
The named-stack request completed at 10:31 UTC with serving commit verification
|
||||
for `d941ed3879`. A new continuous deployment campaign then started. The controller
|
||||
restart guard correctly refuses the allowlist change until that campaign finishes.
|
||||
The original volume remains attached and the fixture is pinned.
|
||||
|
||||
Core `1d348bbef0` published its image, but its post-merge Cloud-source check
|
||||
timed out after 15 seconds in the new cold-folder regression. All other shards
|
||||
passed. The unchanged test passed locally against the owned PostgreSQL fixture;
|
||||
the eight-test file took 10.03 seconds. Native `Buffer.equals` retains complete
|
||||
byte equality while avoiding generic assertion traversal over 40 large buffers.
|
||||
After that assertion-only change, all eight tests passed in 5.90 seconds, with
|
||||
the cold-folder case taking 561 ms. The 15-second test timeout and 30-second
|
||||
production deadline remain unchanged. The original CI failure is retained at
|
||||
`/private/tmp/paperclip-zdt-final-source-verification-failed.log`; local before
|
||||
and after logs are `/private/tmp/paperclip-zdt-buffer-test-{before,after}-pg.log`.
|
||||
A first local attempt failed before tests because embedded PostgreSQL could not
|
||||
start; the existing external test database enabled the comparison. The failed
|
||||
post-merge shard was retried once while the assertion fix proceeds through review.
|
||||
@@ -115,12 +115,12 @@ describe("durable agent file authority", () => {
|
||||
expect(await migrateAgentFiles(db, { companyId, apply: true, writersStopped: true })).toMatchObject({ pageReady: true });
|
||||
await replaceFilesystem();
|
||||
objects.readDelayMs = 25;
|
||||
expect(await store().read(companyId, agentId, "nested/file-39.bin", actor())).toEqual(contents);
|
||||
expect((await store().read(companyId, agentId, "nested/file-39.bin", actor()))?.equals(contents)).toBe(true);
|
||||
expect(objects.peakReads).toBeGreaterThan(1);
|
||||
expect(objects.peakReads).toBeLessThanOrEqual(8);
|
||||
expect(objects.activeReads).toBe(0);
|
||||
for (let index = 0; index < 40; index++) {
|
||||
expect(await fs.readFile(path.join(root, `nested/file-${index}.bin`))).toEqual(contents);
|
||||
expect((await fs.readFile(path.join(root, `nested/file-${index}.bin`))).equals(contents)).toBe(true);
|
||||
}
|
||||
|
||||
// A corrupt replacement must preserve the old projection and await every
|
||||
@@ -130,9 +130,9 @@ describe("durable agent file authority", () => {
|
||||
await expect(store().read(companyId, agentId, "nested/file-39.bin", actor())).rejects.toThrow(/committed (size|digest)/);
|
||||
expect(objects.activeReads).toBe(0);
|
||||
expect((await fs.readdir(path.dirname(root))).filter(name => name.startsWith(".agent-files-"))).toEqual([]);
|
||||
expect(await fs.readFile(path.join(root, "nested/file-39.bin"))).toEqual(contents);
|
||||
expect((await fs.readFile(path.join(root, "nested/file-39.bin"))).equals(contents)).toBe(true);
|
||||
objects.corruptReads = false;
|
||||
expect(await store().read(companyId, agentId, "nested/file-39.bin", actor())).toEqual(contents);
|
||||
expect((await store().read(companyId, agentId, "nested/file-39.bin", actor()))?.equals(contents)).toBe(true);
|
||||
});
|
||||
|
||||
it("does not promote a rejected first save through subsequent local adoption", async () => {
|
||||
|
||||
Reference in new issue
Block a user