Files
PaperClipAI/tests/runner-e2e/zip-inspection-shutdown.test.ts
T
DottaandPaperclip 556b249897 fix: preserve exact Markdown in native document writes (#20605)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents save Markdown documents through native tools and REST routes.
> - The native write_document handler trims the supplied body.
> - This removes final newlines and can change Markdown whitespace.
> - This pull request preserves the original body after required-content
validation.
> - Agents can now write and read the same Markdown string across these
paths.

## Linked Issues or Issue Description

**What happened?**

The native write_document tool removes leading and trailing whitespace.
A body with one final newline loses that newline on read-back. The REST
document PUT and write_task_document preserve it.

**Expected behavior**

The native write_document tool must preserve the supplied Markdown body
exactly. Required-content checks and key and title normalization must
stay in place. REST-backed paths retain surrounding whitespace and keep
their existing escaped-line-break conversion for single-line payloads.

**Steps to reproduce**

1. Write a disposable document through write_document with a final
newline.
2. Read the document and compare the full body with the supplied string.
3. Repeat through write_task_document and REST. These paths preserve the
string.

Related work: Refs #18824. That change aligns document keys. This change
preserves body strings. A search of open and closed PRs found no
duplicate fix.

## What Changed

- Use a separate untrimmed required-string helper for the document body.
Keep the existing requiredString callback signature and all other field
normalization.
- Add real tool-handler, database, and authenticated HTTP tests for
create, update, read-back, and replay. Cover leading spaces, hard
breaks, blank lines, final newlines, no final newline, Unicode, and
mixed line endings.
- Keep blank and invalid bodies rejected. Document whitespace
preservation and the existing REST escaped-line-break exception.
- Fix the Runner support archive-scanner shutdown defect exposed by CI:
rejected ZIP scanners now receive SIGKILL so an ignored SIGTERM cannot
leave publication waiting for process closure. Keep all secret and
archive limits unchanged. A real child-process regression fails before
the change and passes afterward.
- Address all three initial review findings: callback type
compatibility, contribution comment policy, and the REST documentation
limit.

## Verification

Final tested head: `c9ff8cf3f83aa19e5d8313c65816a3a2e6562368`.

- All applicable [CI
checks](https://github.com/paperclipai/paperclip/actions/runs/43019885807)
passed. This includes typecheck, build, Runner support, Rust, native
Runner, general tests, serialized suites, all eight E2E shards, and the
required aggregate gates.
- [Greptile
review](https://github.com/paperclipai/paperclip/runs/130520170309)
passed on this head with confidence 5/5. All three review threads are
resolved. The independent Paperclip review also reports 5/5 with no
findings.
- Red at `7368a0b15c2f02107778a961144d86e2eb59a952`: `pnpm --dir server
exec vitest run
src/services/native-runtime/task-document-body.integration.test.ts`
failed because write_document removed the final newline. Cross-task and
REST cases passed.
- Green on the final source: 176 tests passed across the document body,
document key, native tool authority, task scope, document service, and
document route suites. The tests cover create, update, exact read-back,
replay, blank rejection, revision guards, and company scope.
- The ZIP shutdown regression failed before repair with “scanner did not
stop”. After repair, 54 tests passed across zip-inspection-shutdown,
private-reporter-artifacts, manual-private-cell-report-assets, and
private-evidence-budget. The regression uses a real process that ignores
SIGTERM.
- Earlier CI runs had two archive-related timeouts. The source repair
above replaces blind reruns. The Runner support suite and aggregate
verify gate now pass.
- A disposable live native document reproduced the original whitespace
loss. REST preserved the same string. The fixed handler passed local
integration tests. The running service has not been deployed.
- Local environment: Linux x64, Node 25.9.0, Vitest 5.0.3, and embedded
PostgreSQL. Local Rust preparation was unavailable because cargo is
absent. Final-head CI passed Rust verification. `pnpm verify --explain`
requested broad CI because evidence.ts is a global input. Required CI
supplied that coverage.

## Risks

Low risk. The change retains required-content validation. The product
change affects document body whitespace handling. The CI repair
forcefully stops only already-rejected archive scanners; it retains
fail-closed publication and all existing scan limits. It does not change
schema, authorization, revision guards, receipts, or persistence.

## Model Used

CI repair continuation: GPT-6.1, using repository tools and code
execution.


OpenAI Codex, GPT-6 (the exact serving model ID and context window were
not exposed in this session). Used reasoning, repository tools, and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-11 06:38:32 -06:00

43 lines
1.7 KiB
TypeScript

import { chmod, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { expect, it, vi } from "vitest";
import { inspectZip } from "./evidence.js";
it.skipIf(process.platform === "win32")("finishes a rejected ZIP scan even when the scanner ignores SIGTERM", async () => {
const root = await mkdtemp(path.join(os.tmpdir(), "zip-shutdown-"));
const source = path.join(root, "report.zip");
const executable = path.join(root, "unzip");
const secret = "synthetic-zip-secret-928472";
await writeFile(executable, `#!${process.execPath}
const fs = require('node:fs');
process.on('SIGTERM', () => {});
fs.writeFileSync(process.argv[3] + '.pid', String(process.pid));
process.stdout.write(${JSON.stringify(secret)});
setInterval(() => {}, 1000);
`);
await chmod(executable, 0o700);
vi.stubEnv("PATH", `${root}${path.delimiter}${process.env.PATH ?? ""}`);
let timer: ReturnType<typeof setTimeout> | undefined;
let stopped = false;
const scan = inspectZip(source, [secret]).then(result => { stopped = true; return result; });
try {
const result = await Promise.race([
scan,
new Promise<string>(resolve => { timer = setTimeout(() => resolve("scanner did not stop"), 2000); }),
]);
expect(result).toBe("exact secret value");
} finally {
clearTimeout(timer);
vi.unstubAllEnvs();
const pid = Number(await readFile(`${source}.pid`, "utf8").catch(() => "0"));
if (!stopped && pid > 0) {
try { process.kill(pid, "SIGKILL"); } catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ESRCH") throw error;
}
}
await scan;
await rm(root, { recursive: true, force: true });
}
});