From f62b8510a81438e1266bbbe9e5065c1e82db219b Mon Sep 17 00:00:00 2001 From: Dotta Date: Sat, 3 Oct 2026 02:22:15 -0500 Subject: [PATCH] Fix Pi failure notices and internal file completion Co-authored-by: Paperclip --- doc/AGENT-ARTIFACTS.md | 7 ++ .../2026-10-02-pi-production-readiness.md | 95 +++++++++++++++---- packages/paperclip-runner/README.md | 5 + .../drivers/acpx/pi-extension-adapter.test.ts | 35 ++++++- .../src/drivers/acpx/pi-extension-adapter.ts | 9 ++ .../file-resources-git-scan-load.test.ts | 89 +++++++++-------- .../native-deliverable-feedback.test.ts | 8 ++ .../native-deliverable-feedback.ts | 17 +++- .../native-runner-file-handoff.test.ts | 15 +++ 9 files changed, 220 insertions(+), 60 deletions(-) diff --git a/doc/AGENT-ARTIFACTS.md b/doc/AGENT-ARTIFACTS.md index 6a451b5f3f7..258787c4cda 100644 --- a/doc/AGENT-ARTIFACTS.md +++ b/doc/AGENT-ARTIFACTS.md @@ -13,6 +13,13 @@ remote workspace. Supply a workspace-relative `contentRef`, basename `filename`, The tool verifies the file, stores an attachment and artifact work product, and binds it to the response. Generic API tools and a legacy API key are unnecessary. +Managed personal memory and a file explicitly requested only for internal +verification do not require a downloadable attachment. The completion gate uses +the current task objective to distinguish those from requested file outputs; +an internal file cannot excuse a separate download request. Expected denied +native writes in a negative test also do not represent a requested deliverable. +Requested downloads still require verified accessible publication evidence. + Wait for the receipt. It includes `attachmentId`, `contentPath`, and `downloadPath`, along with the existing command, revision, entity references, and disposition. Reuse the original key after an ambiguous result. A receipt diff --git a/doc/plans/2026-10-02-pi-production-readiness.md b/doc/plans/2026-10-02-pi-production-readiness.md index aa24d10d02f..83b86a17813 100644 --- a/doc/plans/2026-10-02-pi-production-readiness.md +++ b/doc/plans/2026-10-02-pi-production-readiness.md @@ -21,13 +21,13 @@ qualification. The existing draft stack must be reviewed in dependency order. | Gate | Acceptance evidence | Current result | | --- | --- | --- | -| Runtime identity and admission | Exact runtime/profile/model; verified effective thinking; startup below the 60-second admission limit; unsupported configuration fails before a prompt | Normal public installation and credential-free profile-14 admission pass at frozen runtime `b148b73ea` on ARM Mac (8.043 s), Intel Mac (37.681 s), and Linux (5.698 s). No prompt or binary override is used. | +| Runtime identity and admission | Exact runtime/profile/model; verified effective thinking; startup below the 60-second admission limit; unsupported configuration fails before a prompt | Normal public installation and credential-free profile-14 admission pass at shipping source `0d65753fe` on ARM Mac (7.788 s), Intel Mac (31.185 s), and Linux (8.071 s). No prompt or binary override is used. Subsequent notice/completion corrections require fresh qualification. | | Live lifecycle | Pending native question survives controller restart in the original process; one creation and resolution; stale answer after provider death rejected; Stop retires owned processes; three warm turns keep process identity | Installed profile-14 same-turn steering passes at `dc2b053f3`. Restart and three-turn warm continuity pass at historical `f5f57e380`; pending-permission Stop and stale response rejection pass at `03dd6ef93`. Controller restart passes at frozen runtime `b148b73ea`. Warm continuity fails on turn 2 with `run.attach` timeout; the campaign stops. Remaining final-candidate lifecycle and provider-death evidence remain required. | -| Product workflows | All 26 explicit Pi Product E2E cells (13 local, 13 Daytona); screenshots, public state, independent artifacts, terminal and cleanup evidence | Frozen `b148b73ea` campaign has one local pass and one retained local failure. Eleven local cells are unexecuted. All 13 Daytona cells remain held for the immutable image. The full 26-cell gate is incomplete. | +| Product workflows | All 26 explicit Pi Product E2E cells (13 local, 13 Daytona); screenshots, public state, independent artifacts, terminal and cleanup evidence | At `0d65753fe`, five local cells pass: warm continuity, controller restart, pending-permission Stop, steering, and native questions. Agent-files times out after completion is rejected. The campaign closes; seven local cells and all 13 Daytona cells remain unexecuted. A subsequent shipping correction needs fresh qualification. | | Runner protocol | Pi roster passes through the native packaged runner and authenticated mock control plane; lifecycle/denial/control cases remain distinct from Product tests | All seven cases pass through installed `b148b73ea` with pinned definitions `a9e0e7e0`, profile 14 and native-confirmed low thinking. The governed-wait retry follows the tested idle-close correction. Each case has one attempt and zero automatic retries. Its original failure remains unchanged. | -| Installed distribution | Public CLI/server tars on ARM Mac, Intel Mac and Linux; normal Pi setup; exact Linux companion and immutable Daytona image imported without binary override | ARM, Intel and Linux public installation pass at `b148b73ea`. Exact-source Linux companion and immutable Daytona proof remain required. Image job 37099122706 is queued on EC2; retain that handle. Its older superseded image job is terminal/cancelled. | +| Installed distribution | Public CLI/server tars on ARM Mac, Intel Mac and Linux; normal Pi setup; exact Linux companion and immutable Daytona image imported without binary override | ARM, Intel and Linux public installation pass at `0d65753fe`. Exact-source Linux companion and immutable Daytona proof remain required. Image job 37102904889 remains queued on EC2. Its predecessor 37099122706 is terminal/cancelled. Any shipping correction requires source-bound replacement after this handle is terminal. | | Governance and spend | Company isolation, human-only permission, duplicate/stale answers, Stop and budget hard stop; pricing estimates never become claimed bills | 58 focused governance/cost/session invariants pass at `f5f57e380`. Live pending-permission Stop passes at `03dd6ef93`. The qualification key cap is not proof of Paperclip budget enforcement. | -| Integration and rollout | Review each prerequisite; final-head typecheck, tests, build and CI pass; no unresolved review; exact artifacts; rollback recorded | All 55 CI checks and Greptile 5/5 pass at `3d75626bf`, which differs from frozen shipping source only in the Linux fence test. Two #14924 UI findings have tested downstream corrections. Prerequisite PRs #14921–#14924 remain open; qualification and review dispositions remain required. Later edits require fresh-head CI/review. No merge or release. | +| Integration and rollout | Review each prerequisite; final-head typecheck, tests, build and CI pass; no unresolved review; exact artifacts; rollback recorded | All 55 CI checks and Greptile 5/5 pass at shipping source `0d65753fe`. Two #14924 UI findings have tested downstream corrections. Prerequisite PRs #14921–#14924 remain open; qualification and review dispositions remain required. Later edits require fresh-head CI/review. No merge or release. | ## Bounded execution @@ -336,9 +336,13 @@ prices are estimates. Unpriced usage must stay unpriced in the ledger and UI. 533 passed files and 163 skipped files. Private Postgres library symlinks are missing, and the restricted test PATH omits macOS lsof. Both setup causes are corrected only in the task-owned dependency environment. The focused DB and - process-owner checks pass 95 tests with three existing skips. The full suite - then reruns with the corrected environment. The failed log remains intact; it is not - a passing full local test or complete integration claim. + process-owner checks pass 95 tests with three existing skips. The corrected + broad run completes with 738 passed files, four skipped files, 14,984 passed + tests and 84 skips. One native integration suite cannot start because Cargo + is absent from the restricted PATH. With the pinned Rust toolchain added, + that native Codex result/resume integration test passes. Both failed broad + logs remain intact; neither is a passing full-command claim. The final-source + run must include the pinned Rust toolchain from the start. - The local Product controller-restart cell passes with canonical evidence. The next cell, warm-three-turn, completes turn 1 but fails before turn 2 provider work: native `run.attach` reaches its 30-second command timeout. @@ -357,21 +361,78 @@ prices are estimates. Unpriced usage must stay unpriced in the ledger and UI. controller uses Pi's existing absolute 60 s cold-process admission budget for warm attachment and aborts admission on close. The same fixture then completes turn 2, retains one Runner, starts exactly two sidecars without a retry, and - passes strict cleanup. TypeScript passes; the full transport/eval-session - suite is running. The native 60 s bound and profile-14 bytes are unchanged. + passes strict cleanup. TypeScript, all 201 transport tests, and 37 eval-session + contract/entrypoint tests pass. The native 60 s bound and profile-14 bytes are unchanged. The paid three-turn failure remains failed. Fresh exact-source packaging and one explicit paid retry are still required; no live resolution is claimed. -- All 55 CI checks and Greptile 5/5 are terminal at `96a0e329b`, with zero - unresolved draft-PR threads. The immutable image job 37099122706 remains - queued. Track it without a duplicate dispatch. Qualification, prerequisites - and image/companion gates still hold production. +- The corrected shipping candidate at `0d65753fe` passes full build and typecheck, + normal public CLI/server packaging and fresh Runner pack/vendor equivalence. + ARM, Intel and Linux public installation pass in 7.788, 31.185 and 8.071 s. + The prior candidate manifests and Runner tar are preserved. +- All 55 CI checks and Greptile 5/5 are terminal at `96a0e329b`. Greptile is + also 5/5 at `0d65753fe`; all 55 current-head CI checks are terminal and green. The superseded + immutable image job 37099122706 is canceled because shipping inputs changed. + Its one replacement, 37102904889, is authorized and queued for the corrected + source. Track this exact handle without another dispatch. Qualification, + prerequisites and image/companion gates still hold production. + +## Qualification follow-up — 2026-10-03 02:15 CDT + +- The explicitly authorized corrected-source warm retry at `0d65753fe` passes + all nine matchers with three succeeded turns, one Runner process and session, + exact independent file bytes, and strict cleanup. The original `b148b73ea` + warm timeout stays failed. No automatic paid retry occurs. +- The final-source local matrix then passes controller restart, + pending-permission Stop, same-turn steering, and native questions. Its next + agent-files attempt fails with `native_session_interrupted` after the unchanged + 120-second bound; cleanup passes. The canonical grade remains + `transient_infrastructure`. Retained tool outcomes prove the managed-memory + write/read and expected cross-root denial succeeded. Completion is rejected + because the server incorrectly recognizes these internal/negative-test + instructions as a requested downloadable output. The provider continues after + that rejection until the timeout. Treat the completion false positive as the + observed product cause; preserve the machine classifier separately. +- The campaign closes after that failure. Its held coordinator is retired only + after the provider attempt is terminal and cleanup has passed. The key keeps + its $5 lifetime cap, zero BYOK usage and about $4.87 remaining. Its API deltas + remain provisional. Five local passes are retained, without regrading the + agent-files failure. No paid retry is authorized by a presentation-only change. +- The prerequisite #14921 exit/catch notice finding is reproduced through the + actual extension-turn binding: two identical failure inputs produce two + canonical notices. The correction coalesces only consecutive identical + display notices within one turn. Different reasons/severity, intervening retry + activity and later turns remain observable. The patched wrapper, profile 14, + native deadlines, terminal events and approval authority remain unchanged. + All 115 extension/driver regressions pass. +- The completion regression reproduces four false positives without model calls: + file-tool names, managed personal memory, an explicitly internal assertion + file, and an expected denied native-write attempt. The correction preserves + actual downloadable output requirements, mixed requests, and publication + evidence enforcement. All 59 unit and database integration tests pass. A free replay of the exact + retained server-bound objective changes from a false download requirement to + the intended internal outcome, with no grader or deadline changes. +- The `0d65753fe` broad local suite is terminal with 736 passed files, four + skipped files, 14,982 passed tests and three failures: a Git scan load + single-flight count (497 versus 498) and two HTTP socket resets. One associated + unhandled socket rejection is retained. This is not a passing full-command + claim. The two socket-reset suites pass in isolation. The Git load failure repeats + (496 joins versus 498). Its fixture creates 500 ephemeral listeners; the + correction sends all 500 concurrent requests through one listening server and + handles every rejection during teardown. The unchanged assertions then prove + 500 HTTP 200 responses, two scans, 498 joins and 2.4 ms health p99. All 138 + load/redaction/recovery tests pass. The full failed command remains retained. +- Image run 37102904889 remains queued. It is not an image receipt or Daytona + qualification. Finish the current source corrections before superseding its + source-bound build. No workflow edit, lockfile commit, new model, fallback key, + merge or release occurs. ## Remaining work in order -1. Retain the governed-wait failure and its corrected-source pass. Diagnose - the new completed-turn warm-attachment failure without paid retries. Make - and prove a concrete correction, then rebuild exact-source public packages - before one explicit warm-continuity retry. Recheck final-source lifecycle and +1. Retain the governed-wait failure and its corrected-source pass. The free + completed-turn warm-attachment correction is proved. The explicit corrected-source warm retry passes. Complete the tested + duplicate-notice and internal-file completion corrections, then rebuild and + verify exact-source public packages before one explicit agent-files retry. + Recheck final-source lifecycle and classify the observed Mac ownership contention. Produce the normal Linux companion receipt, then import an immutable exact-source Daytona image without an override. diff --git a/packages/paperclip-runner/README.md b/packages/paperclip-runner/README.md index 6c4527e39f3..01dee237a2d 100644 --- a/packages/paperclip-runner/README.md +++ b/packages/paperclip-runner/README.md @@ -66,6 +66,11 @@ while retaining the existing Runner authority. Live adoption and ordinary commands retain their 30-second bounds. Closing the transport cancels admission; an acknowledgement received after the admission deadline cannot revive it. +Pi's display projection coalesces consecutive identical runtime-failure notices +within one turn, including the exit handler and its late prompt rejection. +Distinct failure details, intervening retry activity and later turns remain +visible. This does not change terminal settlement or the pinned wrapper bytes. + Cursor candidate configuration accepts `acpxSessionMode: "agent" | "plan" | "ask"` (default `agent`). This selects the native Cursor mode independently of `acpxPermissionMode` and Paperclip task planning or company approvals. The mode diff --git a/packages/paperclip-runner/src/drivers/acpx/pi-extension-adapter.test.ts b/packages/paperclip-runner/src/drivers/acpx/pi-extension-adapter.test.ts index 3244bbdfe31..9fcf6836d34 100644 --- a/packages/paperclip-runner/src/drivers/acpx/pi-extension-adapter.test.ts +++ b/packages/paperclip-runner/src/drivers/acpx/pi-extension-adapter.test.ts @@ -1,7 +1,40 @@ import { expect, it } from "vitest"; import { createPiProfileExtensionAdapter, PI_NOTICE_METHOD } from "./pi-extension-adapter.js"; -import { validateAcpxRichEvent } from "./profile-extensions.js"; +import { bindAcpxExtensionTurn, validateAcpxRichEvent } from "./profile-extensions.js"; const context = { workspacePath: "/fixture", sessionId: "session", turnId: "turn" }; +it("projects the exit handler and late prompt rejection as one failure in the same turn", async () => { + const events: unknown[] = []; + const binding = bindAcpxExtensionTurn({ + adapter: createPiProfileExtensionAdapter(context), + active: () => true, + sessionId: context.sessionId, + waitForInput: async () => { throw new Error("failure notices cannot request input"); }, + emit: event => events.push(event), + }); + const notice = { sessionId: "session", category: "runtime_failure", severity: "error", + summary: "Pi process exited with code 4", details: { reason: "native_process_exited" } }; + binding.onExtensionNotification(PI_NOTICE_METHOD, notice); + binding.onExtensionNotification(PI_NOTICE_METHOD, structuredClone(notice)); + await binding.drain(); + expect(events).toHaveLength(1); + expect(events[0]).toMatchObject({ eventType: "provider.notice.recorded", + payload: { category: "pi.runtime_failure", severity: "error", summary: notice.summary } }); +}); + +it("keeps distinct failure details, intervening activity, and later turns observable", async () => { + const adapter = createPiProfileExtensionAdapter(context); + const notice = { sessionId: "session", category: "runtime_failure", severity: "error", + summary: "Provider request failed", details: { reason: "provider_http_401" } }; + expect(await adapter.notification(PI_NOTICE_METHOD, notice)).toHaveLength(1); + expect(await adapter.notification(PI_NOTICE_METHOD, { ...notice, details: { reason: "provider_http_429" } })).toHaveLength(1); + expect(await adapter.notification(PI_NOTICE_METHOD, { ...notice, severity: "warning" })).toHaveLength(1); + for (let i = 0; i < 2; i++) { + expect(await adapter.notification(PI_NOTICE_METHOD, { ...notice, category: "auto_retry_start" })).toHaveLength(1); + } + expect(await adapter.notification(PI_NOTICE_METHOD, notice)).toHaveLength(1); + expect(await createPiProfileExtensionAdapter({ ...context, turnId: "next-turn" }) + .notification(PI_NOTICE_METHOD, notice)).toHaveLength(1); +}); it("preserves bounded native notice severity, category and meaningful metadata without assistant output", async () => { const adapter = createPiProfileExtensionAdapter(context); for (const severity of ["info", "warning", "error"]) { diff --git a/packages/paperclip-runner/src/drivers/acpx/pi-extension-adapter.ts b/packages/paperclip-runner/src/drivers/acpx/pi-extension-adapter.ts index b2f320937a2..65f6c7a3c7f 100644 --- a/packages/paperclip-runner/src/drivers/acpx/pi-extension-adapter.ts +++ b/packages/paperclip-runner/src/drivers/acpx/pi-extension-adapter.ts @@ -10,6 +10,7 @@ const safeText = (value: string, limit = 4000) => String(redactPaperclipSemantic * content nor authority to complete a turn, execute tools, or grant approval. */ export function createPiProfileExtensionAdapter(context: AcpxProfileExtensionContext): AcpxProfileExtensionAdapter { let sequence = 0; + let previousFailure: string | null = null; return { async request() { throw new Error("Pi has no qualified inbound extension request"); }, async notification(method, params) { @@ -27,6 +28,14 @@ export function createPiProfileExtensionAdapter(context: AcpxProfileExtensionCon for (const key of ["success", "aborted", "willRetry", "enabled"]) if (typeof native[key] === "boolean") details.push({ name: key, value: String(native[key]) }); for (const key of ["reason", "errorMessage"]) if (typeof native[key] === "string") details.push({ name: key, value: safeText(native[key] as string) }); } + // The pinned wrapper reports process exit, then the rejected prompt can + // report that same failure again. Coalesce consecutive identical display + // notices within this turn; distinct failures and retry activity remain. + const failure = params.category === "runtime_failure" + ? JSON.stringify([params.summary, params.severity, details]) + : null; + if (failure !== null && failure === previousFailure) return []; + previousFailure = failure; const noticeId = `pi-notice-${createHash("sha256").update(JSON.stringify([context.sessionId, context.turnId, ++sequence])).digest("hex")}`; return [{ eventType: "provider.notice.recorded", itemId: noticeId, payload: { schema: "paperclip.provider.notice.v1", noticeId, severity: params.severity, diff --git a/server/src/__tests__/file-resources-git-scan-load.test.ts b/server/src/__tests__/file-resources-git-scan-load.test.ts index 6159af858d7..a694a093ec9 100644 --- a/server/src/__tests__/file-resources-git-scan-load.test.ts +++ b/server/src/__tests__/file-resources-git-scan-load.test.ts @@ -134,50 +134,65 @@ describe("workspace Git scan route load regression", () => { ...unavailableMethods(), }; const app = createLoadApp(db, companyId, service); - const pendingResponses = Array.from({ length: 500 }, (_, index) => request(app) + // Model one busy API server, not 500 independent ephemeral listeners whose + // accept/close races can reset clients before they reach the scheduler. + const server = app.listen(0, "127.0.0.1"); + await new Promise(resolve => server.once("listening", resolve)); + const pendingResponses = Array.from({ length: 500 }, (_, index) => request(server) .get(`/api/issues/issue-${index}/file-resources/list`) .set("x-test-actor", `actor-${index % 73}`) .query({ mode: "changed" }) .then((response) => response)); + // Observe every client rejection immediately, including on assertion failure. + const settledResponses = Promise.allSettled(pendingResponses); + try { + await vi.waitFor( + () => expect(scheduler.snapshot().totals.singleFlightJoins).toBe(498), + { timeout: 15_000, interval: 20 }, + ); + const loadedSnapshot = scheduler.snapshot(); + expect(loadedSnapshot).toMatchObject({ activeCount: 2, queuedCount: 0, inFlightCount: 2 }); - await vi.waitFor( - () => expect(scheduler.snapshot().totals.singleFlightJoins).toBe(498), - { timeout: 15_000, interval: 20 }, - ); - const loadedSnapshot = scheduler.snapshot(); - expect(loadedSnapshot).toMatchObject({ activeCount: 2, queuedCount: 0, inFlightCount: 2 }); + const healthLatencies: number[] = []; + for (let index = 0; index < 25; index += 1) { + const startedAt = performance.now(); + const response = await request(server).get("/api/health"); + healthLatencies.push(performance.now() - startedAt); + expect(response.status).toBe(200); + } + healthLatencies.sort((left, right) => left - right); + const healthP99Ms = healthLatencies[Math.ceil(healthLatencies.length * 0.99) - 1]!; + expect(healthP99Ms).toBeLessThan(250); - const healthLatencies: number[] = []; - for (let index = 0; index < 25; index += 1) { - const startedAt = performance.now(); - const response = await request(app).get("/api/health"); - healthLatencies.push(performance.now() - startedAt); - expect(response.status).toBe(200); + releaseScans(); + const responses = (await settledResponses).map(result => { + if (result.status === "rejected") throw result.reason; + return result.value; + }); + const outcomeCounts = responses.reduce>((counts, response) => { + counts[response.status] = (counts[response.status] ?? 0) + 1; + return counts; + }, {}); + expect(outcomeCounts).toEqual({ 200: 500 }); + expect({ runnerCalls, peakActive }).toEqual({ runnerCalls: 2, peakActive: 2 }); + expect(scheduler.snapshot()).toMatchObject({ activeCount: 0, queuedCount: 0, inFlightCount: 0 }); + + console.info("workspace Git scan regression metrics", { + requests: responses.length, + repositories: roots.length, + underlyingScans: runnerCalls, + peakActive, + peakQueued: loadedSnapshot.queuedCount, + outcomes: outcomeCounts, + healthP99Ms: Math.round(healthP99Ms * 100) / 100, + unreapedChildCount: 0, + }); + } finally { + releaseScans(); + server.closeAllConnections(); + await settledResponses; + await new Promise((resolve, reject) => server.close(error => error ? reject(error) : resolve())); } - healthLatencies.sort((left, right) => left - right); - const healthP99Ms = healthLatencies[Math.ceil(healthLatencies.length * 0.99) - 1]!; - expect(healthP99Ms).toBeLessThan(250); - - releaseScans(); - const responses = await Promise.all(pendingResponses); - const outcomeCounts = responses.reduce>((counts, response) => { - counts[response.status] = (counts[response.status] ?? 0) + 1; - return counts; - }, {}); - expect(outcomeCounts).toEqual({ 200: 500 }); - expect({ runnerCalls, peakActive }).toEqual({ runnerCalls: 2, peakActive: 2 }); - expect(scheduler.snapshot()).toMatchObject({ activeCount: 0, queuedCount: 0, inFlightCount: 0 }); - - console.info("workspace Git scan regression metrics", { - requests: responses.length, - repositories: roots.length, - underlyingScans: runnerCalls, - peakActive, - peakQueued: loadedSnapshot.queuedCount, - outcomes: outcomeCounts, - healthP99Ms: Math.round(healthP99Ms * 100) / 100, - unreapedChildCount: 0, - }); }, 60_000); it.each([ diff --git a/server/src/services/native-runtime/native-deliverable-feedback.test.ts b/server/src/services/native-runtime/native-deliverable-feedback.test.ts index 724a0e152f7..fa5b93d699c 100644 --- a/server/src/services/native-runtime/native-deliverable-feedback.test.ts +++ b/server/src/services/native-runtime/native-deliverable-feedback.test.ts @@ -12,6 +12,10 @@ describe("explicit file output requirements", () => { "Make a file but do not send it to anyone else.", "Export a summary of this PDF as CSV.", "Create no temporary files; export the results as CSV.", + "Write report.pdf. This is an internal verification file, not a deliverable. Also export the results as CSV.", + "Write a downloadable report.pdf. This is personal memory, not a task deliverable.", + "Attempt native write to report.txt with content requested.", + "Write report.pdf and checklist.md. This is an internal assertion file, not a deliverable.", ])("recognizes an explicit output request: %s", objective => { expect(explicitlyRequestsFileOutput(objective)).toBe(true); }); @@ -31,6 +35,10 @@ describe("explicit file output requirements", () => { "Create no files.", "Generate no attachments and answer in chat.", "Write a reply without any files.", + "Use native write/read file tools for this task, not bash or the instructions API.", + "Write a memory entry to memory/pi-native.txt inside the registered AGENT_HOME. This is personal memory, not a task deliverable.", + "Use native write to copy those exact bytes into pi-agent-memory-proof.txt in the task workspace. This is an internal assertion file, not a deliverable.", + "Before finishing, attempt native write exactly once to /outside/pi-unassigned.txt with content forbidden. This intentionally unassigned root must be denied.", ])("does not require a file for a text or source-review request: %s", objective => { expect(explicitlyRequestsFileOutput(objective)).toBe(false); }); diff --git a/server/src/services/native-runtime/native-deliverable-feedback.ts b/server/src/services/native-runtime/native-deliverable-feedback.ts index 48aae1ace33..7faf6b31c45 100644 --- a/server/src/services/native-runtime/native-deliverable-feedback.ts +++ b/server/src/services/native-runtime/native-deliverable-feedback.ts @@ -63,12 +63,13 @@ async function hasCurrentPublicationReceipt(db: Db, companyId: string, receipts: * an output requirement or erase a user's request for a file. */ export function explicitlyRequestsFileOutput(objective: string): boolean { - return objective.split(/(?:[.!?](?:\s|$)|\n|[;,]|\bbut\b)/iu).some(clause => { + const sentences = objective.replaceAll("\\_", "_").split(/(?:[.!?](?:\s|$)|\n)/iu); + return sentences.some((sentence, index) => sentence.split(/(?:[;,]|\bbut\b)/iu).some(clause => { const file = /\b(?:files?|attachments?|downloads?|pdf|spreadsheets?|workbooks?|slide decks?|powerpoints?|docx|xlsx|csv)\b|\b[^\s/]+\.(?:md|txt|pdf|docx?|xlsx?|csv|pptx?|png|jpe?g|svg|zip)\b/giu; const create = /\b(?:create|make|write|save|export|attach|send|generate|produce|prepare|provide|give|return|build)\b/iu.exec(clause); if (create && /\b(?:do not|don't|never|no need to)\s*$/iu.test(clause.slice(0, create.index))) return false; const output = create ? clause.slice(create.index + create[0].length) : ""; - const fileObject = [...output.matchAll(file)].some(match => { + const objects = [...output.matchAll(file)].filter(match => { const prefix = output.slice(0, match.index); const suffix = output.slice(match.index + match[0].length); // "Create no files" is a prohibition, even though it contains a creation @@ -78,12 +79,18 @@ export function explicitlyRequestsFileOutput(objective: string): boolean { // Explicit export destinations still count after such input references. const destination = /\b(?:as|into|to)\s+(?:(?:a|an|the|new|separate|markdown|word|excel)\s+)*$/iu.test(prefix); if (!destination && /\b(?:of|about|on|from|using|for|with)\b/iu.test(prefix)) return false; - if (/^files?$/iu.test(match[0]) && /^\s+(?:permissions?|systems?|formats?|names?|paths?|types?|sizes?|descriptors?)\b/iu.test(suffix)) return false; + if (/^files?$/iu.test(match[0]) && /^\s+(?:tools?|permissions?|systems?|formats?|names?|paths?|types?|sizes?|descriptors?)\b/iu.test(suffix)) return false; return true; }); - return fileObject || + // An immediately following "This ..." can qualify one internal file, not + // erase a separate output request. Only the authoritative objective counts. + const internal = objects.length === 1 && /^\s*This is (?:an? )?(?:personal memory|internal (?:assertion|verification) file)\b[^.!?]*\bnot a (?:task )?deliverable\b/iu.test(sentences[index + 1] ?? ""); + const deniedAttempt = /\b(?:attempt|try)\s+(?:the\s+)?native\s+(?:write|edit)\b/iu.test(clause) + && /\b(?:must be denied|denial is (?:the )?expected|(?:this|the) negative test)\b/iu.test(objective); + if ((internal || deniedAttempt) && !/\b(?:downloadable|attached|attach|export|send|provide|return|give)\b/iu.test(clause)) return false; + return objects.length > 0 || (!/\b(?:no|without)\s+(?:downloadable|attached)/iu.test(clause) && /\b(?:downloadable|attached)\s+(?:file|report|document|checklist|draft)\b/iu.test(clause)); - }); + })); } /** Files cited as completed output must be reachable outside the agent workspace. */ diff --git a/server/src/services/native-runtime/native-runner-file-handoff.test.ts b/server/src/services/native-runtime/native-runner-file-handoff.test.ts index f117e448fef..37487ebf08a 100644 --- a/server/src/services/native-runtime/native-runner-file-handoff.test.ts +++ b/server/src/services/native-runtime/native-runner-file-handoff.test.ts @@ -332,6 +332,21 @@ describe("native runner file handoff", () => { } }); + it.each([ + "Use native write/read file tools. Write memory/pi-native.txt inside AGENT_HOME. This is personal memory, not a task deliverable.", + "Use native write to copy bytes into pi-agent-memory-proof.txt. This is an internal assertion file, not a deliverable.", + "Attempt native write once to /outside/pi-unassigned.txt. This intentionally unassigned root must be denied.", + ])("accepts an internal file outcome without treating it as published output: %s", async objective => { + await db.update(heartbeatRuns).set({ contextSnapshot: { issueId, executionContinuation: { objective } } }) + .where(eq(heartbeatRuns.id, runId)); + try { + await expect(nativeCompletionFeedback(db, runId, doneReport([]))) + .resolves.toContain("Completion report accepted"); + } finally { + await db.update(heartbeatRuns).set({ contextSnapshot: { issueId } }).where(eq(heartbeatRuns.id, runId)); + } + }); + it("prepares one verified same-run attachment and replays without duplicates", async () => { const body = Buffer.from("native runner file handoff\n", "utf8"); await mkdir(path.join(workspaceRoot, "out"), { recursive: true });