mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 12:37:55 +01:00
test(evals): keep the trace screencast out of failure evidence
A failed agent-chat.track-delegated-work attempt (15 min, live run indicator on screen) produced two ~127 MB trace zips; the 256 MB private allowance refused the whole bundle and no evidence survived. Thirteen launch-1474 cells failed the same way. The bulk was the trace's own JPEG screencast, which does not compress. Keep DOM snapshots, network and sources in the trace and rely on the retained failure video for the visual timeline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b4593b361d
commit
6892a88e81
1 file changed
+6
-1
@@ -62,7 +62,12 @@ export default defineConfig({
|
||||
actionTimeout: 30_000,
|
||||
navigationTimeout: 30_000,
|
||||
screenshot: process.env.PAPERCLIP_RUNNER_E2E_PUBLIC_MCP === "1" ? "off" : "only-on-failure",
|
||||
trace: process.env.PAPERCLIP_RUNNER_E2E_PUBLIC_MCP === "1" ? "off" : "retain-on-failure",
|
||||
// Keep DOM snapshots and network, but not the trace's own screencast: a
|
||||
// long chat attempt with a live run indicator streamed thousands of JPEG
|
||||
// frames (incompressible), pushing failure evidence past the 256 MB private
|
||||
// allowance so no evidence survived. The retained failure video still
|
||||
// records the visual timeline.
|
||||
trace: process.env.PAPERCLIP_RUNNER_E2E_PUBLIC_MCP === "1" ? "off" : { mode: "retain-on-failure", screenshots: false },
|
||||
// A developer-supplied system Chromium keeps the local smoke loop
|
||||
// installation-free; CI's managed browser retains failure video as usual.
|
||||
video: chromiumExecutable || process.env.PAPERCLIP_RUNNER_E2E_PUBLIC_MCP === "1" ? "off" : "retain-on-failure",
|
||||
|
||||
Reference in new issue
Block a user