Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-30 21:49:18 +01:00
Code Issues Packages Projects Releases Wiki Activity

test(evals): protect Antigravity setup and two-turn delivery (#18190)

Browse Source 
## Thinking Path

> - Paperclip manages agents and their work.
> - A provider integration must work through ordinary agent setup and
task controls.
> - The recorded Antigravity walkthrough exposed setup, delivery, and
continuation failures.
> - A repeatable check must verify real work and retain failed attempts.
> - This pull request converts all 47 recorded steps into one fixed
Product E2E journey.
> - Maintainers can rerun the case and inspect independent delivery,
cleanup, and accounting evidence.

## Linked Issues or Issue Description

Builds on merged #18187, #18188, and #18189, and consumes the shared
repairs from merged #18230 and the equivalent manual-policy declaration
from #18222. Refs #18214 for the CI task wrapper. Refs #12503 and #14938
for related legacy adapter work.

**Agent or provider**

Google Antigravity native Runner with a saved managed Google
subscription and the exact Low model.

**Why this adapter is useful**

A user must be able to add an agent, connect its account, and receive
real task results across a follow-up. Picker visibility alone is
insufficient.

**How the agent is invoked**

A fixed Playwright driver uses ordinary product controls. The agent runs
through native ACP. Independent checks use public APIs, actual
downloads, owned process observations, and native lifetime fences.

## What Changed

- Register one explicit-only case:
`antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn`.
- Map all 47 source steps and declare saved-account prerequisites,
finite limits, and same-host macOS ARM64 scope.
- Create an agent through the ordinary picker, connect the saved
account, select Low, run Test, and wait for readiness. Save an
instruction marker, deliver a file, and update it through an ordinary
follow-up.
- Require each run's visible completed file activity, exact downloaded
bytes, committed Done result, same tagged conversation, distinct
physical owner birth, released lease, and free native fences. Verify
reload persistence.
- Stop failed native runs that still own a retry. Replay an existing
Stop with its durable request identity and wait for the matching audited
acknowledgement. A terminal reservation without a native intent remains
unconfirmed.
- Keep private evidence in the existing result, report, dashboard, and
accounting pipeline. Retain the paused QA agent and financial history
while preserving the shared account and budget policies.
- Journal only the ordinary wizard's future ID, target, name, intended
harness and expected creator before forwarding its POST unchanged. Bind
the response and interrupted recovery to exact identity reads; a pending
404 stays unconfirmed.
- Serialize private resource snapshots through temporary-file fsync,
atomic rename and directory fsync. Abort forwarding on storage failure
and preserve legacy receipts. If later journal or diagnostic writes
fail, retain the primary error, always close the browser, and attempt
the ordinary failed report.
- Add production-component browser controls and harden task-menu
readiness, alias selection, scroll restoration, failure packaging, and
one hermetic auth-test budget.
- Wait for the existing GitHub recovery fixture to register its next
deadline before asserting the original range; preserve its
nested-transaction and zero-request checks. In the decision-expiry
fixture, verify the exact saved expiry separately from the earlier
global wake of another queue.

## Verification

- Fresh registered acceptance passed on eval source
`9060f3cd01b7c3445ecdf703d72e9cf30fe42b59` and merged serving product
`61bd4ac6919c2309f766a25e62273b074cee3261`. It completed ordinary setup
and two turns with native `.5` and `gemini-3.8-flash-low`. All 16
matchers, 14 checkpoints, and cleanup passed in **165,613 ms**. Actual
downloads were **61 bytes** and **119 bytes**. Both turns passed their
own run-bound browser activity checks and reload persistence.
- Independent read-back confirms Done, a paused retained agent, the same
connected saved default account, and no active owned run. Both accepted
results have the same full tagged conversation and distinct primary
process births. All 21 observed processes retired; both leases released
and all three native fences were free after each turn. The selected `.5`
checkpoints also showed equal agent/backend session IDs; this is an
observed case requirement, not a generic ACP invariant.
- Receipts report 339,273 input tokens, 7,224 output tokens, and 202,846
cached input tokens. Subscription prices are unavailable. Billing
coverage remains incomplete, not free.
- The normal report CLI passed. All 310 files from nine original
attempts and two separate recoveries retained their hashes. The twelve
current screenshots are byte-identical, and 194 local report links are
contained and valid. The eight earlier reports were copied without
regeneration.
- Delivery source `1bf4fb04df04710b82844b3c9355767f46b5f68f` branches
from `d3f2a60e795fc60d64c8b19d1eed4d48657d5492` and merges cleanly with
current master `1d571de5f9f886d66e520800baec8fc2f67d0948`. All 41 prior
`c8b332` authored file blobs remain identical; the 42nd path is the
decision-expiry fixture contract correction. Creation ownership, the
strict durable writer, and the ordinary successful workflow remain
unchanged. Of 43 measured file blobs, 30 remain identical. The
Antigravity oracle, artifact, billing and report gates and its
suite-specific validation block remain unchanged. The actual retained
agent matches its original creator; its original request `setupId` was
not captured and is not reconstructed.
- Later master changes include model-visible workspace guidance and
rejected-terminal-report handling. Their bounded source assessment is
separate: the first actual finish failed, then a same-turn replacement
was accepted and committed. Later read-only results match both original
native-result hashes, and the complete event catalogs have no
`session.failed` event. Master #18228 also expands detection of explicit
JSON file requests. The measured create/update-and-attach `.txt` task
already required publication under both parsers; the publication
validation gate itself is unchanged. Master #18240 changes remote
workspace cache staging and adds an OpenCode index refresh. The selected
local lane returns before those paths. Master #18213 adds optional
numeric turn timing and trace boundaries. Its selected Antigravity
model, text, IDs, result and retirement semantics remain unchanged in
the bounded source review. Local diagnostic rows and awaits can change
event ordinals and latency; the oracle checks relative semantic order.
Collector coverage and latency improvement remain outside this proof.
This does not claim that the whole current runtime is byte-identical to
serving `61bd`, or relabel the measured result as a new run.
- The adopted upstream cache controls passed 3/3 with pinned real Turbo.
Forced work disables local and remote reads and writes; ordinary work
reuses local results. The original genuine-pnpm/Turbo controls remain
historical to `9060`'s earlier write-only policy. The duplicate cache
patch is superseded by #18230. Its qualification-only pnpm addition is
removed because the adopted upstream controls do not require it.
- The E2E typecheck, map checks and 277-control replay are carried from
the unchanged eval source paths. That replay passed, including the 183
Antigravity controls, catalog/connection checks and the separate
product-journey validator. The carried creation controls include 91
config/recovery cases. The final launcher suite passed 19/19, including
five real campaign finalization fault cases; the report suite passed
7/7. These final checks did not rerun the earlier 277-control group. The
canonical production-component browser spec passed in 27.4 s under its
unchanged 60 s limit before the replay. Four paused-composer cases and
the product base's build/typecheck plus 376 focused regressions were
verified earlier.
- The shipped wizard and actual API client cover persistence before
forwarding, lost response JSON, foreign/malformed requests, wrong
response identity/creator and cleanup-error preservation. Real
filesystem controls failed 5/5 before the writer repair and passed 5/5
afterward, preserving the old journal on partial-write, temporary-file
sync and rename faults. Three persistent-sync campaign controls failed
before the finalization repair and passed afterward. All five final
campaign fault cases preserve the primary failure, cleanup failure,
usage and failed report while attempting browser closure. The strict
journal still blocks the next mutation. These are provider-free
ownership controls; the Google task proof remains the original `9060` →
`61bd` acceptance.
- The decision restart check passed in the local 20-case fixture run. A
current-source real-PostgreSQL rollback control reproduced the exact
90,000 ms earlier global wake: the old equality rejected it, while the
exact durable expiry and new future-wake bound passed. The original 60 s
open-state, exact-timestamp, continuation and quiet checks remain. The
broader local run was **FAILED: 19 passed / 1 failed**, with a 15 s
timeout and a 30 s cleanup timeout in an unchanged timestamp-pagination
case. That case passed in canonical `c8b332` CI; the local timeout cause
remains inconclusive. An extra local server typecheck failed against
stale cached Runner declarations; it did not check this excluded test
file. Both limits remain recorded for fresh canonical CI.
- A broader local GitHub recovery group passed 21/22. Its unchanged
pagination case observed an unexpected config request; the log does not
establish the cause. That result remains retained, and fresh
current-head CI is the final broader gate.

- Exact delivery-head CI completed with 55 successes, two skips, and one
optional neutral review. Required `ci / verify` and `ci / e2e`, build,
typecheck, and Runner checks passed. Fresh Greptile is 5/5 with zero
unresolved threads. Optional Paperclip is neutral 4/5; its carried
recommendations are dispositioned against the public Stop contract and
selected live identity evidence.

Seven earlier converted attempts retain their failed grades and partial
evidence. Historical Paid8 and its later read-only per-run UI
observation remain separate from the fresh acceptance above. Unknown
prices and the older partial startup accounting remain explicit. Full
lineage and raw evidence remain private in the normal report bundle.

## Risks

- This case requires an authorized saved personal default Google
subscription and finite operator candidate admission. It does not test
fresh OAuth or establish global qualification.
- The physical observer supports a same-host loopback macOS ARM64
target. Cloud, Linux, and other models remain outside this case.
- A known-cost stop cannot cap missing prices or in-flight subscription
charges. Existing operator policies remain unchanged.
- Raw account/native identities stay private. Missing retirement proof
keeps resources retained. The selected cleanup mode does not test agent
purge.

## Model Used

OpenAI Codex (GPT-6). The host did not expose the exact model revision
or context limit. The model used tools for code edits, tests, browser
automation, and review.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-29 19:51:32 -05:00
1 parent 1d571de5f9
commit ebed7f5237
42 files changed
+4213 -34
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

doc/evals.md
+8
View File
@@ -102,6 +102,14 @@ The explicit-only [live provider connection suite](../tests/runner-e2e/PROVIDER-
is a Product E2E workflow for fresh subscription/API-key/gateway connections,
with attended login and independent artifact checks against local or staging targets.
The explicit-only [Antigravity additional-agent journey](../tests/runner-e2e/ANTIGRAVITY-AGENT-JOURNEY.md)
converts the saved-subscription setup and two-turn file walkthrough into one
local native cell. It measures agent creation through the ordinary UI,
instruction-dependent downloaded bytes, same native conversation, durable Done
lineage and owned runtime cleanup. Current support requires an attached
self-hosted loopback macOS ARM64 candidate; fresh OAuth, Cloud/remote execution
and global release qualification are separate gates.
The bounded [personal Muse receiver probe](muse-personal-agent.md#bounded-24-hour-probe)
uses ordinary task APIs and retained native receipts to measure unattended
receiver behavior. It is a separate manual qualification probe, not a registered
doc/plans/2026-10-29-antigravity-journey-to-eval.md
+275
View File
@@ -0,0 +1,275 @@
# Antigravity setup and two-turn file journey
This is the conversion contract for the recorded user walkthrough, not a runtime
manifest. The runnable contract and commands live in
[Antigravity agent journey](../../tests/runner-e2e/ANTIGRAVITY-AGENT-JOURNEY.md).
Raw recordings and concrete account, company, agent, task, attachment and native
session bindings remain in private evidence. The aliases and hashes below locate
that evidence without publishing an authenticated screenshot report.
## Outcome and requested delivery
A person can add a Google Antigravity agent from the ordinary Agents roster,
reuse their saved Google subscription, test and finish setup, save an instruction,
and receive a readable file. A follow-up on the same task preserves and updates
that file, returns to Done, and survives reload.
- Suite: `antigravity-agent-journey` (explicit only).
- Profile: `runner-antigravity`; case: `saved-subscription-two-turn`.
- Execution ID:
`antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn`.
- Entry: the selected existing company's dashboard as its board operator.
- Measured creation: a fresh additional agent and one task with two work turns.
- Excluded: fresh Google signup/consent, API keys, gateways, other models,
permission/question/Stop interactions, controller or provider loss, remote
execution and a platform matrix. Registration is not global provider release
qualification or unattended campaign qualification.
## Source and confidence
The source contains two v1 logs. Both originals remain immutable. On conversion,
the original `render_journey.py --validate-only` accepted 38/38 and 9/9 screenshots.
Images A08, A17, A18, A19, A26, A36, A38 and B09 were inspected directly: ordinary
picker discovery, disabled assignment during preparation, actual readiness, clean
composer, saved instruction, accurate activity statuses and persisted outputs.
The HTML gallery itself remains unverified: the collection host rejected local
file preview. Schema validation, image inspection and gallery interaction are
separate claims.
| Alias | Original title and UTC interval | SHA-256 of original `journey.json` | Original verdict |
| --- | --- | --- | --- |
| A | Fresh Antigravity agent from setup to useful work — native .5; 2026-10-29 16:49:19.943–16:57:04.231 | `2064abfafffae4b77bdff364c6746a513dbd6df01e165fbc5450e36bc71520da` | passed; experience ready; 38 steps |
| B | Antigravity .5 second turn on the fresh agent; 2026-10-29 16:58:08.303–17:00:24.774 | `d34c581a1875032634a6dc5eecff11a94a5f98319e9b555ae0898848957d6942` | passed; experience ready; 9 steps |
Both attempts disclose implementation knowledge. A corrected first finish call
followed a genuinely rejected completion citation; it remains a failed activity.
The native `.5` candidate fixed the earlier global error broadcast, so completed
file edit/read activities stayed completed. Raw instruction and session-load wire
receipts were not archived. The saved Instructions-only nonce and independent
downloaded bytes establish the resulting behavior; they do not manufacture those
missing receipts. No fresh Google OAuth took place in A or B.
Independent private observation followed each browser-only phase. The retained
two-turn summary has SHA-256
`2f4a4b06357fb5b0f1db4e7df10a287114f3a8b45f05f0bb4101cd7c0b8879aa`.
Its actual downloads were independently read again during conversion:
| Turn | Bytes / newlines | Download SHA-256 |
| --- | --- | --- |
| First | 61 / 1 | `e229f07be3a7c6835f91f9bd07f7daf4bd19f839c91f2fdd17161867750d4483` |
| Follow-up | 119 / 2 | `b1bca87da838f17535613f48932a40043b9e31a820a43e4c9740d619e3dfac05` |
That observation binds each attachment to its own run, the exact requested and
effective `gemini-3.8-flash-low`, the same full tagged native conversation, accepted
results, committed finalization, applied Done decisions and released leases. It
also records both owned Runner/sidecar process groups absent and the provider
lifetime fence acquired/released. The first Done decision was later explicitly
superseded when the user reopened the task. Its accepted-result/assessment/Done
lineage and captured first settlement remain valid; requiring the *current*
coordinator pointer forever to equal the first Done would reject a correct
follow-up. The initial overly strict audit failure is retained privately beside
the lineage-aware observation; it is not a new product repair or erased attempt.
Earlier evidence remains separate:
| Alias | Original scope/verdict | Log SHA-256 | Conversion treatment |
| --- | --- | --- | --- |
| P1 | Additional-agent picker/draft only; passed, needs improvement | `e1f89d1771a495d5c9220ab4a3eb48b7f4b010dd448ea1bf4e4cd024487d05d3` | Discovery evidence only; no task or full setup claim |
| P2 | Native Test failed; failed, needs improvement | `fa3b4c02aec713a55371b833172410da816ab1304a9f77c6cb3451c8de07f83d` | Retain package-owner mismatch failure; no task ran |
| P3 | Native .4 setup and first task; passed, needs improvement | `8d56f5bc6b77d627f4b01dea038ee4b42d2e8b42b4b1587bf266e3cf7ada2fc1` | Preserve stale paused composer and erroneous activity errors; readiness and per-call status regressions |
| P4 | Native .4 follow-up; passed, ready | `0474ef042f1430c210d5175d44b7c35a3e5e48b7338d4e74aa0ae3378c4988dc` | Historical .4 only; does not qualify changed .5 runtime |
## Actors, setup applicability and dependencies
The fixed Playwright driver replaces exploration. The product agent and Google
subscription are real. An independent observer checks public records, downloaded
bytes and native lifecycle evidence. No semantic judge is needed for exact bytes,
identity or lifecycle. Synthetic oracle/report controls are labeled synthetic and
perform no provider work.
The UI promise is portable where the same product capabilities exist. Current
native admission has a complete maintained Antigravity closure only on macOS
ARM64. Application hosting and runner environment are separate axes; `.local.`
names agent execution, not whether the application is Cloud-hosted.
| Setup | Product applicability | Implemented / selected | Actual evidence |
| --- | --- | --- | --- |
| Attached self-hosted development target, same-host loopback local macOS ARM64, saved personal Google subscription, exact low model | Applicable with trusted candidate admission, account reuse and native lifecycle capabilities | Selected explicit cell | A+B exploratory live pass on native .5; converted fixed-driver acceptance is a separate attempt |
| Compatible self-hosted staging target with the same local runner capabilities | Same user promise; revision, auth and target prerequisites differ | Remote observer/setup adapter gap; current observer requires same-host loopback | Not run; missing fixture is not an unsupported-product claim |
| Paperclip Cloud with an admitted local macOS ARM64 execution target and equivalent account route | Conditional applicability; actual service/target capabilities must be established | Current config restricts hosting to self-hosted; Cloud adapter gap | Not run; no Cloud qualification |
| Linux, macOS x64, Daytona or another remote target | Current native closure/admission unavailable for this case | Excluded from current execution cell | No platform/remote proof |
| Fresh OAuth, API key or gateway | Meaningfully different authentication/setup variant | Excluded from saved-subscription case | No fresh-auth proof |
Required prerequisites are an attached, exclusively assigned test company and
local environment, exact expected serving revision and deployment mode, a
dedicated signed-in browser profile, and a current trusted operator candidate
admission. Admission remains a server-startup responsibility. Agent configuration,
environment variables or test payloads cannot grant it. The harness checks the
effective target, native version, platform, profile and closure pins rather than
silently switching provider/model/host. Missing or expired prerequisites block
before measured work. It never restarts or resets the attached target.
The selected implementation requires the same-host loopback target to verify
native process retirement through kernel birth identities and all three public
checkpoint lifetime fences. It has no managed-local bootstrap adapter; existing
workspace ownership establishes the test drive before non-paying preflight. The
roster must already contain a nonterminated agent to exercise the additional-agent
entry rather than first-company onboarding.
| Dependency alias | Consumer / phase | Requirement and delivery | Ownership / cleanup |
| --- | --- | --- | --- |
| `browser-operator` | Browser driver; prerequisite | Existing company member with board agent creation/configuration, instructions, task and connection-use rights; dedicated signed-in QA profile | Supplied user/profile retained; exclusive profile lock |
| `google-subscription-owner` | Browser user and connection; prerequisite | Same user as browser identity; existing personal Google subscription and personal default grant, exact private connection binding | Shared saved credential/grant retained; no default change or revocation |
| `product-model-account` | Newly created product agent; measured binding | Existing connection selected through ordinary Google setup; normal managed credential delivery; exact low model | New agent owned by attempt; managed credential refresh remains product-owned |
| `native-candidate-authority` | Attached server/native admission; prerequisite | Approved version/profile/closure/platform/company/model and finite expiry through trusted startup | Operator owns renewal; never copied into prompt or agent configuration |
| `independent-observer` | Harness; observation | Same session's bounded authorized read APIs and owned-file download; local native evidence only where its declared target permits | Read-only; private evidence retained |
Expected MCP connections: **none** for browser driver, product agent, observer or
authoring conversion. Native product tools are not an MCP dependency. No mailbox,
new passkey, billing-management credential, external App or new OAuth registration
is required. Concrete identity/session/cookie/account/connection and target refs
stay in an owner-only private setup file and existing credential store, not in
tracked examples or public metadata. Readiness verifies the consuming identity,
same-user personal grant, selected native capability and accounting policy before
use. Existing auth may refresh normally; fresh consent is outside this case.
## Source-step disposition
`A` and `B` refer to the immutable logs above. Checkpoint names below describe the
contract rather than source step numbers. Waits become bounded assertions; source
timings are evidence, not sleeps. No measured agent/task creation moves into API
fixture setup. Every one of the 47 numbered steps appears once below.
| Source | Observed action/result | Phase / checkpoint | Disposition | Assertion / evidence |
| --- | --- | --- | --- | --- |
| A01 | Open test-drive root, initially loading | measured / dashboard | retain | Selected company dashboard is the entry |
| A02 | Dashboard loads, Agents visible | measured / dashboard | observation | Bound loading wait; ordinary sidebar route |
| A03 | Click Agents | measured / agent-roster | retain | Navigate through visible sidebar |
| A04 | Roster has New Agent | measured / agent-roster | observation | Correct roster creation control visible |
| A05 | Click New Agent; name required | measured / name-agent | retain | Visible dialog and disabled adapter action |
| A06 | Enter distinct name | measured / name-agent | retain | Attempt-owned name enables continuation |
| A07 | Choose adapter | measured / normal-picker | retain + recovery regression | Ordinary grid, Advanced collapsed; durable request identity before forwarding; response bound to future ID/creator |
| A08 | Scroll to Google card | measured / normal-picker | observation | Google Antigravity card outside Advanced |
| A09 | Select Google card | measured / select-antigravity | retain | Configuration action becomes enabled |
| A10 | Configure agent | measured / google-connection | retain | Google-specific connection choices |
| A11 | Find personal saved default | measured / saved-subscription | observation | Exact personal default binding and reuse control |
| A12 | Use saved subscription | measured / saved-subscription | retain | Ordinary connection reuse, pending state |
| A13 | Configure opens with low model and Local | measured / model-environment | observation | Exact selected low model/environment |
| A14 | Run real setup Test | measured / native-test | retain | Prompt-free native runtime check |
| A15 | Connection successful | measured / native-test | observation | Successful check and enabled Finish |
| A16 | Finish setup | measured / finish-setup | retain | Activate the same owned draft |
| A17 | Finishing; assignment disabled | measured / readiness | observation | Never call pending activation ready |
| A18 | Actual ready; assignment enabled | measured / readiness | observation | Same persisted agent becomes invokable |
| A19 | Open first composer; no paused warning | measured / first-composer | retain | Correct agent/model and no stale pause |
| A20 | Escape empty composer | measured / configure-before-work | retain detour | No task created before instructions |
| A21 | Open agent from sidebar | measured / agent-overview | adapt | Follow ordinary Agents roster and search the exact owned name; the recent sidebar list is bounded |
| A22 | Idle/configured; no runs | measured / agent-overview | observation | Same owned Google agent and exact model |
| A23 | Open Instructions | measured / instructions | retain | Entry AGENTS.md and edit controls |
| A24 | Edit existing entry text | measured / instructions | retain | Preserve default instruction text |
| A25 | Add instruction-only marker | measured / instructions | retain | New nonce supplied only in instructions |
| A26 | Save changes; saved state | measured / instructions-saved | retain | Persisted instruction, no unsaved state |
| A27 | Assign Task from agent | measured / task-composer | retain | Same ready agent/model, no pause |
| A28 | Ask for configured marker, omit its value | measured / first-task-prompt | retain | Prompt cannot supply expected first-line bytes |
| A29 | Create task | measured / create-task | retain | Capture exact owned task immediately |
| A30 | Open Recent Tasks entry | measured / open-owned-task | retain | Follow visible newly created task link |
| A31 | In Progress / native work | verification / first-working | observation | Same task/agent; bounded start |
| A32 | Actual 61-byte result card arrives | verification / first-delivery | observation | New attachment correlated to owned first run |
| A33 | Open actual file; first finish rejected | verification / first-file | retain + recovery regression | Read/download actual bytes; preserve genuine rejection |
| A34 | Expand history after finish rejection | verification / activity-status | adapt | Open ordinary More messages when saved history is compacted, then expand activity; match actual native `tool_call`/`tool_result` (`kind: tool`) and canonical `provider_activity` rows while retaining A36 per-call attribution |
| A35 | Done after corrected finish | verification / first-settlement | observation | Own accepted result, committed finalization, applied Done |
| A36 | Completed edit/read, only actual finish failed | verification / activity-status | observation + recovery regression | Completed tools stay completed; do not force live rejection |
| A37 | Reload first result | verification / first-persistence | retain | Durable navigation boundary |
| A38 | Done and actual file survive reload | verification / first-persistence | observation | Exact visible task status plus independent saved bytes |
| B01 | Begin from same completed task | measured / follow-up-entry | retain | First settlement/output preserved before next trigger |
| B02 | Enter preserve-first-line/append-new-line request | measured / follow-up-prompt | retain | New nonce only; original marker not repeated |
| B03 | Send; task reopens/queues | measured / follow-up-send | retain | Same task and agent; exact new run bound |
| B04 | In Progress; existing file read | verification / follow-up-working | observation | Same native conversation; prior file read evidence |
| B05 | Second Done and new attachment | verification / follow-up-settlement | observation | Own accepted/committed Done lineage |
| B06 | Open actual 119-byte file | verification / follow-up-file | retain | Distinct second attachment, both exact lines and bytes |
| B07 | Reload final result | verification / final-persistence | retain | Durable second navigation boundary |
| B08 | Done and updated file persist | verification / final-persistence | observation | Same task, updated actual file, no stale first artifact |
| B09 | Scroll to latest result/viewer | verification / final-result | observation | Final checkpoint image and actual download, no prose-only pass |
## Acceptance, ownership and execution bounds
The observer is armed before each UI mutation and journals immutable owned
agent/task/run/artifact IDs as they become available. It rejects a wrong company,
agent, task/run, stale first artifact, empty file, missing proof, model mismatch,
conversation mismatch, uncommitted completion, pending lease or uncertain
retirement. Exact synthetic pass/fail/missing-evidence controls calibrate the same
oracle and normal report importer. Visible status checks target the exact task's
property; hidden menu items, SVG titles and narrative “Done” cannot pass them.
The first line derives from a fresh attempt nonce saved only in Instructions;
the second line derives from an independent follow-up nonce. Expected bytes are
computed by the observer. Fixed historical 61/119-byte sizes are source evidence,
not fixture constants for different nonce lengths. Each download must be bound
to its own task/run and actually match the independently computed content/hash.
After each turn, accepted native result, committed finalization and applied Done
decision lineage must agree. Same conversation does not require same physical
daemon: the source retired the first owners and resumed with fresh owners.
The native Runner instance ID is session-scoped in the recorded `.5` runs.
Freshness therefore compares each independently observed primary Runner PID and
OS birth identity, with its exact run and `per_turn` argv verified; changing only
a sidecar or journal digest cannot satisfy that gate.
Exactly two task work turns/runs are admitted in one selected case and one
attempt, with finite whole-attempt/per-turn deadlines, no automatic retries and
the startup authority's TTL. `budgetCents` (at most 1,000) is an observed known-cost
stop threshold, not a hard currency cap. It cannot cap missing prices, unknown
subscription charges or already in-flight usage. The fixture creates no agent
budget policy or QA override: the operator's existing policy and the new agent's
inherited policy remain unchanged. Current budget incidents, held charges and
unpriced-usage rules retain their authority; the fixture does not clear flags,
resume repeatedly or weaken policy. Native Test is a zero-turn
account/model/retirement probe; an unexpected inference edge or extra task run
fails the bound and remains in accounting. Measured token usage and missing
prices survive packaging as unknown cost rather than free work. No recurrence
is requested.
Supplied company, environment, browser user, saved Google connection and default
grant are retained. The ordinary draft POST is forwarded unchanged only after
its safe identity subset is durably journaled: future setup ID, company/name,
origin, intended harness and preflight creator. Serialized private temp-file
fsync, atomic rename and directory fsync keep later interruptions from truncating
that identity. A lost response can be recovered only by exact-ID and creator
readback with unique company/name binding; pending 404 stays unconfirmed. No
historical source request is inferred or reconstructed from a later agent row.
The new measured agent, task and any owned active work are journaled incrementally.
Teardown cancels/stops only owned work, waits for
settlement and physical retirement, then removes or pauses/retains the owned
agent as declared. Retained task history is inspectable; it does not authorize
future dispatch. A forced interruption retains a private ownership receipt for
the existing recovery command. Cleanup failure stays separate from the primary
failure; late cleanup writes a new receipt without rewriting the failed attempt.
An authoritative cancellation before native selection has a separate no-launch
receipt: settled preparation, explicit absence of native ownership, settled
accounting, and the exact environment lease's concrete release time. A lease
marked expired is accepted only for this no-launch variant when its release
precedes preparation settlement. Native success still requires released lease,
captured process births, absence of all owned processes and all three fences.
## Integration and handoff record
The registered catalog/profile/case, measured Playwright flow, independent
oracle and result integration belong in `tests/runner-e2e`, alongside existing
provider-connection target/browser helpers. This is a separate suite because the
existing connection suite requires **fresh** credentials; changing that invariant
would mislabel saved subscription reuse. No second runner, report schema,
dashboard or scheduler is introduced. Private setup/native evidence uses the
existing retained-attempt boundary. Authenticated setup images are never marked
public-fixture screenshots.
Mapped capabilities require actual assertions: `agents/hire`,
`agents/instructions`, `runs-adapters/validation`, `runs-adapters/session`,
`tasks/create`, `tasks/completion`, `documents-artifacts/attachments` and
`documents-artifacts/work-products`. Visiting other surfaces adds no coverage.
The runbook records exact discovery, credential-free calibration, readiness,
run, normal report and owned-resource recovery commands. Fixed-driver live
attempts, serving/driver/report revisions, actual setup, failed attempts, tokens,
unknown prices and cleanup are separate from the historical A+B proof.
Delivery state: source recovered, original logs validated, decisive images and
actual retained file bytes inspected; conversion implemented in the accompanying
case. Credential-free results and any fresh live attempt are recorded in the PR
and normal attempt artifacts. Historical A+B qualifies only its declared
attended local candidate scope. Other hosting/auth/platform cells remain unrun.
maps/agent-access-map/agents.md
+7
View File
@@ -111,6 +111,13 @@ Scoped Evidence: `source-only` — [creation policy](../../server/src/services/a
and [same-harness import preservation](../../server/src/services/company-portability.ts)
at the scoped commit. No permitted-agent journey was executed for this map update.
The board-driven [Antigravity setup/task eval](../../tests/runner-e2e/ANTIGRAVITY-AGENT-JOURNEY.md)
creates its measured agent through the ordinary wizard and reuses only the
signed-in owner's existing personal default Google grant. Its observer may read
the owned records; it cannot seed the hire or manufacture readiness. This case
does not qualify agent-authored hiring, credential administration or a broader
agent permission boundary.
Verification: `not-run` — exercise a listed permitted operation with actual agent
credentials/run context; inspect its result and applicable persistence/attribution.
Assert the relevant denied actor, owner, company or mode from Policy. Review the
maps/agent-access-map/runs-adapters.md
+6
View File
@@ -123,6 +123,12 @@ Policy: Environment tests follow agent creation/configuration authority. Support
managed-account login initiation follows `model-accounts` below; human codes and
consent, raw host login and adapter installation retain their specific gates.
The board-driven [Antigravity saved-subscription journey](../../tests/runner-e2e/ANTIGRAVITY-AGENT-JOURNEY.md)
uses this ordinary environment Test after selecting the same signed-in owner's
saved personal Google grant. It also verifies actual task outputs and native
settlement independently. This does not establish agent-authored environment
validation, fresh provider consent or access under another owner's grant.
Evidence: `source-only` — [implementation](../../server/src/routes/agents.ts#L6608) at
the reviewed commit above.
maps/feature-map/agents.md
+10
View File
@@ -64,6 +64,16 @@ Test verifies native account/model admission and cleanup without a model turn.
Other companies, remote targets, expired authority and other models stay blocked.
Verify this scoped path separately from production qualification and real task acceptance.
The explicit-only [Antigravity agent journey](../../tests/runner-e2e/ANTIGRAVITY-AGENT-JOURNEY.md)
protects the ordinary roster/picker path, saved personal Google subscription,
native Test, actual ready handoff, Instructions-only file marker, task and
same-task follow-up. Its fixed driver starts from the dashboard and creates the
new agent through visible controls. The current runnable cell attaches to a
self-hosted target with the admitted local macOS ARM64 candidate and exact low
model. Saved-account reuse does not qualify fresh OAuth, other platforms or
global provider release; registration and synthetic controls are separate from
the retained exploratory and fresh fixed-driver results.
Agent-owned plugin environments prepare the saved draft in the background when
their driver declares `requiresAgent`. The draft waits in `verifying` with its
setup hold until hiring finishes. Changing environments keeps the selected
maps/feature-map/runs-adapters.md
+10
View File
@@ -178,6 +178,16 @@ Cloud onboarding must not offer shell commands the customer cannot run.
Manual: Select the intended adapter and account, refresh model choices, and run its environment test. Save, reload, then execute a small task. Repeat with a deliberately unavailable account and verify the failure remains visible.
The explicit-only [Antigravity agent journey](../../tests/runner-e2e/ANTIGRAVITY-AGENT-JOURNEY.md)
checks a saved personal subscription, prompt-free native Test and exact low model,
then independently correlates two actual downloaded outputs, accepted/committed
Done lineage, native conversation continuity and owned runtime retirement. Its
first settlement is captured before follow-up reopening; a later coordinator
assessment cannot erase that historical Done decision. A failed finish must not
mark prior completed file operations failed. The current local macOS ARM64
candidate scope leaves fresh authorization, remote targets and release
qualification unproven.
For Legacy Codex, assign a service secret such as `SLACK_BOT_TOKEN` through agent
configuration and run a task that checks its presence without printing its value.
Check both ACP and CLI execution. Remove the binding and repeat on the next run.
packages/paperclip-runner/src/drivers/acpx/antigravity-auth.test.ts
+3 -1
View File
@@ -93,6 +93,8 @@ describe("Antigravity setup-only Google authentication", () => {
expect(fetch).toHaveBeenCalledTimes(2);
});
// The login predicate keeps its five-second bound. Allow ten more seconds
// for fixture startup and the durable mocked credential checks on a busy CI worker.
it("performs native authenticate, retires it, verifies identity, and preserves refresh rotation without repeated status network calls", async () => {
const home = await fixture(), fetch = googleIdentity(); vi.stubGlobal("fetch", fetch);
const login = startAntigravityBrowserLogin(home);
@@ -114,7 +116,7 @@ describe("Antigravity setup-only Google authentication", () => {
await verifyAntigravityCredential(home);
await verifyAntigravityCredential(home);
expect(fetch).toHaveBeenCalledTimes(6);
});
}, 15_000);
it.each(["error", "bad-url"])("rejects %s without leaking provider text or calling Google", async mode => {
const home = await fixture(mode), fetch = vi.fn(); vi.stubGlobal("fetch", fetch);
server/src/__tests__/chat-channels.integration.test.ts
+5 -1
View File
@@ -16297,7 +16297,11 @@ describeEmbeddedPostgres("chat channel control-plane integration", () => {
await expect.poll(async () => (await f.checkpoint()).value, { timeout: 5000 }).toMatchObject({
generation: Number(oldWindow.generation) + 1, outcome: "epoch_initialized" });
expect(f.requests).toEqual([]);
expect(coordinator.nextWakeAt()!).toBeLessThan(Date.now() + 90_000);
// The checkpoint commits before this coordinator pass rearms its deadline.
await expect.poll(() => {
const next = coordinator.nextWakeAt();
return next !== null && next < Date.now() + 90_000;
}, { timeout: 5000 }).toBe(true);
} finally { release?.(); await pending; await coordinator.stop(); }
});
server/src/__tests__/decision-expiry-work.test.ts
+9 -2
View File
@@ -164,8 +164,15 @@ const support = externalUrl ? { supported: true } : await getEmbeddedPostgresTes
});
it("reconstructs a future expiry on restart and does not expire it early", async () => {
const row = await create(); await start();
await vi.waitFor(() => expect(coordinator!.nextWakeAt()).toBe(row.expiresAt.getTime()));
const row = await create(); const { expiry } = await start();
await vi.waitFor(async () => {
expect(await expiry.nextExpiryAt()).toBe(row.expiresAt.getTime());
// Another queue may reconcile sooner than this decision's expiry.
const next = coordinator!.nextWakeAt();
expect(next).not.toBeNull();
expect(next!).toBeGreaterThan(Date.now());
expect(next!).toBeLessThanOrEqual(row.expiresAt.getTime());
});
await vi.advanceTimersByTimeAsync(60_000);
expect((await service().get(row.id))?.status).toBe("open");
await expired(row.id); await quiet(); expect(wake).toHaveBeenCalledTimes(1);
tests/e2e/paused-composer.spec.ts
+54 -11
View File
@@ -1,4 +1,4 @@
import { test, expect, type APIResponse } from "@playwright/test";
import { test, expect, type APIRequestContext, type APIResponse, type Page } from "@playwright/test";
async function json(response: APIResponse) {
const body = await response.json();
@@ -6,11 +6,52 @@ async function json(response: APIResponse) {
return body;
}
async function pauseTask(page: Page) {
const actions = page.getByRole("button", { name: "More task actions", exact: true });
await expect(actions).toBeVisible({ timeout: 10_000 });
// During hydration the loading overlay contains a temporary header copy.
// Use the real revealed header before sending its ordinary key action.
await expect(page.getByTestId("task-chat-history-loading")).toHaveCount(0, { timeout: 10_000 });
const history = page.getByTestId("task-chat-history-content");
if (await history.count()) {
await expect(history).toBeVisible({ timeout: 10_000 });
await expect(history).not.toHaveAttribute("inert", "", { timeout: 10_000 });
}
await actions.focus({ timeout: 10_000 });
await expect(actions).toBeFocused({ timeout: 10_000 });
// The production trigger explicitly supports Enter to open the popover.
await actions.press("Enter", { timeout: 10_000 });
await expect(actions).toHaveAttribute("aria-expanded", "true", { timeout: 10_000 });
await page.getByRole("button", { name: "Pause work", exact: true }).click({ timeout: 10_000 });
}
async function cleanupFixture(request: APIRequestContext, companyId: string, classic: boolean, primaryFailed: boolean) {
const operations = [
{ name: "archive fixture company", run: () => request.patch(`/api/companies/${companyId}`, { data: { status: "archived" } }) },
{ name: "restore task interface", run: () => request.patch("/api/instance/settings/experimental", { data: { enableClassicTaskInterface: classic } }) },
];
const outcomes = await Promise.allSettled(operations.map(async operation => {
const response = await operation.run();
if (!response.ok()) throw new Error(`${operation.name}: HTTP ${response.status()}`);
}));
const failures = outcomes.flatMap((outcome, index) => outcome.status === "rejected"
? [{ operation: operations[index]!.name, error: String(outcome.reason) }] : []);
if (!failures.length) return;
if (!primaryFailed) throw new AggregateError(failures.map(failure => new Error(`${failure.operation}: ${failure.error}`)), "Paused composer fixture cleanup failed");
// A timeout may have disposed the request fixture. Keep that original
// failure while retaining both cleanup outcomes for diagnosis.
console.warn("Paused composer fixture cleanup failed after the primary failure", failures);
await test.info().attach("paused-composer-cleanup-errors", {
body: Buffer.from(JSON.stringify(failures, null, 2)), contentType: "application/json",
}).catch(() => {});
}
for (const classic of [false, true]) {
test(`resume preview: ${classic ? "classic" : "task chat"} does not promise to wake blocked work`, async ({ page, request }) => {
test.setTimeout(120_000);
const company = await json(await request.post("/api/companies", { data: { name: `Resume preview ${Date.now()}` } }));
const settings = await json(await request.get("/api/instance/settings/experimental"));
let primaryFailed = false;
try {
await json(await request.patch("/api/instance/settings/experimental", { data: { enableClassicTaskInterface: classic } }));
const agent = await json(await request.post(`/api/companies/${company.id}/agents`, { data: {
@@ -22,8 +63,7 @@ for (const classic of [false, true]) {
title: "Blocked work stays blocked", status: "blocked", assigneeAgentId: agent.id,
} }));
await page.goto(`/${company.issuePrefix}/issues/${task.identifier}`);
await page.getByRole("button", { name: "More task actions", exact: true }).click();
await page.getByRole("button", { name: "Pause work", exact: true }).click();
await pauseTask(page);
const takeover = page.getByTestId("paused-composer-takeover");
await expect(takeover).toBeVisible();
// Enable dispatch only after the hold exists; assignment itself otherwise starts a run.
@@ -46,9 +86,11 @@ for (const classic of [false, true]) {
await expect(editor).toBeVisible();
expect((await json(await request.get(`/api/issues/${task.id}`))).status).toBe("blocked");
expect(await json(await request.get(`/api/companies/${company.id}/heartbeat-runs?agentId=${agent.id}`))).toEqual([]);
} catch (error) {
primaryFailed = true;
throw error;
} finally {
await request.patch(`/api/companies/${company.id}`, { data: { status: "archived" } });
await request.patch("/api/instance/settings/experimental", { data: { enableClassicTaskInterface: settings.enableClassicTaskInterface } });
await cleanupFixture(request, company.id, settings.enableClassicTaskInterface, primaryFailed);
}
});
@@ -56,6 +98,7 @@ for (const classic of [false, true]) {
test.setTimeout(120_000);
const company = await json(await request.post("/api/companies", { data: { name: `Paused composer ${Date.now()}` } }));
const settings = await json(await request.get("/api/instance/settings/experimental"));
let primaryFailed = false;
try {
await json(await request.patch("/api/instance/settings/experimental", { data: { enableClassicTaskInterface: classic } }));
const agent = await json(await request.post(`/api/companies/${company.id}/agents`, { data: {
@@ -69,8 +112,7 @@ for (const classic of [false, true]) {
await page.goto(`/${company.issuePrefix}/issues/${task.identifier}`);
const editor = (classic ? page.getByTestId("issue-chat-composer") : page).getByRole("textbox", { name: "editable markdown" });
await editor.fill("Keep this draft until I resume.");
await page.getByRole("button", { name: "More task actions", exact: true }).click();
await page.getByRole("button", { name: "Pause work", exact: true }).click();
await pauseTask(page);
const takeover = page.getByTestId("paused-composer-takeover");
await expect(takeover).toBeVisible();
await expect(takeover).toContainText("Your draft is saved.");
@@ -95,8 +137,7 @@ for (const classic of [false, true]) {
await expect.poll(async () => (await json(await request.get(`/api/issues/${task.id}/comments`))).map((comment: { body: string }) => comment.body)).toEqual(["Keep this draft until I resume."]);
// Ancestor pause is effective even when this child was created after it.
await page.getByRole("button", { name: "More task actions", exact: true }).click();
await page.getByRole("button", { name: "Pause work", exact: true }).click();
await pauseTask(page);
await expect(takeover).toBeVisible();
const child = await json(await request.post(`/api/companies/${company.id}/issues`, { data: {
title: "Child held by parent", parentId: task.id, status: "backlog", assigneeAgentId: agent.id,
@@ -122,9 +163,11 @@ for (const classic of [false, true]) {
await page.goto(`/${company.issuePrefix}/issues/${child.identifier}`);
await expect(takeover).toHaveCount(0);
await expect(editor).toBeVisible();
} catch (error) {
primaryFailed = true;
throw error;
} finally {
await request.patch(`/api/companies/${company.id}`, { data: { status: "archived" } });
await request.patch("/api/instance/settings/experimental", { data: { enableClassicTaskInterface: settings.enableClassicTaskInterface } });
await cleanupFixture(request, company.id, settings.enableClassicTaskInterface, primaryFailed);
}
});
}
tests/runner-e2e/ANTIGRAVITY-AGENT-JOURNEY.md
+306
View File
@@ -0,0 +1,306 @@
# Antigravity additional-agent journey
`antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn`
converts the recorded 38-step setup/first-task and 9-step follow-up into one
explicit-only Product E2E cell. It is excluded from `--all` and scheduled paid
matrices. The [conversion brief](../../doc/plans/2026-10-29-antigravity-journey-to-eval.md)
records all 47 source-step dispositions, immutable source hashes, original grades,
earlier failures and evidence limits.
The fixed browser driver starts on the company dashboard, follows Agents → New
Agent, selects Google Antigravity in the ordinary grid with Advanced collapsed,
reuses the saved personal Google subscription, selects the exact low model and
Local environment, and runs the native prompt-free Test. Finish must reach actual
readiness before task assignment is enabled. It checks the first composer for a
stale paused warning, closes that empty composer, saves a nonce only through the
agent's Instructions, then submits a task asking for the configured marker without
supplying its value. An ordinary follow-up asks the same agent/task to preserve the
first line and append a different nonce. Each real file and Done state must remain
available after reload.
## Selected setup and dependencies
The runnable setup attaches to an existing **self-hosted loopback target on the
same macOS ARM64 host**. This narrower implementation lets the independent native
observer verify owned process retirement and all three public-checkpoint provider
lifetime fences. `.local.` identifies agent execution. It does not itself identify
application hosting. Compatible Cloud/staging behavior remains a portability
target, not an implemented or verified remote observer path.
Required existing state:
- A dedicated test company and local execution environment; expected and observed
serving commit and deployment mode must match. The harness never resets or
restarts an attached target. Its roster already contains a nonterminated agent,
so the measured creation is an additional-agent path.
- A dedicated signed-in QA browser profile for the intended board operator, with
company agent/instruction/task creation and saved-connection use rights. The
normal profile ownership marker and exclusive lock apply; do not use a normal
personal Chrome profile.
- The same person's saved managed Google subscription, with its existing personal
default grant. The private setup names exact connection, owner, company and
environment references. Names alone do not establish the binding.
- A current, finite **trusted operator startup admission** for that company,
`gemini-3.8-flash-low`, local darwin-arm64, native version, profile and closure.
The config's pins assert provenance; they cannot grant admission. The global
release remains pending and closed by default.
- Existing operator accounting policy permitting the two task turns, with held
and unpriced usage reconciled through its ordinary owner. The harness preserves
that policy and the new agent's inherited policy; it creates no agent budget
policy or QA override. Missing subscription prices remain unknown.
No MCP connection is required by the browser driver, product agent or independent
observer. Native product tools are used by the agent. No API key, mailbox,
billing-management credential, fresh Google login or OAuth registration is part
of this case. Authentication freshness is **saved browser session + saved product
connection**, not fresh provider consent. Product-managed refresh stays inside
the normal managed-account boundary.
After verified native retirement, a refreshed Google credential transfers to the
existing encrypted, fsynced recovery journal before database rotation. The next
authorized task, Test or model discovery recovers it with the same grant,
generation and account checks. A failed journal write or unconfirmed retirement
keeps the private home and blocks cleanup; a retry does not grant new authority.
One-time test-drive/bootstrap setup remains the operator's supported workspace
owner responsibility. This suite does not implement a managed-local target
bootstrap, mint candidate authority, install a provider through a test-only path,
or import an operator's CLI credential home. Install the maintained native assets
through `paperclipai runtime setup antigravity` as the OS user running Paperclip.
Then use the existing product UI to establish the saved account once. Per-attempt
preflight verifies those declared prerequisites; the measured **new agent** must
remain absent until the visible creation flow.
## Private setup file
Create an owner-only regular JSON file outside the repository (`chmod 600`).
Replace every placeholder with an authorized binding; never put token/cookie,
password, OAuth code or startup authority bytes in it. Relative `profileDir` is
resolved against this file's directory. The browser must declare `signed-in`.
If board sign-in needs assistance, use setup-only `--antigravity-journey-preflight`
to save the dedicated browser session, then start a fresh measured attempt.
A measured attempt that needed assisted sign-in stops before agent or provider
setup and retains a blocked report.
```json
{
"version": 1,
"setupId": "local-antigravity-saved-subscription",
"hosting": "self-hosted",
"executionEnvironment": "local",
"target": {
"mode": "attach",
"baseURL": "http://127.0.0.1:YOUR_PORT",
"expectedCommit": "EXACT_40_CHARACTER_SERVING_COMMIT",
"deploymentMode": "local_trusted",
"companyId": "EXISTING_TEST_COMPANY_UUID",
"environmentId": "EXISTING_LOCAL_ENVIRONMENT_UUID"
},
"browser": {
"headed": true,
"channel": "chrome",
"profileDir": "./qa-browser",
"accountAlias": "browser-operator",
"freshness": "signed-in",
"loginTimeoutMs": 600000
},
"savedConnection": {
"id": "EXISTING_GOOGLE_CONNECTION_UUID",
"name": "YOUR_SAVED_SUBSCRIPTION_LABEL",
"ownerUserId": "EXPECTED_BROWSER_USER_ID"
},
"candidate": {
"platform": "darwin",
"architecture": "arm64",
"nativeVersion": "1.4.3-paperclip.5",
"profileDigest": "sha256:acf5f574b6b9ec4388ff9d081890caa330e66bf1ae3fe3b08e7f0fb0e851b9ce",
"nativeClosureDigest": "sha256:044792d793e203bc8b4524a418f1816c4d34db67bb41fa4bae5947a050c49b4a"
},
"limits": {
"durationMs": 1200000,
"turnTimeoutMs": 300000,
"maxRuns": 2,
"budgetCents": 200
},
"retainAgent": true
}
```
Use `authenticated` when it is the target's actual deployment mode; do not change
the server to match the example. The maximum whole attempt is 30 minutes and each
turn at most 10 minutes. `budgetCents` is an observed known-cost stop threshold
(at most 1,000 cents), not a hard currency cap or a product-policy write. It cannot
cap missing prices, unknown subscription charges or already in-flight usage. The
finite execution boundary is one selected case, one attempt, exactly two task
runs, finite deadlines and the startup authority's TTL. Native Test performs
account/model admission and cleanup without a model prompt. An unexpected work
run or inference edge must be recorded, not omitted from accounting.
## Commands
Run from the Paperclip repository root, with its normal workspace dependencies
installed. The maintained source files own all behavior; no author-local helper
script is required. Discovery and credential-free checks do no provider work:
```sh
pnpm test:e2e:runner -- --list --suite antigravity-agent-journey
pnpm test:e2e:runner:typecheck
pnpm test:e2e:runner:unit
pnpm test:e2e:runner:browser-support -- antigravity-journey-browser.spec.ts
```
The browser calibration is included in the canonical browser-support target and
uses local fixtures, including the real activity renderer. It checks selectors
and stale UI states without an account connection, an agent setup or a provider run; it is separate from live qualification.
Read-only readiness validates the target revision/mode, signed-in consuming
identity, exact personal default grant, local environment and candidate catalog.
It does not run native Test, create a task or dispatch a provider prompt:
```sh
pnpm test:e2e:runner -- \
--id antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn \
--antigravity-journey-config /absolute/private/antigravity.json \
--antigravity-journey-preflight \
--max-automatic-retries 0
```
After readiness, run only the exact authorized cell. A new campaign preserves
previous attempts and uses no automatic retry:
```sh
pnpm test:e2e:runner -- \
--id antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn \
--antigravity-journey-config /absolute/private/antigravity.json \
--max-automatic-retries 0
```
Campaigns live under
`tests/runner-e2e/results/antigravity-journey-<timestamp>-<nonce>/`. Retain their
normal result, evidence manifest, checkpoint images, actual byte/hash receipts,
timings, effective setup/revisions, billing and cleanup alongside failures. Run
the **normal importer separately** to verify a retained report; it makes zero new
model calls. Environment variables, not positional report arguments, select it:
```sh
PAPERCLIP_RUNNER_E2E_REPORT_ROOT=/absolute/private/campaign \
PAPERCLIP_RUNNER_E2E_REPORT_OUT=/absolute/private/rebuilt-report \
PAPERCLIP_RUNNER_E2E_EXPECTED_IDS='["antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn"]' \
pnpm test:e2e:runner:report
```
The importer must preserve the attempt verdict, cleanup outcome, duration, known
input/output/cache tokens and unknown prices. A rebuilt dashboard is a
presentation check, not another provider measurement. Missing final proof cannot
be repaired by inventing a screenshot or rewriting an old grade.
If packaging fails, `attempt-result.json` preserves the primary measurement and
`packaging-error.json` records the separate reporting failure. Any partially
written result and manifest are copied before the retained presentation is
marked failed with `journey_report_packaging_failed`. Cleanup, usage and original
screenshots remain available; the normal importer can rebuild that failed
presentation without changing the original outcome or supplying missing proof.
## Independent acceptance and retained evidence
Success requires the correct browser controls and persisted object identity,
exact actual downloaded bytes for both distinct run-bound artifacts, requested
and effective low model, same full tagged native conversation, accepted result,
committed finalization and an applied Done decision for each own settlement.
The session-ID equality gate is an observed property of this exact native `.5`
integration, corroborated by both retained real task checkpoints; it is not a
general ACP invariant.
The first settlement is captured before reopening; later legitimate reassessment
may supersede its current pointer without invalidating that historical lineage.
The final task's visible status and both outputs must survive reload. Narrative
claims, hidden status labels, a matching filename or a stale first artifact do not
satisfy those gates.
Native public lease release, exact owned process absence and reacquisition/release
of all three observed provider fence ports establish local retirement. The
two turns must have different independently captured primary Runner PID/OS-birth
identities, each verified against the exact run and `per_turn` argv. Their
session-scoped Runner instance ID may remain the same; a different sidecar does
not prove a fresh primary. The observer reads no private database or raw provider
reasoning. Its exact local
target constraint is part of the case, not portable host proof. A genuine failed
finish remains failed while prior successful native file activities stay
completed; the happy path does not deliberately manufacture that rejection.
Synthetic positive/negative/missing-proof controls use the same oracle and normal
report boundary. They cover wrong object/run/model/conversation, stale or wrong
bytes, incomplete settlement/retirement, missing final screenshot and unknown
price preservation. Browser calibration includes hidden duplicate/menu/SVG Done
text beside the intended visible task status. Synthetic fixtures establish these
gates, not Google behavior or discoverability.
Raw authenticated setup screenshots, concrete account/company/agent/task/native
session identifiers, browser state and ownership receipts stay private. Do not
mark these captures `public-runner-fixture`, publish original walkthrough HTML,
or copy provider payloads into public evidence. Source aliases and hashes are
safe provenance; any public projection remains the existing sanitizer's allowlist.
## Cleanup and interrupted recovery
Before forwarding the ordinary wizard's draft POST unchanged, the campaign
journals only its future `setupId`, exact company/name/origin, intended harness
and preflight browser-user identity. It does not store the full request body or
credentials. The private journal serializes snapshots, fsyncs a private temporary
file, atomically renames it, and fsyncs its directory; failure aborts forwarding.
The response must match that future ID and creator. If the response is lost,
recovery reads that exact ID, verifies company/name/creator and uniqueness, then
persists the verified ID before cleanup mutations. A pending creation returning
404 remains unconfirmed; a name match alone grants no ownership. Existing
receipts with a known response and no new optional witness remain supported.
The receipt binds target/company, execution/attempt and immutable created
resources before later task dispatch. Cleanup stops only owned active work, verifies
settlement and native retirement, then pauses/removes the owned agent according
to `retainAgent`. Supplied company/environment/browser identity and saved Google
connection/default grant remain intact and are reverified. It never revokes a
shared credential or resets the target. Retained task history remains inspectable.
Use `retainAgent: true` for accounted work: cleanup leaves the owned agent paused
and preserves its task and cost history. Purge remains subject to the product's
ledger constraints; the fixture never removes accounting rows to force deletion.
For cancellation before native selection, cleanup instead persists a closed
no-launch receipt from the server's settled preparation and explicit null native
ownership fields, settled accounting, and exact company/run/environment lease.
That lease must have a concrete release timestamp no later than preparation
settlement; `expired` is accepted only in this no-launch variant. Missing or
pending receipts retain the agent. This branch creates no physical capture proof.
A Test that fails or times out before the product confirms probe retirement leaves
its setup probe marked unconfirmed. The harness keeps that agent paused and
retained. Public task-run absence cannot prove that a setup probe has retired;
recovery needs authoritative proof from the product cleanup owner.
On interruption, preserve the result and receipt first. Recover with the same
private target binding and the exact receipt; no name-based broad cleanup:
```sh
pnpm test:e2e:runner -- \
--id antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn \
--antigravity-journey-config /absolute/private/antigravity.json \
--antigravity-journey-recover /absolute/private/campaign/owned-resources.json \
--max-automatic-retries 0
```
Recovery validates ownership before stopping recorded/new owned-agent runs or
removing its agent. An uncertain mutation is reconciled before retrying. Cleanup
failure stays separate from primary failure; late recovery produces a separate
receipt without upgrading the original cleanup grade. Do not start another paid
attempt while owned work or the browser profile lock remains unresolved.
## Verification scope
The source `.5` browser walkthrough and independent retained oracle passed on the
declared attended local saved-subscription/low-model candidate. Native instruction
wire receipts were not archived; Instructions-only provenance plus actual file
bytes proves resulting behavior. HTML gallery navigation was blocked by the
host's local-file preview policy and is unverified. Earlier failed/partial `.4`
and package-admission attempts retain their original grades.
The converted fixed driver, credential-free calibration, fresh live attempt and
normal report results are recorded separately in PR/attempt evidence. A registry
listing or unit pass does not qualify a live provider. Fresh OAuth, API-key live
operation, Cloud, staging/remote targets, other models/platforms, failure controls,
reliability repetitions and unattended execution remain outside this selection.
tests/runner-e2e/FIXTURES.md
+11
View File
@@ -18,6 +18,17 @@ See [PROVIDER-CONNECTIONS.md](PROVIDER-CONNECTIONS.md) for the explicit-only
`provider-connections` suite, local/staging target ownership, dedicated browser
profiles, credential handoffs, private evidence, and cleanup contract.
[Antigravity additional-agent setup](ANTIGRAVITY-AGENT-JOURNEY.md) instead reuses
an existing same-user personal default Google subscription on an attached local
macOS ARM64 candidate. That prerequisite does not seed the measured new agent or
task: both are created through ordinary visible controls. The attempt journals
owned resources before task dispatch, preserves the supplied company/account/grant,
and exposes bounded read-only preflight and ownership-checked recovery. Native
Test is prompt-free; exactly two work turns are admitted. Independent actual
downloaded bytes, effective model/conversation, accepted/committed Done lineage
and local physical retirement determine the outcome. No fresh OAuth or
managed-local bootstrap is claimed.
The [public MCP journeys](PUBLIC-MCP.md) reuse the fixture registry with a real
authenticated browser session. `RunnerApi.setBrowserSession` binds that session
to API calls, including encrypted secret provisioning via Node fetch. OAuth
tests/runner-e2e/README.md
+8
View File
@@ -277,6 +277,14 @@ See [PROVIDER-CONNECTIONS.md](PROVIDER-CONNECTIONS.md) for the explicit-only
`provider-connections` suite, local/staging target ownership, dedicated browser
profiles, credential handoffs, private evidence, and cleanup contract.
The explicit-only [Antigravity additional-agent journey](ANTIGRAVITY-AGENT-JOURNEY.md)
protects ordinary picker discovery, reuse of a saved personal Google subscription,
native Test and genuine setup readiness, an Instructions-only file proof and a
same-task follow-up. Its one local macOS ARM64 cell attaches to an already running
self-hosted test target and preserves the supplied account. It has separate
non-paying preflight, owned-resource recovery and normal report commands. Saved
reuse is separate from the fresh-connection suite and global release qualification.
The [public MCP suite](PUBLIC-MCP.md) adds explicit paid assistant/plugin journeys
with authenticated browser consent, real MCP tool use and independently graded
team execution. Select `--suite public-mcp`; it is excluded from `--all`.
tests/runner-e2e/SECURITY.md
+13
View File
@@ -40,6 +40,19 @@ does not attribute a product cause or authorize acceptance.
## Attended connection authentication
The [Antigravity saved-subscription journey](ANTIGRAVITY-AGENT-JOURNEY.md) attaches
only to its declared same-host loopback macOS ARM64 test target. Configured pins
assert the trusted startup candidate authority; they cannot create it. The
signed-in browser user must own the exact pre-existing personal default Google
grant. No provider secret or OAuth code enters test configuration or prompts,
and no MCP connection is required. Ownership receipts, actual authenticated
screenshots and concrete account/session bindings stay private and receive no
public-fixture marker. Recovery validates target/company/execution and immutable
owned resources, stops only owned work and retains the supplied Google account
and default grant. It neither resets the attached instance nor clears accounting
holds. Two work turns, finite deadlines and zero automatic retries bound the
selected cell; unknown subscription prices remain unknown.
See [PROVIDER-CONNECTIONS.md](PROVIDER-CONNECTIONS.md) for the explicit-only
`provider-connections` suite, local/staging target ownership, dedicated browser
profiles, credential handoffs, private evidence, and cleanup contract.
tests/runner-e2e/antigravity-journey-adversarial.test.ts
+72
View File
@@ -0,0 +1,72 @@
import { expect, it } from "vitest";
import { gradeAntigravityJourneyTurn } from "./antigravity-journey-evidence.js";
import { antigravityEvidenceFixture, antigravityTurnFixture } from "./antigravity-journey-test-fixtures.js";
import { validateRetainedRunnerResult } from "./result-validation.js";
import type { RunnerE2EResult } from "./types.js";
// Independent synthetic counterexamples; these are not historical/live receipts.
it("requires a control-plane acceptance receipt even when Done metadata and bytes agree", () => {
const fixture = antigravityTurnFixture();
fixture.events = fixture.events.filter(event => event.eventType !== "run.result.accepted");
expect(() => gradeAntigravityJourneyTurn(fixture)).toThrow("antigravity_evidence_");
});
it("rejects provider-authored acceptance and an accepted result different from the settled result", () => {
for (const wrongSource of [true, false]) {
const fixture = antigravityTurnFixture();
const accepted = fixture.events.find(event => event.eventType === "run.result.accepted");
expect(accepted).toBeDefined();
if (wrongSource) accepted!.payload.prpEvent.sourceKind = "runner";
else (accepted!.payload.prpEvent.payload as Record<string, unknown>).result = { schema: "paperclip.run_result.v1", summary: "Different accepted result" };
expect(() => gradeAntigravityJourneyTurn(fixture)).toThrow("antigravity_evidence_");
}
});
it("requires the accepted result's terminal receipt instead of only a terminal run row", () => {
const fixture = antigravityTurnFixture();
fixture.events = fixture.events.filter(event => event.eventType !== "run.terminal");
expect(() => gradeAntigravityJourneyTurn(fixture)).toThrow("antigravity_evidence_");
});
it("requires a follow-up to read the prior file before writing and read the updated file afterward", () => {
for (const removeBefore of [true, false]) {
const fixture = antigravityTurnFixture(true);
const edits = fixture.events.filter(event => event.eventType === "tool.execution.completed" && ["edit", "write"].includes(event.payload.prpEvent.payload.operation));
const boundary = removeBefore ? Math.min(...edits.map(event => event.seq)) : Math.max(...edits.map(event => event.seq));
const excludedCalls = new Set(fixture.events.filter(event => event.eventType === "tool.execution.completed"
&& event.payload.prpEvent.payload.operation === "read"
&& (removeBefore ? event.seq < boundary : event.seq > boundary)).map(event => event.payload.prpEvent.payload.executionId));
expect(excludedCalls.size).toBeGreaterThan(0);
fixture.events = fixture.events.filter(event => !excludedCalls.has(event.payload.prpEvent.payload.executionId));
expect(() => gradeAntigravityJourneyTurn(fixture)).toThrow("antigravity_evidence_");
}
});
function retainedResult(): RunnerE2EResult {
return {
schema: "paperclip.runner-e2e.result/v2", executionId: "antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn", suiteId: "antigravity-agent-journey",
profileId: "runner-antigravity", environmentId: "local", caseId: "saved-subscription-two-turn", provider: "acpx", model: "gemini-3.8-flash-low", runtimeMode: "native",
attempt: 1, status: "passed", cleanup: "passed", durationMs: 1000, startedAt: "2026-10-29T00:00:00Z", finishedAt: "2026-10-29T00:00:01Z",
issueId: "task", runIds: ["run-first", "run-second"], antigravityJourney: antigravityEvidenceFixture(),
screenshots: [{ id: "persisted", label: "Synthetic assertion calibration only", file: "synthetic.png", sha256: "a".repeat(64) }],
};
}
it("accepts the fully correlated synthetic retained baseline used by the wrong-binding controls", () => {
expect(() => validateRetainedRunnerResult(retainedResult())).not.toThrow();
});
it.each(["companyId", "agentId", "connectionId"] as const)("rejects a passing report with a foreign or missing top-level %s binding", key => {
for (const missing of [false, true]) {
const result = retainedResult();
if (missing) delete result.antigravityJourney![key];
else result.antigravityJourney![key] = "foreign-object";
expect(() => validateRetainedRunnerResult(result)).toThrow("result.antigravityJourney");
}
});
it("rejects retained turn proofs attached to a different report task", () => {
const result = retainedResult();
result.issueId = "different-task";
expect(() => validateRetainedRunnerResult(result)).toThrow("result.antigravityJourney");
});
tests/runner-e2e/antigravity-journey-browser.smoke.ts
+390
View File
@@ -0,0 +1,390 @@
/** Credential-free calibration of the measured selectors in a real browser.
* Synthetic loopback HTML is deliberately not live Antigravity qualification. */
import { createServer } from "node:http";
import { captureAntigravityAgentDraft } from "./antigravity-journey-flow.js";
import { ANTIGRAVITY_JOURNEY_EXECUTION } from "./antigravity-journey-cases.js";
import type { AntigravityJourneyResources } from "./antigravity-journey-fixtures.js";
import { chromium, type Page } from "@playwright/test";
import assert from "node:assert/strict";
import { build } from "esbuild";
import path from "node:path";
import { pathToFileURL } from "node:url";
import { assertAntigravityCleanComposer, assertAntigravityFileActivity, assertAntigravityReady, assertAntigravityTaskDone, inspectAntigravityArtifact, openAntigravityAgentFromRoster, openAntigravityCreatedTask, selectAntigravityFromOrdinaryPicker } from "./antigravity-journey-ui.js";
const agent = "Antigravity QA synthetic", model = "gemini-3.8-flash-low";
const artifactId = "11111111-1111-4111-8111-111111111111", followupId = "22222222-2222-4222-8222-222222222222";
const artifactBytes = Buffer.from("A".repeat(60) + "\n");
const followupBytes = Buffer.concat([artifactBytes, Buffer.from("B".repeat(57) + "\n")]);
const fixtures: Record<string, string> = {
"task-uuid": `<aside><a href="/ANT/issues/synthetic-issue-id">Created task</a><a hidden href="/ANT/issues/ANT-42">Hidden duplicate</a><a href="/ANT/issues/other-id">Other task</a></aside>`,
"task-identifier": `<aside><a href="/ANT/issues/ANT-42">Created task</a><a href="/ANT/issues/ANT-41">Other task</a></aside>`,
"task-duplicate": `<aside><a href="/ANT/issues/ANT-41">Other task first</a><a href="/ANT/issues/synthetic-issue-id">Created task</a><a href="/ANT/issues/ANT-42">Same task identifier</a></aside>`,
"task-repeated": `<aside><a href="/ANT/issues/ANT-41">Other task first</a><a href="/ANT/issues/ANT-42">Recent task</a><a href="/ANT/issues/ANT-42">Project recent task</a></aside>`,
"task-other-only": `<aside><a href="/ANT/issues/ANT-41">Other task</a></aside>`,
"task-hidden": `<aside><a hidden href="/ANT/issues/ANT-42">Created task</a><a href="/ANT/issues/ANT-41">Other task</a></aside>`,
"ANT/issues/synthetic-issue-id": `<main>Created task UUID</main>`,
"ANT/issues/ANT-42": `<main>Created task identifier</main>`,
dashboard: `<aside><a href="/roster">Agents</a><a href="/wrong-agent">An older agent</a></aside><nav aria-label="breadcrumb"><a href="/roster">Agents</a></nav><main>Ready confirmation after closing composer</main>`,
roster: `<aside><a href="/roster">Agents</a><a href="/wrong-agent">${agent}</a></aside><main><div role="toolbar" aria-label="Agents controls"><input aria-label="Search agents"></div><a href="/wrong-agent"><span>${agent} older</span>Engineer idle</a><a href="/owned-agent"><span>${agent}</span>Engineer idle</a><a hidden href="/wrong-agent"><span>${agent}</span>Engineer idle</a></main>`,
"owned-agent": `<main><h1>${agent}</h1></main>`,
picker: `<div role="dialog" aria-labelledby="picker"><h2 id="picker">Choose an adapter</h2><label><input type="radio" name="adapter">Google Antigravity Candidate</label><details><summary>Advanced</summary><label><input type="radio" name="adapter">Other harness</label></details><button>Configure agent</button></div><div hidden><label><input type="radio">Google Antigravity Candidate</label></div>`,
"advanced-only": `<div role="dialog" aria-labelledby="picker"><h2 id="picker">Choose an adapter</h2><details open><summary>Advanced</summary><label><input type="radio">Google Antigravity Candidate</label></details><button>Configure agent</button></div>`,
ready: `<div hidden><h2>Finishing agent setup</h2><button disabled>Assign ${agent} a Task</button></div><h2>Your agent is ready</h2><button>Assign ${agent} a Task</button>`,
"narrative-ready": `<p>The assistant says Your agent is ready.</p><h2>Finishing agent setup</h2><button disabled>Assign ${agent} a Task</button>`,
done: `<div hidden data-testid="issue-detail-header"><button aria-label="Change status (current: Todo)">Todo</button></div><div data-testid="issue-detail-header"><button aria-label="Change status (current: Done)"><svg><title>Done</title></svg></button></div><div role="menu" hidden>Done</div><p>Earlier task: Todo</p>`,
"done-scrolled": `<nav aria-label="breadcrumb"><span aria-current="page"><svg role="img" aria-label="Done"><title>Done</title></svg>Current task</span></nav><div id="thread" style="height:200px;overflow:auto"><div data-testid="issue-detail-header" style="height:1000px"><button aria-label="Change status (current: Done)">Done</button></div><div style="height:1000px">Restored latest messages and open file viewer</div></div><script>document.getElementById('thread').scrollTop=2000</script>`,
"wrong-current-breadcrumb": `<nav aria-label="breadcrumb"><a><svg role="img" aria-label="Done"><title>Done</title></svg>Older task</a><span aria-current="page"><svg role="img" aria-label="Todo"><title>Todo</title></svg>Current task</span></nav><div data-testid="issue-detail-header"><button aria-label="Change status (current: Done)">Done</button></div>`,
"hidden-breadcrumb-done": `<nav aria-label="breadcrumb"><span aria-current="page"><svg style="display:none" role="img" aria-label="Done"><title>Done</title></svg><svg role="img" aria-label="Todo"><title>Todo</title></svg>Current task</span></nav><div data-testid="issue-detail-header"><button aria-label="Change status (current: Done)">Done</button></div>`,
"narrative-done": `<div data-testid="issue-detail-header"><button aria-label="Change status (current: Todo)"><svg><title>Done</title></svg></button></div><div hidden><button aria-label="Change status (current: Done)">Done</button></div><p>The task is Done</p>`,
composer: `<div role="dialog" aria-labelledby="composer"><h2 id="composer">New task</h2><button aria-label="Select assignee">${agent}</button><button aria-label="Select model and effort"><span data-testid="task-chat-composer-model-label">${model}</span></button><div hidden data-testid="new-issue-paused-assignee-note">paused</div></div><div data-testid="new-issue-paused-assignee-note">unrelated paused agent</div>`,
"paused-composer": `<div role="dialog" aria-labelledby="composer"><h2 id="composer">New task</h2><button aria-label="Select assignee">${agent}</button><button aria-label="Select model and effort"><span data-testid="task-chat-composer-model-label">${model}</span></button><div data-testid="new-issue-paused-assignee-note">paused</div></div>`,
"wrong-model": `<div role="dialog" aria-labelledby="composer"><h2 id="composer">New task</h2><button aria-label="Select assignee">${agent}</button><button aria-label="Select model and effort"><span data-testid="task-chat-composer-model-label">gemini-3.8-flash-high</span></button><p>${model}</p></div>`,
};
export async function runAntigravityBrowserCalibration(page: Page) {
// The activity calibration bundles the shipped adapter and React renderer.
// No hand-authored activity markup can accidentally validate invented copy.
const activityBundle = await build({ stdin: { loader: "tsx", resolveDir: path.resolve(import.meta.dirname, "../../ui"), contents: `
import React from 'react';
import { createRoot } from 'react-dom/client';
import { TaskChatRunnerActivityGroup } from './src/components/task-chat/TaskChatRunnerActivityGroup';
import { buildTurnTimelineRows, transcriptToTaskChatItems } from './src/components/task-chat/transcript-adapter';
const route = new URL(location.href).pathname.slice(1);
const legacy = route.startsWith('legacy-'), scenario = route.replace(/^legacy-/, '');
const target = scenario === 'wrong-target-activity' ? 'backup/antigravity-proof.txt' : 'antigravity-proof.txt';
const event = (operation, status, executionId, name = 'client_' + operation + '_file') => ({
kind: 'provider_activity', ts: '2026-10-29T00:00:00Z', family: 'tool_execution',
eventType: status === 'running' ? 'tool.execution.started' : 'tool.execution.completed', status,
title: 'Tool execution', summary: 'Native file operation',
payload: { name, operation, target, status, executionId, transport: 'builtin' },
});
const entries = [event('edit', 'completed', 'edit'), event('read', 'completed', 'read'),
event('execute', 'failed', 'finish', 'paperclip_paperclip_finish')];
if (scenario === 'failed-read-activity') entries.unshift(event('read', 'failed', 'failed-read'));
if (scenario === 'interrupted-edit-activity') entries.unshift(event('edit', 'interrupted', 'interrupted-edit'));
if (scenario === 'running-read-activity') entries.unshift(event('read', 'running', 'running-read'));
if (scenario === 'pending-read-activity') entries.unshift(event('read', 'pending', 'pending-read'));
if (scenario === 'failed-renderer-activity') entries.unshift({ ...event('read', 'completed', 'failed-renderer'), status: 'failed' });
if (scenario === 'interrupted-renderer-activity') entries.unshift({ ...event('edit', 'completed', 'interrupted-renderer'), status: 'interrupted' });
if (scenario === 'write-activity') entries[0] = event('write', 'completed', 'write');
const running = legacy && ['running-read-activity', 'pending-read-activity'].includes(scenario);
const transcriptOf = values => legacy ? values.flatMap(entry => {
const { operation, target, executionId } = entry.payload;
const call = { kind: 'tool_call', ts: entry.ts, toolUseId: executionId,
name: operation === 'read' ? 'Running client_view_file' : operation === 'edit' ? 'exec_command' : operation === 'write' ? 'write' : 'paperclip_paperclip_finish',
input: { operation, target } };
if (['interrupted', 'pending'].includes(entry.status)) return [call];
return [call, { kind: 'tool_result', ts: entry.ts, toolUseId: executionId, isError: entry.status === 'failed',
delta: entry.status === 'running', content: entry.status === 'failed' ? 'Operation rejected' : 'Synthetic actual tool result' }];
}) : values;
const phases = (values, runId, active = false) => buildTurnTimelineRows(
transcriptToTaskChatItems(transcriptOf(values), { runId, agentName: 'Synthetic agent', running: active }), active,
).filter(item => item.kind === 'activity_phase');
const runId = scenario === 'stale-first-only-activity' ? 'synthetic-run-old' : 'synthetic-run';
const current = phases(entries, runId, running);
const older = scenario === 'current-run-mixed-activity'
? phases([event('read', 'failed', 'old-read'), event('edit', 'failed', 'old-edit')], 'synthetic-run-old') : [];
createRoot(document.getElementById('activity-root')).render(<>{[...older, ...current].map(item =>
<TaskChatRunnerActivityGroup key={item.id} item={item} />)}</>);
` }, bundle: true, write: false, format: "iife", platform: "browser", jsx: "automatic", tsconfig: path.resolve(import.meta.dirname, "../../ui/tsconfig.json"),
define: { "process.env.NODE_ENV": '"production"', "import.meta.env": "{}" }, logLevel: "silent" });
// The artifact controls and text viewer are the shipped components. Their
// friendly work-product title deliberately differs from the actual filename.
const artifactBundle = await build({ stdin: { loader: "tsx", resolveDir: path.resolve(import.meta.dirname, "../../ui"), contents: `
import React, { useState } from 'react';
import { createRoot } from 'react-dom/client';
import { RichWorkProductCard } from './src/components/task-chat/RichWorkProductCard';
import { ArtifactRow } from './src/components/artifacts/v1/ArtifactRow';
import { TextAttachmentContext } from './src/context/TextAttachmentContext';
import { TextAttachmentPreview } from './src/components/task-side-panel/TaskAttachmentPanel';
const route = new URL(location.href).pathname.slice(1), currentId = ${JSON.stringify(artifactId)}, secondId = ${JSON.stringify(followupId)};
const wrongId = '33333333-3333-4333-8333-333333333333';
const first = ${JSON.stringify(artifactBytes.toString('utf8'))}, second = ${JSON.stringify(followupBytes.toString('utf8'))};
const bytes = id => id === secondId ? second : first;
const product = (id, size = bytes(id).length) => ({ id: 'product-' + id, companyId: 'synthetic-company', issueId: 'synthetic-issue',
type: 'artifact', provider: 'paperclip', externalId: id, title: 'Prepared proof file', status: 'active', reviewState: 'none',
healthStatus: 'unknown', createdAt: '2026-10-29T00:00:00Z', updatedAt: '2026-10-29T00:00:00Z',
metadata: { attachmentId: id, originalFilename: 'antigravity-proof.txt', contentType: 'text/plain', byteSize: size,
contentPath: '/api/attachments/' + id + '/content', openPath: '/api/attachments/' + id + '/content', downloadPath: '/api/attachments/' + id + '/content?download=1' } });
function ArtifactCalibration() {
const [selected, setSelected] = useState(null);
const open = (id, title) => setSelected({ id: route === 'artifact-wrong-viewer-id' ? wrongId : id, title });
const render = (id, size) => <RichWorkProductCard workProduct={product(id, size)} href={'/api/attachments/' + id + '/content'} variant={route === 'artifact-compact' ? 'compact' : 'card'} />;
const row = <ArtifactRow subject={{ kind: 'file', title: 'Prepared proof file', filename: 'antigravity-proof.txt', sizeLabel: '61 B' }}
onClick={() => open(route === 'artifact-button-wrong-id' ? wrongId : currentId, 'antigravity-proof.txt')} />;
return <main><TextAttachmentContext.Provider value={open}>
{route.startsWith('artifact-button') ? <>{row}{route === 'artifact-button-duplicate' ? row : null}</> :
route === 'artifact-hidden' ? <><div hidden>{render(currentId)}</div>{render(wrongId)}</> :
route === 'artifact-followup' ? <>{render(currentId)}{render(secondId)}</> :
<>{render(currentId, route === 'artifact-wrong-size' ? 119 : 61)}{route === 'artifact-duplicate' ? render(currentId) : null}</>}
{selected ? <div hidden={route === 'artifact-hidden-viewer'}><TextAttachmentPreview title={selected.title}
text={route === 'artifact-wrong-preview' ? 'Wrong file bytes' : bytes(selected.id)} markdown={false}
downloadUrl={'/api/attachments/' + selected.id + '/content?download=1'} /></div> : null}
</TextAttachmentContext.Provider></main>;
}
createRoot(document.getElementById('artifact-root')).render(<ArtifactCalibration />);
` }, bundle: true, write: false, format: "iife", platform: "browser", jsx: "automatic", tsconfig: path.resolve(import.meta.dirname, "../../ui/tsconfig.json"),
loader: { ".svg": "dataurl" }, define: { "process.env.NODE_ENV": '"production"', "import.meta.env": "{}" }, logLevel: "silent" });
// Render the shipped wizard and actual agentsApi: only the selected-company
// context is fixed. All draft creation crosses the real browser POST boundary.
const creationCompanyId = "33333333-3333-4333-8333-333333333333", creatorUserId = "synthetic-user";
const creationName = "Antigravity QA 0123456789ab";
const creationBundle = await build({ stdin: { loader: "tsx", resolveDir: path.resolve(import.meta.dirname, "../../ui"), contents: `
import React from 'react';
import { createRoot } from 'react-dom/client';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { AgentBasicsDialog } from './src/components/new-agent/AgentBasicsDialog';
const cache = new QueryClient({ defaultOptions: { queries: { retry: false } } });
createRoot(document.getElementById('creation-root')).render(<QueryClientProvider client={cache}>
<AgentBasicsDialog open onClose={() => {}} onContinue={() => {}} />
</QueryClientProvider>);
` }, bundle: true, write: false, format: "iife", platform: "browser", jsx: "automatic", tsconfig: path.resolve(import.meta.dirname, "../../ui/tsconfig.json"),
loader: { ".svg": "dataurl" }, define: { "process.env.NODE_ENV": '"production"', "import.meta.env": "{}" }, logLevel: "silent",
plugins: [{ name: "selected-company-calibration", setup(build) {
build.onResolve({ filter: /CompanyContext$/ }, () => ({ path: "selected-company", namespace: "calibration" }));
build.onLoad({ filter: /.*/, namespace: "calibration" }, () => ({ contents: `export const useCompany = () => ({ selectedCompanyId: '${creationCompanyId}' });`, loader: "js" }));
} }] });
let persistedCreation: Record<string, unknown> | undefined;
let creationMode: "normal" | "lost-response" | "wrong-response-id" | "foreign-response-creator" | "malformed-response" = "normal";
const receivedCreatePaths: string[] = [];
const creationRequests: Array<{ body: Record<string, any>; raw: string; durableBeforeForward: boolean }> = [];
const server = createServer(async (request, response) => {
const name = new URL(request.url ?? "/", "http://localhost").pathname.slice(1);
if (request.method === "POST" && /^api\/companies\/[^/]+\/agents$/u.test(name)) receivedCreatePaths.push(name);
if (name === "creation-client.js") { response.writeHead(200, { "Content-Type": "text/javascript" }).end(creationBundle.outputFiles[0]!.text); return; }
if (name === "creation") { response.writeHead(200, { "Content-Type": "text/html; charset=utf-8" }).end('<!doctype html><html><body><div id="creation-root"></div><script src="/creation-client.js"></script></body></html>'); return; }
if (name === "api/instance/settings/experimental") { response.writeHead(200, { "Content-Type": "application/json" }).end(JSON.stringify({ enableNativeRunner: true })); return; }
if (name === "api/adapters") { response.writeHead(200, { "Content-Type": "application/json" }).end(JSON.stringify([{ type: "paperclip_runner", loaded: true, candidateHarnesses: [{ agent: "antigravity", model }] }])); return; }
if (name === `api/companies/${creationCompanyId}/agents`) {
if (request.method === "POST") {
let raw = ""; for await (const chunk of request) raw += chunk;
const body = JSON.parse(raw);
const expected = { setupId: body.setupId, companyId: creationCompanyId, agentName: body.name, provider: "antigravity", creatorUserId, origin: `http://${request.headers.host}` };
creationRequests.push({ body, raw, durableBeforeForward: persistedCreation != null && Object.keys(persistedCreation).length === 6 && Object.entries(expected).every(([key, value]) => persistedCreation![key] === value) });
// Simulate a committed row whose response is lost, without re-sending
// the POST or invoking any agent/provider work.
if (creationMode === "lost-response") {
response.writeHead(201, { "Content-Type": "application/json", "Content-Length": "1000" });
response.flushHeaders(); response.write('{"id":');
setTimeout(() => response.destroy(), 25); return;
}
if (creationMode === "malformed-response") { response.writeHead(201, { "Content-Type": "application/json" }).end("{"); return; }
response.writeHead(201, { "Content-Type": "application/json" }).end(JSON.stringify({ ...body, id: creationMode === "wrong-response-id" ? artifactId : body.setupId,
companyId: creationCompanyId, createdByUserId: creationMode === "foreign-response-creator" ? "another-user" : creatorUserId, lifecycleState: "draft" })); return;
}
response.writeHead(200, { "Content-Type": "application/json" }).end(JSON.stringify([{ id: followupId, companyId: creationCompanyId, role: "ceo", status: "idle" }])); return;
}
if (/^api\/attachments\/[^/]+\/content$/u.test(name)) {
const referer = String(request.headers.referer ?? "");
const bytes = referer.includes("artifact-bad-download") ? Buffer.from("C".repeat(artifactBytes.length)) : name.includes(followupId) ? followupBytes : artifactBytes;
response.writeHead(200, { "Content-Type": "text/plain", "Content-Disposition": 'attachment; filename="antigravity-proof.txt"' }).end(bytes); return;
}
if (name === "artifact-client.js") { response.writeHead(200, { "Content-Type": "text/javascript" }).end(artifactBundle.outputFiles[0]!.text); return; }
if (name.startsWith("artifact")) {
response.writeHead(200, { "Content-Type": "text/html; charset=utf-8" }).end('<!doctype html><html><body><div id="artifact-root"></div><script src="/artifact-client.js"></script></body></html>'); return;
}
if (name === "activity-client.js") { response.writeHead(200, { "Content-Type": "text/javascript" }).end(activityBundle.outputFiles[0]!.text); return; }
if (name.endsWith("activity")) {
response.writeHead(200, { "Content-Type": "text/html; charset=utf-8" }).end(`<!doctype html><html><head><style>svg{width:14px;height:14px}button{display:block}</style></head><body><button onclick="document.getElementById('activity-root').hidden=false;this.remove()">More messages</button><div id="activity-root" hidden></div><p>The file work succeeded; unrelated task failed.</p><script src="/activity-client.js"></script></body></html>`); return;
}
response.writeHead(200, { "Content-Type": "text/html; charset=utf-8" }).end(`<!doctype html><html><body>${fixtures[name] ?? "missing fixture"}</body></html>`);
});
await new Promise<void>(resolve => server.listen(0, "127.0.0.1", resolve));
const origin = `http://127.0.0.1:${(server.address() as { port: number }).port}`;
let removeCreationObserver: (() => void) | undefined;
try {
const visit = async (fixture: string) => page.goto(`${origin}/${fixture}`);
const creationResources = (): AntigravityJourneyResources => ({ version: 1, executionId: ANTIGRAVITY_JOURNEY_EXECUTION, origin, companyId: creationCompanyId,
environmentId: artifactId, connectionId: followupId, agentName: creationName, runIds: [], processes: {}, retained: false, setupProbe: "not_started" });
const beginCreation = async () => {
await visit("creation");
const dialog = page.getByRole("dialog", { name: "Meet your next agent", exact: true });
await dialog.getByRole("textbox", { name: "Agent name", exact: true }).fill(creationName);
return () => dialog.getByRole("button", { name: "Choose adapter", exact: true }).click();
};
const originalPostBodies: string[] = [];
const observeCreation = (request: import("@playwright/test").Request) => {
if (request.method() === "POST" && new URL(request.url()).pathname === `/api/companies/${creationCompanyId}/agents`) originalPostBodies.push(request.postData()!);
};
page.on("request", observeCreation); removeCreationObserver = () => page.off("request", observeCreation);
const owned = creationResources(), submit = await beginCreation();
let writes = 0;
const draft = await captureAntigravityAgentDraft(page, owned, async () => { writes++; persistedCreation = structuredClone((owned as any).agentCreationRequest); }, submit, creatorUserId);
assert.equal(writes, 1, "persist the safe creation identity once before network forwarding");
assert.equal(creationRequests.length, 1);
assert.equal(creationRequests[0]!.durableBeforeForward, true, "creation witness must be durable before the server receives the unchanged POST");
assert.deepEqual((owned as any).agentCreationRequest, { setupId: draft.id, companyId: creationCompanyId, agentName: creationName, provider: "antigravity", creatorUserId, origin });
assert.equal(creationRequests[0]!.raw, originalPostBodies[0], "forward the original UI POST body without mutation");
assert.equal(creationRequests[0]!.body.setupId, draft.id);
assert.equal(creationRequests[0]!.body.metadata.setupAdapterPending, true);
assert.equal(creationRequests[0]!.body.runtimeConfig.heartbeat.enabled, false);
assert.equal(JSON.stringify(persistedCreation).includes("runtimeConfig"), false);
await page.getByRole("dialog", { name: "Choose an adapter", exact: true }).waitFor({ state: "visible" });
await page.locator('input[type="radio"][value="antigravity_runtime"]').waitFor({ state: "attached" });
// A storage barrier proves that request forwarding joins the durable write.
const gateOwned = creationResources(), gateSubmit = await beginCreation(), beforeGate = creationRequests.length;
let releaseWrite!: () => void, enteredWrite!: () => void;
const writeGate = new Promise<void>(resolve => { releaseWrite = resolve; });
const writeEntered = new Promise<void>(resolve => { enteredWrite = resolve; });
const gated = captureAntigravityAgentDraft(page, gateOwned, async () => {
enteredWrite(); await writeGate; persistedCreation = structuredClone((gateOwned as any).agentCreationRequest);
}, gateSubmit, creatorUserId);
try { await writeEntered; assert.equal(creationRequests.length, beforeGate, "do not forward while persistence is blocked"); }
finally { releaseWrite(); }
await gated;
assert.equal(creationRequests.at(-1)!.durableBeforeForward, true);
assert.equal(creationRequests.at(-1)!.raw, originalPostBodies.at(-1));
const storageOwned = creationResources(), storageSubmit = await beginCreation(), beforeStorageFailure = creationRequests.length;
const storageError = new Error("synthetic creation receipt storage unavailable");
await assert.rejects(captureAntigravityAgentDraft(page, storageOwned, async () => { throw storageError; }, storageSubmit, creatorUserId), error => error === storageError);
assert.equal(creationRequests.length, beforeStorageFailure, "failed persistence must abort before any server commit");
for (const mode of ["wrong-response-id", "foreign-response-creator", "malformed-response"] as const) {
creationMode = mode; persistedCreation = undefined;
const wrong = creationResources(), wrongSubmit = await beginCreation();
await assert.rejects(captureAntigravityAgentDraft(page, wrong, async () => { persistedCreation = structuredClone((wrong as any).agentCreationRequest); }, wrongSubmit, creatorUserId));
assert.equal(creationRequests.at(-1)!.durableBeforeForward, true);
assert.equal((wrong as any).agentCreationRequest.setupId, creationRequests.at(-1)!.body.setupId);
assert.equal(wrong.agentId, undefined, "an untrusted response cannot replace the exact request identity");
}
creationMode = "lost-response"; persistedCreation = undefined;
const lost = creationResources(), lostSubmit = await beginCreation(), beforeLost = creationRequests.length;
page.setDefaultTimeout(1500);
try {
await assert.rejects(captureAntigravityAgentDraft(page, lost, async () => { persistedCreation = structuredClone((lost as any).agentCreationRequest); }, lostSubmit, creatorUserId));
assert.equal(creationRequests.length, beforeLost + 1, "a lost response must not replay creation");
assert.equal(creationRequests.at(-1)!.durableBeforeForward, true);
assert.equal((lost as any).agentCreationRequest.setupId, creationRequests.at(-1)!.body.setupId);
assert.equal(lost.agentId, undefined);
} finally { page.setDefaultTimeout(10000); }
creationMode = "normal";
for (const patch of [{ setupId: "not-a-uuid" }, { name: "Existing user agent" }, { setupId: undefined }, { company: artifactId }]) {
const malformed = creationResources(); const before: number = receivedCreatePaths.length;
let malformedWrites = 0;
await assert.rejects(captureAntigravityAgentDraft(page, malformed, async () => { malformedWrites++; }, async () => {
await page.evaluate(({ route, body }) => { void fetch(route, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(body) }).catch(() => {}); },
{ route: `/api/companies/${"company" in patch ? patch.company : creationCompanyId}/agents`, body: { setupId: artifactId, name: creationName, ...patch } });
}, creatorUserId));
assert.equal(malformedWrites, 0); assert.equal(receivedCreatePaths.length, before, "reject an invalid create identity before forwarding");
assert.equal((malformed as any).agentCreationRequest, undefined);
}
// A durable subset must never absorb unrelated fields from an outgoing body.
const privateBodyOwned = creationResources(); let privateWrite: string | undefined;
await captureAntigravityAgentDraft(page, privateBodyOwned, async () => {
persistedCreation = structuredClone((privateBodyOwned as any).agentCreationRequest); privateWrite = JSON.stringify(privateBodyOwned);
}, async () => {
await page.evaluate(({ route, body }) => { void fetch(route, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(body) }).catch(() => {}); },
{ route: `/api/companies/${creationCompanyId}/agents`, body: { setupId: artifactId, name: creationName, runtimeConfig: { env: { SYNTHETIC_PRIVATE_VALUE: "must-never-be-persisted" } } } });
}, creatorUserId);
assert.equal(privateWrite!.includes("must-never-be-persisted"), false);
assert.equal(creationRequests.at(-1)!.body.runtimeConfig.env.SYNTHETIC_PRIVATE_VALUE, "must-never-be-persisted", "the original request still forwards unchanged");
const submitError = new Error("synthetic ordinary submit failure");
await assert.rejects(captureAntigravityAgentDraft(page, creationResources(), async () => {}, async () => { throw submitError; }, creatorUserId), error => error === submitError);
const abortedOwned = creationResources(), abortSubmit = await beginCreation(), beforeAbort = creationRequests.length;
const abortError = new Error("synthetic route abort cleanup failure");
const failingAbortPage = new Proxy(page, { get(target, key) {
if (key === "route") return (url: any, handler: any) => target.route(url, (route, request) => handler(new Proxy(route, { get(owner, operation) {
if (operation === "abort") return async (...args: any[]) => { await owner.abort(...args); throw abortError; };
const value = Reflect.get(owner, operation); return typeof value === "function" ? value.bind(owner) : value;
} }), request));
if (key === "unroute") return (url: any) => target.unroute(url);
const value = Reflect.get(target, key); return typeof value === "function" ? value.bind(target) : value;
} });
await assert.rejects(captureAntigravityAgentDraft(failingAbortPage, abortedOwned, async () => { throw storageError; }, abortSubmit, creatorUserId), error => error === storageError);
assert.equal(creationRequests.length, beforeAbort, "an abort-cleanup error cannot forward an unjournaled create");
const teardownError = new Error("synthetic route teardown failure");
const failingTeardownPage = new Proxy(page, { get(target, key) {
if (key === "unroute") return async () => { throw teardownError; };
const value = Reflect.get(target, key); return typeof value === "function" ? value.bind(target) : value;
} });
try {
await assert.rejects(captureAntigravityAgentDraft(failingTeardownPage, creationResources(), async () => {}, async () => { throw submitError; }, creatorUserId), error => error === submitError);
} finally { await page.unrouteAll({ behavior: "wait" }); }
const teardownOwned = creationResources(), teardownSubmit = await beginCreation();
try {
await assert.rejects(captureAntigravityAgentDraft(failingTeardownPage, teardownOwned, async () => {
persistedCreation = structuredClone((teardownOwned as any).agentCreationRequest);
}, teardownSubmit, creatorUserId), error => error === teardownError);
assert.equal(creationRequests.at(-1)!.durableBeforeForward, true);
assert.equal((teardownOwned as any).agentCreationRequest.setupId, creationRequests.at(-1)!.body.setupId);
} finally { await page.unrouteAll({ behavior: "wait" }); }
// After every error, a new ordinary wizard request still passes exactly one
// interceptor. This also detects a leaked route from the failed attempt.
const afterFailure = creationResources(), afterSubmit = await beginCreation(); let afterWrites = 0;
await captureAntigravityAgentDraft(page, afterFailure, async () => { afterWrites++; persistedCreation = structuredClone((afterFailure as any).agentCreationRequest); }, afterSubmit, creatorUserId);
assert.equal(afterWrites, 1); assert.equal(creationRequests.at(-1)!.durableBeforeForward, true);
removeCreationObserver(); removeCreationObserver = undefined;
await visit("dashboard"); await openAntigravityAgentFromRoster(page, agent, 300);
assert.equal(new URL(page.url()).pathname, "/owned-agent");
for (const fixture of ["artifact", "artifact-compact", "artifact-button-row"]) {
await visit(fixture); try { await inspectAntigravityArtifact(page, artifactId, artifactBytes, 1000); } catch (error) { throw new Error(`${fixture}: ${await page.locator("body").innerText()}`, { cause: error }); }
}
await visit("artifact-followup"); await inspectAntigravityArtifact(page, followupId, followupBytes, 1000);
for (const fixture of ["artifact-hidden", "artifact-duplicate", "artifact-wrong-size", "artifact-wrong-viewer-id", "artifact-wrong-preview", "artifact-hidden-viewer", "artifact-button-wrong-id", "artifact-button-duplicate"]) {
await visit(fixture); await assert.rejects(inspectAntigravityArtifact(page, artifactId, artifactBytes, 300), fixture);
}
await visit("artifact-bad-download");
await assert.rejects(inspectAntigravityArtifact(page, artifactId, artifactBytes, 1000), /browser_download_bytes_mismatch/u);
for (const [fixture, target] of [["task-uuid", "synthetic-issue-id"], ["task-identifier", "ANT-42"], ["task-duplicate", "synthetic-issue-id"], ["task-repeated", "ANT-42"]]) {
await visit(fixture); await openAntigravityCreatedTask(page, { id: "synthetic-issue-id", identifier: "ANT-42" }, 300);
assert.equal(new URL(page.url()).pathname, '/ANT/issues/' + target);
}
for (const fixture of ["task-hidden", "task-other-only"]) {
await visit(fixture); await assert.rejects(openAntigravityCreatedTask(page, { id: "synthetic-issue-id", identifier: "ANT-42" }, 300), fixture);
}
await visit("picker"); await selectAntigravityFromOrdinaryPicker(page, 300);
assert.equal(await page.getByRole("radio", { name: "Google Antigravity Candidate", exact: true }).isChecked(), true);
await visit("advanced-only"); await assert.rejects(selectAntigravityFromOrdinaryPicker(page, 300));
await visit("ready"); await assertAntigravityReady(page, agent, 300);
await visit("narrative-ready"); await assert.rejects(assertAntigravityReady(page, agent, 300));
await visit("done"); await assertAntigravityTaskDone(page, 300);
await visit("done-scrolled");
const restoredScroll = await page.locator("#thread").evaluate(node => node.scrollTop);
assert.ok(restoredScroll > 0, "Done regression must start at a restored nonzero scroll position");
await assertAntigravityTaskDone(page, 300);
assert.equal(await page.locator("#thread").evaluate(node => node.scrollTop), restoredScroll);
for (const fixture of ["narrative-done", "wrong-current-breadcrumb", "hidden-breadcrumb-done"]) {
await visit(fixture); await assert.rejects(assertAntigravityTaskDone(page, 300), fixture);
}
await visit("composer"); await assertAntigravityCleanComposer(page, agent, model, 300);
await visit("paused-composer"); await assert.rejects(assertAntigravityCleanComposer(page, agent, model, 300));
await visit("wrong-model"); await assert.rejects(assertAntigravityCleanComposer(page, agent, model, 300));
for (const fixture of ["stale-first-only-activity", "legacy-stale-first-only-activity"]) {
await visit(fixture); await assert.rejects(assertAntigravityFileActivity(page, "synthetic-run", 1000), fixture);
}
await visit("activity"); await assertAntigravityFileActivity(page, "synthetic-run", 1000);
await visit("write-activity"); await assertAntigravityFileActivity(page, "synthetic-run", 1000);
for (const fixture of ["current-run-mixed-activity", "legacy-current-run-mixed-activity"]) {
await visit(fixture); await assertAntigravityFileActivity(page, "synthetic-run", 1000);
}
for (const fixture of ["failed-read-activity", "interrupted-edit-activity", "running-read-activity", "failed-renderer-activity", "interrupted-renderer-activity", "wrong-target-activity"]) {
await visit(fixture);
let rejected = false;
try { await assertAntigravityFileActivity(page, "synthetic-run", 300); } catch { rejected = true; }
assert.ok(rejected, `${fixture}: ${await page.locator("body").innerText()}`);
}
for (const fixture of ["legacy-activity", "legacy-write-activity"]) {
await visit(fixture); await assertAntigravityFileActivity(page, "synthetic-run", 1000);
assert.equal(await page.locator('[data-activity-family="tool_execution"]').count(), 0);
}
for (const fixture of ["legacy-failed-read-activity", "legacy-interrupted-edit-activity", "legacy-running-read-activity", "legacy-pending-read-activity", "legacy-wrong-target-activity"]) {
await visit(fixture);
let rejected = false;
try { await assertAntigravityFileActivity(page, "synthetic-run", 300); } catch { rejected = true; }
assert.ok(rejected, `${fixture}: ${await page.locator("body").innerText()}`);
}
console.log("PASS Antigravity browser calibration: production AgentBasicsDialog/agentsApi safe creation witness durable before unchanged POST, persistence gate/failure, lost201-body recovery identity, malformed/foreign request and response rejection, no raw-body/env persistence, abort/unroute primary-error preservation; production rich/compact artifact cards and rows, exact attachment viewer/download identity+61/119B, hidden/duplicate/wrong-ID/size/bytes rejection, both task sidebar layouts; More messages plus production transcript adapter/activity renderer for canonical provider activity and native legacy tool calls/results; exact run-bound completed read/edit/write, stale-first-only rejection/current-run mixed-history acceptance, exact target, failed/interrupted/running/pending rejection and unrelated failed finish");
} finally {
removeCreationObserver?.();
await new Promise<void>((resolve, reject) => server.close(error => error ? reject(error) : resolve()));
}
}
// Keep the documented standalone entry while the canonical support spec uses
// Playwright's existing page/browser lifecycle instead of launching a second runner.
if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) {
const browser = await chromium.launch({ headless: true });
try {
await runAntigravityBrowserCalibration(await browser.newPage());
} finally {
await browser.close();
}
}
tests/runner-e2e/antigravity-journey-browser.spec.ts
+7
View File
@@ -0,0 +1,7 @@
import { test } from "@playwright/test";
import { runAntigravityBrowserCalibration } from "./antigravity-journey-browser.smoke.js";
test("Antigravity selectors match the shipped activity and artifact components", async ({ page }) => {
test.setTimeout(60_000);
await runAntigravityBrowserCalibration(page);
});
tests/runner-e2e/antigravity-journey-cases.ts
+31
View File
@@ -0,0 +1,31 @@
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
import { ANTIGRAVITY_JOURNEY_MODEL } from "./antigravity-journey-config.js";
import type { EnvironmentFixture, RunnerProfileFixture, RunnerSuiteFixture } from "./types.js";
export const ANTIGRAVITY_JOURNEY_SUITE = "antigravity-agent-journey";
export const ANTIGRAVITY_JOURNEY_CASE = "saved-subscription-two-turn";
export const ANTIGRAVITY_JOURNEY_EXECUTION = `${ANTIGRAVITY_JOURNEY_SUITE}.runner-antigravity.local.${ANTIGRAVITY_JOURNEY_CASE}`;
export function buildAntigravityJourneySuite(environments: readonly EnvironmentFixture[]): RunnerSuiteFixture {
const profile: RunnerProfileFixture = { id: "runner-antigravity", label: "Google Antigravity candidate", generation: "native", groups: [],
// The catalog's legacy required field is never resolved for this flow;
// requiredCredentials is empty and the saved product connection is verified.
adapterType: "paperclip_runner", provider: "acpx", model: ANTIGRAVITY_JOURNEY_MODEL, credential: "GEMINI_API_KEY",
supportedEnvironments: ["local"], expectedRuntimeMode: "native", expectedRuntimeMetadata: { adapterType: "paperclip_runner", provider: "acpx" },
modelQualification: { source: "candidate_runner_profile", qualificationId: "antigravity:operator-admitted-darwin-arm64-saved-subscription" },
buildAgent: () => { throw new Error("The measured Antigravity agent must be created through the ordinary UI"); } };
const hash = createHash("sha256");
for (const file of ["antigravity-journey-cases.ts", "antigravity-journey-config.ts", "antigravity-journey-fixtures.ts", "antigravity-journey-flow.ts", "antigravity-journey-launch.ts", "antigravity-journey-evidence.ts", "antigravity-journey-report.ts", "antigravity-journey-ui.ts", "copilot-local-fixtures.ts", "user-actions.ts",
"../../packages/paperclip-runner/src/drivers/acpx/generated-profiles.ts", "../../packages/paperclip-runner/antigravity-distributions.json"])
hash.update(file).update(readFileSync(new URL(`./${file}`, import.meta.url)));
return { id: ANTIGRAVITY_JOURNEY_SUITE, label: "Antigravity additional-agent setup and two-turn task", groups: [], manualOnly: true,
description: "Ordinary agent picker, saved Google subscription, exact Low model, native setup test, instruction-only file proof and persisted follow-up.",
profiles: [profile], environments: environments.filter(environment => environment.id === "local"), expectedMatrixSize: 1,
definitionMetadata: { version: 1, digest: hash.digest("hex"), scheduling: "explicit-only", authentication: "existing-same-user-personal-default-google-subscription",
setup: "attached-self-hosted-local-darwin-arm64", admission: "trusted-operator-candidate-startup-required", freshOAuth: false, globalQualification: "pending", mcpConnections: [],
dependencies: [{ alias: "browser_operator", consumer: "browser" }, { alias: "product_agent", consumer: "product-agent", phase: "created-in-journey" }, { alias: "independent_observer", consumer: "harness" }] },
tasks: [{ id: ANTIGRAVITY_JOURNEY_CASE, label: "Add an agent, reuse subscription, complete a task and follow-up", groups: [], flow: "antigravity_agent_journey", workMode: "standard",
expectedRunCount: 2, minimumExpectedRunCount: 2, automaticRetryPolicy: "single_attempt", attemptTimeoutMs: { local: 1_800_000, daytona: 1_800_000 }, turnTimeoutMs: 300_000,
expectedTerminalState: { issue: "done", run: "succeeded" }, buildTitle: nonce => `Antigravity file proof ${nonce}`, buildPrompt: () => "Runtime-generated instruction-only nonce file proof.", buildVisibleMarker: nonce => nonce, buildMatchers: () => [] }] };
}
tests/runner-e2e/antigravity-journey-config.test.ts
+562
View File
@@ -0,0 +1,562 @@
import { expect, it, vi } from "vitest";
import { runnerMatrix } from "./catalog.js";
import { ANTIGRAVITY_JOURNEY_EXECUTION } from "./antigravity-journey-cases.js";
import { parseAntigravityJourneyConfig } from "./antigravity-journey-config.js";
import { ConnectionFailure } from "./connection-config.js";
import { antigravityNoLaunchReceipt, cleanupAntigravityJourney, observeAntigravityRun, parseAntigravityOwnedResources, selectAntigravityJourneyEnvironment, type AntigravityJourneyResources, type AntigravityProcessObservers } from "./antigravity-journey-fixtures.js";
import { parseRunnerSelectors, selectRunnerExecutions } from "./selectors.js";
import type { ConnectionApi } from "./connection-target.js";
import { antigravityKnownCostObservation } from "./antigravity-journey-launch.js";
// These cleanup-ordering calibrations use synthetic retired process receipts.
vi.mock("node:child_process", async original => ({ ...await original<typeof import("node:child_process")>(), execFileSync: vi.fn(() => "") }));
vi.mock("./antigravity-journey-evidence.js", async original => ({ ...await original<typeof import("./antigravity-journey-evidence.js")>(),
reacquireAntigravityNativeFences: vi.fn(async () => {}) }));
const companyId = "11111111-1111-4111-8111-111111111111", environmentId = "22222222-2222-4222-8222-222222222222";
const connectionId = "33333333-3333-4333-8333-333333333333", agentId = "44444444-4444-4444-8444-444444444444", runId = "55555555-5555-4555-8555-555555555555";
function fixture() { return { version: 1, setupId: "synthetic", hosting: "self-hosted", executionEnvironment: "local",
target: { mode: "attach", baseURL: "http://127.0.0.1:3104", expectedCommit: "a".repeat(40), deploymentMode: "local_trusted", companyId, environmentId },
browser: { headed: false, channel: "chromium", freshness: "signed-in" }, savedConnection: { id: connectionId, name: "Synthetic Google subscription", ownerUserId: "synthetic-user" },
candidate: { platform: "darwin", architecture: "arm64", nativeVersion: "1.4.3-paperclip.5", profileDigest: `sha256:${"b".repeat(64)}`, nativeClosureDigest: `sha256:${"c".repeat(64)}` },
limits: { durationMs: 900_000, turnTimeoutMs: 300_000, maxRuns: 2, budgetCents: 100 } }; }
function resources(): AntigravityJourneyResources { return { version: 1, executionId: ANTIGRAVITY_JOURNEY_EXECUTION, origin: "http://127.0.0.1:3104",
companyId, environmentId, connectionId, agentName: "Antigravity QA 0123456789ab", agentId, runIds: [], processes: {}, retained: false, setupProbe: "not_started" }; }
function neverLaunched() {
const owned = resources(); owned.runIds = [runId]; owned.issueId = "66666666-6666-4666-8666-666666666666"; owned.setupProbe = "retired";
const leaseId = "77777777-7777-4777-8777-777777777777";
const run: Record<string, any> = { id: runId, companyId, agentId, issueId: owned.issueId, status: "cancelled", executionStage: "settled", costAccountingPending: false,
runtimeMode: "legacy", runtimeModeResolvedAt: null, controllerLeaseExpiresAt: null, processPid: null, processGroupId: null, processStartedAt: null,
runnerInstanceId: null, nativeSessionId: null, driverKind: null, completionContractId: null, runnerProfileJson: { adapterDispatch: { adapterType: "paperclip_runner" } },
resultJson: { startupCancellation: { beforeNativeSelection: true, errorCode: "cancelled", requestedAt: "2026-01-01T00:00:00Z" }, startupPreparationSettledAt: "2026-01-01T00:00:02Z" },
contextSnapshot: { paperclipEnvironment: { id: environmentId, driver: "local", leaseId } } };
const lease: Record<string, any> = { id: leaseId, companyId, heartbeatRunId: runId, environmentId, status: "expired", releasedAt: "2026-01-01T00:00:01Z" };
return { owned, run, lease };
}
it("retains accepted physical ownership when a primary report projection throws", () => {
const { owned, run } = neverLaunched(), pid = 900001, start = "2026-01-01T00:00:00Z";
Object.assign(run, { processPid: pid, processGroupId: pid, processStartedAt: start });
owned.processes[runId] = { captured: false, identityChanged: false, journal: [] };
const journal = [{ pid, parent: 1, start }, { pid: pid + 1, parent: pid, start }];
const observers = new Map([[runId, { sample: () => ({ captured: true, identityChanged: false, live: [pid, pid + 1], journal,
authorityCheck: { accepted: true, commandSha256: "malformed-projection-digest", authority: { runId, pid, groupId: pid, startedAt: start }, observed: journal[0] } }) }]]) as unknown as AntigravityProcessObservers;
expect(() => observeAntigravityRun(owned, run, observers)).toThrow(/primary_owner_/u);
expect(owned.processes[runId]).toEqual({ captured: true, identityChanged: false, journal });
expect(owned.processes[runId]!.primaryOwner).toBeUndefined();
});
it.each(["released", "expired"])("persists authoritative pre-native cancellation with a %s lease before purge", async status => {
const { owned, run, lease } = neverLaunched(); lease.status = status;
let lifecycleState = "ready", saved = false;
const agent = () => ({ id: agentId, companyId, name: owned.agentName, lifecycleState });
const get = vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [agent()];
if (route.includes("heartbeat-runs?")) return [run];
if (route === `/api/heartbeat-runs/${runId}`) return run;
if (route === `/api/environment-leases/${lease.id}`) return lease;
if (route === `/api/issues/${owned.issueId}`) return { id: owned.issueId, companyId, assigneeAgentId: agentId, status: "cancelled" };
if (route === `/api/agents/${agentId}`) return agent();
if (route.endsWith("/ai-connections")) return { connections: [{ id: connectionId, provider: "google", method: "subscription", ownerUserId: "synthetic-user", isDefault: true, status: "connected" }] };
throw new Error(`Unexpected cleanup read ${route}`);
});
const remove = vi.fn(async () => { expect(saved).toBe(true); expect(owned.processes[runId]?.captured).toBe(false); });
const post = vi.fn(async (route: string) => { expect(route).toBe(`/api/agents/${agentId}/terminate`); lifecycleState = "terminated"; });
const cleanup = { agent: false, runs: false, connectionPreserved: false };
await cleanupAntigravityJourney({ api: { get, post, patch: async () => {}, delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: owned, cleanup, captureRun: async () => {}, persistResources: async () => {
if (owned.noLaunch?.[runId]) { expect(parseAntigravityOwnedResources(owned, parseAntigravityJourneyConfig(fixture()))).toBe(owned); saved = true; }
} });
expect(remove).toHaveBeenCalledOnce(); expect(cleanup).toEqual({ agent: true, runs: true, connectionPreserved: true });
expect(owned.noLaunch?.[runId]).toMatchObject({ kind: "cancelled-before-native-selection", runId, leaseStatus: status, leaseReleasedAt: lease.releasedAt });
});
it.each([
["missing settlement", (f: ReturnType<typeof neverLaunched>) => { delete f.run.resultJson.startupPreparationSettledAt; }],
["false before-selection receipt", (f: ReturnType<typeof neverLaunched>) => { f.run.resultJson.startupCancellation.beforeNativeSelection = false; }],
["preparation pending", (f: ReturnType<typeof neverLaunched>) => { f.run.executionStage = "preparing"; }],
["unsettled accounting", (f: ReturnType<typeof neverLaunched>) => { f.run.costAccountingPending = true; }],
["native owner", (f: ReturnType<typeof neverLaunched>) => { f.run.runnerInstanceId = "native-owner"; }],
["missing null owner", (f: ReturnType<typeof neverLaunched>) => { delete f.run.processPid; }],
["captured process", (f: ReturnType<typeof neverLaunched>) => { f.owned.processes[runId] = { captured: true, identityChanged: false, journal: [] }; }],
["unreleased expired lease", (f: ReturnType<typeof neverLaunched>) => { f.lease.releasedAt = null; }],
["active lease", (f: ReturnType<typeof neverLaunched>) => { f.lease.status = "active"; }],
["late lease release", (f: ReturnType<typeof neverLaunched>) => { f.lease.releasedAt = "2026-01-01T00:00:03Z"; }],
["foreign lease", (f: ReturnType<typeof neverLaunched>) => { f.lease.heartbeatRunId = agentId; }],
["foreign environment", (f: ReturnType<typeof neverLaunched>) => { f.lease.environmentId = agentId; }],
])("rejects no-launch cleanup with %s", (_label, edit) => {
const f = neverLaunched(); edit(f);
expect(() => antigravityNoLaunchReceipt(f.run, f.lease, f.owned)).toThrow("cleanup_no_launch_receipt_unconfirmed");
});
it("retains the agent if authoritative no-launch evidence cannot be persisted", async () => {
const { owned, run, lease } = neverLaunched(), post = vi.fn(), remove = vi.fn();
const get = vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [{ id: agentId, companyId, name: owned.agentName }];
if (route.includes("heartbeat-runs?")) return [run];
if (route === `/api/heartbeat-runs/${runId}`) return run;
if (route === `/api/environment-leases/${lease.id}`) return lease;
throw new Error("Unexpected cleanup read");
});
await expect(cleanupAntigravityJourney({ api: { get, post, patch: vi.fn(), delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: owned, cleanup: { agent: false, runs: false, connectionPreserved: false }, captureRun: async () => {},
persistResources: async () => { if (owned.noLaunch?.[runId]) throw new Error("fixture receipt storage unavailable"); } })).rejects.toThrow("fixture receipt storage unavailable");
expect(remove).not.toHaveBeenCalled(); expect(post).not.toHaveBeenCalled();
});
it("rejects altered retained no-launch identity, release timing, or extra fields", () => {
for (const patch of [{ runId: agentId }, { leaseStatus: "active" }, { leaseReleasedAt: "2026-01-01T00:00:03Z" }, { rawPrivatePayload: "unexpected" }]) {
const { owned, run, lease } = neverLaunched();
owned.noLaunch = { [runId]: { ...antigravityNoLaunchReceipt(run, lease, owned), ...patch } as any };
expect(() => parseAntigravityOwnedResources(owned, parseAntigravityJourneyConfig(fixture()))).toThrow("invalid_antigravity_owned_resources");
}
});
it.each(["settles", "timeout", "wrong_owner", "malformed"])("waits for an original pre-native cleanup owner without weakening proof (%s)", async scenario => {
vi.useFakeTimers();
try {
const { owned, run, lease } = neverLaunched(), settledRun = structuredClone(run), settledLease = structuredClone(lease);
run.executionStage = "preparing"; run.costAccountingPending = true; delete run.resultJson.startupPreparationSettledAt;
run.controllerLeaseExpiresAt = "2026-01-01T00:00:04Z"; lease.status = "active"; lease.releasedAt = null;
if (scenario === "wrong_owner") lease.companyId = agentId;
if (scenario === "malformed") run.resultJson.startupCancellation.requestedAt = "invalid-timestamp";
let fullReads = 0, lifecycleState = "ready", persisted = false;
const agent = () => ({ id: agentId, companyId, name: owned.agentName, lifecycleState });
const get = vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [agent()];
if (route.includes("heartbeat-runs?")) return [structuredClone(run)];
if (route === `/api/heartbeat-runs/${runId}`) {
if (++fullReads >= 3 && scenario === "settles") { Object.assign(run, settledRun); Object.assign(lease, settledLease); }
return structuredClone(run);
}
if (route === `/api/environment-leases/${lease.id}`) return structuredClone(lease);
if (route === `/api/issues/${owned.issueId}`) return { id: owned.issueId, companyId, assigneeAgentId: agentId, status: "cancelled" };
if (route === `/api/agents/${agentId}`) return agent();
if (route.endsWith("/ai-connections")) return { connections: [{ id: connectionId, provider: "google", method: "subscription", ownerUserId: "synthetic-user", isDefault: true, status: "connected" }] };
throw new Error("Unexpected cleanup read");
});
const remove = vi.fn(async () => { expect(persisted).toBe(true); });
const post = vi.fn(async (route: string) => { expect(route).toBe(`/api/agents/${agentId}/terminate`); lifecycleState = "terminated"; });
const outcome: { completed: boolean; error?: unknown } = { completed: false };
const operation = cleanupAntigravityJourney({ api: { get, post, patch: vi.fn(), delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: owned, cleanup: { agent: false, runs: false, connectionPreserved: false }, captureRun: async () => {},
persistResources: async () => { persisted ||= Boolean(owned.noLaunch?.[runId]); } }).then(() => { outcome.completed = true; }, error => { outcome.error = error; });
await vi.advanceTimersByTimeAsync(scenario === "timeout" ? 60_500 : 1_000);
await operation;
if (scenario === "settles") {
expect(outcome.error).toBeUndefined(); expect(outcome.completed).toBe(true); expect(fullReads).toBeGreaterThanOrEqual(3);
expect(remove).toHaveBeenCalledOnce(); expect(owned.noLaunch?.[runId]).toMatchObject({ leaseStatus: "expired", startupPreparationSettledAt: settledRun.resultJson.startupPreparationSettledAt });
} else {
expect(String(outcome.error)).toContain(scenario === "timeout" ? "cleanup_run_retirement_unconfirmed" : scenario === "wrong_owner" ? "cleanup_lease_owner_mismatch" : "cleanup_no_launch_receipt_unconfirmed");
expect(outcome.completed).toBe(false); expect(remove).not.toHaveBeenCalled(); expect(post).not.toHaveBeenCalled(); expect(persisted).toBe(false);
if (scenario !== "timeout") expect(fullReads).toBeLessThan(3);
}
} finally { vi.useRealTimers(); }
});
it("discovers one explicit-only real-agent cell without ambient API-key requirements", () => {
const selected = selectRunnerExecutions(parseRunnerSelectors(["--id", ANTIGRAVITY_JOURNEY_EXECUTION, "--antigravity-journey-config", "/private/binding.json", "--max-automatic-retries", "0"]), runnerMatrix);
expect(selected).toHaveLength(1); expect(selected[0]!.suite.manualOnly).toBe(true); expect(selected[0]!.requiredCredentials).toEqual([]);
expect(selected[0]!.task.automaticRetryPolicy).toBe("single_attempt");
expect(selectRunnerExecutions(parseRunnerSelectors(["--all"]), runnerMatrix).some(row => row.id === ANTIGRAVITY_JOURNEY_EXECUTION)).toBe(false);
expect(() => selected[0]!.profile.buildAgent({} as never)).toThrow("must be created through the ordinary UI");
});
it.each(["https://paperclip.example", "http://localhost:3104", "http://192.168.0.1:3104"])("does not claim same-host native retirement on %s", baseURL => {
const value = fixture(); value.target.baseURL = baseURL;
expect(() => parseAntigravityJourneyConfig(value)).toThrow("invalid_antigravity_journey_configuration");
});
it("rejects widened turns, Linux inventory, live credential values and a signed-out setup", () => {
for (const edit of [
(v: any) => { v.limits.maxRuns = 3; }, (v: any) => { v.candidate.platform = "linux"; },
(v: any) => { v.savedConnection.accessToken = "must-never-be-a-config-value"; }, (v: any) => { v.browser.freshness = "signed-out"; },
(v: any) => { v.limits.turnTimeoutMs = "300000"; },
]) { const value = fixture(); edit(value); expect(() => parseAntigravityJourneyConfig(value)).toThrow("invalid_antigravity_journey_configuration"); }
});
it("binds owned-resource recovery to the exact company, environment and saved account", () => {
const config = parseAntigravityJourneyConfig(fixture());
expect(parseAntigravityOwnedResources(resources(), config).agentId).toBe(agentId);
for (const patch of [{ companyId: environmentId }, { environmentId: companyId }, { connectionId: agentId }, { origin: "https://other.example" }, { agentName: "Existing user agent" }])
expect(() => parseAntigravityOwnedResources({ ...resources(), ...patch }, config)).toThrow("invalid_antigravity_owned_resources");
});
it("accepts a company-allowed instance environment without inventing company ownership", () => {
const environment = { id: environmentId, name: "Local", driver: "local", status: "active" };
expect(selectAntigravityJourneyEnvironment([environment], environmentId)).toBe(environment);
expect(() => selectAntigravityJourneyEnvironment([{ ...environment, id: agentId }], environmentId)).toThrow("configured_execution_environment_unavailable");
expect(() => selectAntigravityJourneyEnvironment([{ ...environment, status: "disabled" }], environmentId)).toThrow("configured_execution_environment_unavailable");
});
it.each(["failed", "interrupted", "timed_out"])("retains an owned %s run when physical ownership was never captured", async status => {
const owned = resources(); owned.runIds = [runId];
const agent = { id: agentId, companyId, name: owned.agentName }, run = { id: runId, agentId, companyId, status };
const get = vi.fn(async (route: string) => route.includes("/agents") ? [agent] : route.includes("heartbeat-runs?") ? [run] : run);
const remove = vi.fn(), patch = vi.fn();
await expect(cleanupAntigravityJourney({ api: { get, patch, delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: owned, cleanup: { agent: false, runs: false, connectionPreserved: false }, persistResources: async () => {}, captureRun: async () => {} }))
.rejects.toThrow("cleanup_physical_owner_not_captured");
expect(remove).not.toHaveBeenCalled(); expect(patch).toHaveBeenCalledWith(`/api/agents/${agentId}`, { status: "paused" });
});
it("terminates the owned ready agent before purge and preserves the shared account", async () => {
const owned = resources(); owned.setupProbe = "retired";
const cleanup = { agent: false, runs: false, connectionPreserved: false };
let lifecycleState = "ready";
const agent = () => ({ id: agentId, companyId, name: owned.agentName, lifecycleState });
const get = vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [agent()];
if (route.includes("heartbeat-runs?")) return [];
if (route === `/api/agents/${agentId}`) return agent();
if (route.endsWith("/ai-connections")) return { connections: [{ id: connectionId, provider: "google", method: "subscription", ownerUserId: "synthetic-user", isDefault: true, status: "connected" }] };
throw new Error("Unexpected cleanup read");
});
const post = vi.fn(async (route: string) => { expect(route).toBe(`/api/agents/${agentId}/terminate`); lifecycleState = "terminated"; return agent(); });
const remove = vi.fn(async () => { expect(lifecycleState).toBe("terminated"); });
await cleanupAntigravityJourney({ api: { get, post, patch: async () => { lifecycleState = "paused"; }, delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: owned, cleanup, persistResources: async () => {}, captureRun: async () => {} });
expect(remove).toHaveBeenCalledWith(`/api/agents/${agentId}`);
expect(cleanup).toEqual({ agent: true, runs: true, connectionPreserved: true });
});
it.each([false, true])("captures a cleanup-discovered run's settled usage before purge (storage failure: %s)", async storageFailure => {
const owned = resources(); owned.setupProbe = "retired";
owned.processes[runId] = { captured: true, identityChanged: false, journal: [
{ pid: 9000001, parent: 1, start: "2020-01-01T00:00:00Z" }, { pid: 9000002, parent: 9000001, start: "2020-01-01T00:00:00Z" },
] };
let status = "running", lifecycleState = "ready";
const agent = () => ({ id: agentId, companyId, name: owned.agentName, lifecycleState });
const run = () => ({ id: runId, agentId, companyId, status, costAccountingPending: status === "running",
usageJson: { inputTokens: 100, outputTokens: 20, costUsd: status === "running" ? 0.01 : 0.25 },
contextSnapshot: { paperclipEnvironment: { leaseId: "lease" } } });
const snapshots: any[] = [];
const captureRun = vi.fn(async row => {
if (storageFailure && row.status === "cancelled") throw new Error("fixture accounting storage unavailable");
snapshots.push(structuredClone(row));
});
const get = vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [agent()];
if (route.includes("heartbeat-runs?")) return [run()];
if (route === `/api/heartbeat-runs/${runId}`) return run();
if (route === "/api/environment-leases/lease") return { companyId, heartbeatRunId: runId, environmentId, status: "released" };
if (route === `/api/agents/${agentId}`) return agent();
if (route.endsWith("/ai-connections")) return { connections: [{ id: connectionId, provider: "google", method: "subscription", ownerUserId: "synthetic-user", isDefault: true, status: "connected" }] };
throw new Error("Unexpected cleanup read");
});
const post = vi.fn(async (route: string) => {
if (route.endsWith("/cancel")) status = "cancelled";
else if (route.endsWith("/terminate")) lifecycleState = "terminated";
else throw new Error("Unexpected cleanup mutation");
});
const remove = vi.fn(async () => { expect(snapshots.at(-1)).toMatchObject({ status: "cancelled", costAccountingPending: false, usageJson: { costUsd: 0.25 } }); });
const cleanup = cleanupAntigravityJourney({ api: { get, post, patch: async () => {}, delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: owned, cleanup: { agent: false, runs: false, connectionPreserved: false }, persistResources: async () => {}, captureRun });
if (storageFailure) { await expect(cleanup).rejects.toThrow("fixture accounting storage unavailable"); expect(remove).not.toHaveBeenCalled(); }
else { await cleanup; expect(remove).toHaveBeenCalledOnce(); }
expect(owned.runIds).toEqual([runId]);
expect(snapshots[0]).toMatchObject({ status: "running", usageJson: { costUsd: 0.01 } });
});
it("does not erase a failed native setup Test's unconfirmed cleanup with an empty task-run list", async () => {
const owned = resources(); owned.setupProbe = "retirement_unconfirmed";
const agent = { id: agentId, companyId, name: owned.agentName }, remove = vi.fn();
const get = vi.fn(async (route: string) => route.endsWith("/agents") ? [agent] : []);
await expect(cleanupAntigravityJourney({ api: { get, patch: vi.fn(), delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: owned, cleanup: { agent: false, runs: true, connectionPreserved: true }, persistResources: async () => {}, captureRun: async () => {} }))
.rejects.toThrow("setup_probe_retirement_unconfirmed");
expect(remove).not.toHaveBeenCalled();
});
it.each(["interrupted", "timed_out"].flatMap(status => [false, true].map(pending => ({ status, pending }))))("retires a $status run only after its accounting settles (pending: $pending)", async ({ status, pending }) => {
vi.useFakeTimers();
try {
const owned = resources(); owned.setupProbe = "retired"; owned.runIds = [runId];
owned.processes[runId] = { captured: true, identityChanged: false, journal: [
{ pid: 9000001, parent: 1, start: "2020-01-01T00:00:00Z" }, { pid: 9000002, parent: 9000001, start: "2020-01-01T00:00:00Z" },
] };
const agent = { id: agentId, companyId, name: owned.agentName, status: "active" }, run = { id: runId, agentId, companyId, status,
costAccountingPending: pending, contextSnapshot: { paperclipEnvironment: { leaseId: "lease" } } };
const get = vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [agent];
if (route === `/api/agents/${agentId}`) return agent;
if (route.includes("heartbeat-runs?")) return [run];
if (route === `/api/heartbeat-runs/${runId}`) return run;
if (route === "/api/environment-leases/lease") return { companyId, heartbeatRunId: runId, environmentId, status: "released" };
if (route.endsWith("/ai-connections")) return { connections: [{ id: connectionId, provider: "google", method: "subscription", ownerUserId: "synthetic-user", isDefault: true, status: "connected" }] };
throw new Error("Unexpected cleanup read");
});
const post = vi.fn(), remove = vi.fn(), cleanup = { agent: false, runs: false, connectionPreserved: false };
const started = Date.now(); let error: unknown, finishedAt: number | undefined;
const operation = cleanupAntigravityJourney({ api: { get, post, patch: vi.fn(async () => { agent.status = "paused"; }), delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig({ ...fixture(), retainAgent: true }), resources: owned, cleanup, captureRun: async () => {}, persistResources: async () => {} })
.then(() => { finishedAt = Date.now(); }, caught => { error = caught; finishedAt = Date.now(); });
await vi.advanceTimersByTimeAsync(60_500); await operation;
if (pending) { expect(String(error)).toContain("cleanup_run_retirement_unconfirmed"); expect(cleanup.runs).toBe(false); }
else { expect(error).toBeUndefined(); expect(finishedAt! - started).toBeLessThan(1_000); expect(cleanup).toEqual({ agent: true, runs: true, connectionPreserved: true }); expect(owned.retained).toBe(true); }
expect(post).not.toHaveBeenCalled(); expect(remove).not.toHaveBeenCalled();
} finally { vi.useRealTimers(); }
});
it("does not claim retained-agent cleanup when the pause PATCH was ignored", async () => {
const owned = resources(); owned.setupProbe = "retired";
const agent = { id: agentId, companyId, name: owned.agentName, status: "active" };
const get = vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [agent];
if (route === `/api/agents/${agentId}`) return agent;
if (route.includes("heartbeat-runs?")) return [];
if (route.endsWith("/ai-connections")) return { connections: [{ id: connectionId, provider: "google", method: "subscription", ownerUserId: "synthetic-user", isDefault: true, status: "connected" }] };
throw new Error("Unexpected cleanup read");
});
const post = vi.fn(), remove = vi.fn(), patch = vi.fn(), cleanup = { agent: false, runs: false, connectionPreserved: false };
await expect(cleanupAntigravityJourney({ api: { get, post, patch, delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig({ ...fixture(), retainAgent: true }), resources: owned, cleanup, captureRun: async () => {}, persistResources: async () => {} }))
.rejects.toThrow("cleanup_retained_agent_pause_unconfirmed");
expect(patch).toHaveBeenCalledWith(`/api/agents/${agentId}`, { status: "paused" });
expect(cleanup.agent).toBe(false); expect(post).not.toHaveBeenCalled(); expect(remove).not.toHaveBeenCalled();
});
it("stops on combined reported and estimated known cost while retaining unknown subscription prices", () => {
const usage = (id: string, usageJson: Record<string, unknown>) => ({ id, usageJson });
expect(antigravityKnownCostObservation([
usage("first", { inputTokens: 100, outputTokens: 20, costUsd: 0.6 }),
usage("second", { inputTokens: 100, outputTokens: 20, costUsd: 0.41, costStatus: "estimated" }),
], 100)).toMatchObject({ knownCostUsd: 1.01, exceeded: true, hardCurrencyCap: false });
expect(antigravityKnownCostObservation([
usage("first", { inputTokens: 100, outputTokens: 20, costUsd: 0.1 }),
usage("second", { inputTokens: 100, outputTokens: 20, costUsd: 0.2, costStatus: "estimated" }),
], 30).exceeded).toBe(false);
expect(antigravityKnownCostObservation([
usage("first", { inputTokens: 100, outputTokens: 20, costUsd: 0, costStatus: "unpriced", billingType: "subscription" }),
], 1)).toMatchObject({ exceeded: false, costStatus: "unpriced", priceCoverageComplete: false, hardCurrencyCap: false });
});
function deferredOwnerCleanup(discovery: "initial" | "poll" | "transition" = "initial") {
const owned = resources(); owned.setupProbe = "retired"; owned.issueId = "66666666-6666-4666-8666-666666666666";
owned.processes[runId] = { captured: true, identityChanged: false, journal: [
{ pid: 9000001, parent: 1, start: "2020-01-01T00:00:00Z" }, { pid: 9000002, parent: 9000001, start: "2020-01-01T00:00:00Z" },
] };
const agent = { id: agentId, companyId, name: owned.agentName, status: "active" };
const run: Record<string, any> = { id: runId, companyId, agentId, issueId: owned.issueId, nativeIssueId: owned.issueId,
status: "failed", runtimeMode: "native", nativePhase: "retryable_failure", costAccountingPending: false,
contextSnapshot: { paperclipEnvironment: { leaseId: "owned-lease" } }, resultJson: {} };
const lease = { id: "owned-lease", companyId, heartbeatRunId: runId, environmentId, status: "released" };
const state = { listReads: 0, fullReads: 0, stopAtListReads: [] as number[], fullPatch: {} as Record<string, unknown>,
onStop: async (_body: Record<string, any>) => {} };
const acknowledge = (body: Record<string, any>, patch: Record<string, unknown> = {}) => {
run.status = "cancelled"; run.nativePhase = "terminal_failure";
run.resultJson = { startupCancellation: { cancellationRequestId: body.cancellationRequestId }, nativeCancellation: {
schema: "paperclip.native-cancellation.v1", intentId: `native-cancellation:${body.cancellationRequestId}`,
companyId, runId, issueId: owned.issueId, scope: "run", reasonCode: "cancellation_run_only", dispatched: false,
dispatchState: "acknowledged", intentAuditId: "owned-intent-audit", acknowledgementAuditId: "owned-ack-audit", ...patch,
} };
};
state.onStop = async body => { if (run.runtimeMode === "native") acknowledge(body); else { run.status = "cancelled"; run.resultJson = {}; } };
const get = vi.fn(async (route: string) => {
if (route.endsWith("/agents")) return [structuredClone(agent)];
if (route === `/api/agents/${agentId}`) return structuredClone(agent);
if (route.includes("heartbeat-runs?")) {
state.listReads++;
if (discovery === "poll" && state.listReads === 1) return [];
return [{ ...structuredClone(run), ...(discovery === "transition" ? { nativePhase: "committed" } : {}) }];
}
if (route === `/api/heartbeat-runs/${runId}`) {
state.fullReads++;
return { ...structuredClone(run), ...(discovery === "transition" && state.fullReads === 1 ? { nativePhase: "committed" } : {}), ...state.fullPatch };
}
if (route === "/api/environment-leases/owned-lease") return structuredClone(lease);
if (route === `/api/issues/${owned.issueId}`) return { id: owned.issueId, companyId, assigneeAgentId: agentId, status: "done" };
if (route.endsWith("/ai-connections")) return { connections: [{ id: connectionId, provider: "google", method: "subscription", ownerUserId: "synthetic-user", isDefault: true, status: "connected" }] };
throw new Error(`Unexpected deferred cleanup read ${route}`);
});
const post = vi.fn(async (route: string, body: Record<string, any> = {}) => {
expect(route).toBe(`/api/heartbeat-runs/${runId}/cancel`);
expect(owned.runIds).toContain(runId);
if (run.runtimeMode === "native") expect(body.cancellationRequestId).toMatch(/^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/iu);
else expect(body.cancellationRequestId).toBeUndefined();
state.stopAtListReads.push(state.listReads); await state.onStop(body); return structuredClone(run);
});
const remove = vi.fn(), cleanup = { agent: false, runs: false, connectionPreserved: false };
const start = () => cleanupAntigravityJourney({ api: { get, post, patch: vi.fn(async () => { agent.status = "paused"; }), delete: remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig({ ...fixture(), retainAgent: true }), resources: owned, cleanup, captureRun: async () => {}, persistResources: async () => {} });
return { owned, run, lease, state, post, remove, cleanup, acknowledge, start };
}
it.each(["initial", "poll"] as const)("owned deferred cleanup stops a failed native retry at %s discovery and waits for acknowledgement and lease retirement", async discovery => {
vi.useFakeTimers();
const f = deferredOwnerCleanup(discovery); f.lease.status = "active";
let release!: () => void; const stop = new Promise<void>(resolve => { release = resolve; });
f.state.onStop = async body => { await stop; f.acknowledge(body, { dispatched: discovery === "initial" }); };
const outcome: { done: boolean; error?: unknown } = { done: false };
const operation = f.start().then(() => { outcome.done = true; }, error => { outcome.error = error; });
try {
await vi.advanceTimersByTimeAsync(0);
expect(f.post).toHaveBeenCalledOnce(); expect(f.state.stopAtListReads).toEqual([discovery === "initial" ? 1 : 2]);
expect(outcome.done).toBe(false); expect(f.cleanup.runs).toBe(false); expect(f.cleanup.agent).toBe(false);
release(); await vi.advanceTimersByTimeAsync(0);
expect(outcome.done).toBe(false); expect(f.cleanup.runs).toBe(false);
f.lease.status = "released"; await vi.advanceTimersByTimeAsync(300); await operation;
expect(outcome.error).toBeUndefined(); expect(outcome.done).toBe(true);
expect(f.cleanup).toEqual({ agent: true, runs: true, connectionPreserved: true }); expect(f.owned.retained).toBe(true); expect(f.remove).not.toHaveBeenCalled();
} finally { release(); f.lease.status = "released"; await vi.advanceTimersByTimeAsync(61_000); await operation; vi.useRealTimers(); }
});
it.each(["initial", "poll"] as const)("owned deferred cleanup requires Stop despite already retired processes, released lease and settled accounting (%s)", async discovery => {
const f = deferredOwnerCleanup(discovery); await f.start();
expect(f.post).toHaveBeenCalledOnce(); expect(f.state.stopAtListReads).toEqual([discovery === "initial" ? 1 : 2]);
expect(f.cleanup.runs).toBe(true);
});
it("owned deferred cleanup observes a known run's transition into retry ownership", async () => {
const f = deferredOwnerCleanup("transition"); await f.start();
expect(f.state.fullReads).toBeGreaterThanOrEqual(2); expect(f.post).toHaveBeenCalledOnce(); expect(f.cleanup.runs).toBe(true);
});
it.each(["initial", "poll"] as const)("owned deferred cleanup stops a scheduled retry at %s discovery", async discovery => {
vi.useFakeTimers();
try {
const f = deferredOwnerCleanup(discovery); Object.assign(f.run, { status: "scheduled_retry", runtimeMode: "legacy", nativePhase: null });
let error: unknown; const operation = f.start().catch(caught => { error = caught; });
await vi.advanceTimersByTimeAsync(61_000); await operation;
expect(error).toBeUndefined(); expect(f.post).toHaveBeenCalledOnce(); expect(f.state.stopAtListReads).toEqual([discovery === "initial" ? 1 : 2]); expect(f.cleanup.runs).toBe(true);
} finally { vi.useRealTimers(); }
});
it.each([
["failed", "native", "terminal_failure"], ["failed", "legacy", "retryable_failure"],
["succeeded", "native", "retryable_failure"], ["cancelled", "native", "retryable_failure"], ["timed_out", "native", "retryable_failure"],
])("owned deferred cleanup does not Stop ordinary terminal %s/%s/%s", async (status, runtimeMode, nativePhase) => {
const f = deferredOwnerCleanup(); Object.assign(f.run, { status, runtimeMode, nativePhase });
await f.start(); expect(f.post).not.toHaveBeenCalled(); expect(f.cleanup.runs).toBe(true);
});
it.each(["id", "companyId", "agentId", "issueId"])("owned deferred cleanup rejects foreign hydrated %s before Stop", async field => {
const f = deferredOwnerCleanup(); f.state.fullPatch[field] = "ffffffff-ffff-4fff-8fff-ffffffffffff";
await expect(f.start()).rejects.toThrow(/owner_mismatch/u); expect(f.post).not.toHaveBeenCalled(); expect(f.cleanup.agent).toBe(false);
});
it.each(["pending", "foreign_run", "foreign_company", "foreign_issue", "missing_audit", "same_audit", "stale_intent", "accounting_pending", "stop_error"])("owned deferred cleanup keeps failure when Stop proof is %s", async scenario => {
vi.useFakeTimers();
try {
const f = deferredOwnerCleanup();
f.state.onStop = async body => {
if (scenario === "stop_error") throw new Error("synthetic Stop rejected");
f.acknowledge(body, scenario === "pending" ? { dispatchState: "pending" }
: scenario === "foreign_run" ? { runId: agentId } : scenario === "foreign_company" ? { companyId: agentId }
: scenario === "foreign_issue" ? { issueId: agentId } : scenario === "missing_audit" ? { acknowledgementAuditId: "" }
: scenario === "same_audit" ? { acknowledgementAuditId: "owned-intent-audit" }
: scenario === "stale_intent" ? { intentId: "native-cancellation:ffffffff-ffff-4fff-8fff-ffffffffffff" } : {});
if (scenario === "accounting_pending") f.run.costAccountingPending = true;
};
let error: unknown; const operation = f.start().catch(caught => { error = caught; });
await vi.advanceTimersByTimeAsync(61_000); await operation;
expect(f.post).toHaveBeenCalledOnce(); expect(error).toBeDefined(); expect(f.cleanup.agent).toBe(false); expect(f.cleanup.runs).toBe(false); expect(f.remove).not.toHaveBeenCalled();
} finally { vi.useRealTimers(); }
});
it("owned deferred cleanup recovers a pending native Stop with its original request identity", async () => {
const f = deferredOwnerCleanup(), requestId = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa";
f.acknowledge({ cancellationRequestId: requestId }, { dispatchState: "pending", acknowledgementAuditId: null });
f.run.status = "failed";
await f.start(); expect(f.post).toHaveBeenCalledOnce(); expect(f.post.mock.calls[0]![1]).toEqual({ cancellationRequestId: requestId });
expect(f.cleanup.runs).toBe(true);
});
it("owned deferred cleanup preserves cancelled unknown-accounting debt without a new Stop", async () => {
vi.useFakeTimers();
try {
const f = deferredOwnerCleanup(); Object.assign(f.run, { status: "cancelled", costAccountingPending: true });
let error: unknown; const operation = f.start().catch(caught => { error = caught; });
await vi.advanceTimersByTimeAsync(61_000); await operation;
expect(error).toBeDefined(); expect(f.post).not.toHaveBeenCalled(); expect(f.cleanup.runs).toBe(false); expect(f.cleanup.agent).toBe(false);
} finally { vi.useRealTimers(); }
});
it("owned deferred cleanup retains a terminal native reservation without inventing Stop authority", async () => {
vi.useFakeTimers();
try {
const f = deferredOwnerCleanup(); f.run.nativePhase = "terminal_failure";
f.run.resultJson = { startupCancellation: { cancellationRequestId: "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa" } };
let error: unknown; const operation = f.start().catch(caught => { error = caught; });
await vi.advanceTimersByTimeAsync(61_000); await operation;
expect(error).toBeDefined(); expect(f.post).not.toHaveBeenCalled(); expect(f.cleanup.runs).toBe(false); expect(f.cleanup.agent).toBe(false); expect(f.remove).not.toHaveBeenCalled();
} finally { vi.useRealTimers(); }
});
it("owned deferred cleanup reuses a reservation-only UUID while its native retry is eligible", async () => {
const f = deferredOwnerCleanup(), requestId = "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa";
f.run.resultJson = { startupCancellation: { cancellationRequestId: requestId } };
await f.start(); expect(f.post).toHaveBeenCalledOnce(); expect(f.post.mock.calls[0]![1]).toEqual({ cancellationRequestId: requestId });
expect(f.cleanup.runs).toBe(true);
});
function interruptedAgentCreation() {
const owned = resources(); delete owned.agentId;
const witness = { setupId: agentId, companyId, agentName: owned.agentName, provider: "antigravity", creatorUserId: "synthetic-user", origin: owned.origin };
(owned as any).agentCreationRequest = witness;
let lifecycleState = "draft", persisted = false;
const record = () => ({ id: agentId, companyId, name: owned.agentName, createdByUserId: "synthetic-user", lifecycleState });
const get = vi.fn(async (route: string): Promise<Record<string, any> | Record<string, any>[]> => {
if (route === `/api/companies/${companyId}/agents`) return [record()];
if (route === `/api/agents/${agentId}?companyId=${companyId}`) return record();
if (route.includes("heartbeat-runs?")) return [];
if (route === `/api/agents/${agentId}`) return record();
if (route.endsWith("/ai-connections")) return { connections: [{ id: connectionId, provider: "google", method: "subscription", ownerUserId: "synthetic-user", isDefault: true, status: "connected" }] };
throw new Error(`Unexpected creation recovery read ${route}`);
});
const patch = vi.fn(async (route: string) => {
expect(route).toBe(`/api/agents/${agentId}`); expect(persisted).toBe(true); lifecycleState = "paused";
});
const post = vi.fn(async (route: string) => { expect(route).toBe(`/api/agents/${agentId}/terminate`); lifecycleState = "terminated"; });
const remove = vi.fn();
const persistResources = vi.fn(async () => { expect(owned.agentId).toBe(agentId); persisted = true; });
return { owned, witness, record, get, patch, post, remove, persistResources };
}
it("parses the minimal pre-forward creation witness without storing the UI request body", () => {
const f = interruptedAgentCreation();
expect(parseAntigravityOwnedResources(f.owned, parseAntigravityJourneyConfig(fixture()))).toMatchObject({ agentCreationRequest: f.witness });
});
it("recovers a committed UI draft by exact setupId before any pause or purge", async () => {
const f = interruptedAgentCreation(), cleanup = { agent: false, runs: false, connectionPreserved: false };
await cleanupAntigravityJourney({ api: { get: f.get, patch: f.patch, post: f.post, delete: f.remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: f.owned, cleanup, persistResources: f.persistResources, captureRun: async () => {} });
expect(f.owned.agentId).toBe(agentId);
expect(f.get).toHaveBeenCalledWith(`/api/agents/${agentId}?companyId=${companyId}`);
expect(f.persistResources).toHaveBeenCalled(); expect(f.remove).toHaveBeenCalledWith(`/api/agents/${agentId}`);
expect(cleanup).toEqual({ agent: true, runs: true, connectionPreserved: true });
});
it.each([
["foreign company", { companyId: environmentId }], ["different name", { agentName: "Antigravity QA fedcba987654" }],
["wrong provider", { provider: "codex" }], ["noncanonical setup UUID", { setupId: "NOT-A-UUID" }],
["uppercase setup UUID", { setupId: "AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA" }],
["foreign origin", { origin: "http://127.0.0.1:9999" }], ["foreign creator", { creatorUserId: "another-user" }], ["missing creator", { creatorUserId: undefined }],
["extra private payload", { rawBody: { runtimeConfig: { env: { PRIVATE_VALUE: "never-store" } } } }],
["mismatched known agent", { setupId: environmentId }],
])("rejects a %s creation witness before recovery", (_label, patch) => {
const f = interruptedAgentCreation();
(f.owned as any).agentCreationRequest = { ...f.witness, ...patch };
if (_label === "mismatched known agent") f.owned.agentId = agentId;
expect(() => parseAntigravityOwnedResources(f.owned, parseAntigravityJourneyConfig(fixture()))).toThrow("invalid_antigravity_owned_resources");
});
it.each(["company", "name", "id", "creator", "missing-creator", "same-name-only", "duplicate-name"])("does not reclaim a creation draft with mismatched %s", async field => {
const f = interruptedAgentCreation(), originalGet = f.get.getMockImplementation()!;
f.get.mockImplementation(async route => {
if (route === `/api/companies/${companyId}/agents` && field === "same-name-only") return [{ ...f.record(), id: environmentId }];
if (route === `/api/companies/${companyId}/agents` && field === "duplicate-name") return [f.record(), { ...f.record(), id: environmentId }];
if (route === `/api/agents/${agentId}?companyId=${companyId}`) return { ...f.record(),
...(field === "company" ? { companyId: environmentId } : field === "name" ? { name: "Existing user agent" } : field === "id" ? { id: environmentId } : field === "creator" ? { createdByUserId: "another-user" } : field === "missing-creator" ? { createdByUserId: undefined } : {}) };
return originalGet(route);
});
await expect(cleanupAntigravityJourney({ api: { get: f.get, patch: f.patch, post: f.post, delete: f.remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: f.owned, cleanup: { agent: false, runs: false, connectionPreserved: false },
persistResources: f.persistResources, captureRun: async () => {} })).rejects.toThrow(/cleanup_.*(?:mismatch|unconfirmed)/u);
expect(f.patch).not.toHaveBeenCalled(); expect(f.post).not.toHaveBeenCalled(); expect(f.remove).not.toHaveBeenCalled();
});
it("keeps a witnessed404 unconfirmed rather than treating an in-flight creation as absent", async () => {
const f = interruptedAgentCreation(), cleanup = { agent: false, runs: false, connectionPreserved: false };
f.get.mockImplementation(async route => {
if (route === `/api/companies/${companyId}/agents`) return [];
if (route === `/api/agents/${agentId}?companyId=${companyId}`) throw new ConnectionFailure("api_status_404");
throw new Error(`Unexpected witnessed404 read ${route}`);
});
await expect(cleanupAntigravityJourney({ api: { get: f.get, patch: f.patch, post: f.post, delete: f.remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: f.owned, cleanup, persistResources: f.persistResources, captureRun: async () => {} }))
.rejects.toThrow(/cleanup_agent_creation_unconfirmed/u);
expect(f.patch).not.toHaveBeenCalled(); expect(f.post).not.toHaveBeenCalled(); expect(f.remove).not.toHaveBeenCalled();
expect(cleanup.agent).toBe(false); expect(cleanup.runs).toBe(false);
});
it("does not mutate a recovered draft if its exact identity cannot be persisted", async () => {
const f = interruptedAgentCreation(); f.persistResources.mockRejectedValue(new Error("owned creation receipt write failed"));
await expect(cleanupAntigravityJourney({ api: { get: f.get, patch: f.patch, post: f.post, delete: f.remove } as unknown as ConnectionApi,
config: parseAntigravityJourneyConfig(fixture()), resources: f.owned, cleanup: { agent: false, runs: false, connectionPreserved: false },
persistResources: f.persistResources, captureRun: async () => {} })).rejects.toThrow("owned creation receipt write failed");
expect(f.patch).not.toHaveBeenCalled(); expect(f.post).not.toHaveBeenCalled(); expect(f.remove).not.toHaveBeenCalled();
});
tests/runner-e2e/antigravity-journey-config.ts
+75
View File
@@ -0,0 +1,75 @@
import { readFile } from "node:fs/promises";
import path from "node:path";
import { ConnectionBlock, assertPrivateFile, parseConnectionConfig, type ConnectionConfig } from "./connection-config.js";
export const ANTIGRAVITY_JOURNEY_MODEL = "gemini-3.8-flash-low";
export interface AntigravityJourneyConfig {
version: 1;
setupId: string;
hosting: "self-hosted";
executionEnvironment: "local";
target: Extract<ConnectionConfig["target"], { mode: "attach" }> & { companyId: string; environmentId: string };
browser: ConnectionConfig["browser"];
savedConnection: { id: string; name: string; ownerUserId: string };
candidate: { platform: "darwin"; architecture: "arm64"; nativeVersion: string; profileDigest: string; nativeClosureDigest: string };
limits: { durationMs: number; turnTimeoutMs: number; maxRuns: 2; budgetCents: number };
retainAgent: boolean;
}
const invalid = (): never => { throw new ConnectionBlock("blocked_target", "invalid_antigravity_journey_configuration"); };
function object(value: unknown, keys: readonly string[]): Record<string, any> {
if (!value || typeof value !== "object" || Array.isArray(value) || Object.keys(value).some(key => !keys.includes(key))) return invalid();
return value as Record<string, any>;
}
function text(value: unknown, pattern?: RegExp): string {
if (typeof value !== "string" || !value.trim() || value !== value.trim() || value.length > 1024 || (pattern && !pattern.test(value))) return invalid();
return value;
}
function number(value: unknown, min: number, max: number): number {
if (typeof value !== "number" || !Number.isSafeInteger(value) || value < min || value > max) return invalid();
return value;
}
/** Binding references only. Candidate pins assert provenance; they do not grant admission. */
export function parseAntigravityJourneyConfig(input: unknown): AntigravityJourneyConfig {
try {
const v = object(input, ["version", "setupId", "hosting", "executionEnvironment", "target", "browser", "savedConnection", "candidate", "limits", "retainAgent"]);
if (v.version !== 1 || v.hosting !== "self-hosted" || v.executionEnvironment !== "local") return invalid();
const common = parseConnectionConfig({ target: v.target, browser: v.browser });
if (common.target.mode !== "attach" || common.browser.freshness !== "signed-in") return invalid();
if (new URL(common.target.baseURL).hostname !== "127.0.0.1") return invalid();
const uuid = /^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/i;
const connection = object(v.savedConnection, ["id", "name", "ownerUserId"]);
const candidate = object(v.candidate, ["platform", "architecture", "nativeVersion", "profileDigest", "nativeClosureDigest"]);
if (candidate.platform !== "darwin" || candidate.architecture !== "arm64") return invalid();
const limits = object(v.limits, ["durationMs", "turnTimeoutMs", "maxRuns", "budgetCents"]);
if (limits.maxRuns !== 2 || (v.retainAgent !== undefined && typeof v.retainAgent !== "boolean")) return invalid();
const durationMs = number(limits.durationMs, 60_000, 1_800_000);
const turnTimeoutMs = number(limits.turnTimeoutMs, 10_000, 600_000);
if (turnTimeoutMs * 2 > durationMs) return invalid();
return { version: 1, setupId: text(v.setupId, /^[a-z0-9-]{1,80}$/), hosting: "self-hosted", executionEnvironment: "local",
target: { ...common.target, companyId: text(common.target.companyId, uuid), environmentId: text(common.target.environmentId, uuid) }, browser: common.browser,
savedConnection: { id: text(connection.id, uuid), name: text(connection.name), ownerUserId: text(connection.ownerUserId) },
candidate: { platform: "darwin", architecture: "arm64", nativeVersion: text(candidate.nativeVersion, /^\d+\.\d+\.\d+-paperclip\.\d+$/),
profileDigest: text(candidate.profileDigest, /^sha256:[a-f0-9]{64}$/), nativeClosureDigest: text(candidate.nativeClosureDigest, /^sha256:[a-f0-9]{64}$/) },
limits: { durationMs, turnTimeoutMs, maxRuns: 2, budgetCents: number(limits.budgetCents, 1, 1000) }, retainAgent: v.retainAgent ?? false };
} catch { return invalid(); }
}
export async function loadAntigravityJourneyConfig(file?: string) {
if (!file) throw new ConnectionBlock("blocked_target", "antigravity_journey_config_required");
try {
await assertPrivateFile(file);
const input = JSON.parse(await readFile(file, "utf8"));
if (input.browser?.profileDir && !path.isAbsolute(input.browser.profileDir)) input.browser.profileDir = path.resolve(path.dirname(file), input.browser.profileDir);
return parseAntigravityJourneyConfig(input);
} catch (error) {
if (error instanceof ConnectionBlock) throw error;
return invalid();
}
}
export function antigravityConnectionConfig(config: AntigravityJourneyConfig): ConnectionConfig {
return { target: config.target, browser: config.browser, models: {}, routes: {}, credentials: {}, budgetCents: config.limits.budgetCents,
maxRuns: config.limits.maxRuns, turnTimeoutMs: config.limits.turnTimeoutMs, retainCompany: true };
}
tests/runner-e2e/antigravity-journey-evidence.test.ts
+191
View File
@@ -0,0 +1,191 @@
import { readFileSync } from "node:fs";
import { execFileSync, spawn } from "node:child_process";
import { once } from "node:events";
import { expect, it, vi } from "vitest";
import { antigravityEvidenceFixture, antigravityTurnFixture } from "./antigravity-journey-test-fixtures.js";
import { antigravityPrimaryOwner, gradeAntigravityJourney, gradeAntigravityJourneyTurn, sameAntigravityConversation, selectAntigravityIssueRun, validateAntigravityJourneyEvidence, verifyAntigravityJourneyTurn } from "./antigravity-journey-evidence.js";
import type { ConnectionApi } from "./connection-target.js";
import { observeRunProcesses } from "./copilot-local-fixtures.js";
import { readLinuxProcessStartedAt } from "../../packages/paperclip-runner/src/live/linux-process-start.js";
it("consumes the activity router's public runId summary rather than a full heartbeat row", () => {
const service = readFileSync(new URL("../../server/src/services/activity.ts", import.meta.url), "utf8");
const router = readFileSync(new URL("../../server/src/routes/activity.ts", import.meta.url), "utf8");
const client = readFileSync(new URL("../../ui/src/api/activity.ts", import.meta.url), "utf8");
expect(service).toMatch(/runsForIssue:[\s\S]*?\.select\(\{\s*runId: heartbeatRuns\.id/u);
expect(router).toMatch(/router\.get\("\/issues\/:id\/runs"[\s\S]*?svc\.runsForIssue\(issue\.companyId, issue\.id/u);
expect(client).toMatch(/interface RunForIssue\s*\{\s*runId: string/u);
expect(selectAntigravityIssueRun([{ runId: "old", agentId: "agent" }, { runId: "new", agentId: "agent" }], "agent", ["old"])).toBe("new");
expect(selectAntigravityIssueRun([], "agent", [])).toBeNull();
});
it.each([
[{ id: "wrong-public-shape", agentId: "agent" }],
[{ runId: "run", id: "conflicting", agentId: "agent" }],
[{ runId: "run", agentId: "other" }],
[{ runId: "run", agentId: "agent" }, { runId: "run", agentId: "agent" }],
[{ runId: "run", agentId: "agent" }, { runId: "extra", agentId: "agent" }],
].map(summaries => ({ summaries })))("rejects an invalid or ambiguous public run summary %#", ({ summaries }) => {
expect(() => selectAntigravityIssueRun(summaries, "agent", [])).toThrow("antigravity_evidence_");
});
it.skipIf(process.platform !== "darwin" || process.arch !== "arm64")("fetches the public runId and verifies the full owner before any host observation", async () => {
const fixture = antigravityTurnFixture(), routes: string[] = []; let observed = false;
const api = { origin: "http://127.0.0.1:3104", get: async (route: string) => {
routes.push(route);
if (route.endsWith("/runs")) return [{ runId: fixture.run.id, agentId: fixture.agentId }];
return { ...fixture.run, companyId: "foreign" };
} } as unknown as ConnectionApi;
await expect(verifyAntigravityJourneyTurn({ ...fixture, api, deadlineAt: Date.now() + 1000,
observeRun: async () => { observed = true; } })).rejects.toThrow("observed_run_binding");
expect(routes).toEqual(["/api/issues/task/runs", "/api/heartbeat-runs/run-first"]);
expect(observed).toBe(false);
});
it("accepts real-bound exact file evidence despite an unrelated genuine finish rejection", () => {
const proof = gradeAntigravityJourneyTurn(antigravityTurnFixture());
expect(proof.toolLifecycle).toEqual({ completedFileOperations: 2, failedFileOperations: 0, failedFinishOperations: 1 });
const evidence = antigravityEvidenceFixture();
expect(() => validateAntigravityJourneyEvidence(evidence)).not.toThrow();
expect(gradeAntigravityJourney(evidence).every(check => check.passed)).toBe(true);
});
it("accepts Rust allocator and executor-hashed event IDs with independent source sequences", () => {
const fixture = antigravityTurnFixture();
const first = fixture.events[0]!;
first.sourceEventId = first.payload.prpEvent.sourceEventId = `event_${fixture.run.runnerInstanceId}_0000000000000001`;
expect(fixture.events[1]!.sourceEventId).toMatch(/^event_executor_[a-f0-9]{64}$/u);
let controlSeq = 0;
for (const row of fixture.events.filter(row => row.payload.prpEvent.sourceKind === "control_plane")) {
row.sourceSeq = row.payload.prpEvent.sourceSeq = ++controlSeq;
}
expect(() => gradeAntigravityJourneyTurn(fixture)).not.toThrow();
});
const changes: Array<[string, (f: ReturnType<typeof antigravityTurnFixture>) => void]> = [
["different task", f => { f.attachment.issueId = "another-task"; }],
["different assignee", f => { f.issue.assigneeAgentId = "other-agent"; }],
["stale previous run", f => { f.afterRunIds.push(f.run.id); }],
["copied attachment", f => { f.attachment.originatingRunId = "old-run"; }],
["missing file", f => { f.downloadedBytes = Buffer.alloc(0); }],
["wrong actual bytes", f => { f.downloadedBytes[0] = 0; }],
["success claim without delivery", f => { f.run.resultJson.semanticToolReceipts = {}; }],
["narrative Done without committed result", f => { f.run.resultJson.finalizationPhase = "prepared"; }],
["unsettled cost accounting", f => { f.run.costAccountingPending = true; }],
["different selected connection", f => { f.run.contextSnapshot.aiConnection.connectionId = "ambient-connection"; }],
["different effective model", f => { f.run.runnerProfileJson.sessionCheckpoint.providerIdentity.effectiveModel = "automatic"; }],
["old candidate profile", f => { f.expectedProfileDigest = `sha256:${"c".repeat(64)}`; }],
["foreign native evidence", f => { f.events[0]!.payload.prpEvent.sourceInstanceId = "other-runner"; }],
["missing event ID", f => { delete (f.events[0] as any).sourceEventId; delete (f.events[0]!.payload.prpEvent as any).sourceEventId; }],
["empty event ID", f => { f.events[0]!.sourceEventId = f.events[0]!.payload.prpEvent.sourceEventId = ""; }],
["oversized event ID", f => { f.events[0]!.sourceEventId = f.events[0]!.payload.prpEvent.sourceEventId = "x".repeat(161); }],
["unsafe event ID", f => { f.events[0]!.sourceEventId = f.events[0]!.payload.prpEvent.sourceEventId = "event\nforeign"; }],
["duplicate event ID", f => { f.events[1]!.sourceEventId = f.events[1]!.payload.prpEvent.sourceEventId = f.events[0]!.sourceEventId; }],
["foreign envelope event ID", f => { f.events[0]!.payload.prpEvent.sourceEventId = "event_from_another_row"; }],
["foreign row event ID", f => { f.events[0]!.sourceEventId = "event_from_another_row"; }],
["foreign paired source", f => { f.events[0]!.sourceInstanceId = f.events[0]!.payload.prpEvent.sourceInstanceId = "foreign-runner"; }],
["replayed source sequence", f => { f.events[1]!.sourceSeq = f.events[1]!.payload.prpEvent.sourceSeq = f.events[0]!.sourceSeq; }],
["reordered control source sequence", f => { f.events.at(-1)!.sourceSeq = f.events.at(-1)!.payload.prpEvent.sourceSeq = 1; }],
["missing source sequence", f => { delete (f.events[0] as any).sourceSeq; delete (f.events[0]!.payload.prpEvent as any).sourceSeq; }],
["failed file read", f => { f.events[3]!.payload.prpEvent.payload.status = "failed"; }],
["terminal task with active lease", f => { f.lease.status = "active"; }],
["unobserved physical owner", f => { f.retirement.captured = false; }],
["missing primary birth", f => { delete (f.retirement as any).primaryOwner; }],
["forged primary birth", f => { f.retirement.primaryOwner.startedAt = "forged"; }],
["different primary run", f => { f.retirement.primaryOwner.runId = "another-run"; }],
["different primary PID", f => { f.retirement.primaryOwner.pid++; }],
["different primary OS birth", f => { f.retirement.primaryOwner.startedAt = "2026-10-29T00:00:01Z"; }],
["unverified primary argv", f => { (f.retirement.primaryOwner as any).perTurnArgvVerified = false; }],
["remaining native process", f => { f.retirement.remainingProcessCount = 1; }],
["PID reused", f => { f.retirement.identityChanged = true; }],
["native child still owns lifetime fence", f => { f.retirement.nativeFencesReacquired = false; }],
];
it.each(changes)("rejects %s rather than trusting Done prose", (_label, change) => {
const fixture = antigravityTurnFixture(); change(fixture);
expect(() => gradeAntigravityJourneyTurn(fixture)).toThrow("antigravity_evidence_");
});
it.each(["acpxRecordId", "agentSessionId", "backendSessionId", "normalizedSessionId", "workspaceDigest", "profileDigest", "effectiveModel"] as const)("rejects replaced native conversation %s", key => {
const e = antigravityEvidenceFixture(); e.turnEvidence[1]!.conversation[key] = "replaced";
expect(sameAntigravityConversation(e.turnEvidence[0]!, e.turnEvidence[1]!)).toBe(false);
});
it("allows the real session-scoped runner ID only with a different primary process birth", () => {
const e = antigravityEvidenceFixture(), [first, second] = e.turnEvidence;
second!.runnerInstanceId = first!.runnerInstanceId;
expect(sameAntigravityConversation(first!, second!)).toBe(true);
second!.retirement.primaryOwner = { ...first!.retirement.primaryOwner, runId: second!.runId };
// A different sidecar/journal size does not prove a different primary Runner.
second!.retirement.observedProcessCount++;
expect(sameAntigravityConversation(first!, second!)).toBe(false);
});
it("binds the retained primary to the actual observer's per-turn argv, run, and OS birth", () => {
const f = antigravityTurnFixture(), owner = f.retirement.primaryOwner;
const observation = { captured: true, identityChanged: false, live: [], journal: [{ pid: owner.pid, parent: 1, start: owner.startedAt }], rootObserved: undefined,
authorityCheck: { accepted: true, commandSha256: `sha256:${owner.commandSha256}`, authority: { pid: owner.pid, groupId: owner.groupId, startedAt: owner.startedAt, runId: owner.runId },
observed: { pid: owner.pid, parent: 1, start: owner.startedAt } } };
expect(antigravityPrimaryOwner(f.run, observation)).toEqual(owner);
expect(antigravityPrimaryOwner(f.run, { ...observation, authorityCheck: { ...observation.authorityCheck, accepted: false } })).toBeUndefined();
observation.authorityCheck.authority.runId = "another-run";
expect(() => antigravityPrimaryOwner(f.run, observation)).toThrow("primary_owner_binding");
});
it.each(["", "c".repeat(64), "sha256:short", `sha256:${"C".repeat(64)}`, `sha256:${"c".repeat(65)}`])("rejects malformed shared-observer command digest %#", commandSha256 => {
const f = antigravityTurnFixture(), owner = f.retirement.primaryOwner;
expect(() => antigravityPrimaryOwner(f.run, { captured: true, identityChanged: false, live: [], rootObserved: undefined,
journal: [{ pid: owner.pid, parent: 1, start: owner.startedAt }], authorityCheck: { accepted: true, commandSha256,
authority: { pid: owner.pid, groupId: owner.groupId, startedAt: owner.startedAt, runId: owner.runId },
observed: { pid: owner.pid, parent: 1, start: owner.startedAt } } })).toThrow("primary_owner_command_digest");
});
it.skipIf(!["darwin", "linux"].includes(process.platform))("normalizes the real live observer's tagged command digest for a bounded owned child", async () => {
const runId = "hermetic-primary-owner", observer = observeRunProcesses();
// No Runner/provider/model: two finite test-owned Node processes expose the
// same argv/birth contract consumed by the selected local native observer.
const script = `const {spawn}=require('node:child_process');
const child=spawn(process.execPath,['-e','setTimeout(()=>{},3500)'],{stdio:'ignore'});
process.on('SIGTERM',()=>{child.kill();setTimeout(()=>process.exit(0),50)});
setTimeout(()=>{child.kill();process.exit(0)},4000);
console.log('ready');`;
const child = spawn(process.execPath, ["-e", script, "--", "--run-id", runId, "--lifecycle-mode", "per_turn"],
{ detached: true, stdio: ["ignore", "pipe", "ignore"], env: { PATH: process.env.PATH } });
const exited = once(child, "exit");
try {
await once(child.stdout!, "data");
const pid = child.pid!;
const startedAt = process.platform === "linux" ? readLinuxProcessStartedAt(pid)
: new Date(Date.parse(execFileSync("/bin/ps", ["-p", String(pid), "-o", "lstart="],
{ encoding: "utf8", env: { PATH: "/usr/bin:/bin", LC_ALL: "C" } }).trim())).toISOString();
const run = { id: runId, processPid: pid, processGroupId: pid, processStartedAt: startedAt };
let observation = observer.sample({ runId, pid, groupId: pid, startedAt });
await vi.waitFor(() => {
observation = observer.sample({ runId, pid, groupId: pid, startedAt });
expect(observation.captured).toBe(true); expect(observation.journal.length).toBeGreaterThanOrEqual(2);
}, { timeout: 2500 });
expect(observation.authorityCheck?.accepted).toBe(true);
expect(observation.authorityCheck!.commandSha256).toMatch(/^sha256:[a-f0-9]{64}$/u);
const owner = antigravityPrimaryOwner(run, observation)!;
expect(owner).toMatchObject({ runId, pid, groupId: pid, startedAt, perTurnArgvVerified: true });
expect(owner.commandSha256).toBe(observation.authorityCheck!.commandSha256.slice("sha256:".length));
expect(owner.commandSha256).toMatch(/^[a-f0-9]{64}$/u);
expect(() => antigravityPrimaryOwner({ ...run, id: "different-run" }, observation)).toThrow("primary_owner_binding");
expect(() => antigravityPrimaryOwner({ ...run, processStartedAt: "2026-01-01T00:00:00Z" }, observation)).toThrow("primary_owner_binding");
} finally {
if (child.exitCode === null && child.signalCode === null) child.kill("SIGTERM");
await exited;
await vi.waitFor(() => expect(observer.sample().live).toHaveLength(0), { timeout: 5000 });
}
}, 10_000);
it("retains first completion as a snapshot when later task reopening changes coordinator pointers", () => {
const first = antigravityTurnFixture(), observed = gradeAntigravityJourneyTurn(first);
first.run.resultJson.authoritativeDecision = "in_progress";
first.run.resultJson.assessmentId = "superseding-assessment";
expect(observed.finalization.decision).toBe("done");
expect(observed.finalization.assessmentId).toBe("assessment-run-first");
expect(() => gradeAntigravityJourneyTurn(first)).toThrow("committed_done");
});
it("rejects an arbitrary retained native payload and altered download bytes", () => {
const e = antigravityEvidenceFixture();
(e.turnEvidence[0]!.conversation as any).rawNativePrompt = "private";
expect(() => validateAntigravityJourneyEvidence(e)).toThrow("retained_shape");
delete (e.turnEvidence[0]!.conversation as any).rawNativePrompt;
e.turnEvidence[0]!.artifact.base64 = Buffer.from("not the proof").toString("base64");
expect(() => validateAntigravityJourneyEvidence(e)).toThrow("retained_artifact");
});
tests/runner-e2e/antigravity-journey-evidence.ts
+324
View File
@@ -0,0 +1,324 @@
import { createHash } from "node:crypto";
import { createServer } from "node:net";
import { isDeepStrictEqual } from "node:util";
import type { ConnectionApi } from "./connection-target.js";
import { connectionDelay } from "./connection-target.js";
import { observeRunProcesses, retainRunProcessIdentity } from "./copilot-local-fixtures.js";
import { isValidNativePrpEnvelope } from "./native-event-envelope.js";
import { collectRunEvents } from "./run-observations.js";
import type { Matcher } from "./types.js";
type Row = Record<string, any>;
const rec = (value: unknown): Row => value && typeof value === "object" && !Array.isArray(value) ? value as Row : {};
const hash = (bytes: Buffer | string) => createHash("sha256").update(bytes).digest("hex");
const requireProof = (condition: unknown, code: string) => { if (!condition) throw new Error(`antigravity_evidence_${code}`); };
const id = (value: unknown): value is string => typeof value === "string" && /^[A-Za-z0-9_-]{1,128}$/u.test(value);
const eventId = (value: unknown): value is string => typeof value === "string" && value.length > 0 && value.length <= 160 && !/[\x00-\x20\x7f]/u.test(value);
export const antigravityJourneyCheckpoints = ["target", "dependencies", "ordinary_picker", "saved_subscription", "exact_model", "setup_probe", "agent_ready", "clean_composer", "instructions_saved", "first_run", "first_artifact", "followup_run", "followup_artifact", "persisted"] as const;
export interface AntigravityPrimaryOwner {
runId: string; pid: number; groupId: number; startedAt: string; commandSha256: string; perTurnArgvVerified: true;
}
export function validAntigravityPrimaryOwner(value: unknown): value is AntigravityPrimaryOwner {
const owner = rec(value);
return Object.keys(owner).length === 6 && Object.keys(owner).every(key => ["runId", "pid", "groupId", "startedAt", "commandSha256", "perTurnArgvVerified"].includes(key))
&& id(owner.runId) && Number.isSafeInteger(owner.pid) && owner.pid > 1 && owner.groupId === owner.pid
&& typeof owner.startedAt === "string" && Number.isFinite(Date.parse(owner.startedAt))
&& /^[a-f0-9]{64}$/u.test(owner.commandSha256) && owner.perTurnArgvVerified === true;
}
/** Capture only a primary whose live argv and OS birth were independently bound
* by the process observer to this exact run and per_turn lifecycle. */
export function antigravityPrimaryOwner(run: Row, observation: ReturnType<ReturnType<typeof observeRunProcesses>["sample"]>): AntigravityPrimaryOwner | undefined {
const check = observation.authorityCheck;
if (!check?.accepted) return;
requireProof(observation.captured && Number.isFinite(Date.parse(check.observed.start)), "primary_owner_birth");
// The shared live observer emits a tagged digest. Normalize only that exact
// producer format into the retained witness's bare SHA-256 field.
requireProof(/^sha256:[a-f0-9]{64}$/u.test(check.commandSha256), "primary_owner_command_digest");
const owner: AntigravityPrimaryOwner = { runId: check.authority.runId, pid: check.observed.pid, groupId: check.authority.groupId,
startedAt: new Date(Date.parse(check.observed.start)).toISOString(), commandSha256: check.commandSha256.slice("sha256:".length), perTurnArgvVerified: true };
requireProof(validAntigravityPrimaryOwner(owner) && owner.runId === run.id && owner.pid === run.processPid && check.authority.pid === owner.pid
&& Date.parse(check.authority.startedAt) === Date.parse(owner.startedAt)
&& owner.groupId === run.processGroupId && Date.parse(owner.startedAt) === Date.parse(run.processStartedAt)
&& observation.journal.some(process => process.pid === owner.pid && Date.parse(process.start) === Date.parse(owner.startedAt)), "primary_owner_binding");
return owner;
}
export interface AntigravityJourneyTurnEvidence {
runId: string; companyId: string; agentId: string; issueId: string; connectionId: string; environmentId: string;
nativeSessionId: string; runnerInstanceId: string; model: string;
conversation: { kind: "acpx"; acpxRecordId: string; agentSessionId: string; backendSessionId: string; normalizedSessionId: string;
requestedModel: string; effectiveModel: string; permissionMode: string; workspaceDigest: string; profileDigest: string };
artifact: { attachmentId: string; filename: string; sha256: string; byteSize: number; base64: string };
finalization: { phase: "committed"; decision: "done"; assessmentId: string; decisionId: string; observedAt: string;
acceptedEventSeq: number; terminalEventSeq: number; nativeResultSha256: string };
toolLifecycle: { completedFileOperations: number; failedFileOperations: number; failedFinishOperations: number };
retirement: { captured: boolean; observedProcessCount: number; remainingProcessCount: number; identityChanged: boolean;
primaryOwner: AntigravityPrimaryOwner; nativeFencesReacquired: boolean; nativeFenceCount: number; leaseId: string; leaseStatus: string };
}
export interface AntigravityJourneyEvidence {
version: 1; setupId: string; hosting: "self-hosted"; targetCommit: string | null;
phase: string; outcome: "running" | "passed" | "failed" | "awaiting_user" | "missing_credential" | "blocked_target";
assisted: boolean; privacy: "private"; authentication: "saved-subscription"; model: string;
companyId?: string; agentId?: string; connectionId?: string;
checkpoints: Partial<Record<typeof antigravityJourneyCheckpoints[number], boolean>>;
turnEvidence: AntigravityJourneyTurnEvidence[];
cleanup: { agent: boolean; runs: boolean; connectionPreserved: boolean };
}
interface TurnScope {
companyId: string; agentId: string; issueId: string; connectionId: string; environmentId: string;
model: string; expectedProfileDigest: string; expectedBytes: Buffer; afterRunIds: readonly string[];
}
/** Grade actual public objects and downloaded bytes, never an agent's summary.
* Capture this before reopening the task: finalizer coordinator pointers are mutable. */
export function gradeAntigravityJourneyTurn(input: TurnScope & {
run: Row; issue: Row; attachment: Row; downloadedBytes: Buffer; events: Row[]; lease: Row;
retirement: Omit<AntigravityJourneyTurnEvidence["retirement"], "leaseId" | "leaseStatus">;
}): AntigravityJourneyTurnEvidence {
const r = input.run, result = rec(r.resultJson), profile = rec(r.runnerProfileJson), checkpoint = rec(profile.sessionCheckpoint);
const native = rec(profile.nativeExecutionInput), provider = rec(native.provider), binding = rec(native.binding);
const connection = rec(rec(r.contextSnapshot).aiConnection), environment = rec(rec(r.contextSnapshot).paperclipEnvironment);
const identity = rec(checkpoint.providerIdentity), a = input.attachment;
requireProof(id(r.id) && !input.afterRunIds.includes(r.id) && r.companyId === input.companyId && r.agentId === input.agentId
&& r.issueId === input.issueId && r.status === "succeeded" && r.runtimeMode === "native"
&& r.driverKind === "acpx_runtime" && id(r.nativeSessionId) && id(r.runnerInstanceId) && r.costAccountingPending === false, "run_binding");
requireProof(binding.runId === r.id && binding.companyId === input.companyId && binding.agentId === input.agentId && binding.issueId === input.issueId
&& profile.backend === "acpx_runtime" && provider.kind === "acpx" && provider.agent === "antigravity" && provider.model === input.model
&& environment.id === input.environmentId && environment.driver === "local" && connection.provider === "google"
&& connection.method === "subscription" && connection.connectionId === input.connectionId, "selected_provider");
requireProof(identity.kind === "acpx" && id(identity.acpxRecordId) && id(identity.agentSessionId) && id(identity.backendSessionId)
&& identity.agentSessionId === identity.backendSessionId && identity.normalizedSessionId === r.nativeSessionId
&& identity.requestedModel === input.model && identity.effectiveModel === input.model
&& identity.profileDigest === input.expectedProfileDigest && identity.profileDigest === rec(provider.profile).commandDigest && /^sha256:[a-f0-9]{64}$/u.test(identity.profileDigest)
&& /^sha256:[a-f0-9]{64}$/u.test(identity.workspaceDigest) && identity.permissionMode === "approve-all", "native_identity");
requireProof(input.issue.id === input.issueId && input.issue.companyId === input.companyId && input.issue.assigneeAgentId === input.agentId
&& input.issue.status === "done" && result.finalizationPhase === "committed" && result.authoritativeDecision === "done"
&& result.issueStatusAfter === "done" && result.workspaceFinalizeStatus === "succeeded" && id(result.assessmentId) && id(result.decisionId)
&& rec(result.nativeResult).schema === "paperclip.run_result.v1" && rec(result.nativeTerminal).runTerminalState === "succeeded", "committed_done");
const expectedSha = hash(input.expectedBytes);
requireProof(input.expectedBytes.length > 0 && input.expectedBytes.length <= 4096 && id(a.id) && a.companyId === input.companyId
&& a.issueId === input.issueId && a.createdByAgentId === input.agentId && a.originatingRunId === r.id
&& a.originalFilename === "antigravity-proof.txt" && a.contentType === "text/plain" && a.byteSize === input.expectedBytes.length
&& a.sha256 === expectedSha && input.downloadedBytes.equals(input.expectedBytes), "exact_download");
const registrations = Object.values(rec(result.semanticToolReceipts)).map(rec).filter(receipt => receipt.operationId === "register_deliverable"
&& rec(receipt.result).attachmentId === a.id);
requireProof(registrations.length === 1 && rec(registrations[0]!.result).disposition === "applied"
&& rec(registrations[0]!.input).filename === a.originalFilename && rec(registrations[0]!.input).byteSize === a.byteSize
&& rec(registrations[0]!.input).sha256 === a.sha256, "registered_delivery");
requireProof(input.events.length > 0 && input.events.length <= 20_000, "bounded_events");
const seen = new Set<string>(), sourceSequences = new Map<string, number>(); let lastSeq = 0;
const tools: Array<{ event: Row; payload: Row; seq: number }> = [];
const accepted: Array<{ event: Row; seq: number }> = [], terminals: Array<{ event: Row; seq: number }> = [];
for (const row of input.events) {
const event = rec(rec(row.payload).prpEvent);
if (!Object.keys(event).length) continue;
requireProof(row.companyId === input.companyId && row.runId === r.id && event.runId === r.id
&& isValidNativePrpEnvelope(event, row.protocolSchemaVersion) && row.eventType === event.eventType
&& Number.isSafeInteger(row.seq) && row.seq > lastSeq && eventId(event.sourceEventId) && !seen.has(event.sourceEventId)
&& row.sourceEventId === event.sourceEventId && row.sourceInstanceId === event.sourceInstanceId
&& row.sourceSeq === event.sourceSeq && Number.isSafeInteger(event.sourceSeq) && event.sourceSeq > (sourceSequences.get(event.sourceInstanceId) ?? 0)
&& ((event.sourceKind === "runner" && event.sourceInstanceId === r.runnerInstanceId)
|| (event.sourceKind === "control_plane" && event.sourceInstanceId === `${r.runnerInstanceId}:control`))
&& event.normalizedSessionId === r.nativeSessionId, "event_binding");
lastSeq = row.seq; seen.add(event.sourceEventId); sourceSequences.set(event.sourceInstanceId, event.sourceSeq);
if (event.eventType === "run.result.accepted") accepted.push({ event, seq: row.seq });
if (event.eventType === "run.terminal") terminals.push({ event, seq: row.seq });
if (event.eventType.startsWith("tool.execution.")) tools.push({ event, payload: rec(event.payload), seq: row.seq });
}
const fileTools = tools.filter(t => t.event.eventType === "tool.execution.completed" && ["read", "edit", "write"].includes(t.payload.operation) && t.payload.target === "antigravity-proof.txt");
requireProof(fileTools.length > 0 && fileTools.every(t => t.event.sourceKind === "runner" && t.event.sourceInstanceId === r.runnerInstanceId
&& t.event.normalizedSessionId === r.nativeSessionId && t.payload.status === "completed"), "file_tool_lifecycle");
requireProof(fileTools.some(t => t.payload.operation === "read") && fileTools.some(t => ["edit", "write"].includes(t.payload.operation)), "actual_read_and_write");
const edits = fileTools.filter(t => ["edit", "write"].includes(t.payload.operation)), reads = fileTools.filter(t => t.payload.operation === "read");
requireProof(reads.some(t => t.seq > edits.at(-1)!.seq) && (!input.afterRunIds.length || reads.some(t => t.seq < edits[0]!.seq)), "file_read_order");
for (const terminal of fileTools) {
const correlated = tools.filter(t => t.payload.executionId === terminal.payload.executionId);
const starts = correlated.filter(t => t.event.eventType === "tool.execution.started");
requireProof(id(terminal.payload.executionId) && starts.length === 1 && starts[0]!.seq < terminal.seq
&& correlated.filter(t => t.event.eventType === "tool.execution.completed").length === 1
&& correlated.every(t => t.event.sourceInstanceId === terminal.event.sourceInstanceId && t.event.normalizedSessionId === terminal.event.normalizedSessionId
&& t.event.turnId === terminal.event.turnId && t.payload.operation === terminal.payload.operation && t.payload.target === terminal.payload.target
&& t.seq >= starts[0]!.seq && t.seq <= terminal.seq && t.payload.status === (t === terminal ? "completed" : "running")), "correlated_file_lifecycle");
}
requireProof(accepted.length === 1 && terminals.length === 1 && accepted[0]!.seq > fileTools.at(-1)!.seq && terminals[0]!.seq > accepted[0]!.seq
&& [...accepted, ...terminals].every(({ event }) => event.sourceKind === "control_plane" && event.sourceInstanceId === `${r.runnerInstanceId}:control`
&& event.normalizedSessionId === r.nativeSessionId && id(event.turnId))
&& accepted[0]!.event.turnId === terminals[0]!.event.turnId
&& fileTools.every(t => t.event.turnId === accepted[0]!.event.turnId)
&& isDeepStrictEqual(rec(accepted[0]!.event.payload).result, result.nativeResult)
&& rec(terminals[0]!.event.payload).runTerminalState === "succeeded" && rec(terminals[0]!.event.payload).turnTerminalState === "completed"
&& rec(terminals[0]!.event.payload).reportedWorkDisposition === "done", "accepted_native_result");
const lease = input.lease;
requireProof(id(lease.id) && lease.id === environment.leaseId && lease.companyId === input.companyId && lease.environmentId === input.environmentId
&& lease.heartbeatRunId === r.id && lease.status === "released" && Boolean(lease.releasedAt), "lease_retirement");
const retirement = input.retirement;
requireProof(retirement.captured && retirement.observedProcessCount >= 2 && retirement.remainingProcessCount === 0
&& !retirement.identityChanged && retirement.nativeFencesReacquired && retirement.nativeFenceCount === 3
&& validAntigravityPrimaryOwner(retirement.primaryOwner) && retirement.primaryOwner.runId === r.id
&& retirement.primaryOwner.pid === r.processPid && retirement.primaryOwner.groupId === r.processGroupId
&& Date.parse(retirement.primaryOwner.startedAt) === Date.parse(r.processStartedAt), "physical_retirement");
const conversation = { kind: "acpx" as const, acpxRecordId: identity.acpxRecordId, agentSessionId: identity.agentSessionId,
backendSessionId: identity.backendSessionId, normalizedSessionId: identity.normalizedSessionId,
requestedModel: identity.requestedModel, effectiveModel: identity.effectiveModel, permissionMode: identity.permissionMode,
workspaceDigest: identity.workspaceDigest, profileDigest: identity.profileDigest };
return { runId: r.id, companyId: input.companyId, agentId: input.agentId, issueId: input.issueId, connectionId: input.connectionId,
environmentId: input.environmentId, nativeSessionId: r.nativeSessionId, runnerInstanceId: r.runnerInstanceId, model: input.model, conversation,
artifact: { attachmentId: a.id, filename: a.originalFilename, sha256: expectedSha, byteSize: input.downloadedBytes.length, base64: input.downloadedBytes.toString("base64") },
finalization: { phase: "committed", decision: "done", assessmentId: result.assessmentId, decisionId: result.decisionId, observedAt: new Date().toISOString(),
acceptedEventSeq: accepted[0]!.seq, terminalEventSeq: terminals[0]!.seq, nativeResultSha256: hash(JSON.stringify(result.nativeResult)) },
toolLifecycle: { completedFileOperations: fileTools.length, failedFileOperations: 0,
failedFinishOperations: tools.filter(t => t.event.eventType === "tool.execution.completed" && /(?:^|_)paperclip_finish$/u.test(t.payload.name) && t.payload.status === "failed").length },
retirement: { ...retirement, leaseId: lease.id, leaseStatus: lease.status } };
}
/** Exact same tagged conversation, with a fresh physical Runner per turn. */
export function sameAntigravityConversation(first: AntigravityJourneyTurnEvidence, followup: AntigravityJourneyTurnEvidence) {
const left = first.retirement.primaryOwner, right = followup.retirement.primaryOwner;
return first.runId !== followup.runId && validAntigravityPrimaryOwner(left) && validAntigravityPrimaryOwner(right)
&& left.runId === first.runId && right.runId === followup.runId
&& (left.pid !== right.pid || Date.parse(left.startedAt) !== Date.parse(right.startedAt))
&& Date.parse(right.startedAt) > Date.parse(left.startedAt)
&& ["companyId", "agentId", "issueId", "connectionId", "environmentId", "model", "nativeSessionId"].every(key => first[key as keyof typeof first] === followup[key as keyof typeof followup])
&& Object.keys(first.conversation).every(key => first.conversation[key as keyof typeof first.conversation] === followup.conversation[key as keyof typeof followup.conversation]);
}
/** Reacquire all three production lifetime fences together, then release them.
* Valid only for the declared same-host loopback target; never probe a remote host. */
export async function reacquireAntigravityNativeFences(value: unknown) {
requireProof(Array.isArray(value) && value.length === 3 && new Set(value).size === 3
&& value.every(p => Number.isSafeInteger(p) && p >= 49152 && p <= 65535), "native_fence_identity");
const servers: ReturnType<typeof createServer>[] = [];
try {
for (const port of value as number[]) {
const server = createServer(); servers.push(server);
await new Promise<void>((resolve, reject) => { server.once("error", reject); server.listen({ host: "127.0.0.1", port, exclusive: true }, () => resolve()); });
}
return true;
} finally {
await Promise.all(servers.map(server => new Promise<void>((resolve, reject) => {
if (!server.listening) { resolve(); return; }
server.close(error => error ? reject(error) : resolve());
})));
}
}
/** /issues/:id/runs is an activity summary, whose public identity is runId.
* Fetch the full row separately before observing any host process. */
export function selectAntigravityIssueRun(summaries: unknown, agentId: string, afterRunIds: readonly string[]): string | null {
requireProof(Array.isArray(summaries) && summaries.length <= 100, "issue_run_summaries");
const seen = new Set<string>();
const selected: string[] = [];
for (const value of summaries as unknown[]) {
const summary = rec(value);
requireProof(id(summary.runId) && summary.agentId === agentId && !seen.has(summary.runId)
&& (summary.id === undefined || summary.id === summary.runId), "issue_run_summary_binding");
seen.add(summary.runId);
if (!afterRunIds.includes(summary.runId)) selected.push(summary.runId);
}
requireProof(selected.length <= 1, "extra_provider_run");
return selected[0] ?? null;
}
export async function verifyAntigravityJourneyTurn(input: TurnScope & {
api: ConnectionApi; deadlineAt: number; observeRun?: (run: Row) => Promise<void>; assertActive?: () => void;
}): Promise<AntigravityJourneyTurnEvidence> {
requireProof(process.platform === "darwin" && process.arch === "arm64" && new URL(input.api.origin).hostname === "127.0.0.1", "same_host_target_required");
const observer = observeRunProcesses();
let processes: ReturnType<typeof observer.sample> | undefined;
while (Date.now() < input.deadlineAt) {
input.assertActive?.();
const summaries = await input.api.get<unknown>(`/api/issues/${input.issueId}/runs`);
const selected = selectAntigravityIssueRun(summaries, input.agentId, input.afterRunIds);
if (selected) {
const run = await input.api.get<Row>(`/api/heartbeat-runs/${selected}`);
requireProof(run.id === selected && run.companyId === input.companyId && run.agentId === input.agentId && run.issueId === input.issueId, "observed_run_binding");
if (input.observeRun) await input.observeRun(run);
const sampled = observer.sample(run.processPid ? { pid: run.processPid, groupId: run.processGroupId, startedAt: run.processStartedAt, runId: run.id } : undefined);
processes = processes ? retainRunProcessIdentity(processes, sampled) : sampled;
requireProof(!["failed", "cancelled", "interrupted", "timed_out"].includes(run.status), "terminal_run_failure");
if (run.status === "succeeded" && run.costAccountingPending === false && rec(run.resultJson).finalizationPhase === "committed") {
const issue = await input.api.get<Row>(`/api/issues/${input.issueId}`);
const attachments = await input.api.get<Row[]>(`/api/issues/${input.issueId}/attachments`);
const matching = attachments.filter(a => a.originatingRunId === run.id && a.originalFilename === "antigravity-proof.txt");
const leaseId = rec(rec(run.contextSnapshot).paperclipEnvironment).leaseId;
requireProof(id(leaseId), "lease_identity");
const lease = await input.api.get<Row>(`/api/environment-leases/${leaseId}`);
if (issue.status === "done" && lease.status === "released" && processes.live.length === 0) {
input.assertActive?.();
requireProof(matching.length === 1 && id(matching[0]!.id) && matching[0]!.byteSize <= 4096, "attachment_selection");
const downloadedBytes = await input.api.bytes(`/api/attachments/${matching[0]!.id}/content`);
const events = await collectRunEvents<Row>((afterSeq, limit) => input.api.get(`/api/heartbeat-runs/${run.id}/events?afterSeq=${afterSeq}&limit=${limit}`));
const fences = rec(rec(rec(run.runnerProfileJson).sessionCheckpoint).providerIdentity).providerLifetimeFenceCandidates;
const nativeFencesReacquired = await reacquireAntigravityNativeFences(fences);
const primaryOwner = antigravityPrimaryOwner(run, processes);
requireProof(primaryOwner, "primary_owner_missing");
return gradeAntigravityJourneyTurn({ ...input, run, issue, attachment: matching[0]!, downloadedBytes, events, lease,
retirement: { captured: processes.captured, observedProcessCount: processes.journal.length, remainingProcessCount: processes.live.length,
identityChanged: processes.identityChanged, primaryOwner: primaryOwner!, nativeFencesReacquired, nativeFenceCount: 3 } });
}
}
}
await connectionDelay(250);
}
throw new Error("antigravity_evidence_turn_deadline");
}
export function gradeAntigravityJourney(evidence: AntigravityJourneyEvidence) {
const checkpointChecks = antigravityJourneyCheckpoints.map(key => ({ matcher: { kind: "json_path", path: `antigravityJourney.checkpoints.${key}`, expected: true } as Matcher,
passed: evidence.checkpoints[key] === true, detail: `${key}: ${evidence.checkpoints[key] === true ? "verified" : "missing"}` }));
const [first, second] = evidence.turnEvidence;
const fileSequence = first && second && /^[A-Za-z0-9+/=]+$/u.test(first.artifact.base64) && /^[A-Za-z0-9+/=]+$/u.test(second.artifact.base64)
&& /^ANTIGRAVITY-INSTRUCTION-[a-f0-9-]{36}\n$/u.test(Buffer.from(first.artifact.base64, "base64").toString("utf8"))
&& Buffer.from(second.artifact.base64, "base64").toString("utf8").startsWith(Buffer.from(first.artifact.base64, "base64").toString("utf8"))
&& /^ANTIGRAVITY-FOLLOWUP-[a-f0-9-]{36}\n$/u.test(Buffer.from(second.artifact.base64, "base64").toString("utf8").slice(first.artifact.byteSize));
return [...checkpointChecks,
{ matcher: { kind: "json_path", path: "antigravityJourney.turnEvidence", expected: 2 } as Matcher,
passed: evidence.turnEvidence.length === 2 && sameAntigravityConversation(first!, second!) && Boolean(fileSequence), detail: "Two independently observed turns in the same tagged native conversation with exact original and appended bytes" },
{ matcher: { kind: "json_path", path: "antigravityJourney.cleanup", expected: true } as Matcher,
passed: Object.values(evidence.cleanup).every(Boolean), detail: "Owned runs retired and agent cleanup completed; shared saved connection preserved" }];
}
/** Closed retained projection; reject arbitrary native payloads before rendering. */
export function validateAntigravityJourneyEvidence(value: unknown): asserts value is AntigravityJourneyEvidence {
const closed = (value: unknown, keys: string[]) => {
const row = rec(value);
requireProof(Object.keys(row).length > 0 && Object.keys(row).every(key => keys.includes(key)), "retained_shape");
return row;
};
const row = closed(value, ["version", "setupId", "hosting", "targetCommit", "phase", "outcome", "assisted", "privacy", "authentication", "model", "companyId", "agentId", "connectionId", "checkpoints", "turnEvidence", "cleanup"]);
requireProof(row.version === 1 && /^[a-z0-9-]{1,80}$/u.test(row.setupId) && row.hosting === "self-hosted" && row.privacy === "private"
&& (row.targetCommit === null || /^[a-f0-9]{40}$/u.test(row.targetCommit)) && /^[a-z0-9_]{1,100}$/u.test(row.phase)
&& ["running", "passed", "failed", "awaiting_user", "missing_credential", "blocked_target"].includes(row.outcome)
&& typeof row.assisted === "boolean" && row.authentication === "saved-subscription" && row.model === "gemini-3.8-flash-low"
&& ["companyId", "agentId", "connectionId"].every(key => row[key] === undefined || id(row[key])), "retained_identity");
requireProof(row.checkpoints && typeof row.checkpoints === "object" && !Array.isArray(row.checkpoints)
&& Object.entries(row.checkpoints).every(([key, v]) => antigravityJourneyCheckpoints.includes(key as typeof antigravityJourneyCheckpoints[number]) && typeof v === "boolean"), "retained_checkpoints");
const cleanup = closed(row.cleanup, ["agent", "runs", "connectionPreserved"]);
requireProof(["agent", "runs", "connectionPreserved"].every(key => typeof cleanup[key] === "boolean"), "retained_cleanup");
requireProof(Array.isArray(row.turnEvidence) && row.turnEvidence.length <= 2, "retained_turn_count");
for (const turn of row.turnEvidence) {
const t = closed(turn, ["runId", "companyId", "agentId", "issueId", "connectionId", "environmentId", "nativeSessionId", "runnerInstanceId", "model", "conversation", "artifact", "finalization", "toolLifecycle", "retirement"]);
requireProof(["runId", "companyId", "agentId", "issueId", "connectionId", "environmentId", "nativeSessionId", "runnerInstanceId"].every(key => id(t[key])) && t.model === row.model, "retained_turn_identity");
const c = closed(t.conversation, ["kind", "acpxRecordId", "agentSessionId", "backendSessionId", "normalizedSessionId", "requestedModel", "effectiveModel", "permissionMode", "workspaceDigest", "profileDigest"]);
requireProof(c.kind === "acpx" && ["acpxRecordId", "agentSessionId", "backendSessionId", "normalizedSessionId"].every(key => id(c[key]))
&& c.agentSessionId === c.backendSessionId && c.normalizedSessionId === t.nativeSessionId && c.requestedModel === row.model && c.effectiveModel === row.model
&& c.permissionMode === "approve-all" && [c.workspaceDigest, c.profileDigest].every(v => /^sha256:[a-f0-9]{64}$/u.test(v)), "retained_native_identity");
const a = closed(t.artifact, ["attachmentId", "filename", "sha256", "byteSize", "base64"]);
requireProof(id(a.attachmentId) && a.filename === "antigravity-proof.txt" && Number.isSafeInteger(a.byteSize) && a.byteSize > 0 && a.byteSize <= 4096
&& typeof a.base64 === "string" && a.base64.length <= 5464 && /^[A-Za-z0-9+/=]+$/u.test(a.base64)
&& Buffer.from(a.base64, "base64").toString("base64") === a.base64 && Buffer.from(a.base64, "base64").length === a.byteSize
&& hash(Buffer.from(a.base64, "base64")) === a.sha256, "retained_artifact");
const f = closed(t.finalization, ["phase", "decision", "assessmentId", "decisionId", "observedAt", "acceptedEventSeq", "terminalEventSeq", "nativeResultSha256"]);
requireProof(f.phase === "committed" && f.decision === "done" && id(f.assessmentId) && id(f.decisionId) && Number.isFinite(Date.parse(f.observedAt))
&& Number.isSafeInteger(f.acceptedEventSeq) && f.acceptedEventSeq > 0 && Number.isSafeInteger(f.terminalEventSeq) && f.terminalEventSeq > f.acceptedEventSeq
&& /^[a-f0-9]{64}$/u.test(f.nativeResultSha256), "retained_finalization");
const tools = closed(t.toolLifecycle, ["completedFileOperations", "failedFileOperations", "failedFinishOperations"]);
requireProof(Number.isSafeInteger(tools.completedFileOperations) && tools.completedFileOperations >= 2 && tools.completedFileOperations <= 20_000
&& tools.failedFileOperations === 0 && Number.isSafeInteger(tools.failedFinishOperations) && tools.failedFinishOperations >= 0 && tools.failedFinishOperations <= 20_000, "retained_tools");
const retired = closed(t.retirement, ["captured", "observedProcessCount", "remainingProcessCount", "identityChanged", "primaryOwner", "nativeFencesReacquired", "nativeFenceCount", "leaseId", "leaseStatus"]);
requireProof(retired.captured === true && Number.isSafeInteger(retired.observedProcessCount) && retired.observedProcessCount >= 2
&& retired.observedProcessCount <= 1000 && retired.remainingProcessCount === 0 && retired.identityChanged === false
&& validAntigravityPrimaryOwner(retired.primaryOwner) && retired.primaryOwner.runId === t.runId
&& retired.nativeFencesReacquired === true && retired.nativeFenceCount === 3 && id(retired.leaseId) && retired.leaseStatus === "released", "retained_retirement");
}
}
tests/runner-e2e/antigravity-journey-fixtures.ts
+406
View File
@@ -0,0 +1,406 @@
import { QUALIFIED_ACPX_PROFILES } from "../../packages/paperclip-runner/src/drivers/acpx/qualified-profiles.js";
import { ANTIGRAVITY_DISTRIBUTION_PINS } from "../../packages/paperclip-runner/src/drivers/acpx/generated-profiles.js";
import { readFile } from "node:fs/promises";
import { execFileSync } from "node:child_process";
import { randomUUID } from "node:crypto";
import { ANTIGRAVITY_JOURNEY_EXECUTION } from "./antigravity-journey-cases.js";
import { ANTIGRAVITY_JOURNEY_MODEL, antigravityConnectionConfig, type AntigravityJourneyConfig } from "./antigravity-journey-config.js";
import { ConnectionBlock, ConnectionFailure } from "./connection-config.js";
import { ConnectionApi, connectionDelay, verifyConnectionTarget } from "./connection-target.js";
import type { AntigravityJourneyEvidence, AntigravityPrimaryOwner } from "./antigravity-journey-evidence.js";
import { antigravityPrimaryOwner, reacquireAntigravityNativeFences, validAntigravityPrimaryOwner } from "./antigravity-journey-evidence.js";
import { observeRunProcesses } from "./copilot-local-fixtures.js";
type Row = Record<string, any>;
/** Work failure and retirement are separate: every finished run status still
* needs authoritative accounting and the same physical cleanup proof. */
export function isAntigravityRunAccountingSettled(run: Row) {
return ["succeeded", "failed", "cancelled", "interrupted", "timed_out"].includes(run.status) && run.costAccountingPending === false;
}
export interface AntigravityJourneyResources {
version: 1; executionId: string; origin: string; companyId: string; environmentId: string; connectionId: string;
agentName: string; agentId?: string; issueId?: string; issueIdentifier?: string; runIds: string[];
/** Intended harness; the ordinary name-only wizard creates a held provisional draft. */
agentCreationRequest?: { setupId: string; companyId: string; agentName: string; provider: "antigravity"; creatorUserId: string; origin: string };
retained: boolean; setupProbe: "not_started" | "pending" | "retired" | "retirement_unconfirmed"; processes: Record<string, AntigravityProcessReceipt>;
noLaunch?: Record<string, AntigravityNoLaunchReceipt>;
sourceProvenance?: { head: string; trackedDiffSha256: string; untrackedSourceSha256: string; untrackedSourceFileCount: number; suiteDefinitionHash: string; dirty: boolean };
}
export interface AntigravityProcessReceipt {
captured: boolean; identityChanged: boolean; primaryOwner?: AntigravityPrimaryOwner; journal: Array<{ pid: number; parent: number; start: string }>;
}
export interface AntigravityNoLaunchReceipt {
kind: "cancelled-before-native-selection"; runId: string; companyId: string; agentId: string; issueId: string;
environmentId: string; leaseId: string; leaseStatus: "released" | "expired";
cancelRequestedAt: string; startupPreparationSettledAt: string; leaseReleasedAt: string; observedAt: string;
}
/** The server writes startupPreparationSettledAt only after preparation and
* lease cleanup finish without native ownership. Absence of a PID alone is not proof. */
export function antigravityNoLaunchReceipt(run: Row, lease: Row, resources: AntigravityJourneyResources): AntigravityNoLaunchReceipt {
const cancellation = run.resultJson?.startupCancellation;
const requestedAt = cancellation?.requestedAt, settledAt = run.resultJson?.startupPreparationSettledAt;
const environment = run.contextSnapshot?.paperclipEnvironment, profile = run.runnerProfileJson;
const prior = resources.processes[run.id];
const instant = (value: unknown): value is string => typeof value === "string" && Number.isFinite(Date.parse(value));
const uuid = (value: unknown) => typeof value === "string" && /^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/iu.test(value);
if (!resources.runIds.includes(run.id) || run.companyId !== resources.companyId || run.agentId !== resources.agentId
|| !resources.issueId || run.issueId !== resources.issueId || run.status !== "cancelled" || run.executionStage !== "settled"
|| run.costAccountingPending !== false || run.runtimeMode !== "legacy" || run.runtimeModeResolvedAt !== null
|| run.controllerLeaseExpiresAt !== null || cancellation?.beforeNativeSelection !== true || cancellation.errorCode !== "cancelled"
|| !instant(requestedAt) || !instant(settledAt) || Date.parse(requestedAt) > Date.parse(settledAt)
|| ["processPid", "processGroupId", "processStartedAt", "runnerInstanceId", "nativeSessionId", "driverKind", "completionContractId"].some(key => run[key] !== null)
|| !profile || Object.keys(profile).length !== 1 || !profile.adapterDispatch || Object.keys(profile.adapterDispatch).length !== 1
|| profile.adapterDispatch.adapterType !== "paperclip_runner" || prior?.captured || prior?.identityChanged || (prior?.journal.length ?? 0) > 0
|| environment?.id !== resources.environmentId || environment.driver !== "local" || !uuid(environment.leaseId)
|| lease.id !== environment.leaseId || lease.companyId !== resources.companyId || lease.heartbeatRunId !== run.id
|| lease.environmentId !== resources.environmentId || !["released", "expired"].includes(lease.status)
|| !instant(lease.releasedAt) || Date.parse(lease.releasedAt) < Date.parse(requestedAt) || Date.parse(lease.releasedAt) > Date.parse(settledAt)
|| Date.parse(settledAt) > Date.now()) throw new ConnectionFailure("cleanup_no_launch_receipt_unconfirmed");
return { kind: "cancelled-before-native-selection", runId: run.id, companyId: resources.companyId, agentId: run.agentId,
issueId: run.issueId, environmentId: resources.environmentId, leaseId: lease.id, leaseStatus: lease.status,
cancelRequestedAt: requestedAt, startupPreparationSettledAt: settledAt, leaseReleasedAt: lease.releasedAt, observedAt: new Date().toISOString() };
}
export type AntigravityProcessObservers = Map<string, ReturnType<typeof observeRunProcesses>>;
export function selectAntigravityJourneyEnvironment(allowedEnvironments: Row[], environmentId: string): Row {
const matches = allowedEnvironments.filter(environment => environment.id === environmentId && environment.driver === "local" && environment.status === "active");
if (matches.length !== 1) throw new ConnectionBlock("blocked_target", "configured_execution_environment_unavailable");
return matches[0]!;
}
export function observeAntigravityRun(resources: AntigravityJourneyResources, run: Row, observers: AntigravityProcessObservers) {
if (run.companyId !== resources.companyId || run.agentId !== resources.agentId || (resources.issueId && run.issueId !== resources.issueId))
throw new ConnectionFailure("observed_run_owner_mismatch");
if (!resources.runIds.includes(run.id)) resources.runIds.push(run.id);
let observer = observers.get(run.id);
if (!observer && !run.processPid) {
resources.processes[run.id] ??= { captured: false, identityChanged: false, journal: [] };
return;
}
if (!observer) { observer = observeRunProcesses(); observers.set(run.id, observer); }
const observation = observer.sample(run.processPid ? { pid: run.processPid, groupId: run.processGroupId, startedAt: run.processStartedAt, runId: run.id } : undefined);
const previous = resources.processes[run.id];
const journal = previous?.journal ?? [];
for (const process of observation.journal) if (!journal.some(row => row.pid === process.pid && row.start === process.start)) journal.push({ pid: process.pid, parent: process.parent, start: process.start });
// Retain accepted physical ownership before deriving a report projection.
// The launcher's error path persists this record even if projection fails.
const retained: AntigravityProcessReceipt = { captured: Boolean(previous?.captured || observation.captured),
identityChanged: Boolean(previous?.identityChanged || observation.identityChanged), ...(previous?.primaryOwner ? { primaryOwner: previous.primaryOwner } : {}), journal };
resources.processes[run.id] = retained;
retained.primaryOwner ??= antigravityPrimaryOwner(run, observation);
}
/** Read only the same host's kernel birth identities; never signal a reported PID. */
function remainingRecordedProcesses(receipt: AntigravityProcessReceipt) {
const table = execFileSync("/bin/ps", ["-axo", "pid=,ppid=,lstart="], { encoding: "utf8", timeout: 3000, maxBuffer: 8 * 1024 * 1024,
env: { PATH: "/usr/bin:/bin", LC_ALL: "C" } }).split("\n").flatMap(line => {
const row = /^\s*(\d+)\s+(\d+)\s+(.+?)\s*$/u.exec(line);
return row ? [{ pid: Number(row[1]), start: row[3] }] : [];
});
return { live: receipt.journal.filter(process => table.some(row => row.pid === process.pid && row.start === process.start)),
identityChanged: receipt.identityChanged || receipt.journal.some(process => table.some(row => row.pid === process.pid && row.start !== process.start)) };
}
/** Read-only fixture readiness. No credential read, refresh, model discovery,
* setup Test, admission override or measured-agent provisioning occurs here. */
export async function prepareAntigravityJourney(api: ConnectionApi, config: AntigravityJourneyConfig) {
if (process.platform !== "darwin" || process.arch !== "arm64" || new URL(api.origin).hostname !== "127.0.0.1")
throw new ConnectionBlock("blocked_target", "same_host_darwin_arm64_required");
const identity = await verifyConnectionTarget(api.origin, antigravityConnectionConfig(config), api.request);
if (config.candidate.profileDigest !== QUALIFIED_ACPX_PROFILES.antigravity.commandDigest
|| config.candidate.nativeClosureDigest !== `sha256:${ANTIGRAVITY_DISTRIBUTION_PINS["darwin-arm64"].closureSha256}`)
throw new ConnectionBlock("blocked_target", "candidate_pin_mismatch");
const distribution = JSON.parse(await readFile(new URL("../../packages/paperclip-runner/antigravity-distributions.json", import.meta.url), "utf8"));
if (distribution.platforms?.["darwin-arm64"]?.patchVersion !== config.candidate.nativeVersion)
throw new ConnectionBlock("blocked_target", "candidate_native_version_mismatch");
const session = await api.get<Row>("/api/auth/get-session");
const company = await api.get<Row>(`/api/companies/${config.target.companyId}`);
// Environments are instance-scoped. This company-filtered list establishes
// permission; a fabricated environment.companyId would reject valid locals.
const environment = selectAntigravityJourneyEnvironment(await api.get<Row[]>(`/api/companies/${company.id}/environments`), config.target.environmentId);
const connections = await api.get<Row>(`/api/companies/${company.id}/ai-connections`);
const connection = connections.connections.find((row: Row) => row.id === config.savedConnection.id);
if (session.user?.id !== config.savedConnection.ownerUserId || connections.currentUserId !== session.user.id
|| company.id !== config.target.companyId || environment.id !== config.target.environmentId
|| environment.driver !== "local" || environment.status !== "active")
throw new ConnectionBlock("blocked_target", "configured_actor_or_environment_unavailable");
if (!connection || connection.companyId !== company.id || connection.name !== config.savedConnection.name
|| connection.provider !== "google" || connection.method !== "subscription" || connection.ownership !== "personal"
|| connection.ownerUserId !== session.user.id || connection.status !== "connected" || connection.isDefault !== true || !connection.grantId)
throw new ConnectionBlock("missing_credential", "same_user_saved_google_default_required");
const agents = await api.get<Row[]>(`/api/companies/${company.id}/agents`);
if (!agents.some(agent => agent.status !== "terminated")) throw new ConnectionBlock("blocked_target", "additional_agent_requires_existing_roster");
const budgets = await api.get<Row>(`/api/companies/${company.id}/budgets/overview`);
const companyPolicies = budgets.policies.filter((policy: Row) => policy.scopeType === "company" && policy.scopeId === company.id && policy.isActive);
if (company.status !== "active" || companyPolicies.some((policy: Row) => policy.hardStopEnabled && (policy.status === "hard_stop"
|| policy.remainingAmount <= 0 || policy.pendingRunCount > 0 || (policy.unpricedUsagePolicy !== "allow" && policy.unpricedEventCount > 0))))
throw new ConnectionBlock("blocked_target", "company_budget_admission_blocked");
const adapters = await api.get<Row[]>(`/api/adapters?${new URLSearchParams({ companyId: company.id, environmentId: environment.id })}`);
const runner = adapters.find(adapter => adapter.type === "paperclip_runner");
if (!runner?.loaded || runner.disabled || !runner.candidateHarnesses?.some((candidate: Row) => candidate.agent === "antigravity" && candidate.model === ANTIGRAVITY_JOURNEY_MODEL))
throw new ConnectionBlock("blocked_target", "trusted_antigravity_candidate_admission_required");
const accounting = companyPolicies.map((policy: Row) => ({ scopeType: "company", amount: policy.amount, remainingAmount: policy.remainingAmount,
hardStopEnabled: policy.hardStopEnabled, unpricedUsagePolicy: policy.unpricedUsagePolicy, pendingRunCount: policy.pendingRunCount, unpricedEventCount: policy.unpricedEventCount }));
return { company, environment, connection, identity, accounting };
}
/** Recovery accepts only the current attempt's persisted exact identities. */
export function parseAntigravityOwnedResources(input: unknown, config: AntigravityJourneyConfig): AntigravityJourneyResources {
const v = input as Partial<AntigravityJourneyResources> | null;
const uuid = /^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/i;
const allowed = ["version", "executionId", "origin", "companyId", "environmentId", "connectionId", "agentName", "agentId", "agentCreationRequest", "issueId", "issueIdentifier", "runIds", "retained", "setupProbe", "processes", "noLaunch", "sourceProvenance"];
if (!v || typeof v !== "object" || Array.isArray(v) || Object.keys(v).some(key => !allowed.includes(key)) || v.version !== 1 || v.executionId !== ANTIGRAVITY_JOURNEY_EXECUTION
|| v.origin !== config.target.baseURL || v.companyId !== config.target.companyId || v.environmentId !== config.target.environmentId
|| v.connectionId !== config.savedConnection.id || !/^Antigravity QA [a-f0-9]{12}$/.test(v.agentName ?? "")
|| (v.agentId !== undefined && !uuid.test(v.agentId)) || (v.issueId !== undefined && !uuid.test(v.issueId))
|| !Array.isArray(v.runIds) || v.runIds.length > 100 || new Set(v.runIds).size !== v.runIds.length || v.runIds.some(id => typeof id !== "string" || !uuid.test(id)) || typeof v.retained !== "boolean"
|| !["not_started", "pending", "retired", "retirement_unconfirmed"].includes(v.setupProbe ?? "")
|| !v.processes || typeof v.processes !== "object" || Array.isArray(v.processes)
|| Object.entries(v.processes).some(([runId, receipt]) => !uuid.test(runId) || typeof receipt?.captured !== "boolean" || typeof receipt.identityChanged !== "boolean"
|| (receipt.primaryOwner !== undefined && (!validAntigravityPrimaryOwner(receipt.primaryOwner) || receipt.primaryOwner.runId !== runId))
|| !Array.isArray(receipt.journal) || receipt.journal.length > 1000 || receipt.journal.some(process => !Number.isSafeInteger(process.pid)
|| process.pid <= 1 || !Number.isSafeInteger(process.parent) || process.parent < 0 || typeof process.start !== "string" || !Number.isFinite(Date.parse(process.start)))))
throw new ConnectionFailure("invalid_antigravity_owned_resources");
const creation = v.agentCreationRequest;
if (creation !== undefined && (!creation || typeof creation !== "object" || Array.isArray(creation)
|| Object.keys(creation).length !== 6 || Object.keys(creation).some(key => !["setupId", "companyId", "agentName", "provider", "creatorUserId", "origin"].includes(key))
|| typeof creation.setupId !== "string" || !/^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/u.test(creation.setupId) || (v.agentId !== undefined && v.agentId !== creation.setupId)
|| creation.companyId !== v.companyId || creation.agentName !== v.agentName || creation.provider !== "antigravity"
|| creation.creatorUserId !== config.savedConnection.ownerUserId || creation.origin !== v.origin))
throw new ConnectionFailure("invalid_antigravity_owned_resources");
if (v.noLaunch !== undefined && (!v.noLaunch || typeof v.noLaunch !== "object" || Array.isArray(v.noLaunch)
|| Object.entries(v.noLaunch).some(([runId, receipt]) => !v.runIds!.includes(runId) || !receipt || typeof receipt !== "object" || Array.isArray(receipt)
|| Object.keys(receipt).length !== 12 || Object.keys(receipt).some(key => !["kind", "runId", "companyId", "agentId", "issueId", "environmentId", "leaseId", "leaseStatus", "cancelRequestedAt", "startupPreparationSettledAt", "leaseReleasedAt", "observedAt"].includes(key))
|| receipt.kind !== "cancelled-before-native-selection" || receipt.runId !== runId || receipt.companyId !== v.companyId
|| receipt.agentId !== v.agentId || receipt.issueId !== v.issueId || receipt.environmentId !== v.environmentId || !uuid.test(receipt.leaseId)
|| !["released", "expired"].includes(receipt.leaseStatus)
|| [receipt.cancelRequestedAt, receipt.startupPreparationSettledAt, receipt.leaseReleasedAt, receipt.observedAt].some(value => typeof value !== "string" || !Number.isFinite(Date.parse(value)))
|| Date.parse(receipt.cancelRequestedAt) > Date.parse(receipt.leaseReleasedAt) || Date.parse(receipt.leaseReleasedAt) > Date.parse(receipt.startupPreparationSettledAt)
|| Date.parse(receipt.startupPreparationSettledAt) > Date.parse(receipt.observedAt)))) throw new ConnectionFailure("invalid_antigravity_owned_resources");
if (v.sourceProvenance && (Object.keys(v.sourceProvenance).some(key => !["head", "trackedDiffSha256", "untrackedSourceSha256", "untrackedSourceFileCount", "suiteDefinitionHash", "dirty"].includes(key))
|| !/^[a-f0-9]{40}$/.test(v.sourceProvenance.head) || [v.sourceProvenance.trackedDiffSha256, v.sourceProvenance.untrackedSourceSha256, v.sourceProvenance.suiteDefinitionHash].some(value => !/^[a-f0-9]{64}$/.test(value))
|| !Number.isSafeInteger(v.sourceProvenance.untrackedSourceFileCount) || v.sourceProvenance.untrackedSourceFileCount < 0 || typeof v.sourceProvenance.dirty !== "boolean"))
throw new ConnectionFailure("invalid_antigravity_owned_resources");
return v as AntigravityJourneyResources;
}
export async function cleanupAntigravityJourney(input: {
api: ConnectionApi; config: AntigravityJourneyConfig; resources: AntigravityJourneyResources;
cleanup: AntigravityJourneyEvidence["cleanup"]; persistResources(): Promise<void>;
/** Save public usage while these exact owned rows still exist. No launch or admission checks. */
captureRun(run: Row): Promise<void>;
observers?: AntigravityProcessObservers;
}) {
const { api, config, resources, cleanup } = input;
cleanup.connectionPreserved = false;
let pendingDraft: Row | undefined;
if (!resources.agentId && resources.agentCreationRequest) {
cleanup.agent = false;
const creation = resources.agentCreationRequest;
if (!/^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/u.test(creation.setupId)
|| creation.companyId !== resources.companyId || creation.agentName !== resources.agentName || creation.origin !== resources.origin
|| creation.creatorUserId !== config.savedConnection.ownerUserId || creation.provider !== "antigravity")
throw new ConnectionFailure("cleanup_agent_owner_mismatch");
let draft: Row;
try { draft = await api.get<Row>(`/api/agents/${creation.setupId}?companyId=${resources.companyId}`); }
catch (error) {
if (error instanceof ConnectionFailure && error.message === "api_status_404")
throw new ConnectionFailure("cleanup_agent_creation_unconfirmed");
throw error;
}
if (!draft || draft.id !== creation.setupId || draft.companyId !== resources.companyId || draft.name !== resources.agentName
|| draft.createdByUserId !== creation.creatorUserId)
throw new ConnectionFailure("cleanup_agent_owner_mismatch");
pendingDraft = draft;
}
if (resources.agentId) cleanup.agent = false;
if (resources.runIds.length) cleanup.runs = false;
const agents = await api.get<Row[]>(`/api/companies/${resources.companyId}/agents`);
const named = agents.filter(agent => agent.name === resources.agentName);
if (pendingDraft) {
if (named.length !== 1 || named[0]!.id !== pendingDraft.id || named[0]!.companyId !== resources.companyId)
throw new ConnectionFailure("cleanup_agent_owner_mismatch");
resources.agentId = pendingDraft.id;
await input.persistResources();
}
if (!resources.agentId && named.length) throw new ConnectionFailure("cleanup_requires_agent_creation_receipt");
const agent = agents.find(agent => agent.id === resources.agentId);
if (agent && (agent.companyId !== resources.companyId || agent.name !== resources.agentName || named.length !== 1))
throw new ConnectionFailure("cleanup_agent_owner_mismatch");
if (agent && agents.some(row => row.reportsTo === agent.id)) throw new ConnectionFailure("cleanup_agent_has_unowned_reports");
if (agent) {
cleanup.agent = false; cleanup.runs = false;
const requestedStops = new Map<string, string | null>();
const assertRunOwner = (run: Row, runId: string) => {
if (run.id !== runId || run.companyId !== resources.companyId || run.agentId !== agent.id
|| (resources.issueId && run.issueId !== resources.issueId))
throw new ConnectionFailure("cleanup_run_owner_mismatch");
};
const nativeStop = (run: Row): { requestId?: string; intent?: Row } | undefined => {
if (run.runtimeMode !== "native") return;
const requestId = run.resultJson?.startupCancellation?.cancellationRequestId;
const intent = run.resultJson?.nativeCancellation;
if (requestId == null && intent == null) return;
if (typeof run.issueId !== "string" || !run.issueId || (resources.issueId && run.issueId !== resources.issueId)
|| (requestId != null && (typeof requestId !== "string" || !/^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/u.test(requestId))))
throw new ConnectionFailure("cleanup_native_stop_identity_mismatch");
if (intent != null && (intent.schema !== "paperclip.native-cancellation.v1" || typeof intent.intentId !== "string" || !intent.intentId
|| intent.runId !== run.id || intent.companyId !== resources.companyId || intent.issueId !== run.issueId
|| intent.scope !== "run" || intent.reasonCode !== "cancellation_run_only" || typeof intent.dispatched !== "boolean"
|| !["pending", "acknowledged"].includes(intent.dispatchState) || typeof intent.intentAuditId !== "string" || !intent.intentAuditId
|| (requestId != null && intent.intentId !== `native-cancellation:${requestId}`)
|| (intent.dispatchState === "acknowledged" && (typeof intent.acknowledgementAuditId !== "string" || !intent.acknowledgementAuditId
|| intent.acknowledgementAuditId === intent.intentAuditId))))
throw new ConnectionFailure("cleanup_native_stop_identity_mismatch");
return { ...(requestId == null ? {} : { requestId }), ...(intent == null ? {} : { intent }) };
};
const stopOwnedRun = async (run: Row) => {
const stop = nativeStop(run);
const retryOwner = run.runtimeMode === "native" && run.status === "failed" && run.nativePhase === "retryable_failure";
// A prior Stop advances the coordinator before its acknowledgement. Its
// public correlation is durable; recovery must replay the same request.
const resumeStop = run.runtimeMode === "native" && run.status === "failed" && stop?.intent != null;
// A UUID reservation alone is not retry authority. A terminal row with
// no durable native intent remains unconfirmed instead of issuing Stop.
if (!["queued", "running", "scheduled_retry"].includes(run.status) && !retryOwner && !resumeStop) return;
if (resources.issueId && run.issueId !== resources.issueId) throw new ConnectionFailure("cleanup_run_owner_mismatch");
if (requestedStops.has(run.id)) return;
if (run.runtimeMode === "native") {
if (typeof run.issueId !== "string" || !run.issueId || (stop?.intent && !stop.requestId))
throw new ConnectionFailure("cleanup_native_stop_identity_mismatch");
const requestId = stop?.requestId ?? randomUUID();
await api.post(`/api/heartbeat-runs/${run.id}/cancel`, { cancellationRequestId: requestId });
requestedStops.set(run.id, requestId);
} else {
await api.post(`/api/heartbeat-runs/${run.id}/cancel`);
requestedStops.set(run.id, null);
}
};
const nativeStopAcknowledged = (run: Row) => {
const stop = nativeStop(run), requested = requestedStops.get(run.id);
if (requested == null && !stop) return true;
return run.status === "cancelled" && stop?.intent?.dispatchState === "acknowledged"
&& (requested == null || stop.requestId === requested);
};
await api.patch(`/api/agents/${agent.id}`, { status: "paused" });
const runs = await api.get<Row[]>(`/api/companies/${resources.companyId}/heartbeat-runs?agentId=${agent.id}&limit=100`);
for (const run of runs) {
if (run.agentId !== agent.id || run.companyId !== resources.companyId) throw new ConnectionFailure("cleanup_run_owner_mismatch");
if (!resources.runIds.includes(run.id)) resources.runIds.push(run.id);
const observed = await api.get<Row>(`/api/heartbeat-runs/${run.id}`);
assertRunOwner(observed, run.id);
await input.captureRun(observed);
observeAntigravityRun(resources, observed, input.observers ?? new Map());
await input.persistResources();
await stopOwnedRun(observed);
}
await input.persistResources();
const deadline = Date.now() + 60_000;
while (true) {
const latest = await api.get<Row[]>(`/api/companies/${resources.companyId}/heartbeat-runs?agentId=${agent.id}&limit=100`);
for (const run of latest) {
if (run.companyId !== resources.companyId || run.agentId !== agent.id) throw new ConnectionFailure("cleanup_run_owner_mismatch");
if (resources.runIds.includes(run.id)) continue;
resources.runIds.push(run.id);
const observed = await api.get<Row>(`/api/heartbeat-runs/${run.id}`);
assertRunOwner(observed, run.id);
await input.captureRun(observed);
observeAntigravityRun(resources, observed, input.observers ?? new Map());
await input.persistResources();
await stopOwnedRun(observed);
}
let retired = !latest.some(run => run.agentId === agent.id && ["queued", "running", "scheduled_retry"].includes(run.status));
for (const runId of resources.runIds) {
const run = await api.get<Row>(`/api/heartbeat-runs/${runId}`);
assertRunOwner(run, runId);
await input.captureRun(run);
await stopOwnedRun(run);
if (!nativeStopAcknowledged(run)) { retired = false; continue; }
if (!isAntigravityRunAccountingSettled(run)) retired = false;
const receipt = resources.processes[runId];
if ((!receipt?.captured || receipt.journal.length < 2) && run.resultJson?.startupCancellation?.beforeNativeSelection === true) {
const cancellation = run.resultJson.startupCancellation, settledAt = run.resultJson.startupPreparationSettledAt;
const environment = run.contextSnapshot?.paperclipEnvironment, profile = run.runnerProfileJson;
const instant = (value: unknown) => typeof value === "string" && Number.isFinite(Date.parse(value));
// Cancellation may return while its original preparation owner is
// still settling. Only missing progress receipts may wait; ownership
// contradictions and malformed evidence must not become a timeout.
if (run.issueId !== resources.issueId || run.status !== "cancelled" || run.runtimeMode !== "legacy" || run.runtimeModeResolvedAt !== null
|| !["preparing", "settled"].includes(run.executionStage) || typeof run.costAccountingPending !== "boolean"
|| cancellation.errorCode !== "cancelled" || !instant(cancellation.requestedAt) || (settledAt != null && !instant(settledAt))
|| (run.controllerLeaseExpiresAt !== null && !instant(run.controllerLeaseExpiresAt))
|| ["processPid", "processGroupId", "processStartedAt", "runnerInstanceId", "nativeSessionId", "driverKind", "completionContractId"].some(key => run[key] !== null)
|| !profile || Object.keys(profile).length !== 1 || !profile.adapterDispatch || Object.keys(profile.adapterDispatch).length !== 1
|| profile.adapterDispatch.adapterType !== "paperclip_runner" || receipt?.captured || receipt?.identityChanged || (receipt?.journal.length ?? 0) > 0
|| environment?.id !== resources.environmentId || environment.driver !== "local") throw new ConnectionFailure("cleanup_no_launch_receipt_unconfirmed");
const leaseId = environment.leaseId;
if (typeof leaseId !== "string") throw new ConnectionFailure("cleanup_lease_identity_missing");
const lease = await api.get<Row>(`/api/environment-leases/${leaseId}`);
if (lease.id !== leaseId || lease.companyId !== resources.companyId || lease.heartbeatRunId !== runId || lease.environmentId !== resources.environmentId)
throw new ConnectionFailure("cleanup_lease_owner_mismatch");
if (!["active", "released", "expired", "failed", "retained", "pending_cleanup"].includes(lease.status)
|| (lease.releasedAt != null && !instant(lease.releasedAt))
|| (settledAt != null && Date.parse(settledAt) < Date.parse(cancellation.requestedAt))
|| (lease.releasedAt != null && (Date.parse(lease.releasedAt) < Date.parse(cancellation.requestedAt)
|| (settledAt != null && Date.parse(lease.releasedAt) > Date.parse(settledAt))))) throw new ConnectionFailure("cleanup_no_launch_receipt_unconfirmed");
if (run.executionStage !== "settled" || settledAt == null || run.costAccountingPending || run.controllerLeaseExpiresAt !== null
|| !["released", "expired"].includes(lease.status) || lease.releasedAt == null) { retired = false; continue; }
const noLaunch = antigravityNoLaunchReceipt(run, lease, resources);
(resources.noLaunch ??= {})[runId] = noLaunch;
await input.persistResources();
continue;
}
if (!receipt?.captured || receipt.journal.length < 2) throw new ConnectionFailure("cleanup_physical_owner_not_captured");
const observed = remainingRecordedProcesses(receipt);
if (observed.identityChanged) throw new ConnectionFailure("cleanup_physical_identity_changed");
if (observed.live.length) { retired = false; continue; }
const leaseId = run.contextSnapshot?.paperclipEnvironment?.leaseId;
if (typeof leaseId !== "string") throw new ConnectionFailure("cleanup_lease_identity_missing");
const lease = await api.get<Row>(`/api/environment-leases/${leaseId}`);
if (lease.companyId !== resources.companyId || lease.heartbeatRunId !== runId || lease.environmentId !== resources.environmentId)
throw new ConnectionFailure("cleanup_lease_owner_mismatch");
if (lease.status !== "released") { retired = false; continue; }
try { await reacquireAntigravityNativeFences(run.runnerProfileJson?.sessionCheckpoint?.providerIdentity?.providerLifetimeFenceCandidates); }
catch { throw new ConnectionFailure("cleanup_native_fences_unconfirmed"); }
}
if (retired) break;
if (Date.now() > deadline) throw new ConnectionFailure("cleanup_run_retirement_unconfirmed");
await connectionDelay(250);
}
cleanup.runs = true;
if (resources.issueId) {
const issue = await api.get<Row>(`/api/issues/${resources.issueId}`);
if (issue.companyId !== resources.companyId || issue.assigneeAgentId !== agent.id) throw new ConnectionFailure("cleanup_issue_owner_mismatch");
if (!["done", "cancelled"].includes(issue.status)) await api.patch(`/api/issues/${issue.id}`, { status: "cancelled" });
}
if (["pending", "retirement_unconfirmed"].includes(resources.setupProbe)) throw new ConnectionFailure("setup_probe_retirement_unconfirmed");
if (!config.retainAgent) {
// Purge is allowed only after the ordinary lifecycle owner completes termination.
await api.post(`/api/agents/${agent.id}/terminate`);
const terminateDeadline = Date.now() + 30_000;
while (true) {
const current = await api.get<Row>(`/api/agents/${agent.id}`);
if (current.companyId !== resources.companyId || current.name !== resources.agentName) throw new ConnectionFailure("cleanup_agent_owner_mismatch");
if (["terminated", "rejected"].includes(current.lifecycleState)) break;
if (Date.now() >= terminateDeadline) throw new ConnectionFailure("cleanup_agent_termination_unconfirmed");
await connectionDelay(250);
}
await api.delete(`/api/agents/${agent.id}`);
} else {
const retained = await api.get<Row>(`/api/agents/${agent.id}`);
if (retained.id !== agent.id || retained.companyId !== resources.companyId || retained.name !== resources.agentName || retained.status !== "paused")
throw new ConnectionFailure("cleanup_retained_agent_pause_unconfirmed");
}
resources.retained = config.retainAgent;
cleanup.agent = true;
} else {
if (resources.runIds.length) throw new ConnectionFailure("cleanup_agent_removed_before_retirement_verification");
cleanup.agent = true; cleanup.runs = true;
}
if (["pending", "retirement_unconfirmed"].includes(resources.setupProbe)) throw new ConnectionFailure("setup_probe_retirement_unconfirmed");
const accounts = await api.get<Row>(`/api/companies/${resources.companyId}/ai-connections`);
const connection = accounts.connections.find((row: Row) => row.id === config.savedConnection.id);
cleanup.connectionPreserved = Boolean(connection && connection.provider === "google" && connection.method === "subscription"
&& connection.ownerUserId === config.savedConnection.ownerUserId && connection.isDefault === true && connection.status === "connected");
if (!cleanup.connectionPreserved) throw new ConnectionFailure("existing_connection_not_preserved");
await input.persistResources();
}
tests/runner-e2e/antigravity-journey-flow.ts
+241
View File
@@ -0,0 +1,241 @@
import { randomUUID } from "node:crypto";
import { expect, type Page, type Request, type Route } from "@playwright/test";
import { ConnectionFailure } from "./connection-config.js";
import { ConnectionApi } from "./connection-target.js";
import { ANTIGRAVITY_JOURNEY_MODEL, type AntigravityJourneyConfig } from "./antigravity-journey-config.js";
import { verifyAntigravityJourneyTurn, type AntigravityJourneyEvidence } from "./antigravity-journey-evidence.js";
import type { AntigravityJourneyResources, prepareAntigravityJourney } from "./antigravity-journey-fixtures.js";
import { openAntigravityCreatedTask, assertAntigravityCleanComposer, assertAntigravityFileActivity, assertAntigravityReady, assertAntigravityTaskDone, inspectAntigravityArtifact, openAntigravityAgentFromRoster, selectAntigravityFromOrdinaryPicker } from "./antigravity-journey-ui.js";
import { submitTaskReply } from "./user-actions.js";
type Row = Record<string, any>;
export interface AntigravityJourneyFlowInput {
page: Page; api: ConnectionApi; config: AntigravityJourneyConfig; prepared: Awaited<ReturnType<typeof prepareAntigravityJourney>>;
resources: AntigravityJourneyResources; evidence: AntigravityJourneyEvidence;
checkpoint(id: keyof AntigravityJourneyEvidence["checkpoints"], page: Page): Promise<void>;
phase(id: string): Promise<void>;
persistResources(): Promise<void>;
observeRun(run: Row): Promise<void>;
beforeAdmission(): Promise<void>;
assertActive(): void;
}
export async function captureAntigravityAgentDraft(page: Page, resources: AntigravityJourneyResources,
persistResources: () => Promise<void>, submit: () => Promise<void>, expectedCreatorUserId: string): Promise<Row> {
const matches = (url: URL) => /^\/api\/companies\/[^/]+\/agents$/u.test(url.pathname);
let captured: Request | undefined;
let rejectRoute!: (error: unknown) => void;
const routeFailure = new Promise<never>((_resolve, reject) => { rejectRoute = reject; });
const handler = async (route: Route) => {
const request = route.request();
try {
if (request.method() !== "POST") { await route.continue(); return; }
const url = new URL(request.url());
let body: Row;
try { body = request.postDataJSON(); } catch { throw new ConnectionFailure("agent_creation_request_mismatch"); }
if (captured || url.origin !== resources.origin || url.pathname !== `/api/companies/${resources.companyId}/agents`
|| !body || typeof body !== "object" || Array.isArray(body) || body.name !== resources.agentName
|| typeof body.setupId !== "string" || !/^[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}$/u.test(body.setupId)
|| typeof expectedCreatorUserId !== "string" || !expectedCreatorUserId)
throw new ConnectionFailure("agent_creation_request_mismatch");
const witness = { setupId: body.setupId, companyId: resources.companyId, agentName: resources.agentName,
provider: "antigravity" as const, creatorUserId: expectedCreatorUserId, origin: resources.origin };
if ((resources.agentId && resources.agentId !== witness.setupId) || (resources.agentCreationRequest
&& Object.entries(witness).some(([key, value]) => resources.agentCreationRequest![key as keyof typeof witness] !== value)))
throw new ConnectionFailure("agent_creation_request_mismatch");
captured = request;
resources.agentCreationRequest = witness;
resources.retained = true;
await persistResources();
// Continue the ordinary UI request unchanged only after its recovery identity is durable.
await route.continue();
} catch (error) {
await route.abort("failed").catch(() => {});
rejectRoute(error);
}
};
let primary: unknown;
await page.route(matches, handler);
try {
const creation = page.waitForResponse(response => response.request() === captured);
const receipt = Promise.race([creation, routeFailure]);
// A submit failure must not leave a later response timeout unhandled.
void receipt.catch(() => {});
await Promise.race([submit(), routeFailure]);
const created = await receipt;
if (!created.ok()) throw new ConnectionFailure("agent_draft_creation_failed");
const draft = await created.json();
if (!draft || typeof draft !== "object" || Array.isArray(draft) || !resources.agentCreationRequest || draft.id !== resources.agentCreationRequest.setupId
|| draft.companyId !== resources.companyId || draft.name !== resources.agentName || draft.createdByUserId !== expectedCreatorUserId)
throw new ConnectionFailure("agent_creation_receipt_mismatch");
return draft;
} catch (error) { primary = error; throw error; }
finally {
try { await page.unroute(matches, handler); }
catch (error) {
if (!primary && !page.isClosed()) throw error;
if (primary instanceof Error) Object.assign(primary, { agentCreationRouteCleanupFailed: true });
}
}
}
export async function runAntigravityJourneyFlow(input: AntigravityJourneyFlowInput) {
const { page, api, config, prepared, resources, evidence } = input;
const { company, connection, environment } = prepared;
const marker = `ANTIGRAVITY-INSTRUCTION-${randomUUID()}`;
const followupMarker = `ANTIGRAVITY-FOLLOWUP-${randomUUID()}`;
const firstBytes = Buffer.from(`${marker}\n`), followupBytes = Buffer.from(`${marker}\n${followupMarker}\n`);
const checkpoint = async (id: keyof AntigravityJourneyEvidence["checkpoints"]) => { input.assertActive(); await input.checkpoint(id, page); };
if (!/^[A-Za-z0-9]+$/.test(company.issuePrefix)) throw new ConnectionFailure("invalid_journey_company_prefix");
const dashboard = `/${company.issuePrefix}/dashboard`;
await input.phase("agent_navigation");
await page.goto(`${api.origin}${dashboard}`);
await page.waitForURL(url => url.origin === api.origin && url.pathname === dashboard);
await page.getByRole("link", { name: "Agents", exact: true }).click();
const newAgent = page.getByRole("toolbar", { name: "Agents controls", exact: true }).getByRole("button", { name: "New Agent", exact: true });
await expect(newAgent).toHaveCount(1);
await newAgent.click();
await input.phase("agent_creation");
const nameDialog = page.getByRole("dialog", { name: "Meet your next agent", exact: true });
await expect(nameDialog.getByRole("button", { name: "Choose adapter", exact: true })).toBeDisabled();
await nameDialog.getByRole("textbox", { name: "Agent name", exact: true }).fill(resources.agentName);
const draft = await captureAntigravityAgentDraft(page, resources, input.persistResources,
() => nameDialog.getByRole("button", { name: "Choose adapter", exact: true }).click(), config.savedConnection.ownerUserId);
resources.agentId = draft.id; evidence.agentId = draft.id; evidence.cleanup.agent = false;
await input.persistResources();
await input.phase("picker_selection");
await selectAntigravityFromOrdinaryPicker(page);
await checkpoint("ordinary_picker");
await page.getByRole("dialog", { name: "Choose an adapter", exact: true }).getByRole("button", { name: "Configure agent", exact: true }).click();
await input.phase("saved_subscription");
const saved = page.getByRole("combobox", { name: "Saved subscription", exact: true });
await expect(saved).toBeVisible({ timeout: 60_000 });
await saved.selectOption(`ai:${connection.grantId}`);
await expect(saved).toHaveValue(`ai:${connection.grantId}`);
await expect(saved.locator("option:checked")).toHaveText(`${connection.name} (Your default)`);
await page.getByRole("button", { name: "Use saved subscription", exact: true }).click();
await expect(page.getByRole("heading", { name: "Configure your agent", exact: true })).toBeVisible({ timeout: 120_000 });
await checkpoint("saved_subscription");
await input.phase("model_selection");
const model = page.getByRole("combobox", { name: "Model", exact: true });
await expect(model).not.toHaveAttribute("aria-busy", "true", { timeout: 60_000 });
// Select a catalog option. A custom model shortcut would miss exact account discovery.
await expect(model.locator(`option[value="${ANTIGRAVITY_JOURNEY_MODEL}"]`)).toHaveCount(1);
await model.selectOption(ANTIGRAVITY_JOURNEY_MODEL);
await expect(model).toHaveValue(ANTIGRAVITY_JOURNEY_MODEL);
const environmentSelect = page.getByRole("combobox", { name: "Environment", exact: true });
const selectedEnvironment = await environmentSelect.inputValue();
// Empty means the instance default; do not silently replace the selected environment.
if (selectedEnvironment && selectedEnvironment !== environment.id) throw new ConnectionFailure("ui_environment_mismatch");
await checkpoint("exact_model");
await input.phase("setup_probe");
const beforeProbe = await api.get<Row[]>(`/api/companies/${company.id}/heartbeat-runs?agentId=${draft.id}&limit=100`);
if (beforeProbe.length) throw new ConnectionFailure("new_agent_already_has_runs");
const probeResponse = page.waitForResponse(response => response.request().method() === "POST"
&& new URL(response.url()).pathname === `/api/companies/${company.id}/adapters/paperclip_runner/test-environment`, { timeout: 120_000 });
resources.setupProbe = "pending";
await input.persistResources();
await page.getByRole("button", { name: "Run test", exact: true }).click();
const probe = await probeResponse, result = await probe.json();
if (!probe.ok() || result.status !== "pass" || !result.checks?.some((check: Row) => check.code === "antigravity_metadata_verified" && check.level === "info")
|| (await api.get<Row[]>(`/api/companies/${company.id}/heartbeat-runs?agentId=${draft.id}&limit=100`)).length)
throw new ConnectionFailure("prompt_free_native_setup_test_failed");
resources.setupProbe = "retired";
await input.persistResources();
await expect(page.getByText("Connection successful", { exact: true })).toBeVisible();
await checkpoint("setup_probe");
await input.phase("readiness");
await page.getByRole("button", { name: "Finish setup", exact: true }).click();
await assertAntigravityReady(page, resources.agentName);
const ready = await api.get<Row>(`/api/agents/${draft.id}`);
if (ready.companyId !== company.id || ready.name !== resources.agentName || ready.status !== "idle" || ready.lifecycleState !== "ready"
|| ready.adapterType !== "paperclip_runner" || ready.adapterConfig?.provider !== "acpx" || ready.adapterConfig?.acpxAgent !== "antigravity"
|| ready.adapterConfig?.model !== ANTIGRAVITY_JOURNEY_MODEL || ready.runtimeConfig?.aiConnection?.mode !== "responsible_user"
|| ready.runtimeConfig.aiConnection.provider !== "google" || ready.runtimeConfig.aiConnection.method !== "subscription")
throw new ConnectionFailure("saved_ready_agent_binding_mismatch");
const effectiveEnvironment = ready.defaultEnvironmentId ?? (await api.get<Row>("/api/instance/settings")).defaultEnvironmentId;
if (effectiveEnvironment && effectiveEnvironment !== environment.id) throw new ConnectionFailure("saved_agent_environment_mismatch");
// Preserve the operator's inherited accounting policy, as in the source journey.
await checkpoint("agent_ready");
await input.phase("first_composer");
await page.getByRole("button", { name: `Assign ${resources.agentName} a Task`, exact: true }).click();
await assertAntigravityCleanComposer(page, resources.agentName, ANTIGRAVITY_JOURNEY_MODEL);
await checkpoint("clean_composer");
await input.phase("close_empty_composer");
await page.keyboard.press("Escape");
await expect(page.getByRole("dialog", { name: "New task", exact: true })).toBeHidden();
await input.phase("agent_navigation");
await openAntigravityAgentFromRoster(page, resources.agentName);
await input.phase("instructions");
await page.getByRole("navigation", { name: `${resources.agentName} navigation`, exact: true })
.getByRole("link", { name: "Instructions", exact: true }).click();
await page.getByRole("group", { name: "Instruction file view", exact: true }).getByRole("button", { name: "edit", exact: true }).click();
const editor = page.locator('[contenteditable="true"]:visible,textarea[aria-label="Instruction file editor"]:visible');
await expect(editor).toHaveCount(1);
const current = await editor.evaluate(element => element instanceof HTMLTextAreaElement ? element.value : (element as HTMLElement).innerText);
if (!current.includes("You are an agent in a Paperclip company.")) throw new ConnectionFailure("default_instructions_missing");
const addition = `\nFor file verification tasks, your configured verification marker is:\n${marker}\n\nWhen a task asks for your configured verification marker, write exactly that marker as the first line of the requested proof file followed by one newline. Read the actual file back and attach it as the task result.`;
// Follow the existing instruction-persistence journey's keyboard path. The
// rich editor distinguishes real interaction from initial normalization.
await editor.click();
await editor.press("ControlOrMeta+End");
for (const line of addition.split("\n")) {
await editor.press("Enter");
if (line) await editor.pressSequentially(line);
}
const save = page.getByRole("button", { name: "Save changes", exact: true });
await expect(save).toBeEnabled();
await save.click();
await expect(save).toBeDisabled({ timeout: 30_000 });
const storedInstructions = await api.get<Row>(`/api/agents/${draft.id}/instructions-bundle/file?path=AGENTS.md&companyId=${company.id}`);
if (!storedInstructions.content?.includes(marker) || !storedInstructions.content.includes(current.trim())) throw new ConnectionFailure("instruction_ui_save_not_durable");
await checkpoint("instructions_saved");
await input.phase("task_creation");
await page.getByRole("button", { name: "Assign Task", exact: true }).click();
await assertAntigravityCleanComposer(page, resources.agentName, ANTIGRAVITY_JOURNEY_MODEL);
const prompt = "Create a proof file from your configured instruction\n\nCreate antigravity-proof.txt containing exactly your configured verification marker as its only line, followed by one newline. Read back the actual file, attach it as the task result, report its byte count, and complete this task. I will follow up to ask you to update this same file.";
if (prompt.includes(marker)) throw new ConnectionFailure("instruction_nonce_leaked_to_task_prompt");
const dialog = page.getByRole("dialog", { name: "New task", exact: true });
await dialog.getByRole("textbox", { name: "editable markdown", exact: true }).fill(prompt);
await input.beforeAdmission();
const issueResponse = page.waitForResponse(response => response.request().method() === "POST" && new URL(response.url()).pathname === `/api/companies/${company.id}/issues`);
await dialog.getByRole("button", { name: "Create task", exact: true }).click();
const response = await issueResponse;
if (response.status() !== 201) throw new ConnectionFailure("task_creation_failed");
const issue = await response.json();
if (issue.companyId !== company.id || issue.assigneeAgentId !== draft.id || typeof issue.id !== "string" || issue.description?.includes(marker)) throw new ConnectionFailure("task_creation_binding_mismatch");
resources.issueId = issue.id; resources.issueIdentifier = issue.identifier;
evidence.cleanup.runs = false;
await input.persistResources();
await expect(dialog).toBeHidden();
// Follow the persistent created-task link, as the walkthrough does.
await openAntigravityCreatedTask(page, issue);
const verifyTurn = async (expectedBytes: Buffer, afterRunIds: readonly string[]) => verifyAntigravityJourneyTurn({ api, companyId: company.id,
agentId: draft.id, issueId: issue.id, connectionId: connection.id, environmentId: environment.id, model: ANTIGRAVITY_JOURNEY_MODEL,
expectedProfileDigest: config.candidate.profileDigest, expectedBytes, afterRunIds, deadlineAt: Date.now() + config.limits.turnTimeoutMs, observeRun: input.observeRun, assertActive: input.assertActive });
await input.phase("first_turn");
const first = await verifyTurn(firstBytes, []);
evidence.turnEvidence.push(first);
await checkpoint("first_run");
await assertAntigravityTaskDone(page);
await assertAntigravityFileActivity(page, first.runId);
await inspectAntigravityArtifact(page, first.artifact.attachmentId, firstBytes);
await checkpoint("first_artifact");
await page.reload();
await assertAntigravityTaskDone(page);
await inspectAntigravityArtifact(page, first.artifact.attachmentId, firstBytes);
await input.phase("followup");
await input.beforeAdmission();
await submitTaskReply(page, `Read your existing antigravity-proof.txt. Preserve its first line exactly and append this second line followed by one newline:\n${followupMarker}\nRead back the actual updated file, attach it as the task result, report its byte count, and complete this task. Do not recreate the original first line from memory or from a task message.`);
const second = await verifyTurn(followupBytes, [first.runId]);
evidence.turnEvidence.push(second);
await checkpoint("followup_run");
await assertAntigravityTaskDone(page);
await assertAntigravityFileActivity(page, second.runId);
await inspectAntigravityArtifact(page, second.artifact.attachmentId, followupBytes);
await checkpoint("followup_artifact");
await page.reload();
await assertAntigravityTaskDone(page);
await inspectAntigravityArtifact(page, second.artifact.attachmentId, followupBytes);
await checkpoint("persisted");
}
tests/runner-e2e/antigravity-journey-launch.test.ts
+370
View File
@@ -0,0 +1,370 @@
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { mkdtemp, readFile, readdir, rm, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { spawnSync } from "node:child_process";
import { runnerMatrix } from "./catalog.js";
import { ANTIGRAVITY_JOURNEY_EXECUTION } from "./antigravity-journey-cases.js";
import { parseAntigravityJourneyConfig } from "./antigravity-journey-config.js";
import { createAntigravityResourceWriter, runAntigravityJourneyCampaign } from "./antigravity-journey-launch.js";
import { parseAntigravityOwnedResources, type AntigravityJourneyResources } from "./antigravity-journey-fixtures.js";
import { buildMatrixJobs, parseRunnerSelectors, selectRunnerExecutions } from "./selectors.js";
import { ConnectionFailure } from "./connection-config.js";
const scenario = vi.hoisted(() => ({ config: undefined as any, authReads: 0, prepare: vi.fn(), flow: vi.fn(), close: vi.fn(), cleanup: vi.fn(), reportOutput: "", failBranding: false }));
const journalFs = vi.hoisted(() => ({ directory: "", fault: "", events: [] as string[], snapshots: [] as unknown[],
persistentFault: false, captureWriterDirectory: false, diagnosticFailure: "",
gate: undefined as { entered(): void; wait: Promise<void>; used: boolean } | undefined }));
vi.mock("node:fs/promises", async original => {
const actual = await original<typeof import("node:fs/promises")>();
const owned = (file: unknown) => typeof file === "string" && journalFs.directory !== "" && file.startsWith(`${journalFs.directory}/`)
&& (file === `${journalFs.directory}/owned-resources.json` || file.startsWith(`${journalFs.directory}/.owned-resources-`));
const fail = (stage: string) => {
if (journalFs.fault !== stage) return false;
if (!journalFs.persistentFault) journalFs.fault = "";
return true;
};
const recordWrite = async (data: unknown) => {
journalFs.events.push("write");
journalFs.snapshots.push(JSON.parse(String(data)));
if (journalFs.gate && !journalFs.gate.used) {
journalFs.gate.used = true;
journalFs.gate.entered();
await journalFs.gate.wait;
}
};
return { ...actual,
writeFile: async (...args: Parameters<typeof actual.writeFile>) => {
if (journalFs.diagnosticFailure && args[0] === `${journalFs.directory}/${journalFs.diagnosticFailure}`)
throw new Error("fixture diagnostic write failed");
if (owned(args[0])) {
await recordWrite(args[1]);
if (fail("write")) {
await actual.writeFile(args[0], '{"interrupted":', args[2]);
throw new Error("fixture journal write failed");
}
}
return actual.writeFile(...args);
},
open: async (...args: Parameters<typeof actual.open>) => {
if (journalFs.captureWriterDirectory && typeof args[0] === "string" && /\/\.owned-resources-[a-f0-9]+\.tmp$/u.test(args[0]))
journalFs.directory = args[0].slice(0, args[0].lastIndexOf("/"));
const handle = await actual.open(...args);
const directory = args[0] === journalFs.directory;
if (!directory && !owned(args[0])) return handle;
journalFs.events.push(directory ? "directory-open" : "file-open");
return new Proxy(handle, { get(target, key) {
if (key === "writeFile") return async (...writeArgs: Parameters<typeof handle.writeFile>) => {
await recordWrite(writeArgs[0]);
if (fail("write")) {
await target.writeFile('{"interrupted":');
throw new Error("fixture journal write failed");
}
return target.writeFile(...writeArgs);
};
if (key === "sync") return async () => {
const stage = directory ? "directory-sync" : "file-sync";
journalFs.events.push(stage);
if (fail(stage)) throw new Error(`fixture journal ${stage} failed`);
return target.sync();
};
const value = Reflect.get(target, key, target);
return typeof value === "function" ? value.bind(target) : value;
} });
},
rename: async (...args: Parameters<typeof actual.rename>) => {
if (owned(args[1])) {
journalFs.events.push("rename");
if (fail("rename")) throw new Error("fixture journal rename failed");
}
return actual.rename(...args);
},
};
});
vi.mock("./antigravity-journey-config.js", async original => ({ ...await original<typeof import("./antigravity-journey-config.js")>(),
loadAntigravityJourneyConfig: async () => scenario.config }));
vi.mock("./antigravity-journey-fixtures.js", async original => ({ ...await original<typeof import("./antigravity-journey-fixtures.js")>(),
prepareAntigravityJourney: (...args: unknown[]) => scenario.prepare(...args),
cleanupAntigravityJourney: (...args: unknown[]) => scenario.cleanup(...args) }));
vi.mock("./antigravity-journey-flow.js", () => ({ runAntigravityJourneyFlow: (...args: unknown[]) => scenario.flow(...args) }));
vi.mock("./connection-launch.js", () => ({
withConnectionCampaignCancellation: (run: (signal: AbortSignal) => Promise<unknown>) => run(new AbortController().signal),
openConnectionBrowser: async () => ({ close: scenario.close, context: {
pages: () => [{ goto: async () => {}, isClosed: () => true }],
request: { fetch: async () => {
const status = ++scenario.authReads === 1 ? 401 : 200;
return { ok: () => status === 200, status: () => status, json: async () => ({}) };
} },
} }),
}));
vi.mock("./connection-target.js", async original => ({ ...await original<typeof import("./connection-target.js")>(), connectionDelay: async () => {} }));
vi.mock("./antigravity-journey-report.js", async original => {
const actual = await original<typeof import("./antigravity-journey-report.js")>();
return { ...actual, writeAntigravityJourneyReport: async (input: Parameters<typeof actual.writeAntigravityJourneyReport>[0]) => {
scenario.reportOutput = input.output;
return actual.writeAntigravityJourneyReport(input);
} };
});
vi.mock("./report-assets.js", async original => {
const actual = await original<typeof import("./report-assets.js")>();
return { ...actual, stageDashboardBrandAssets: async (...args: Parameters<typeof actual.stageDashboardBrandAssets>) => {
if (scenario.failBranding) throw new Error("fixture late branding failure");
return actual.stageDashboardBrandAssets(...args);
} };
});
const repositoryRoot = fileURLToPath(new URL("../../", import.meta.url));
const outputs: string[] = [];
let originalExitCode: typeof process.exitCode;
beforeEach(() => {
journalFs.directory = ""; journalFs.fault = ""; journalFs.events = []; journalFs.snapshots = []; journalFs.gate = undefined;
journalFs.persistentFault = false; journalFs.captureWriterDirectory = false; journalFs.diagnosticFailure = "";
originalExitCode = process.exitCode;
scenario.authReads = 0; scenario.prepare.mockReset(); scenario.flow.mockReset(); scenario.close.mockReset();
scenario.cleanup.mockReset();
scenario.reportOutput = "";
scenario.failBranding = false;
scenario.config = parseAntigravityJourneyConfig({ version: 1, setupId: "assisted-login-calibration", hosting: "self-hosted", executionEnvironment: "local",
target: { mode: "attach", baseURL: "http://127.0.0.1:3104", expectedCommit: "a".repeat(40), deploymentMode: "authenticated",
companyId: "11111111-1111-4111-8111-111111111111", environmentId: "22222222-2222-4222-8222-222222222222" },
browser: { headed: true, channel: "chromium", freshness: "signed-in" },
savedConnection: { id: "33333333-3333-4333-8333-333333333333", name: "Synthetic subscription", ownerUserId: "synthetic-operator" },
candidate: { platform: "darwin", architecture: "arm64", nativeVersion: "1.4.3-paperclip.5", profileDigest: `sha256:${"b".repeat(64)}`, nativeClosureDigest: `sha256:${"c".repeat(64)}` },
limits: { durationMs: 900_000, turnTimeoutMs: 300_000, maxRuns: 2, budgetCents: 100 } });
scenario.prepare.mockResolvedValue({ identity: { commit: "a".repeat(40) }, company: { issuePrefix: "QA" },
connection: { id: scenario.config.savedConnection.id, grantId: "synthetic-grant", ownerUserId: "synthetic-operator", isDefault: true, method: "subscription" }, accounting: {} });
});
afterEach(async () => {
process.exitCode = originalExitCode;
await Promise.all([...new Set([...outputs.splice(0), scenario.reportOutput].filter(Boolean))].map(output => rm(output, { recursive: true, force: true })));
});
const executions = () => selectRunnerExecutions(parseRunnerSelectors(["--id", ANTIGRAVITY_JOURNEY_EXECUTION]), runnerMatrix);
async function resourceJournal() {
const directory = await mkdtemp(path.join(tmpdir(), "antigravity-owned-journal-"));
outputs.push(directory);
const file = path.join(directory, "owned-resources.json");
const resources: AntigravityJourneyResources = { version: 1, executionId: ANTIGRAVITY_JOURNEY_EXECUTION,
origin: scenario.config.target.baseURL, companyId: scenario.config.target.companyId, environmentId: scenario.config.target.environmentId,
connectionId: scenario.config.savedConnection.id, agentName: "Antigravity QA abcdef123456", runIds: [], retained: false,
setupProbe: "not_started", processes: {} };
await writeFile(file, JSON.stringify(resources), { mode: 0o600 });
journalFs.directory = directory;
return { directory, file, resources, persist: createAntigravityResourceWriter(file) };
}
it.each(["write", "file-sync", "rename"])("preserves the previous real ownership journal when replacement %s fails", async fault => {
const f = await resourceJournal();
const previous = await readFile(f.file, "utf8");
f.resources.agentId = "66666666-6666-4666-8666-666666666666";
journalFs.fault = fault;
await expect(f.persist(f.resources)).rejects.toThrow(`fixture journal ${fault} failed`);
expect(await readFile(f.file, "utf8")).toBe(previous);
expect(await readdir(f.directory)).toEqual(["owned-resources.json"]);
await f.persist(f.resources);
expect(JSON.parse(await readFile(f.file, "utf8")).agentId).toBe(f.resources.agentId);
});
it("requires private synced replacement and directory sync before ownership journaling succeeds", async () => {
const f = await resourceJournal();
await f.persist(f.resources);
expect((await stat(f.file)).mode & 0o777).toBe(0o600);
expect(journalFs.events).toEqual(["file-open", "write", "file-sync", "rename", "directory-open", "directory-sync"]);
journalFs.fault = "directory-sync";
f.resources.agentId = "66666666-6666-4666-8666-666666666666";
await expect(f.persist(f.resources)).rejects.toThrow("fixture journal directory-sync failed");
expect(JSON.parse(await readFile(f.file, "utf8")).agentId).toBe(f.resources.agentId);
expect(await readdir(f.directory)).toEqual(["owned-resources.json"]);
});
it("serializes real ownership replacements and captures each snapshot before later mutations", async () => {
const f = await resourceJournal();
let entered!: () => void, release!: () => void;
const waiting = new Promise<void>(resolve => { entered = resolve; });
journalFs.gate = { entered, wait: new Promise<void>(resolve => { release = resolve; }), used: false };
const first = f.persist(f.resources);
await waiting;
const pending = (await readdir(f.directory)).filter(file => file !== "owned-resources.json");
const pendingModes = await Promise.all(pending.map(async file => (await stat(path.join(f.directory, file))).mode & 0o777));
f.resources.agentId = "66666666-6666-4666-8666-666666666666";
const second = f.persist(f.resources);
f.resources.issueId = "88888888-8888-4888-8888-888888888888";
const third = f.persist(f.resources);
release();
await Promise.all([first, second, third]);
expect(pending).toHaveLength(1);
expect(pendingModes).toEqual([0o600]);
expect(journalFs.snapshots).toHaveLength(3);
expect(journalFs.snapshots[0]).not.toHaveProperty("agentId");
expect(journalFs.snapshots[1]).toMatchObject({ agentId: f.resources.agentId });
expect(journalFs.snapshots[1]).not.toHaveProperty("issueId");
expect(journalFs.snapshots[2]).toMatchObject({ agentId: f.resources.agentId, issueId: f.resources.issueId });
expect(JSON.parse(await readFile(f.file, "utf8"))).toMatchObject({ agentId: f.resources.agentId, issueId: f.resources.issueId });
expect(journalFs.events).toEqual(Array.from({ length: 3 }, () => ["file-open", "write", "file-sync", "rename", "directory-open", "directory-sync"]).flat());
});
it("blocks a measured attempt after assisted board login before agent or provider setup and retains a valid failed report", async () => {
const output = await runAntigravityJourneyCampaign({ executions: executions(), catalog: runnerMatrix, repositoryRoot }); outputs.push(output);
expect(scenario.prepare).not.toHaveBeenCalled(); expect(scenario.flow).not.toHaveBeenCalled();
expect(scenario.authReads).toBe(2);
const result = JSON.parse(await readFile(path.join(output, "evidence/result.json"), "utf8"));
expect(result).toMatchObject({ status: "failed", error: "assisted_board_login_requires_fresh_case", runIds: [],
antigravityJourney: { assisted: true, outcome: "blocked_target", phase: "board_authentication", turnEvidence: [] } });
expect(result.antigravityJourney.agentId).toBeUndefined();
expect(scenario.close).toHaveBeenCalledOnce();
});
it("allows assisted setup-only preflight to save board authentication without creating an agent or running a provider", async () => {
const output = await runAntigravityJourneyCampaign({ executions: executions(), catalog: runnerMatrix, repositoryRoot, preflight: true }); outputs.push(output);
expect(scenario.prepare).toHaveBeenCalledOnce(); expect(scenario.flow).not.toHaveBeenCalled();
expect(JSON.parse(await readFile(path.join(output, "preflight.json"), "utf8"))).toMatchObject({ outcome: "ready", measured: false, providerPrompts: 0 });
expect(scenario.close).toHaveBeenCalledOnce();
});
it("rejects explicit private Antigravity selection from paid matrix generation", () => {
expect(executions()).toHaveLength(1);
expect(() => buildMatrixJobs(executions())).toThrow("not admitted to paid CI matrix jobs");
});
it.each(["cancelled", "interrupted", "timed_out"])("packages settled %s usage captured during cleanup before public run rows are purged", async terminalStatus => {
scenario.authReads = 1;
const first = "44444444-4444-4444-8444-444444444444", discovered = "55555555-5555-4555-8555-555555555555";
const agentId = "66666666-6666-4666-8666-666666666666";
const issueId = "88888888-8888-4888-8888-888888888888";
const run = (id: string, cost: number) => ({ id, agentId, issueId, companyId: scenario.config.target.companyId,
status: terminalStatus, costAccountingPending: false, usageJson: { inputTokens: 100, outputTokens: 20, costUsd: cost },
startedAt: "2026-01-01T00:00:00Z", finishedAt: "2026-01-01T00:00:01Z" });
scenario.flow.mockImplementationOnce(async ({ resources, observeRun }) => {
resources.agentId = agentId;
resources.issueId = issueId;
await observeRun({ ...run(first, 0.01), status: "running", costAccountingPending: true });
throw new Error("fixture interrupted turn");
});
scenario.cleanup.mockImplementationOnce(async input => {
input.resources.runIds.push(discovered);
await input.captureRun(run(first, 0.6));
await input.captureRun(run(discovered, 0.4));
Object.assign(input.cleanup, { agent: true, runs: true, connectionPreserved: true });
});
const output = await runAntigravityJourneyCampaign({ executions: executions(), catalog: runnerMatrix, repositoryRoot }); outputs.push(output);
const result = JSON.parse(await readFile(path.join(output, "evidence/result.json"), "utf8"));
expect(result.runIds).toEqual([first, discovered]);
expect(result.issueId).toBe(issueId);
expect(result.billing.llm.reportedCostUsd).toBe(1);
expect(result.runtimeUsage.agentRunDurationMs).toBe(2000);
expect(result.cleanup).toBe("passed");
expect(result.status).toBe("failed");
expect(result.usage.runs.every((row: any) => row.usage.accountingReceiptReady !== false)).toBe(true);
expect(JSON.parse(await readFile(path.join(output, "run-usage-observations.json"), "utf8")).runs).toHaveLength(2);
});
it("retains cleanup-discovered run identities with unknown usage when their public rows are unavailable", async () => {
scenario.authReads = 1;
const missing = "77777777-7777-4777-8777-777777777777";
scenario.flow.mockImplementationOnce(async ({ resources }) => { resources.agentId = "66666666-6666-4666-8666-666666666666"; throw new Error("fixture interruption"); });
scenario.cleanup.mockImplementationOnce(async ({ resources }) => { resources.runIds.push(missing); throw new Error("fixture run unavailable"); });
const output = await runAntigravityJourneyCampaign({ executions: executions(), catalog: runnerMatrix, repositoryRoot }); outputs.push(output);
const result = JSON.parse(await readFile(path.join(output, "evidence/result.json"), "utf8"));
expect(result.runIds).toEqual([missing]);
expect(result.usage.runs).toEqual([{ usage: null }]);
expect(result.billing.complete).toBe(false);
expect(result.runtimeUsage.agentRunDurationMs).toBeNull();
expect(result.cleanup).toBe("failed");
});
it.each(["", "attempt-error.json", "progress.json", "attempt-result.json", "browser-close"])("closes the browser and packages a failed campaign after persistent resource sync failure (diagnostic=%s)", async diagnosticFailure => {
scenario.authReads = 1;
if (diagnosticFailure === "browser-close") scenario.close.mockRejectedValueOnce(new Error("fixture browser close failed"));
journalFs.captureWriterDirectory = true;
const agentId = "66666666-6666-4666-8666-666666666666";
const forbiddenMutation = vi.fn();
let previousJournal = "";
scenario.flow.mockImplementationOnce(async ({ resources, persistResources, observeRun, phase }) => {
await phase("agent_creation");
resources.agentId = agentId;
resources.agentCreationRequest = { setupId: agentId, companyId: resources.companyId, agentName: resources.agentName,
provider: "antigravity", creatorUserId: scenario.config.savedConnection.ownerUserId, origin: resources.origin };
resources.retained = true;
await persistResources();
await observeRun({ id: "44444444-4444-4444-8444-444444444444", agentId, companyId: resources.companyId,
status: "failed", costAccountingPending: false, usageJson: { inputTokens: 100, outputTokens: 20, costUsd: 0.25 },
startedAt: "2026-01-01T00:00:00Z", finishedAt: "2026-01-01T00:00:01Z" });
previousJournal = await readFile(path.join(journalFs.directory, "owned-resources.json"), "utf8");
journalFs.fault = "file-sync";
journalFs.persistentFault = true;
journalFs.diagnosticFailure = diagnosticFailure === "browser-close" ? "" : diagnosticFailure;
throw new ConnectionFailure("fixture_primary_failure");
});
scenario.cleanup.mockImplementationOnce(async ({ persistResources }) => {
await persistResources();
forbiddenMutation();
});
let output: string | undefined;
try {
output = await runAntigravityJourneyCampaign({ executions: executions(), catalog: runnerMatrix, repositoryRoot });
outputs.push(output);
} finally {
// Also remove red-control output when the broken finally rejects before returning it.
if (journalFs.directory) outputs.push(journalFs.directory);
expect(scenario.close).toHaveBeenCalledOnce();
}
expect(scenario.flow).toHaveBeenCalledOnce();
expect(scenario.cleanup).toHaveBeenCalledOnce();
expect(forbiddenMutation).not.toHaveBeenCalled();
expect(journalFs.events.filter(event => event === "file-sync").length).toBeGreaterThanOrEqual(3);
expect(await readFile(path.join(output!, "owned-resources.json"), "utf8")).toBe(previousJournal);
expect(parseAntigravityOwnedResources(JSON.parse(previousJournal), scenario.config)).toMatchObject({ agentId, retained: true,
agentCreationRequest: { setupId: agentId, creatorUserId: scenario.config.savedConnection.ownerUserId } });
expect((await stat(path.join(output!, "owned-resources.json"))).mode & 0o777).toBe(0o600);
expect((await readdir(output!)).filter(file => file.startsWith(".owned-resources-"))).toEqual([]);
expect(process.exitCode).toBe(1);
expect(scenario.reportOutput).toBe(output);
const primary = diagnosticFailure === "attempt-result.json" ? undefined
: JSON.parse(await readFile(path.join(output!, "attempt-result.json"), "utf8"));
const retained = JSON.parse(await readFile(path.join(output!, "evidence/result.json"), "utf8"));
for (const result of [retained, ...(primary ? [primary] : [])]) {
expect(result).toMatchObject({ status: "failed", cleanup: "failed", failureClass: "cleanup_failure", antigravityJourney: { outcome: "failed" } });
expect(result.error).toContain("fixture_primary_failure");
expect(result.error).toContain("fixture_cleanup_failed");
expect(result.billing.llm.reportedCostUsd).toBe(0.25);
expect(result.usage.runs).toEqual([{ usage: { inputTokens: 100, outputTokens: 20, costUsd: 0.25 } }]);
}
if (diagnosticFailure === "attempt-result.json") {
await expect(readFile(path.join(output!, "attempt-result.json"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
expect(retained.error).toContain("journey_attempt_result_write_failed");
}
if (diagnosticFailure === "browser-close") expect(retained.error).toContain("qa_browser_cleanup_failed");
if (diagnosticFailure !== "attempt-error.json")
expect(JSON.parse(await readFile(path.join(output!, "attempt-error.json"), "utf8")).error).toBe("fixture_primary_failure");
});
it.each(["missing_screenshot", "late_assets"])("preserves the primary failure and accounting when real report packaging fails at %s", async failure => {
scenario.authReads = 1;
const run = { id: "44444444-4444-4444-8444-444444444444", agentId: "66666666-6666-4666-8666-666666666666",
companyId: scenario.config.target.companyId, status: "failed", costAccountingPending: false,
usageJson: { inputTokens: 100, outputTokens: 20, costUsd: 0.25 }, startedAt: "2026-01-01T00:00:00Z", finishedAt: "2026-01-01T00:00:01Z" };
scenario.flow.mockImplementationOnce(async ({ resources, observeRun, checkpoint }) => {
resources.agentId = run.agentId;
await observeRun(run);
if (failure === "missing_screenshot") await checkpoint("ordinary_picker", { url: () => `${scenario.config.target.baseURL}/QA/agents`,
locator: () => ({ count: async () => 0 }), screenshot: async () => Buffer.from("fixture screenshot was not saved") });
throw new ConnectionFailure("fixture_primary_failure");
});
scenario.failBranding = failure === "late_assets";
scenario.cleanup.mockImplementationOnce(async ({ cleanup }) => Object.assign(cleanup, { agent: true, runs: true, connectionPreserved: true }));
const output = await runAntigravityJourneyCampaign({ executions: executions(), catalog: runnerMatrix, repositoryRoot }); outputs.push(output);
expect(process.exitCode).toBe(1);
const source = JSON.parse(await readFile(path.join(output, "attempt-result.json"), "utf8"));
const retained = JSON.parse(await readFile(path.join(output, "evidence/result.json"), "utf8"));
expect(source).toMatchObject({ status: "failed", cleanup: "passed", failureClass: "candidate_failure", error: "fixture_primary_failure",
antigravityJourney: { outcome: "failed" }, billing: { llm: { reportedCostUsd: 0.25 } } });
expect(retained).toMatchObject({ status: "failed", cleanup: "passed", failureClass: "candidate_failure",
error: "fixture_primary_failure; journey_report_packaging_failed" });
for (const key of ["usage", "runtimeUsage", "billing", "screenshots", "antigravityJourney"]) expect(retained[key]).toEqual(source[key]);
expect(JSON.parse(await readFile(path.join(output, "attempt-error.json"), "utf8")).error).toBe("fixture_primary_failure");
expect(JSON.parse(await readFile(path.join(output, "packaging-error.json"), "utf8"))).toMatchObject({ code: "journey_report_packaging_failed",
primary: { status: "failed", error: "fixture_primary_failure", outcome: "failed" } });
if (failure === "late_assets") {
expect(JSON.parse(await readFile(path.join(output, "packaging-source-result.json"), "utf8"))).toMatchObject({ error: "fixture_primary_failure" });
expect(JSON.parse(await readFile(path.join(output, "packaging-source-evidence-manifest.json"), "utf8")).missing).toEqual([]);
}
const env = Object.fromEntries(["PATH", "HOME", "USER", "TMPDIR", "LANG"].flatMap(key => process.env[key] ? [[key, process.env[key]!]] : []));
Object.assign(env, { PAPERCLIP_RUNNER_E2E_REPORT_ROOT: output, PAPERCLIP_RUNNER_E2E_REPORT_OUT: path.join(output, "rebuilt"),
PAPERCLIP_RUNNER_E2E_EXPECTED_IDS: JSON.stringify([ANTIGRAVITY_JOURNEY_EXECUTION]), PAPERCLIP_E2E_CAMPAIGN_ID: path.basename(output) });
const rebuilt = spawnSync(process.execPath, ["cli/node_modules/tsx/dist/cli.mjs", "tests/runner-e2e/report.ts"], { cwd: repositoryRoot, env, timeout: 25_000, encoding: "utf8" });
expect(rebuilt.status).toBe(1);
const normalized = JSON.parse(await readFile(path.join(output, "rebuilt/normalized-results.json"), "utf8")).results[0];
expect(normalized).toMatchObject({ status: "failed", cleanup: "passed", error: retained.error, usage: retained.usage, runtimeUsage: retained.runtimeUsage });
expect(normalized.billing.llm.reportedCostUsd).toBe(0.25);
});
tests/runner-e2e/antigravity-journey-launch.ts
+334
View File
@@ -0,0 +1,334 @@
import { createHash, randomBytes } from "node:crypto";
import { execFileSync } from "node:child_process";
import { lstat, mkdir, open, readFile, rename, rm, writeFile } from "node:fs/promises";
import path from "node:path";
import { ANTIGRAVITY_JOURNEY_EXECUTION } from "./antigravity-journey-cases.js";
import { ANTIGRAVITY_JOURNEY_MODEL, antigravityConnectionConfig, loadAntigravityJourneyConfig } from "./antigravity-journey-config.js";
import { gradeAntigravityJourney, type AntigravityJourneyEvidence } from "./antigravity-journey-evidence.js";
import { cleanupAntigravityJourney, isAntigravityRunAccountingSettled, observeAntigravityRun, parseAntigravityOwnedResources, prepareAntigravityJourney, type AntigravityJourneyResources, type AntigravityProcessObservers } from "./antigravity-journey-fixtures.js";
import { runAntigravityJourneyFlow } from "./antigravity-journey-flow.js";
import { writeAntigravityJourneyReport } from "./antigravity-journey-report.js";
import { buildRuntimeUsage, summarizeExecutionBilling } from "./billing.js";
import { ConnectionBlock, ConnectionFailure, assertPrivateFile } from "./connection-config.js";
import { openConnectionBrowser, withConnectionCampaignCancellation } from "./connection-launch.js";
import { ConnectionApi, connectionDelay, verifyConnectionTarget } from "./connection-target.js";
import { FixtureRegistry } from "./fixture-registry.js";
import { assertSecretFree, sanitizeJson } from "./redaction.js";
import type { MatrixExecution, RunnerE2EResult } from "./types.js";
export function createAntigravityResourceWriter(file: string) {
let writes = Promise.resolve();
return async (resources: AntigravityJourneyResources) => {
const text = JSON.stringify(sanitizeJson(resources, []), null, 2) + "\n";
assertSecretFree(text, [], "owned-resources.json");
const next = writes.catch(() => {}).then(async () => {
const directory = path.dirname(file), parent = await lstat(directory);
if (!parent.isDirectory() || parent.isSymbolicLink() || (parent.mode & 0o077)
|| (process.getuid && parent.uid !== process.getuid())) throw new ConnectionFailure("unsafe_antigravity_resource_directory");
const existing = await lstat(file).catch(error => { if (error.code === "ENOENT") return; throw error; });
if (existing && (!existing.isFile() || existing.isSymbolicLink() || (existing.mode & 0o077)
|| (process.getuid && existing.uid !== process.getuid()))) throw new ConnectionFailure("unsafe_antigravity_resource_file");
const temporary = path.join(directory, `.owned-resources-${randomBytes(12).toString("hex")}.tmp`);
const handle = await open(temporary, "wx", 0o600);
try {
try { await handle.writeFile(text); await handle.sync(); } finally { await handle.close(); }
await rename(temporary, file);
const parentHandle = await open(directory, "r");
try { await parentHandle.sync(); } finally { await parentHandle.close(); }
} finally { await rm(temporary, { force: true }); }
});
writes = next;
await next;
};
}
type Row = Record<string, any>;
/** Known charges are an observed stop threshold. Missing subscription prices
* stay unknown, and this does not install or weaken any product budget policy. */
export function antigravityKnownCostObservation(runs: readonly Row[], budgetCents: number) {
const billing = summarizeExecutionBilling({ environmentId: "local", durationMs: 0, runIds: runs.map(run => run.id),
usage: { runs: runs.map(run => ({ usage: run.usageJson })) } });
const knownCostUsd = billing.llm.reportedCostUsd + (billing.llm.estimatedCostUsd ?? 0);
const thresholdUsd = budgetCents / 100;
return { budgetCents, knownCostUsd, costStatus: billing.llm.costStatus, priceCoverageComplete: billing.complete,
hardCurrencyCap: false, exceeded: knownCostUsd - thresholdUsd > Number.EPSILON * Math.max(1, knownCostUsd) };
}
async function sourceProvenance(repositoryRoot: string, suiteDefinitionHash: string) {
const git = (args: string[]) => execFileSync("git", args, { cwd: repositoryRoot, maxBuffer: 64 * 1024 * 1024 });
const untracked = git(["ls-files", "--others", "--exclude-standard", "-z"]).toString("utf8").split("\0").filter(Boolean)
.filter(file => /^(?:packages|server|ui|cli|tests|doc|maps|rules)\//.test(file) && /\.(?:md|ts|tsx|js|mjs|cjs|json|toml|rs|yaml|yml|sh|py|sql|css|html)$/.test(file)).sort();
const hash = createHash("sha256"); let count = 0;
for (const file of untracked) {
const absolute = path.resolve(repositoryRoot, file);
if (!absolute.startsWith(`${path.resolve(repositoryRoot)}${path.sep}`)) throw new ConnectionFailure("unsafe_source_provenance_path");
const stat = await lstat(absolute);
if (!stat.isFile() || stat.isSymbolicLink() || stat.size > 4 * 1024 * 1024) throw new ConnectionFailure("untracked_source_provenance_unavailable");
hash.update(file).update("\0").update(await readFile(absolute)).update("\0"); count++;
}
return { head: git(["rev-parse", "HEAD"]).toString("utf8").trim(), trackedDiffSha256: createHash("sha256").update(git(["diff", "--binary", "HEAD"])).digest("hex"),
untrackedSourceSha256: hash.digest("hex"), untrackedSourceFileCount: count, suiteDefinitionHash, dirty: git(["status", "--porcelain"]).length > 0 };
}
export async function runAntigravityJourneyCampaign(input: {
executions: readonly MatrixExecution[]; catalog: readonly MatrixExecution[]; configFile?: string; repositoryRoot: string;
preflight?: boolean; recoverFile?: string;
}) {
if (input.executions.length !== 1 || input.executions[0]!.id !== ANTIGRAVITY_JOURNEY_EXECUTION)
throw new ConnectionFailure("Select exactly the registered Antigravity journey cell separately");
if (process.env.PAPERCLIP_E2E_OPERATIONS_MANIFEST) throw new ConnectionFailure("Antigravity journey has no budgeted routing qualification");
if (input.preflight && input.recoverFile) throw new ConnectionFailure("Preflight and recovery are separate operations");
const config = await loadAntigravityJourneyConfig(input.configFile);
return withConnectionCampaignCancellation(async campaignSignal => {
const execution = input.executions[0]!;
const startedAt = new Date().toISOString(), nonce = randomBytes(6).toString("hex");
const campaignId = `antigravity-journey-${startedAt.replace(/[:.]/g, "-")}-${nonce}`;
const output = path.join(input.repositoryRoot, "tests/runner-e2e/results", campaignId);
await mkdir(output, { recursive: true, mode: 0o700 });
const provenance = await sourceProvenance(input.repositoryRoot, execution.suiteDefinitionHash);
const save = async (name: string, value: unknown) => {
const text = JSON.stringify(sanitizeJson(value, []), null, 2) + "\n";
assertSecretFree(text, [], name); await writeFile(path.join(output, name), text, { mode: 0o600 });
};
const deadline = new AbortController(), signal = AbortSignal.any([campaignSignal, deadline.signal]);
const timer = setTimeout(() => deadline.abort(), config.limits.durationMs);
const evidence: AntigravityJourneyEvidence = { version: 1, setupId: config.setupId, hosting: "self-hosted", targetCommit: null,
phase: "configuration", outcome: "running", assisted: false, privacy: "private", authentication: "saved-subscription", model: ANTIGRAVITY_JOURNEY_MODEL,
companyId: config.target.companyId, connectionId: config.savedConnection.id, checkpoints: {}, turnEvidence: [],
cleanup: { agent: true, runs: true, connectionPreserved: false } };
const resources: AntigravityJourneyResources = { version: 1, executionId: execution.id, origin: config.target.baseURL,
companyId: config.target.companyId, environmentId: config.target.environmentId, connectionId: config.savedConnection.id,
agentName: `Antigravity QA ${nonce}`, runIds: [], retained: false, setupProbe: "not_started", processes: {}, sourceProvenance: provenance };
const result: RunnerE2EResult = { schema: "paperclip.runner-e2e.result/v2", executionId: execution.id, suiteId: execution.suite.id,
suiteDefinitionHash: execution.suiteDefinitionHash, source: { sha: provenance.head,
ref: `${execFileSync("git", ["branch", "--show-current"], { cwd: input.repositoryRoot, encoding: "utf8" }).trim()}${provenance.dirty ? " (working tree)" : ""}`, workflowRunUrl: null }, attempt: 1, status: "failed", profileId: execution.profile.id, environmentId: execution.environment.id,
caseId: execution.task.id, provider: "acpx", model: ANTIGRAVITY_JOURNEY_MODEL, runtimeMode: "native", startedAt, finishedAt: startedAt,
durationMs: 0, cleanup: "not_started", antigravityJourney: evidence, screenshots: [] };
const observed = new Map<string, Row>(), observers: AntigravityProcessObservers = new Map();
let browser: Awaited<ReturnType<typeof openConnectionBrowser>> | undefined;
let fixtures: Awaited<ReturnType<FixtureRegistry["setupAll"]>> | undefined;
let api: ConnectionApi | undefined;
let operationOnly = Boolean(input.preflight || input.recoverFile), operationPassed = false;
const assertActive = () => { if (signal.aborted) throw new ConnectionBlock("blocked_target", campaignSignal.aborted ? "test_interrupted" : "journey_deadline_reached"); };
const writeResources = createAntigravityResourceWriter(path.join(output, "owned-resources.json"));
const persistResources = () => writeResources(resources);
let finalizationFailed = false;
const retainFinalizationFailure = (code: string, ownershipCleanup = false) => {
finalizationFailed = true;
result.status = "failed";
if (!result.error?.split("; ").includes(code)) result.error = `${result.error ? `${result.error}; ` : ""}${code}`;
if (ownershipCleanup) {
result.cleanup = "failed"; evidence.outcome = "failed"; result.failureClass = "cleanup_failure";
resources.retained = Boolean(resources.agentId || resources.agentCreationRequest);
} else result.failureClass ??= "permanent_infrastructure";
};
// Mutation-bound journal writes remain strict. Only failure-path retries are best effort.
const persistAfterFailure = async () => {
try { await persistResources(); }
catch { retainFinalizationFailure("resource_journal_failed", true); }
};
const captureRun = async (run: Row) => {
if (typeof run.id !== "string" || run.companyId !== resources.companyId || run.agentId !== resources.agentId)
throw new ConnectionFailure("observed_run_owner_mismatch");
observed.set(run.id, run);
// Preserve only accounting fields, before teardown can purge public rows.
// This also runs after the attempt deadline while cleanup settles work.
await save("run-usage-observations.json", { version: 1, runs: [...observed.values()].map(row => ({
id: row.id, companyId: row.companyId, agentId: row.agentId, status: row.status,
startedAt: row.startedAt ?? null, finishedAt: row.finishedAt ?? null,
costAccountingPending: row.costAccountingPending ?? null, usageJson: row.usageJson ?? null,
})) });
};
const abort = () => { for (const page of browser?.context.pages() ?? []) void page.close().catch(() => {}); };
signal.addEventListener("abort", abort, { once: true });
try {
assertActive();
await save("setup.json", { version: 1, setupId: config.setupId, hosting: config.hosting, executionEnvironment: config.executionEnvironment,
target: config.target, candidate: config.candidate, limits: config.limits, authentication: "saved-subscription", realProductAgent: true, sourceProvenance: provenance,
costBoundary: { budgetCents: config.limits.budgetCents, mode: "observed-known-cost-stop", hardCurrencyCap: false,
productBudgetPolicy: "inherited-unchanged", missingSubscriptionPrices: "unknown", maximumTaskRuns: 2, attempts: 1 },
configSha256: createHash("sha256").update(JSON.stringify(config)).digest("hex"), browser: { freshness: config.browser.freshness, accountAlias: config.browser.accountAlias } });
await persistResources();
browser = await openConnectionBrowser(antigravityConnectionConfig(config), input.repositoryRoot, config.target.baseURL);
const page = browser.context.pages()[0] ?? await browser.context.newPage();
await page.goto(config.target.baseURL);
api = new ConnectionApi(browser.context.request, config.target.baseURL);
evidence.phase = "board_authentication";
const loginDeadline = Date.now() + config.browser.loginTimeoutMs;
while (true) {
assertActive();
try { await api.get("/api/auth/get-session"); break; }
catch (error) {
if (!(error instanceof ConnectionFailure) || !["api_status_401", "api_status_403"].includes(error.message)) throw error;
if (!config.browser.headed || Date.now() >= loginDeadline) throw new ConnectionBlock("awaiting_user", "board_login_required");
if (!evidence.assisted) {
evidence.assisted = true;
console.log("[antigravity-agent-journey] Complete board sign-in in the dedicated QA browser; no Google provider login is measured.");
await save("progress.json", evidence);
}
await connectionDelay(1000);
}
}
if (evidence.assisted && !operationOnly) throw new ConnectionBlock("blocked_target", "assisted_board_login_requires_fresh_case");
evidence.phase = "preflight";
if (input.recoverFile) {
evidence.phase = "recovery";
// Recovery must still stop owned work after admission expires or a
// budget becomes blocked. It needs authenticated target identity only.
await verifyConnectionTarget(api.origin, antigravityConnectionConfig(config), api.request);
if (process.platform !== "darwin" || process.arch !== "arm64") throw new ConnectionBlock("blocked_target", "same_host_darwin_arm64_required");
await assertPrivateFile(input.recoverFile);
const owned = parseAntigravityOwnedResources(JSON.parse(await readFile(input.recoverFile, "utf8")), config);
Object.assign(resources, owned);
await persistResources();
await cleanupAntigravityJourney({ api, config, resources, cleanup: evidence.cleanup, persistResources, observers, captureRun });
await save("recovery.json", { version: 1, outcome: "released", cleanup: evidence.cleanup, sourceReceipt: createHash("sha256").update(await readFile(input.recoverFile)).digest("hex") });
operationPassed = true; console.log(`[antigravity-agent-journey] Owned resource recovery complete: ${output}`); return output;
}
const registry = new FixtureRegistry();
registry.register({ id: "antigravity-existing-state", setup: () => prepareAntigravityJourney(api!, config) });
if (!operationOnly) registry.register({ id: "antigravity-owned-resources", dependencies: ["antigravity-existing-state"], setup: async () => resources,
teardown: () => cleanupAntigravityJourney({ api: api!, config, resources, cleanup: evidence.cleanup, persistResources, observers, captureRun }) });
fixtures = await registry.setupAll();
const prepared = fixtures.values.get("antigravity-existing-state") as Awaited<ReturnType<typeof prepareAntigravityJourney>>;
evidence.targetCommit = prepared.identity.commit;
evidence.checkpoints.target = true; evidence.checkpoints.dependencies = true; evidence.cleanup.connectionPreserved = true;
await save("preflight.json", { version: 1, outcome: "ready", setupId: config.setupId, measured: false, providerPrompts: 0,
targetCommit: prepared.identity.commit, actor: "browser_operator", savedConnection: { id: prepared.connection.id, grantId: prepared.connection.grantId,
ownerUserId: prepared.connection.ownerUserId, isDefault: prepared.connection.isDefault, method: prepared.connection.method },
accounting: prepared.accounting, candidate: config.candidate });
if (input.preflight) { operationPassed = true; console.log(`[antigravity-agent-journey] Read-only preflight ready: ${output}`); return output; }
const observeRun = async (run: Row) => {
assertActive();
await captureRun(run); observeAntigravityRun(resources, run, observers); await persistResources();
if (observed.size > config.limits.maxRuns) throw new ConnectionFailure("provider_run_limit_exceeded");
const cost = antigravityKnownCostObservation([...observed.values()], config.limits.budgetCents);
await save("known-cost.json", cost);
if (cost.exceeded) throw new ConnectionBlock("blocked_target", "observed_known_cost_limit_exceeded");
};
await runAntigravityJourneyFlow({ page, api, config, prepared, resources, evidence, assertActive, persistResources, observeRun,
beforeAdmission: async () => {
assertActive();
// Read any settled accounting update before admitting the next UI turn.
for (const runId of observed.keys()) await observeRun(await api!.get<Row>(`/api/heartbeat-runs/${runId}`));
}, phase: async id => { assertActive(); evidence.phase = id; await save("progress.json", evidence); },
checkpoint: async (id, tab) => {
evidence.phase = id; evidence.checkpoints[id] = true;
const url = new URL(tab.url());
if (url.origin === config.target.baseURL && url.pathname.startsWith(`/${prepared.company.issuePrefix}/`)
&& !await tab.locator('input[type="password"]:visible,input[autocomplete="one-time-code"]:visible').count()) {
const file = `${id}.png`, bytes = await tab.screenshot({ path: path.join(output, file) });
result.screenshots!.push({ id, label: id.replaceAll("_", " "), file, sha256: createHash("sha256").update(bytes).digest("hex") });
}
await save("progress.json", evidence); await persistResources();
} });
result.issueId = resources.issueId;
evidence.outcome = "passed"; result.status = "passed";
} catch (caught) {
const error = signal.aborted ? new ConnectionBlock("blocked_target", campaignSignal.aborted ? "test_interrupted" : "journey_deadline_reached") : caught;
evidence.outcome = error instanceof ConnectionBlock ? error.reason : "failed";
result.failureClass = error instanceof ConnectionBlock ? "permanent_infrastructure" : "candidate_failure";
if (resources.setupProbe === "pending") resources.setupProbe = "retirement_unconfirmed";
if (evidence.phase === "recovery") { result.cleanup = "failed"; result.failureClass = "cleanup_failure"; resources.retained = Boolean(resources.agentId || resources.agentCreationRequest); }
result.error = error instanceof ConnectionBlock ? error.code : error instanceof ConnectionFailure ? error.message
: error instanceof Error && /^antigravity_evidence_[a-z0-9_]+$/.test(error.message) ? error.message : `${evidence.phase}_failed`;
let failureScreenshot = false;
const failedPage = browser?.context.pages()[0];
if (failedPage && !failedPage.isClosed()) {
try {
const url = new URL(failedPage.url());
if (url.origin === config.target.baseURL && url.pathname.startsWith("/")
&& !await failedPage.locator('input[type="password"]:visible,input[autocomplete="one-time-code"]:visible').count()) {
const file = "failure.png", bytes = await failedPage.screenshot({ path: path.join(output, file) });
result.screenshots!.push({ id: "failure", label: `Failed at ${evidence.phase.replaceAll("_", " ")}`, file, sha256: createHash("sha256").update(bytes).digest("hex") });
failureScreenshot = true;
}
} catch { /* Preserve the primary failure even when the browser is unavailable. */ }
}
const sourceLocation = error instanceof Error ? error.stack?.match(/antigravity-journey-(?:flow|ui)\.ts:\d+:\d+/u)?.[0] : undefined;
try { await save("attempt-error.json", { error: result.error, phase: evidence.phase, outcome: evidence.outcome, sourceLocation, failureScreenshot }); }
catch { retainFinalizationFailure("journey_error_diagnostic_write_failed"); }
await persistAfterFailure();
} finally {
clearTimeout(timer); signal.removeEventListener("abort", abort);
try {
try { await fixtures?.teardown(); if (result.cleanup !== "failed") result.cleanup = "passed"; }
catch (error) {
const code = error instanceof AggregateError && error.errors[0] instanceof ConnectionFailure ? error.errors[0].message : "fixture_cleanup_failed";
retainFinalizationFailure(code, true);
await persistAfterFailure();
}
} finally {
try { await browser?.close(); }
catch { retainFinalizationFailure("qa_browser_cleanup_failed", true); }
}
result.finishedAt = new Date().toISOString(); result.durationMs = Date.parse(result.finishedAt) - Date.parse(startedAt);
result.issueId = resources.issueId;
try { await save("progress.json", evidence); }
catch { retainFinalizationFailure("journey_progress_write_failed"); }
if (operationOnly) {
if (!operationPassed || result.cleanup === "failed" || finalizationFailed) process.exitCode = 1;
} else {
result.runIds = [...new Set([...resources.runIds, ...observed.keys()])];
const runs = result.runIds.map(id => observed.get(id) ?? { id });
result.usage = { runs: runs.map(run => ({ usage: !run.usageJson ? null
: !isAntigravityRunAccountingSettled(run)
? { ...run.usageJson, accountingReceiptReady: false } : run.usageJson })) };
result.runtimeUsage = buildRuntimeUsage({ environmentId: execution.environment.id, runs, fallbackFinishedAt: result.finishedAt });
result.billing = summarizeExecutionBilling(result);
result.matcherResults = gradeAntigravityJourney(evidence);
if (result.status === "passed" && !result.matcherResults.every(check => check.passed)) {
result.status = "failed"; evidence.outcome = "failed"; result.failureClass = "candidate_failure"; result.error = "antigravity_journey_proof_incomplete";
}
const primary = structuredClone(result);
try { await save("attempt-result.json", primary); }
catch { retainFinalizationFailure("journey_attempt_result_write_failed"); }
const presentation = structuredClone(result);
try {
const packaged = await writeAntigravityJourneyReport({ output, campaignId, result, resources, catalog: input.catalog, secrets: [] });
if (packaged.status !== "passed" || packaged.cleanup !== "passed") process.exitCode = 1;
console.log(`[antigravity-agent-journey] ${packaged.status}: ${path.join(output, "dashboard.html")}`);
} catch {
process.exitCode = 1;
const code = "journey_report_packaging_failed";
let retainedFailure = false;
try {
// Keep partially written normal evidence before replacing its
// presentation grade with an explicit packaging failure.
const sources: Record<string, string> = {};
let manifest: Row = {};
for (const name of ["result.json", "evidence-manifest.json"]) {
const bytes = await readFile(path.join(output, "evidence", name)).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return undefined;
throw error;
});
if (!bytes) continue;
await writeFile(path.join(output, `packaging-source-${name}`), bytes, { mode: 0o600 });
sources[name] = createHash("sha256").update(bytes).digest("hex");
if (name === "evidence-manifest.json") {
try {
const parsed: unknown = JSON.parse(bytes.toString("utf8"));
if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) manifest = parsed;
} catch { /* Retain malformed source bytes; packaging remains failed. */ }
}
}
const failed: RunnerE2EResult = { ...presentation, status: "failed",
failureClass: presentation.failureClass ?? "permanent_infrastructure",
error: `${presentation.error ? `${presentation.error}; ` : ""}${code}` };
await mkdir(path.join(output, "evidence"), { recursive: true, mode: 0o700 });
await save("evidence/result.json", failed);
await save("evidence/evidence-manifest.json", { ...manifest,
files: Array.isArray(manifest.files) ? [...new Set([...manifest.files, "result.json"])] : ["result.json"],
leaks: Array.isArray(manifest.leaks) ? manifest.leaks : [],
missing: [...new Set([...(Array.isArray(manifest.missing) ? manifest.missing : []), code])] });
await save("packaging-error.json", { version: 1, code, sources,
primary: { status: primary.status, error: primary.error ?? null, failureClass: primary.failureClass ?? null,
outcome: primary.antigravityJourney?.outcome, cleanup: primary.cleanup },
presentationOnlyRecovery: true });
retainedFailure = true;
} catch { /* Disk or permission failure cannot replace the primary outcome or the failed exit code. */ }
console.log(`[antigravity-agent-journey] failed: ${code}; ${retainedFailure ? "retained evidence/result.json" : "retained diagnostics unavailable"}: ${output}`);
}
}
}
return output;
});
}
tests/runner-e2e/antigravity-journey-report.test.ts
+106
View File
@@ -0,0 +1,106 @@
import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
import { mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { promisify } from "node:util";
import { afterEach, expect, it } from "vitest";
import { runnerMatrix } from "./catalog.js";
import { writeAntigravityJourneyReport } from "./antigravity-journey-report.js";
import { antigravityEvidenceFixture } from "./antigravity-journey-test-fixtures.js";
import { validateRetainedRunnerResult } from "./result-validation.js";
import { assertPublicCampaign } from "./history-publish.js";
import { summarizeExecutionBilling } from "./billing.js";
import type { RunnerE2EResult } from "./types.js";
const temporary: string[] = [];
afterEach(async () => { await Promise.all(temporary.splice(0).map(root => rm(root, { recursive: true, force: true }))); });
// Synthetic report calibration image; never a screenshot of a product attempt.
const pixel = Buffer.from("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jBv0AAAAASUVORK5CYII=", "base64");
function fixture(passed = true): RunnerE2EResult {
const cell = runnerMatrix.find(row => row.id === "antigravity-agent-journey.runner-antigravity.local.saved-subscription-two-turn")!;
const evidence = antigravityEvidenceFixture();
if (!passed) { evidence.outcome = "failed"; evidence.checkpoints.persisted = false; }
return { schema: "paperclip.runner-e2e.result/v2", executionId: cell.id, suiteId: cell.suite.id, suiteDefinitionHash: cell.suiteDefinitionHash,
profileId: cell.profile.id, environmentId: "local", caseId: cell.task.id, provider: "acpx", model: "gemini-3.8-flash-low", runtimeMode: "native",
attempt: 1, status: passed ? "passed" : "failed", ...(!passed ? { error: "antigravity_evidence_exact_download" } : {}), cleanup: "passed", durationMs: 1000,
startedAt: "2026-10-29T00:00:00Z", finishedAt: "2026-10-29T00:00:01Z", issueId: "task", runIds: ["run-first", "run-second"], antigravityJourney: evidence,
screenshots: [{ id: passed ? "persisted" : "failure", label: "Synthetic calibration only", file: "calibration.png", sha256: createHash("sha256").update(pixel).digest("hex") }] };
}
it.each([true, false])("roundtrips %s through ordinary packaging, validation and campaign dashboard", async passed => {
const output = await mkdtemp(path.join(os.tmpdir(), "antigravity-report-calibration-")); temporary.push(output);
await writeFile(path.join(output, "calibration.png"), pixel);
await writeFile(path.join(output, "private-auth.png"), pixel);
const result = await writeAntigravityJourneyReport({ output, campaignId: "calibration", result: fixture(passed), resources: { runIds: ["run-first", "run-second"] }, catalog: runnerMatrix, secrets: [] });
expect(() => validateRetainedRunnerResult(result)).not.toThrow();
expect(result.status).toBe(passed ? "passed" : "failed");
const manifest = JSON.parse(await readFile(path.join(output, "evidence/evidence-manifest.json"), "utf8"));
expect(manifest).toMatchObject({ leaks: [], missing: [] });
expect(manifest.files).toContain(`snapshots/${passed ? "persisted" : "failure"}.png`);
await expect(stat(path.join(output, "evidence/private-auth.png"))).rejects.toThrow();
const campaign = JSON.parse(await readFile(path.join(output, "campaign.json"), "utf8"));
expect(campaign).toMatchObject({ passed: passed ? 1 : 0, failed: passed ? 0 : 1 });
expect(() => assertPublicCampaign(campaign)).toThrow("Private Product E2E");
expect(await readFile(path.join(output, "dashboard.html"), "utf8")).toContain(`evidence/snapshots/${passed ? "persisted" : "failure"}.png`);
});
it("rejects a missing or changed required screenshot before a passing report", async () => {
const output = await mkdtemp(path.join(os.tmpdir(), "antigravity-report-calibration-")); temporary.push(output);
const input = fixture();
input.screenshots = [];
await expect(writeAntigravityJourneyReport({ output, campaignId: "missing", result: input, resources: {}, catalog: runnerMatrix, secrets: [] })).rejects.toThrow();
await writeFile(path.join(output, "calibration.png"), "altered");
await expect(writeAntigravityJourneyReport({ output, campaignId: "changed", result: fixture(), resources: {}, catalog: runnerMatrix, secrets: [] })).rejects.toThrow("journey_screenshot_digest_mismatch");
});
it("does not permit a green normalized receipt with incomplete owned cleanup", () => {
const input = fixture(); input.antigravityJourney!.cleanup.connectionPreserved = false;
expect(() => validateRetainedRunnerResult(input)).toThrow("result.antigravityJourney");
});
it("preserves measured tokens, duration and unknown subscription prices through packaging", async () => {
const output = await mkdtemp(path.join(os.tmpdir(), "antigravity-report-calibration-")); temporary.push(output);
await writeFile(path.join(output, "calibration.png"), pixel);
const input = fixture();
input.usage = { runs: [{ usage: { inputTokens: 100, outputTokens: 20, cachedInputTokens: 10, costUsd: 0, costStatus: "unpriced" } },
{ usage: { inputTokens: 200, outputTokens: 30, cachedInputTokens: 40, costUsd: 0, costStatus: "unpriced" } }] };
input.billing = summarizeExecutionBilling(input);
const result = await writeAntigravityJourneyReport({ output, campaignId: "unpriced", result: input, resources: {}, catalog: runnerMatrix, secrets: [] });
const retained = JSON.parse(await readFile(path.join(output, "evidence/result.json"), "utf8"));
expect(retained.durationMs).toBe(1000);
expect(retained.billing).toEqual(result.billing);
expect(summarizeExecutionBilling(retained)).toEqual(input.billing);
expect(retained.billing).toMatchObject({ complete: false, llm: { runCount: 2, inputTokens: 300, outputTokens: 50, cachedInputTokens: 50,
runsWithReportedCost: 0, costStatus: "unpriced" } });
});
it.each([true, false])("imports a synthetic %s receipt through the ordinary report CLI without changing its verdict or unknown prices", async passed => {
const output = await mkdtemp(path.join(os.tmpdir(), "antigravity-report-cli-calibration-")); temporary.push(output);
await writeFile(path.join(output, "calibration.png"), pixel);
const input = fixture(passed);
input.usage = { runs: [
{ usage: { inputTokens: 100, outputTokens: 20, cachedInputTokens: 10, costUsd: 0, costStatus: "unpriced" } },
{ usage: { inputTokens: 200, outputTokens: 30, cachedInputTokens: 40, costUsd: 0, costStatus: "unpriced" } },
] };
input.billing = summarizeExecutionBilling(input);
await writeAntigravityJourneyReport({ output, campaignId: "cli-calibration", result: input, resources: {}, catalog: runnerMatrix, secrets: [] });
const repositoryRoot = path.resolve(import.meta.dirname, "../.."), rebuilt = path.join(output, "rebuilt");
let exitCode = 0;
try {
await promisify(execFile)(process.execPath, ["--import", path.join(repositoryRoot, "cli/node_modules/tsx/dist/loader.mjs"), path.join(repositoryRoot, "tests/runner-e2e/report.ts")], {
cwd: repositoryRoot, env: { ...process.env, PAPERCLIP_RUNNER_E2E_REPORT_ROOT: output, PAPERCLIP_RUNNER_E2E_REPORT_OUT: rebuilt,
PAPERCLIP_RUNNER_E2E_EXPECTED_IDS: JSON.stringify([input.executionId]) }, timeout: 30_000, maxBuffer: 1024 * 1024,
});
} catch (error) {
if (typeof (error as { code?: unknown }).code !== "number") throw error;
exitCode = (error as { code: number }).code;
}
expect(exitCode).toBe(passed ? 0 : 1);
const normalized = JSON.parse(await readFile(path.join(rebuilt, "normalized-results.json"), "utf8"));
expect(normalized.results).toHaveLength(1);
expect(normalized.results[0]).toMatchObject({ status: input.status, durationMs: 1000, evidenceValid: true, antigravityJourney: { privacy: "private", outcome: input.status } });
expect(normalized.results[0].billing).toEqual(input.billing);
expect(normalized.results[0].antigravityJourney.turnEvidence).toHaveLength(2);
expect(await readFile(path.join(rebuilt, "dashboard.html"), "utf8")).toContain("antigravity-agent-journey");
});
tests/runner-e2e/antigravity-journey-report.ts
+67
View File
@@ -0,0 +1,67 @@
import { createHash } from "node:crypto";
import { mkdir, mkdtemp, readFile, realpath, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { packageEvidence } from "./evidence.js";
import { buildRunnerCampaign } from "./history.js";
import { renderRunnerE2EDashboard } from "./dashboard.js";
import { stageDashboardBrandAssets } from "./report-assets.js";
import { assertSecretFree, sanitizeJson } from "./redaction.js";
import { validateRetainedRunnerResult } from "./result-validation.js";
import type { MatrixExecution, RunnerE2EResult } from "./types.js";
/** Use the normal sanitizer, artifact manifest, validator, and dashboard. Raw
* operator receipts stay in the private attempt; they are not auto-published. */
export async function writeAntigravityJourneyReport(input: {
output: string; campaignId: string; result: RunnerE2EResult; resources: object;
catalog: readonly MatrixExecution[]; secrets: readonly string[];
}) {
const stage = await mkdtemp(path.join(os.tmpdir(), "antigravity-journey-report-"));
const output = path.resolve(input.output);
const evidenceDir = path.join(output, "evidence");
const realOutput = await realpath(output);
const save = async (root: string, name: string, value: unknown) => {
const text = JSON.stringify(sanitizeJson(value, input.secrets), null, 2) + "\n";
assertSecretFree(text, input.secrets, name);
await writeFile(path.join(root, name), text, { mode: 0o600 });
};
try {
await mkdir(path.join(stage, "snapshots"), { mode: 0o700 });
const result = structuredClone(input.result);
let finalStateStaged = false;
for (const shot of result.screenshots ?? []) {
const source = path.resolve(output, shot.file);
if (!source.startsWith(`${output}${path.sep}`) || !/^[a-z0-9_-]+$/.test(shot.id) || path.extname(source) !== ".png") throw new Error("unsafe_journey_screenshot_reference");
if (!(await realpath(source)).startsWith(`${realOutput}${path.sep}`)) throw new Error("unsafe_journey_screenshot_reference");
const bytes = await readFile(source);
if (shot.sha256 && createHash("sha256").update(bytes).digest("hex") !== shot.sha256) throw new Error("journey_screenshot_digest_mismatch");
shot.file = `snapshots/${shot.id}.png`;
await writeFile(path.join(stage, shot.file), bytes, { mode: 0o600 });
if (result.status === "passed" && shot.id === "persisted" && result.antigravityJourney?.checkpoints.persisted === true) {
await writeFile(path.join(stage, "final-state.png"), bytes, { mode: 0o600 });
finalStateStaged = true;
}
}
validateRetainedRunnerResult(result);
if (result.status === "passed" && !finalStateStaged) throw new Error("journey_final_state_missing");
await save(stage, "result.json", result);
await save(stage, "snapshots/journey-resources.json", input.resources);
await save(stage, "snapshots/journey-evidence.json", result.antigravityJourney);
const manifest = await packageEvidence({ privateDir: stage, uploadDir: evidenceDir, secrets: input.secrets, expectPassScreenshot: false });
if (manifest.leaks.length || manifest.missing.length) {
result.status = "failed"; result.failureClass = "secret_leak"; result.error = "journey_evidence_packaging_failed";
if (result.antigravityJourney) result.antigravityJourney.outcome = "failed";
}
await save(evidenceDir, "result.json", result);
await save(evidenceDir, "evidence-manifest.json", manifest);
const campaign = buildRunnerCampaign({ campaignId: input.campaignId, generatedAt: result.finishedAt, expected: [result.executionId], results: [result] });
await save(output, "campaign.json", campaign);
await save(output, "normalized-results.json", { ...campaign, results: [{ ...result, evidenceValid: !manifest.leaks.length && !manifest.missing.length, evidenceErrors: [] }] });
await stageDashboardBrandAssets(output);
const html = renderRunnerE2EDashboard({ title: "Google Antigravity setup and two-turn task", generatedAt: result.finishedAt, expected: [result.executionId], catalog: input.catalog, campaign,
entries: [{ result, valid: !manifest.leaks.length && !manifest.missing.length, errors: [], evidenceBaseHref: "evidence" }] });
assertSecretFree(html, input.secrets, "dashboard.html");
await writeFile(path.join(output, "dashboard.html"), html, { mode: 0o600 });
return result;
} finally { await rm(stage, { recursive: true, force: true }); }
}
tests/runner-e2e/antigravity-journey-test-fixtures.ts
+67
View File
@@ -0,0 +1,67 @@
import { createHash } from "node:crypto";
import { gradeAntigravityJourneyTurn, antigravityJourneyCheckpoints, type AntigravityJourneyEvidence } from "./antigravity-journey-evidence.js";
/** Synthetic calibration only. No provider, account, or historical session data. */
export function antigravityTurnFixture(second = false) {
const firstBytes = "ANTIGRAVITY-INSTRUCTION-00000000-0000-4000-8000-000000000001\n";
const expectedBytes = Buffer.from(firstBytes + (second ? "ANTIGRAVITY-FOLLOWUP-00000000-0000-4000-8000-000000000002\n" : ""));
const sha256 = createHash("sha256").update(expectedBytes).digest("hex");
const runId = second ? "run-second" : "run-first", runnerInstanceId = second ? "runner-second" : "runner-first";
const attachmentId = second ? "attachment-second" : "attachment-first";
const model = "gemini-3.8-flash-low", profileDigest = `sha256:${"a".repeat(64)}`;
const providerIdentity = { kind: "acpx", acpxRecordId: "acpx-record", agentSessionId: "provider-session", backendSessionId: "provider-session",
normalizedSessionId: "native-session", requestedModel: model, effectiveModel: model, permissionMode: "approve-all", workspaceDigest: `sha256:${"b".repeat(64)}`, profileDigest };
const run: Record<string, any> = { id: runId, companyId: "company", agentId: "agent", issueId: "task", status: "succeeded", runtimeMode: "native", driverKind: "acpx_runtime",
nativeSessionId: "native-session", runnerInstanceId, costAccountingPending: false,
processPid: second ? 900002 : 900001, processGroupId: second ? 900002 : 900001, processStartedAt: second ? "2026-10-29T01:00:00.000Z" : "2026-10-29T00:00:00.000Z",
runnerProfileJson: { backend: "acpx_runtime", nativeExecutionInput: { binding: { runId, companyId: "company", agentId: "agent", issueId: "task" },
provider: { kind: "acpx", agent: "antigravity", model, profile: { commandDigest: profileDigest } } }, sessionCheckpoint: { providerIdentity } },
contextSnapshot: { aiConnection: { provider: "google", method: "subscription", connectionId: "connection" }, paperclipEnvironment: { id: "environment", driver: "local", leaseId: `lease-${runId}` } },
resultJson: { finalizationPhase: "committed", authoritativeDecision: "done", issueStatusAfter: "done", workspaceFinalizeStatus: "succeeded", assessmentId: `assessment-${runId}`, decisionId: `decision-${runId}`,
nativeResult: { schema: "paperclip.run_result.v1" }, nativeTerminal: { runTerminalState: "succeeded" },
semanticToolReceipts: { register: { operationId: "register_deliverable", input: { filename: "antigravity-proof.txt", byteSize: expectedBytes.length, sha256 }, result: { attachmentId, disposition: "applied" } } } } };
const event = (operation: string, seq: number, status = "completed", name = `client_${operation}_file`, execution = `call-${seq}`) => ({
companyId: "company", runId, seq, sourceInstanceId: runnerInstanceId, sourceSeq: seq, sourceEventId: executorSourceEventId(runnerInstanceId, seq),
protocolSchemaVersion: 1, eventType: status === "running" ? "tool.execution.started" : "tool.execution.completed", payload: { prpEvent: {
schema: "paperclip.prp.event.v1", schemaVersion: 1, eventType: status === "running" ? "tool.execution.started" : "tool.execution.completed", runId, sourceKind: "runner", sourceInstanceId: runnerInstanceId,
sourceSeq: seq, sourceEventId: executorSourceEventId(runnerInstanceId, seq), normalizedSessionId: "native-session", turnId: `turn-${runId}`,
payload: { operation, status, name, target: "antigravity-proof.txt", executionId: execution } } } });
const events = second ? [event("read", 1, "running", "client_read_file", "before-read"), event("read", 2, "completed", "client_read_file", "before-read"),
event("edit", 3, "running", "client_edit_file", "edit-call"), event("edit", 4, "completed", "client_edit_file", "edit-call"),
event("read", 5, "running", "client_read_file", "read-call"), event("read", 6, "completed", "client_read_file", "read-call")]
: [event("edit", 1, "running", "client_edit_file", "edit-call"), event("edit", 2, "completed", "client_edit_file", "edit-call"), event("read", 3, "running", "client_read_file", "read-call"), event("read", 4, "completed", "client_read_file", "read-call")];
events.push(event("execute", events.length + 1, "failed", "paperclip_paperclip_finish"));
for (const eventType of ["run.result.accepted", "run.terminal"]) {
const seq = events.length + 1, sourceInstanceId = `${runnerInstanceId}:control`;
const payload = eventType === "run.result.accepted" ? { result: structuredClone(run.resultJson.nativeResult) }
: { schema: "paperclip.prp.terminal.v1", runTerminalState: "succeeded", turnTerminalState: "completed", reportedWorkDisposition: "done" };
events.push({ companyId: "company", runId, seq, sourceInstanceId, sourceSeq: seq, sourceEventId: `${sourceInstanceId}:${runId}:${seq}`,
protocolSchemaVersion: 1, eventType, payload: { prpEvent: { schema: "paperclip.prp.event.v1", schemaVersion: 1, eventType, runId,
sourceKind: "control_plane", sourceInstanceId, sourceSeq: seq, sourceEventId: `${sourceInstanceId}:${runId}:${seq}`,
normalizedSessionId: "native-session", turnId: `turn-${runId}`, payload } } } as any);
}
return { companyId: "company", agentId: "agent", issueId: "task", connectionId: "connection", environmentId: "environment", model, expectedProfileDigest: profileDigest,
expectedBytes, afterRunIds: second ? ["run-first"] : [], run,
issue: { id: "task", companyId: "company", assigneeAgentId: "agent", status: "done" },
attachment: { id: attachmentId, companyId: "company", issueId: "task", createdByAgentId: "agent", originatingRunId: runId,
originalFilename: "antigravity-proof.txt", contentType: "text/plain", byteSize: expectedBytes.length, sha256 },
downloadedBytes: Buffer.from(expectedBytes), events,
lease: { id: `lease-${runId}`, companyId: "company", environmentId: "environment", heartbeatRunId: runId, status: "released", releasedAt: "2026-10-29T00:00:00Z" },
retirement: { captured: true, observedProcessCount: 3, remainingProcessCount: 0, identityChanged: false,
primaryOwner: { runId, pid: run.processPid as number, groupId: run.processGroupId as number, startedAt: run.processStartedAt as string, commandSha256: "c".repeat(64), perTurnArgvVerified: true as const },
nativeFencesReacquired: true, nativeFenceCount: 3 } };
}
/** Rust durable/state.rs hashes the ACPX executor's stable event identity. */
function executorSourceEventId(runnerInstanceId: string, seq: number) {
return `event_executor_${createHash("sha256").update("paperclip.executor-event.v1\0").update(runnerInstanceId)
.update("\0").update(`acpx_provider_${String(seq).padStart(16, "0")}`).digest("hex")}`;
}
export function antigravityEvidenceFixture(): AntigravityJourneyEvidence {
return { version: 1, setupId: "calibration", hosting: "self-hosted", targetCommit: "a".repeat(40), phase: "persisted", outcome: "passed", assisted: false,
privacy: "private", authentication: "saved-subscription", model: "gemini-3.8-flash-low", companyId: "company", agentId: "agent", connectionId: "connection",
checkpoints: Object.fromEntries(antigravityJourneyCheckpoints.map(key => [key, true])),
turnEvidence: [gradeAntigravityJourneyTurn(antigravityTurnFixture()), gradeAntigravityJourneyTurn(antigravityTurnFixture(true))],
cleanup: { agent: true, runs: true, connectionPreserved: true } };
}
tests/runner-e2e/antigravity-journey-ui.ts
+176
View File
@@ -0,0 +1,176 @@
import { expect, type Page } from "@playwright/test";
import { readFile } from "node:fs/promises";
import { ConnectionFailure } from "./connection-config.js";
/** Recent sidebar entries are bounded; the searchable roster contains every
* agent. Scope the exact visible name to the main roster, not sidebar recency. */
export async function openAntigravityAgentFromRoster(page: Page, agentName: string, timeout = 30_000) {
const navigation = page.locator("aside").getByRole("link", { name: "Agents", exact: true });
await expect(navigation).toHaveCount(1, { timeout });
await navigation.click({ timeout });
const toolbar = page.getByRole("toolbar", { name: "Agents controls", exact: true });
await toolbar.getByRole("textbox", { name: "Search agents", exact: true }).fill(agentName, { timeout });
const row = page.getByRole("main").getByRole("link").filter({ has: page.getByText(agentName, { exact: true }) });
await expect(row).toHaveCount(1, { timeout });
await row.click({ timeout });
}
/** Both sidebar presentations use ordinary task links: the coordinator uses
* the identifier and recent lists can repeat the same task or its immutable ID. */
export async function openAntigravityCreatedTask(page: Page, issue: { id: string; identifier?: string | null }, timeout = 30_000) {
const paths = [issue.id, issue.identifier].filter((value): value is string => Boolean(value))
.map(value => `a[href$="/issues/${encodeURIComponent(value)}"]:visible`);
// Every match is an exact alias of this captured task. Recents and expanded
// project lists may legitimately expose it more than once.
const task = page.locator("aside").locator(paths.join(", ")).first();
await expect(task).toHaveCount(1, { timeout });
await task.click({ timeout });
}
/** The ordinary picker is measured before any direct setup-route navigation. */
export async function selectAntigravityFromOrdinaryPicker(page: Page, timeout = 30_000) {
const dialog = page.getByRole("dialog", { name: "Choose an adapter", exact: true });
await expect(dialog).toBeVisible({ timeout });
const advanced = dialog.locator("details").filter({ has: page.locator("summary", { hasText: /^Advanced$/ }) });
if (await advanced.count()) await expect(advanced).not.toHaveAttribute("open", "", { timeout });
const radio = dialog.getByRole("radio", { name: /^Google Antigravity(?:\s+Candidate)?$/ });
await radio.scrollIntoViewIfNeeded({ timeout });
// The native input is visually hidden; its enclosing ordinary card is visible.
await expect(radio.locator("..")).toBeVisible({ timeout });
await radio.locator("..").click({ timeout });
await expect(radio).toBeChecked();
await expect(dialog.getByRole("button", { name: "Configure agent", exact: true })).toBeEnabled();
}
export async function assertAntigravityReady(page: Page, agentName: string, timeout = 120_000) {
const assign = page.getByRole("button", { name: `Assign ${agentName} a Task`, exact: true });
const finishing = page.getByRole("heading", { name: "Finishing agent setup", exact: true });
if (await finishing.isVisible()) await expect(assign).toBeDisabled({ timeout });
await expect(page.getByRole("heading", { name: "Your agent is ready", exact: true })).toBeVisible({ timeout });
await expect(assign).toBeEnabled({ timeout });
}
export async function assertAntigravityCleanComposer(page: Page, agentName: string, model: string, timeout = 30_000) {
const dialog = page.getByRole("dialog", { name: "New task", exact: true });
await expect(dialog).toBeVisible({ timeout });
await expect(dialog.getByTestId("new-issue-paused-assignee-note")).toBeHidden({ timeout });
await expect(dialog.getByRole("button", { name: "Select assignee", exact: true })).toContainText(agentName, { timeout });
const modelLabel = dialog.getByRole("button", { name: "Select model and effort", exact: true }).getByTestId("task-chat-composer-model-label");
await expect(modelLabel).toHaveText(model === "gemini-3.8-flash-low" ? /^(?:gemini-3\.8-flash-low|Gemini 3\.8 Flash \(Low\))$/ : model, { timeout });
}
export async function assertAntigravityTaskDone(page: Page, timeout = 30_000) {
const header = page.locator('[data-testid="issue-detail-header"]:visible');
await expect(header).toBeVisible({ timeout });
// The thread restores its bottom anchor after reload. Scrolling its entire
// large header fights that restoration and is unnecessary to read status.
const control = header.getByRole("button", { name: "Change status (current: Done)", exact: true });
await expect(control).toHaveCount(1, { timeout });
await expect(control).toBeVisible({ timeout });
const current = page.getByRole("navigation", { name: "breadcrumb", exact: true }).locator('[aria-current="page"]:visible');
if (await current.count()) {
await expect(current).toHaveCount(1, { timeout });
await expect(current.getByRole("img", { name: "Done", exact: true })).toBeVisible({ timeout });
}
}
export async function assertAntigravityFileActivity(page: Page, runId: string, timeout = 30_000) {
if (typeof runId !== "string" || !/^[a-zA-Z0-9_-]{1,128}$/u.test(runId)) throw new ConnectionFailure("browser_activity_run_identity_invalid");
// The shipped adapter prefixes both phase and item IDs with their run ID.
// Older successful rows must not satisfy this turn, or poison its checks.
const groups = page.locator(`section[data-testid="task-chat-activity-phase"][data-activity-group^="${runId}:"]`);
const histories = groups.getByTestId("task-chat-activity-phase-toggle");
const older = page.getByRole("button", { name: "More messages", exact: true });
await expect.poll(async () => await older.isVisible() || await histories.count() > 0, { timeout }).toBe(true);
if (await older.isVisible()) await older.click();
await expect.poll(() => histories.count(), { timeout }).toBeGreaterThan(0);
for (const toggle of await histories.all()) if (await toggle.getAttribute("aria-expanded") !== "true") await toggle.click();
const items = groups.locator(`li[data-activity-item-id^="${runId}:"]:visible`).filter({ has: page.locator("[data-activity-row]") });
await expect.poll(() => items.count(), { timeout }).toBeGreaterThanOrEqual(2);
await expect.poll(async () => {
let reads = 0, edits = 0;
for (const item of await items.all()) {
const row = item.locator("[data-activity-row]");
const family = await row.getAttribute("data-activity-family");
if (family !== null && family !== "tool_execution") continue;
let name: string | undefined;
if (family === null) {
// Persisted native tool_call/result entries use the legacy tool model.
// Its mono target is the adapter's summarized real call input.
const input = row.locator("span[title].font-mono");
if (await input.count() !== 1) continue;
const match = /^operation:\s*(read|edit|write),\s*target:\s*([^,]+)(?:,|$)/u.exec(await input.getAttribute("title") ?? "");
if (!match || match[2]!.trim() !== "antigravity-proof.txt") continue;
name = match[1]!;
}
const toggle = item.locator(":scope > button");
await expect(toggle).toHaveCount(1, { timeout });
if (await toggle.getAttribute("aria-expanded") !== "true") await toggle.click();
if (family === "tool_execution") {
const details = item.getByTestId("task-chat-provider-detail-list");
const value = (label: string) => details.locator(":scope > div").filter({ has: page.locator("dt").getByText(label, { exact: true }) }).locator("dd");
const operation = value("Operation");
if (await operation.count() !== 1) continue;
name = (await operation.innerText()).trim();
if (!["read", "edit", "write"].includes(name)) continue;
const target = value("Target");
await expect(target).toHaveCount(1, { timeout });
if ((await target.innerText()).trim() !== "antigravity-proof.txt") continue;
await expect(value("Status")).toHaveText("completed", { timeout });
} else {
// A non-latest legacy call can display completed-style copy while its
// phase is live. Require the visible settled-phase summary as well.
const phase = item.locator('xpath=ancestor::section[@data-testid="task-chat-activity-phase"][1]');
await expect(phase.locator("[data-completed-summary]")).toBeVisible({ timeout });
const result = item.getByTestId("task-chat-runner-activity-detail").locator("pre");
await expect(result).toBeVisible({ timeout });
await expect(result).toContainText(/\S/u, { timeout });
await expect(row.locator(":scope > span:nth-child(2) > span:first-child"))
.toHaveText(/^(?:Ran|Read|Edited|Wrote|Updated|Applied|Created)\b/u, { timeout });
}
await expect(row).toBeVisible({ timeout });
await expect(row).not.toContainText(/·\s*(?:failed|stopped)/u, { timeout });
await expect(row.locator('[data-conversation-tool-running="true"]')).toHaveCount(0, { timeout });
if (name === "read") reads++; else if (name === "edit" || name === "write") edits++;
}
return reads > 0 && edits > 0;
}, { timeout }).toBe(true);
}
/** Open the card bound to the independently observed attachment, then download
* through its real file viewer. A stale same-named first-turn card cannot pass. */
export async function inspectAntigravityArtifact(page: Page, attachmentId: string, expected: Buffer, timeout = 30_000) {
if (!/^[a-zA-Z0-9-]+$/u.test(attachmentId)) throw new ConnectionFailure("browser_attachment_identity_invalid");
const contentPath = `/api/attachments/${attachmentId}/content`;
const boundLink = page.locator(`a[href="${contentPath}"], a[href="${contentPath}?download=1"]`);
const cards = page.locator('article[data-testid="task-chat-rich-work-product-artifact"]:visible').filter({ has: boundLink });
// Work-product titles are human labels, for example "Antigravity Proof File".
// Their exact attachment link, not that title, binds the ordinary open action.
const opening = page.getByRole("button", { name: /^Open in tab: /u }).and(cards.getByRole("button", { name: /^Open in tab: /u }));
const rows = page.locator('[data-artifact-row]:visible, [data-slot="attachment"]:visible')
.filter({ has: page.getByRole("button", { name: "Open antigravity-proof.txt", exact: true }).or(page.getByRole("link", { name: "Open antigravity-proof.txt", exact: true })) })
.filter({ has: page.getByText(new RegExp(`(?:^|\\s)${expected.length} B$`, "u")) });
const boundRows = rows.filter({ has: boundLink });
const rowOpening = boundRows.getByRole("link", { name: "Open antigravity-proof.txt", exact: true });
const buttonOnly = rows.filter({ hasNot: page.locator("a[href]") }).getByRole("button", { name: "Open antigravity-proof.txt", exact: true });
await expect.poll(async () => await opening.count() + await rowOpening.count() + await buttonOnly.count(), { timeout }).toBeGreaterThan(0);
if (await opening.count()) {
await expect(opening).toHaveCount(1, { timeout });
const card = opening.locator('xpath=ancestor::*[self::article or @data-slot="card"][1]');
await expect(card.getByText(new RegExp(`(?:^|\\s)${expected.length} B$`, "u"))).toBeVisible({ timeout });
await opening.click({ timeout });
} else {
const control = await rowOpening.count() ? rowOpening : buttonOnly;
await expect(control).toHaveCount(1, { timeout });
await control.click({ timeout });
}
const raw = page.locator('pre[aria-label="antigravity-proof.txt raw text"]:visible');
await expect(raw).toHaveCount(1, { timeout });
await expect(raw).toHaveText(expected.toString("utf8"), { useInnerText: false, timeout });
const download = page.getByRole("link", { name: "Download antigravity-proof.txt", exact: true })
.and(page.locator(`a[download][href="${contentPath}?download=1"]`));
await expect(download).toHaveAttribute("href", `${contentPath}?download=1`, { timeout });
const [file] = await Promise.all([page.waitForEvent("download", { timeout }), download.click({ timeout })]);
const downloaded = await file.path();
if (!downloaded || !(await readFile(downloaded)).equals(expected)) throw new ConnectionFailure("browser_download_bytes_mismatch");
}
tests/runner-e2e/catalog.test.ts
+5 -5
View File
@@ -219,10 +219,10 @@ describe("runner E2E catalog", () => {
expect(localIntegrityTasks).toHaveLength(2);
expect(openRouterBreadthTasks).toHaveLength(3);
expect(runnerSuites.map((suite) => suite.expectedMatrixSize)).toEqual([
2, 1, 3, 2, 2, 4, 4, 4, 4, 4, 4, 8, 10, 6, 63, 12, 6, 4, 2, 4, 2, 30, 3, 16, 16, 2, 6, 8, 46, 2, 23, 15, 52, 6, 6, 20, 26, 52, 2, 28, 18, 2, 6, 6, 13, 10, 4, 48, 16, 10, 2, 1, 1, 4, 116, 2, 2,
2, 1, 3, 2, 2, 4, 4, 4, 4, 4, 4, 8, 10, 6, 63, 12, 6, 4, 2, 4, 2, 30, 3, 16, 16, 2, 6, 8, 46, 2, 23, 15, 52, 6, 6, 20, 26, 52, 2, 28, 18, 2, 6, 6, 13, 10, 4, 48, 16, 10, 2, 1, 1, 4, 116, 2, 1, 2,
]);
expect(validateRunnerCatalog()).toHaveLength(771);
expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(771);
expect(validateRunnerCatalog()).toHaveLength(772);
expect(new Set(runnerMatrix.map((entry) => entry.id)).size).toBe(772);
expect(
runnerMatrix.filter((entry) => entry.suite.id === "core-compatibility"),
).toHaveLength(48);
@@ -535,13 +535,13 @@ describe("runner E2E catalog", () => {
it("uses only declared secret references in generated payloads", () => {
expect(
runnerMatrix.filter(entry => !["provider_connection", "github_review_journey"].includes(entry.task.flow)).every((entry) =>
runnerMatrix.filter(entry => !["provider_connection", "github_review_journey", "antigravity_agent_journey"].includes(entry.task.flow)).every((entry) =>
entry.requiredCredentials.includes(entry.profile.credential),
),
).toBe(true);
expect(
runnerMatrix
.filter((entry) => entry.environment.id === "daytona" && !["provider_connection", "github_review_journey"].includes(entry.task.flow))
.filter((entry) => entry.environment.id === "daytona" && !["provider_connection", "github_review_journey", "antigravity_agent_journey"].includes(entry.task.flow))
.every((entry) =>
entry.requiredCredentials.includes("DAYTONA_API_KEY"),
),
tests/runner-e2e/catalog.ts
+4 -2
View File
@@ -20,6 +20,7 @@ import { nativeCompletionProfile, NATIVE_COMPLETION_BUDGET_CENTS } from "./nativ
import { chatConfirmationTasks } from "./chat-cases.js";
import { buildConnectionSuite } from "./connection-cases.js";
import { buildGitHubReviewJourneySuite } from "./github-review-cases.js";
import { buildAntigravityJourneySuite } from "./antigravity-journey-cases.js";
import { hiringTemplateTasks, hiringTemplateProfile, hiringTemplateDefinitionDigest } from "./hiring-template-cases.js";
import { copilotProviderDeathTasks } from "./copilot-active-stop-tasks.js";
import { instructionPersistenceTask } from "./instruction-persistence.js";
@@ -1829,6 +1830,7 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
},
buildConnectionSuite(runnerProfiles, runnerEnvironments),
buildGitHubReviewJourneySuite(runnerProfiles, runnerEnvironments),
buildAntigravityJourneySuite(runnerEnvironments),
{
id: "repository-skills", label: "Repository skill packages", manualOnly: true,
description: "Live private Poteto import and byte-for-byte delivery to local and Daytona agent runtimes.",
@@ -1904,8 +1906,8 @@ export function buildRunnerMatrix(
]),
],
requiredCredentials: [
...(["provider_connection", "github_review_journey"].includes(task.flow) ? [] : [profile.credential]),
...(!["provider_connection", "github_review_journey"].includes(task.flow) && environment.credential ? [environment.credential] : []),
...(["provider_connection", "github_review_journey", "antigravity_agent_journey"].includes(task.flow) ? [] : [profile.credential]),
...(!["provider_connection", "github_review_journey", "antigravity_agent_journey"].includes(task.flow) && environment.credential ? [environment.credential] : []),
...(task.requiredCredentials ?? []),
],
}))
tests/runner-e2e/daytona-image.test.ts
+9 -6
View File
@@ -11,6 +11,12 @@ import {
} from "./daytona-image-content.js";
const repositoryRoot = path.resolve(import.meta.dirname, "../..");
function lastDockerignoreRule(dockerignore: string, fixturePath: string) {
return dockerignore.split(/\r?\n/).filter(rule => rule && !rule.startsWith("#")).filter(rule => {
const pattern = rule.startsWith("!") ? rule.slice(1) : rule;
return path.matchesGlob(fixturePath, pattern) || fixturePath.startsWith(`${pattern}/`);
}).at(-1);
}
describe("runner E2E Daytona image contract", () => {
it("retains the active Copilot identity declaration in the Docker context", async () => {
@@ -24,11 +30,7 @@ describe("runner E2E Daytona image contract", () => {
expect(DAYTONA_IMAGE_INPUT_PATHS).toContain(fixturePath);
await expect(readFile(path.join(repositoryRoot, fixturePath), "utf8")).resolves.toContain('"commandDigest"');
// Docker's last matching rule wins, including the parent-directory exclusion.
const rules = dockerignore.split(/\r?\n/).filter(rule => rule && !rule.startsWith("#"));
const lastRule = rules.filter(rule => {
const pattern = rule.startsWith("!") ? rule.slice(1) : rule;
return path.matchesGlob(fixturePath, pattern) || fixturePath.startsWith(`${pattern}/`);
}).at(-1);
const lastRule = lastDockerignoreRule(dockerignore, fixturePath);
expect(lastRule?.startsWith("!")).toBe(true);
// The exception also covers the next version without exposing other live evidence.
const exception = lastRule!.slice(1);
@@ -146,9 +148,10 @@ describe("runner E2E Daytona image contract", () => {
"test-fixtures/pi-acp/profile-v19-identity.json",
"test/fixtures/copilot-profile-v16-identity.json",
]) {
expect(dockerignore).toContain(`!packages/paperclip-runner/${declaration}`);
expect(lastDockerignoreRule(dockerignore, `packages/paperclip-runner/${declaration}`)?.startsWith("!")).toBe(true);
await expect(readFile(path.join(repositoryRoot, "packages/paperclip-runner", declaration), "utf8")).resolves.toBeTruthy();
}
expect(lastDockerignoreRule(dockerignore, "packages/paperclip-runner/test-fixtures/pi-acp/private-live-session.json")?.startsWith("!")).toBe(false);
for (const developmentOnlyInput of [
"packages/paperclip-runner/devtools",
"packages/paperclip-runner/docs",
tests/runner-e2e/history-publish.ts
+1 -1
View File
@@ -121,7 +121,7 @@ export function publicScreenshotPaths(campaign: RunnerE2ECampaign) {
}
export function assertPublicCampaign(campaign: Pick<RunnerE2ECampaign, "results">) {
if (campaign.results.some(result => result.productJourney?.privacy === "private")) {
if (campaign.results.some(result => result.productJourney?.privacy === "private" || result.antigravityJourney?.privacy === "private")) {
throw new Error("Private Product E2E campaigns cannot enter the public history publisher");
}
}
tests/runner-e2e/history.test.ts
+8
View File
@@ -33,6 +33,7 @@ import { renderPublicCampaignSummary } from "./public-summary-image.js";
import { isPublicRunnerScreenshotRoute } from "./screenshot-policy.js";
import { connectionCheckpoints, type ConnectionEvidence } from "./connection-evidence.js";
import { githubJourneyCheckpoints, type GitHubReviewJourneyEvidence } from "./github-review-evidence.js";
import { antigravityEvidenceFixture } from "./antigravity-journey-test-fixtures.js";
import type { MatrixExecution, RunnerE2EResult } from "./types.js";
const temporaryDirectories: string[] = [];
@@ -72,6 +73,13 @@ function result(execution: MatrixExecution, status: "passed" | "failed") {
checkpoints: Object.fromEntries(githubJourneyCheckpoints.map(key => [key, true])),
cleanup: { endpoint: true, pullRequest: true, branch: true, app: true },
} satisfies GitHubReviewJourneyEvidence } : {}),
// Synthetic campaign calibration must satisfy the same strict retained
// contract as a live result; it is never provider qualification evidence.
...(execution.task.flow === "antigravity_agent_journey" ? {
issueId: "task", runIds: ["run-first", "run-second"],
antigravityJourney: { ...antigravityEvidenceFixture(), outcome: status },
screenshots: [{ id: "persisted", label: "Synthetic Antigravity persistence calibration", file: "synthetic-persisted.png", sha256: "a".repeat(64) }],
} : {}),
} satisfies RunnerE2EResult;
}
tests/runner-e2e/journey-features.ts
+4
View File
@@ -20,6 +20,10 @@ function associate(suites: readonly string[], cases: Record<string, readonly str
associate(["product-journeys"], { "task-create": ["tasks/create"], "ask-question": ["questions-and-approvals/questions"],
"task-properties": ["tasks/properties"], "cloud-onboarding": ["onboarding/first-task"] });
associate(["antigravity-agent-journey"], {
"saved-subscription-two-turn": ["agents/hire", "agents/instructions", "runs-adapters/validation", "runs-adapters/session",
"tasks/create", "tasks/completion", "documents-artifacts/attachments", "documents-artifacts/work-products"],
});
associate(["core-compatibility", "openrouter-model-breadth", "extended-harnesses", "grok-qualification", "grok-subscription-qualification"], {
"message-marker": ["runs-adapters/runs"],
"hello-complete": ["runs-adapters/runs"],
tests/runner-e2e/launch.ts
+8
View File
@@ -1086,6 +1086,14 @@ async function main() {
}
if (process.env.PAPERCLIP_E2E_OPERATIONS_MANIFEST && executions.some(execution => execution.task.flow === "provider_connection")) throw new Error("Provider connection campaign has no budgeted routing qualification");
if (executions.some(execution => execution.task.flow === "antigravity_agent_journey")) {
if (options.connectionConfig || options.githubReviewConfig || options.provisionGitHubRepository) throw new Error("Use --antigravity-journey-config for the Antigravity journey.");
const { runAntigravityJourneyCampaign } = await import("./antigravity-journey-launch.js");
await runAntigravityJourneyCampaign({ executions, catalog: runnerMatrix, configFile: options.antigravityJourneyConfig, repositoryRoot,
preflight: options.antigravityJourneyPreflight, recoverFile: options.antigravityJourneyRecover });
return;
}
if (options.antigravityJourneyConfig || options.antigravityJourneyPreflight || options.antigravityJourneyRecover) throw new Error("Antigravity journey options require its explicit registered cell.");
if (executions.some(execution => execution.task.flow === "github_review_journey")) {
if (options.connectionConfig) throw new Error("Use --github-review-config for the GitHub journey.");
const { runGitHubReviewCampaign } = await import("./github-review-launch.js");
tests/runner-e2e/pi-controls-evidence.test.ts
+3 -3
View File
@@ -188,8 +188,8 @@ describe("Pi controls catalog admission", () => {
// Extended v3 states the strict file oracle's task-wide Bash limit.
const pi = runnerMatrix.find(c => c.profile.qualificationCandidate === "pi")!.profile;
expect(pi.modelQualification?.qualificationId).toBe("pi:0.0.33:1.0.0:openrouter");
expect(runnerSuites.find(s => s.id === "pi-native")!.definitionMetadata).toMatchObject({ version: 23, remoteProcExit: "separately-confirmed-absence-after-read-failure", taskCreation: "explicit-title-and-creation-response-id", agentMemoryParent: "public-managed-file-seed-before-admission", incompleteTerminalCleanup: "retirement-retained-with-failed-watch", profileVersion: 30, agentMemoryContent: "utf8-nonce-plus-final-lf", agentMemoryPrompt: "single-json-write-and-content-bound-native-read-both-runs", agentMemoryReadAuthority: "local-withheld-or-exact-remote-agent-run-file", taskPromptTransport: "fenced-markdown-paste-and-multiline-literal-escapes", nativeFinish: "current-contract-objective-evidence-refs", providerFaultExecutable: "stable-preinstalled-runner-link-and-snapshot-node-inode-with-held-bootstrap-fd-3-or-7" });
expect(runnerSuites.find(s => s.id === "pi-controls")!.definitionMetadata).toMatchObject({ version: 10, remoteProcExit: "separately-confirmed-absence-after-read-failure", taskCreation: "explicit-title-and-creation-response-id", profileVersion: 30,
expect(runnerSuites.find(s => s.id === "pi-native")!.definitionMetadata).toMatchObject({ version: 23, remoteProcExit: "separately-confirmed-absence-after-read-failure", taskCreation: "explicit-title-and-creation-response-id", agentMemoryParent: "public-managed-file-seed-before-admission", incompleteTerminalCleanup: "retirement-retained-with-failed-watch", profileVersion: 35, agentMemoryContent: "utf8-nonce-plus-final-lf", agentMemoryPrompt: "single-json-write-and-content-bound-native-read-both-runs", agentMemoryReadAuthority: "local-withheld-or-exact-remote-agent-run-file", taskPromptTransport: "fenced-markdown-paste-and-multiline-literal-escapes", nativeFinish: "current-contract-objective-evidence-refs", providerFaultExecutable: "stable-preinstalled-runner-link-and-snapshot-node-inode-with-held-bootstrap-fd-3-or-7" });
expect(runnerSuites.find(s => s.id === "pi-controls")!.definitionMetadata).toMatchObject({ version: 10, remoteProcExit: "separately-confirmed-absence-after-read-failure", taskCreation: "explicit-title-and-creation-response-id", profileVersion: 35,
remoteProcessIdentity: "observer-pid-startTicks-bootId",
controlPlaneSettlement: "required-scoped-result-and-terminal-after-runner" });
expect(runnerSuites.find(s => s.id === "extended-harnesses")!.definitionMetadata).toMatchObject({ version: 6, piFileArtifactTitle: "exact-filename", piFileCommandTransport: "fenced-bash-markdown-paste" });
@@ -199,7 +199,7 @@ describe("Pi controls catalog admission", () => {
}
const hashes = Object.fromEntries(runnerSuites.filter(s => ["pi-native", "native-active-stop", "extended-harnesses", "rich-acp-warm-continuity"].includes(s.id)).map(s => [s.id, suiteDefinitionHash(s)]));
expect(hashes).toEqual({
"pi-native": "04d49c6e098c24091d2939108a97722d1b8353e39192b87e98ee33377f183001",
"pi-native": "8107706b60dd20aa34856728be19f3a8bf25bc21812587b3f115a4fe57da3972",
"native-active-stop": "4654a2169bff96cf8377fdd62441f5cabea5c613f447e518830f6b1efd82c051",
"rich-acp-warm-continuity": "7e8a0012f2f67fba03cd94403b096a07cfc95cdb7aa568b4c3fd18b852d735f3",
"extended-harnesses": "d5bcf5de0ade3a54e4e1560e76feb5d8a787039d56d2379cd501626d85ed7418",
tests/runner-e2e/playwright-support.config.ts
+1 -1
View File
@@ -3,7 +3,7 @@ import { defineConfig } from "@playwright/test";
/** Browser-only harness regressions: no Paperclip instance or provider credentials. */
export default defineConfig({
testDir: ".",
testMatch: ["screenshot-readiness.spec.ts", "service-worker-reload.spec.ts", "lost-send.spec.ts", "chat-restart.spec.ts", "settings-toggle.spec.ts", "browser-bootstrap-diagnostics.spec.ts", "service-worker-module-loading.spec.ts", "browser-bootstrap-recovery.spec.ts", "blocker-input.spec.ts", "native-blocker-visible.spec.ts", "copilot-permission-card.spec.ts", "github-review-check.spec.ts"],
testMatch: ["screenshot-readiness.spec.ts", "service-worker-reload.spec.ts", "lost-send.spec.ts", "chat-restart.spec.ts", "settings-toggle.spec.ts", "browser-bootstrap-diagnostics.spec.ts", "service-worker-module-loading.spec.ts", "browser-bootstrap-recovery.spec.ts", "blocker-input.spec.ts", "native-blocker-visible.spec.ts", "copilot-permission-card.spec.ts", "github-review-check.spec.ts", "antigravity-journey-browser.spec.ts"],
workers: 1,
retries: 0,
timeout: 10_000,
tests/runner-e2e/result-validation.ts
+11
View File
@@ -2,6 +2,7 @@ import { agentJourneyRuns, journeyRequiredChecks } from "./product-journeys/succ
import { validateWorkerTiming } from "./worker-timing.js";
import { connectionCheckpoints, connectionEvidencePasses, connectionRunSignals } from "./connection-evidence.js";
import { githubJourneyCheckpoints, gradeGitHubReviewJourney } from "./github-review-evidence.js";
import { gradeAntigravityJourney, validateAntigravityJourneyEvidence } from "./antigravity-journey-evidence.js";
import { STARTUP_REUSE_LAYERS, STARTUP_REUSE_REASONS } from "../../packages/adapter-utils/src/startup-reuse.js";
import type { RunnerE2ECampaignAccounting, RunnerE2EResult } from "./types.js";
import { DEPENDABLE_CELLS } from "./dependable-qualification.js";
@@ -397,6 +398,7 @@ const fields = {
scores: array(shape({ dimension: oneOf("questionRelevance", "useOfFacts", "proposalUsefulness", "clarity", "lowFriction"), score: integer, rationale: string, evidence: array(string) })),
inputTokens: nullable(integer), outputTokens: nullable(integer), estimatedCostUsd: nullable(number), reservedCostUsd: number, recordedAt: date, error: optional(string),
})),
antigravityJourney: optional((value, at) => { try { validateAntigravityJourneyEvidence(value); } catch { invalid(at); } }),
githubReviewJourney: optional(closedShape({ version: integer, outcome: oneOf("running", "passed", "failed", "awaiting_user", "missing_credential", "blocked_target"),
phase: string, setupId: string, hosting: oneOf("self-hosted", "paperclip-cloud"), targetCommit: nullable(string), assisted: boolean,
dependencies: array(closedShape({ alias: string, consumer: oneOf("harness", "browser", "product-agent"), status: oneOf("ready", "unverified", "created-in-journey"), source: oneOf("onepassword", "environment", "browser-session", "product-connection") })),
@@ -439,4 +441,13 @@ export function validateRetainedRunnerResult(
}
if (checked.suiteId === "provider-connections" && checked.status === "passed" && (!checked.providerConnection || !connectionEvidencePasses(checked.providerConnection))) invalid("result.providerConnection");
if (checked.suiteId === "github-review-journey" && checked.status === "passed" && (!checked.githubReviewJourney || checked.githubReviewJourney.outcome !== "passed" || !checked.githubReviewJourney.targetCommit || !gradeGitHubReviewJourney(checked.githubReviewJourney).every(check => check.passed))) invalid("result.githubReviewJourney");
if (checked.suiteId === "antigravity-agent-journey" && checked.status === "passed") {
const evidence = checked.antigravityJourney;
if (!evidence || evidence.outcome !== "passed" || evidence.assisted || !evidence.targetCommit || !gradeAntigravityJourney(evidence).every(check => check.passed)
|| checked.cleanup !== "passed" || checked.runtimeMode !== "native" || checked.model !== evidence.model || checked.provider !== "acpx"
|| checked.runIds?.length !== 2 || !evidence.turnEvidence.every(turn => checked.runIds!.includes(turn.runId))
|| !evidence.companyId || !evidence.agentId || !evidence.connectionId || !checked.issueId
|| !evidence.turnEvidence.every(turn => turn.companyId === evidence.companyId && turn.agentId === evidence.agentId && turn.connectionId === evidence.connectionId && turn.issueId === checked.issueId)
|| !checked.screenshots?.some(shot => shot.id === "persisted" && /^[a-f0-9]{64}$/u.test(shot.sha256 ?? ""))) invalid("result.antigravityJourney");
}
}
tests/runner-e2e/selectors.ts
+13 -1
View File
@@ -20,6 +20,9 @@ export interface RunnerSelectorOptions {
connectionConfig?: string;
githubReviewConfig?: string;
provisionGitHubRepository?: boolean;
antigravityJourneyConfig?: string;
antigravityJourneyPreflight?: boolean;
antigravityJourneyRecover?: string;
}
export class RunnerSelectorError extends Error {}
@@ -69,6 +72,15 @@ export function parseRunnerSelectors(
index += 1;
}
else if (flag === "--provision-github-repository") options.provisionGitHubRepository = true;
else if (flag === "--antigravity-journey-config") {
options.antigravityJourneyConfig = valueFor(args, index, flag);
index += 1;
}
else if (flag === "--antigravity-journey-preflight") options.antigravityJourneyPreflight = true;
else if (flag === "--antigravity-journey-recover") {
options.antigravityJourneyRecover = valueFor(args, index, flag);
index += 1;
}
else if (flag === "--max-parallel") {
const value = valueFor(args, index, flag);
index += 1;
@@ -214,7 +226,7 @@ export function selectRunnerExecutions(
export function buildMatrixJobs(
executions: readonly MatrixExecution[],
): MatrixJob[] {
if (executions.some(e => ["provider_connection", "github_review_journey"].includes(e.task.flow))) throw new RunnerSelectorError("Attended connection journeys require explicit private setup profiles; they are not admitted to paid CI matrix jobs yet.");
if (executions.some(e => ["provider_connection", "github_review_journey", "antigravity_agent_journey"].includes(e.task.flow))) throw new RunnerSelectorError("Attended connection journeys require explicit private setup profiles; they are not admitted to paid CI matrix jobs yet.");
return executions
.map((execution) => ({
executionId: execution.id,
tests/runner-e2e/types.ts
+2
View File
@@ -27,6 +27,7 @@ export type RunnerTaskFlow =
| "repository_skills"
| "provider_connection"
| "github_review_journey"
| "antigravity_agent_journey"
| "plan_task_guidance"
| "blocker_guidance"
| "public_mcp"
@@ -368,6 +369,7 @@ export interface RunnerE2EResult {
taskChat?: import("./task-chat.js").TaskChatEvidence;
providerConnection?: import("./connection-evidence.js").ConnectionEvidence;
githubReviewJourney?: import("./github-review-evidence.js").GitHubReviewJourneyEvidence;
antigravityJourney?: import("./antigravity-journey-evidence.js").AntigravityJourneyEvidence;
completionQuality?: import("./completion-quality.js").CompletionQualityRecord[];
firstTaskQuality?: import("./first-task-quality.js").FirstTaskQuality;
/** External assistant API calls, separate from the team's heartbeat executions. */