Files
PaperClipAI/server/src/__tests__/antigravity-ai-runtime.test.ts
T
DottaandClaude Opus 5.5 4879a146fb fix(secrets): stop credential reads from aborting concurrent budget admissions (#21452)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Paid model calls (company decisions, fast responses, browser use,
prompt-injection checks) pass a budget admission before they start
> - Admission runs in a REPEATABLE READ transaction and share-locks the
credential's secret row, so a key rotation cannot slip between admission
and the call
> - Every credential read also wrote "Last used" (`last_resolved_at`,
`updated_at`) to that same row
> - When two calls ran at once, one call's credential read aborted the
other's admission with a serialization failure (40001). The retry then
saw the first call's new budget hold and denied the work as
`budget_blocked`
> - This is the flaky `decision-models.test.ts` failure that held Cloud
readiness, and so releases, on 2026-11-13
> - This pull request stops credential reads from writing the secret
row. "Last used" now comes from the access event that every read already
records
> - The benefit is that concurrent paid calls are no longer wrongly
denied or retried, and the release gate no longer fails on this test

## Linked Issues or Issue Description

No public issue exists. The bug is described here.

**What happened?**
Cloud readiness `verify / General tests (server (4/10))` failed on
master commits 6917d0568d and 5ede64f247. The test "admits concurrent
work from the same budget observation and records both obligations"
reported `expected "vi.fn()" to be called 2 times, but got 1 times`.
#21387 replaced the 1-second poll with an unbounded wait. That does not
fix it: one decision really is denied, so the test now waits until the
test timeout.

**Root cause**
I logged the accounting retry loop under load. Each batch of 50 runs had
6 to 10 retries with `could not serialize access due to concurrent
update` on `select … from company_secrets … for share`. The other
decision's credential read had updated the row. In 4 of 5 batches, the
retried admission saw the other decision's committed reservation and
returned `budget_blocked`.

**Expected behavior**
A credential read does not change admission. Two decisions admitted from
the same budget observation both run and both record their obligation.

**Steps to reproduce**
Run the updated test on the old `secrets.ts`. The test makes the second
credential read land after the first admission's snapshot. It then
commits an unrelated hold after both observations. The old code fails
every run with `first settled before its provider call:
{"status":"unavailable","reason":"budget_blocked"}`.

## What Changed

- `secretService` no longer writes `last_resolved_at` / `updated_at`
when it resolves a value. It still records the `secret_access_events`
row, as before.
- `secretService.list` reports `lastResolvedAt` as the later of the
stored column (legacy values) and the newest successful access event. It
reads one row per secret, backwards on the existing `(secret_id,
created_at)` index.
- A read without a consumer context (for example a GitHub API credential
read) now records a `system` access event, so it still counts as "Last
used".
- The removed `UPDATE` also made a credential read wait for a concurrent
rotation to commit. `credential()` in `ai-connections.ts` depends on
that wait to re-read the rotated version instead of handing a run a
consumed single-use token. Its existing version re-read now takes `FOR
SHARE`, so it keeps that wait. `prepareManagedAiRuntime` has its own
before/after freshness read around `runtimeCredential()`; that read
takes `FOR SHARE` too. A share lock creates no new row version and
cannot abort an admission. It adds no statement.
- The first version of this PR throttled the write to once a minute.
Review found that a read crossing the minute could still abort an
admission, so the write is removed instead.
- The concurrent-admission test is now deterministic. It starts decision
A and waits until A is blocked, then starts decision B and waits again.
It commits an unrelated 1-cent hold after both observations, then checks
that both decisions are admitted. A decision that settles before its
provider call fails the test immediately with its result, instead of
hanging.

## Verification

- Red: the new test against the old `secrets.ts` fails on every run with
the `budget_blocked` result above.
- Green: the same test with `repeats: 49` (50 runs) passes 50/50 locally
on a machine at load average 20 to 60.
- Before the change, 3 parallel batches of 50 runs of the old test each
produced a false denial.
- `npx tsc --noEmit -p server`: no errors.
- New `secrets-service.test.ts` test: a resolution leaves the secret row
unchanged, and `list()` still shows "Last used".
- `ai-connections.test.ts` quota test now checks that a read leaves
`updated_at` unchanged and records a success access event.

## Risks

- `updated_at` on a secret no longer moves when the secret is read. It
still moves on every real change.
- "Last used" on the Secrets page costs one indexed lookup per listed
secret.
- Only `list()` derives "Last used". Other endpoints that return a
secret row show the stored column, which is now set only by older
releases.
- No migration. No API change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 19:46:50 -06:00

66 lines
4.1 KiB
TypeScript

import { afterEach, describe, expect, it, vi } from "vitest";
import { access, readFile, stat } from "node:fs/promises";
import path from "node:path";
import type { Db } from "@paperclipai/db";
const account = vi.hoisted(() => {
const refs = [{ configPath: "ai.credential", secretId: "22222222-2222-4222-8222-222222222222", versionSelector: "latest" }];
return {
refs,
credential: "fixture-selected-google-key",
selection: {
grant: { id: "11111111-1111-4111-8111-111111111111", subjectUserId: "alice", credentialSecretRefs: refs },
connection: { id: "33333333-3333-4333-8333-333333333333", name: "Google AI fixture", config: { ai: { provider: "google", method: "api_key" } } },
attribution: { connectionId: "33333333-3333-4333-8333-333333333333", grantId: "11111111-1111-4111-8111-111111111111",
provider: "google", method: "api_key", mode: "responsible_user", responsibleUserId: "alice" },
},
};
});
// This fixture exercises real private-home preparation and deletion. The DB-backed
// ai-connections suite separately checks membership, selected grants and revocation.
vi.mock("../services/ai-connections.js", () => ({ aiConnectionService: () => ({
select: async () => account.selection,
runtimeCredential: async () => account.credential,
}) }));
vi.mock("../services/subscription-refresh.js", () => ({ refreshSubscriptionConnection: vi.fn() }));
import { managedAiSessionFingerprintConfig, prepareManagedAiRuntime } from "../services/ai-connection-runtime.js";
describe("Antigravity private managed AI home", () => {
afterEach(() => vi.unstubAllEnvs());
it("uses the selected key and explicit native auth settings, then deletes both private homes", async () => {
vi.stubEnv("GEMINI_HOME", "/ambient/google-home");
vi.stubEnv("GOOGLE_APPLICATION_CREDENTIALS", "/ambient/google-credentials.json");
const version = [{ epoch: 1, version: 1 }];
const db = { select: (fields: Record<string, unknown>) => ({ from: () => ({
where: () => Object.assign(Promise.resolve("refs" in fields ? [{ refs: account.refs, status: "active" }] : version),
{ limit: async () => version, for: async () => version }),
}) }) } as unknown as Db;
const input = {
companyId: "44444444-4444-4444-8444-444444444444", agentId: "55555555-5555-4555-8555-555555555555", responsibleUserId: "alice",
adapterType: "paperclip_runner", binding: { provider: "google", method: "api_key", mode: "responsible_user" } as const,
config: { provider: "acpx", acpxAgent: "antigravity", model: "caller-exact-model", env: {
GEMINI_API_KEY: "fixture-ambient-key", GOOGLE_API_KEY: "fixture-other-key", GEMINI_HOME: "/ambient/configured-home",
GOOGLE_APPLICATION_CREDENTIALS: "/ambient/configured-credentials.json", GOOGLE_CLOUD_ACCESS_TOKEN: "fixture-oauth-token",
} },
};
const first = await prepareManagedAiRuntime(db, input);
const second = await prepareManagedAiRuntime(db, input);
try {
expect(first.config).toMatchObject({ provider: "acpx", acpxAgent: "antigravity", model: "caller-exact-model", env: {
GEMINI_API_KEY: account.credential, GOOGLE_API_KEY: "", GOOGLE_APPLICATION_CREDENTIALS: "", GOOGLE_CLOUD_ACCESS_TOKEN: "",
} });
expect(first.home).not.toBe(second.home);
expect(first.config.env.GEMINI_HOME).toBe(path.join(first.home!, "provider"));
const settingsPath = path.join(String(first.config.env.GEMINI_HOME), "antigravity-acp", "settings.json");
expect(JSON.parse(await readFile(settingsPath, "utf8"))).toEqual({ auth: { type: "gemini-api-key" } });
expect((await stat(settingsPath)).mode & 0o777).toBe(0o600);
expect(managedAiSessionFingerprintConfig(first.config, first.home)).toEqual(managedAiSessionFingerprintConfig(second.config, second.home));
expect(input.config.env.GEMINI_HOME).toBe("/ambient/configured-home");
} finally { await Promise.all([first.cleanup(), second.cleanup()]); }
await expect(access(first.home!)).rejects.toMatchObject({ code: "ENOENT" });
await expect(access(second.home!)).rejects.toMatchObject({ code: "ENOENT" });
});
});