mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-08 12:33:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server has about 1,100 test and helper files in `server/src/__tests__` > - `server/tsconfig.json` excludes that directory, Vitest does not check types, and no script runs the tsconfig in that directory > - So a test can call a function with the wrong arguments and still pass, and no tool reports it > - This gap hid 2,168 real type errors, and some of them hid tests that did not test what they claim > - This pull request fixes all of these errors and adds the directory to the server layers from paperclipai/paperclip#18737 > - The benefit is that a type error in a server test now fails the typecheck, with no baseline and no backlog ## Linked Issues or Issue Description Refs paperclipai/paperclip#18737, which is merged and added the server layers. Refs paperclipai/paperclip#18738, which is merged and checks the test projects outside `pnpm -r`, so it is not a duplicate. Refs paperclipai/paperclip#18137, which records that this directory has no check. **What happened?** No tool checked the types of 1,074 of the 1,101 files in `server/src/__tests__`. I searched `package.json`, the workflows, and all tsconfig files. 27 files are in the program of another tsconfig, and these 27 have 0 errors. 11 of the 27 are in `helpers/maintenance/tsconfig.json`. No script ran that project when I measured this. paperclipai/paperclip#18738 now runs it. **Expected behavior** The server typecheck checks these files, and a type error fails the check. **Steps to reproduce** 1. On the base commit, run `tsc -p server/src/__tests__/tsconfig.json --noEmit`. It reports 6,536 errors and needs 12.3 GiB. 2. Add a type error to any file in `server/src/__tests__`. 3. Run `pnpm --filter @paperclipai/server typecheck`. It passes. **Paperclip version or commit** I measured the error breakdown below on commit `8dfd632452`, with TypeScript 7.0.2. The branch now includes master up to paperclipai/paperclip#18865. **Deployment mode** Not applicable. This issue applies to development and CI. No production source file in the server changes. ### Error breakdown Each row adds one change to the row above. | Step | Total errors | Change | Cause | | --- | --- | --- | --- | | Config as committed | 6,536 | | | | Add `src/types/*.d.ts` | 4,447 | -2,089 | Config. The Express request types were missing. | | Set `rootDir` to the repository root | 2,436 | -2,011 | Config. | | Set `allowImportingTsExtensions` | 2,181 | -255 | Config. Vitest accepts `.ts` imports. | | Add `ES2024.Promise` to `lib` | 2,138 | -43 | Config. Node 24 has `Promise.withResolvers`. | | Set `allowJs` for `.mjs` imports | 2,148 | +10 | Config. It removes 8 `TS7016` errors and shows new real errors. | | Set `declaration` to false | 2,142 | -6 | Config. Tests emit nothing. | | Count only files in `server/src` | 2,058 | -84 | Other packages: 53 Daytona, 29 Runner test, 2 UI. | So 4,478 of the 6,536 errors (69%) came from config or scope, and 2,058 are real type errors in 314 files. The 29 Runner test errors are also real, which gives 2,087 errors. Tests that master added after this measurement had 81 more errors, so this pull request fixes 2,168 errors. This table classifies the first 2,058 errors. | Kind of real error | Errors | | --- | --- | | Wrong argument or assignment type | 1,023 | | Stale or partial fixture objects | 525 | | Property that the type does not have | 317 | | Missing narrowing of null or unknown | 101 | | Implicit any | 60 | | Missing name or module | 16 | | Other | 16 | ### What the type errors hid Most errors were stale fixtures and loose mocks. These errors hid real test problems, and this pull request fixes them: - **A test that never ran.** `instrumentation.test.ts` required `sdk-trace-base`, which is not installed. So 3 span tests were skipped. They now run and pass. - **Assertions that could not fail.** - `issue-thread-interactions-service` used a company ID as an issue ID. - `status-card-evaluation-work` compared `[]` with `[]` in a rollback check. - `slack-conversation-lifecycle` read `item.issueId`, which does not exist. - `email-channels` used a permission key that does not exist. - `heartbeat-project-env` imported `LOW_TRUST_REVIEW_PRESET`, which is not exported. The value was undefined. - **Postgres processes that did not stop.** 11 test files called `stop` on the test database helper. That method does not exist, and the helper method is `cleanup`. - **Arguments that the service ignored.** - `methodKey` is now `connectionMethodKey` in the Daytona, Unikraft, 1Password, and Railway tests. - 108 calls passed `"owner-user"` to a parameter that expects an options object. - 9 chat wizard fixtures passed `managementContext` as `source`, so they ran `existing_app` and not `manifest`. - 2 tests passed a `{ userId }` actor that the function ignores. - Inserts into `environments` passed a `companyId` column that the table does not have. - **Problems in tests that master added during this work.** - `sandbox-creation-operations` read `database.connectionString` when an external test database was set. In that mode `database` is undefined, so the fixture would stop with a TypeError. It now uses the external URL. - `idle-completed-history` set a Dot operation to `dispatched`. That is a Muse status, and the Dot type does not have it. The fixture now uses `admitted`, which is a real in-flight Dot status. ### Possible product problems, not changed These may be real problems in production code or types. I did not change them, because this pull request does not change production behavior: - `decisions-service` `service.create` does not validate the lenient `cancel_issue_tree` input. - `createConnection` ignores `input.connectionPurpose`. - The `createConnection` route schema default stores `{}` for `transportConfig`. - `prepareManagedAiRuntime` accepted the invalid value `mode: "connection"`. - `createBinding` accepts invalid `projectionClass` strings outside class 3. - `ensureManagedProjectWorkspace` forwards an untyped `repoRef` by spread. - `routineVariableSchema.label` is optional, but `RoutineVariable.label` is required. - `ToolRunContext.projectId` may be too strict. - `withdrawInteraction` reads `issue.status`, which its type omits. - `issueService.list` and `reconcileNativeFinalizations` omit fields that they return at runtime. - `createGitHubExternalObjectProvider` handles `tokenProvider: null`, but its type forbids this value. ## What Changed - Fixed the 2,168 type errors. The diff changes 322 test and helper files. - Corrected `server/src/__tests__/tsconfig.json`. Editors and the typecheck now read the same options and report the same errors. - Added three `test-tree` layers to `server/scripts/typecheck-layers.mjs`. They read the server layers as declarations, and any error fails them. A hash of the path selects the shard, so a new file does not move other files. - Added `server/scripts/typecheck-stubs/daytona-sdk.d.ts`. Server tests reach Daytona plugin source, and the root install does not have `@daytonaio/sdk`. - Changed two imports in `ui/src/lib/issue-execution-policy.ts` to use `.js` extensions, because server tests import this file. - Made `declarationsConfig` and `--declarations-only` skip the `test-tree` layers, because these layers emit nothing. Other projects read only the source layers. Added a test for this. - Updated `doc/DEVELOPING.md`. ## Verification Run these commands: ```sh pnpm --filter @paperclipai/server typecheck pnpm exec vitest run server/scripts/typecheck-layers.test.mjs ``` Results on macOS (arm64), Node 24, 16 cores. Each check ran after the merge of paperclipai/paperclip#18738 unless the row says otherwise: | Check | Commit | Result | | --- | --- | --- | | Server typecheck, `PAPERCLIP_TYPECHECK_LOW_MEMORY=1`, all 7 layers | `6e45a08573` | Exit 0, 0 errors, 62 seconds, peak 3.52 GiB | | `node scripts/typecheck-test-projects.mjs --low-memory` | `a3306bd13e` | Exit 0, 0 errors, 42 seconds, peak 3.25 GiB | | The same script with `--reuse-declarations` | `1b878aabc0` | Exit 0, 0 errors | | `typecheck-layers.test.mjs` and `live-recovery-work.test.ts` | `01db5634e4` | 90 passed. These include real compiler runs for a pass, a type error, and an invalid tsconfig. | | `tsconfig.declarations.json` after a full server typecheck | `0fb1e4fb0a` | It references only `below`, `cycle`, and `above`. No generated config names a `tsconfig.layers.json` file. | | Owned runtime test in `idle-completed-history` | `e8f2f448ac` | Passed, with the fix that master later replaced | | `idle-incoming-wake`, `plugin-idle-manager`, and the changed `chat-channels` test | `6e45a08573` | 30 passed | | Runner `native-execution.test.ts` | `eab4dcc1f4` | 114 passed | | The 19 server test files that changed since `eab4dcc1f4`, and `typecheck-layers.test.mjs` | `259737b700` | 4,275 of 4,277 passed. The 2 failures are in `chat-channels` and are in the list of 7 below. | The peak is the largest single process, from `/usr/bin/time -l`. Peaks for the server typecheck were between 3.42 GiB and 3.56 GiB across 8 runs. Peaks for the test projects were 3.25 GiB and 3.37 GiB across 2 runs. I did not measure a cgroup. **Runtime check of the fixes.** Types do not prove that a test still works. So I ran all 309 changed server test files, 11,929 tests, before the rebase onto paperclipai/paperclip#18737: | Result | Tests | Detail | | --- | --- | --- | | Passed | 11,913 | Includes the 2 tests and the 54 `dot-runner` tests that passed in a later run, as described below. | | Failed | 7 | The same 7 tests also fail on the commit before the fixes. | | Skipped | 9 | These tests have their own skip conditions: `sentry` 5, and 1 each in `bundled-plugin-registration`, `customer-success-cloud`, `onepassword-connection`, and `workspace-runtime-https-live-exercise`. | The 7 tests that fail with and without this change, on my machine: - 3 tests in `chat-channels` and `email-channels` stop with `ENOENT` on a `SKILL.md` file. The path is in the parent checkout, above the git worktree that I used, so I expect that the worktree is the cause. I did not prove this. - `workspace-instance-cleanup` compares `/private/var/...` with `/var/...`. This is a macOS symlink difference. - 2 tests in `git-reproducible-managed-checkouts` get `unknown_base_git_files` and expect `unpushed_managed_checkout`. I did not find the cause. - 1 more test in `chat-channels`, `materializes direct external-chat finals once`, gets `undefined`. I did not find the cause. CI is the check for these 7 tests. `ci / verify` passed on `eab4dcc1f4`. Failures during this work: - My fix to `agent-runtime-connector-access` changed an invalid permission key in the setup, but not in the assertion. The runtime check found this. I corrected the assertion, and the file passes. - `heartbeat-plugin-environment` failed one wait in the full run. It passed three times alone, so I think that load caused the failure. It can fail again under load. - `dot-runner` skipped all 54 tests in the full run because `cargo` was not on my `PATH`. It then passed 54 of 54 with the toolchain on the `PATH`. - My first `lib` value removed the default DOM types, and the error count went up by 375. I added the DOM libraries. - My Daytona stub with bare `any` left 48 errors, because generic helpers then inferred `unknown`. I added shapes that return promises. - One fix to `agent-secrets-routes` used `class_3_static_lease`, and the test failed. That class allows only specific paths. I reverted to `unclassified`. - `ci / General tests (server (8/12))` failed on `259737b700` because of the flaky port in `idle-completed-history`. paperclipai/paperclip#18853 fixed this test on master, so this pull request keeps that fix. **Audit of the diff.** I counted these patterns in added and removed lines, to find fixes that only hide an error: | Pattern | Added | Removed | | --- | --- | --- | | `@ts-ignore` | 0 | 0 | | `@ts-expect-error` | 0 | 1 | | `.skip` | 0 | 0 | | `as any` | 9 | 9 | | `: any` | 11 | 2 | | `as never` | 8 | 2 | | `as unknown as` | 35 | 4 | | `Promise<unknown>` | 16 | 5 | | `expect(` | 128 | 113 | No hunk removes more `expect(` calls than it adds. Three removals of test content were intentional, and I checked each one: - `muse-control` removed `signalToken`. Line 1012 already asserts that it is absent. - `merged-pr-confirmation-sweep` removed legacy aliases. Other tests cover them. - `heartbeat-workspace-branch-containment` removed a `finalize` branch that no test row can reach. Some partial fixtures stay unchecked through `as unknown as` and `Promise<unknown>`, mostly in `issue-agent-mutation-ownership-routes` and `issue-comment-reopen-routes`. **Accuracy of the check.** One program over the tree and the three shards reported the same errors at the same positions. Memory, as the peak of the largest process on one thread, was 3.93 GiB for one program and 2.43 GiB for the largest shard. I measured these two values before the fixes. Not run or not proven: - I did not run the full `pnpm test:run`, or a root `pnpm build`. I expect CI to run the full suites. - I did not run the 4 GiB container check, or the full `scripts/typecheck-low-memory.mjs`. The peaks above come from macOS, so they are not a Linux cgroup measurement. - I did not run the UI typecheck after the merges. - `workspace-runtime-https-live-exercise` is skipped, so its `cleanup` fix did not run. ## Risks - **A large diff in test files.** 328 files change. A pull request that edits the same tests can get merge conflicts. New tests on master can also add type errors before this merges. Any such error fails this check and needs a fix here. - **A higher cost for changes to source types.** A change to a type can add errors in test files that the author did not touch. The author must then fix them. This is the purpose of the check, but it is new work. - **Changed test behavior.** Some tests now run paths that they did not run before, such as the 9 chat wizard fixtures and the 3 span tests. They pass here, but they can be more sensitive to timing or to the environment. - **The Daytona stub can go stale.** The stub is written by hand. If the plugin uses a new SDK member, the typecheck reports an error until someone updates the stub. - **The release path.** `release-verify.yml` and `scripts/release.sh` run `pnpm -r typecheck`, so they now run this check. - **The tsconfig change.** The test transform reads `server/src/__tests__/tsconfig.json`. The options that I changed are for type checks, and the runs above passed. - **A gap that remains.** Tsconfigs in `packages/paperclip-runner` and in most sandbox provider plugins exclude `*.test.ts`. This check does not read most of those test files, so they can still drift. That work is separate from this pull request. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with reasoning, tool use, code execution, and subagents. The runtime does not show the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (7 tests fail locally, with and without this change. See Verification.) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
3034 lines
316 KiB
TypeScript
3034 lines
316 KiB
TypeScript
import { subscribeDatabaseWork } from "@paperclipai/db";
|
|
import { DELIVERY_QUEUES } from "../services/delivery-work-notifications.js";
|
|
import express from "express";
|
|
import { execFileSync } from "node:child_process";
|
|
import { fileURLToPath } from "node:url";
|
|
import { activityLog, agentCommentary, documents, issueDocuments } from "@paperclipai/db";
|
|
import request from "supertest";
|
|
import { createHash, randomBytes, randomUUID } from "node:crypto";
|
|
import { and, eq } from "drizzle-orm";
|
|
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
|
import { agents, agentWakeupRequests, authUsers, companies, companyMemberships, createDb, closeRegisteredClients, applyPendingMigrations, externalAgentHolds, heartbeatRuns, issueAccessGrants, issueComments, issues, projects, principalPermissionGrants, mcpOauthGrants, mcpOauthRequests, mcpOauthTokens, dotAgentBindings, dotRunnerAssignments, dotRunnerOperations, museIdleReceipts, museAgentBindings as bindings, museCredentials, museInputDeliveries, museReceiverContactBuckets, museMailboxItems as mailbox, museRunnerAssignments as assignments, museRunnerOperations as operations, nativeRunFinalizations } from "@paperclipai/db";
|
|
import { heartbeatService } from "../services/heartbeat.js";
|
|
import { startAgentLifecycle } from "../services/agent-lifecycle.js";
|
|
import { agentHarnessVerificationService } from "../services/agent-harness-verification.js";
|
|
import type { PluginWorkerManager } from "../services/plugin-worker-manager.js";
|
|
import { MUSE_PUBLIC_ROUTES, MUSE_CLIENT_VERSION, type MuseCredentials } from "@paperclipai/shared";
|
|
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
|
import { museOAuth } from "../services/muse-oauth.js";
|
|
import { createPublicMcpOAuth, hashMcpSecret } from "../services/public-mcp/oauth.js";
|
|
import { dotRunnerBroker } from "../services/dot-runner-broker.js";
|
|
import { inspectMuseConnectionCredential, assertMuseSubjectCredential } from "../services/muse-identity.js";
|
|
import { sql } from "drizzle-orm";
|
|
import { museIdentity, museCredentialHash } from "../services/muse-identity.js";
|
|
import * as museIdentityModule from "../services/muse-identity.js";
|
|
import { museTransportRoutes, museBoardRoutes } from "../routes/muse.js";
|
|
import { projectRoutes } from "../routes/projects.js";
|
|
import { activityRoutes } from "../routes/activity.js";
|
|
import * as activityLogModule from "../services/activity-log.js";
|
|
import { errorHandler } from "../middleware/error-handler.js";
|
|
import { museRunnerBroker } from "../services/muse-runner-broker.js";
|
|
import { museReceiver } from "../services/muse-receiver.js";
|
|
import { instanceSettingsService } from "../services/instance-settings.js";
|
|
import { boardAuthService } from "../services/board-auth.js";
|
|
import { ensureHumanRoleDefaultGrants } from "../services/principal-access-compatibility.js";
|
|
import { subscribeCompanyLiveEvents } from "../services/live-events.js";
|
|
import { LOW_TRUST_QUARANTINED_BODY, buildLowTrustSourceTrust, buildPromotedSourceTrust, sanitizeQuarantinedCommentForHigherTrust } from "../services/source-trust.js";
|
|
import { PaperclipRunnerToolAuthority } from "../services/native-runtime/paperclip-runner-tool-authority.js";
|
|
import { documentService } from "../services/documents.js";
|
|
import { assertNoExternalOverlap, withExternalAdmissionGuard } from "../modules/external-agents/index.js";
|
|
import { digestPaperclipSemanticContent, externalOperationDigest, parseNativeExecutionInput, canonicalNativeRuntimeContextDigest, nativeRuntimePromptDigest, NATIVE_RUNTIME_ASSET_SCHEMA, PAPERCLIP_EXECUTION_PROMPT, PAPERCLIP_EXECUTION_PROMPT_REVISION, type ExternalProviderOperation } from "../vendor/paperclip-runner/index.js";
|
|
import { actorMiddleware } from "../middleware/auth.js";
|
|
import { createLocalAgentJwt } from "../agent-auth-jwt.js";
|
|
import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js";
|
|
|
|
/** Real database/transport authority tests; synthetic provider receipts do not
|
|
* count as live Muse/native qualification evidence. */
|
|
describe("personal Muse control plane",()=>{
|
|
let temporary:Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>|undefined,db:ReturnType<typeof createDb>,externalDatabaseUrl:string|undefined;
|
|
beforeAll(async()=>{
|
|
externalDatabaseUrl=process.env.PAPERCLIP_MUSE_CONTROL_TEST_DATABASE_URL?.trim();
|
|
if(externalDatabaseUrl){await applyPendingMigrations(externalDatabaseUrl);db=createDb(externalDatabaseUrl);}
|
|
else {temporary=await startEmbeddedPostgresTestDatabase("muse-control-");db=createDb(temporary.connectionString);}
|
|
await instanceSettingsService(db).updateExperimental({enableNativeRunner:true,enableMuse:true});
|
|
},30000);
|
|
afterAll(async()=>{vi.unstubAllEnvs();await museReceiver(db).stop();if(externalDatabaseUrl)await closeRegisteredClients(externalDatabaseUrl);await temporary?.cleanup();});
|
|
async function fixture(ready=true) {
|
|
const operatorId=randomUUID();await db.insert(authUsers).values({id:operatorId,name:"Muse authorizer",email:operatorId+"@example.test",createdAt:new Date(),updatedAt:new Date()});
|
|
const [company]=await db.insert(companies).values({name:"Muse control",issuePrefix:"MU"+operatorId.replaceAll("-","")}).returning();
|
|
const [agent]=await db.insert(agents).values({companyId:company.id,name:"Muse",role:"engineer",status:"active",adapterType:"paperclip_runner",adapterConfig:{provider:"muse",allowUnmeteredProvider:true,lifecycle:"per_turn"},permissions:{canCreateTasks:true}}).returning();
|
|
await db.insert(companyMemberships).values([{companyId:company.id,principalType:"user",principalId:operatorId,membershipRole:"owner",status:"active"},{companyId:company.id,principalType:"agent",principalId:agent.id,membershipRole:"member",status:"active"}]);
|
|
const ticket=randomBytes(32).toString("base64url");const [binding]=await db.insert(bindings).values({companyId:company.id,agentId:agent.id,operatorId,ticketHash:museCredentialHash(ticket),ticketExpiresAt:new Date(Date.now()+600000)}).returning();
|
|
await db.update(agents).set({adapterConfig:{...agent.adapterConfig,museBindingId:binding.id}}).where(eq(agents.id,agent.id));
|
|
const identity=museIdentity(db),credentials=await identity.pair(ticket,"2"),broker=museRunnerBroker(db,{publicOrigin:"https://paperclip.example",heartbeat:{wakeup:vi.fn().mockResolvedValue(null),cancelRun:vi.fn()}});
|
|
if(ready)await db.update(bindings).set({status:"ready",receiverContactAt:new Date(),verifiedReplyAt:new Date()}).where(eq(bindings.id,binding.id));
|
|
return {company,agent,operatorId,binding,ticket,identity,credentials,broker,subject:await identity.authenticate(credentials.accessToken)};
|
|
}
|
|
function transport(allowLegacyCredentials=false) { const app=express();app.use(museTransportRoutes(db,"https://paperclip.example",{allowLegacyCredentials}));app.use(errorHandler);return app; }
|
|
const oauthOrigin="https://paperclip.example",oauthResource=oauthOrigin+"/api/muse/v1",oauthCallback="https://authd.example/callback";
|
|
async function oauthFixture(ready=false,authorize=true) {
|
|
const f=await fixture(false),ticket=randomBytes(32).toString("base64url");
|
|
// This fixture starts the production OAuth journey from an unredeemed binding.
|
|
await db.delete(museCredentials).where(eq(museCredentials.bindingId,f.binding.id));
|
|
await db.update(bindings).set({status:"pairing",pairedAt:null,workerActivityAt:null,ticketHash:museCredentialHash(ticket),ticketExpiresAt:new Date(Date.now()+600000)}).where(eq(bindings.id,f.binding.id));
|
|
const oauth=museOAuth(db,oauthOrigin),client=await oauth.register({client_name:"Muse authd",redirect_uris:[oauthCallback],token_endpoint_auth_method:"none",grant_types:["authorization_code","refresh_token"]},randomUUID());
|
|
const verifier=randomBytes(32).toString("base64url"),input={client_id:client.client_id,redirect_uri:oauthCallback,response_type:"code",scope:"paperclip:agent offline_access",state:"opaque-state+/☂",code_challenge:createHash("sha256").update(verifier).digest("base64url"),code_challenge_method:"S256",pairing_ticket:ticket};
|
|
const authorizationUrl=authorize?await oauth.authorize(input):undefined;
|
|
const tokenInput=()=>({grant_type:"authorization_code",client_id:client.client_id,redirect_uri:oauthCallback,code:new URL(authorizationUrl!).searchParams.get("code")!,code_verifier:verifier});
|
|
const tokens=ready?await oauth.token(tokenInput()):undefined;
|
|
const subject=tokens?await f.identity.authenticateOAuthAccess(tokens.access_token,oauthOrigin):f.subject;
|
|
if(ready)await db.update(bindings).set({status:"ready",receiverContactAt:new Date(),verifiedReplyAt:new Date()}).where(eq(bindings.id,f.binding.id));
|
|
const [binding]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));
|
|
return {...f,binding,ticket,oauth,client,verifier,input,authorizationUrl,tokenInput,tokens,subject};
|
|
}
|
|
it("signals maintenance from real qualification, assignment, and OAuth revocation producers", async () => {
|
|
const events: string[] = [];
|
|
const unsubscribe = subscribeDatabaseWork(db, DELIVERY_QUEUES.museMaintenance, event => events.push(event));
|
|
let detach: (() => Promise<void>) | undefined;
|
|
try {
|
|
const f = await oauthFixture(true);
|
|
expect(events).toContain("intent");
|
|
expect(events.at(-1)).toBe("settled");
|
|
events.length = 0;
|
|
const [binding] = await db.select().from(bindings).where(eq(bindings.id, f.binding.id));
|
|
await f.broker.beginQualification({ companyId: f.company.id, agentId: f.agent.id,
|
|
bindingId: binding.id, generation: binding.generation, expectedRevision: binding.revision,
|
|
qualificationId: randomUUID(), expiresAt: new Date(Date.now() + 60_000), operatorId: f.operatorId });
|
|
expect(events).toEqual(["intent", "settled"]);
|
|
events.length = 0;
|
|
const running = await active(f);
|
|
detach = running.detach;
|
|
expect(events).toContain("intent");
|
|
expect(events.at(-1)).toBe("settled");
|
|
events.length = 0;
|
|
await f.oauth.revokeConnection(binding.grantId!, f.operatorId);
|
|
expect(events).toContain("intent");
|
|
expect(events.at(-1)).toBe("settled");
|
|
expect((await db.select().from(bindings).where(eq(bindings.id, binding.id)))[0].revokedAt).not.toBeNull();
|
|
} finally { unsubscribe(); await detach?.(); }
|
|
});
|
|
it("permits a paused handoff recipient authority read while retaining connection revocation checks", async () => {
|
|
const f = await oauthFixture(true);
|
|
const ref = { companyId: f.company.id, agentId: f.agent.id, bindingId: f.binding.id, bindingGeneration: f.binding.generation };
|
|
await expect(f.broker.authorizingUserIdForBinding(ref)).resolves.toBe(f.operatorId);
|
|
await db.update(agents).set({ status: "paused" }).where(eq(agents.id, f.agent.id));
|
|
await expect(f.broker.authorizingUserIdForBinding(ref)).rejects.toMatchObject({ status: 403 });
|
|
await expect(f.broker.authorizingUserIdForBinding(ref, { allowPaused: true })).resolves.toBe(f.operatorId);
|
|
await db.update(bindings).set({ revokedAt: new Date() }).where(eq(bindings.id, f.binding.id));
|
|
await expect(f.broker.authorizingUserIdForBinding(ref, { allowPaused: true })).rejects.toMatchObject({ status: 403 });
|
|
});
|
|
async function active(provided?:Awaited<ReturnType<typeof fixture>>,afterSettle?: (command:ExternalProviderOperation)=>Promise<void>) {
|
|
const f=provided??await fixture(),runId=randomUUID(),session=randomUUID(),turnId="turn_"+randomUUID();
|
|
const [issue]=await db.insert(issues).values({companyId:f.company.id,title:"Muse task",status:"in_progress",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
await db.insert(heartbeatRuns).values({id:runId,companyId:f.company.id,agentId:f.agent.id,status:"running",runtimeMode:"native",driverKind:"muse_external",nativeIssueId:issue.id,nativeSessionId:session,nativePhase:"turn_running",responsibleUserId:f.operatorId});
|
|
await db.update(issues).set({checkoutRunId:runId,executionRunId:runId}).where(eq(issues.id,issue.id));
|
|
await db.insert(nativeRunFinalizations).values({runId,companyId:f.company.id,issueId:issue.id,phase:"running",controllerGeneration:1,leaseExpiresAt:new Date(Date.now()+600000)});
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId},provider:{binding:await f.broker.snapshot(f.company.id,f.agent.id,f.binding.id)}};
|
|
const port=f.broker.port(execution),sent:ExternalProviderOperation[]=[];
|
|
const detach=await port.attach(async command=>{sent.push(command);await port.settle({sourceEventId:"settle_"+command.requestId,payload:{binding:ref,requestId:command.requestId,outcome:command.action==="consume_input"?{status:"consumed",requestId:command.input.requestId,inputDigest:command.input.inputDigest}:{status:command.action==="accept"?"accepted":"completed"}}});await afterSettle?.(command);});
|
|
const ref={...execution.provider.binding,runId,normalizedSessionId:session,turnId,assignmentRevision:1};
|
|
await port.dispatch({sourceEventId:"offer_"+randomUUID(),payload:{kind:"assignment",binding:ref,tools:[],acceptByUnixMs:Date.now()+600000,expiresAtUnixMs:Date.now()+3600000}});
|
|
const [assignment]=await db.select().from(assignments).where(eq(assignments.runId,runId));
|
|
return {...f,runId,issue,port,ref,assignment,sent,detach};
|
|
}
|
|
function projectWriteExecution(f: Awaited<ReturnType<typeof active>>) {
|
|
const digest="0".repeat(64),now=Date.now();
|
|
const context={
|
|
prompt:{revision:PAPERCLIP_EXECUTION_PROMPT_REVISION,text:PAPERCLIP_EXECUTION_PROMPT,digest:nativeRuntimePromptDigest()},
|
|
instructions:{entryPath:"AGENTS.md",bundle:{schema:NATIVE_RUNTIME_ASSET_SCHEMA,digest,manifestDigest:digest,rootPath:"/synthetic/muse-jwt-instructions",fileCount:1,totalBytes:4}},
|
|
skills:[],mcp:{assignmentSetId:"none",digest,bindingId:null},
|
|
};
|
|
const execution=parseNativeExecutionInput({
|
|
schema:"paperclip.native-execution-input.v8",
|
|
binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId,issueId:f.issue.id,executionWorkspaceId:randomUUID()},
|
|
task:{identifier:f.issue.identifier??"MUSE-JWT",title:f.issue.title,description:null,prompt:"Read the current actor.",workMode:"standard"},
|
|
provider:{kind:"muse",model:null,bridgeRevision:"muse-v1",binding:{companyId:f.company.id,agentId:f.agent.id,bindingId:f.binding.id,bindingGeneration:f.binding.generation,acceptByUnixMs:now+600000,expiresAtUnixMs:now+3600000}},
|
|
workspace:{access:"none",cwd:null,repoUrl:null,repoRef:null,branchName:null},
|
|
session:{normalizedSessionId:f.ref.normalizedSessionId,driverKind:"muse_external",protocolVersion:1,lifecyclePolicy:{mode:"per_turn",idleTimeoutMs:null}},
|
|
executionMode:"default",planningContext:null,
|
|
completionContract:{id:randomUUID(),sha256:"sha256:"+digest,schemaVersion:"paperclip.completion-contract.v1",contract:{revision:"1",objective:"Read the actor",criteria:[{id:"actor",requirement:"Read the actor"}]}},
|
|
runtimeContext:{...context,aggregateDigest:canonicalNativeRuntimeContextDigest(context)},interactionResponses:[],credentialBindings:[],
|
|
});
|
|
if(execution.schema!=="paperclip.native-execution-input.v8")throw new Error("Muse JWT fixture is not V8");
|
|
return execution;
|
|
}
|
|
it.each([
|
|
{ wait: "task", fault: "assignment_expired", replay: false },
|
|
{ wait: "task", fault: "grant_revoked", replay: false },
|
|
{ wait: "receipt", fault: "assignment_expired", replay: true },
|
|
{ wait: "receipt", fault: "grant_revoked", replay: true },
|
|
{ wait: "task", fault: "unchanged", replay: false },
|
|
{ wait: "receipt", fault: "unchanged", replay: true },
|
|
] as const)("rechecks Muse project-write authority after a real $wait lock wait ($fault, replay=$replay)", async ({ wait, fault, replay }) => {
|
|
const f = await active(await oauthFixture(true));
|
|
const previousSecret = process.env.PAPERCLIP_AGENT_JWT_SECRET;
|
|
process.env.PAPERCLIP_AGENT_JWT_SECRET = randomBytes(32).toString("hex");
|
|
let releaseAfterAuth = () => {}, releaseHolder = () => {};
|
|
let holder: Promise<void> | undefined;
|
|
let operation: Promise<{ response?: { status: number; body: Record<string, unknown> }; error?: unknown }> | undefined;
|
|
try {
|
|
await ensureHumanRoleDefaultGrants(db, { companyId: f.company.id, principalId: f.operatorId, membershipRole: "owner", grantedByUserId: f.operatorId });
|
|
expect((await f.broker.act(f.subject, { version: 1, command: "accept", assignmentId: f.assignment.id, requestId: randomUUID() })).status).toBe("accepted");
|
|
// The strict V8 profile is an HTTP authentication fixture, not a provider
|
|
// launch, scheduler admission, or release of the accepted Muse holds.
|
|
const execution = projectWriteExecution(f);
|
|
await db.update(heartbeatRuns).set({ runnerProfileJson: { nativeExecutionInput: execution } }).where(eq(heartbeatRuns.id, f.runId));
|
|
await expect(f.broker.assertRunAuthority(execution)).resolves.toBeUndefined();
|
|
const token = createLocalAgentJwt(f.agent.id, f.company.id, "paperclip_runner", f.runId, f.operatorId, undefined, f.operatorId);
|
|
expect(typeof token).toBe("string");
|
|
let armed = false, authenticated = () => {};
|
|
let observedActor: unknown;
|
|
const reachedAuth = new Promise<void>(resolve => { authenticated = resolve; });
|
|
const afterAuth = new Promise<void>(resolve => { releaseAfterAuth = resolve; });
|
|
const app = express();
|
|
app.use(express.json());
|
|
app.use(actorMiddleware(db, { deploymentMode: "authenticated", resolveSession: async () => null }));
|
|
app.use((req, _res, next) => {
|
|
if (!armed) { next(); return; }
|
|
observedActor = req.actor;
|
|
authenticated();
|
|
void afterAuth.then(() => next(), next);
|
|
});
|
|
app.use("/api", projectRoutes(db));
|
|
app.use(errorHandler);
|
|
const input = { name: "Muse project authority canary", description: "A bounded project result", idempotencyKey: randomUUID() };
|
|
const receiptKey = `project:${f.company.id}:${f.agent.id}:${f.issue.id}:${input.idempotencyKey}`;
|
|
const send = (body = input) => request(app).post(`/api/companies/${f.company.id}/projects`)
|
|
.set("Authorization", "Bearer " + token).set("X-Paperclip-Run-Id", f.runId).send(body);
|
|
let priorId: string | undefined;
|
|
if (replay) {
|
|
const first = await send();
|
|
expect(first.status).toBe(201);
|
|
priorId = first.body.id;
|
|
expect(typeof priorId).toBe("string");
|
|
const duplicate = await send();
|
|
expect(duplicate.status).toBe(200);
|
|
expect(duplicate.body.id).toBe(priorId);
|
|
}
|
|
const snapshot = async () => ({
|
|
projects: await db.select().from(projects).where(eq(projects.companyId, f.company.id)).orderBy(projects.id),
|
|
activity: await db.select().from(activityLog).where(eq(activityLog.companyId, f.company.id)).orderBy(activityLog.id),
|
|
run: (await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId)))[0],
|
|
assignment: await db.select({ status: assignments.status, acceptedResultAt: assignments.acceptedResultAt, finalizedAt: assignments.finalizedAt }).from(assignments).where(eq(assignments.id, f.assignment.id)),
|
|
holds: await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId, f.assignment.id)).orderBy(externalAgentHolds.id),
|
|
});
|
|
const before = await snapshot();
|
|
const oldReceipts = before.activity.filter(row => row.action === "project.created" && row.details?.idempotencyKey === receiptKey);
|
|
expect(oldReceipts).toHaveLength(replay ? 1 : 0);
|
|
expect(before.projects).toHaveLength(replay ? 1 : 0);
|
|
armed = true;
|
|
operation = send().then(response => ({ response: { status: response.status, body: response.body } }), error => ({ error }));
|
|
await expect.poll(() => observedActor !== undefined, { timeout: 5000, interval: 25 }).toBe(true);
|
|
await reachedAuth;
|
|
expect(observedActor).toMatchObject({ type: "agent", source: "agent_jwt", companyId: f.company.id, agentId: f.agent.id, runId: f.runId, authorizingUserId: f.operatorId });
|
|
// The holder starts only after real middleware authorization. Holding it
|
|
// before the request could block middleware instead of the project path.
|
|
let locked = () => {}, holderPid = 0;
|
|
const ready = new Promise<void>(resolve => { locked = resolve; });
|
|
const gate = new Promise<void>(resolve => { releaseHolder = resolve; });
|
|
holder = db.transaction(async tx => {
|
|
holderPid = (await tx.execute<{ pid: number }>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
if (wait === "task") await tx.select({ id: issues.id }).from(issues).where(and(eq(issues.companyId, f.company.id), eq(issues.id, f.issue.id))).for("update");
|
|
else await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${receiptKey}, 0))`);
|
|
locked();
|
|
await gate;
|
|
});
|
|
await ready;
|
|
releaseAfterAuth();
|
|
await expect.poll(async () => {
|
|
const [observed] = wait === "task"
|
|
? await db.execute<{ waiting: boolean }>(sql`select exists(select 1 from pg_stat_activity waiter
|
|
inner join pg_locks waiting on waiting.pid=waiter.pid and waiting.locktype='transactionid' and not waiting.granted
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='transactionid' and held.granted and held.transactionid=waiting.transactionid
|
|
where waiter.datname=current_database() and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and exists(select 1 from pg_locks tuple_lock where tuple_lock.pid=waiter.pid and tuple_lock.locktype='tuple'
|
|
and tuple_lock.relation='issues'::regclass and tuple_lock.mode in ('ExclusiveLock','AccessExclusiveLock') and tuple_lock.granted)) as waiting`)
|
|
: await db.execute<{ waiting: boolean }>(sql`select exists(select 1 from pg_stat_activity waiter
|
|
inner join pg_locks waiting on waiting.pid=waiter.pid and waiting.locktype='advisory' and not waiting.granted
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='advisory' and held.granted
|
|
and held.database=waiting.database and held.classid=waiting.classid and held.objid=waiting.objid and held.objsubid=waiting.objsubid
|
|
where waiter.datname=current_database() and ${holderPid}=any(pg_blocking_pids(waiter.pid))) as waiting`);
|
|
return observed.waiting;
|
|
}, { timeout: 5000, interval: 25 }).toBe(true);
|
|
await expect(f.broker.assertRunAuthority(execution)).resolves.toBeUndefined();
|
|
if (fault === "assignment_expired") await db.update(assignments).set({ expiresAt: new Date(Date.now() - 1000) }).where(eq(assignments.id, f.assignment.id));
|
|
if (fault === "grant_revoked") await db.update(mcpOauthGrants).set({ revokedAt: new Date() }).where(eq(mcpOauthGrants.id, f.binding.grantId!));
|
|
if (fault === "unchanged") await expect(f.broker.assertRunAuthority(execution)).resolves.toBeUndefined();
|
|
else await expect(f.broker.assertRunAuthority(execution)).rejects.toMatchObject({ status: 403 });
|
|
releaseHolder();
|
|
await holder;
|
|
const outcome = await operation;
|
|
expect(outcome.error).toBeUndefined();
|
|
armed = false;
|
|
if (fault !== "unchanged") {
|
|
// The old route returns 201/200 here; denial must precede any effect or replay.
|
|
expect(outcome.response?.status).toBe(403);
|
|
expect(outcome.response?.body).toEqual({ error: "Muse native tool authority is unavailable" });
|
|
expect(await snapshot()).toEqual(before);
|
|
} else {
|
|
expect(outcome.response?.status).toBe(replay ? 200 : 201);
|
|
const id = outcome.response?.body.id;
|
|
expect(typeof id).toBe("string");
|
|
if (replay) expect(id).toBe(priorId);
|
|
const committed = await snapshot();
|
|
expect(committed.projects).toHaveLength(1);
|
|
expect(committed.projects[0].id).toBe(id);
|
|
expect(committed.activity.filter(row => row.action === "project.created" && row.details?.idempotencyKey === receiptKey)).toHaveLength(1);
|
|
const duplicate = await send();
|
|
expect(duplicate.status).toBe(200);
|
|
expect(duplicate.body.id).toBe(id);
|
|
expect(await snapshot()).toEqual(committed);
|
|
const conflict = await send({ ...input, name: input.name + " changed" });
|
|
expect(conflict.status).toBe(409);
|
|
expect(await snapshot()).toEqual(committed);
|
|
expect(committed.holds).toEqual(before.holds);
|
|
expect(committed.assignment).toEqual(before.assignment);
|
|
expect(committed.run).toEqual(before.run);
|
|
}
|
|
} finally {
|
|
releaseAfterAuth();
|
|
releaseHolder();
|
|
await Promise.allSettled([holder, operation].filter(value => value !== undefined));
|
|
if (previousSecret === undefined) delete process.env.PAPERCLIP_AGENT_JWT_SECRET;
|
|
else process.env.PAPERCLIP_AGENT_JWT_SECRET = previousSecret;
|
|
await f.detach();
|
|
}
|
|
}, 15000);
|
|
it("authenticates current Muse V8 JWT actors and denies changed native bindings and grants",async()=>{
|
|
const f=await active(await oauthFixture(true)),previousSecret=process.env.PAPERCLIP_AGENT_JWT_SECRET;
|
|
process.env.PAPERCLIP_AGENT_JWT_SECRET=randomBytes(32).toString("hex");
|
|
try {
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
// Strict launch data for an authentication fixture; no provider launch,
|
|
// scheduler admission, or runtime-asset qualification is claimed here.
|
|
const digest="0".repeat(64),now=Date.now();
|
|
const context={
|
|
prompt:{revision:PAPERCLIP_EXECUTION_PROMPT_REVISION,text:PAPERCLIP_EXECUTION_PROMPT,digest:nativeRuntimePromptDigest()},
|
|
instructions:{entryPath:"AGENTS.md",bundle:{schema:NATIVE_RUNTIME_ASSET_SCHEMA,digest,manifestDigest:digest,rootPath:"/synthetic/muse-jwt-instructions",fileCount:1,totalBytes:4}},
|
|
skills:[],mcp:{assignmentSetId:"none",digest,bindingId:null},
|
|
};
|
|
const execution=parseNativeExecutionInput({
|
|
schema:"paperclip.native-execution-input.v8",
|
|
binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId,issueId:f.issue.id,executionWorkspaceId:randomUUID()},
|
|
task:{identifier:f.issue.identifier??"MUSE-JWT",title:f.issue.title,description:null,prompt:"Read the current actor.",workMode:"standard"},
|
|
provider:{kind:"muse",model:null,bridgeRevision:"muse-v1",binding:{companyId:f.company.id,agentId:f.agent.id,bindingId:f.binding.id,bindingGeneration:f.binding.generation,acceptByUnixMs:now+600000,expiresAtUnixMs:now+3600000}},
|
|
workspace:{access:"none",cwd:null,repoUrl:null,repoRef:null,branchName:null},
|
|
session:{normalizedSessionId:f.ref.normalizedSessionId,driverKind:"muse_external",protocolVersion:1,lifecyclePolicy:{mode:"per_turn",idleTimeoutMs:null}},
|
|
executionMode:"default",planningContext:null,
|
|
completionContract:{id:randomUUID(),sha256:"sha256:"+digest,schemaVersion:"paperclip.completion-contract.v1",contract:{revision:"1",objective:"Read the actor",criteria:[{id:"actor",requirement:"Read the actor"}]}},
|
|
runtimeContext:{...context,aggregateDigest:canonicalNativeRuntimeContextDigest(context)},interactionResponses:[],credentialBindings:[],
|
|
});
|
|
if(execution.schema!=="paperclip.native-execution-input.v8")throw new Error("Muse JWT fixture is not V8");
|
|
const profile={nativeExecutionInput:execution};
|
|
await db.update(heartbeatRuns).set({runnerProfileJson:profile}).where(eq(heartbeatRuns.id,f.runId));
|
|
await expect(f.broker.assertRunAuthority(execution)).resolves.toBeUndefined();
|
|
const token=createLocalAgentJwt(f.agent.id,f.company.id,"paperclip_runner",f.runId,f.operatorId,undefined,f.operatorId);
|
|
expect(typeof token).toBe("string");
|
|
const app=express();app.use(actorMiddleware(db,{deploymentMode:"authenticated",resolveSession:async()=>null}));
|
|
app.get("/actor",(req,res)=>res.json(req.actor));app.use(errorHandler);
|
|
const read=()=>request(app).get("/actor").set("Authorization","Bearer "+token).set("X-Paperclip-Run-Id",f.runId);
|
|
const positive=await read();
|
|
expect(positive.status).toBe(200);expect(positive.body).toMatchObject({type:"agent",source:"agent_jwt",companyId:f.company.id,agentId:f.agent.id,runId:f.runId,authorizingUserId:f.operatorId,onBehalfOfUserId:f.operatorId});
|
|
const beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const beforeAssignment=(await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0];
|
|
const beforeActivity=await db.select().from(activityLog).where(eq(activityLog.companyId,f.company.id)).orderBy(activityLog.id);
|
|
await db.update(heartbeatRuns).set({runtimeMode:"legacy"}).where(eq(heartbeatRuns.id,f.runId));
|
|
const nonNative=await read();expect(nonNative.status).toBe(403);expect(nonNative.body).toEqual({error:"Muse native tool authority is unavailable"});
|
|
await db.update(heartbeatRuns).set({runtimeMode:"native"}).where(eq(heartbeatRuns.id,f.runId));
|
|
expect((await read()).status).toBe(200);
|
|
// This remains a valid V8 shape, but names a different current run.
|
|
const otherRun={...execution,binding:{...execution.binding,runId:randomUUID()}};
|
|
expect(parseNativeExecutionInput(otherRun).schema).toBe("paperclip.native-execution-input.v8");
|
|
await db.update(heartbeatRuns).set({runnerProfileJson:{nativeExecutionInput:otherRun}}).where(eq(heartbeatRuns.id,f.runId));
|
|
const wrongRun=await read();expect(wrongRun.status).toBe(403);expect(wrongRun.body).toEqual({error:"Muse native tool authority is unavailable"});
|
|
await db.update(heartbeatRuns).set({runnerProfileJson:profile}).where(eq(heartbeatRuns.id,f.runId));
|
|
expect((await read()).status).toBe(200);
|
|
const otherAuthorizer=createLocalAgentJwt(f.agent.id,f.company.id,"paperclip_runner",f.runId,f.operatorId,undefined,randomUUID());
|
|
expect(typeof otherAuthorizer).toBe("string");
|
|
const wrongAuthorizer=await request(app).get("/actor").set("Authorization","Bearer "+otherAuthorizer).set("X-Paperclip-Run-Id",f.runId);
|
|
expect(wrongAuthorizer.status).toBe(403);expect(wrongAuthorizer.body).toEqual({error:"Muse native tool authority is unavailable"});
|
|
// A synthetic current-DB grant revocation must affect the same signed JWT.
|
|
await db.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
const revoked=await read();expect(revoked.status).toBe(403);expect(revoked.body).toEqual({error:"Muse native tool authority is unavailable"});
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual(beforeAssignment);
|
|
expect(await db.select().from(activityLog).where(eq(activityLog.companyId,f.company.id)).orderBy(activityLog.id)).toEqual(beforeActivity);
|
|
expect((await db.select({status:heartbeatRuns.status,runtimeMode:heartbeatRuns.runtimeMode,runnerProfileJson:heartbeatRuns.runnerProfileJson,resultJson:heartbeatRuns.resultJson}).from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId)))[0]).toMatchObject({status:"running",runtimeMode:"native",runnerProfileJson:profile,resultJson:null});
|
|
}finally{
|
|
if(previousSecret===undefined)delete process.env.PAPERCLIP_AGENT_JWT_SECRET;else process.env.PAPERCLIP_AGENT_JWT_SECRET=previousSecret;
|
|
await f.detach();
|
|
}
|
|
});
|
|
it.each([true,false])("retains native Muse progress source trust and idempotent history (low trust: %s)",async lowTrust=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const executionPolicy={authorizationPolicy:{trustBoundary:{mode:"low_trust_review",companyId:f.company.id,rootIssueId:f.issue.id}}};
|
|
const writerContext={issueId:f.issue.id,...(lowTrust?{executionPolicy}:{})};
|
|
await db.update(heartbeatRuns).set({contextSnapshot:writerContext}).where(eq(heartbeatRuns.id,f.runId));
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}};
|
|
const checks:string[]=[],authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,
|
|
museRuntime:true,externalAuthorizingUserId:f.operatorId,assertBridgeAuthority:async(database=db)=>{
|
|
await f.broker.assertRunAuthority(execution,database);checks.push(database===db?"discovery":"transaction");
|
|
}});
|
|
const body="NATIVE_MUSE_PROGRESS_QUARANTINE_CANARY",idempotencyKey=randomUUID();
|
|
const call={tool:"report_progress",callId:randomUUID(),arguments:{body,idempotencyKey}};
|
|
const beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const first=await authority.execute(call) as {commentId:string,issueId:string,disposition:string};
|
|
expect(first).toMatchObject({issueId:f.issue.id,disposition:"applied"});
|
|
expect(checks).toEqual(["discovery","transaction"]);
|
|
const [comment]=await db.select().from(issueComments).where(eq(issueComments.id,first.commentId));
|
|
const expectedTrust=lowTrust?buildLowTrustSourceTrust({issueId:f.issue.id,agentId:f.agent.id,runId:f.runId}):null;
|
|
// The baseline must reach the actual persisted tag; the previous producer stores null.
|
|
expect(comment.sourceTrust).toEqual(expectedTrust);
|
|
expect(comment).toMatchObject({companyId:f.company.id,issueId:f.issue.id,authorAgentId:f.agent.id,createdByRunId:f.runId,body});
|
|
const snapshot=async()=>({
|
|
comments:await db.select().from(issueComments).where(eq(issueComments.issueId,f.issue.id)).orderBy(issueComments.id),
|
|
activity:await db.select().from(activityLog).where(eq(activityLog.companyId,f.company.id)).orderBy(activityLog.id),
|
|
run:(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId)))[0],
|
|
});
|
|
const committed=await snapshot();
|
|
expect(committed.comments).toHaveLength(1);
|
|
expect(committed.activity.filter(row=>row.action==="issue.comment_added")).toHaveLength(1);
|
|
expect((committed.run.resultJson as {semanticToolReceipts:Record<string,unknown>}).semanticToolReceipts).toEqual({
|
|
[idempotencyKey]:{operationId:"report_progress",input:call.arguments,result:first},
|
|
});
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(first);
|
|
expect(await snapshot()).toEqual(committed);
|
|
await expect(authority.execute({...call,callId:randomUUID(),arguments:{...call.arguments,body:body+" changed"}})).rejects.toThrow("paperclip_runner_tool_idempotency_conflict");
|
|
expect(await snapshot()).toEqual(committed);
|
|
const history=await authority.execute({tool:"get_task_history",callId:randomUUID(),arguments:{limit:50}}) as {comments:Array<{id:string,body:string,sourceTrust:unknown}>};
|
|
expect(history.comments).toHaveLength(1);
|
|
expect(history.comments[0]).toMatchObject({id:comment.id,body,sourceTrust:expectedTrust});
|
|
if(lowTrust){
|
|
const promotedId=randomUUID(),standardId=randomUUID(),at=comment.createdAt.getTime();
|
|
const promotedTrust=buildPromotedSourceTrust({sourceIssueId:f.issue.id,sourceArtifactKind:"comment",sourceArtifactId:comment.id,promotedByActorType:"user",promotedByActorId:f.operatorId});
|
|
await db.insert(issueComments).values([
|
|
{id:promotedId,companyId:f.company.id,issueId:f.issue.id,authorUserId:f.operatorId,authorType:"user",body:"Reviewed progress summary",sourceTrust:promotedTrust,createdAt:new Date(at+1000)},
|
|
{id:standardId,companyId:f.company.id,issueId:f.issue.id,authorUserId:f.operatorId,authorType:"user",body:"Ordinary coordination",createdAt:new Date(at+2000)},
|
|
]);
|
|
// Construct a distinct, genuinely standard read context. This is a DB
|
|
// read fixture, not scheduler admission, a handoff, or worker release.
|
|
const [reader]=await db.insert(agents).values({companyId:f.company.id,name:"Standard history reader",role:"engineer",status:"active",adapterType:"paperclip_runner",adapterConfig:{provider:"codex"}}).returning();
|
|
await db.insert(companyMemberships).values({companyId:f.company.id,principalType:"agent",principalId:reader.id,membershipRole:"member",status:"active"});
|
|
const readerRunId=randomUUID();
|
|
await db.insert(heartbeatRuns).values({id:readerRunId,companyId:f.company.id,agentId:reader.id,status:"running",runtimeMode:"native",nativeIssueId:f.issue.id,responsibleUserId:f.operatorId,contextSnapshot:{issueId:f.issue.id}});
|
|
await db.update(issues).set({assigneeAgentId:reader.id,executionRunId:readerRunId,checkoutRunId:readerRunId}).where(eq(issues.id,f.issue.id));
|
|
const readerAuthority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:reader.id,issueId:f.issue.id,runId:readerRunId});
|
|
const stored=await db.select().from(issueComments).where(eq(issueComments.issueId,f.issue.id)).orderBy(issueComments.id);
|
|
const read=await readerAuthority.execute({tool:"get_task_history",callId:randomUUID(),arguments:{limit:50}}) as {comments:Array<{id:string,body:string,sourceTrust:unknown}>};
|
|
expect(read.comments.find(row=>row.id===comment.id)).toMatchObject({body:LOW_TRUST_QUARANTINED_BODY,sourceTrust:expectedTrust});
|
|
expect(JSON.stringify(read)).not.toContain(body);
|
|
expect(read.comments.find(row=>row.id===promotedId)).toMatchObject({body:"Reviewed progress summary",sourceTrust:promotedTrust});
|
|
expect(read.comments.find(row=>row.id===standardId)).toMatchObject({body:"Ordinary coordination",sourceTrust:null});
|
|
expect(await db.select().from(issueComments).where(eq(issueComments.issueId,f.issue.id)).orderBy(issueComments.id)).toEqual(stored);
|
|
}
|
|
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId)))[0]).toEqual(committed.run);
|
|
expect(committed.run.contextSnapshot).toEqual(writerContext);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
expect((await db.select({status:assignments.status,acceptedResultAt:assignments.acceptedResultAt,finalizedAt:assignments.finalizedAt}).from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual({status:"accepted",acceptedResultAt:null,finalizedAt:null});
|
|
}finally{await f.detach();}
|
|
});
|
|
it.each(["refresh_expired","muse_disabled","runner_disabled","grant_revoked","binding_revoked","assignment_fenced","qualification_expired","unchanged"] as const)("rechecks native Muse document authority after the real task-lock wait (%s)",async fault=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},holder:Promise<void>|undefined,write:Promise<{result?:unknown;error?:unknown}>|undefined;
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}};
|
|
const checks:string[]=[],authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,
|
|
museRuntime:true,externalAuthorizingUserId:f.operatorId,assertBridgeAuthority:async(database=db)=>{
|
|
await museRunnerBroker(database,{publicOrigin:oauthOrigin}).assertRunAuthority(execution);
|
|
checks.push(database===db?"discovery":"transaction");
|
|
}});
|
|
const idempotencyKey=randomUUID(),call={tool:"write_document",callId:randomUUID(),arguments:{key:"report",title:"Report",body:"MUSE_LOCK_WAIT_DOCUMENT_CANARY",baseRevisionId:null,idempotencyKey}};
|
|
const beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
let ready=()=>{},holderPid=0;const locked=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:issues.id}).from(issues).where(eq(issues.id,f.issue.id)).for("update");ready();await gate;
|
|
});
|
|
await locked;
|
|
write=authority.execute(call).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [observed]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_stat_activity waiter
|
|
where waiter.datname=current_database() and waiter.wait_event_type='Lock' and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and waiter.query ~* '(from|join)[[:space:]]+"?issues"?' and waiter.query ~* 'for[[:space:]]+update') as waiting`);
|
|
return observed.waiting;
|
|
},{timeout:5000,interval:25}).toBe(true);
|
|
expect(checks).toEqual(["discovery"]);
|
|
if(fault==="refresh_expired")await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(and(eq(mcpOauthTokens.grantId,f.binding.grantId!),eq(mcpOauthTokens.kind,"refresh")));
|
|
if(fault==="muse_disabled")await instanceSettingsService(db).updateExperimental({enableMuse:false});
|
|
if(fault==="runner_disabled")await instanceSettingsService(db).updateExperimental({enableNativeRunner:false});
|
|
if(fault==="grant_revoked")await db.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
if(fault==="binding_revoked")await db.update(bindings).set({revokedAt:new Date(),status:"revoked"}).where(eq(bindings.id,f.binding.id));
|
|
if(fault==="assignment_fenced")await db.update(assignments).set({status:"fenced"}).where(eq(assignments.id,f.assignment.id));
|
|
if(fault==="qualification_expired")await db.update(bindings).set({qualificationId:randomUUID(),qualificationExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
if(fault==="unchanged")await expect(f.broker.assertRunAuthority(execution)).resolves.toBeUndefined();
|
|
else await expect(f.broker.assertRunAuthority(execution)).rejects.toMatchObject({status:403});
|
|
release();await holder;
|
|
const outcome=await write;
|
|
const [run]=await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId));
|
|
const receipts=(run.resultJson as {semanticToolReceipts?:Record<string,unknown>}|null)?.semanticToolReceipts;
|
|
if(fault==="unchanged"){
|
|
expect(outcome.error).toBeUndefined();expect(outcome.result).toMatchObject({disposition:"applied",document:{body:call.arguments.body}});
|
|
expect(receipts?.[idempotencyKey]).toMatchObject({operationId:"write_document",input:call.arguments,result:outcome.result});
|
|
expect(await documentService(db).getIssueDocumentByKey(f.issue.id,"report")).toMatchObject({body:call.arguments.body});
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(outcome.result);
|
|
expect(await documentService(db).listIssueDocumentRevisions(f.issue.id,"report")).toHaveLength(1);
|
|
}else{
|
|
expect(outcome.error).toMatchObject({status:403});expect(outcome.result).toBeUndefined();
|
|
expect(await db.select({id:issueDocuments.id}).from(issueDocuments).where(eq(issueDocuments.issueId,f.issue.id))).toEqual([]);
|
|
expect(await db.select({id:documents.id}).from(documents).where(eq(documents.companyId,f.company.id))).toEqual([]);
|
|
expect(receipts?.[idempotencyKey]).toBeUndefined();
|
|
expect(await db.select({id:activityLog.id}).from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"issue.document_created")))).toEqual([]);
|
|
}
|
|
expect(run.status).toBe("running");
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
}finally{release();await holder;await write;await instanceSettingsService(db).updateExperimental({enableMuse:true,enableNativeRunner:true});await f.detach();}
|
|
});
|
|
it.each([
|
|
{deadline:"expired" as const,replay:false},
|
|
{deadline:"nonfinite" as const,replay:false},
|
|
{deadline:"expired" as const,replay:true},
|
|
{deadline:"nonfinite" as const,replay:true},
|
|
])("rejects native document effects and receipt replay after assignment deadline changes during the real task-lock wait ($deadline, replay=$replay)",async({deadline,replay})=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},holder:Promise<void>|undefined,operation:Promise<{result?:unknown;error?:unknown}>|undefined;
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}};
|
|
const checks:string[]=[],authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,
|
|
museRuntime:true,externalAuthorizingUserId:f.operatorId,assertBridgeAuthority:async(database=db)=>{
|
|
checks.push(database===db?"discovery":"transaction");await f.broker.assertRunAuthority(execution,database);
|
|
}});
|
|
const idempotencyKey=randomUUID(),call={tool:"write_document",callId:randomUUID(),arguments:{key:"report",title:"Assignment authority",body:"MUSE_ASSIGNMENT_DEADLINE_DOCUMENT_CANARY",baseRevisionId:null,idempotencyKey}};
|
|
let priorResult:unknown;
|
|
if(replay){
|
|
priorResult=await authority.execute(call);
|
|
expect(priorResult).toMatchObject({disposition:"applied",document:{body:call.arguments.body}});
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(priorResult);
|
|
expect(await documentService(db).listIssueDocumentRevisions(f.issue.id,"report")).toHaveLength(1);
|
|
}
|
|
const snapshot=async()=>({
|
|
run:await db.select({status:heartbeatRuns.status,resultJson:heartbeatRuns.resultJson}).from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId)),
|
|
issueDocuments:await db.select().from(issueDocuments).where(eq(issueDocuments.issueId,f.issue.id)).orderBy(issueDocuments.id),
|
|
documents:await db.select().from(documents).where(eq(documents.companyId,f.company.id)).orderBy(documents.id),
|
|
revisions:await documentService(db).listIssueDocumentRevisions(f.issue.id,"report"),
|
|
activity:await db.select({id:activityLog.id,action:activityLog.action,entityId:activityLog.entityId}).from(activityLog).where(eq(activityLog.companyId,f.company.id)).orderBy(activityLog.id),
|
|
assignment:await db.select({status:assignments.status,acceptedResultAt:assignments.acceptedResultAt,finalizedAt:assignments.finalizedAt}).from(assignments).where(eq(assignments.id,f.assignment.id)),
|
|
});
|
|
const [initialAssignment]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));
|
|
expect(initialAssignment.expiresAt).toBeInstanceOf(Date);
|
|
expect(Number.isFinite(initialAssignment.expiresAt.getTime())).toBe(true);
|
|
expect(initialAssignment.expiresAt.getTime()).toBeGreaterThan(Date.now());
|
|
const before=await snapshot(),beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
checks.length=0;
|
|
let ready=()=>{},holderPid=0;const locked=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:issues.id}).from(issues).where(and(eq(issues.companyId,f.company.id),eq(issues.id,f.issue.id))).for("update");ready();await gate;
|
|
});
|
|
await locked;
|
|
operation=authority.execute({...call,callId:randomUUID()}).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [observed]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_stat_activity waiter
|
|
inner join pg_locks waiting on waiting.pid=waiter.pid and waiting.locktype='transactionid' and not waiting.granted
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='transactionid' and held.granted and held.transactionid=waiting.transactionid
|
|
where waiter.datname=current_database() and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and exists(select 1 from pg_locks tuple_lock where tuple_lock.pid=waiter.pid and tuple_lock.locktype='tuple'
|
|
and tuple_lock.relation='issues'::regclass and tuple_lock.mode='AccessExclusiveLock' and tuple_lock.granted)) as waiting`);
|
|
return observed.waiting;
|
|
},{timeout:5000,interval:25}).toBe(true);
|
|
expect(checks).toEqual(["discovery"]);
|
|
if(deadline==="expired")await db.update(assignments).set({expiresAt:new Date(Date.now()-1)}).where(eq(assignments.id,f.assignment.id));
|
|
else await db.execute(sql`update muse_runner_assignments set expires_at='infinity'::timestamptz where id=${f.assignment.id}::uuid`);
|
|
const [changedAssignment]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));
|
|
expect(changedAssignment.status).toBe("accepted");expect(changedAssignment.expiresAt).toBeInstanceOf(Date);
|
|
if(deadline==="expired")expect(changedAssignment.expiresAt.getTime()).toBeLessThan(Date.now());
|
|
else expect(Number.isFinite(changedAssignment.expiresAt.getTime())).toBe(false);
|
|
// Connection authority stays live; only the assignment deadline changed.
|
|
await expect(assertMuseSubjectCredential(db,f.subject,oauthResource)).resolves.toBeDefined();
|
|
expect((await inspectMuseConnectionCredential(db,f.binding,oauthResource)).state).toBe("active");
|
|
expect(checks).toEqual(["discovery"]);
|
|
release();await holder;
|
|
const outcome=await operation;
|
|
expect(checks).toEqual(["discovery","transaction"]);
|
|
expect(outcome.error).toMatchObject({status:403,message:"Muse assignment unavailable."});expect(outcome.result).toBeUndefined();
|
|
const after=await snapshot();expect(after).toEqual(before);expect(after.run[0].status).toBe("running");
|
|
const receipts=(after.run[0].resultJson as {semanticToolReceipts?:Record<string,unknown>}|null)?.semanticToolReceipts;
|
|
if(replay)expect(receipts?.[idempotencyKey]).toMatchObject({operationId:"write_document",input:call.arguments,result:priorResult});
|
|
else expect(receipts?.[idempotencyKey]).toBeUndefined();
|
|
expect(after.assignment).toEqual([{status:"accepted",acceptedResultAt:null,finalizedAt:null}]);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
}finally{release();await Promise.allSettled([holder,operation]);await f.detach();}
|
|
});
|
|
it("preserves the ordinary non-Muse document receipt path without invoking Muse authority",async()=>{
|
|
const f=await active();try{
|
|
await db.update(agents).set({adapterConfig:{provider:"codex"}}).where(eq(agents.id,f.agent.id));
|
|
await db.update(heartbeatRuns).set({driverKind:"codex_app_server"}).where(eq(heartbeatRuns.id,f.runId));
|
|
const assertBridgeAuthority=vi.fn(async()=>{throw new Error("Muse authority must not guard this ordinary native binding");});
|
|
const authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,assertBridgeAuthority});
|
|
const call={tool:"write_document",callId:randomUUID(),arguments:{key:"report",title:"Report",body:"Ordinary native document",baseRevisionId:null,idempotencyKey:randomUUID()}};
|
|
const result=await authority.execute(call);expect(result).toMatchObject({disposition:"applied",document:{body:call.arguments.body}});
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(result);
|
|
expect(await documentService(db).listIssueDocumentRevisions(f.issue.id,"report")).toHaveLength(1);expect(assertBridgeAuthority).not.toHaveBeenCalled();
|
|
}finally{await f.detach();}
|
|
});
|
|
it.each(["prepare","commit","replay"] as const)("rejects contended Muse reassignment targets without an idle-comment deadlock (%s)",async phase=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},native:Promise<{result?:unknown;error?:unknown}>|undefined,idle:Promise<{result?:Record<string,unknown>;error?:unknown}>|undefined;
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const [target]=await db.insert(issues).values({companyId:f.company.id,title:"Reassignment target",status:phase==="commit"?"in_progress":"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
const oldRun=phase==="commit"?randomUUID():null;
|
|
if(oldRun){
|
|
await db.insert(heartbeatRuns).values({id:oldRun,companyId:f.company.id,agentId:f.agent.id,nativeIssueId:target.id,status:"running",runtimeMode:"native",invocationSource:"assignment",triggerDetail:"system"});
|
|
await db.update(issues).set({executionRunId:oldRun,checkoutRunId:oldRun}).where(eq(issues.id,target.id));
|
|
}
|
|
const stopTaskForReassignment=vi.fn(async()=>{
|
|
// Existing stop callback boundary: persist a cancellation acknowledgment
|
|
// outside row locks, so the production compensation path is exercised.
|
|
if(oldRun)await db.update(heartbeatRuns).set({status:"cancelled",errorCode:"issue_reassigned"}).where(eq(heartbeatRuns.id,oldRun));
|
|
});
|
|
const enqueueWakeup=vi.fn(async()=>({id:randomUUID()}));
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}};
|
|
let armed=false,txChecks=0,nativePid=0,idlePid=0,ready=()=>{};
|
|
const entered=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
const authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,
|
|
museRuntime:true,externalAuthorizingUserId:f.operatorId,stopTaskForReassignment,enqueueWakeup,
|
|
assertBridgeAuthority:async(database=db)=>{
|
|
await f.broker.assertRunAuthority(execution,database);
|
|
if(database!==db&&armed&&++txChecks===(phase==="prepare"?1:2)){
|
|
nativePid=(await database.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
ready();await gate;
|
|
}
|
|
}});
|
|
const call={tool:"reassign_task",callId:randomUUID(),arguments:{taskId:target.id,assigneeActorId:null,assigneeUserId:f.operatorId,expectedAssigneeActorId:f.agent.id,expectedStatusVersion:0,reason:"Retain target context while handing work to its operator",idempotencyKey:randomUUID()}};
|
|
let priorReceipt:unknown;
|
|
if(phase==="replay"){
|
|
priorReceipt=await authority.execute(call);
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(priorReceipt);
|
|
expect((await db.select().from(issues).where(eq(issues.id,target.id)))[0]).toMatchObject({assigneeAgentId:null,assigneeUserId:f.operatorId,statusVersion:1});
|
|
}
|
|
const readTarget=async()=>{
|
|
const [row]=await db.select({assigneeAgentId:issues.assigneeAgentId,assigneeUserId:issues.assigneeUserId,status:issues.status,statusVersion:issues.statusVersion,executionRunId:issues.executionRunId,checkoutRunId:issues.checkoutRunId}).from(issues).where(eq(issues.id,target.id));return row;
|
|
};
|
|
const readReassignments=()=>db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.entityId,target.id),eq(activityLog.action,"issue.reassigned")));
|
|
const beforeTarget=await readTarget(),beforeReassignments=await readReassignments();
|
|
const beforeRun=(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId)))[0];
|
|
const beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const beforeAssignment=(await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0];
|
|
const beforeStops=stopTaskForReassignment.mock.calls.length,beforeWakes=enqueueWakeup.mock.calls.length;
|
|
const command={version:1 as const,command:"task.comment" as const,requestId:randomUUID(),issueId:target.id,body:"Concurrent idle coordination"};
|
|
armed=true;native=authority.execute({...call,callId:randomUUID()}).then(result=>({result}),error=>({error}));
|
|
await expect.poll(()=>nativePid,{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
await entered;
|
|
const admissionKey=`paperclip:external-admission:${f.company.id}:${f.agent.id}`;
|
|
const nativeAdmission=await db.execute<{pid:number}>(sql`select pid from pg_locks where pid=${nativePid} and locktype='advisory' and granted and objsubid=1
|
|
and classid=((hashtextextended(${admissionKey},0)>>32)&4294967295)::oid and objid=(hashtextextended(${admissionKey},0)&4294967295)::oid`);
|
|
expect(nativeAdmission).toHaveLength(1);
|
|
idle=f.broker.act(f.subject,command).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [waiter]=await db.execute<{pid:number}>(sql`select waiting.pid from pg_locks waiting
|
|
inner join pg_locks held on held.pid=${nativePid} and held.locktype='advisory' and held.granted
|
|
and held.classid=waiting.classid and held.objid=waiting.objid and held.objsubid=waiting.objsubid
|
|
where waiting.locktype='advisory' and not waiting.granted and ${nativePid}=any(pg_blocking_pids(waiting.pid))
|
|
and waiting.objsubid=1 and waiting.classid=((hashtextextended(${admissionKey},0)>>32)&4294967295)::oid
|
|
and waiting.objid=(hashtextextended(${admissionKey},0)&4294967295)::oid`);
|
|
idlePid=waiter?.pid??0;return idlePid;
|
|
},{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
// The real idle path acquired target before requesting this admission key.
|
|
// Releasing the native callback now permits the original B <-> admission cycle.
|
|
release();
|
|
const [nativeOutcome,idleOutcome]=await Promise.all([native,idle]);
|
|
const observed=JSON.stringify({phase,nativePid,idlePid,nativeCode:(nativeOutcome.error as {cause?:{code?:string};code?:string}|undefined)?.cause?.code??(nativeOutcome.error as {code?:string}|undefined)?.code,idleCode:(idleOutcome.error as {cause?:{code?:string};code?:string}|undefined)?.cause?.code??(idleOutcome.error as {code?:string}|undefined)?.code});
|
|
expect(nativeOutcome.error,observed).toMatchObject({status:409,message:"paperclip_runner_reassignment_conflict: target is busy; retry after current work settles"});
|
|
expect(idleOutcome.error,observed).toBeUndefined();expect(idleOutcome.result).toMatchObject({status:"commented",issueId:target.id});
|
|
expect(await readTarget()).toEqual(beforeTarget);expect(await readReassignments()).toEqual(beforeReassignments);
|
|
const [afterRun]=await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId));
|
|
expect(afterRun.resultJson).toEqual(beforeRun.resultJson);expect(afterRun.status).toBe(beforeRun.status);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual(beforeAssignment);
|
|
expect(stopTaskForReassignment).toHaveBeenCalledTimes(beforeStops+(phase==="commit"?1:0));
|
|
expect(enqueueWakeup).toHaveBeenCalledTimes(beforeWakes+(phase==="commit"?1:0));
|
|
if(oldRun){
|
|
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,oldRun)))[0]).toMatchObject({status:"cancelled",errorCode:"issue_reassigned"});
|
|
expect(enqueueWakeup).toHaveBeenLastCalledWith(f.agent.id,expect.objectContaining({payload:{issueId:target.id,mutation:"reassign_task_rollback",interruptedRunId:oldRun},issueStateGuard:{statuses:["todo","in_progress"],assigneeAgentId:f.agent.id,statusVersion:0}}));
|
|
}
|
|
const idleReceipts=await db.select().from(museIdleReceipts).where(and(eq(museIdleReceipts.bindingId,f.binding.id),eq(museIdleReceipts.bindingGeneration,f.binding.generation),eq(museIdleReceipts.requestId,command.requestId)));
|
|
expect(idleReceipts).toHaveLength(1);expect(idleReceipts[0].outcome).toEqual(idleOutcome.result);
|
|
expect(await db.select().from(issueComments).where(and(eq(issueComments.issueId,target.id),eq(issueComments.body,command.body)))).toHaveLength(1);
|
|
armed=false;
|
|
const result=await authority.execute({...call,callId:randomUUID()});
|
|
if(phase==="replay")expect(result).toEqual(priorReceipt);
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(result);
|
|
expect(await f.broker.act(f.subject,command)).toEqual(idleOutcome.result);
|
|
expect(await readTarget()).toMatchObject({assigneeAgentId:null,assigneeUserId:f.operatorId,status:"todo",statusVersion:1,executionRunId:null,checkoutRunId:null});
|
|
expect(await readReassignments()).toHaveLength(1);
|
|
expect(await db.select().from(issueComments).where(eq(issueComments.issueId,target.id))).toHaveLength(2);
|
|
const [settledRun]=await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId));
|
|
expect((settledRun.resultJson as {semanticToolReceipts:Record<string,unknown>}).semanticToolReceipts[call.arguments.idempotencyKey]).toMatchObject({operationId:"reassign_task",input:call.arguments,result});
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual(beforeAssignment);
|
|
}finally{release();await Promise.allSettled([native,idle]);await f.detach();}
|
|
});
|
|
it("preserves ordinary native reassignment waiting on a held target",async()=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},holder:Promise<void>|undefined,native:Promise<{result?:unknown;error?:unknown}>|undefined;
|
|
try {
|
|
await db.update(heartbeatRuns).set({driverKind:"codex_app_server"}).where(eq(heartbeatRuns.id,f.runId));
|
|
const [target]=await db.insert(issues).values({companyId:f.company.id,title:"Ordinary reassignment target",status:"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
let ready=()=>{},holderPid=0;const locked=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:issues.id}).from(issues).where(and(eq(issues.id,target.id),eq(issues.companyId,f.company.id))).for("update");ready();await gate;
|
|
});
|
|
await locked;
|
|
const assertBridgeAuthority=vi.fn(async()=>{throw new Error("Muse authority must not guard ordinary reassignment");});
|
|
const authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,assertBridgeAuthority,stopTaskForReassignment:vi.fn(async()=>{})});
|
|
const call={tool:"reassign_task",callId:randomUUID(),arguments:{taskId:target.id,assigneeActorId:null,assigneeUserId:f.operatorId,expectedAssigneeActorId:f.agent.id,expectedStatusVersion:0,reason:"Ordinary handoff",idempotencyKey:randomUUID()}};
|
|
native=authority.execute(call).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [observed]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_locks waiting
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='transactionid' and held.granted and held.transactionid=waiting.transactionid
|
|
where waiting.locktype='transactionid' and not waiting.granted and ${holderPid}=any(pg_blocking_pids(waiting.pid))) as waiting`);
|
|
return observed.waiting;
|
|
},{timeout:5000,interval:25}).toBe(true);
|
|
release();await holder;
|
|
const result=await native;expect(result.error).toBeUndefined();expect(result.result).toMatchObject({disposition:"applied",stateRevision:1});
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(result.result);
|
|
expect((await db.select().from(issues).where(eq(issues.id,target.id)))[0]).toMatchObject({assigneeAgentId:null,assigneeUserId:f.operatorId,statusVersion:1});
|
|
expect(assertBridgeAuthority).not.toHaveBeenCalled();
|
|
}finally{release();await Promise.allSettled([holder,native]);await f.detach();}
|
|
});
|
|
it("serializes native and idle child creation without a privacy-tree deadlock",async()=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},holder:Promise<void>|undefined,native:Promise<{result?:unknown;error?:unknown}>|undefined,idle:Promise<{result?:unknown;error?:unknown}>|undefined;
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}};
|
|
const authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,
|
|
museRuntime:true,externalAuthorizingUserId:f.operatorId,assertBridgeAuthority:(database=db)=>f.broker.assertRunAuthority(execution,database)});
|
|
const call={tool:"create_task",callId:randomUUID(),arguments:{title:"Native child",idempotencyKey:randomUUID()}};
|
|
const command={version:1 as const,command:"task.create" as const,requestId:randomUUID(),title:"Idle child",parentId:f.issue.id};
|
|
const beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
let ready=()=>{},holderPid=0,nativePid=0,idlePid=0;
|
|
const locked=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:agents.id}).from(agents).where(and(eq(agents.id,f.agent.id),eq(agents.companyId,f.company.id))).for("update");
|
|
ready();await gate;
|
|
});
|
|
await locked;
|
|
native=authority.execute(call).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [waiter]=await db.execute<{pid:number}>(sql`select waiter.pid from pg_stat_activity waiter
|
|
inner join pg_locks waiting on waiting.pid=waiter.pid and waiting.locktype='transactionid' and not waiting.granted
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='transactionid' and held.granted
|
|
and held.transactionid=waiting.transactionid
|
|
where waiter.datname=current_database() and waiter.wait_event_type='Lock' and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and exists(select 1 from pg_locks tuple_lock where tuple_lock.pid=waiter.pid and tuple_lock.locktype='tuple'
|
|
and tuple_lock.relation='agents'::regclass and tuple_lock.mode='AccessExclusiveLock' and tuple_lock.granted)
|
|
and (select count(distinct relation) from pg_locks context_lock where context_lock.pid=waiter.pid
|
|
and context_lock.locktype='relation' and context_lock.mode='RowShareLock' and context_lock.granted
|
|
and context_lock.relation in ('issues'::regclass,'heartbeat_runs'::regclass,'agents'::regclass))=3`);
|
|
nativePid=waiter?.pid??0;return nativePid;
|
|
},{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
idle=f.broker.act(f.subject,command).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [waiter]=await db.execute<{pid:number}>(sql`select waiter.pid from pg_stat_activity waiter
|
|
where waiter.datname=current_database() and waiter.wait_event_type='Lock' and ${nativePid}=any(pg_blocking_pids(waiter.pid))`);
|
|
idlePid=waiter?.pid??0;return idlePid;
|
|
},{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
const privacyKey=`issue-privacy-tree:${f.company.id}`;
|
|
const privacyOwners=await db.execute<{pid:number}>(sql`select pid from pg_locks where locktype='advisory' and granted
|
|
and pid in (${nativePid},${idlePid}) and objsubid=1
|
|
and classid=((hashtextextended(${privacyKey},0)>>32)&4294967295)::oid
|
|
and objid=(hashtextextended(${privacyKey},0)&4294967295)::oid order by pid`);
|
|
expect(privacyOwners).toHaveLength(1);
|
|
const observed=JSON.stringify({holderPid,nativePid,idlePid,privacyOwners:privacyOwners.map(row=>row.pid)});
|
|
release();await holder;
|
|
const [nativeOutcome,idleOutcome]=await Promise.all([native,idle]);
|
|
expect(nativeOutcome.error,observed).toBeUndefined();expect(idleOutcome.error,observed).toBeUndefined();
|
|
expect(nativeOutcome.result).toMatchObject({disposition:"applied",task:{parentId:f.issue.id,status:"todo",assigneeActorId:f.agent.id}});
|
|
expect(idleOutcome.result).toMatchObject({status:"created"});
|
|
const nativeChildId=(nativeOutcome.result as {task:{id:string}}).task.id,idleChildId=String((idleOutcome.result as {issueId:string}).issueId);
|
|
expect(nativeChildId).not.toBe(idleChildId);
|
|
const children=await db.select().from(issues).where(and(eq(issues.companyId,f.company.id),eq(issues.parentId,f.issue.id)));
|
|
expect(children.map(child=>child.id).sort()).toEqual([nativeChildId,idleChildId].sort());
|
|
const [run]=await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId));
|
|
expect(run.status).toBe("running");
|
|
expect((run.resultJson as {semanticToolReceipts:Record<string,unknown>}).semanticToolReceipts[call.arguments.idempotencyKey])
|
|
.toMatchObject({operationId:"create_task",input:call.arguments,result:nativeOutcome.result});
|
|
const receipts=await db.select().from(museIdleReceipts).where(and(eq(museIdleReceipts.bindingId,f.binding.id),eq(museIdleReceipts.bindingGeneration,f.binding.generation),eq(museIdleReceipts.requestId,command.requestId)));
|
|
expect(receipts).toHaveLength(1);expect(receipts[0].outcome).toEqual(idleOutcome.result);
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(nativeOutcome.result);
|
|
expect(await f.broker.act(f.subject,command)).toEqual(idleOutcome.result);
|
|
await expect(authority.execute({...call,callId:randomUUID(),arguments:{...call.arguments,title:"Changed native child"}})).rejects.toThrow("idempotency_conflict");
|
|
await expect(f.broker.act(f.subject,{...command,title:"Changed idle child"})).rejects.toThrow("changed input");
|
|
expect(await db.select({id:issues.id}).from(issues).where(and(eq(issues.companyId,f.company.id),eq(issues.parentId,f.issue.id)))).toHaveLength(2);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toMatchObject({status:"accepted",acceptedResultAt:null,finalizedAt:null});
|
|
}finally{release();await Promise.allSettled([holder,native,idle]);await f.detach();}
|
|
});
|
|
it("keeps native document writes independent of the task-creation privacy lock",async()=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},holder:Promise<void>|undefined,write:Promise<{result?:unknown;error?:unknown}>|undefined,outcome:{result?:unknown;error?:unknown}|undefined;
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}};
|
|
const authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,
|
|
museRuntime:true,externalAuthorizingUserId:f.operatorId,assertBridgeAuthority:(database=db)=>f.broker.assertRunAuthority(execution,database)});
|
|
let ready=()=>{};const locked=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${`issue-privacy-tree:${f.company.id}`},0))`);ready();await gate;
|
|
});
|
|
await locked;
|
|
const call={tool:"write_document",callId:randomUUID(),arguments:{key:"report",title:"Report",body:"Unrelated native mutation",baseRevisionId:null,idempotencyKey:randomUUID()}};
|
|
write=authority.execute(call).then(result=>(outcome={result}),error=>(outcome={error}));
|
|
await expect.poll(()=>outcome!==undefined,{timeout:5000,interval:25}).toBe(true);
|
|
expect(outcome?.error).toBeUndefined();expect(outcome?.result).toMatchObject({disposition:"applied",document:{body:call.arguments.body}});
|
|
expect(await documentService(db).getIssueDocumentByKey(f.issue.id,"report")).toMatchObject({body:call.arguments.body});
|
|
expect(await authority.execute({...call,callId:randomUUID()})).toEqual(outcome?.result);
|
|
expect(await documentService(db).listIssueDocumentRevisions(f.issue.id,"report")).toHaveLength(1);
|
|
}finally{release();await Promise.allSettled([holder,write]);await f.detach();}
|
|
});
|
|
it.each(["report_progress","set_task_title","submit_complaint","submit_suggestion"].flatMap(tool=>[
|
|
{tool,replay:false,fault:"grant_revoked" as const},
|
|
{tool,replay:true,fault:"qualification_expired" as const},
|
|
]))("rechecks local Muse mutation authority after a real task-lock wait ($tool/$fault/replay=$replay)",async({tool,replay,fault})=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},holder:Promise<void>|undefined,operation:Promise<{result?:unknown;error?:unknown}>|undefined;
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}};
|
|
const checks:string[]=[],authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,
|
|
museRuntime:true,externalAuthorizingUserId:f.operatorId,assertBridgeAuthority:async(database=db)=>{
|
|
await f.broker.assertRunAuthority(execution,database);checks.push(database===db?"discovery":"transaction");
|
|
}});
|
|
const idempotencyKey=randomUUID(),body="Muse current-authority regression",title="Current-authority title";
|
|
const call={tool,callId:randomUUID(),arguments:tool==="set_task_title"?{title,onlyIfProvisional:false,idempotencyKey}:{body,idempotencyKey}};
|
|
const snapshot=async()=>({
|
|
issue:await db.select({title:issues.title,status:issues.status,statusVersion:issues.statusVersion}).from(issues).where(eq(issues.id,f.issue.id)),
|
|
run:await db.select({status:heartbeatRuns.status,resultJson:heartbeatRuns.resultJson}).from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId)),
|
|
comments:await db.select({id:issueComments.id,body:issueComments.body}).from(issueComments).where(eq(issueComments.issueId,f.issue.id)).orderBy(issueComments.id),
|
|
commentary:await db.select({id:agentCommentary.id,kind:agentCommentary.kind,body:agentCommentary.body,idempotencyKey:agentCommentary.idempotencyKey}).from(agentCommentary).where(eq(agentCommentary.runId,f.runId)).orderBy(agentCommentary.id),
|
|
activity:await db.select({id:activityLog.id,action:activityLog.action,entityId:activityLog.entityId}).from(activityLog).where(eq(activityLog.companyId,f.company.id)).orderBy(activityLog.id),
|
|
});
|
|
if(replay){
|
|
const first=await authority.execute(call),second=await authority.execute({...call,callId:randomUUID()});
|
|
if(tool==="report_progress"){
|
|
expect(first).toMatchObject({issueId:f.issue.id,disposition:"applied"});expect(second).toEqual(first);
|
|
expect(await db.select({body:issueComments.body}).from(issueComments).where(eq(issueComments.issueId,f.issue.id))).toEqual([{body}]);
|
|
}else if(tool==="set_task_title"){
|
|
expect(first).toMatchObject({id:f.issue.id,title,changed:true});expect(second).toEqual(first);
|
|
expect((await db.select({title:issues.title}).from(issues).where(eq(issues.id,f.issue.id)))[0]).toEqual({title});
|
|
}else{
|
|
const kind=tool==="submit_complaint"?"complaint":"suggestion";
|
|
expect(first).toMatchObject({kind,replayed:false});expect(second).toEqual({...(first as Record<string,unknown>),replayed:true});
|
|
expect(await db.select({kind:agentCommentary.kind,body:agentCommentary.body}).from(agentCommentary).where(eq(agentCommentary.runId,f.runId))).toEqual([{kind,body}]);
|
|
}
|
|
}
|
|
const before=await snapshot(),beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
checks.length=0;
|
|
let ready=()=>{},holderPid=0;const locked=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:issues.id}).from(issues).where(and(eq(issues.companyId,f.company.id),eq(issues.id,f.issue.id))).for("update");ready();await gate;
|
|
});
|
|
await locked;
|
|
operation=authority.execute({...call,callId:randomUUID()}).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [observed]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_stat_activity waiter
|
|
inner join pg_locks waiting on waiting.pid=waiter.pid and waiting.locktype='transactionid' and not waiting.granted
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='transactionid' and held.granted and held.transactionid=waiting.transactionid
|
|
where waiter.datname=current_database() and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and exists(select 1 from pg_locks tuple_lock where tuple_lock.pid=waiter.pid and tuple_lock.locktype='tuple'
|
|
and tuple_lock.relation='issues'::regclass and tuple_lock.mode='AccessExclusiveLock' and tuple_lock.granted)) as waiting`);
|
|
return observed.waiting;
|
|
},{timeout:5000,interval:25}).toBe(true);
|
|
expect(checks).toEqual(["discovery"]);
|
|
if(fault==="grant_revoked")await db.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
else await db.update(bindings).set({qualificationId:randomUUID(),qualificationExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
await expect(f.broker.assertRunAuthority(execution)).rejects.toMatchObject({status:403});
|
|
release();await holder;
|
|
const outcome=await operation;
|
|
expect(outcome.error).toMatchObject({status:403});expect(outcome.result).toBeUndefined();
|
|
expect(await snapshot()).toEqual(before);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
expect((await db.select({status:assignments.status,acceptedResultAt:assignments.acceptedResultAt,finalizedAt:assignments.finalizedAt}).from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual({status:"accepted",acceptedResultAt:null,finalizedAt:null});
|
|
}finally{release();await Promise.allSettled([holder,operation]);await f.detach();}
|
|
});
|
|
it("rechecks native child creation authority after the real company privacy-lock wait",async()=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},holder:Promise<void>|undefined,operation:Promise<{result?:unknown;error?:unknown}>|undefined;
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const execution={binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}};
|
|
const checks:string[]=[],authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:f.issue.id,runId:f.runId,
|
|
museRuntime:true,externalAuthorizingUserId:f.operatorId,assertBridgeAuthority:async(database=db)=>{
|
|
await f.broker.assertRunAuthority(execution,database);checks.push(database===db?"discovery":"transaction");
|
|
}});
|
|
const idempotencyKey=randomUUID(),call={tool:"create_task",callId:randomUUID(),arguments:{title:"Must not create after qualification expiry",idempotencyKey}};
|
|
const beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
let ready=()=>{},holderPid=0;const locked=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${`issue-privacy-tree:${f.company.id}`},0))`);ready();await gate;
|
|
});
|
|
await locked;
|
|
operation=authority.execute(call).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [observed]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_locks waiting
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='advisory' and held.granted
|
|
and held.database=waiting.database and held.classid=waiting.classid and held.objid=waiting.objid and held.objsubid=waiting.objsubid
|
|
where waiting.locktype='advisory' and not waiting.granted and ${holderPid}=any(pg_blocking_pids(waiting.pid))) as waiting`);
|
|
return observed.waiting;
|
|
},{timeout:5000,interval:25}).toBe(true);
|
|
expect(checks).toEqual(["discovery"]);
|
|
await db.update(bindings).set({qualificationId:randomUUID(),qualificationExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
await expect(f.broker.assertRunAuthority(execution)).rejects.toMatchObject({status:403});
|
|
release();await holder;
|
|
const outcome=await operation;
|
|
expect(outcome.error).toMatchObject({status:403});expect(outcome.result).toBeUndefined();
|
|
expect(await db.select({id:issues.id}).from(issues).where(and(eq(issues.companyId,f.company.id),eq(issues.parentId,f.issue.id)))).toEqual([]);
|
|
expect(await db.select({id:activityLog.id}).from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"issue.created")))).toEqual([]);
|
|
const [run]=await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,f.runId));
|
|
expect((run.resultJson as {semanticToolReceipts?:Record<string,unknown>}|null)?.semanticToolReceipts?.[idempotencyKey]).toBeUndefined();expect(run.status).toBe("running");
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
expect((await db.select({status:assignments.status,acceptedResultAt:assignments.acceptedResultAt,finalizedAt:assignments.finalizedAt}).from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual({status:"accepted",acceptedResultAt:null,finalizedAt:null});
|
|
}finally{release();await Promise.allSettled([holder,operation]);await f.detach();}
|
|
});
|
|
async function detectorCleanupFixture(provided?:Awaited<ReturnType<typeof oauthFixture>>) {
|
|
const f=provided??await oauthFixture(true),qualificationId=randomUUID(),expiresAt=new Date(Date.now()-60_000);
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
await db.update(bindings).set({qualificationId,qualificationStartedAt:new Date(expiresAt.getTime()-86_400_000),qualificationExpiresAt:expiresAt}).where(eq(bindings.id,f.binding.id));
|
|
await f.broker.endQualification({companyId:f.company.id,agentId:f.agent.id,bindingId:f.binding.id,qualificationId,operatorId:f.operatorId});
|
|
const [binding]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));
|
|
const input={bindingId:binding.id,generation:binding.generation,expectedRevision:binding.revision,qualificationId,hookScript:`~/hooks/scripts/paperclip-muse-${binding.id}.sh`,detectorRemoved:true as const,
|
|
observation:{method:"hooks.list" as const,includeDisabled:true as const,hookId:"paperclip-synthetic-hook",outcome:"exact_hook_absent" as const,observedAt:new Date().toISOString(),sha256:"d".repeat(64)}};
|
|
const app=express();app.use(express.json());app.use(async(req,_res,next)=>{
|
|
const access=await boardAuthService(db).resolveBoardAccess(f.operatorId);
|
|
Object.assign(req,{actor:{type:"board",source:"session",userId:access.user?.id,userName:access.user?.name??null,userEmail:access.user?.email??null,companyIds:access.companyIds,memberships:access.memberships,isInstanceAdmin:access.isInstanceAdmin}});next();
|
|
});app.use(museBoardRoutes(db,oauthOrigin));app.use(activityRoutes(db));app.use(errorHandler);
|
|
const path=`/companies/${f.company.id}/agents/${f.agent.id}/muse-binding/attest-detector-cleanup`;
|
|
return {...f,binding,input,app,path};
|
|
}
|
|
it("records exact supported-runtime detector removal after OAuth expiry without restoring worker authority",async()=>{
|
|
const f=await detectorCleanupFixture();
|
|
await expect(f.identity.authenticateOAuthAccess(f.tokens!.access_token,oauthOrigin)).rejects.toThrow();
|
|
const beforeTokens=await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!));
|
|
const beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.agentId,f.agent.id));
|
|
await instanceSettingsService(db).updateExperimental({enableMuse:false});
|
|
try {
|
|
const result=await request(f.app).post(f.path).send(f.input);expect(result.status).toBe(200);expect(result.headers["cache-control"]).toBe("no-store");
|
|
expect(result.body).toMatchObject({revision:f.binding.revision+1,evidence:{source:"operator_runtime_attestation",operatorId:f.operatorId,hookId:f.input.observation.hookId,bindingId:f.binding.id,generation:f.binding.generation,qualificationId:f.input.qualificationId}});
|
|
const projected=await f.broker.bindingForAgent(f.company.id,f.agent.id);expect(projected!.cleanup.detectorEvidence).toEqual(result.body.evidence);expect(projected!.cleanup.attestationEligible).toBe(false);
|
|
const evidence=await f.broker.readQualificationEvidence(f.company.id,f.agent.id,f.binding.id,f.input.qualificationId);expect(evidence.detectorCleanup).toEqual(result.body.evidence);
|
|
expect((await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!)))).toEqual(beforeTokens);expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.agentId,f.agent.id)))).toEqual(beforeHolds);
|
|
await expect(f.identity.authenticateOAuthAccess(f.tokens!.access_token,oauthOrigin)).rejects.toThrow();
|
|
const rows=await db.select().from(activityLog).where(and(eq(activityLog.entityId,f.agent.id),eq(activityLog.action,"muse.detector_cleanup_attested")));expect(rows).toHaveLength(1);expect(rows[0]).toMatchObject({actorType:"user",actorId:f.operatorId,responsibleUserId:f.operatorId,agentId:f.agent.id});
|
|
expect(rows[0].details).toMatchObject({method:"hooks.list",includeDisabled:true,outcome:"exact_hook_absent",externalStopConfirmed:false});
|
|
expect(JSON.stringify(rows)).not.toContain(f.tokens!.access_token);expect(JSON.stringify(rows)).not.toContain(f.tokens!.refresh_token);
|
|
}finally{await instanceSettingsService(db).updateExperimental({enableMuse:true});}
|
|
});
|
|
it.each(["generation","revision","qualification","script","binding"])("rejects stale detector cleanup %s",async field=>{
|
|
const f=await detectorCleanupFixture(),input=structuredClone(f.input);
|
|
if(field==="generation")input.generation++;if(field==="revision")input.expectedRevision++;if(field==="qualification")input.qualificationId=randomUUID();if(field==="binding")input.bindingId=randomUUID();if(field==="script")input.hookScript="~/hooks/scripts/other.sh";
|
|
expect((await request(f.app).post(f.path).send(input)).status).toBe(409);
|
|
const [b]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));expect(b.detectorRemovedAt).toBeNull();expect(b.revision).toBe(f.binding.revision);
|
|
});
|
|
it("supersedes only stale worker detector provenance with an exact post-deadline operator observation",async()=>{
|
|
const original=await oauthFixture(true);
|
|
const reported=await request(transport()).post("/api/muse/v1/detector-cleanup").set("Authorization",`Bearer ${original.tokens!.access_token}`).send({version:1,requestId:randomUUID(),bindingId:original.binding.id,generation:original.binding.generation,detectorRemoved:true});expect(reported.status).toBe(200);
|
|
const [workerRow]=await db.select().from(activityLog).where(and(eq(activityLog.entityId,original.agent.id),eq(activityLog.action,"muse.detector_removal_reported")));
|
|
const [workerBinding]=await db.select().from(bindings).where(eq(bindings.id,original.binding.id));
|
|
const f=await detectorCleanupFixture(original);
|
|
await db.update(bindings).set({qualificationExpiresAt:new Date(workerRow.createdAt.getTime()+1),qualificationStartedAt:new Date(workerRow.createdAt.getTime()+1-86_400_000)}).where(eq(bindings.id,f.binding.id));
|
|
const before=await f.broker.bindingForAgent(f.company.id,f.agent.id);expect(before!.cleanup.detectorEvidence).toMatchObject({source:"authenticated_worker_report",recordedAt:workerRow.createdAt.toISOString()});expect(before!.cleanup.attestationEligible).toBe(true);
|
|
const result=await request(f.app).post(f.path).send(f.input);expect(result.status).toBe(200);
|
|
const after=await f.broker.bindingForAgent(f.company.id,f.agent.id);expect(after!.cleanup.detectorEvidence).toEqual(result.body.evidence);expect(after!.cleanup.detectorEvidence?.source).toBe("operator_runtime_attestation");expect(after!.cleanup.attestationEligible).toBe(false);
|
|
expect(await db.select().from(activityLog).where(eq(activityLog.id,workerRow.id))).toEqual([workerRow]);
|
|
expect(workerBinding.detectorRemovedAt).not.toBeNull();
|
|
expect((await f.broker.readQualificationEvidence(f.company.id,f.agent.id,f.binding.id,f.input.qualificationId)).detectorCleanup).toEqual(result.body.evidence);
|
|
await expect(f.identity.authenticateOAuthAccess(f.tokens!.access_token,oauthOrigin)).rejects.toThrow();
|
|
});
|
|
it("does not clear claimed-worker or unknown-native-effect holds when recording detector removal",async()=>{
|
|
const oauth=await oauthFixture(true),activeRun=await active(oauth);
|
|
try {
|
|
expect((await activeRun.broker.act(activeRun.subject,{version:1,command:"accept",assignmentId:activeRun.assignment.id,requestId:randomUUID()})).status).toBe("accepted");
|
|
const f=await detectorCleanupFixture({...oauth,...activeRun});
|
|
await db.update(externalAgentHolds).set({nativeEffectsUnknown:true}).where(eq(externalAgentHolds.assignmentId,activeRun.assignment.id));
|
|
const before=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,activeRun.assignment.id));expect(before).toHaveLength(1);expect(before[0]).toMatchObject({workerUnknown:true,nativeEffectsUnknown:true,releasedAt:null});
|
|
expect((await request(f.app).post(f.path).send(f.input)).status).toBe(200);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,activeRun.assignment.id))).toEqual(before);
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))!.stop).toMatchObject({status:"cannot_confirm",nativeEffectsUnknown:true});
|
|
}finally{await activeRun.detach();}
|
|
});
|
|
it("requires a full supported hook listing and exact absence, not disablement or arbitrary evidence",async()=>{
|
|
const f=await detectorCleanupFixture();
|
|
for(const observation of [{...f.input.observation,includeDisabled:false},{...f.input.observation,includeDisabled:undefined},{...f.input.observation,outcome:"disabled"},{...f.input.observation,method:"summary"},{...f.input.observation,hookId:"https://private.example"},{...f.input.observation,sha256:"private-canary"},{...f.input.observation,url:"https://private.example"}])expect((await request(f.app).post(f.path).send({...f.input,observation})).status).toBe(400);
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))!.cleanup.detectorEvidence).toBeNull();
|
|
});
|
|
it("keeps detector evidence atomic, serialized and exact-replay idempotent",async()=>{
|
|
const f=await detectorCleanupFixture(),spy=vi.spyOn(activityLogModule,"logActivity").mockRejectedValueOnce(new Error("synthetic audit storage failure"));
|
|
try{expect((await request(f.app).post(f.path).send(f.input)).status).toBe(500);}finally{spy.mockRestore();}
|
|
let [b]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));expect(b.revision).toBe(f.binding.revision);expect(b.detectorRemovedAt).toBeNull();expect(b.detectorRemovalRequestedAt).toBeNull();
|
|
const replies=await Promise.all([request(f.app).post(f.path).send(f.input),request(f.app).post(f.path).send(f.input)]);expect(replies.map(r=>r.status)).toEqual([200,200]);expect(replies[0].body).toEqual(replies[1].body);
|
|
expect(await db.select().from(activityLog).where(and(eq(activityLog.entityId,f.agent.id),eq(activityLog.action,"muse.detector_cleanup_attested")))).toHaveLength(1);
|
|
expect((await request(f.app).post(f.path).send({...f.input,observation:{...f.input.observation,sha256:"e".repeat(64)}})).status).toBe(409);
|
|
[b]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));await db.update(bindings).set({revision:b.revision+1}).where(eq(bindings.id,b.id));expect((await request(f.app).post(f.path).send(f.input)).status).toBe(409);
|
|
});
|
|
it("does not trust public activity forgery as detector evidence or a completed replay",async()=>{
|
|
const f=await detectorCleanupFixture();
|
|
const forged=await request(f.app).post(`/companies/${f.company.id}/activity`).send({actorType:"user",actorId:f.operatorId,agentId:f.agent.id,responsibleUserId:f.operatorId,action:"muse.detector_cleanup_attested",entityType:"agent",entityId:f.agent.id,details:{...f.input,source:"operator_runtime_attestation",method:"hooks.list",includeDisabled:true,hookId:f.input.observation.hookId,outcome:"exact_hook_absent",recordedAt:new Date().toISOString(),observedAt:f.input.observation.observedAt}});expect(forged.status).toBe(201);expect(forged.body.responsibleUserId).toBeNull();
|
|
await db.update(bindings).set({detectorRemovalRequestedAt:new Date(),detectorRemovedAt:new Date()}).where(eq(bindings.id,f.binding.id));
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))!.cleanup.detectorEvidence).toBeNull();
|
|
expect((await request(f.app).post(f.path).send(f.input)).status).toBe(200);
|
|
const trusted=await f.broker.bindingForAgent(f.company.id,f.agent.id);expect(trusted!.cleanup.detectorEvidence?.source).toBe("operator_runtime_attestation");
|
|
});
|
|
it("rejects premature, future, replaced and no-longer-authorized detector attestations",async()=>{
|
|
const f=await detectorCleanupFixture();
|
|
for(const observedAt of [new Date(f.binding.revokedAt!.getTime()-1).toISOString(),new Date(Date.now()+60_000).toISOString()])expect((await request(f.app).post(f.path).send({...f.input,observation:{...f.input.observation,observedAt}})).status).toBe(409);
|
|
await db.update(companyMemberships).set({membershipRole:"viewer"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));
|
|
expect((await request(f.app).post(f.path).send(f.input)).status).toBe(403);
|
|
await db.update(companyMemberships).set({membershipRole:"owner"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));
|
|
await db.insert(bindings).values({companyId:f.company.id,agentId:f.agent.id,operatorId:f.operatorId,generation:f.binding.generation+1,status:"pairing"});
|
|
expect((await request(f.app).post(f.path).send(f.input)).status).toBe(409);
|
|
expect((await f.broker.readQualificationEvidence(f.company.id,f.agent.id,f.binding.id,f.input.qualificationId)).detectorCleanup).toBeNull();
|
|
});
|
|
it("consumes a ten-minute ticket once and separates credential lanes",async()=>{
|
|
const f=await fixture(false);await expect(f.identity.pair(f.ticket,"2")).rejects.toThrow("consumed");
|
|
expect(f.credentials).not.toHaveProperty("signalToken");
|
|
await expect(f.identity.authenticate(f.credentials.cleanupToken)).rejects.toThrow();
|
|
await expect(f.identity.authenticate(f.credentials.accessToken,"signal")).rejects.toThrow();
|
|
const legacySignal=randomBytes(32).toString("base64url");await db.insert(museCredentials).values({companyId:f.company.id,bindingId:f.binding.id,bindingGeneration:1,kind:"signal",familyId:randomUUID(),tokenHash:museCredentialHash(legacySignal),expiresAt:new Date(Date.now()+600000)});
|
|
await expect(f.identity.authenticate(legacySignal,"signal")).rejects.toThrow("no longer supported");
|
|
await db.delete(museCredentials).where(eq(museCredentials.tokenHash,museCredentialHash(legacySignal)));
|
|
const rows=await db.select().from(museCredentials).where(eq(museCredentials.bindingId,f.binding.id));
|
|
expect(rows).toHaveLength(4);expect(rows.some(r=>r.kind==="signal")).toBe(false);expect(rows.some(r=>r.tokenHash===f.credentials.accessToken)).toBe(false);
|
|
expect(new Date(f.credentials.accessExpiresAt).getTime()-Date.now()).toBeLessThanOrEqual(15*60000);
|
|
});
|
|
it("rotates refresh once and commits family revocation on consumed-token replay",async()=>{
|
|
const f=await fixture(),rotated=await f.identity.refresh(f.credentials.refreshToken);
|
|
await expect(f.identity.authenticate(f.credentials.accessToken)).rejects.toThrow();
|
|
await expect(f.identity.refresh(f.credentials.refreshToken)).rejects.toThrow("replay");
|
|
await expect(f.identity.authenticate(rotated.accessToken)).rejects.toThrow();
|
|
await expect(museReceiver(db).signal(f.binding.id)).rejects.toMatchObject({status:410});
|
|
});
|
|
it("polling does not extend refresh inactivity and independent contact creates the first challenge",async()=>{
|
|
const f=await fixture(false);
|
|
const [before]=await db.select().from(museCredentials).where(and(eq(museCredentials.bindingId,f.binding.id),eq(museCredentials.kind,"refresh")));
|
|
expect((await museReceiver(db).signal(f.binding.id)).signal).toBeNull();await museReceiver(db).flush();
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))?.backgroundReplyVerified).toBe(false);
|
|
const [after]=await db.select().from(museCredentials).where(eq(museCredentials.id,before.id));expect(after.expiresAt).toEqual(before.expiresAt);
|
|
const items=await f.broker.inspect(f.subject,{version:1,query:"mailbox",after:0}) as {items:Array<{references:{nonce:string}}>};expect(items.items).toHaveLength(1);
|
|
const requestId = randomUUID(), nonce = items.items[0].references.nonce;
|
|
await f.broker.act(f.subject,{version:1,command:"challenge.confirm",nonce,requestId});
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))?.backgroundReplyVerified).toBe(true);
|
|
const logs = await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"muse.background_reply_verified")));
|
|
expect(logs).toHaveLength(1);
|
|
expect(logs[0]).toMatchObject({actorType:"agent",actorId:f.agent.id,agentId:f.agent.id,responsibleUserId:f.operatorId,details:{bindingId:f.binding.id,generation:1,authorizingUserId:f.operatorId,requestId}});
|
|
expect(JSON.stringify(logs)).not.toContain(nonce);
|
|
});
|
|
it("keeps expired public readiness quiet after an acknowledged item becomes eligible for rearm",async()=>{
|
|
const f=await fixture();
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});
|
|
const [challenge]=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge")));
|
|
await db.update(bindings).set({challengeExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
const poll=()=>museReceiver(db).signal(f.binding.id);
|
|
expect(await poll()).toEqual({version:1,signal:null});
|
|
await f.broker.inspect(f.subject,{version:1,query:"mailbox",after:challenge.id});
|
|
await db.update(mailbox).set({signalNotifiedAt:new Date(Date.now()-60000)}).where(eq(mailbox.id,challenge.id));
|
|
expect(await poll()).toEqual({version:1,signal:null});
|
|
expect((await db.select().from(mailbox).where(eq(mailbox.id,challenge.id)))[0].signalAttempt).toBe(0);
|
|
});
|
|
it("signals only the current unread public readiness nonce with a stable reference",async()=>{
|
|
const f=await fixture();
|
|
const verify=async()=>{const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;return f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});};
|
|
const firstPending=await verify();expect(await verify()).toEqual(firstPending);
|
|
expect(await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge")))).toHaveLength(1);
|
|
await db.update(bindings).set({challengeExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));await verify();
|
|
const challenges=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge"))).orderBy(mailbox.id);
|
|
expect(challenges).toHaveLength(2);
|
|
const current=(await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0];
|
|
expect(museCredentialHash(String(challenges[1].references.nonce))).toBe(current.challengeHash);
|
|
const poll=()=>museReceiver(db).signal(f.binding.id),first=await poll();
|
|
expect(first).toEqual({version:1,signal:{reference:"sha256:"+createHash("sha256").update(`${f.binding.id}:1:${challenges[1].sourceEventId}:1`).digest("hex")}});expect(await poll()).toEqual(first);
|
|
const saved=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge"))).orderBy(mailbox.id);
|
|
expect(saved.map(item=>item.signalAttempt)).toEqual([0,1]);
|
|
await f.broker.inspect(f.subject,{version:1,query:"mailbox",after:challenges[1].id});expect(await poll()).toEqual({version:1,signal:null});
|
|
await db.update(bindings).set({challengeExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));await verify();
|
|
const all=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge"))).orderBy(mailbox.id);
|
|
expect(all).toHaveLength(3);expect(await poll()).toEqual({version:1,signal:{reference:"sha256:"+createHash("sha256").update(`${f.binding.id}:1:${all[2].sourceEventId}:1`).digest("hex")}});
|
|
});
|
|
it.each(["automatic","board"] as const)("audits %s readiness only on first change with the actual actor",async source=>{
|
|
const f=await fixture(false),operatorId=source==="board"?randomUUID():undefined;
|
|
if(operatorId){
|
|
await db.insert(authUsers).values({id:operatorId,name:"Actual board caller",email:operatorId+"@example.test",createdAt:new Date(),updatedAt:new Date()});
|
|
await db.insert(companyMemberships).values({companyId:f.company.id,principalType:"user",principalId:operatorId,membershipRole:"owner",status:"active"});
|
|
await db.update(bindings).set({receiverContactAt:new Date()}).where(eq(bindings.id,f.binding.id));
|
|
}
|
|
const events:Array<Record<string,unknown>>=[];
|
|
const unsubscribe=subscribeCompanyLiveEvents(f.company.id,event=>{if(event.type==="activity.logged")events.push(event.payload);});
|
|
try {
|
|
if(source==="automatic"){
|
|
await museReceiver(db).signal(f.binding.id);await museReceiver(db).flush();
|
|
} else {
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision},operatorId);
|
|
}
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
const challenge=(await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge"))))[0]!;
|
|
const pending=await f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision},operatorId);
|
|
expect(pending).toMatchObject({revision:state.revision,expiresAt:state.challengeExpiresAt});
|
|
await expect(f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:2,expectedRevision:state.revision},operatorId)).rejects.toThrow("before verifying");
|
|
const logs=await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"muse.readiness_challenge_created")));
|
|
expect(logs).toHaveLength(1);
|
|
expect(logs[0]).toMatchObject({actorType:source==="board"?"user":"system",actorId:operatorId??"muse-runner",agentId:f.agent.id,responsibleUserId:operatorId??f.operatorId,entityType:"agent",entityId:f.agent.id,details:{bindingId:f.binding.id,generation:1,revision:state.revision,authorizingUserId:f.operatorId,expiresAt:state.challengeExpiresAt}});
|
|
expect(events.filter(event=>event.action==="muse.readiness_challenge_created")).toHaveLength(1);
|
|
expect((await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge"))))).toHaveLength(1);
|
|
const serialized=JSON.stringify({logs,events});
|
|
for(const secret of [f.ticket,f.credentials.accessToken,f.credentials.refreshToken,f.credentials.cleanupToken,f.credentials.detectorCleanupToken,String(challenge.references.nonce)])expect(serialized).not.toContain(secret);
|
|
} finally {unsubscribe();}
|
|
});
|
|
it("audits first qualification with the actual operator and preserves idempotent reads",async()=>{
|
|
const f=await fixture(),operatorId=randomUUID();
|
|
await db.insert(authUsers).values({id:operatorId,name:"Qualification board caller",email:operatorId+"@example.test",createdAt:new Date(),updatedAt:new Date()});
|
|
await db.insert(companyMemberships).values({companyId:f.company.id,principalType:"user",principalId:operatorId,membershipRole:"owner",status:"active"});
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!,qualificationId=randomUUID();
|
|
const input={companyId:f.company.id,agentId:f.agent.id,bindingId:f.binding.id,generation:1,expectedRevision:state.revision,qualificationId,expiresAt:new Date(Date.now()+24*60*60_000),operatorId};
|
|
const events:Array<Record<string,unknown>>=[];
|
|
const unsubscribe=subscribeCompanyLiveEvents(f.company.id,event=>{if(event.type==="activity.logged")events.push(event.payload);});
|
|
try {
|
|
const first=await f.broker.beginQualification(input);
|
|
expect(await f.broker.beginQualification({...input,expiresAt:new Date(Date.now()+60_000)})).toEqual(first);
|
|
expect(await f.broker.beginQualification({...input,expectedRevision:first.revision})).toEqual(first);
|
|
await f.broker.readQualificationEvidence(f.company.id,f.agent.id,f.binding.id,qualificationId);
|
|
await expect(f.broker.beginQualification({...input,companyId:randomUUID()})).rejects.toThrow("Connection changed");
|
|
const logs=await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"muse.qualification_started")));
|
|
expect(logs).toHaveLength(1);
|
|
expect(logs[0]).toMatchObject({actorType:"user",actorId:operatorId,agentId:f.agent.id,responsibleUserId:operatorId,entityType:"agent",entityId:f.agent.id,details:{bindingId:f.binding.id,generation:1,revision:first.revision,authorizingUserId:f.operatorId,qualificationId,startedAt:first.startedAt,expiresAt:first.expiresAt}});
|
|
expect(events.filter(event=>event.action==="muse.qualification_started")).toHaveLength(1);
|
|
const serialized=JSON.stringify({logs,events});
|
|
for(const secret of [f.ticket,f.credentials.accessToken,f.credentials.refreshToken,f.credentials.cleanupToken,f.credentials.detectorCleanupToken])expect(serialized).not.toContain(secret);
|
|
} finally {unsubscribe();}
|
|
});
|
|
it("retains a mailbox batch until its prior durable cursor is acknowledged",async()=>{
|
|
const f=await fixture();await db.insert(mailbox).values({companyId:f.company.id,bindingId:f.binding.id,bindingGeneration:1,kind:"follow_up",sourceEventId:randomUUID(),references:{}});
|
|
const batch=await f.broker.inspect(f.subject,{version:1,query:"mailbox",after:0}) as {nextCursor:number};
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].workerCursor).toBe(0);
|
|
await f.broker.inspect(f.subject,{version:1,query:"mailbox",after:batch.nextCursor});
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].workerCursor).toBe(batch.nextCursor);
|
|
});
|
|
it("filters current idle task visibility even with global privacy disabled",async()=>{
|
|
vi.stubEnv("PAPERCLIP_ISSUE_PRIVACY_MODE","off");const f=await fixture();
|
|
const other=randomUUID();await db.insert(authUsers).values({id:other,name:"other",email:other+"@example.test",createdAt:new Date(),updatedAt:new Date()});
|
|
const [issue]=await db.insert(issues).values({companyId:f.company.id,title:"secret",visibility:"private",responsibleUserId:other,assigneeAgentId:f.agent.id}).returning();
|
|
expect((await f.broker.inspect(f.subject,{version:1,query:"task.list"}) as {tasks:unknown[]}).tasks).toEqual([]);
|
|
await db.insert(issueAccessGrants).values({issueId:issue.id,subjectType:"user",subjectId:f.operatorId,source:"explicit"});
|
|
expect((await f.broker.inspect(f.subject,{version:1,query:"task.list"}) as {tasks:unknown[]}).tasks).toHaveLength(1);
|
|
await db.update(companyMemberships).set({status:"inactive"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));
|
|
await expect(f.broker.inspect(f.subject,{version:1,query:"task.read",issueId:issue.id})).rejects.toThrow();vi.unstubAllEnvs();
|
|
});
|
|
async function idleReadFixture() {
|
|
const f=await oauthFixture(true);
|
|
const [allowedProject,outsideProject]=await db.insert(projects).values([
|
|
{companyId:f.company.id,name:"Allowed project"},{companyId:f.company.id,name:"Outside project"},
|
|
]).returning();
|
|
await db.update(agents).set({permissions:{trustPreset:"low_trust_review",authorizationPolicy:{trustBoundary:{mode:"low_trust_review",companyId:f.company.id,projectIds:[allowedProject.id]}}}}).where(eq(agents.id,f.agent.id));
|
|
const [allowed,outside]=await db.insert(issues).values([
|
|
{companyId:f.company.id,projectId:allowedProject.id,title:"idle-scope allowed",description:"allowed task content"},
|
|
{companyId:f.company.id,projectId:outsideProject.id,title:"idle-scope forbidden",description:"outside task content"},
|
|
]).returning();
|
|
for(const issue of [allowed,outside]) {
|
|
await db.insert(issueComments).values({companyId:f.company.id,issueId:issue.id,authorType:"user",authorUserId:f.operatorId,body:issue.title+" history"});
|
|
const [document]=await db.insert(documents).values({companyId:f.company.id,title:issue.title,latestBody:issue.title+" document"}).returning();
|
|
await db.insert(issueDocuments).values({companyId:f.company.id,issueId:issue.id,documentId:document.id,key:"plan"});
|
|
}
|
|
return {...f,allowedProject,outsideProject,allowed,outside};
|
|
}
|
|
it.each(["task.read","task.history","task.document.read"] as const)("enforces ordinary low-trust authorization for idle %s",async(query)=>{
|
|
const f=await idleReadFixture();
|
|
const request=(issueId:string)=>query==="task.document.read"?{version:1 as const,query,issueId,key:"plan"}:{version:1 as const,query,issueId};
|
|
await expect(f.broker.inspect(f.subject,request(f.outside.id))).rejects.toThrow("unavailable");
|
|
expect(JSON.stringify(await f.broker.inspect(f.subject,request(f.allowed.id)))).toContain("idle-scope allowed");
|
|
await db.update(agents).set({permissions:{trustPreset:"low_trust_review",authorizationPolicy:{trustBoundary:{mode:"low_trust_review",companyId:f.company.id,projectIds:[f.outsideProject.id]}}}}).where(eq(agents.id,f.agent.id));
|
|
await expect(f.broker.inspect(f.subject,request(f.allowed.id))).rejects.toThrow("unavailable");
|
|
expect(JSON.stringify(await f.broker.inspect(f.subject,request(f.outside.id)))).toContain("idle-scope forbidden");
|
|
});
|
|
it.each(["task.list","task.search"] as const)("filters idle %s with ordinary authorization without exposing restricted cursors",async(query)=>{
|
|
const f=await idleReadFixture();
|
|
const request=query==="task.search"?{version:1 as const,query,text:"idle-scope"}:{version:1 as const,query};
|
|
const page=await f.broker.inspect(f.subject,request) as {tasks:Array<{id:string;title:string}>,nextCursor:string|null};
|
|
expect(page.tasks.map(task=>task.id)).toEqual([f.allowed.id]);expect(page.nextCursor).toBe(f.allowed.id);
|
|
expect(JSON.stringify(page)).not.toContain(f.outside.id);expect(JSON.stringify(page)).not.toContain(f.outside.title);
|
|
});
|
|
it.each(["task.list","task.search"] as const)("fills bounded authorized pages after rejected idle %s candidates",async(query)=>{
|
|
const f=await idleReadFixture();await db.delete(issues).where(eq(issues.companyId,f.company.id));
|
|
const ids=(high:boolean)=>Array.from({length:51},()=>`${high?"f0000000":"00000000"}-${randomBytes(2).toString("hex")}-4${randomBytes(2).toString("hex").slice(1)}-8${randomBytes(2).toString("hex").slice(1)}-${randomBytes(6).toString("hex")}`).sort();
|
|
const denied=ids(false),allowed=ids(true);
|
|
await db.insert(issues).values([...denied.map(id=>({id,companyId:f.company.id,projectId:f.outsideProject.id,title:"pagination-forbidden"})),...allowed.map(id=>({id,companyId:f.company.id,projectId:f.allowedProject.id,title:"pagination-allowed"}))]);
|
|
const request=query==="task.search"?{version:1 as const,query,text:"pagination"}:{version:1 as const,query};
|
|
const first=await f.broker.inspect(f.subject,request) as {tasks:Array<{id:string}>,nextCursor:string};
|
|
expect(first.tasks.map(task=>task.id)).toEqual(allowed.slice(0,50));expect(first.nextCursor).toBe(allowed[49]);
|
|
const last=await f.broker.inspect(f.subject,{...request,after:first.nextCursor}) as {tasks:Array<{id:string}>,nextCursor:string};
|
|
expect(last.tasks.map(task=>task.id)).toEqual(allowed.slice(50));expect(last.nextCursor).toBe(allowed[50]);
|
|
expect((await f.broker.inspect(f.subject,{...request,after:last.nextCursor}) as {tasks:unknown[]}).tasks).toEqual([]);
|
|
for(const id of denied)expect(JSON.stringify([first,last])).not.toContain(id);
|
|
});
|
|
it.each(["task.list","task.search"] as const)("fails closed at the authorized scan limit for idle %s and accepts narrower search",async(query)=>{
|
|
const f=await idleReadFixture();await db.delete(issues).where(eq(issues.companyId,f.company.id));
|
|
const denied=Array.from({length:201},()=>`00000000-${randomBytes(2).toString("hex")}-4${randomBytes(2).toString("hex").slice(1)}-8${randomBytes(2).toString("hex").slice(1)}-${randomBytes(6).toString("hex")}`);
|
|
const allowedId=`f0000000-${randomBytes(2).toString("hex")}-4${randomBytes(2).toString("hex").slice(1)}-8${randomBytes(2).toString("hex").slice(1)}-${randomBytes(6).toString("hex")}`;
|
|
await db.insert(issues).values([...denied.map(id=>({id,companyId:f.company.id,projectId:f.outsideProject.id,title:"bounded-collection forbidden"})),{id:allowedId,companyId:f.company.id,projectId:f.allowedProject.id,title:"bounded-collection narrow-match"}]);
|
|
const request=query==="task.search"?{version:1 as const,query,text:"bounded-collection"}:{version:1 as const,query};
|
|
await expect(f.broker.inspect(f.subject,request)).rejects.toMatchObject({status:422,message:"Task collection exceeds its read limit. Use task.search with a narrower query."});
|
|
const narrow=await f.broker.inspect(f.subject,{version:1,query:"task.search",text:"narrow-match"}) as {tasks:Array<{id:string}>,nextCursor:string};
|
|
expect(narrow.tasks.map(task=>task.id)).toEqual([allowedId]);expect(narrow.nextCursor).toBe(allowedId);
|
|
},30000);
|
|
it("uses only the current server-derived human-directed run context for native assignment reads",async()=>{
|
|
const f=await active(await oauthFixture(true));try {
|
|
const [project]=await db.insert(projects).values({companyId:f.company.id,name:"Unrelated allowed project"}).returning();
|
|
await db.update(agents).set({permissions:{trustPreset:"low_trust_review",authorizationPolicy:{trustBoundary:{mode:"low_trust_review",companyId:f.company.id,projectIds:[project.id]}}}}).where(eq(agents.id,f.agent.id));
|
|
await db.update(issues).set({visibility:"private"}).where(eq(issues.id,f.issue.id));
|
|
await db.update(heartbeatRuns).set({contextSnapshot:{issueId:f.issue.id}}).where(eq(heartbeatRuns.id,f.runId));
|
|
const [wake]=await db.insert(agentWakeupRequests).values({companyId:f.company.id,agentId:f.agent.id,runId:f.runId,source:"assignment",status:"claimed",requestedByActorType:"user",requestedByActorId:f.operatorId,payload:{issueId:f.issue.id,_paperclipWakeContext:{source:"issue.update"}}}).returning();
|
|
const idleRequest={version:1 as const,query:"task.read" as const,issueId:f.issue.id,runId:f.runId};
|
|
await expect(f.broker.inspect(f.subject,idleRequest)).rejects.toThrow("unavailable");
|
|
expect(await f.broker.inspect(f.subject,{version:1,query:"assignment.read",assignmentId:f.assignment.id})).toMatchObject({assignmentId:f.assignment.id,status:"offered"});
|
|
await db.update(agentWakeupRequests).set({status:"cancelled"}).where(eq(agentWakeupRequests.id,wake.id));
|
|
await expect(f.broker.inspect(f.subject,{version:1,query:"assignment.read",assignmentId:f.assignment.id})).rejects.toThrow("unavailable");
|
|
}finally{await f.detach();}
|
|
});
|
|
it("rechecks current authorizer grants for every idle task read surface",async()=>{
|
|
const f=await idleReadFixture(),other=randomUUID();
|
|
await db.insert(authUsers).values({id:other,name:"Other author",email:other+"@example.test",createdAt:new Date(),updatedAt:new Date()});
|
|
await db.update(issues).set({visibility:"private",responsibleUserId:other,assigneeAgentId:f.agent.id}).where(eq(issues.id,f.allowed.id));
|
|
const reads=[{version:1 as const,query:"task.read" as const,issueId:f.allowed.id},{version:1 as const,query:"task.history" as const,issueId:f.allowed.id},{version:1 as const,query:"task.document.read" as const,issueId:f.allowed.id,key:"plan"}];
|
|
for(const request of reads)await expect(f.broker.inspect(f.subject,request)).rejects.toThrow();
|
|
const [grant]=await db.insert(issueAccessGrants).values({issueId:f.allowed.id,subjectType:"user",subjectId:f.operatorId,source:"explicit"}).returning();
|
|
for(const request of reads)expect(JSON.stringify(await f.broker.inspect(f.subject,request))).toContain("idle-scope allowed");
|
|
await db.delete(issueAccessGrants).where(eq(issueAccessGrants.id,grant.id));
|
|
for(const request of reads)await expect(f.broker.inspect(f.subject,request)).rejects.toThrow();
|
|
expect((await f.broker.inspect(f.subject,{version:1,query:"task.list"}) as {tasks:unknown[]}).tasks).toEqual([]);
|
|
});
|
|
it("creates visible idle tasks with supported creation authority and a stable mutation receipt",async()=>{
|
|
const f=await fixture(),command={version:1 as const,command:"task.create" as const,requestId:randomUUID(),title:"Idle intake",description:"Ordinary task intake"};
|
|
const result=await f.broker.act(f.subject,command);
|
|
expect(result.status).toBe("created");expect(await f.broker.act(f.subject,command)).toEqual(result);
|
|
const [issue]=await db.select().from(issues).where(eq(issues.id,String(result.issueId)));
|
|
expect(issue.createdByAgentId).toBe(f.agent.id);expect(issue.responsibleUserId).toBe(f.operatorId);expect(issue.status).toBe("todo");expect(issue.sourceTrust).toBeNull();
|
|
await expect(f.broker.act(f.subject,{...command,title:"Changed intake"})).rejects.toThrow("changed input");
|
|
});
|
|
it("comments on an idle task using its complete current permission resource",async()=>{
|
|
const f=await fixture();const [issue]=await db.insert(issues).values({companyId:f.company.id,title:"Idle comment",status:"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
const command={version:1 as const,command:"task.comment" as const,requestId:randomUUID(),issueId:issue.id,body:"Idle coordination"};
|
|
const result=await f.broker.act(f.subject,command);expect(result.status).toBe("commented");expect(await f.broker.act(f.subject,command)).toEqual(result);
|
|
const rows=await db.select().from(issueComments).where(eq(issueComments.issueId,issue.id));
|
|
expect(rows).toHaveLength(1);expect(rows[0]).toMatchObject({authorAgentId:f.agent.id,authorUserId:null,onBehalfOfUserId:f.operatorId,body:command.body,sourceTrust:null});
|
|
await db.update(companyMemberships).set({membershipRole:"viewer"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));
|
|
await expect(f.broker.act(f.subject,{...command,requestId:randomUUID()})).rejects.toThrow("authority");
|
|
expect(await db.select().from(issueComments).where(eq(issueComments.issueId,issue.id))).toHaveLength(1);
|
|
});
|
|
it.each(["agent","project"] as const)("quarantines idle Muse output and retains the resolved %s trust boundary",async source=>{
|
|
const f=await fixture();
|
|
const [project]=await db.insert(projects).values({companyId:f.company.id,name:"Restricted idle coordination"}).returning();
|
|
const boundary={mode:"low_trust_review",companyId:f.company.id,projectIds:[project.id]};
|
|
const policy={trustPreset:"low_trust_review",trustBoundary:boundary};
|
|
if(source==="agent")await db.update(agents).set({permissions:{canCreateTasks:true,authorizationPolicy:policy}}).where(eq(agents.id,f.agent.id));
|
|
else await db.update(projects).set({executionWorkspacePolicy:{authorizationPolicy:policy}}).where(eq(projects.id,project.id));
|
|
const command={version:1 as const,command:"task.create" as const,requestId:randomUUID(),title:"Unreviewed idle task",description:"Untrusted task instructions",projectId:project.id};
|
|
const created=await f.broker.act(f.subject,command);
|
|
const [issue]=await db.select().from(issues).where(eq(issues.id,String(created.issueId)));
|
|
expect(issue.executionPolicy).toMatchObject({authorizationPolicy:policy});
|
|
expect(issue.sourceTrust).toMatchObject({preset:"low_trust_review",disposition:"quarantined",sourceIssueId:issue.id,sourceAgentId:f.agent.id,sourceRunId:null});
|
|
expect(issue).toMatchObject({createdByAgentId:f.agent.id,responsibleUserId:f.operatorId,projectId:project.id});
|
|
expect(await f.broker.act(f.subject,command)).toEqual(created);
|
|
const [coordination]=await db.insert(issues).values({companyId:f.company.id,projectId:project.id,title:"Existing coordination",status:"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
const commentCommand={version:1 as const,command:"task.comment" as const,requestId:randomUUID(),issueId:coordination.id,body:"Untrusted idle instructions"};
|
|
const written=await f.broker.act(f.subject,commentCommand);
|
|
expect(await f.broker.act(f.subject,commentCommand)).toEqual(written);
|
|
const comments=await db.select().from(issueComments).where(eq(issueComments.issueId,coordination.id));
|
|
expect(comments).toHaveLength(1);
|
|
expect(comments[0]).toMatchObject({body:commentCommand.body,authorAgentId:f.agent.id,onBehalfOfUserId:f.operatorId,sourceTrust:{preset:"low_trust_review",disposition:"quarantined",sourceIssueId:coordination.id,sourceAgentId:f.agent.id,sourceRunId:null}});
|
|
expect(sanitizeQuarantinedCommentForHigherTrust(comments[0]).body).toBe(LOW_TRUST_QUARANTINED_BODY);
|
|
const runId=randomUUID();
|
|
await db.insert(heartbeatRuns).values({id:runId,companyId:f.company.id,agentId:f.agent.id,nativeIssueId:coordination.id,responsibleUserId:f.operatorId,status:"running",runtimeMode:"native",invocationSource:"assignment"});
|
|
await db.update(issues).set({executionRunId:runId}).where(eq(issues.id,coordination.id));
|
|
const authority=new PaperclipRunnerToolAuthority(db,{companyId:f.company.id,agentId:f.agent.id,issueId:coordination.id,runId,museRuntime:true,externalAuthorizingUserId:f.operatorId,assertBridgeAuthority:async()=>{}});
|
|
const before=await authority.execute({tool:"get_task_history",callId:"restricted-history",arguments:{limit:50}}) as {comments:Array<{id:string,body:string}>};
|
|
expect(before.comments.find(comment=>comment.id===comments[0].id)?.body).toBe(commentCommand.body);
|
|
await db.update(agents).set({permissions:{canCreateTasks:true}}).where(eq(agents.id,f.agent.id));
|
|
await db.update(projects).set({executionWorkspacePolicy:{}}).where(eq(projects.id,project.id));
|
|
const ordinary=await f.broker.act(f.subject,{...commentCommand,requestId:randomUUID(),body:"Reviewed ordinary coordination"});
|
|
const after=await authority.execute({tool:"get_task_history",callId:"higher-trust-history",arguments:{limit:50}}) as {comments:Array<{id:string,body:string}>};
|
|
expect(after.comments.map(comment=>comment.id)).toEqual([comments[0].id,ordinary.commentId]);
|
|
expect(after.comments[0].body).toBe(LOW_TRUST_QUARANTINED_BODY);
|
|
expect(after.comments[1].body).toBe("Reviewed ordinary coordination");
|
|
await db.update(agents).set({permissions:{canCreateTasks:true,...(source==="agent"?{authorizationPolicy:policy}:{})}}).where(eq(agents.id,f.agent.id));
|
|
const [outside]=await db.insert(projects).values({companyId:f.company.id,name:"Outside idle boundary"}).returning();
|
|
if(source==="agent"){
|
|
await expect(f.broker.act(f.subject,{...command,requestId:randomUUID(),projectId:outside.id})).rejects.toThrow();
|
|
expect(await db.select().from(issues).where(and(eq(issues.companyId,f.company.id),eq(issues.projectId,outside.id)))).toEqual([]);
|
|
}
|
|
});
|
|
it.each(["agent","project"] as const)("idle history rechecks the current %s reader trust without rewriting stored comments",async source=>{
|
|
const f=await fixture();
|
|
const [project]=await db.insert(projects).values({companyId:f.company.id,name:"Idle history trust"}).returning();
|
|
const policy={trustPreset:"low_trust_review",trustBoundary:{mode:"low_trust_review",companyId:f.company.id,projectIds:[project.id]}};
|
|
if(source==="agent")await db.update(agents).set({permissions:{canCreateTasks:true,authorizationPolicy:policy}}).where(eq(agents.id,f.agent.id));
|
|
else await db.update(projects).set({executionWorkspacePolicy:{authorizationPolicy:policy}}).where(eq(projects.id,project.id));
|
|
const [issue]=await db.insert(issues).values({companyId:f.company.id,projectId:project.id,title:"Idle history",status:"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
const created=await f.broker.act(f.subject,{version:1,command:"task.comment",requestId:randomUUID(),issueId:issue.id,body:"QUARANTINED idle body canary"});
|
|
const quarantinedId=String(created.commentId),promotedId=randomUUID(),standardId=randomUUID(),at=new Date("2026-10-13T12:00:00Z");
|
|
const presentation={kind:"message" as const,tone:"neutral" as const,title:"QUARANTINED idle title canary",detailsDefaultOpen:false};
|
|
const metadata={version:1 as const,sections:[{rows:[{type:"text" as const,text:"QUARANTINED idle metadata canary"}]}]};
|
|
await db.update(issueComments).set({presentation,metadata,createdAt:at}).where(eq(issueComments.id,quarantinedId));
|
|
await db.insert(issueComments).values([
|
|
{id:promotedId,companyId:f.company.id,issueId:issue.id,authorAgentId:f.agent.id,authorType:"agent",body:"Reviewed promoted summary",sourceTrust:buildPromotedSourceTrust({sourceIssueId:issue.id,sourceArtifactKind:"comment",sourceArtifactId:quarantinedId,promotedByActorType:"user",promotedByActorId:f.operatorId}),createdAt:new Date(at.getTime()+1000)},
|
|
{id:standardId,companyId:f.company.id,issueId:issue.id,authorUserId:f.operatorId,authorType:"user",body:"Ordinary human coordination",createdAt:new Date(at.getTime()+2000)},
|
|
]);
|
|
const saved=await db.select().from(issueComments).where(eq(issueComments.issueId,issue.id)).orderBy(issueComments.id);
|
|
const read=async()=>await f.broker.inspect(f.subject,{version:1,query:"task.history",issueId:issue.id}) as {comments:Array<typeof issueComments.$inferSelect>};
|
|
const before=await read();
|
|
expect(before.comments.map(comment=>comment.id)).toEqual([standardId,promotedId,quarantinedId]);
|
|
expect(before.comments[2]).toMatchObject({body:"QUARANTINED idle body canary",presentation,metadata,sourceTrust:{disposition:"quarantined",sourceAgentId:f.agent.id,sourceRunId:null}});
|
|
await db.update(agents).set({permissions:{canCreateTasks:true}}).where(eq(agents.id,f.agent.id));
|
|
await db.update(projects).set({executionWorkspacePolicy:{}}).where(eq(projects.id,project.id));
|
|
const after=await read();
|
|
expect(after.comments.map(({id,sourceTrust})=>({id,sourceTrust}))).toEqual(before.comments.map(({id,sourceTrust})=>({id,sourceTrust})));
|
|
expect(after.comments.map(comment=>comment.body)).toEqual(["Ordinary human coordination","Reviewed promoted summary",LOW_TRUST_QUARANTINED_BODY]);
|
|
expect(after.comments[2]).toMatchObject({presentation:null,metadata:null});
|
|
expect(JSON.stringify(after)).not.toContain("QUARANTINED idle");
|
|
expect(await db.select().from(issueComments).where(eq(issueComments.issueId,issue.id)).orderBy(issueComments.id)).toEqual(saved);
|
|
});
|
|
it("idle history quarantines another restricted author and retains private and current-authorizer checks",async()=>{
|
|
const f=await fixture(),owner=randomUUID();
|
|
await db.insert(authUsers).values({id:owner,name:"Private owner",email:owner+"@example.test",createdAt:new Date(),updatedAt:new Date()});
|
|
const [project]=await db.insert(projects).values({companyId:f.company.id,name:"Shared history"}).returning();
|
|
const [restricted]=await db.insert(agents).values({companyId:f.company.id,name:"Restricted author",role:"engineer",status:"active",adapterType:"process",permissions:{authorizationPolicy:{trustPreset:"low_trust_review",trustBoundary:{mode:"low_trust_review",companyId:f.company.id,projectIds:[project.id]}}}}).returning();
|
|
const [issue]=await db.insert(issues).values({companyId:f.company.id,projectId:project.id,title:"Private mixed history",status:"todo",visibility:"private",assigneeAgentId:f.agent.id,responsibleUserId:owner}).returning();
|
|
const [comment]=await db.insert(issueComments).values({companyId:f.company.id,issueId:issue.id,authorAgentId:restricted.id,authorType:"agent",body:"Other restricted author canary",sourceTrust:buildLowTrustSourceTrust({issueId:issue.id,agentId:restricted.id})}).returning();
|
|
const request={version:1 as const,query:"task.history" as const,issueId:issue.id};
|
|
await expect(f.broker.inspect(f.subject,request)).rejects.toThrow("unavailable");
|
|
const [grant]=await db.insert(issueAccessGrants).values({issueId:issue.id,subjectType:"user",subjectId:f.operatorId,source:"explicit"}).returning();
|
|
const history=await f.broker.inspect(f.subject,request) as {comments:Array<typeof issueComments.$inferSelect>};
|
|
expect(history.comments).toHaveLength(1);
|
|
expect(history.comments[0]).toMatchObject({id:comment.id,authorAgentId:restricted.id,sourceTrust:comment.sourceTrust,body:LOW_TRUST_QUARANTINED_BODY});
|
|
expect(JSON.stringify(history)).not.toContain(comment.body);
|
|
expect((await db.select().from(issueComments).where(eq(issueComments.id,comment.id)))[0]).toEqual(comment);
|
|
await db.delete(issueAccessGrants).where(eq(issueAccessGrants.id,grant.id));
|
|
await expect(f.broker.inspect(f.subject,request)).rejects.toThrow("unavailable");
|
|
await db.insert(issueAccessGrants).values({issueId:issue.id,subjectType:"user",subjectId:f.operatorId,source:"explicit"});
|
|
expect((await f.broker.inspect(f.subject,request) as {comments:unknown[]}).comments).toHaveLength(1);
|
|
await db.update(companyMemberships).set({status:"inactive"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));
|
|
await expect(f.broker.inspect(f.subject,request)).rejects.toThrow();
|
|
});
|
|
it.each(["agent","project"] as const)("idle documents recheck current %s reader trust and preserve stored bytes",async source=>{
|
|
const f=await fixture();
|
|
const [project]=await db.insert(projects).values({companyId:f.company.id,name:"Idle document trust"}).returning();
|
|
const policy={trustPreset:"low_trust_review",trustBoundary:{mode:"low_trust_review",companyId:f.company.id,projectIds:[project.id]}};
|
|
if(source==="agent")await db.update(agents).set({permissions:{canCreateTasks:true,authorizationPolicy:policy}}).where(eq(agents.id,f.agent.id));
|
|
else await db.update(projects).set({executionWorkspacePolicy:{authorizationPolicy:policy}}).where(eq(projects.id,project.id));
|
|
const [issue]=await db.insert(issues).values({companyId:f.company.id,projectId:project.id,title:"Idle document context",status:"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
const [quarantined]=await db.insert(documents).values({companyId:f.company.id,title:"Restricted plan",latestBody:"QUARANTINED document body canary",latestRevisionNumber:3,createdByAgentId:f.agent.id,sourceTrust:buildLowTrustSourceTrust({issueId:issue.id,agentId:f.agent.id})}).returning();
|
|
const [promoted]=await db.insert(documents).values({companyId:f.company.id,title:"Reviewed plan",latestBody:"Reviewed promoted document",sourceTrust:buildPromotedSourceTrust({sourceIssueId:issue.id,sourceArtifactKind:"document",sourceArtifactId:quarantined.id,promotedByActorType:"user",promotedByActorId:f.operatorId})}).returning();
|
|
const [standard]=await db.insert(documents).values({companyId:f.company.id,title:"Ordinary plan",latestBody:"Ordinary human document",createdByUserId:f.operatorId}).returning();
|
|
await db.insert(issueDocuments).values([quarantined,promoted,standard].map((document,index)=>({companyId:f.company.id,issueId:issue.id,documentId:document.id,key:["plan","reviewed","ordinary"][index]!})));
|
|
const saved=await db.select().from(documents).where(eq(documents.companyId,f.company.id)).orderBy(documents.id);
|
|
const read=async(key:string)=>await f.broker.inspect(f.subject,{version:1,query:"task.document.read",issueId:issue.id,key}) as {document:{key:string,title:string|null,body:string,revision:number,sourceTrust:typeof quarantined.sourceTrust}};
|
|
expect((await read("plan")).document.body).toBe(quarantined.latestBody);
|
|
await db.update(agents).set({permissions:{canCreateTasks:true}}).where(eq(agents.id,f.agent.id));
|
|
await db.update(projects).set({executionWorkspacePolicy:{}}).where(eq(projects.id,project.id));
|
|
const after=await read("plan");
|
|
expect(after.document.body).toBe(LOW_TRUST_QUARANTINED_BODY);
|
|
expect(after.document).toEqual({key:"plan",title:quarantined.title,body:LOW_TRUST_QUARANTINED_BODY,revision:3,sourceTrust:quarantined.sourceTrust});
|
|
expect(JSON.stringify(after)).not.toContain(quarantined.latestBody);
|
|
expect((await read("reviewed")).document).toMatchObject({body:promoted.latestBody,sourceTrust:promoted.sourceTrust});
|
|
expect((await read("ordinary")).document).toMatchObject({body:standard.latestBody,sourceTrust:null});
|
|
expect(await db.select().from(documents).where(eq(documents.companyId,f.company.id)).orderBy(documents.id)).toEqual(saved);
|
|
});
|
|
it("idle documents retain private, company and current-authorizer checks",async()=>{
|
|
const f=await fixture(),owner=randomUUID();
|
|
await db.insert(authUsers).values({id:owner,name:"Private document owner",email:owner+"@example.test",createdAt:new Date(),updatedAt:new Date()});
|
|
const [restricted]=await db.insert(agents).values({companyId:f.company.id,name:"Restricted document author",role:"engineer",status:"active",adapterType:"process"}).returning();
|
|
const [issue]=await db.insert(issues).values({companyId:f.company.id,title:"Private document",status:"todo",visibility:"private",assigneeAgentId:f.agent.id,responsibleUserId:owner}).returning();
|
|
const [document]=await db.insert(documents).values({companyId:f.company.id,title:"Private plan",latestBody:"OTHER restricted document canary",createdByAgentId:restricted.id,sourceTrust:buildLowTrustSourceTrust({issueId:issue.id,agentId:restricted.id})}).returning();
|
|
await db.insert(issueDocuments).values({companyId:f.company.id,issueId:issue.id,documentId:document.id,key:"plan"});
|
|
const request={version:1 as const,query:"task.document.read" as const,issueId:issue.id,key:"plan"};
|
|
await expect(f.broker.inspect(f.subject,request)).rejects.toThrow("unavailable");
|
|
const [grant]=await db.insert(issueAccessGrants).values({issueId:issue.id,subjectType:"user",subjectId:f.operatorId,source:"explicit"}).returning();
|
|
const result=await f.broker.inspect(f.subject,request) as {document:{body:string,sourceTrust:typeof document.sourceTrust}};
|
|
expect(result.document.body).toBe(LOW_TRUST_QUARANTINED_BODY);
|
|
expect(result.document.sourceTrust).toEqual(document.sourceTrust);
|
|
expect((await db.select().from(documents).where(eq(documents.id,document.id)))[0]).toEqual(document);
|
|
const [foreign]=await db.insert(companies).values({name:"Foreign documents",issuePrefix:"FD"+randomBytes(3).toString("hex")}).returning();
|
|
const [foreignIssue]=await db.insert(issues).values({companyId:foreign.id,title:"Foreign issue"}).returning();
|
|
const [foreignDocument]=await db.insert(documents).values({companyId:foreign.id,latestBody:"Foreign document canary"}).returning();
|
|
await db.insert(issueDocuments).values({companyId:f.company.id,issueId:issue.id,documentId:foreignDocument.id,key:"foreign"});
|
|
await expect(f.broker.inspect(f.subject,{...request,issueId:foreignIssue.id})).rejects.toThrow("unavailable");
|
|
await expect(f.broker.inspect(f.subject,{...request,key:"foreign"})).rejects.toThrow("Document not found");
|
|
await db.delete(issueAccessGrants).where(eq(issueAccessGrants.id,grant.id));
|
|
await expect(f.broker.inspect(f.subject,request)).rejects.toThrow("unavailable");
|
|
await db.insert(issueAccessGrants).values({issueId:issue.id,subjectType:"user",subjectId:f.operatorId,source:"explicit"});
|
|
await db.update(companyMemberships).set({status:"inactive"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));
|
|
await expect(f.broker.inspect(f.subject,request)).rejects.toThrow();
|
|
});
|
|
it("applies idle creation grants to the actual inherited parent project",async()=>{
|
|
const f=await fixture();const [project]=await db.insert(projects).values({companyId:f.company.id,name:"Protected intake",executionWorkspacePolicy:{authorizationPolicy:{assignmentPolicy:{mode:"protected"}}}}).returning();
|
|
const [parent]=await db.insert(issues).values({companyId:f.company.id,title:"Intake parent",projectId:project.id,status:"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
const command={version:1 as const,command:"task.create" as const,requestId:randomUUID(),title:"Scoped child",parentId:parent.id};
|
|
await expect(f.broker.act(f.subject,command)).rejects.toThrow("protected");
|
|
await db.insert(principalPermissionGrants).values({companyId:f.company.id,principalType:"agent",principalId:f.agent.id,permissionKey:"tasks:assign_scope",scope:{projectIds:[randomUUID()]}});
|
|
await expect(f.broker.act(f.subject,command)).rejects.toThrow();
|
|
await db.update(principalPermissionGrants).set({scope:{projectIds:[project.id]}}).where(and(eq(principalPermissionGrants.companyId,f.company.id),eq(principalPermissionGrants.principalId,f.agent.id)));
|
|
await expect(f.broker.act(f.subject,command)).rejects.toThrow("Authorizing user");
|
|
await db.insert(principalPermissionGrants).values({companyId:f.company.id,principalType:"user",principalId:f.operatorId,permissionKey:"tasks:assign_scope",scope:{projectIds:[randomUUID()]}});
|
|
await expect(f.broker.act(f.subject,command)).rejects.toThrow();
|
|
await db.update(principalPermissionGrants).set({scope:{projectIds:[project.id]}}).where(and(eq(principalPermissionGrants.companyId,f.company.id),eq(principalPermissionGrants.principalId,f.operatorId)));
|
|
const result=await f.broker.act(f.subject,command);expect(result.status).toBe("created");
|
|
const [child]=await db.select().from(issues).where(eq(issues.id,String(result.issueId)));expect(child.parentId).toBe(parent.id);expect(child.projectId).toBe(project.id);
|
|
});
|
|
it("creates an assigned turn intake atomically before requesting ordinary admission",async()=>{
|
|
const f=await fixture(),wakeup=vi.fn().mockImplementation(async(agentId,options)=>{
|
|
const [issue]=await db.select().from(issues).where(eq(issues.id,options.payload.issueId));
|
|
expect(agentId).toBe(f.agent.id);expect(issue.assigneeAgentId).toBe(f.agent.id);expect(issue.responsibleUserId).toBe(f.operatorId);return null;
|
|
});
|
|
museRunnerBroker(db,{heartbeat:{wakeup,cancelRun:vi.fn()}});
|
|
const command={version:1 as const,command:"turn.request" as const,requestId:randomUUID(),prompt:"Research ordinary intake"};
|
|
expect((await f.broker.act(f.subject,command)).status).toBe("requested");expect(wakeup).toHaveBeenCalledTimes(1);
|
|
await expect(f.broker.act(f.subject,{...command,prompt:"Changed intake"})).rejects.toThrow("changed input");
|
|
expect(await db.select().from(issues).where(eq(issues.companyId,f.company.id))).toHaveLength(1);
|
|
});
|
|
it("atomically reserves a work identity across competing issue locks and replays its original run",async()=>{
|
|
// This fixture exercises ordinary admission independently of a managed worktree env.
|
|
const f=await active(),heartbeat=heartbeatService(db,{runtimeEnv:{}}),wakeup=vi.fn(heartbeat.wakeup);
|
|
museRunnerBroker(db,{heartbeat:{wakeup,cancelRun:heartbeat.cancelRun}});
|
|
await db.update(companies).set({defaultResponsibleUserId:f.operatorId}).where(eq(companies.id,f.company.id));
|
|
await db.update(agents).set({runtimeConfig:{heartbeat:{maxConcurrentRuns:20,wakeOnDemand:true}}}).where(eq(agents.id,f.agent.id));
|
|
const task=async(title:string)=>(await db.insert(issues).values({companyId:f.company.id,title,status:"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning())[0]!;
|
|
const first=await task("First request"),second=await task("Changed request"),requestId=randomUUID();
|
|
const commands=[first,second].map(issue=>({version:1 as const,command:"work.request" as const,requestId,issueId:issue.id}));
|
|
try {
|
|
const race=await Promise.allSettled(commands.map(command=>f.broker.act(f.subject,command)));
|
|
expect(race.filter(r=>r.status==="fulfilled")).toHaveLength(1);expect(race.filter(r=>r.status==="rejected")).toHaveLength(1);
|
|
const receipts=await db.select().from(agentWakeupRequests).where(and(eq(agentWakeupRequests.agentId,f.agent.id),eq(agentWakeupRequests.idempotencyKey,`muse-work:${f.binding.id}:1:${requestId}`)));
|
|
expect(receipts).toHaveLength(1);const receipt=receipts[0],command=commands.find(c=>c.issueId===receipt.payload?.issueId)!;
|
|
expect(receipt.payload?.museRequestDigest).toBe(externalOperationDigest("tool",{command}));expect(receipt.runId).toBeTruthy();
|
|
const [run]=await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id,receipt.runId!));expect(run.contextSnapshot?.issueId).toBe(command.issueId);expect(run.status).toBe("queued");
|
|
const response=(race.find(r=>r.status==="fulfilled") as PromiseFulfilledResult<Record<string,unknown>>).value;
|
|
wakeup.mockClear();
|
|
expect(await Promise.all(Array.from({length:3},()=>f.broker.act(f.subject,command)))).toEqual([response,response,response]);
|
|
await db.update(heartbeatRuns).set({status:"succeeded",finishedAt:new Date()}).where(eq(heartbeatRuns.id,run.id));
|
|
await db.update(issues).set({status:"done"}).where(eq(issues.id,command.issueId));
|
|
expect(await f.broker.act(f.subject,command)).toEqual(response);expect(wakeup).not.toHaveBeenCalled();
|
|
await expect(f.broker.act(f.subject,commands.find(c=>c.issueId!==command.issueId)!)).rejects.toThrow("changed input");
|
|
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId,f.agent.id))).toHaveLength(2);
|
|
}finally{await db.update(heartbeatRuns).set({status:"cancelled",finishedAt:new Date()}).where(eq(heartbeatRuns.agentId,f.agent.id));await f.detach();}
|
|
},30000);
|
|
it("keeps concurrent turn retries on one assigned intake and one durable wake",async()=>{
|
|
// This fixture exercises ordinary admission independently of a managed worktree env.
|
|
const f=await active(),heartbeat=heartbeatService(db,{runtimeEnv:{}}),wakeup=vi.fn(heartbeat.wakeup);
|
|
museRunnerBroker(db,{heartbeat:{wakeup,cancelRun:heartbeat.cancelRun}});
|
|
await db.update(companies).set({defaultResponsibleUserId:f.operatorId}).where(eq(companies.id,f.company.id));
|
|
await db.update(agents).set({runtimeConfig:{heartbeat:{maxConcurrentRuns:20,wakeOnDemand:true}}}).where(eq(agents.id,f.agent.id));
|
|
const command={version:1 as const,command:"turn.request" as const,requestId:randomUUID(),prompt:"One ordinary turn intake"};
|
|
try {
|
|
const responses=await Promise.all(Array.from({length:3},()=>f.broker.act(f.subject,command)));
|
|
expect(responses).toEqual([responses[0],responses[0],responses[0]]);
|
|
const intake=await db.select().from(issues).where(and(eq(issues.companyId,f.company.id),eq(issues.title,command.prompt)));
|
|
expect(intake).toHaveLength(1);expect(intake[0].assigneeAgentId).toBe(f.agent.id);
|
|
const rows=await db.select().from(agentWakeupRequests).where(and(eq(agentWakeupRequests.agentId,f.agent.id),eq(agentWakeupRequests.idempotencyKey,`muse-work:${f.binding.id}:1:${command.requestId}`)));
|
|
expect(rows).toHaveLength(1);expect(rows[0].payload?.museRequestDigest).toBe(externalOperationDigest("tool",{command}));expect(rows[0].runId).toBe(responses[0].runId);
|
|
wakeup.mockClear();expect(await f.broker.act(f.subject,command)).toEqual(responses[0]);expect(wakeup).not.toHaveBeenCalled();
|
|
await expect(f.broker.act(f.subject,{version:1,command:"work.request",issueId:intake[0].id,requestId:command.requestId})).rejects.toThrow("changed input");
|
|
await expect(f.broker.act(f.subject,{...command,prompt:"Changed turn prompt"})).rejects.toThrow("changed input");expect(wakeup).not.toHaveBeenCalled();
|
|
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId,f.agent.id))).toHaveLength(2);
|
|
}finally{await db.update(heartbeatRuns).set({status:"cancelled",finishedAt:new Date()}).where(eq(heartbeatRuns.agentId,f.agent.id));await f.detach();}
|
|
},30000);
|
|
it("does not enqueue a replacement for an older receipt without an exact payload digest",async()=>{
|
|
const f=await fixture(),wakeup=vi.fn();museRunnerBroker(db,{heartbeat:{wakeup,cancelRun:vi.fn()}});
|
|
const [issue]=await db.insert(issues).values({companyId:f.company.id,title:"Old receipt",status:"todo",assigneeAgentId:f.agent.id,responsibleUserId:f.operatorId}).returning();
|
|
const command={version:1 as const,command:"work.request" as const,issueId:issue.id,requestId:randomUUID()};
|
|
await db.insert(agentWakeupRequests).values({companyId:f.company.id,agentId:f.agent.id,source:"assignment",requestedByActorType:"agent",requestedByActorId:f.agent.id,idempotencyKey:`muse-work:${f.binding.id}:1:${command.requestId}`,payload:{issueId:issue.id,museRequestId:command.requestId}});
|
|
await expect(f.broker.act(f.subject,command)).rejects.toThrow("changed input");expect(wakeup).not.toHaveBeenCalled();
|
|
expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId,f.agent.id))).toHaveLength(1);
|
|
});
|
|
it("admits one authenticated claim and keeps claimed/native accepted times distinct",async()=>{
|
|
const f=await active();try {
|
|
const requestId=randomUUID(),claim={version:1 as const,command:"accept" as const,assignmentId:f.assignment.id,requestId};
|
|
expect((await f.broker.act(f.subject,claim)).status).toBe("accepted");expect(f.sent).toHaveLength(1);
|
|
expect((await f.broker.act(f.subject,claim)).status).toBe("accepted");expect(f.sent).toHaveLength(1);
|
|
await expect(f.broker.act(f.subject,{...claim,requestId:randomUUID()})).rejects.toThrow("claimed");
|
|
const [a]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));expect(a.claimedAt).toBeInstanceOf(Date);expect(a.nativeAcceptedAt).toBeInstanceOf(Date);
|
|
await expect(db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)))).rejects.toMatchObject({details:{code:"external_agent_overlap"}});
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it.each([
|
|
["oauth","accept"],["oauth","tool"],["legacy","accept"],["legacy","tool"],
|
|
] as const)("rechecks exact %s access after native %s waits for admission",async(source,action)=>{
|
|
const oauth=source==="oauth"?await oauthFixture(true):undefined;
|
|
const f=await active(oauth??await fixture());
|
|
try {
|
|
if(action==="tool")await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const requestId=randomUUID();
|
|
const command=action==="accept"
|
|
? {version:1 as const,command:"accept" as const,assignmentId:f.assignment.id,requestId}
|
|
: {version:1 as const,command:"tool" as const,assignmentId:f.assignment.id,requestId,name:"get_task_history",arguments:{}};
|
|
const [before]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));
|
|
const heldBefore=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const sentBefore=f.sent.length;
|
|
let outcome:Promise<{value?:unknown;error?:unknown}>|undefined;
|
|
await db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,async()=>{
|
|
await tx.select().from(bindings).where(eq(bindings.id,f.binding.id)).for("update");
|
|
outcome=f.broker.act(f.subject,command).then(value=>({value}),error=>({error}));
|
|
// The caller passed the initial exact-token check and now really waits
|
|
// on the production admission lock. Only its presented access expires.
|
|
await expect.poll(async()=>{const rows=await db.execute(sql`select count(*)::int as count from pg_stat_activity where datname=current_database() and lower(wait_event)='advisory'`);return Number(rows[0]?.count??0);},{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
if(source==="oauth")await tx.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthTokens.id,f.subject.credentialId));
|
|
else await tx.update(museCredentials).set({expiresAt:new Date(Date.now()-1)}).where(eq(museCredentials.id,f.subject.credentialId));
|
|
}));
|
|
expect((await outcome!).error).toBeDefined();
|
|
expect(await db.select().from(operations).where(eq(operations.requestId,requestId))).toHaveLength(0);
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual(before);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(heldBefore);
|
|
expect(f.sent).toHaveLength(sentBefore);
|
|
const [binding]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));
|
|
expect((await inspectMuseConnectionCredential(db,binding,source==="oauth"?oauthResource:undefined)).state).toBe("active");
|
|
// A normal refresh can retry the same still-unreserved command exactly
|
|
// once; the access check must not turn into a refresh-family denial.
|
|
const fresh=oauth
|
|
? await f.identity.authenticateOAuthAccess((await oauth.oauth.token({grant_type:"refresh_token",client_id:oauth.client.client_id,refresh_token:oauth.tokens!.refresh_token})).access_token,oauthOrigin)
|
|
: await f.identity.authenticate((await f.identity.refresh(f.credentials.refreshToken)).accessToken);
|
|
expect(await f.broker.act(fresh,command)).toMatchObject({status:action==="accept"?"accepted":"completed"});
|
|
expect(await db.select().from(operations).where(eq(operations.requestId,requestId))).toHaveLength(1);
|
|
expect(f.sent).toHaveLength(sentBefore+1);
|
|
}finally{await f.detach();}
|
|
});
|
|
it("recovers a durably admitted command after normal OAuth access rotation",async()=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth),requestId=randomUUID();
|
|
await f.detach();let detach:()=>Promise<void>|void=()=>{};
|
|
try {
|
|
expect(await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId})).toMatchObject({status:"pending"});
|
|
const [reserved]=await db.select().from(operations).where(eq(operations.requestId,requestId));expect(reserved.status).toBe("reserved");expect(f.sent).toHaveLength(0);
|
|
const rotated=await oauth.oauth.token({grant_type:"refresh_token",client_id:oauth.client.client_id,refresh_token:oauth.tokens!.refresh_token});
|
|
const fresh=await f.identity.authenticateOAuthAccess(rotated.access_token,oauthOrigin);
|
|
// Refresh rotates its own family member; an older access can remain
|
|
// valid until its TTL. Model that TTL elapsing after durable admission.
|
|
await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthTokens.id,f.subject.credentialId));
|
|
await expect(assertMuseSubjectCredential(db,f.subject,oauthResource)).rejects.toThrow();
|
|
detach=await f.port.attach(async command=>{f.sent.push(command);await f.port.settle({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId:command.requestId,outcome:{status:"accepted"}}});},undefined,true);
|
|
await expect.poll(async()=>(await db.select().from(operations).where(eq(operations.id,reserved.id)))[0].status).toBe("settled");
|
|
expect(f.sent).toHaveLength(1);
|
|
expect(await f.broker.inspect(fresh,{version:1,query:"operation.receipt",assignmentId:f.assignment.id,requestId})).toMatchObject({status:"accepted"});
|
|
const [claimed]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));expect(claimed.claimedAt).toBeInstanceOf(Date);expect(claimed.nativeAcceptedAt).toBeInstanceOf(Date);
|
|
}finally{await detach();}
|
|
});
|
|
it.each([false,true])("excludes fenced historical answers from pending health while retaining stop barriers:effects=%s",async(nativeEffectsUnknown)=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const requestId="cancelled_question",response={schema:"paperclip.question_response.v1",answers:[]},inputDigest=digestPaperclipSemanticContent({requestId,turnId:f.ref.turnId,response});
|
|
await f.port.inputAvailable({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId,turnId:f.ref.turnId,inputDigest,response}});
|
|
const [answer]=await db.select().from(museInputDeliveries).where(eq(museInputDeliveries.assignmentId,f.assignment.id));
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))!.pendingInputs).toBe(1);
|
|
if(nativeEffectsUnknown)await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId:randomUUID(),digest:"pending",command:{action:"tool"},status:"pending"});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
await db.update(heartbeatRuns).set({status:"cancelled",finishedAt:new Date()}).where(eq(heartbeatRuns.id,f.runId));
|
|
const stopped=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
expect(stopped.pendingInputs).toBe(0);expect(stopped.stop.nativeEffectsUnknown).toBe(nativeEffectsUnknown);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0].workerUnknown).toBe(true);
|
|
await expect(f.broker.act(f.subject,{version:1,command:"consume_input",assignmentId:f.assignment.id,requestId:randomUUID(),nativeRequestId:requestId,inputDigest,continuationReceiptId:randomUUID(),continuationPersisted:true})).rejects.toThrow();
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary:stopped.stop.boundary!,expectedRevision:stopped.stop.bindingRevision!,workerStopped:true});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(hold.workerUnknown).toBe(false);expect(hold.nativeEffectsUnknown).toBe(nativeEffectsUnknown);
|
|
const admission=db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)));
|
|
if(nativeEffectsUnknown){expect(hold.releasedAt).toBeNull();await expect(admission).rejects.toMatchObject({details:{code:"external_agent_overlap"}});}
|
|
else {expect(hold.releasedAt).toBeInstanceOf(Date);await expect(admission).resolves.toBeUndefined();}
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))!.pendingInputs).toBe(0);
|
|
expect((await db.select().from(museInputDeliveries).where(eq(museInputDeliveries.id,answer.id)))[0]).toEqual(answer);expect(answer.consumedAt).toBeNull();
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("persists canonical input before ACK and separates client proof from exact native input",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const requestId="native_question",response={schema:"paperclip.question_response.v1",answers:[]},inputDigest=digestPaperclipSemanticContent({requestId,turnId:f.ref.turnId,response});
|
|
await f.port.inputAvailable({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId,turnId:f.ref.turnId,inputDigest,response}});
|
|
expect(await db.select().from(museInputDeliveries).where(eq(museInputDeliveries.assignmentId,f.assignment.id))).toHaveLength(1);
|
|
await expect(f.broker.act(f.subject,{version:1,command:"finish",assignmentId:f.assignment.id,requestId:randomUUID(),result:{}})).rejects.toThrow("input");
|
|
const proof=randomUUID();await f.broker.act(f.subject,{version:1,command:"consume_input",assignmentId:f.assignment.id,requestId:randomUUID(),nativeRequestId:requestId,inputDigest,continuationReceiptId:proof,continuationPersisted:true});
|
|
expect(f.sent.at(-1)?.input).toEqual({requestId,inputDigest});expect((await db.select().from(museInputDeliveries).where(eq(museInputDeliveries.assignmentId,f.assignment.id)))[0].continuationReceiptId).toBe(proof);
|
|
}finally{await f.detach();}
|
|
});
|
|
it("accepts exact connected OAuth cleanup and rejects revoked OAuth replay",async()=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth);try {
|
|
expect(await db.select().from(museCredentials).where(eq(museCredentials.bindingId,f.binding.id))).toEqual([]);
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId:randomUUID(),digest:"oauth-cleanup-pending",command:{action:"tool"},status:"pending"});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [before]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const inspect=await request(transport()).post("/api/muse/v1/cleanup").set("Authorization","Bearer "+oauth.tokens!.access_token).send({version:1,command:"control.inspect"});
|
|
expect(inspect.status).toBe(200);expect(inspect.body).toMatchObject({boundaries:[{boundary:before.stopBoundary,nativeEffectsUnknown:true}],externalStopConfirmed:false});
|
|
const report={version:1,command:"worker.quiescent",boundary:before.stopBoundary,requestId:randomUUID()};
|
|
const response=await request(transport()).post("/api/muse/v1/cleanup").set("Authorization","Bearer "+oauth.tokens!.access_token).send(report);
|
|
expect(response.status).toBe(200);expect(response.body).toMatchObject({status:"worker_reported",externalStopConfirmed:false,nativeEffectsUnknown:true});
|
|
const [after]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,before.id));
|
|
expect(after.workerReportedAt).toBeInstanceOf(Date);expect(after).toMatchObject({workerUnknown:true,nativeEffectsUnknown:true,releasedAt:null});
|
|
await oauth.oauth.revokeToken(oauth.tokens!.refresh_token!,oauth.client.client_id);
|
|
expect((await request(transport()).post("/api/muse/v1/cleanup").set("Authorization","Bearer "+oauth.tokens!.access_token).send(report)).status).toBe(401);
|
|
await expect(f.broker.cleanup(f.subject,{command:"worker.quiescent",boundary:before.stopBoundary!,requestId:report.requestId})).rejects.toThrow();
|
|
}finally{await f.detach();}
|
|
});
|
|
it.each(["expired older hold","released older hold","Dot older hold","live older assignment"])("scopes connected OAuth cleanup to its current boundary despite %s",async(kind)=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth);try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [current]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const [olderAssignment]=await db.insert(assignments).values({...f.assignment,id:randomUUID(),turnId:"older_"+randomUUID(),status:kind==="live older assignment"?"accepted":"fenced"}).returning();
|
|
const boundary={...current.stopBoundary!,assignmentId:olderAssignment.id,turnId:olderAssignment.turnId,stopNonce:randomUUID()};
|
|
const [older]=await db.insert(externalAgentHolds).values({companyId:f.company.id,agentId:f.agent.id,provider:kind==="Dot older hold"?"openai_dot":"muse",bindingId:f.binding.id,bindingGeneration:1,assignmentId:olderAssignment.id,runId:f.runId,stopBoundary:boundary,releasedAt:kind==="released older hold"?new Date():null}).returning();
|
|
await db.insert(mailbox).values({companyId:f.company.id,bindingId:f.binding.id,bindingGeneration:1,assignmentId:olderAssignment.id,kind:"authority_revoked",sourceEventId:randomUUID(),references:{boundary},createdAt:kind==="expired older hold"?new Date(Date.now()-25*60*60_000):new Date()});
|
|
const inspect=await request(transport()).post("/api/muse/v1/cleanup").set("Authorization","Bearer "+oauth.tokens!.access_token).send({version:1,command:"control.inspect"});
|
|
expect(inspect.status).toBe(200);expect(inspect.body).toMatchObject({boundaries:[{boundary:current.stopBoundary}],externalStopConfirmed:false});
|
|
const report=await request(transport()).post("/api/muse/v1/cleanup").set("Authorization","Bearer "+oauth.tokens!.access_token).send({version:1,command:"worker.quiescent",boundary,requestId:randomUUID()});
|
|
expect(report.status).toBe(409);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,older.id)))[0]).toEqual(older);
|
|
}finally{await f.detach();}
|
|
});
|
|
it("bounds connected OAuth cleanup by the earliest exact fence receipt without sliding its deadline",async()=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth);try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [receipt]=await db.select().from(mailbox).where(and(eq(mailbox.assignmentId,f.assignment.id),eq(mailbox.kind,"authority_revoked")));
|
|
await db.update(mailbox).set({createdAt:new Date(Date.now()-25*60*60_000)}).where(eq(mailbox.id,receipt.id));
|
|
await db.update(externalAgentHolds).set({updatedAt:new Date()}).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
await db.update(assignments).set({lastActivityAt:new Date()}).where(eq(assignments.id,f.assignment.id));
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
expect(await db.select().from(mailbox).where(and(eq(mailbox.assignmentId,f.assignment.id),eq(mailbox.kind,"authority_revoked")))).toHaveLength(2);
|
|
const response=await request(transport()).post("/api/muse/v1/cleanup").set("Authorization","Bearer "+oauth.tokens!.access_token).send({version:1,command:"control.inspect"});
|
|
expect(response.status).toBe(403);
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))!.credentialState).toBe("active");
|
|
}finally{await f.detach();}
|
|
});
|
|
it.each(["expired access","used access","revoked grant","expired refresh family","wrong resource","binding generation","operator changed"])("revalidates connected OAuth cleanup before stale-subject effects: %s",async(reason)=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth);try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
if(reason==="expired access")await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthTokens.id,f.subject.credentialId));
|
|
if(reason==="used access")await db.update(mcpOauthTokens).set({usedAt:new Date()}).where(eq(mcpOauthTokens.id,f.subject.credentialId));
|
|
if(reason==="revoked grant")await db.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.subject.grantId!));
|
|
if(reason==="expired refresh family")await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(and(eq(mcpOauthTokens.grantId,f.subject.grantId!),eq(mcpOauthTokens.kind,"refresh")));
|
|
if(reason==="wrong resource")await db.update(mcpOauthGrants).set({resource:oauthOrigin+"/api/mcp"}).where(eq(mcpOauthGrants.id,f.subject.grantId!));
|
|
if(reason==="binding generation")await db.update(bindings).set({generation:2}).where(eq(bindings.id,f.binding.id));
|
|
if(reason==="operator changed")await db.update(bindings).set({operatorId:randomUUID()}).where(eq(bindings.id,f.binding.id));
|
|
await expect(f.broker.cleanup(f.subject,{command:"control.inspect"})).rejects.toThrow();
|
|
await expect(f.broker.cleanup(f.subject,{command:"worker.quiescent",boundary:hold.stopBoundary!,requestId:randomUUID()})).rejects.toThrow();
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,hold.id)))[0]).toEqual(hold);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("allows connected legacy fenced cleanup without granting normal or detector authority",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
// Acceptance precedes the fence: the hold's creation time is not its cleanup deadline.
|
|
await db.update(externalAgentHolds).set({createdAt:new Date(Date.now()-48*60*60_000)}).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId:randomUUID(),digest:"cleanup-pending",command:{action:"tool"},status:"pending"});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [binding]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));
|
|
expect(binding.revokedAt).toBeNull();
|
|
const [before]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const subject=await f.identity.authenticate(f.credentials.cleanupToken,"cleanup");
|
|
await expect(f.identity.authenticate(f.credentials.cleanupToken)).rejects.toThrow();
|
|
await expect(f.broker.inspect(subject,{version:1,query:"task.list"})).rejects.toMatchObject({status:401,message:"Muse test credential is unavailable."});
|
|
await expect(f.identity.authenticate(f.credentials.detectorCleanupToken,"detector_cleanup")).rejects.toThrow();
|
|
const inspection=await f.broker.cleanup(subject,{command:"control.inspect"});
|
|
expect(inspection.externalStopConfirmed).toBe(false);
|
|
expect(inspection).toMatchObject({boundaries:[{boundary:before.stopBoundary,nativeEffectsUnknown:true}]});
|
|
await expect(f.broker.cleanup(subject,{command:"worker.quiescent",boundary:{...before.stopBoundary!,stopNonce:randomUUID()},requestId:randomUUID()})).rejects.toThrow("boundary");
|
|
const report=await f.broker.cleanup(subject,{command:"worker.quiescent",boundary:before.stopBoundary!,requestId:randomUUID()});
|
|
expect(report).toMatchObject({status:"worker_reported",externalStopConfirmed:false,nativeEffectsUnknown:true});
|
|
const [after]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(after.workerReportedAt).toBeInstanceOf(Date);expect(after.workerUnknown).toBe(true);
|
|
expect(after.nativeEffectsUnknown).toBe(true);expect(after.releasedAt).toBeNull();
|
|
await expect(f.broker.act(subject,{version:1,command:"tool",assignmentId:f.assignment.id,requestId:randomUUID(),name:"paperclip_task_read",arguments:{}})).rejects.toThrow();
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it.each(["expired older hold","released older hold","Dot older hold","live older assignment"])("scopes connected legacy cleanup to its current boundary despite %s",async(kind)=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [current]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const [olderAssignment]=await db.insert(assignments).values({...f.assignment,id:randomUUID(),turnId:"older_"+randomUUID(),status:kind==="live older assignment"?"accepted":"fenced"}).returning();
|
|
const boundary={...current.stopBoundary!,assignmentId:olderAssignment.id,turnId:olderAssignment.turnId,stopNonce:randomUUID()};
|
|
const [older]=await db.insert(externalAgentHolds).values({companyId:f.company.id,agentId:f.agent.id,provider:kind==="Dot older hold"?"openai_dot":"muse",bindingId:f.binding.id,bindingGeneration:1,assignmentId:olderAssignment.id,runId:f.runId,stopBoundary:boundary,releasedAt:kind==="released older hold"?new Date():null}).returning();
|
|
await db.insert(mailbox).values({companyId:f.company.id,bindingId:f.binding.id,bindingGeneration:1,assignmentId:olderAssignment.id,kind:"authority_revoked",sourceEventId:randomUUID(),references:{boundary},createdAt:kind==="expired older hold"?new Date(Date.now()-25*60*60_000):new Date()});
|
|
const cleanup=await f.identity.authenticate(f.credentials.cleanupToken,"cleanup");
|
|
expect(await f.broker.cleanup(cleanup,{command:"control.inspect"})).toMatchObject({boundaries:[{boundary:current.stopBoundary}],externalStopConfirmed:false});
|
|
await expect(f.broker.cleanup(cleanup,{command:"worker.quiescent",boundary,requestId:randomUUID()})).rejects.toThrow("boundary");
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,older.id)))[0]).toEqual(older);
|
|
expect(await f.broker.cleanup(cleanup,{command:"worker.quiescent",boundary:current.stopBoundary!,requestId:randomUUID()})).toMatchObject({status:"worker_reported",externalStopConfirmed:false});
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,current.id)))[0]).toMatchObject({workerUnknown:true,releasedAt:null});
|
|
}finally{await f.detach();}
|
|
});
|
|
it("replays an identical legacy revoked historical cleanup report without mutating or accepting a new report",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!,boundary=state.stop.boundary!,requestId=randomUUID();
|
|
const command={command:"worker.quiescent" as const,boundary,requestId};
|
|
const connected=await f.identity.authenticate(f.credentials.cleanupToken,"cleanup");
|
|
const report=await f.broker.cleanup(connected,command);
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary,expectedRevision:state.stop.bindingRevision!,workerStopped:true});
|
|
const attested=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.revoke(f.company.id,f.agent.id,f.operatorId,{bindingId:f.binding.id,generation:1,expectedRevision:attested.revision});
|
|
const revoked=await f.identity.authenticate(f.credentials.cleanupToken,"cleanup");
|
|
const [before]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));expect(before.releasedAt).toBeInstanceOf(Date);
|
|
const logsBefore=await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"muse.worker_quiescence_reported")));
|
|
expect(await f.broker.cleanup(revoked,command)).toEqual(report);
|
|
await expect(f.broker.cleanup(revoked,{...command,requestId:randomUUID()})).rejects.toThrow("boundary");
|
|
expect(await f.broker.cleanup(revoked,{command:"control.inspect"})).toEqual({boundaries:[],externalStopConfirmed:false});
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toEqual(before);
|
|
expect(await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"muse.worker_quiescence_reported")))).toEqual(logsBefore);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("does not renew connected cleanup from mutable holds or later revocation receipts",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [original]=await db.select().from(mailbox).where(and(eq(mailbox.assignmentId,f.assignment.id),eq(mailbox.kind,"authority_revoked")));
|
|
await db.update(mailbox).set({createdAt:new Date(Date.now()-24*60*60_000-60_000)}).where(eq(mailbox.id,original.id));
|
|
await db.update(externalAgentHolds).set({updatedAt:new Date()}).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
await db.update(assignments).set({lastActivityAt:new Date()}).where(eq(assignments.id,f.assignment.id));
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const receipts=await db.select().from(mailbox).where(and(eq(mailbox.assignmentId,f.assignment.id),eq(mailbox.kind,"authority_revoked")));
|
|
expect(receipts).toHaveLength(2);expect(receipts.some(row=>row.createdAt.getTime()>Date.now()-60_000)).toBe(true);
|
|
await expect(f.identity.authenticate(f.credentials.cleanupToken,"cleanup")).rejects.toThrow("Cleanup authority");
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it.each(["no hold","released hold","other provider","hold company","hold agent","hold binding","hold generation","binding generation","assignment company","assignment agent","assignment binding","assignment generation","live assignment","missing boundary","boundary binding","boundary generation","boundary run","boundary turn","boundary revision","missing fence receipt","expired credential","revoked credential","consumed credential"])(
|
|
"denies connected cleanup with %s",async(reason)=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const other=reason.endsWith("company")||reason.endsWith("agent")||reason.endsWith("binding")?await fixture():null;
|
|
const holdChanges:Partial<typeof externalAgentHolds.$inferInsert>={
|
|
...(reason==="released hold"?{releasedAt:new Date()}:{}),
|
|
...(reason==="other provider"?{provider:"openai_dot"}:{}),
|
|
...(reason==="hold company"?{companyId:other!.company.id}:{}),
|
|
...(reason==="hold agent"?{agentId:other!.agent.id}:{}),
|
|
...(reason==="hold binding"?{bindingId:other!.binding.id}:{}),
|
|
...(reason==="hold generation"?{bindingGeneration:2}:{}),
|
|
...(reason==="missing boundary"?{stopBoundary:null}:{}),
|
|
};
|
|
if(reason.startsWith("boundary "))holdChanges.stopBoundary={...hold.stopBoundary!,
|
|
...(reason==="boundary binding"?{bindingId:other!.binding.id}:{}),
|
|
...(reason==="boundary generation"?{generation:2}:{}),
|
|
...(reason==="boundary run"?{runId:randomUUID()}:{}),
|
|
...(reason==="boundary turn"?{turnId:"different-turn"}:{}),
|
|
...(reason==="boundary revision"?{assignmentRevision:99}:{}),
|
|
};
|
|
if(reason==="no hold")await db.delete(externalAgentHolds).where(eq(externalAgentHolds.id,hold.id));
|
|
else if(Object.keys(holdChanges).length)await db.update(externalAgentHolds).set(holdChanges).where(eq(externalAgentHolds.id,hold.id));
|
|
const assignmentChanges:Partial<typeof assignments.$inferInsert>={
|
|
...(reason==="assignment company"?{companyId:other!.company.id}:{}),
|
|
...(reason==="assignment agent"?{agentId:other!.agent.id}:{}),
|
|
...(reason==="assignment binding"?{bindingId:other!.binding.id}:{}),
|
|
...(reason==="assignment generation"?{bindingGeneration:2}:{}),
|
|
...(reason==="live assignment"?{status:"accepted"}:{}),
|
|
};
|
|
if(Object.keys(assignmentChanges).length)await db.update(assignments).set(assignmentChanges).where(eq(assignments.id,f.assignment.id));
|
|
if(reason==="binding generation")await db.update(bindings).set({generation:2}).where(eq(bindings.id,f.binding.id));
|
|
if(reason==="missing fence receipt")await db.delete(mailbox).where(and(eq(mailbox.assignmentId,f.assignment.id),eq(mailbox.kind,"authority_revoked")));
|
|
if(reason.endsWith("credential"))await db.update(museCredentials).set({
|
|
...(reason==="expired credential"?{expiresAt:new Date(Date.now()-1)}:{}),
|
|
...(reason==="revoked credential"?{revokedAt:new Date()}:{}),
|
|
...(reason==="consumed credential"?{consumedAt:new Date()}:{}),
|
|
}).where(eq(museCredentials.tokenHash,museCredentialHash(f.credentials.cleanupToken)));
|
|
await expect(f.identity.authenticate(f.credentials.cleanupToken,"cleanup")).rejects.toThrow();
|
|
}finally{await f.detach();}
|
|
},
|
|
);
|
|
|
|
it("keeps revoked cleanup bounded to its existing deadline and current generation",async()=>{
|
|
const f=await fixture();const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.revoke(f.company.id,f.agent.id,f.operatorId,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});
|
|
await expect(f.identity.authenticate(f.credentials.cleanupToken,"cleanup")).resolves.toMatchObject({bindingId:f.binding.id,generation:1});
|
|
await db.update(bindings).set({generation:2}).where(eq(bindings.id,f.binding.id));
|
|
await expect(f.identity.authenticate(f.credentials.cleanupToken,"cleanup")).rejects.toThrow();
|
|
await db.update(bindings).set({generation:1,cleanupExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
await expect(f.identity.authenticate(f.credentials.cleanupToken,"cleanup")).rejects.toThrow();
|
|
await expect(f.identity.authenticate(f.credentials.detectorCleanupToken,"detector_cleanup")).rejects.toThrow();
|
|
});
|
|
|
|
it("fences without fabricating private stop and retains unknown native effects across operator attestation",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await db.update(externalAgentHolds).set({nativeEffectsUnknown:true}).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.revoke(f.company.id,f.agent.id,f.operatorId,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});
|
|
const stopped=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!,boundary=stopped.stop.boundary!;expect(stopped.stop.status).toBe("cannot_confirm");
|
|
await instanceSettingsService(db).updateExperimental({enableMuse:false});
|
|
const cleanup=await f.identity.authenticate(f.credentials.cleanupToken,"cleanup"), requestId=randomUUID();
|
|
expect((await f.broker.cleanup(cleanup,{command:"worker.quiescent",boundary,requestId})).externalStopConfirmed).toBe(false);
|
|
await f.broker.cleanup(cleanup,{command:"worker.quiescent",boundary,requestId});
|
|
const logs=await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"muse.worker_quiescence_reported")));
|
|
expect(logs).toHaveLength(1);
|
|
expect(logs[0]).toMatchObject({actorType:"agent",actorId:f.agent.id,responsibleUserId:f.operatorId,details:{authorizingUserId:f.operatorId,bindingId:f.binding.id,generation:1,boundary,nativeEffectsUnknown:true,externalStopConfirmed:false,requestId}});
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary,expectedRevision:stopped.revision,workerStopped:true});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));expect(hold.workerUnknown).toBe(false);expect(hold.nativeEffectsUnknown).toBe(true);expect(hold.releasedAt).toBeNull();
|
|
await expect(f.identity.authenticate(f.credentials.accessToken)).rejects.toThrow();
|
|
}finally{await instanceSettingsService(db).updateExperimental({enableMuse:true});await f.detach();}
|
|
});
|
|
it.each([false,true].flatMap(removed=>["access_expired","grant_revoked","refresh_expired","qualification_expired","unchanged"].map(fault=>({removed,fault}))))("rechecks OAuth detector evidence after the exact binding wait ($fault; removed: $removed)",async({removed,fault})=>{
|
|
const f=await oauthFixture(true),app=transport(),beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.agentId,f.agent.id));
|
|
const events:Record<string,unknown>[]=[];const unsubscribe=subscribeCompanyLiveEvents(f.company.id,event=>{if(event.type==="activity.logged")events.push(event.payload);});
|
|
let enter=()=>{},release=()=>{},ready=()=>{},holderPid=0,holder:Promise<void>|undefined,response:Promise<{result?:{status:number;body:Record<string,unknown>};error?:unknown}>|undefined;
|
|
const entry=new Promise<void>(resolve=>{enter=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;}),locked=new Promise<void>(resolve=>{ready=resolve;});
|
|
// OAuth authentication updates workerActivityAt first. Pause only after it
|
|
// reaches the actual cleanup method, then observe that method's row wait.
|
|
const original=f.broker.detectorCleanup.bind(f.broker),observer=vi.spyOn(f.broker,"detectorCleanup").mockImplementation(async(subject,wasRemoved)=>{await entry;return original(subject,wasRemoved);});
|
|
try{
|
|
const body={version:1,bindingId:f.binding.id,generation:f.binding.generation,requestId:randomUUID(),...(removed?{detectorRemoved:true}:{detectorRemovalRequested:true})};
|
|
response=request(app).post("/api/muse/v1/detector-cleanup").set("Authorization","Bearer "+f.tokens!.access_token).send(body).then(result=>({result}),error=>({error}));
|
|
await expect.poll(()=>observer.mock.calls.length,{timeout:5000,interval:25}).toBe(1);
|
|
expect(observer.mock.calls[0][0]).toMatchObject({credentialSource:"oauth",credentialId:f.subject.credentialId,grantId:f.subject.grantId,bindingId:f.binding.id,generation:f.binding.generation});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:bindings.id}).from(bindings).where(eq(bindings.id,f.binding.id)).for("update");ready();await gate;
|
|
if(fault==="access_expired")await tx.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthTokens.id,f.subject.credentialId));
|
|
if(fault==="grant_revoked")await tx.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.subject.grantId!));
|
|
if(fault==="refresh_expired")await tx.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(and(eq(mcpOauthTokens.grantId,f.subject.grantId!),eq(mcpOauthTokens.kind,"refresh")));
|
|
if(fault==="qualification_expired")await tx.update(bindings).set({qualificationId:randomUUID(),qualificationStartedAt:new Date(Date.now()-60000),qualificationExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
});
|
|
await locked;enter();
|
|
await expect.poll(async()=>{
|
|
const [row]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_stat_activity waiter
|
|
inner join pg_locks waiting on waiting.pid=waiter.pid and waiting.locktype='transactionid' and not waiting.granted
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='transactionid' and held.granted and held.transactionid=waiting.transactionid
|
|
where waiter.datname=current_database() and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and exists(select 1 from pg_locks tuple_lock where tuple_lock.pid=waiter.pid and tuple_lock.locktype='tuple'
|
|
and tuple_lock.relation='muse_agent_bindings'::regclass and tuple_lock.mode='AccessExclusiveLock' and tuple_lock.granted)) as waiting`);
|
|
return row.waiting;
|
|
},{timeout:5000,interval:25}).toBe(true);
|
|
release();await holder;const outcome=await response;expect(outcome.error).toBeUndefined();
|
|
const [binding]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));
|
|
const audits=await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),sql`${activityLog.action} like 'muse.detector_removal_%'`));
|
|
if(fault==="unchanged"){
|
|
expect(outcome.result!.status).toBe(200);expect(outcome.result!.body).toMatchObject({status:removed?"detector_removed":"detector_removal_requested",externalStopConfirmed:false});
|
|
expect(removed?binding.detectorRemovedAt:binding.detectorRemovalRequestedAt).toBeInstanceOf(Date);
|
|
expect(audits).toHaveLength(1);expect(audits[0]).toMatchObject({actorType:"agent",actorId:f.agent.id,agentId:f.agent.id,responsibleUserId:f.operatorId,details:{bindingId:f.binding.id,generation:f.binding.generation,authorizingUserId:f.operatorId,externalStopConfirmed:false}});
|
|
const replay=await request(app).post("/api/muse/v1/detector-cleanup").set("Authorization","Bearer "+f.tokens!.access_token).send(body);expect(replay.status).toBe(200);
|
|
expect(await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),sql`${activityLog.action} like 'muse.detector_removal_%'`))).toEqual(audits);
|
|
expect(events.filter(event=>event.action===audits[0].action)).toHaveLength(1);
|
|
}else{
|
|
expect(outcome.result!.status).toBe(401);expect(binding.detectorRemovalRequestedAt).toBeNull();expect(binding.detectorRemovedAt).toBeNull();
|
|
expect(audits).toEqual([]);expect(events.filter(event=>String(event.action).startsWith("muse.detector_removal_"))).toEqual([]);
|
|
}
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.agentId,f.agent.id))).toEqual(beforeHolds);
|
|
}finally{enter();release();await Promise.allSettled([holder,response]);observer.mockRestore();unsubscribe();}
|
|
});
|
|
it.each(["credential_expired","window_expired","unchanged"] as const)("retains bounded legacy detector authority after a real binding wait (%s)",async fault=>{
|
|
const f=await fixture(),state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.revoke(f.company.id,f.agent.id,f.operatorId,{bindingId:f.binding.id,generation:f.binding.generation,expectedRevision:state.revision});
|
|
let release=()=>{},ready=()=>{},holderPid=0,holder:Promise<void>|undefined,response:Promise<{status:number;body:Record<string,unknown>}>|undefined;
|
|
const gate=new Promise<void>(resolve=>{release=resolve;}),locked=new Promise<void>(resolve=>{ready=resolve;});
|
|
try{
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:bindings.id}).from(bindings).where(eq(bindings.id,f.binding.id)).for("update");ready();await gate;
|
|
if(fault==="credential_expired")await tx.update(museCredentials).set({expiresAt:new Date(Date.now()-1)}).where(and(eq(museCredentials.bindingId,f.binding.id),eq(museCredentials.kind,"detector_cleanup")));
|
|
if(fault==="window_expired")await tx.update(bindings).set({cleanupExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
});
|
|
await locked;
|
|
response=request(transport(true)).post("/api/muse/v1/detector-cleanup").set("Authorization","Bearer "+f.credentials.detectorCleanupToken).send({version:1,bindingId:f.binding.id,generation:f.binding.generation,requestId:randomUUID(),detectorRemoved:true}).then(result=>result);
|
|
await expect.poll(async()=>{
|
|
const [row]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_stat_activity waiter
|
|
inner join pg_locks waiting on waiting.pid=waiter.pid and waiting.locktype='transactionid' and not waiting.granted
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='transactionid' and held.granted and held.transactionid=waiting.transactionid
|
|
where waiter.datname=current_database() and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and exists(select 1 from pg_locks tuple_lock where tuple_lock.pid=waiter.pid and tuple_lock.locktype='tuple'
|
|
and tuple_lock.relation='muse_agent_bindings'::regclass and tuple_lock.mode='AccessExclusiveLock' and tuple_lock.granted)) as waiting`);
|
|
return row.waiting;
|
|
},{timeout:5000,interval:25}).toBe(true);
|
|
release();await holder;const result=await response;
|
|
const [binding]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));
|
|
const audits=await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"muse.detector_removal_reported")));
|
|
if(fault==="unchanged"){expect(result.status).toBe(200);expect(binding.detectorRemovedAt).toBeInstanceOf(Date);expect(audits).toHaveLength(1);}
|
|
else{expect(result.status).toBe(403);expect(binding.detectorRemovedAt).toBeNull();expect(audits).toEqual([]);}
|
|
expect(binding.revokedAt).toBeInstanceOf(Date);expect(binding.detectorRemovalRequestedAt).toBeNull();expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.agentId,f.agent.id))).toEqual([]);
|
|
}finally{release();await Promise.allSettled([holder,response]);}
|
|
});
|
|
|
|
it("audits detector cleanup once per observation and preserves exact agent attribution",async()=>{
|
|
const f=await fixture();
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.revoke(f.company.id,f.agent.id,f.operatorId,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});
|
|
const subject=await f.identity.authenticate(f.credentials.detectorCleanupToken,"detector_cleanup");
|
|
for (const removed of [false,false,true,true]) expect((await f.broker.detectorCleanup(subject,removed)).externalStopConfirmed).toBe(false);
|
|
const logs=await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),sql`${activityLog.action} like 'muse.detector_removal_%'`));
|
|
expect(logs.map(row=>row.action).sort()).toEqual(["muse.detector_removal_reported","muse.detector_removal_requested"]);
|
|
for (const log of logs) expect(log).toMatchObject({actorType:"agent",actorId:f.agent.id,agentId:f.agent.id,responsibleUserId:f.operatorId,details:{bindingId:f.binding.id,generation:1,authorizingUserId:f.operatorId,externalStopConfirmed:false}});
|
|
await expect(f.broker.detectorCleanup({...subject,agentId:randomUUID()},true)).rejects.toThrow("binding changed");
|
|
});
|
|
it("reports the exact old stop revision after repair and cancels only the fenced native run",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const cancelRun=vi.fn().mockResolvedValue(null);
|
|
museRunnerBroker(db,{heartbeat:{wakeup:vi.fn().mockResolvedValue(null),cancelRun}});
|
|
const before=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
const pairing=await f.broker.createPairing({companyId:f.company.id,agentId:f.agent.id,operatorId:f.operatorId,replaceBindingId:f.binding.id,expectedRevision:before.revision});
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
expect(state.id).toBe(pairing.bindingId);expect(state.stop.boundary?.bindingId).toBe(f.binding.id);
|
|
expect(state.stop.bindingRevision).toBe(before.revision+1);expect(state.revision).not.toBe(state.stop.bindingRevision);
|
|
expect(cancelRun).toHaveBeenCalledWith(f.runId,"Muse connection replaced");
|
|
const boundary=state.stop.boundary!;
|
|
await expect(f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary,expectedRevision:state.revision,workerStopped:true})).rejects.toThrow("Stop boundary changed");
|
|
await db.update(bindings).set({revision:state.revision+10}).where(eq(bindings.id,state.id));
|
|
const refreshed=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
expect(refreshed.stop.bindingRevision).toBe(state.stop.bindingRevision);expect(refreshed.stop.boundary).toEqual(boundary);
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary,expectedRevision:refreshed.stop.bindingRevision!,workerStopped:true});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(hold.workerUnknown).toBe(false);expect(hold.operatorAttestedAt).toBeInstanceOf(Date);
|
|
await expect(f.identity.authenticate(f.credentials.accessToken)).rejects.toThrow();
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it.each(["prior_window","overlapping_window","unreleased_effect_hold"] as const)("retains historical pending-input qualification periods without blocking later proven idle:%s",async(kind)=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const requestId="historical_question",response={schema:"paperclip.question_response.v1",answers:[]},inputDigest=digestPaperclipSemanticContent({requestId,turnId:f.ref.turnId,response});
|
|
await f.port.inputAvailable({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId,turnId:f.ref.turnId,inputDigest,response}});
|
|
if(kind==="unreleased_effect_hold")await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId:randomUUID(),digest:"pending",command:{action:"tool"},status:"pending"});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
await db.update(heartbeatRuns).set({status:"cancelled",finishedAt:new Date()}).where(eq(heartbeatRuns.id,f.runId));
|
|
const stop=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!.stop;
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary:stop.boundary!,expectedRevision:stop.bindingRevision!,workerStopped:true});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
// Reconstruct exact historical fixture times in this disposable database.
|
|
// The retained answer was never consumed; only native terminal evidence
|
|
// ends its active-input period, and private/effect holds end separately.
|
|
const start=new Date(Date.now()-90000),at=(ms:number)=>new Date(start.getTime()+ms),overlap=kind==="overlapping_window";
|
|
const workStart=at(overlap?10000:-60000),nativeEnd=at(overlap?30000:-40000),privateEnd=at(overlap?50000:-30000),qualificationId=randomUUID();
|
|
await db.update(assignments).set({createdAt:workStart,offeredAt:workStart,claimedAt:workStart,nativeAcceptedAt:workStart,finalizedAt:null}).where(eq(assignments.id,f.assignment.id));
|
|
await db.update(heartbeatRuns).set({startedAt:workStart,finishedAt:nativeEnd}).where(eq(heartbeatRuns.id,f.runId));
|
|
await db.update(museInputDeliveries).set({createdAt:workStart}).where(eq(museInputDeliveries.assignmentId,f.assignment.id));
|
|
await db.update(externalAgentHolds).set({createdAt:workStart,releasedAt:kind==="unreleased_effect_hold"?null:privateEnd}).where(eq(externalAgentHolds.id,hold.id));
|
|
await db.update(bindings).set({qualificationId,qualificationStartedAt:start,qualificationExpiresAt:at(86400000)}).where(eq(bindings.id,f.binding.id));
|
|
const before=(await db.select().from(museInputDeliveries).where(eq(museInputDeliveries.assignmentId,f.assignment.id)))[0];
|
|
const buckets=new Map<number,string[]>(),replicaId=randomUUID();
|
|
for(let ms=0;ms<=80000;ms+=5000){const time=at(ms),bucket=Math.floor(time.getTime()/30000)*30000;const stamps=buckets.get(bucket)??[];stamps.push(time.toISOString());buckets.set(bucket,stamps);}
|
|
await db.insert(museReceiverContactBuckets).values([...buckets].map(([bucketAt,timestamps])=>({companyId:f.company.id,bindingId:f.binding.id,bindingGeneration:1,replicaId,bucketAt:new Date(bucketAt),firstAt:new Date(timestamps[0]),lastAt:new Date(timestamps.at(-1)!),timestamps,contacts:timestamps.length,gapsAtMostSevenSeconds:timestamps.length-1,maxGapMs:5000})));
|
|
const evidence=await f.broker.readQualificationEvidence(f.company.id,f.agent.id,f.binding.id,qualificationId);
|
|
const expected=kind==="unreleased_effect_hold"?[]:overlap
|
|
? [{startedAt:start.toISOString(),endedAt:at(10000).toISOString()},{startedAt:at(50000).toISOString(),endedAt:at(80000).toISOString()}]
|
|
: [{startedAt:start.toISOString(),endedAt:at(80000).toISOString()}];
|
|
expect(evidence.idleWindows).toEqual(expected);
|
|
expect((await db.select().from(museInputDeliveries).where(eq(museInputDeliveries.id,before.id)))[0]).toEqual(before);expect(before.consumedAt).toBeNull();
|
|
const [after]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,hold.id));
|
|
expect(after.workerUnknown).toBe(false);expect(after.nativeEffectsUnknown).toBe(kind==="unreleased_effect_hold");
|
|
expect(after.releasedAt).toEqual(kind==="unreleased_effect_hold"?null:privateEnd);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("preserves a persisted qualification deadline on retry and enforces it without a watcher",async()=>{
|
|
const f=await fixture(),state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!,qualificationId=randomUUID(),input={companyId:f.company.id,agentId:f.agent.id,bindingId:f.binding.id,generation:1,expectedRevision:state.revision,operatorId:f.operatorId,qualificationId,expiresAt:new Date(Date.now()+86400000)};
|
|
const start=await f.broker.beginQualification(input),retry=await f.broker.beginQualification({...input,expiresAt:new Date(Date.now()+86400000)});expect(retry.expiresAt).toEqual(start.expiresAt);expect(Date.parse(start.expiresAt)-Date.parse(start.startedAt)).toBe(86400000);
|
|
await db.update(bindings).set({qualificationExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
await expect(f.identity.authenticate(f.credentials.accessToken)).rejects.toThrow("deadline");
|
|
const [ended]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));expect(ended.revokedAt).toBeInstanceOf(Date);expect(ended.deadlineEnforcedAt).toBeInstanceOf(Date);
|
|
});
|
|
it("reads actual raw assignment timestamps as strings and schema timestamps as Dates",async()=>{
|
|
const f=await active(await oauthFixture(true));
|
|
try {
|
|
const [raw]=await db.execute<{acceptBy:unknown;expiresAt:unknown;createdAt:unknown}>(sql`select accept_by as "acceptBy",expires_at as "expiresAt",created_at as "createdAt" from muse_runner_assignments where id=${f.assignment.id}::uuid`);
|
|
const [mapped]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));
|
|
for(const key of ["acceptBy","expiresAt","createdAt"] as const){
|
|
expect(typeof raw[key]).toBe("string");expect(mapped[key]).toBeInstanceOf(Date);
|
|
expect(Date.parse(String(raw[key]))).toBe(mapped[key].getTime());
|
|
}
|
|
}finally{await f.detach();}
|
|
});
|
|
it.each(["accept_by","expires_at","nonfinite_accept_by","nonfinite_expires_at","unchanged"] as const)("rechecks actual assignment deadlines after a proven claim admission wait (%s)",async deadline=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
let release=()=>{},holder:Promise<void>|undefined,claim:Promise<{result?:unknown;error?:unknown}>|undefined;
|
|
try {
|
|
expect(f.assignment.acceptBy.getTime()).toBeGreaterThan(Date.now());expect(f.assignment.expiresAt.getTime()).toBeGreaterThan(Date.now());
|
|
const requestId=randomUUID(),beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(beforeHolds).toEqual([]);
|
|
let ready=()=>{},holderPid=0;const locked=new Promise<void>(resolve=>{ready=resolve;}),gate=new Promise<void>(resolve=>{release=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:issues.id}).from(issues).where(and(eq(issues.companyId,f.company.id),eq(issues.id,f.issue.id))).for("update");ready();await gate;
|
|
});
|
|
await locked;
|
|
claim=f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId}).then(result=>({result}),error=>({error}));
|
|
await expect.poll(async()=>{
|
|
const [observed]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_stat_activity waiter
|
|
inner join pg_locks waiting on waiting.pid=waiter.pid and waiting.locktype='transactionid' and not waiting.granted
|
|
inner join pg_locks held on held.pid=${holderPid} and held.locktype='transactionid' and held.granted and held.transactionid=waiting.transactionid
|
|
where waiter.datname=current_database() and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and exists(select 1 from pg_locks tuple_lock where tuple_lock.pid=waiter.pid and tuple_lock.locktype='tuple'
|
|
and tuple_lock.relation='issues'::regclass and tuple_lock.mode='AccessExclusiveLock' and tuple_lock.granted)) as waiting`);
|
|
return observed.waiting;
|
|
},{timeout:5000,interval:25}).toBe(true);
|
|
if(deadline==="accept_by")await db.update(assignments).set({acceptBy:new Date(Date.now()-1)}).where(eq(assignments.id,f.assignment.id));
|
|
if(deadline==="expires_at")await db.update(assignments).set({expiresAt:new Date(Date.now()-1)}).where(eq(assignments.id,f.assignment.id));
|
|
if(deadline==="nonfinite_accept_by")await db.execute(sql`update muse_runner_assignments set accept_by='infinity'::timestamptz where id=${f.assignment.id}::uuid`);
|
|
if(deadline==="nonfinite_expires_at")await db.execute(sql`update muse_runner_assignments set expires_at='infinity'::timestamptz where id=${f.assignment.id}::uuid`);
|
|
release();await holder;
|
|
const outcome=await claim;
|
|
const [current]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));
|
|
const receipts=await db.select().from(operations).where(eq(operations.assignmentId,f.assignment.id));
|
|
if(deadline==="unchanged"){
|
|
expect(outcome.error).toBeUndefined();expect(outcome.result).toMatchObject({status:"accepted"});
|
|
expect(current.status).toBe("accepted");expect(current.claimedAt).toBeInstanceOf(Date);expect(current.nativeAcceptedAt).toBeInstanceOf(Date);
|
|
expect(receipts).toHaveLength(1);expect(receipts[0]).toMatchObject({requestId,status:"settled",outcome:{status:"accepted"}});
|
|
const holds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(holds).toHaveLength(1);expect(holds[0]).toMatchObject({workerUnknown:true,nativeEffectsUnknown:false,releasedAt:null});
|
|
expect(await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId})).toEqual(outcome.result);
|
|
expect(await db.select().from(operations).where(eq(operations.assignmentId,f.assignment.id))).toEqual(receipts);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(holds);expect(f.sent).toHaveLength(1);
|
|
}else{
|
|
expect(outcome.error).toMatchObject({status:409});expect(outcome.result).toBeUndefined();
|
|
expect(current).toMatchObject({status:"offered",claimedAt:null,nativeAcceptedAt:null,acceptedResultAt:null,finalizedAt:null});
|
|
expect(receipts).toEqual([]);expect(f.sent).toEqual([]);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
}
|
|
}finally{release();await Promise.allSettled([holder,claim]);await f.detach();}
|
|
});
|
|
it.each(["expired","unchanged"] as const)("checks actual assignment expiry before forwarding a recovered reserved receipt (%s)",async deadline=>{
|
|
const f=await active(await oauthFixture(true));
|
|
await ensureHumanRoleDefaultGrants(db,{companyId:f.company.id,principalId:f.operatorId,membershipRole:"owner",grantedByUserId:f.operatorId});
|
|
const send=vi.fn(async(_command:ExternalProviderOperation)=>{});
|
|
let transaction:ReturnType<typeof vi.spyOn>|undefined,detach:(()=>Promise<void>)|undefined,failure:unknown,requestId=randomUUID();
|
|
try {
|
|
expect((await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe("accepted");await f.detach();
|
|
const input={text:"Recovered progress"},command={requestId,bindingId:f.binding.id,bindingGeneration:f.binding.generation,runId:f.runId,normalizedSessionId:f.assignment.normalizedSessionId,turnId:f.assignment.turnId,assignmentRevision:f.assignment.revision,action:"progress",input,digest:externalOperationDigest("progress",input)};
|
|
const [reserved]=await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId,digest:command.digest,command,status:"reserved"}).returning();
|
|
const beforeHolds=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
if(deadline==="expired")await db.update(assignments).set({expiresAt:new Date(Date.now()-1)}).where(eq(assignments.id,f.assignment.id));
|
|
transaction=vi.spyOn(db,"transaction");
|
|
detach=await f.port.attach(send,undefined,true);
|
|
await expect.poll(()=>transaction!.mock.results.length,{timeout:5000,interval:25}).toBeGreaterThanOrEqual(2);
|
|
expect(transaction.mock.results[1].type).toBe("return");await transaction.mock.results[1].value;
|
|
if(deadline==="expired"){
|
|
expect(send).not.toHaveBeenCalled();
|
|
expect((await db.select().from(operations).where(eq(operations.id,reserved.id)))[0]).toEqual(reserved);
|
|
expect(await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id))).toEqual(beforeHolds);
|
|
}else{
|
|
await expect.poll(async()=>(await db.select().from(operations).where(eq(operations.id,reserved.id)))[0].status,{timeout:5000,interval:25}).toBe("pending");
|
|
expect(send).toHaveBeenCalledTimes(1);expect(send.mock.calls[0][0]).toEqual(command);
|
|
await f.port.settle({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId,outcome:{status:"completed"}}});
|
|
expect(await f.broker.act(f.subject,{version:1,command:"progress",assignmentId:f.assignment.id,requestId,text:input.text})).toEqual({status:"completed"});
|
|
expect(send).toHaveBeenCalledTimes(1);
|
|
const receipts=await db.select().from(operations).where(eq(operations.requestId,requestId));expect(receipts).toHaveLength(1);expect(receipts[0]).toMatchObject({id:reserved.id,status:"settled",outcome:{status:"completed"}});
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toMatchObject({workerUnknown:true,nativeEffectsUnknown:false,releasedAt:null});
|
|
}
|
|
}catch(error){failure=error;}
|
|
finally{
|
|
try {
|
|
if(send.mock.calls.length)await expect.poll(async()=>["pending","settled"].includes((await db.select().from(operations).where(eq(operations.requestId,requestId)))[0]?.status??""),{timeout:5000,interval:25}).toBe(true);
|
|
}catch(error){if(failure)console.error("MUSE_ASSIGNMENT_FORWARD_CLEANUP_ACK_FAILED");else failure=error;}
|
|
await detach?.();transaction?.mockRestore();await f.detach();
|
|
}
|
|
if(failure)throw failure;
|
|
});
|
|
it("rejects a claim when reassignment commits while its exact issue lock is held",async()=>{
|
|
const f=await active();let unlock!:()=>void,held!:()=>void,holderPid!:number;
|
|
const gate=new Promise<void>(resolve=>{unlock=resolve;}),locked=new Promise<void>(resolve=>{held=resolve;});
|
|
const reassignment=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
await tx.select({id:issues.id}).from(issues).where(eq(issues.id,f.issue.id)).for("update");held();await gate;
|
|
await tx.update(issues).set({assigneeAgentId:null,checkoutRunId:null,executionRunId:null}).where(eq(issues.id,f.issue.id));
|
|
});
|
|
await locked;let settled=false;
|
|
const claim=f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()}).finally(()=>{settled=true;});
|
|
void claim.catch(()=>{});
|
|
try {
|
|
// Observe the real DB wait, rather than assume asynchronous preflight
|
|
// checks have reached reservation after an arbitrary sleep. This holder
|
|
// locks only this task; blocker PID + issues FOR UPDATE identifies the
|
|
// claim's admission barrier without recording private SQL or parameters.
|
|
let waiting=false;const deadline=Date.now()+5000;
|
|
while(Date.now()<deadline&&!settled) {
|
|
const waiters=await db.execute<{pid:number}>(sql`
|
|
select waiter.pid from pg_stat_activity waiter
|
|
where waiter.datname=current_database() and waiter.wait_event_type='Lock'
|
|
and ${holderPid}=any(pg_blocking_pids(waiter.pid))
|
|
and waiter.query ~* '(from|join)[[:space:]]+"?issues"?'
|
|
and waiter.query ~* 'for[[:space:]]+update'
|
|
limit 2`);
|
|
if(waiters.length===1){waiting=true;break;}
|
|
await new Promise(resolve=>setTimeout(resolve,25));
|
|
}
|
|
expect(waiting).toBe(true);expect(settled).toBe(false);
|
|
unlock();await reassignment;
|
|
await expect(claim).rejects.toThrow("current task");
|
|
expect(f.sent).toEqual([]);
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0].claimedAt).toBeNull();
|
|
} finally {unlock();await reassignment;await claim.catch(()=>{});await f.detach();}
|
|
});
|
|
it("reconciles pending sends to an independent unknown barrier on controller recovery",async()=>{
|
|
const f=await active();await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});await f.detach();
|
|
const requestId=randomUUID(),input={name:"write_document",arguments:{}},command={requestId,bindingId:f.binding.id,bindingGeneration:1,runId:f.runId,normalizedSessionId:f.assignment.normalizedSessionId,turnId:f.assignment.turnId,assignmentRevision:1,action:"tool",input,digest:externalOperationDigest("tool",input)};
|
|
await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId,digest:command.digest,command,status:"pending"});
|
|
const send=vi.fn(),detach=await f.port.attach(send,undefined,true);
|
|
try {expect((await db.select().from(operations).where(eq(operations.requestId,requestId)))[0].status).toBe("unknown");expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0].nativeEffectsUnknown).toBe(true);expect((await f.broker.act(f.subject,{version:1,command:"tool",assignmentId:f.assignment.id,requestId,name:input.name,arguments:{}})).status).toBe("unknown");expect(send).not.toHaveBeenCalled();}finally{await detach();}
|
|
});
|
|
it("does not restore native uncertainty when a saved result precedes a lost command acknowledgement",async()=>{
|
|
let releaseFailure!:()=>void;
|
|
const failAfterResult=new Promise<void>(resolve=>{releaseFailure=resolve;});
|
|
const f=await active(undefined,async command=>{if(command.action==="accept"){await failAfterResult;throw new Error("Command acknowledgement lost after result");}});
|
|
let transaction:ReturnType<typeof vi.spyOn>|undefined;
|
|
try {
|
|
const requestId=randomUUID();
|
|
expect(await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId})).toMatchObject({status:"accepted"});
|
|
await f.broker.act(f.subject,{version:1,command:"finish",assignmentId:f.assignment.id,requestId:randomUUID(),result:{}});
|
|
const [saved]=await db.select().from(operations).where(eq(operations.requestId,requestId));
|
|
// Observe completion of the real failure transaction, without timing the
|
|
// negative hold assertion or replacing the database implementation.
|
|
transaction=vi.spyOn(db,"transaction");
|
|
releaseFailure();
|
|
await expect.poll(()=>transaction!.mock.results.length).toBeGreaterThan(0);
|
|
await transaction.mock.results[0].value;
|
|
transaction.mockRestore();transaction=undefined;
|
|
expect((await db.select().from(operations).where(eq(operations.id,saved.id)))[0]).toEqual(saved);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0].nativeEffectsUnknown).toBe(false);
|
|
await db.update(heartbeatRuns).set({status:"succeeded",finishedAt:new Date()}).where(eq(heartbeatRuns.id,f.runId));
|
|
await f.broker.reconcileTerminalAssignments(f.company.id,f.agent.id,f.binding.id);
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(hold.workerUnknown).toBe(false);expect(hold.nativeEffectsUnknown).toBe(false);expect(hold.releasedAt).toBeInstanceOf(Date);
|
|
await expect(db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)))).resolves.toBeUndefined();
|
|
}finally{releaseFailure();transaction?.mockRestore();await f.detach();}
|
|
});
|
|
|
|
it("retains native uncertainty for an unconfirmed failed send until its actual result arrives",async()=>{
|
|
const f=await active();await f.detach();
|
|
const detach=await f.port.attach(async()=>{throw new Error("No command acknowledgement or result");},undefined,true);
|
|
try {
|
|
const requestId=randomUUID();
|
|
expect(await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId})).toMatchObject({status:"unknown",canReplay:false});
|
|
const [pending]=await db.select().from(operations).where(eq(operations.requestId,requestId));expect(pending.status).toBe("unknown");expect(pending.outcome).toBeNull();
|
|
const [held]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));expect(held.workerUnknown).toBe(true);expect(held.nativeEffectsUnknown).toBe(true);expect(held.releasedAt).toBeNull();
|
|
await f.port.settle({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId,outcome:{status:"accepted"}}});
|
|
const [resolved]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,held.id));expect(resolved.workerUnknown).toBe(true);expect(resolved.nativeEffectsUnknown).toBe(false);expect(resolved.releasedAt).toBeNull();
|
|
}finally{await detach();}
|
|
});
|
|
|
|
it("notifies maintenance when a real Muse finish receipt commits", async () => {
|
|
const f = await active();
|
|
const events: string[] = [];
|
|
let unsubscribe = () => {};
|
|
try {
|
|
await f.broker.act(f.subject, { version: 1, command: "accept", assignmentId: f.assignment.id, requestId: randomUUID() });
|
|
unsubscribe = subscribeDatabaseWork(db, DELIVERY_QUEUES.museMaintenance, event => events.push(event));
|
|
await f.broker.act(f.subject, { version: 1, command: "finish", assignmentId: f.assignment.id, requestId: randomUUID(), result: {} });
|
|
expect(events).toContain("intent");
|
|
expect(events.at(-1)).toBe("settled");
|
|
expect((await db.select().from(assignments).where(eq(assignments.id, f.assignment.id)))[0].acceptedResultAt).not.toBeNull();
|
|
} finally { unsubscribe(); await f.detach(); }
|
|
});
|
|
|
|
it("does not rewrite settled Muse terminal history", async () => {
|
|
const f = await active();
|
|
try {
|
|
await f.broker.act(f.subject, { version: 1, command: "accept", assignmentId: f.assignment.id, requestId: randomUUID() });
|
|
await f.broker.act(f.subject, { version: 1, command: "finish", assignmentId: f.assignment.id, requestId: randomUUID(), result: {} });
|
|
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, f.runId));
|
|
await f.broker.reconcileTerminalAssignments(f.company.id, f.agent.id, f.binding.id);
|
|
const version = async () => (await db.select({ version: sql<string>`${assignments}.xmin::text` })
|
|
.from(assignments).where(eq(assignments.id, f.assignment.id)))[0].version;
|
|
const settled = await version();
|
|
await f.broker.reconcileTerminalAssignments(f.company.id, f.agent.id, f.binding.id);
|
|
expect(await version()).toBe(settled);
|
|
const [hold] = await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId, f.assignment.id));
|
|
expect(hold).toMatchObject({ workerUnknown: false, nativeEffectsUnknown: false, releasedAt: expect.any(Date) });
|
|
} finally { await f.detach(); }
|
|
});
|
|
|
|
it("makes identical native result replay idempotent after finish and finalization",async()=>{
|
|
const f=await active();try {
|
|
const acceptId=randomUUID(),finishId=randomUUID();
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:acceptId});
|
|
await f.broker.act(f.subject,{version:1,command:"finish",assignmentId:f.assignment.id,requestId:finishId,result:{}});
|
|
const [finished]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));expect(finished.status).toBe("settled");
|
|
const receipts=await db.select().from(operations).where(eq(operations.assignmentId,f.assignment.id));
|
|
const replay=(requestId:string,status:string)=>f.port.settle({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId,outcome:{status}}});
|
|
await Promise.all([replay(acceptId,"accepted"),replay(acceptId,"accepted"),replay(finishId,"completed")]);
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual(finished);
|
|
expect(await db.select().from(operations).where(eq(operations.assignmentId,f.assignment.id)).orderBy(operations.id)).toEqual([...receipts].sort((a,b)=>a.id.localeCompare(b.id)));
|
|
await db.update(heartbeatRuns).set({status:"succeeded",finishedAt:new Date()}).where(eq(heartbeatRuns.id,f.runId));
|
|
await f.broker.reconcileTerminalAssignments(f.company.id,f.agent.id,f.binding.id);
|
|
const [finalized]=await db.select().from(assignments).where(eq(assignments.id,f.assignment.id));
|
|
const [released]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));expect(released.releasedAt).toBeInstanceOf(Date);
|
|
await Promise.all([replay(acceptId,"accepted"),replay(finishId,"completed")]);
|
|
await expect(replay(acceptId,"completed")).rejects.toThrow("conflicts");
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0]).toEqual(finalized);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,released.id)))[0]).toEqual(released);
|
|
await expect(db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)))).resolves.toBeUndefined();
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("keeps historical Dot result replay idempotent after provider replacement",async()=>{
|
|
const f=await fixture(),runId=randomUUID(),session=randomUUID(),turnId="dot_"+randomUUID();
|
|
// A retained Dot journal can settle after its agent switches provider. It
|
|
// grants no fresh work authority and does not require a live connection.
|
|
const [binding]=await db.insert(dotAgentBindings).values({companyId:f.company.id,agentId:f.agent.id,operatorId:f.operatorId,status:"revoked",revokedAt:new Date()}).returning();
|
|
await db.insert(heartbeatRuns).values({id:runId,companyId:f.company.id,agentId:f.agent.id,status:"succeeded",runtimeMode:"native",driverKind:"openai_dot_mcp",nativeSessionId:session});
|
|
const [assignment]=await db.insert(dotRunnerAssignments).values({companyId:f.company.id,agentId:f.agent.id,bindingId:binding.id,bindingGeneration:1,runId,normalizedSessionId:session,turnId,controllerGeneration:1,catalogDigest:"dot",status:"offered",projection:{},acceptBy:new Date(),expiresAt:new Date(Date.now()+60000)}).returning();
|
|
const acceptId=randomUUID(),finishId=randomUUID();
|
|
await db.insert(dotRunnerOperations).values([{companyId:f.company.id,assignmentId:assignment.id,requestId:acceptId,digest:"accept",command:{action:"accept"},status:"admitted"},{companyId:f.company.id,assignmentId:assignment.id,requestId:finishId,digest:"finish",command:{action:"finish"},status:"admitted"}]);
|
|
const ref={bindingId:binding.id,bindingGeneration:1,companyId:f.company.id,agentId:f.agent.id,acceptByUnixMs:Date.now(),expiresAtUnixMs:Date.now()+60000,runId,normalizedSessionId:session,turnId,assignmentRevision:1};
|
|
const port=dotRunnerBroker(db).port({binding:{companyId:f.company.id,agentId:f.agent.id,runId},provider:{binding:ref}});
|
|
const settle=(requestId:string,status:string)=>port.settle({sourceEventId:randomUUID(),payload:{binding:ref,requestId,outcome:{status}}});
|
|
await settle(acceptId,"accepted");await settle(finishId,"completed");
|
|
const [finished]=await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.id,assignment.id));expect(finished.status).toBe("settled");
|
|
const receipts=await db.select().from(dotRunnerOperations).where(eq(dotRunnerOperations.assignmentId,assignment.id)).orderBy(dotRunnerOperations.id);
|
|
await Promise.all([settle(acceptId,"accepted"),settle(finishId,"completed")]);
|
|
await expect(settle(acceptId,"completed")).rejects.toThrow("conflicts");
|
|
expect((await db.select().from(dotRunnerAssignments).where(eq(dotRunnerAssignments.id,assignment.id)))[0]).toEqual(finished);
|
|
expect(await db.select().from(dotRunnerOperations).where(eq(dotRunnerOperations.assignmentId,assignment.id)).orderBy(dotRunnerOperations.id)).toEqual(receipts);
|
|
});
|
|
|
|
it("rearms an unclaimed offer after a lost wake but never restarts a claimed assignment",async()=>{
|
|
const f=await active();try {
|
|
const first=await museReceiver(db).signal(f.binding.id);
|
|
await db.update(mailbox).set({signalNotifiedAt:new Date(Date.now()-31000)}).where(eq(mailbox.assignmentId,f.assignment.id));
|
|
const retry=await museReceiver(db).signal(f.binding.id);expect(retry.signal?.reference).not.toEqual(first.signal?.reference);
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});expect((await museReceiver(db).signal(f.binding.id)).signal).toBeNull();
|
|
}finally{await f.detach();}
|
|
});
|
|
it("wakes the exact v9 detector for a distinct offer after native success and mailbox acknowledgement",async()=>{
|
|
const first=await active(await oauthFixture(true));
|
|
let second:Awaited<ReturnType<typeof active>>|undefined;
|
|
try {
|
|
const poll=()=>museReceiver(db).signal(first.binding.id,oauthResource);
|
|
const offeredA=await poll();expect(offeredA.signal?.reference).toMatch(/^sha256:[a-f0-9]{64}$/);
|
|
expect(await first.broker.act(first.subject,{version:1,command:"accept",assignmentId:first.assignment.id,requestId:randomUUID()})).toMatchObject({status:"accepted"});
|
|
expect(await first.broker.act(first.subject,{version:1,command:"finish",assignmentId:first.assignment.id,requestId:randomUUID(),result:{}})).toMatchObject({status:"completed"});
|
|
await db.update(heartbeatRuns).set({status:"succeeded",finishedAt:new Date()}).where(eq(heartbeatRuns.id,first.runId));
|
|
await first.broker.reconcileTerminalAssignments(first.company.id,first.agent.id,first.binding.id);
|
|
const [completed]=await db.select().from(assignments).where(eq(assignments.id,first.assignment.id));
|
|
expect(completed).toMatchObject({status:"settled",nativeAcceptedAt:expect.any(Date),acceptedResultAt:expect.any(Date),finalizedAt:expect.any(Date)});
|
|
const inbox=await first.broker.inspect(first.subject,{version:1,query:"mailbox",after:0}) as {items:Array<{assignmentId:string|null}>;nextCursor:number};
|
|
expect(inbox.items.some(item=>item.assignmentId===first.assignment.id)).toBe(true);
|
|
await first.broker.inspect(first.subject,{version:1,query:"mailbox",after:inbox.nextCursor});
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,first.binding.id)))[0].workerCursor).toBe(inbox.nextCursor);
|
|
const idleAfterSuccess=await poll();expect(idleAfterSuccess).toEqual({version:1,signal:null});
|
|
await first.detach();
|
|
second=await active(first);
|
|
expect(second.assignment.id).not.toBe(first.assignment.id);expect(second.runId).not.toBe(first.runId);
|
|
expect(second.assignment.bindingId).toBe(first.binding.id);expect(second.assignment.bindingGeneration).toBe(first.binding.generation);
|
|
const [offerB]=await db.select().from(mailbox).where(and(eq(mailbox.assignmentId,second.assignment.id),eq(mailbox.kind,"assignment")));
|
|
const offeredB=await poll();
|
|
expect(offeredB).toEqual({version:1,signal:{reference:"sha256:"+createHash("sha256").update(`${first.binding.id}:${first.binding.generation}:${offerB.sourceEventId}:1`).digest("hex")}});
|
|
expect(offeredB.signal?.reference).not.toBe(offeredA.signal?.reference);
|
|
const duplicateB=await poll();expect(duplicateB).toEqual(offeredB);
|
|
await db.update(mailbox).set({signalNotifiedAt:new Date(Date.now()-31000)}).where(eq(mailbox.id,offerB.id));
|
|
const rearmedB=await poll();
|
|
expect(rearmedB).toEqual({version:1,signal:{reference:"sha256:"+createHash("sha256").update(`${first.binding.id}:${first.binding.generation}:${offerB.sourceEventId}:2`).digest("hex")}});
|
|
expect(rearmedB.signal?.reference).not.toBe(offeredB.signal?.reference);
|
|
expect(await second.broker.act(second.subject,{version:1,command:"accept",assignmentId:second.assignment.id,requestId:randomUUID()})).toMatchObject({status:"accepted"});
|
|
const [claimed]=await db.select().from(assignments).where(eq(assignments.id,second.assignment.id));
|
|
expect(claimed).toMatchObject({claimedAt:expect.any(Date),nativeAcceptedAt:expect.any(Date)});
|
|
const idleAfterClaim=await poll();expect(idleAfterClaim).toEqual({version:1,signal:null});
|
|
const proof=JSON.parse(execFileSync("python3",[
|
|
fileURLToPath(new URL("./fixtures/muse-client-contract.py",import.meta.url)),
|
|
fileURLToPath(new URL("../services/scripts/v9/client.py",import.meta.url)),"--receiver-sequence",
|
|
],{input:JSON.stringify({bindingId:first.binding.id,origin:oauthOrigin,scope:{companyId:first.company.id,companyName:first.company.name,agentId:first.agent.id,agentName:first.agent.name},responses:[offeredA,idleAfterSuccess,offeredB,duplicateB,rearmedB,idleAfterClaim]}),encoding:"utf8",timeout:10000}));
|
|
expect(proof).toEqual({outputs:["wake","idle","wake","duplicate","wake","idle"],providerAcknowledgement:"not_observed"});
|
|
}finally{await second?.detach();await first.detach();}
|
|
});
|
|
it("rejects missing public origin before replacing a durable connection",async()=>{
|
|
const f=await fixture(),state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
// A broker instance keeps its configured public origin; use a fresh DB
|
|
// wrapper to exercise the same persisted binding with a bad setup origin.
|
|
const bad=museRunnerBroker(createDb(externalDatabaseUrl??temporary!.connectionString),{publicOrigin:"http://invalid.example"});
|
|
await expect(bad.createPairing({companyId:f.company.id,agentId:f.agent.id,operatorId:f.operatorId,replaceBindingId:f.binding.id,expectedRevision:state.revision})).rejects.toThrow("HTTPS");
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].revokedAt).toBeNull();
|
|
});
|
|
|
|
it("denies an existing subject after its same-provider configured binding changes",async()=>{
|
|
const f=await fixture();await db.update(agents).set({adapterConfig:{provider:"muse",museBindingId:randomUUID(),allowUnmeteredProvider:true}}).where(eq(agents.id,f.agent.id));
|
|
await expect(f.broker.inspect(f.subject,{version:1,query:"task.list"})).rejects.toThrow("authority");
|
|
await expect(f.identity.authenticate(f.credentials.accessToken)).rejects.toThrow("authority");
|
|
});
|
|
|
|
it("gives a settled result an exact stop boundary when native finalization fails",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await db.update(assignments).set({status:"settled",acceptedResultAt:new Date()}).where(eq(assignments.id,f.assignment.id));
|
|
await db.update(heartbeatRuns).set({status:"failed",finishedAt:new Date()}).where(eq(heartbeatRuns.id,f.runId));
|
|
await f.broker.reconcileTerminalAssignments(f.company.id,f.agent.id,f.binding.id);
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;expect(state.stop.status).toBe("cannot_confirm");expect(state.stop.boundary?.assignmentId).toBe(f.assignment.id);
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary:state.stop.boundary!,expectedRevision:state.revision,workerStopped:true});
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0].releasedAt).toBeInstanceOf(Date);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("keeps the native-effect barrier when a Runner fence precedes worker attestation",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId:randomUUID(),digest:"pending",command:{action:"tool"},status:"pending"});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;expect(state.stop.nativeEffectsUnknown).toBe(true);
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary:state.stop.boundary!,expectedRevision:state.revision,workerStopped:true});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));expect(hold.releasedAt).toBeNull();expect(hold.nativeEffectsUnknown).toBe(true);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("preserves the original stop receipt across concurrent Runner replays and worker reporting",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const event={sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}};
|
|
await f.port.dispatch(event);
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!,boundary=state.stop.boundary!;
|
|
const [before]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
await Promise.all([f.port.dispatch(event),f.port.dispatch(event)]);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toEqual(before);
|
|
await f.broker.revoke(f.company.id,f.agent.id,f.operatorId,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});
|
|
const cleanup=await f.identity.authenticate(f.credentials.cleanupToken,"cleanup");
|
|
await f.broker.cleanup(cleanup,{command:"worker.quiescent",boundary,requestId:randomUUID()});
|
|
const [reported]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(reported.workerReportedAt).toBeInstanceOf(Date);
|
|
await f.port.dispatch(event);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toEqual(reported);
|
|
const receipts=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.sourceEventId,event.sourceEventId)));
|
|
expect(receipts).toHaveLength(1);expect(receipts[0].references.boundary).toEqual(boundary);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("does not revive an attested and released hold when its Runner stop event replays",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const event={sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}};
|
|
await f.port.dispatch(event);
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary:state.stop.boundary!,expectedRevision:state.stop.bindingRevision!,workerStopped:true});
|
|
await db.update(heartbeatRuns).set({status:"cancelled",finishedAt:new Date()}).where(eq(heartbeatRuns.id,f.runId));
|
|
const [attested]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(attested.workerUnknown).toBe(false);expect(attested.releasedAt).toBeInstanceOf(Date);
|
|
await db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)));
|
|
await f.port.dispatch(event);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toEqual(attested);
|
|
await db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)));
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("preserves reported worker stop and independent native uncertainty on Runner replay",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId:randomUUID(),digest:"pending",command:{action:"tool"},status:"pending"});
|
|
const event={sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}};
|
|
await f.port.dispatch(event);
|
|
const before=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.revoke(f.company.id,f.agent.id,f.operatorId,{bindingId:f.binding.id,generation:1,expectedRevision:before.revision});
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
const cleanup=await f.identity.authenticate(f.credentials.cleanupToken,"cleanup");
|
|
await f.broker.cleanup(cleanup,{command:"worker.quiescent",boundary:state.stop.boundary!,requestId:randomUUID()});
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary:state.stop.boundary!,expectedRevision:state.stop.bindingRevision!,workerStopped:true});
|
|
const [attested]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(attested.workerUnknown).toBe(false);expect(attested.nativeEffectsUnknown).toBe(true);expect(attested.releasedAt).toBeNull();
|
|
expect(attested.workerReportedAt).toBeInstanceOf(Date);expect(attested.operatorAttestedAt).toBeInstanceOf(Date);
|
|
await f.port.dispatch(event);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toEqual(attested);
|
|
await db.update(heartbeatRuns).set({status:"cancelled",finishedAt:new Date()}).where(eq(heartbeatRuns.id,f.runId));
|
|
await expect(db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)))).rejects.toThrow("unresolved");
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("retains an attested server fence when its distinct late Runner stop fact arrives",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const before=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.revoke(f.company.id,f.agent.id,f.operatorId,{bindingId:f.binding.id,generation:1,expectedRevision:before.revision});
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!,boundary=state.stop.boundary!;
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary,expectedRevision:state.stop.bindingRevision!,workerStopped:true});
|
|
const [attested]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const event={sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}};
|
|
await f.port.dispatch(event);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toEqual(attested);
|
|
const [receipt]=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.sourceEventId,event.sourceEventId)));
|
|
expect(receipt.references.boundary).toEqual(boundary);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("holds newly unresolved native effects without erasing an attested worker stop",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.attestStop(f.company.id,f.agent.id,f.operatorId,{boundary:state.stop.boundary!,expectedRevision:state.stop.bindingRevision!,workerStopped:true});
|
|
await db.update(heartbeatRuns).set({status:"cancelled",finishedAt:new Date()}).where(eq(heartbeatRuns.id,f.runId));
|
|
const [attested]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(attested.releasedAt).toBeInstanceOf(Date);
|
|
await db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)));
|
|
await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId:randomUUID(),digest:"recovered-pending",command:{action:"tool"},status:"pending"});
|
|
const event={sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}};
|
|
await f.port.dispatch(event);
|
|
const [held]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(held.stopBoundary).toEqual(attested.stopBoundary);expect(held.operatorAttestedAt).toEqual(attested.operatorAttestedAt);
|
|
expect(held.workerReportedAt).toEqual(attested.workerReportedAt);expect(held.workerUnknown).toBe(false);
|
|
expect(held.nativeEffectsUnknown).toBe(true);expect(held.releasedAt).toBeNull();
|
|
await expect(db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,()=>assertNoExternalOverlap(tx,f.company.id,f.agent.id)))).rejects.toThrow("unresolved");
|
|
await f.port.dispatch(event);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toEqual(held);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("rejects a conflicting Runner source event before changing assignment or stop state",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const [offer]=await db.select().from(mailbox).where(and(eq(mailbox.assignmentId,f.assignment.id),eq(mailbox.kind,"assignment")));
|
|
const [before]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
await expect(f.port.dispatch({sourceEventId:offer.sourceEventId,payload:{kind:"authority_revoked",binding:f.ref}})).rejects.toThrow("event identity");
|
|
expect((await db.select().from(assignments).where(eq(assignments.id,f.assignment.id)))[0].status).toBe("accepted");
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id)))[0]).toEqual(before);
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("denies already-dispatched native authority immediately at the qualification deadline",async()=>{
|
|
const f=await active();try {
|
|
await db.update(bindings).set({qualificationId:randomUUID(),qualificationExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
await expect(f.broker.assertRunAuthority({binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}})).rejects.toThrow("deadline");
|
|
await expect(f.broker.authorizingUserIdForBinding(f.ref)).rejects.toThrow("deadline");
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].revokedAt).toBeNull();
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it("promotes a verifying Muse hire through the ordinary lifecycle after authenticated background confirmation",async()=>{
|
|
const f=await fixture(false);
|
|
await db.update(agents).set({status:"paused",lifecycleState:"verifying",lifecycleOperation:{id:randomUUID(),hostComplete:false,completedPluginIds:[],attempts:0,responsibleUserId:f.operatorId},lifecycleRequiredPluginIds:[]}).where(eq(agents.id,f.agent.id));
|
|
const harness=agentHarnessVerificationService(db,{} as PluginWorkerManager),onReady=vi.fn();
|
|
const lifecycle=startAgentLifecycle(db,{requiredPluginIds:async()=>[],runPlugin:async()=>"complete",runHost:snapshot=>harness.verify(snapshot),onReady},()=>true);
|
|
const deliveryErrors=vi.fn(),coordinator=createDeliveryWorkCoordinator({owner:db,canRun:()=>true,onError:deliveryErrors});
|
|
const delivery=coordinator.register(DELIVERY_QUEUES.agentLifecycle,{retryMs:1000,run:()=>lifecycle.sweep(),hasPending:()=>lifecycle.hasPending(),nextRunAt:()=>lifecycle.nextRunAt()});
|
|
lifecycle.setWake(delivery.wake);
|
|
try {
|
|
await delivery.ready;expect((await db.select().from(agents).where(eq(agents.id,f.agent.id)))[0].lifecycleState).toBe("verifying");
|
|
// The authenticated confirmation must retry this hire, not wait for a
|
|
// periodic scan or a previously persisted long backoff to expire.
|
|
await db.update(agents).set({lifecycleOperation:sql`${agents.lifecycleOperation} || jsonb_build_object('retryAt', ${new Date(Date.now()+60_000).toISOString()}::text)`}).where(eq(agents.id,f.agent.id));
|
|
await museReceiver(db).signal(f.binding.id);await museReceiver(db).flush();
|
|
const items=await f.broker.inspect(f.subject,{version:1,query:"mailbox",after:0}) as {items:Array<{references:{nonce:string}}>};
|
|
await f.broker.act(f.subject,{version:1,command:"challenge.confirm",nonce:items.items[0].references.nonce,requestId:randomUUID()});
|
|
await expect.poll(async()=> (await db.select().from(agents).where(eq(agents.id,f.agent.id)))[0].lifecycleState,{timeout:8000,interval:100}).toBe("ready");
|
|
expect((await db.select().from(agents).where(eq(agents.id,f.agent.id)))[0].status).toBe("idle");expect(onReady).toHaveBeenCalledOnce();
|
|
expect(deliveryErrors).not.toHaveBeenCalled();
|
|
}finally{await coordinator.stop();await lifecycle.stop();}
|
|
});
|
|
|
|
it("serves opaque public wake digests while worker and cleanup APIs still require private bearer authority",async()=>{
|
|
const f=await active();try {
|
|
const app=transport(),reply=await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id);
|
|
expect(reply.status).toBe(200);expect(Object.keys(reply.body)).toEqual(["version","signal"]);expect(Object.keys(reply.body.signal)).toEqual(["reference"]);
|
|
const [item]=await db.select().from(mailbox).where(eq(mailbox.assignmentId,f.assignment.id));
|
|
expect(reply.body.signal.reference).toBe("sha256:"+createHash("sha256").update(`${f.binding.id}:1:${item.sourceEventId}:${item.signalAttempt}`).digest("hex"));
|
|
expect(JSON.stringify(reply.body)).not.toContain(f.assignment.id);expect(JSON.stringify(reply.body)).not.toContain(f.issue.title);
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id)).body).toEqual(reply.body);
|
|
expect((await request(app).get("/api/muse/v1/signal")).status).toBe(400);
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding","invalid")).status).toBe(400);
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",randomUUID())).status).toBe(410);
|
|
expect((await request(app).get("/api/muse/v1/signal?binding="+f.binding.id)).status).toBe(404);
|
|
expect((await request(app).post("/api/muse/v1/commands").set("X-Paperclip-Muse-Binding",f.binding.id).send({version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).status).toBe(401);
|
|
expect((await request(app).post("/api/muse/v1/detector-cleanup").set("X-Paperclip-Muse-Binding",f.binding.id).send({version:1,bindingId:f.binding.id,generation:1,requestId:randomUUID(),detectorRemoved:true})).status).toBe(401);
|
|
expect((await request(app).post("/api/muse/v1/queries").set("X-Paperclip-Muse-Binding",f.binding.id).send({version:1,query:"identify"})).status).toBe(401);
|
|
expect((await request(app).post("/api/muse/v1/cleanup").set("X-Paperclip-Muse-Binding",f.binding.id).send({version:1,command:"control.inspect"})).status).toBe(401);
|
|
}finally{await f.detach();}
|
|
});
|
|
it.each(["input", "challenge"] as const)("signals pending stop control while paused and keeps ordinary %s quiet",async(kind)=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth);try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
if(kind==="input") {
|
|
const requestId="paused_question",response={schema:"paperclip.question_response.v1",answers:[]},inputDigest=digestPaperclipSemanticContent({requestId,turnId:f.ref.turnId,response});
|
|
await f.port.inputAvailable({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId,turnId:f.ref.turnId,inputDigest,response}});
|
|
} else {
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});
|
|
}
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
await db.update(agents).set({status:"paused"}).where(eq(agents.id,f.agent.id));
|
|
const app=transport(),poll=()=>request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id);
|
|
const first=await poll();expect(first.status).toBe(200);expect(first.body.signal?.reference).toMatch(/^sha256:/);
|
|
const [stop]=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"authority_revoked")));
|
|
expect(stop.signalAttempt).toBe(1);expect(stop.signalNotifiedAt).toBeInstanceOf(Date);
|
|
const ordinary=await db.select().from(mailbox).where(eq(mailbox.bindingId,f.binding.id));
|
|
expect(ordinary.filter(item=>item.kind!=="authority_revoked").every(item=>item.signalNotifiedAt===null)).toBe(true);
|
|
const inbox=await request(app).post("/api/muse/v1/queries").set("Authorization","Bearer "+oauth.tokens!.access_token).send({version:1,query:"mailbox",after:0});
|
|
expect(inbox.status).toBe(200);expect(inbox.body).toMatchObject({items:expect.arrayContaining([expect.objectContaining({kind:"authority_revoked",references:stop.references})])});
|
|
const ack=await request(app).post("/api/muse/v1/queries").set("Authorization","Bearer "+oauth.tokens!.access_token).send({version:1,query:"mailbox",after:stop.id});
|
|
expect(ack.status).toBe(200);expect((await poll()).body).toEqual({version:1,signal:null});
|
|
const accept=await request(app).post("/api/muse/v1/commands").set("Authorization","Bearer "+oauth.tokens!.access_token).send({version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
expect(accept.status).toBe(403);expect(f.sent).toHaveLength(1);
|
|
expect((await f.broker.bindingForAgent(f.company.id,f.agent.id))!.stop.status).toBe("cannot_confirm");
|
|
}finally{await f.detach();}
|
|
});
|
|
|
|
it.each(["enableMuse","enableNativeRunner"] as const)("delivers only feature-disabled stop control while %s is off",async flag=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth),app=transport();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});
|
|
const nativeRequestId="disabled_question",response={schema:"paperclip.question_response.v1",answers:{answer:{text:"PRIVATE pending input canary"}}};
|
|
const inputDigest=digestPaperclipSemanticContent({requestId:nativeRequestId,turnId:f.ref.turnId,response});
|
|
await f.port.inputAvailable({sourceEventId:randomUUID(),payload:{binding:f.ref,requestId:nativeRequestId,turnId:f.ref.turnId,inputDigest,response}});
|
|
await db.insert(operations).values({companyId:f.company.id,assignmentId:f.assignment.id,requestId:randomUUID(),digest:"pending",command:{action:"tool"},status:"pending"});
|
|
await instanceSettingsService(db).updateExperimental({[flag]:false});
|
|
// A real native fence fact must remain ingestible after ordinary work is
|
|
// disabled. Both exact replay and a later fact preserve the same boundary.
|
|
const fence={sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}};
|
|
await f.port.dispatch(fence);await f.port.dispatch(fence);
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
const [stop]=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"authority_revoked")));
|
|
const authorization="Bearer "+oauth.tokens!.access_token;
|
|
const poll=()=>request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id);
|
|
const [signal,inbox,inspection]=await Promise.all([
|
|
poll(),request(app).post("/api/muse/v1/queries").set("Authorization",authorization).send({version:1,query:"mailbox",after:0}),
|
|
request(app).post("/api/muse/v1/cleanup").set("Authorization",authorization).send({version:1,command:"control.inspect"}),
|
|
]);
|
|
expect({signal:!!signal.body.signal,mailbox:inbox.status,cleanup:inspection.status}).toEqual({signal:true,mailbox:200,cleanup:200});
|
|
expect(signal.body.signal.reference).toMatch(/^sha256:/);expect((await poll()).body).toEqual(signal.body);
|
|
expect(inbox.body.items.map((item:{kind:string})=>item.kind)).toEqual(["authority_revoked"]);
|
|
expect(inbox.body.items[0].references.boundary).toEqual(hold.stopBoundary);
|
|
expect(JSON.stringify(inbox.body)).not.toContain("PRIVATE pending input canary");
|
|
expect(inspection.body).toMatchObject({boundaries:[{boundary:hold.stopBoundary,nativeEffectsUnknown:true}],externalStopConfirmed:false});
|
|
const workQueries=[{query:"identify"},{query:"assignment.read",assignmentId:f.assignment.id},{query:"input.pending",assignmentId:f.assignment.id,nativeRequestId},
|
|
{query:"task.list"},{query:"task.search",text:"Muse"},{query:"task.read",issueId:f.issue.id},{query:"task.history",issueId:f.issue.id},
|
|
{query:"task.document.read",issueId:f.issue.id,key:"memo"},{query:"operation.receipt",assignmentId:f.assignment.id,requestId:f.sent[0].requestId}];
|
|
for(const query of workQueries)expect((await request(app).post("/api/muse/v1/queries").set("Authorization",authorization).send({version:1,...query})).status).toBe(503);
|
|
const workCommands=[{command:"accept",assignmentId:f.assignment.id},{command:"progress",assignmentId:f.assignment.id,text:"no new effect"},
|
|
{command:"tool",assignmentId:f.assignment.id,name:"read_task",input:{}},{command:"finish",assignmentId:f.assignment.id,result:{schema:"paperclip.run_result.v1"}},
|
|
{command:"request_user_input",assignmentId:f.assignment.id,nativeRequestId:"new_question",questionSet:{schema:"paperclip.question_set.v1",questions:[{id:"answer",prompt:"No new question",required:true,answerMode:"text"}]}},
|
|
{command:"consume_input",assignmentId:f.assignment.id,nativeRequestId,inputDigest,continuationReceiptId:randomUUID(),continuationPersisted:true},
|
|
{command:"challenge.confirm",nonce:"unchanged-private-challenge"},{command:"work.request",issueId:f.issue.id},
|
|
{command:"task.comment",issueId:f.issue.id,body:"No idle effect"},{command:"task.create",title:"No idle task"}];
|
|
for(const command of workCommands)expect((await request(app).post("/api/muse/v1/commands").set("Authorization",authorization).send({version:1,requestId:randomUUID(),...command})).status).toBe(503);
|
|
await expect(f.broker.snapshot(f.company.id,f.agent.id,f.binding.id)).rejects.toThrow("disabled");
|
|
await expect(f.broker.act(f.subject,{version:1,command:"task.comment",issueId:f.issue.id,requestId:randomUUID(),body:"No direct idle effect"})).rejects.toThrow();
|
|
await expect(f.broker.assertRunAuthority({binding:{companyId:f.company.id,agentId:f.agent.id,runId:f.runId},provider:{binding:f.ref}} as Parameters<typeof f.broker.assertRunAuthority>[0])).rejects.toThrow();
|
|
const report={version:1,command:"worker.quiescent",boundary:hold.stopBoundary,requestId:randomUUID()};
|
|
for(let retry=0;retry<2;retry++)expect((await request(app).post("/api/muse/v1/cleanup").set("Authorization",authorization).send(report)).body).toEqual({status:"worker_reported",externalStopConfirmed:false,nativeEffectsUnknown:true});
|
|
for(let retry=0;retry<2;retry++)expect((await request(app).post("/api/muse/v1/queries").set("Authorization",authorization).send({version:1,query:"mailbox",after:stop.id})).body).toMatchObject({items:[],nextCursor:stop.id});
|
|
expect((await poll()).body).toEqual({version:1,signal:null});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:fence.payload});
|
|
const [saved]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,hold.id));
|
|
expect(saved).toMatchObject({stopBoundary:hold.stopBoundary,workerUnknown:true,nativeEffectsUnknown:true,releasedAt:null});expect(saved.workerReportedAt).toBeInstanceOf(Date);
|
|
expect(f.sent).toHaveLength(1);
|
|
const ordinary=await db.select().from(mailbox).where(eq(mailbox.bindingId,f.binding.id));
|
|
expect(ordinary.filter(item=>item.kind!=="authority_revoked").every(item=>item.signalNotifiedAt===null)).toBe(true);
|
|
expect((await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.action,"muse.worker_quiescence_reported"))))).toHaveLength(1);
|
|
}finally{await instanceSettingsService(db).updateExperimental({enableMuse:true,enableNativeRunner:true});await f.detach();}
|
|
});
|
|
it.each(["access_expired","grant_revoked","connection_revoked","refresh_expired","membership_lost","resource_changed","generation_changed","released","stop_window_expired"] as const)("disabled stop control retains current credential and boundary denial: %s",async fault=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth),app=transport();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
await instanceSettingsService(db).updateExperimental({enableMuse:false});
|
|
if(fault==="access_expired")await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthTokens.id,f.subject.credentialId));
|
|
if(fault==="connection_revoked")await f.broker.revoke(f.company.id,f.agent.id,f.operatorId);
|
|
if(fault==="grant_revoked")await db.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
if(fault==="refresh_expired")await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(and(eq(mcpOauthTokens.grantId,f.binding.grantId!),eq(mcpOauthTokens.kind,"refresh")));
|
|
if(fault==="membership_lost")await db.update(companyMemberships).set({status:"inactive"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));
|
|
if(fault==="resource_changed")await db.update(mcpOauthGrants).set({resource:"https://other.example/api/muse/v1"}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
if(fault==="generation_changed")await db.update(bindings).set({generation:2}).where(eq(bindings.id,f.binding.id));
|
|
if(fault==="released")await db.update(externalAgentHolds).set({releasedAt:new Date(),workerUnknown:false}).where(eq(externalAgentHolds.id,hold.id));
|
|
if(fault==="stop_window_expired")await db.update(mailbox).set({createdAt:new Date(Date.now()-24*60*60000-1)}).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"authority_revoked")));
|
|
const authorization="Bearer "+oauth.tokens!.access_token;
|
|
const expected=["generation_changed","released","stop_window_expired"].includes(fault)?403:401;
|
|
for(const command of [{command:"control.inspect"},{command:"worker.quiescent",boundary:hold.stopBoundary,requestId:randomUUID()}])expect((await request(app).post("/api/muse/v1/cleanup").set("Authorization",authorization).send({version:1,...command})).status).toBe(expected);
|
|
expect((await request(app).post("/api/muse/v1/queries").set("Authorization",authorization).send({version:1,query:"mailbox",after:0})).status).toBe(expected);
|
|
const [saved]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,hold.id));expect(saved.workerReportedAt).toBeNull();expect(saved.stopBoundary).toEqual(hold.stopBoundary);expect(f.sent).toHaveLength(1);
|
|
const signal=await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id);
|
|
if(["grant_revoked","connection_revoked","refresh_expired","membership_lost","resource_changed"].includes(fault))expect(signal.status).toBe(410);
|
|
else expect(signal.body).toEqual({version:1,signal:null});
|
|
}finally{await instanceSettingsService(db).updateExperimental({enableMuse:true,enableNativeRunner:true});await f.detach();}
|
|
});
|
|
it.each(["mailbox","report"] as const)("disabled stop control rechecks an access token that expires while its %s lock is held",async kind=>{
|
|
const oauth=await oauthFixture(true),f=await active(oauth),app=transport();let release=()=>{},holder:Promise<void>|undefined;try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});
|
|
const [hold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
await instanceSettingsService(db).updateExperimental({enableMuse:false});
|
|
let ready=()=>{},holderPid=0;const gate=new Promise<void>(resolve=>{release=resolve;}),locked=new Promise<void>(resolve=>{ready=resolve;});
|
|
holder=db.transaction(async tx=>{
|
|
holderPid=(await tx.execute<{pid:number}>(sql`select pg_backend_pid() as pid`))[0].pid;
|
|
const wait=async()=>{ready();await gate;};
|
|
if(kind==="report")await withExternalAdmissionGuard(tx,f.company.id,f.agent.id,wait);
|
|
else {await tx.select({id:bindings.id}).from(bindings).where(eq(bindings.id,f.binding.id)).for("update");await wait();}
|
|
});
|
|
await locked;let settled=false;
|
|
const body=kind==="mailbox"?{version:1,query:"mailbox",after:0}:{version:1,command:"worker.quiescent",boundary:hold.stopBoundary,requestId:randomUUID()};
|
|
const call=request(app).post(kind==="mailbox"?"/api/muse/v1/queries":"/api/muse/v1/cleanup").set("Authorization","Bearer "+oauth.tokens!.access_token).send(body).then(response=>{settled=true;return response;});
|
|
let waiting=false;const deadline=Date.now()+5000;
|
|
while(Date.now()<deadline&&!settled) {
|
|
const [observed]=await db.execute<{waiting:boolean}>(sql`select exists(select 1 from pg_stat_activity where datname=current_database() and wait_event_type='Lock' and ${holderPid} = any(pg_blocking_pids(pid)) and (query ilike '%muse_agent_bindings%' or query ilike '%pg_advisory_xact_lock%')) as waiting`);
|
|
if(observed.waiting){waiting=true;break;}await new Promise(resolve=>setTimeout(resolve,20));
|
|
}
|
|
expect(waiting).toBe(true);expect(settled).toBe(false);
|
|
await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthTokens.id,f.subject.credentialId));
|
|
release();await holder;expect((await call).status).toBe(401);
|
|
const [current]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));expect(current.workerCursor).toBe(0);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,hold.id)))[0].workerReportedAt).toBeNull();expect(f.sent).toHaveLength(1);
|
|
}finally{release();await holder;await instanceSettingsService(db).updateExperimental({enableMuse:true,enableNativeRunner:true});await f.detach();}
|
|
});
|
|
it.each(["enableMuse","enableNativeRunner"] as const)("disabled stop control requires a durable fence before using %s current credentials",async flag=>{
|
|
const f=await oauthFixture(true),app=transport();try {
|
|
await instanceSettingsService(db).updateExperimental({[flag]:false});
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id)).body).toEqual({version:1,signal:null});
|
|
expect((await request(app).post("/api/muse/v1/queries").set("Authorization","Bearer "+f.tokens!.access_token).send({version:1,query:"mailbox",after:0})).status).toBe(403);
|
|
expect((await request(app).post("/api/muse/v1/cleanup").set("Authorization","Bearer "+f.tokens!.access_token).send({version:1,command:"control.inspect"})).status).toBe(403);
|
|
}finally{await instanceSettingsService(db).updateExperimental({enableMuse:true,enableNativeRunner:true});}
|
|
});
|
|
it("disabled stop control cannot report or disclose another company's boundary",async()=>{
|
|
const firstOauth=await oauthFixture(true),secondOauth=await oauthFixture(true),first=await active(firstOauth),second=await active(secondOauth),app=transport();try {
|
|
for(const f of [first,second]){await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});await f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"authority_revoked",binding:f.ref}});}
|
|
const [firstHold]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,first.assignment.id));
|
|
await instanceSettingsService(db).updateExperimental({enableMuse:false});
|
|
const authorization="Bearer "+secondOauth.tokens!.access_token;
|
|
const inbox=await request(app).post("/api/muse/v1/queries").set("Authorization",authorization).send({version:1,query:"mailbox",after:0});expect(inbox.status).toBe(200);
|
|
expect(inbox.body.items.every((item:{kind:string;bindingId:string})=>item.kind==="authority_revoked"&&item.bindingId===second.binding.id)).toBe(true);
|
|
expect(JSON.stringify(inbox.body)).not.toContain(first.assignment.id);
|
|
const report=await request(app).post("/api/muse/v1/cleanup").set("Authorization",authorization).send({version:1,command:"worker.quiescent",boundary:firstHold.stopBoundary,requestId:randomUUID()});expect(report.status).toBe(409);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,firstHold.id)))[0].workerReportedAt).toBeNull();
|
|
}finally{await instanceSettingsService(db).updateExperimental({enableMuse:true,enableNativeRunner:true});await first.detach();await second.detach();}
|
|
});
|
|
|
|
it("keeps expired readiness quiet even after an acknowledged item becomes eligible for rearm",async()=>{
|
|
const f=await oauthFixture(true),app=transport();
|
|
const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
await f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});
|
|
const [challenge]=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge")));
|
|
await db.update(bindings).set({challengeExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));
|
|
const poll=()=>request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id);
|
|
expect((await poll()).body).toEqual({version:1,signal:null});
|
|
const ack=await request(app).post("/api/muse/v1/queries").set("Authorization","Bearer "+f.tokens!.access_token).send({version:1,query:"mailbox",after:challenge.id});
|
|
expect(ack.status).toBe(200);
|
|
await db.update(mailbox).set({signalNotifiedAt:new Date(Date.now()-60000)}).where(eq(mailbox.id,challenge.id));
|
|
expect((await poll()).body).toEqual({version:1,signal:null});
|
|
expect((await db.select().from(mailbox).where(eq(mailbox.id,challenge.id)))[0].signalAttempt).toBe(0);
|
|
});
|
|
it("signals only the current unread readiness nonce and keeps its opaque reference stable",async()=>{
|
|
const f=await oauthFixture(true),app=transport();
|
|
const verify=async()=>{const state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;await f.broker.verify(f.company.id,f.agent.id,{bindingId:f.binding.id,generation:1,expectedRevision:state.revision});};
|
|
await verify();await db.update(bindings).set({challengeExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));await verify();
|
|
const challenges=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge"))).orderBy(mailbox.id);
|
|
expect(challenges).toHaveLength(2);
|
|
const current=(await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0];
|
|
expect(museCredentialHash(String(challenges[1].references.nonce))).toBe(current.challengeHash);
|
|
const poll=()=>request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id);
|
|
const first=await poll(),second=await poll();expect(first.status).toBe(200);expect(first.body.signal?.reference).toMatch(/^sha256:/);expect(second.body).toEqual(first.body);
|
|
const saved=await db.select().from(mailbox).where(and(eq(mailbox.bindingId,f.binding.id),eq(mailbox.kind,"readiness_challenge"))).orderBy(mailbox.id);
|
|
expect(saved.map(item=>item.signalAttempt)).toEqual([0,1]);
|
|
const ack=await request(app).post("/api/muse/v1/queries").set("Authorization","Bearer "+f.tokens!.access_token).send({version:1,query:"mailbox",after:challenges[1].id});
|
|
expect(ack.status).toBe(200);expect((await poll()).body).toEqual({version:1,signal:null});
|
|
await db.update(bindings).set({challengeExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));await verify();expect((await poll()).body.signal?.reference).toMatch(/^sha256:/);
|
|
});
|
|
|
|
it("keeps paused and feature-disabled public receivers idle, resumes them, and fences lost current authority",async()=>{
|
|
const f=await active();try {
|
|
const app=transport(),poll=()=>request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id);
|
|
await db.update(agents).set({status:"paused"}).where(eq(agents.id,f.agent.id));expect((await poll()).body).toEqual({version:1,signal:null});
|
|
await db.update(agents).set({status:"active"}).where(eq(agents.id,f.agent.id));expect((await poll()).body.signal.reference).toMatch(/^sha256:/);
|
|
await instanceSettingsService(db).updateExperimental({enableMuse:false});expect((await poll()).body).toEqual({version:1,signal:null});
|
|
await instanceSettingsService(db).updateExperimental({enableMuse:true});expect((await poll()).body.signal.reference).toMatch(/^sha256:/);
|
|
await db.update(companyMemberships).set({status:"inactive"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));expect((await poll()).status).toBe(410);
|
|
}finally{await instanceSettingsService(db).updateExperimental({enableMuse:true});await f.detach();}
|
|
});
|
|
it("returns410 for expired refresh authority and thirty days of authenticated inactivity",async()=>{
|
|
const expired=await fixture(false),inactive=await fixture(false),app=transport();
|
|
await db.update(museCredentials).set({expiresAt:new Date(Date.now()-1)}).where(and(eq(museCredentials.bindingId,expired.binding.id),eq(museCredentials.kind,"refresh")));
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",expired.binding.id)).status).toBe(410);
|
|
await db.update(bindings).set({workerActivityAt:new Date(Date.now()-30*86400000)}).where(eq(bindings.id,inactive.binding.id));
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",inactive.binding.id)).status).toBe(410);
|
|
});
|
|
it("keeps transient authority-check failures retryable and records no failed public contact",async()=>{
|
|
const f=await fixture(false),app=transport(),check=vi.spyOn(museIdentityModule,"assertMuseReceiverAuthority").mockRejectedValueOnce(new Error("Synthetic database transport failure"));
|
|
try {
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id)).status).toBe(500);
|
|
await museReceiver(db).flush();expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].receiverContactAt).toBeNull();
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id)).status).toBe(200);
|
|
}finally{check.mockRestore();}
|
|
});
|
|
it("retains immutable asset1 through asset8 bytes and advertises finite asset9 routes",async()=>{
|
|
const app=transport(),manifests=await Promise.all([1,2,3,4,5,6,7,8,9].map(version=>request(app).get(`/api/muse/v1/assets/${version}/manifest.json`)));
|
|
expect(manifests[0].body.assets).toEqual({"client.py":{sha256:"8926534cce48e11d5ea157dfd4deb435c9f8d7d8ea58d3d8a5424b45ab31d1f7"},"detector.sh":{sha256:"09981df12ebce8c50b68e5da64edfe5ab5350d56bf6392ec52023c4990ca658d"},"instructions.md":{sha256:"d347f4d0e7a03bb0397dd4b2ddf38350c8b73ee3c0a91b50cd6c3cd84458a45f"}});
|
|
expect(manifests[1].body.assets).toEqual({"client.py":{sha256:"a89a90cf6ca6ae76be1662a9efda83f69dba3bccdd4c37cfe1935563b1f7330f"},"detector.sh":{sha256:"8e9f5de4f7dc18cbca7121fff6bcab4039dc66056441396826460d7f396c3b41"},"instructions.md":{sha256:"6770734ec4494a5d0403e5cc56ac69613faab96b66a96cf56108cec8dcda07a0"}});
|
|
expect(manifests[2].body.assets).toEqual({"client.py":{sha256:"ed171e60dac5d1ea92b5cce40323ad10d5c87448a926678e9e32aacdd5feca0f"},"detector.sh":{sha256:"8e9f5de4f7dc18cbca7121fff6bcab4039dc66056441396826460d7f396c3b41"},"instructions.md":{sha256:"2e7a4677b1b55ca391b62c7530736441c51e9f7ec9354ed50c776bc786bfa901"}});
|
|
expect(manifests[3].body.assets).toEqual({"client.py":{sha256:"42d9a1a9d795da54393bc9616e9e3ae6e9d9f6735d7728148e6080608aafdfc7"},"detector.sh":{sha256:"8e9f5de4f7dc18cbca7121fff6bcab4039dc66056441396826460d7f396c3b41"},"instructions.md":{sha256:"d57f1c43c0d3c7a76f9ed99e73a5a5cdb46c2b7e470efd451519bc3cc3d7d8f3"}});
|
|
expect(manifests[4].body.assets).toEqual({"client.py":{sha256:"55001af1ac3f5e2e904af1254180d4852025e41ed5c424c0814831a9537e50ca"},"detector.sh":{sha256:"1cbd1dd379987c7dadebbc466af312d72065e322d19c1ac2d8931ae2779885d7"},"instructions.md":{sha256:"46488fc1f50520e5f0a53e2a1cd4f944fcf4bd5cacad57de1d68d41d10904041"}});
|
|
expect(manifests[5].body.assets).toEqual({"client.py":{sha256:"cf207b0823bc59271d5d4ca930e8960eb9362b1244aea1a4e16eabc3889613bb"},"detector.sh":{sha256:"1cbd1dd379987c7dadebbc466af312d72065e322d19c1ac2d8931ae2779885d7"},"instructions.md":{sha256:"7957130682c5a8d2a1faf0e56e9505f548a2d95c294048e979c8499b698aa8d5"}});
|
|
expect(manifests[6].body.assets).toEqual({"client.py":{sha256:"bd3a70f650b16b7d28a67ec78e96a465741be1203e062e6680ebb6dc4d0db427"},"detector.sh":{sha256:"1cbd1dd379987c7dadebbc466af312d72065e322d19c1ac2d8931ae2779885d7"},"instructions.md":{sha256:"ad86ec766cd009fb11095407460f0e0362f0dd10749b94d51ebb1f3fd13bd357"}});
|
|
expect(manifests[7].body.assets).toEqual({"client.py":{sha256:"4eb06c9a0399f1cb94faba4f83ef945c509d39c6eb1c650cf6083e9c2363956a"},"detector.sh":{sha256:"1cbd1dd379987c7dadebbc466af312d72065e322d19c1ac2d8931ae2779885d7"},"instructions.md":{sha256:"fb76062f9f4179e39a317cf7f0635847996132195f3cbe7a5a071232b2fe0eb0"}});
|
|
for(const version of [1,2,3,4,5,6,7,8,9]){
|
|
const manifest=manifests[version-1];expect(manifest.status).toBe(200);expect(manifest.body.assetVersion).toBe(version);expect(manifest.body.clientVersion).toBe(String(version));expect(manifest.headers["cache-control"]).toContain("immutable");
|
|
for(const name of ["client.py","detector.sh","instructions.md"]){const response=await request(app).get(`/api/muse/v1/assets/${version}/${name}`);expect(response.status).toBe(200);expect(response.headers["cache-control"]).toContain("immutable");expect(createHash("sha256").update(response.text).digest("hex")).toBe(manifest.body.assets[name].sha256);}
|
|
}
|
|
expect(MUSE_PUBLIC_ROUTES).toHaveLength(48);
|
|
for(const path of ["/api/muse/v1/assets/10/client.py","/api/muse/v1/assets/6/extra.py","/api/muse/v1/assets/6/client.py?version=6"]){expect((await request(app).get(path)).status).toBe(404);}
|
|
expect((await request(app).post("/api/muse/v1/assets/4/client.py")).status).toBe(404);
|
|
expect((await request(transport(true)).post("/api/muse/v1/pair").send({version:1,ticket:"synthetic-unused-ticket",clientVersion:"1"})).status).toBe(400);
|
|
});
|
|
it("creates setup standing delegation from the actual scoped pairing company and agent",async()=>{
|
|
const f=await fixture(false),state=(await f.broker.bindingForAgent(f.company.id,f.agent.id))!;
|
|
const pairing=await f.broker.createPairing({companyId:f.company.id,agentId:f.agent.id,operatorId:f.operatorId,replaceBindingId:f.binding.id,expectedRevision:state.revision});
|
|
expect(pairing.setupInstruction).toContain(JSON.stringify({companyId:f.company.id,companyName:f.company.name,agentId:f.agent.id,agentName:f.agent.name,bindingId:pairing.bindingId}));
|
|
expect(pairing.setupInstruction).toContain("standing authorization");expect(pairing.setupInstruction).toContain("autonomously accept");expect(pairing.setupInstruction).toContain("native accepted lease before any task effect");
|
|
expect(pairing.setupInstruction).toContain(`--company-id '${f.company.id}' --company-name '${f.company.name}' --agent-id '${f.agent.id}' --agent-name '${f.agent.name}'`);
|
|
expect(pairing.setupInstruction).toContain(`/paperclip-muse-${pairing.bindingId}/SKILL.md`);
|
|
});
|
|
|
|
it("redeems Muse PKCE ticket into its exact agent/operator grant and keeps pre-token polling pending",async()=>{
|
|
const f=await oauthFixture(),redirect=new URL(f.authorizationUrl!);expect(redirect.origin+redirect.pathname).toBe(oauthCallback);expect(redirect.searchParams.get("state")).toBe(f.input.state);expect(redirect.searchParams.get("iss")).toBe(oauthOrigin+"/api/muse/v1/oauth");
|
|
const [grant]=await db.select().from(mcpOauthGrants).where(eq(mcpOauthGrants.id,f.binding.grantId!));expect(grant).toMatchObject({companyId:f.company.id,agentId:f.agent.id,userId:f.operatorId,purpose:"agent",resource:oauthResource,scopes:["paperclip:agent","offline_access"]});
|
|
expect(f.binding.ticketHash).toBeNull();expect(f.binding.pairedAt).toBeInstanceOf(Date);expect(await db.select().from(museCredentials).where(eq(museCredentials.bindingId,f.binding.id))).toHaveLength(0);
|
|
expect(await inspectMuseConnectionCredential(db,f.binding,oauthResource)).toEqual({state:"pending",refreshExpiresAt:null});
|
|
expect((await request(transport()).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id)).body).toEqual({version:1,signal:null});
|
|
const tokens=await f.oauth.token(f.tokenInput()),subject=await f.identity.authenticateOAuthAccess(tokens.access_token,oauthOrigin);
|
|
expect(subject).toMatchObject({credentialSource:"oauth",grantId:grant.id,provider:"muse",agentId:f.agent.id,authorizingUserId:f.operatorId});
|
|
const [stored]=await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.tokenHash,hashMcpSecret(tokens.access_token)));expect(subject.credentialId).toBe(stored.id);
|
|
expect((await inspectMuseConnectionCredential(db,(await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0],oauthResource)).state).toBe("active");
|
|
expect((await request(transport()).post("/api/muse/v1/queries").set("Authorization","Bearer "+tokens.access_token).send({version:1,query:"identify"})).body).toMatchObject({bindingId:f.binding.id,agentId:f.agent.id});
|
|
await expect(f.oauth.authorize(f.input)).rejects.toThrow();await expect(f.oauth.token(f.tokenInput())).rejects.toThrow();
|
|
});
|
|
it("preserves the first successful worker release across maintenance and later idle evidence",async()=>{
|
|
const f=await active();try {
|
|
await f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()});
|
|
const start=new Date(Date.now()-60000),at=(ms:number)=>new Date(start.getTime()+ms),releasedAt=at(-5000),workAt=at(-30000);
|
|
await db.update(assignments).set({status:"settled",acceptedResultAt:at(-10000),offeredAt:workAt,createdAt:workAt,claimedAt:workAt,nativeAcceptedAt:workAt,finalizedAt:at(-10000)}).where(eq(assignments.id,f.assignment.id));
|
|
await db.update(heartbeatRuns).set({status:"succeeded",startedAt:workAt,finishedAt:at(-10000)}).where(eq(heartbeatRuns.id,f.runId));
|
|
await db.update(externalAgentHolds).set({createdAt:workAt,releasedAt:null,workerUnknown:true,nativeEffectsUnknown:false}).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
vi.useFakeTimers({toFake:["Date"]});vi.setSystemTime(releasedAt);
|
|
await f.broker.reconcileTerminalAssignments(f.company.id,f.agent.id,f.binding.id);
|
|
const [first]=await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.assignmentId,f.assignment.id));
|
|
expect(first).toMatchObject({workerUnknown:false,nativeEffectsUnknown:false,releasedAt});
|
|
const qualificationId=randomUUID(),replicaId=randomUUID(),buckets=new Map<number,string[]>();
|
|
await db.update(bindings).set({qualificationId,qualificationStartedAt:start,qualificationExpiresAt:at(86400000)}).where(eq(bindings.id,f.binding.id));
|
|
for(let ms=0;ms<=50000;ms+=5000){const time=at(ms),bucket=Math.floor(time.getTime()/30000)*30000;const times=buckets.get(bucket)??[];times.push(time.toISOString());buckets.set(bucket,times);}
|
|
await db.insert(museReceiverContactBuckets).values([...buckets].map(([bucket,timestamps])=>({companyId:f.company.id,bindingId:f.binding.id,bindingGeneration:1,replicaId,bucketAt:new Date(bucket),firstAt:new Date(timestamps[0]),lastAt:new Date(timestamps.at(-1)!),timestamps,contacts:timestamps.length,gapsAtMostSevenSeconds:timestamps.length-1,maxGapMs:5000,incomplete:false})));
|
|
vi.setSystemTime(at(55000));
|
|
await f.broker.reconcileTerminalAssignments(f.company.id,f.agent.id,f.binding.id);
|
|
expect((await db.select().from(externalAgentHolds).where(eq(externalAgentHolds.id,first.id)))[0]).toEqual(first);
|
|
const evidence=await f.broker.readQualificationEvidence(f.company.id,f.agent.id,f.binding.id,qualificationId);
|
|
expect(evidence.idleWindows).toEqual([{startedAt:start.toISOString(),endedAt:at(50000).toISOString()}]);
|
|
}finally{vi.useRealTimers();await f.detach();}
|
|
});
|
|
|
|
describe("Muse OAuth code exchange diagnostics",()=>{
|
|
const prefix="muse.oauth_code_exchange_",generic={error:"invalid_grant",error_description:"Authorization is expired, revoked, or invalid."};
|
|
async function events(f:Awaited<ReturnType<typeof oauthFixture>>,rejectionsOnly=true) {
|
|
const rows=await db.select().from(activityLog).where(and(eq(activityLog.companyId,f.company.id),eq(activityLog.entityType,"agent"),eq(activityLog.entityId,f.agent.id)));
|
|
return rows.filter(row=>row.action.startsWith(prefix+(rejectionsOnly?"rejected_":"")));
|
|
}
|
|
async function codeState(f:Awaited<ReturnType<typeof oauthFixture>>) {
|
|
const [code]=await db.select().from(mcpOauthRequests).where(eq(mcpOauthRequests.grantId,f.binding.grantId!));return code;
|
|
}
|
|
it.each(["code_expired","redirect_mismatch","pkce_mismatch","client_mismatch","code_consumed"] as const)("records only scoped static %s evidence while preserving the public error",async(check)=>{
|
|
const f=await oauthFixture(),input=f.tokenInput();
|
|
if(check==="code_expired")await db.update(mcpOauthRequests).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthRequests.grantId,f.binding.grantId!));
|
|
if(check==="redirect_mismatch")input.redirect_uri="https://other-authd.example/callback";
|
|
if(check==="pkce_mismatch")input.code_verifier="a".repeat(43);
|
|
if(check==="client_mismatch")input.client_id="pcmcp_client_wrong";
|
|
if(check==="code_consumed")await f.oauth.token(f.tokenInput());
|
|
const response=await request(transport()).post("/api/muse/v1/oauth/token").type("form").send(input);
|
|
expect(response.status).toBe(400);expect(response.body).toEqual(generic);
|
|
const rows=await events(f);expect(rows).toHaveLength(1);
|
|
expect(rows[0]).toMatchObject({actorType:"system",actorId:"muse-oauth",responsibleUserId:f.operatorId,agentId:f.agent.id,action:prefix+"rejected_"+check});
|
|
expect(rows[0].details).toMatchObject({schemaVersion:1,bindingId:f.binding.id,generation:f.binding.generation,grantId:f.binding.grantId,failedChecks:[check]});
|
|
const details=rows[0].details!;
|
|
for(const field of ["receivedAt","decisionAt","authorizationAt","deadlineAt"])expect(Number.isFinite(Date.parse(String(details[field])))).toBe(true);
|
|
expect(details.decisionLatencyMs).toBeGreaterThanOrEqual(0);expect(details.codeAgeMs).toBeTypeOf("number");
|
|
expect((await codeState(f)).consumedAt===null).toBe(check!=="code_consumed");
|
|
expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(check==="code_consumed"?2:0);
|
|
});
|
|
it.each(["wrong_resource","missing_client","both"] as const)("records known-code early invalid_target evidence for %s without changing public errors",async(kind)=>{
|
|
const f=await oauthFixture(),input=f.tokenInput(),resource="https://resource-canary.example/unsupported",claimedCompany=randomUUID();
|
|
const submitted:Record<string,unknown>={...input},failedChecks:string[]=[];
|
|
if(kind!=="wrong_resource"){delete submitted.client_id;failedChecks.push("client_mismatch");}
|
|
if(kind!=="missing_client"){submitted.resource=resource;failedChecks.push("resource_mismatch");}
|
|
const expected={error:"invalid_target",error_description:"Client and matching resource are required."};
|
|
const unknown=await request(transport()).post("/api/muse/v1/oauth/token").type("form").send({...submitted,code:"pcmcp_code_unknown_resource_canary",company_id:claimedCompany});
|
|
expect(unknown.status).toBe(400);expect(unknown.body).toEqual(expected);expect(await events(f)).toHaveLength(0);
|
|
const response=await request(transport()).post("/api/muse/v1/oauth/token").type("form").set("X-Private-Canary","resource-header-canary")
|
|
.send({...submitted,company_id:claimedCompany,reason:"resource-client-reason-canary"});
|
|
expect(response.status).toBe(400);expect(response.body).toEqual(expected);
|
|
const rows=await events(f);expect(rows).toHaveLength(failedChecks.length);expect(rows.map(row=>row.action).sort()).toEqual(failedChecks.map(check=>prefix+"rejected_"+check).sort());
|
|
for(const row of rows){expect(row).toMatchObject({actorType:"system",actorId:"muse-oauth",responsibleUserId:f.operatorId,entityType:"agent",entityId:f.agent.id,agentId:f.agent.id});expect(row.details).toMatchObject({bindingId:f.binding.id,generation:f.binding.generation,grantId:f.binding.grantId,failedChecks});}
|
|
const serialized=JSON.stringify(rows),code=await codeState(f);
|
|
for(const value of [resource,claimedCompany,input.code,input.code_verifier,f.client.client_id,f.input.code_challenge,f.input.state,oauthCallback,code.id,code.codeHash,"resource-header-canary","resource-client-reason-canary"])expect(serialized).not.toContain(value);
|
|
expect(code.consumedAt).toBeNull();expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(0);
|
|
await expect(f.oauth.token(input)).resolves.toMatchObject({token_type:"Bearer"});
|
|
});
|
|
it.each(["operator_removed","operator_viewer","agent_removed"] as const)("normalizes expected %s code authority loss to the scoped invalid_grant error",async(kind)=>{
|
|
const f=await oauthFixture();
|
|
const user=and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalType,"user"),eq(companyMemberships.principalId,f.operatorId));
|
|
if(kind==="operator_removed")await db.delete(companyMemberships).where(user);
|
|
else if(kind==="operator_viewer")await db.update(companyMemberships).set({membershipRole:"viewer"}).where(user);
|
|
else await db.delete(companyMemberships).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalType,"agent"),eq(companyMemberships.principalId,f.agent.id)));
|
|
const response=await request(transport()).post("/api/muse/v1/oauth/token").type("form").send(f.tokenInput());
|
|
expect(response.status).toBe(400);expect(response.body).toEqual(generic);
|
|
const rows=await events(f);expect(rows).toHaveLength(1);expect(rows[0]).toMatchObject({action:prefix+"rejected_current_authority_unavailable",responsibleUserId:f.operatorId,entityId:f.agent.id});
|
|
expect((await codeState(f)).consumedAt).toBeNull();expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(0);
|
|
});
|
|
it("normalizes expected refresh authority loss without consuming its credential",async()=>{
|
|
const f=await oauthFixture(true);
|
|
await db.delete(companyMemberships).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalType,"agent"),eq(companyMemberships.principalId,f.agent.id)));
|
|
const response=await request(transport()).post("/api/muse/v1/oauth/token").type("form").send({grant_type:"refresh_token",client_id:f.client.client_id,refresh_token:f.tokens!.refresh_token});
|
|
expect(response.status).toBe(400);expect(response.body).toEqual(generic);
|
|
const tokens=await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!));
|
|
expect(tokens).toHaveLength(2);expect(tokens.find(token=>token.kind==="refresh")?.usedAt).toBeNull();
|
|
});
|
|
it("preserves unexpected authority storage failures without inventing a protocol denial",async()=>{
|
|
const f=await oauthFixture(),fault=new Error("Synthetic authority storage failure");
|
|
const spy=vi.spyOn(museIdentityModule,"assertMuseCurrentAuthority").mockRejectedValueOnce(fault);
|
|
try{await expect(f.oauth.token(f.tokenInput())).rejects.toBe(fault);}finally{spy.mockRestore();}
|
|
expect((await codeState(f)).consumedAt).toBeNull();expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(0);expect(await events(f)).toHaveLength(0);
|
|
});
|
|
it("retains overlapping failed checks without claiming a single cause",async()=>{
|
|
const f=await oauthFixture();await db.update(mcpOauthRequests).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthRequests.grantId,f.binding.grantId!));
|
|
await expect(f.oauth.token({...f.tokenInput(),redirect_uri:oauthCallback+"?changed=1",code_verifier:"z".repeat(43)})).rejects.toMatchObject({code:"invalid_grant",status:400});
|
|
const rows=await events(f);expect(rows.map(row=>row.action).sort()).toEqual(["code_expired","pkce_mismatch","redirect_mismatch"].map(check=>prefix+"rejected_"+check).sort());
|
|
for(const row of rows)expect(row.details?.failedChecks).toEqual(["redirect_mismatch","pkce_mismatch","code_expired"]);
|
|
});
|
|
it("attributes a malformed verifier only from a known code and excludes all private request material",async()=>{
|
|
const f=await oauthFixture(),input=f.tokenInput(),code=await codeState(f);
|
|
const response=await request(transport()).post("/api/muse/v1/oauth/token").type("form").set("X-Private-Canary","header-private-canary")
|
|
.send({...input,code_verifier:"malformed-verifier-canary",reason:"client-chosen-reason-canary",client_secret:"body-private-canary"});
|
|
expect(response.status).toBe(400);expect(response.body).toEqual(generic);
|
|
const rows=await events(f);expect(rows).toHaveLength(1);expect(rows[0].action).toBe(prefix+"rejected_verifier_format");
|
|
expect(Object.keys(rows[0].details!).sort()).toEqual(["schemaVersion","bindingId","generation","grantId","failedChecks","receivedAt","decisionAt","authorizationAt","deadlineAt","codeAgeMs","decisionLatencyMs"].sort());
|
|
const serialized=JSON.stringify(rows);
|
|
for(const value of [input.code,input.code_verifier,f.verifier,f.input.code_challenge,f.input.state,f.ticket,f.client.client_id,oauthCallback,code.id,code.codeHash,"header-private-canary","malformed-verifier-canary","client-chosen-reason-canary","body-private-canary"])expect(serialized).not.toContain(value);
|
|
expect((await codeState(f)).consumedAt).toBeNull();
|
|
});
|
|
it("commits independent S256 proof success and its timing evidence with the credential family",async()=>{
|
|
const f=await oauthFixture(false,false),verifier="dBjftJeZ4CVP-mB92K27uhbUJU1p1r_wW1gFWFOEjXk";
|
|
const redirect=await f.oauth.authorize({...f.input,code_challenge:"E9Melhoa2OwvFrEMTJguCHaoeK1t8URWbuGJSstw-cM"});
|
|
const code=new URL(redirect).searchParams.get("code")!;
|
|
const response=await request(transport()).post("/api/muse/v1/oauth/token").set("Content-Type","application/x-www-form-urlencoded; charset=UTF-8")
|
|
.send({grant_type:"authorization_code",client_id:f.client.client_id,redirect_uri:oauthCallback,code,code_verifier:verifier});
|
|
expect(response.status).toBe(200);expect(response.body.token_type).toBe("Bearer");
|
|
const [binding]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));
|
|
const rows=await events(f,false);expect(rows).toHaveLength(1);expect(rows[0].action).toBe(prefix+"succeeded");expect(rows[0].responsibleUserId).toBe(f.operatorId);
|
|
expect((await db.select().from(mcpOauthRequests).where(eq(mcpOauthRequests.grantId,binding.grantId!)))[0].consumedAt).toBeInstanceOf(Date);
|
|
expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,binding.grantId!))).toHaveLength(2);
|
|
expect((await inspectMuseConnectionCredential(db,binding,oauthResource)).state).toBe("active");
|
|
});
|
|
it("rolls back consumption and tokens if atomic success evidence cannot persist",async()=>{
|
|
const f=await oauthFixture(),original=activityLogModule.logActivity,fault=new Error("synthetic diagnostic storage failure");
|
|
const spy=vi.spyOn(activityLogModule,"logActivity").mockImplementation(async(...args)=>{if(args[1].action===prefix+"succeeded")throw fault;return original(...args);});
|
|
try {await expect(f.oauth.token(f.tokenInput())).rejects.toBe(fault);}finally{spy.mockRestore();}
|
|
expect((await codeState(f)).consumedAt).toBeNull();expect(await events(f,false)).toHaveLength(0);
|
|
expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(0);
|
|
await expect(f.oauth.token(f.tokenInput())).resolves.toMatchObject({token_type:"Bearer"});expect(await events(f,false)).toHaveLength(1);
|
|
});
|
|
it("keeps a committed exchange successful when postcommit publication fails",async()=>{
|
|
const f=await oauthFixture(),original=activityLogModule.publishActivity;
|
|
const spy=vi.spyOn(activityLogModule,"publishActivity").mockImplementation(publication=>{if(publication.payload.action===prefix+"succeeded")throw new Error("synthetic publication failure");return original(publication);});
|
|
try {const response=await request(transport()).post("/api/muse/v1/oauth/token").type("form").send(f.tokenInput());expect(response.status).toBe(200);}finally{spy.mockRestore();}
|
|
expect((await codeState(f)).consumedAt).toBeInstanceOf(Date);expect(await events(f,false)).toHaveLength(1);
|
|
expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(2);
|
|
});
|
|
it("preserves the original public rejection and rolls back partial diagnostic writes on audit failure",async()=>{
|
|
const f=await oauthFixture(),original=activityLogModule.logActivity;
|
|
await db.update(mcpOauthRequests).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthRequests.grantId,f.binding.grantId!));
|
|
const spy=vi.spyOn(activityLogModule,"logActivity").mockImplementation(async(...args)=>{if(args[1].action===prefix+"rejected_code_expired")throw new Error("synthetic private audit error");return original(...args);});
|
|
try {const response=await request(transport()).post("/api/muse/v1/oauth/token").type("form").send({...f.tokenInput(),code_verifier:"z".repeat(43)});expect(response.status).toBe(400);expect(response.body).toEqual(generic);}finally{spy.mockRestore();}
|
|
expect(await events(f)).toHaveLength(0);expect((await codeState(f)).consumedAt).toBeNull();
|
|
expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(0);
|
|
await expect(f.oauth.token({...f.tokenInput(),code_verifier:"z".repeat(43)})).rejects.toMatchObject({code:"invalid_grant"});expect(await events(f)).toHaveLength(2);
|
|
});
|
|
it("does not let a public Board activity forgery suppress trusted first-denial evidence",async()=>{
|
|
const f=await oauthFixture(),app=express();app.use(express.json());
|
|
app.use((req,_res,next)=>{req.actor={type:"board",source:"session",userId:f.operatorId,companyIds:[f.company.id],isInstanceAdmin:false};next();});
|
|
app.use("/api",activityRoutes(db));app.use(errorHandler);
|
|
const action=prefix+"rejected_pkce_mismatch",details={bindingId:f.binding.id,generation:f.binding.generation,grantId:f.binding.grantId};
|
|
const forged=await request(app).post(`/api/companies/${f.company.id}/activity`).send({actorType:"system",actorId:"muse-oauth",action,entityType:"agent",entityId:f.agent.id,agentId:f.agent.id,responsibleUserId:f.operatorId,details});
|
|
expect(forged.status).toBe(201);expect(forged.body.responsibleUserId).toBeNull();
|
|
await expect(f.oauth.token({...f.tokenInput(),code_verifier:"z".repeat(43)})).rejects.toMatchObject({code:"invalid_grant"});
|
|
const rows=await events(f);expect(rows).toHaveLength(2);expect(rows.filter(row=>row.responsibleUserId===f.operatorId)).toHaveLength(1);
|
|
});
|
|
it("caps concurrent repeated denials and preserves the first recorded timing",async()=>{
|
|
const f=await oauthFixture(),input={...f.tokenInput(),code_verifier:"z".repeat(43)};
|
|
const results=await Promise.allSettled(Array.from({length:4},()=>f.oauth.token(input)));expect(results.every(result=>result.status==="rejected")).toBe(true);
|
|
const first=await events(f);expect(first).toHaveLength(1);
|
|
await expect(f.oauth.token(input)).rejects.toMatchObject({code:"invalid_grant"});expect(await events(f)).toEqual(first);
|
|
expect((await codeState(f)).consumedAt).toBeNull();
|
|
await expect(f.oauth.token(f.tokenInput())).resolves.toMatchObject({token_type:"Bearer"});
|
|
});
|
|
it("does not attribute unknown or pruned codes from claimed client or company fields",async()=>{
|
|
const f=await oauthFixture(),other=await oauthFixture();
|
|
await expect(f.oauth.token({...f.tokenInput(),code:"pcmcp_code_unknown_canary",company_id:other.company.id,agent_id:other.agent.id,reason:"code_expired"})).rejects.toMatchObject({code:"invalid_grant"});
|
|
expect(await events(f)).toHaveLength(0);expect(await events(other)).toHaveLength(0);
|
|
await db.update(mcpOauthRequests).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthRequests.grantId,f.binding.grantId!));
|
|
await f.oauth.register({client_name:"Prune control",redirect_uris:[oauthCallback]},randomUUID());
|
|
expect(await codeState(f)).toBeUndefined();await expect(f.oauth.token(f.tokenInput())).rejects.toMatchObject({code:"invalid_grant"});
|
|
expect(await events(f)).toHaveLength(0);
|
|
await expect(other.oauth.token(other.tokenInput())).resolves.toMatchObject({token_type:"Bearer"});
|
|
});
|
|
it("retains historical scope when revocation and replacement occur during the admission wait",async()=>{
|
|
const f=await oauthFixture();let operation:Promise<{error?:unknown}>|undefined;
|
|
try {await db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,async()=>{
|
|
await tx.select().from(bindings).where(eq(bindings.id,f.binding.id)).for("update");
|
|
operation=f.oauth.token(f.tokenInput()).then(()=>({}),error=>({error}));
|
|
await expect.poll(async()=>{const result=await db.execute(sql`select count(*)::int as count from pg_stat_activity where datname=current_database() and lower(wait_event)='advisory'`);return Number(result[0]?.count??0);},{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
await tx.update(bindings).set({revokedAt:new Date(),status:"revoked"}).where(eq(bindings.id,f.binding.id));
|
|
const [replacement]=await tx.insert(bindings).values({companyId:f.company.id,agentId:f.agent.id,operatorId:f.operatorId,generation:f.binding.generation+1}).returning();
|
|
await tx.update(agents).set({adapterConfig:{...f.agent.adapterConfig,museBindingId:replacement.id}}).where(eq(agents.id,f.agent.id));
|
|
}));}finally{await operation;}
|
|
expect((await operation!).error).toMatchObject({code:"invalid_grant"});const rows=await events(f);expect(rows).toHaveLength(1);
|
|
expect(rows[0].details).toMatchObject({bindingId:f.binding.id,generation:f.binding.generation,grantId:f.binding.grantId,failedChecks:["binding_unavailable"]});
|
|
expect((await codeState(f)).consumedAt).toBeNull();expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(0);
|
|
});
|
|
it("records simulated Date-only expiry after a proven admission wait without changing the deadline",async()=>{
|
|
const f=await oauthFixture(),code=await codeState(f),enteredAt=Date.now();let operation:Promise<{error?:unknown}>|undefined;
|
|
vi.useFakeTimers({toFake:["Date"]});vi.setSystemTime(enteredAt);
|
|
try {
|
|
await db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,async()=>{
|
|
await tx.select().from(bindings).where(eq(bindings.id,f.binding.id)).for("update");operation=f.oauth.token(f.tokenInput()).then(()=>({}),error=>({error}));
|
|
await expect.poll(async()=>{const result=await db.execute(sql`select count(*)::int as count from pg_stat_activity where datname=current_database() and lower(wait_event)='advisory'`);return Number(result[0]?.count??0);},{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
vi.setSystemTime(code.expiresAt.getTime()+1);
|
|
}));
|
|
expect((await operation!).error).toMatchObject({code:"invalid_grant"});
|
|
} finally {await operation;vi.useRealTimers();}
|
|
const rows=await events(f);expect(rows).toHaveLength(1);const details=rows[0].details!;
|
|
expect(Date.parse(String(details.receivedAt))).toBeLessThan(code.expiresAt.getTime());expect(Date.parse(String(details.decisionAt))).toBeGreaterThan(code.expiresAt.getTime());
|
|
expect(details.deadlineAt).toBe(code.expiresAt.toISOString());expect(details.failedChecks).toEqual(["code_expired"]);
|
|
expect((await codeState(f)).consumedAt).toBeNull();expect(await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.grantId,f.binding.grantId!))).toHaveLength(0);
|
|
});
|
|
});
|
|
it("enforces registered redirects,S256,current authority and ticket expiry before Muse redemption",async()=>{
|
|
const f=await oauthFixture(false,false);
|
|
for(const extra of [{redirect_uri:"https://attacker.example/callback"},{code_challenge_method:"plain"},{resource:oauthOrigin+"/mcp/paperclip"},{scope:"paperclip:agent"},{company_id:randomUUID()}])await expect(f.oauth.authorize({...f.input,...extra})).rejects.toThrow();
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].ticketHash).not.toBeNull();
|
|
await db.update(companyMemberships).set({status:"inactive"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));await expect(f.oauth.authorize(f.input)).rejects.toThrow();
|
|
await db.update(companyMemberships).set({status:"active"}).where(and(eq(companyMemberships.companyId,f.company.id),eq(companyMemberships.principalId,f.operatorId)));
|
|
await db.update(bindings).set({ticketExpiresAt:new Date(Date.now()-1)}).where(eq(bindings.id,f.binding.id));await expect(f.oauth.authorize(f.input)).rejects.toThrow();
|
|
});
|
|
it("keeps only one Muse OAuth grant when ticket authorization races and rejects expired PKCE codes",async()=>{
|
|
const f=await oauthFixture(false,false),results=await Promise.allSettled([f.oauth.authorize(f.input),f.oauth.authorize(f.input)]);expect(results.filter(r=>r.status==="fulfilled")).toHaveLength(1);
|
|
const [binding]=await db.select().from(bindings).where(eq(bindings.id,f.binding.id));expect(await db.select().from(mcpOauthGrants).where(eq(mcpOauthGrants.agentId,f.agent.id))).toHaveLength(1);
|
|
await db.update(mcpOauthRequests).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthRequests.grantId,binding.grantId!));
|
|
expect((await request(transport()).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id)).status).toBe(410);
|
|
});
|
|
it("rotates finite Muse OAuth refresh,defaults absent resource only here,and fences replay without cleanup exception",async()=>{
|
|
const f=await oauthFixture(true),original=f.tokens!;
|
|
const [before]=await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.tokenHash,hashMcpSecret(original.refresh_token!)));expect(before.expiresAt).toBeInstanceOf(Date);expect(before.expiresAt!.getTime()-Date.now()).toBeLessThanOrEqual(30*86400000);
|
|
const rotated=await f.oauth.token({grant_type:"refresh_token",client_id:f.client.client_id,refresh_token:original.refresh_token});expect(rotated.refresh_token).not.toBe(original.refresh_token);
|
|
await expect(f.oauth.token({grant_type:"refresh_token",client_id:f.client.client_id,refresh_token:rotated.refresh_token,resource:oauthOrigin+"/mcp/runner"})).rejects.toThrow();
|
|
await expect(f.oauth.token({grant_type:"refresh_token",client_id:f.client.client_id,refresh_token:original.refresh_token})).rejects.toThrow();
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].revokedAt).toBeInstanceOf(Date);
|
|
for(const path of ["queries","cleanup","detector-cleanup"]){const body=path==="queries"?{version:1,query:"identify"}:path==="cleanup"?{version:1,command:"control.inspect"}:{version:1,requestId:randomUUID(),bindingId:f.binding.id,generation:1,detectorRemoved:true};expect((await request(transport()).post("/api/muse/v1/"+path).set("Authorization","Bearer "+rotated.access_token).send(body)).status).toBe(401);}
|
|
});
|
|
it("keeps public OAuth contact from extending authenticated inactivity and denies wrong resource/offline scope",async()=>{
|
|
const f=await oauthFixture(true),[refresh]=await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.tokenHash,hashMcpSecret(f.tokens!.refresh_token!)));
|
|
await museReceiver(db).signal(f.binding.id,oauthResource);await museReceiver(db).flush();expect((await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.id,refresh.id)))[0].expiresAt).toEqual(refresh.expiresAt);
|
|
await db.update(mcpOauthGrants).set({resource:"https://another.example/api/muse/v1"}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
expect((await inspectMuseConnectionCredential(db,f.binding,oauthResource)).state).toBe("revoked");await expect(assertMuseSubjectCredential(db,f.subject,oauthResource)).rejects.toThrow();
|
|
await db.update(mcpOauthGrants).set({resource:oauthResource,scopes:["paperclip:agent"]}).where(eq(mcpOauthGrants.id,f.binding.grantId!));expect((await inspectMuseConnectionCredential(db,f.binding,oauthResource)).state).toBe("revoked");
|
|
await db.update(mcpOauthGrants).set({scopes:["paperclip:agent","offline_access"]}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthTokens.id,refresh.id));expect((await inspectMuseConnectionCredential(db,f.binding,oauthResource)).state).toBe("expired");
|
|
expect((await request(transport()).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id)).status).toBe(410);
|
|
});
|
|
it("serves PKCE OAuth ingress and disables raw HTTPlegacy pairing by default",async()=>{
|
|
const f=await oauthFixture(false,false),app=transport();
|
|
expect((await request(app).get("/api/muse/v1/oauth/metadata")).body).toMatchObject({issuer:oauthOrigin+"/api/muse/v1/oauth",token_endpoint_auth_methods_supported:["none"],code_challenge_methods_supported:["S256"]});
|
|
expect((await request(app).post("/api/muse/v1/pair").send({version:1,ticket:f.ticket,clientVersion:"2"})).status).toBe(404);
|
|
expect((await request(app).post("/api/muse/v1/refresh").send({version:1,refreshToken:f.credentials.refreshToken})).status).toBe(404);
|
|
const reply=await request(app).get("/api/muse/v1/oauth/authorize").query(f.input);expect(reply.status).toBe(303);expect(reply.headers["referrer-policy"]).toBe("no-referrer");
|
|
const code=new URL(reply.headers.location).searchParams.get("code")!;
|
|
const exchange=await request(app).post("/api/muse/v1/oauth/token").type("form").send({grant_type:"authorization_code",client_id:f.client.client_id,redirect_uri:oauthCallback,code,code_verifier:f.verifier});expect(exchange.status).toBe(200);expect(exchange.body.token_type).toBe("Bearer");
|
|
expect((await request(app).post("/api/muse/v1/oauth/revoke").type("form").send({client_id:f.client.client_id,token:exchange.body.access_token})).status).toBe(200);
|
|
expect((await request(app).get("/api/muse/v1/signal").set("X-Paperclip-Muse-Binding",f.binding.id)).status).toBe(410);
|
|
});
|
|
it.each(["access_expired","grant_revoked"])("rechecks idle OAuth authority after waiting for its transaction lock:%s",async(reason)=>{
|
|
const f=await oauthFixture(true),requestId=randomUUID(),title="Must not create "+randomUUID();let outcome:Promise<{value?:unknown;error?:unknown}>|undefined;
|
|
await db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,async()=>{
|
|
await tx.select().from(bindings).where(eq(bindings.id,f.binding.id)).for("update");
|
|
outcome=f.broker.act(f.subject,{version:1,command:"task.create",requestId,title}).then(value=>({value}),error=>({error}));
|
|
await expect.poll(async()=>{const result=await db.execute(sql`select count(*)::int as count from pg_stat_activity where datname=current_database() and lower(wait_event)='advisory'`);return Number(result[0]?.count??0);},{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
if(reason==="access_expired")await tx.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(eq(mcpOauthTokens.id,f.subject.credentialId));
|
|
else await tx.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
}));
|
|
expect((await outcome!).error).toBeDefined();expect(await db.select().from(issues).where(eq(issues.title,title))).toHaveLength(0);expect(await db.select().from(museIdleReceipts).where(eq(museIdleReceipts.requestId,requestId))).toHaveLength(0);
|
|
});
|
|
it.each(["code","refresh","revoke"])("serializes Muse OAuth%s behind the broker admission/binding prefix without grant/token lock inversion",async(kind)=>{
|
|
const f=await oauthFixture(kind!=="code");let operation:Promise<{value?:unknown;error?:unknown}>|undefined;
|
|
await db.transaction(tx=>withExternalAdmissionGuard(tx,f.company.id,f.agent.id,async()=>{
|
|
await tx.select().from(bindings).where(eq(bindings.id,f.binding.id)).for("update");
|
|
operation=(kind==="code"?f.oauth.token(f.tokenInput()):kind==="refresh"?f.oauth.token({grant_type:"refresh_token",client_id:f.client.client_id,refresh_token:f.tokens!.refresh_token}):f.oauth.revokeToken(f.tokens!.access_token,f.client.client_id)).then(value=>({value}),error=>({error}));
|
|
await expect.poll(async()=>{const result=await db.execute(sql`select count(*)::int as count from pg_stat_activity where datname=current_database() and lower(wait_event)='advisory'`);return Number(result[0]?.count??0);},{timeout:5000,interval:25}).toBeGreaterThan(0);
|
|
// Before the repair,token/code acquired this grant before waiting on binding.
|
|
await tx.execute(sql`set local lock_timeout='1s'`);await tx.select().from(mcpOauthGrants).where(eq(mcpOauthGrants.id,f.binding.grantId!)).for("update");
|
|
await tx.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
}));
|
|
const result=await operation!;if(kind!=="revoke")expect(result.error).toBeDefined();else expect(result.error).toBeUndefined();
|
|
});
|
|
it("withholds fresh native dispatch when OAuthgrant is revoked before the binding fence or its refresh family expires",async()=>{
|
|
const base=await oauthFixture(true),f=await active(base);try {
|
|
await db.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,base.binding.grantId!));
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].revokedAt).toBeNull();
|
|
await expect(f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).rejects.toThrow();expect(f.sent).toHaveLength(0);
|
|
await expect(f.port.dispatch({sourceEventId:randomUUID(),payload:{kind:"assignment",binding:f.ref,tools:[],acceptByUnixMs:Date.now()+600000,expiresAtUnixMs:Date.now()+3600000}})).rejects.toThrow();
|
|
await db.update(mcpOauthGrants).set({revokedAt:null}).where(eq(mcpOauthGrants.id,base.binding.grantId!));await db.update(mcpOauthTokens).set({expiresAt:new Date(Date.now()-1)}).where(and(eq(mcpOauthTokens.grantId,base.binding.grantId!),eq(mcpOauthTokens.kind,"refresh")));
|
|
await expect(f.broker.act(f.subject,{version:1,command:"accept",assignmentId:f.assignment.id,requestId:randomUUID()})).rejects.toThrow();expect(f.sent).toHaveLength(0);
|
|
}finally{await f.detach();}
|
|
});
|
|
it("revokes a persisted Muse grant through generic connection management after a public-origin change",async()=>{
|
|
const f=await oauthFixture(true),otherOrigin="https://new.paperclip.example",management=createPublicMcpOAuth(db,{origin:otherOrigin,resource:otherOrigin+"/mcp/paperclip"});
|
|
await instanceSettingsService(db).updateExperimental({enablePublicMcp:true});
|
|
await management.revokeConnection(f.binding.grantId!,f.operatorId);
|
|
expect((await db.select().from(mcpOauthGrants).where(eq(mcpOauthGrants.id,f.binding.grantId!)))[0].revokedAt).toBeInstanceOf(Date);
|
|
expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].revokedAt).toBeInstanceOf(Date);
|
|
expect(await db.select().from(dotAgentBindings).where(eq(dotAgentBindings.agentId,f.agent.id))).toHaveLength(0);
|
|
});
|
|
it("reports installed client version only after a currently authenticated identify declaration",async()=>{
|
|
const f=await oauthFixture();expect(f.binding.clientVersion).toBeNull();const tokens=await f.oauth.token(f.tokenInput()),app=transport();
|
|
const identify=(clientVersion?:string)=>request(app).post("/api/muse/v1/queries").set("Authorization","Bearer "+tokens.access_token).send({version:1,query:"identify",...(clientVersion?{clientVersion}:{})});
|
|
expect((await identify("3")).status).toBe(200);expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].clientVersion).toBe("3");
|
|
expect((await identify()).status).toBe(200);expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].clientVersion).toBe("3");
|
|
expect((await identify("4")).status).toBe(200);expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].clientVersion).toBe("4");
|
|
expect((await identify()).status).toBe(200);expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].clientVersion).toBe("4");
|
|
expect((await identify("5")).status).toBe(200);expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].clientVersion).toBe("5");
|
|
expect((await identify(MUSE_CLIENT_VERSION)).status).toBe(200);expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].clientVersion).toBe(MUSE_CLIENT_VERSION);
|
|
expect((await identify()).status).toBe(200);expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].clientVersion).toBe(MUSE_CLIENT_VERSION);
|
|
expect((await identify(String(Number(MUSE_CLIENT_VERSION)+1))).status).toBe(400);
|
|
await db.update(bindings).set({clientVersion:"2"}).where(eq(bindings.id,f.binding.id));await db.update(mcpOauthGrants).set({revokedAt:new Date()}).where(eq(mcpOauthGrants.id,f.binding.grantId!));
|
|
expect((await identify("4")).status).toBe(401);expect((await db.select().from(bindings).where(eq(bindings.id,f.binding.id)))[0].clientVersion).toBe("2");
|
|
expect((await request(app).post("/api/muse/v1/queries").send({version:1,query:"identify",clientVersion:"3"})).status).toBe(401);
|
|
});
|
|
it("preserves personal and Dot default OAuth resources and their deployed refresh policy",async()=>{
|
|
await instanceSettingsService(db).updateExperimental({enablePublicMcp:true,enableOpenAiDot:true});const f=await fixture();
|
|
const personal=createPublicMcpOAuth(db,{origin:oauthOrigin,resource:oauthOrigin+"/mcp/paperclip"}),client=await personal.register({client_name:"Personal",redirect_uris:[oauthCallback]},randomUUID()),verifier=randomBytes(32).toString("base64url"),challenge=createHash("sha256").update(verifier).digest("base64url");
|
|
await expect(personal.authorize({client_id:client.client_id,redirect_uri:oauthCallback,response_type:"code",code_challenge:challenge,code_challenge_method:"S256"})).rejects.toThrow();
|
|
const personalId=(await personal.authorize({client_id:client.client_id,redirect_uri:oauthCallback,response_type:"code",resource:personal.config.resource,scope:"paperclip:read offline_access",code_challenge:challenge,code_challenge_method:"S256"})).split("/").at(-1)!;
|
|
const consent=await personal.consent(personalId,{type:"board",source:"session",userId:f.operatorId},{decision:"approve",companyId:f.company.id,allowWrites:false}),tokens=await personal.token({grant_type:"authorization_code",client_id:client.client_id,redirect_uri:oauthCallback,resource:personal.config.resource,code:new URL(consent.redirectUrl).searchParams.get("code")!,code_verifier:verifier});
|
|
expect((await personal.authenticate(tokens.access_token)).actor).toMatchObject({type:"board",userId:f.operatorId,isInstanceAdmin:false});
|
|
const [personalRefresh]=await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.tokenHash,hashMcpSecret(tokens.refresh_token!)));expect(personalRefresh.expiresAt).toBeInstanceOf(Date);
|
|
const [agent]=await db.insert(agents).values({companyId:f.company.id,name:"Dot",status:"active",adapterType:"paperclip_runner",adapterConfig:{provider:"openai_dot"}}).returning();const dot=createPublicMcpOAuth(db,{origin:oauthOrigin,resource:oauthOrigin+"/mcp/runner"}),dotClient=await dot.register({client_name:"Dot",redirect_uris:[oauthCallback]},randomUUID()),pairing=await dotRunnerBroker(db).createPairing({companyId:f.company.id,agentId:agent.id,operatorId:f.operatorId});
|
|
const dotId=(await dot.authorize({client_id:dotClient.client_id,redirect_uri:oauthCallback,response_type:"code",resource:dot.config.resource,scope:"paperclip:agent offline_access",code_challenge:challenge,code_challenge_method:"S256"})).split("/").at(-1)!;
|
|
const linked=await dot.consentDotPairing(dotId,pairing.pairingCode),dotTokens=await dot.token({grant_type:"authorization_code",client_id:dotClient.client_id,redirect_uri:oauthCallback,resource:dot.config.resource,code:new URL(linked.redirectUrl).searchParams.get("code")!,code_verifier:verifier});
|
|
expect((await dot.authenticate(dotTokens.access_token)).actor).toMatchObject({type:"agent",agentId:agent.id});const [dotRefresh]=await db.select().from(mcpOauthTokens).where(eq(mcpOauthTokens.tokenHash,hashMcpSecret(dotTokens.refresh_token!)));expect(dotRefresh.expiresAt).toBeNull();
|
|
});
|
|
|
|
});
|