Files
PaperClipAI/server/src/__tests__/connection-aggregator-fallback.test.ts
T
Nicky LeachandClaude Opus 5.5 11ac90e31f fix(build): typecheck server/src/__tests__ and fix every type error (#18739)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The server has about 1,100 test and helper files in
`server/src/__tests__`
> - `server/tsconfig.json` excludes that directory, Vitest does not
check types, and no script runs the tsconfig in that directory
> - So a test can call a function with the wrong arguments and still
pass, and no tool reports it
> - This gap hid 2,168 real type errors, and some of them hid tests that
did not test what they claim
> - This pull request fixes all of these errors and adds the directory
to the server layers from paperclipai/paperclip#18737
> - The benefit is that a type error in a server test now fails the
typecheck, with no baseline and no backlog

## Linked Issues or Issue Description

Refs paperclipai/paperclip#18737, which is merged and added the server
layers. Refs paperclipai/paperclip#18738, which is merged and checks the
test projects outside `pnpm -r`, so it is not a duplicate. Refs
paperclipai/paperclip#18137, which records that this directory has no
check.

**What happened?**

No tool checked the types of 1,074 of the 1,101 files in
`server/src/__tests__`. I searched `package.json`, the workflows, and
all tsconfig files. 27 files are in the program of another tsconfig, and
these 27 have 0 errors. 11 of the 27 are in
`helpers/maintenance/tsconfig.json`. No script ran that project when I
measured this. paperclipai/paperclip#18738 now runs it.

**Expected behavior**

The server typecheck checks these files, and a type error fails the
check.

**Steps to reproduce**

1. On the base commit, run `tsc -p server/src/__tests__/tsconfig.json
--noEmit`. It reports 6,536 errors and needs 12.3 GiB.
2. Add a type error to any file in `server/src/__tests__`.
3. Run `pnpm --filter @paperclipai/server typecheck`. It passes.

**Paperclip version or commit**

I measured the error breakdown below on commit `8dfd632452`, with
TypeScript 7.0.2. The branch now includes master up to
paperclipai/paperclip#18865.

**Deployment mode**

Not applicable. This issue applies to development and CI. No production
source file in the server changes.

### Error breakdown

Each row adds one change to the row above.

| Step | Total errors | Change | Cause |
| --- | --- | --- | --- |
| Config as committed | 6,536 | | |
| Add `src/types/*.d.ts` | 4,447 | -2,089 | Config. The Express request
types were missing. |
| Set `rootDir` to the repository root | 2,436 | -2,011 | Config. |
| Set `allowImportingTsExtensions` | 2,181 | -255 | Config. Vitest
accepts `.ts` imports. |
| Add `ES2024.Promise` to `lib` | 2,138 | -43 | Config. Node 24 has
`Promise.withResolvers`. |
| Set `allowJs` for `.mjs` imports | 2,148 | +10 | Config. It removes 8
`TS7016` errors and shows new real errors. |
| Set `declaration` to false | 2,142 | -6 | Config. Tests emit nothing.
|
| Count only files in `server/src` | 2,058 | -84 | Other packages: 53
Daytona, 29 Runner test, 2 UI. |

So 4,478 of the 6,536 errors (69%) came from config or scope, and 2,058
are real type errors in 314 files. The 29 Runner test errors are also
real, which gives 2,087 errors. Tests that master added after this
measurement had 81 more errors, so this pull request fixes 2,168 errors.

This table classifies the first 2,058 errors.

| Kind of real error | Errors |
| --- | --- |
| Wrong argument or assignment type | 1,023 |
| Stale or partial fixture objects | 525 |
| Property that the type does not have | 317 |
| Missing narrowing of null or unknown | 101 |
| Implicit any | 60 |
| Missing name or module | 16 |
| Other | 16 |

### What the type errors hid

Most errors were stale fixtures and loose mocks. These errors hid real
test problems, and this pull request fixes them:

- **A test that never ran.** `instrumentation.test.ts` required
`sdk-trace-base`, which is not installed. So 3 span tests were skipped.
They now run and pass.
- **Assertions that could not fail.**
- `issue-thread-interactions-service` used a company ID as an issue ID.
- `status-card-evaluation-work` compared `[]` with `[]` in a rollback
check.
- `slack-conversation-lifecycle` read `item.issueId`, which does not
exist.
  - `email-channels` used a permission key that does not exist.
- `heartbeat-project-env` imported `LOW_TRUST_REVIEW_PRESET`, which is
not exported. The value was undefined.
- **Postgres processes that did not stop.** 11 test files called `stop`
on the test database helper. That method does not exist, and the helper
method is `cleanup`.
- **Arguments that the service ignored.**
- `methodKey` is now `connectionMethodKey` in the Daytona, Unikraft,
1Password, and Railway tests.
- 108 calls passed `"owner-user"` to a parameter that expects an options
object.
- 9 chat wizard fixtures passed `managementContext` as `source`, so they
ran `existing_app` and not `manifest`.
  - 2 tests passed a `{ userId }` actor that the function ignores.
- Inserts into `environments` passed a `companyId` column that the table
does not have.
- **Problems in tests that master added during this work.**
- `sandbox-creation-operations` read `database.connectionString` when an
external test database was set. In that mode `database` is undefined, so
the fixture would stop with a TypeError. It now uses the external URL.
- `idle-completed-history` set a Dot operation to `dispatched`. That is
a Muse status, and the Dot type does not have it. The fixture now uses
`admitted`, which is a real in-flight Dot status.

### Possible product problems, not changed

These may be real problems in production code or types. I did not change
them, because this pull request does not change production behavior:

- `decisions-service` `service.create` does not validate the lenient
`cancel_issue_tree` input.
- `createConnection` ignores `input.connectionPurpose`.
- The `createConnection` route schema default stores `{}` for
`transportConfig`.
- `prepareManagedAiRuntime` accepted the invalid value `mode:
"connection"`.
- `createBinding` accepts invalid `projectionClass` strings outside
class 3.
- `ensureManagedProjectWorkspace` forwards an untyped `repoRef` by
spread.
- `routineVariableSchema.label` is optional, but `RoutineVariable.label`
is required.
- `ToolRunContext.projectId` may be too strict.
- `withdrawInteraction` reads `issue.status`, which its type omits.
- `issueService.list` and `reconcileNativeFinalizations` omit fields
that they return at runtime.
- `createGitHubExternalObjectProvider` handles `tokenProvider: null`,
but its type forbids this value.

## What Changed

- Fixed the 2,168 type errors. The diff changes 322 test and helper
files.
- Corrected `server/src/__tests__/tsconfig.json`. Editors and the
typecheck now read the same options and report the same errors.
- Added three `test-tree` layers to
`server/scripts/typecheck-layers.mjs`. They read the server layers as
declarations, and any error fails them. A hash of the path selects the
shard, so a new file does not move other files.
- Added `server/scripts/typecheck-stubs/daytona-sdk.d.ts`. Server tests
reach Daytona plugin source, and the root install does not have
`@daytonaio/sdk`.
- Changed two imports in `ui/src/lib/issue-execution-policy.ts` to use
`.js` extensions, because server tests import this file.
- Made `declarationsConfig` and `--declarations-only` skip the
`test-tree` layers, because these layers emit nothing. Other projects
read only the source layers. Added a test for this.
- Updated `doc/DEVELOPING.md`.

## Verification

Run these commands:

```sh
pnpm --filter @paperclipai/server typecheck
pnpm exec vitest run server/scripts/typecheck-layers.test.mjs
```

Results on macOS (arm64), Node 24, 16 cores. Each check ran after the
merge of paperclipai/paperclip#18738 unless the row says otherwise:

| Check | Commit | Result |
| --- | --- | --- |
| Server typecheck, `PAPERCLIP_TYPECHECK_LOW_MEMORY=1`, all 7 layers |
`6e45a08573` | Exit 0, 0 errors, 62 seconds, peak 3.52 GiB |
| `node scripts/typecheck-test-projects.mjs --low-memory` | `a3306bd13e`
| Exit 0, 0 errors, 42 seconds, peak 3.25 GiB |
| The same script with `--reuse-declarations` | `1b878aabc0` | Exit 0, 0
errors |
| `typecheck-layers.test.mjs` and `live-recovery-work.test.ts` |
`01db5634e4` | 90 passed. These include real compiler runs for a pass, a
type error, and an invalid tsconfig. |
| `tsconfig.declarations.json` after a full server typecheck |
`0fb1e4fb0a` | It references only `below`, `cycle`, and `above`. No
generated config names a `tsconfig.layers.json` file. |
| Owned runtime test in `idle-completed-history` | `e8f2f448ac` |
Passed, with the fix that master later replaced |
| `idle-incoming-wake`, `plugin-idle-manager`, and the changed
`chat-channels` test | `6e45a08573` | 30 passed |
| Runner `native-execution.test.ts` | `eab4dcc1f4` | 114 passed |
| The 19 server test files that changed since `eab4dcc1f4`, and
`typecheck-layers.test.mjs` | `259737b700` | 4,275 of 4,277 passed. The
2 failures are in `chat-channels` and are in the list of 7 below. |

The peak is the largest single process, from `/usr/bin/time -l`. Peaks
for the server typecheck were between 3.42 GiB and 3.56 GiB across 8
runs. Peaks for the test projects were 3.25 GiB and 3.37 GiB across 2
runs. I did not measure a cgroup.

**Runtime check of the fixes.** Types do not prove that a test still
works. So I ran all 309 changed server test files, 11,929 tests, before
the rebase onto paperclipai/paperclip#18737:

| Result | Tests | Detail |
| --- | --- | --- |
| Passed | 11,913 | Includes the 2 tests and the 54 `dot-runner` tests
that passed in a later run, as described below. |
| Failed | 7 | The same 7 tests also fail on the commit before the
fixes. |
| Skipped | 9 | These tests have their own skip conditions: `sentry` 5,
and 1 each in `bundled-plugin-registration`, `customer-success-cloud`,
`onepassword-connection`, and `workspace-runtime-https-live-exercise`. |

The 7 tests that fail with and without this change, on my machine:

- 3 tests in `chat-channels` and `email-channels` stop with `ENOENT` on
a `SKILL.md` file. The path is in the parent checkout, above the git
worktree that I used, so I expect that the worktree is the cause. I did
not prove this.
- `workspace-instance-cleanup` compares `/private/var/...` with
`/var/...`. This is a macOS symlink difference.
- 2 tests in `git-reproducible-managed-checkouts` get
`unknown_base_git_files` and expect `unpushed_managed_checkout`. I did
not find the cause.
- 1 more test in `chat-channels`, `materializes direct external-chat
finals once`, gets `undefined`. I did not find the cause.

CI is the check for these 7 tests. `ci / verify` passed on `eab4dcc1f4`.

Failures during this work:

- My fix to `agent-runtime-connector-access` changed an invalid
permission key in the setup, but not in the assertion. The runtime check
found this. I corrected the assertion, and the file passes.
- `heartbeat-plugin-environment` failed one wait in the full run. It
passed three times alone, so I think that load caused the failure. It
can fail again under load.
- `dot-runner` skipped all 54 tests in the full run because `cargo` was
not on my `PATH`. It then passed 54 of 54 with the toolchain on the
`PATH`.
- My first `lib` value removed the default DOM types, and the error
count went up by 375. I added the DOM libraries.
- My Daytona stub with bare `any` left 48 errors, because generic
helpers then inferred `unknown`. I added shapes that return promises.
- One fix to `agent-secrets-routes` used `class_3_static_lease`, and the
test failed. That class allows only specific paths. I reverted to
`unclassified`.
- `ci / General tests (server (8/12))` failed on `259737b700` because of
the flaky port in `idle-completed-history`. paperclipai/paperclip#18853
fixed this test on master, so this pull request keeps that fix.

**Audit of the diff.** I counted these patterns in added and removed
lines, to find fixes that only hide an error:

| Pattern | Added | Removed |
| --- | --- | --- |
| `@ts-ignore` | 0 | 0 |
| `@ts-expect-error` | 0 | 1 |
| `.skip` | 0 | 0 |
| `as any` | 9 | 9 |
| `: any` | 11 | 2 |
| `as never` | 8 | 2 |
| `as unknown as` | 35 | 4 |
| `Promise<unknown>` | 16 | 5 |
| `expect(` | 128 | 113 |

No hunk removes more `expect(` calls than it adds. Three removals of
test content were intentional, and I checked each one:

- `muse-control` removed `signalToken`. Line 1012 already asserts that
it is absent.
- `merged-pr-confirmation-sweep` removed legacy aliases. Other tests
cover them.
- `heartbeat-workspace-branch-containment` removed a `finalize` branch
that no test row can reach.

Some partial fixtures stay unchecked through `as unknown as` and
`Promise<unknown>`, mostly in `issue-agent-mutation-ownership-routes`
and `issue-comment-reopen-routes`.

**Accuracy of the check.** One program over the tree and the three
shards reported the same errors at the same positions. Memory, as the
peak of the largest process on one thread, was 3.93 GiB for one program
and 2.43 GiB for the largest shard. I measured these two values before
the fixes.

Not run or not proven:

- I did not run the full `pnpm test:run`, or a root `pnpm build`. I
expect CI to run the full suites.
- I did not run the 4 GiB container check, or the full
`scripts/typecheck-low-memory.mjs`. The peaks above come from macOS, so
they are not a Linux cgroup measurement.
- I did not run the UI typecheck after the merges.
- `workspace-runtime-https-live-exercise` is skipped, so its `cleanup`
fix did not run.

## Risks

- **A large diff in test files.** 328 files change. A pull request that
edits the same tests can get merge conflicts. New tests on master can
also add type errors before this merges. Any such error fails this check
and needs a fix here.
- **A higher cost for changes to source types.** A change to a type can
add errors in test files that the author did not touch. The author must
then fix them. This is the purpose of the check, but it is new work.
- **Changed test behavior.** Some tests now run paths that they did not
run before, such as the 9 chat wizard fixtures and the 3 span tests.
They pass here, but they can be more sensitive to timing or to the
environment.
- **The Daytona stub can go stale.** The stub is written by hand. If the
plugin uses a new SDK member, the typecheck reports an error until
someone updates the stub.
- **The release path.** `release-verify.yml` and `scripts/release.sh`
run `pnpm -r typecheck`, so they now run this check.
- **The tsconfig change.** The test transform reads
`server/src/__tests__/tsconfig.json`. The options that I changed are for
type checks, and the runs above passed.
- **A gap that remains.** Tsconfigs in `packages/paperclip-runner` and
in most sandbox provider plugins exclude `*.test.ts`. This check does
not read most of those test files, so they can still drift. That work is
separate from this pull request.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with
reasoning, tool use, code execution, and subagents. The runtime does not
show the context window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass (7 tests fail locally, with
and without this change. See Verification.)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 15:27:38 -08:00

911 lines
39 KiB
TypeScript

import { Ajv2020 } from "ajv/dist/2020.js";
import { requestHumanInputAction } from "../../../packages/paperclip-runner/src/protocol-actions/request-human-input.js";
import { questionSetToAskUserQuestionsPayload } from "@paperclipai/shared";
import { randomUUID } from "node:crypto";
import { eq } from "drizzle-orm";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import {
agents,
authUsers,
companies,
companyMemberships,
goals,
heartbeatRuns,
issueThreadInteractions,
issueComments,
issues,
createDb,
toolApplications,
toolConnections,
connectionGrants,
toolCatalogEntries,
toolProfiles,
toolProfileBindings,
toolConnectionInstalls,
} from "@paperclipai/db";
import type { RuntimeToolsTokenClaims } from "../runtime-tools-token.js";
import { connectionIntentService } from "../services/connection-intents.js";
import { instanceSettingsService } from "../services/instance-settings.js";
import { issueThreadInteractionService } from "../services/issue-thread-interactions.js";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
const support = await getEmbeddedPostgresTestSupport();
(support.supported ? describe : describe.skip)(
"aggregator connection fallback",
() => {
let db!: ReturnType<typeof createDb>;
let connectionString!: string;
let cleanup: (() => Promise<void>) | undefined;
let claims!: RuntimeToolsTokenClaims;
let runId!: string;
beforeAll(async () => {
const tempDb = await startEmbeddedPostgresTestDatabase(
"paperclip-connection-intents-",
);
cleanup = tempDb.cleanup;
connectionString = tempDb.connectionString;
db = createDb(connectionString);
const companyId = randomUUID();
const agentId = randomUUID();
const goalId = randomUUID();
const issueId = randomUUID();
runId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Connection tests",
issuePrefix: "AGG",
requireBoardApprovalForNewAgents: false,
});
await db.insert(authUsers).values({
id: "responsible-user", name: "Responsible user", email: "responsible-user@example.test",
createdAt: new Date(), updatedAt: new Date(),
});
await db.insert(companyMemberships).values({
companyId,
principalType: "user",
principalId: "responsible-user",
status: "active",
membershipRole: "member",
});
await db.insert(goals).values({
id: goalId,
companyId,
title: "Connect a service",
level: "task",
status: "active",
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "Researcher",
role: "researcher",
status: "active",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(issues).values({
id: issueId,
companyId,
goalId,
title: "Read Notion",
status: "in_progress",
priority: "medium",
assigneeAgentId: agentId,
});
await db.insert(heartbeatRuns).values({
id: runId,
companyId,
agentId,
status: "running",
responsibleUserId: "responsible-user",
contextSnapshot: { issueId },
});
claims = {
sub: agentId,
company_id: companyId,
run_id: runId,
responsible_user_id: "responsible-user",
scope: "connection_intents",
iat: 1,
exp: 2,
instance_id: "test",
};
}, 60_000);
afterAll(async () => {
await cleanup?.();
});
async function userRequest(body: string, userId = "responsible-user") {
const [run] = await db
.select()
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, runId));
await db
.insert(issueComments)
.values({
companyId: claims.company_id,
issueId: run!.contextSnapshot!.issueId as string,
authorUserId: userId,
body,
});
}
async function resetQuestions() {
await db
.delete(issueComments)
.where(eq(issueComments.companyId, claims.company_id));
await db
.delete(issueThreadInteractions)
.where(eq(issueThreadInteractions.companyId, claims.company_id));
await db
.delete(toolConnections)
.where(eq(toolConnections.companyId, claims.company_id));
await db
.delete(toolProfiles)
.where(eq(toolProfiles.companyId, claims.company_id));
await db
.delete(toolApplications)
.where(eq(toolApplications.companyId, claims.company_id));
}
async function selectProvider(
option: string,
userId = "responsible-user",
query = "hubspot",
) {
const result = await connectionIntentService(db).search(claims, query);
const [run] = await db
.select()
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, runId));
const issue = {
id: run!.contextSnapshot!.issueId as string,
companyId: claims.company_id,
};
const interaction = await issueThreadInteractionService(db).create(
issue,
{
kind: "ask_user_questions",
payload: { version: 1, questions: [result.providerQuestion!] },
sourceRunId: runId,
addresseeUserId: "responsible-user",
},
{ agentId: claims.sub, runId },
);
// Public resolution service enforces human answer semantics; company membership
// and wrong-identity request checks are also exercised below.
await issueThreadInteractionService(db).answerQuestions(
issue,
interaction.id,
{
answers: [
{ questionId: result.providerQuestion!.id, optionIds: [option] },
],
},
{ userId },
);
return interaction;
}
async function seedProvider(
provider: string,
toolName: string,
userId = "responsible-user",
allowed = true,
) {
const [app] = await db
.insert(toolApplications)
.values({
companyId: claims.company_id,
applicationKey: randomUUID(),
name: provider,
type: "mcp_http",
status: "active",
metadata: { sourceTemplateKey: provider },
})
.returning();
const [connection] = await db
.insert(toolConnections)
.values({
companyId: claims.company_id,
applicationId: app!.id,
uid: randomUUID(),
name: provider,
transport: "mcp_remote",
authKind: "none",
credentialPolicy: "per_user",
status: "active",
enabled: true,
healthStatus: "ok",
config: { sourceTemplateKey: provider },
})
.returning();
await db.insert(connectionGrants).values({
companyId: claims.company_id,
connectionId: connection!.id,
kind: "user",
subjectUserId: userId,
status: "active",
});
await db.insert(toolCatalogEntries).values({
companyId: claims.company_id,
connectionId: connection!.id,
toolName,
name: toolName,
versionHash: "fixture",
status: "active",
entryKind: "tool",
lastSeenAt: new Date("2026-09-20T00:00:00Z"),
});
await db.insert(toolConnectionInstalls).values({
companyId: claims.company_id,
connectionId: connection!.id,
targetType: "agent",
targetId: claims.sub,
});
if (allowed) {
const [profile] = await db
.insert(toolProfiles)
.values({
companyId: claims.company_id,
profileKey: randomUUID(),
name: "Fixture reads",
defaultAction: "allow",
status: "active",
})
.returning();
await db.insert(toolProfileBindings).values({
companyId: claims.company_id,
profileId: profile!.id,
targetType: "agent",
targetId: claims.sub,
});
}
return connection!;
}
it.each([
["AgentMail create an email address and manage an agent mailbox", "agentmail"],
["Please acquire an email address through agent mail and remember it", "agentmail"],
["I need an Agentmial inbox for receiving mail from customers", "agentmail"],
["Please connect my Linear workspace so I can triage the team's backlog", "linear"],
["Find a Notion connection to search our engineering documentation", "notion"],
["Can you find the OpenRouter connection for the model I want to use?", "openrouter"],
["Please help me connect to Git Hub to look at my pull requests", "github"],
])("finds native connections in natural-language queries: %s", async (query, service) => {
await resetQuestions();
await instanceSettingsService(db).updateExperimental({ enableChatConnectors: true });
try {
const result = await connectionIntentService(db).search(claims, query);
expect(result.results[0]).toMatchObject({ service, state: "available" });
expect(result.providerQuestion).toBeUndefined();
} finally {
await instanceSettingsService(db).updateExperimental({ enableChatConnectors: false });
}
});
it.each([false, true])("discovers AgentMail with truthful setup actions regardless of the chat setting (%s)", async (enabled) => {
await resetQuestions();
const service = connectionIntentService(db);
await instanceSettingsService(db).updateExperimental({ enableChatConnectors: enabled });
try {
const result = await service.search(claims, "agentmail");
expect(result.results[0]?.methods).toEqual([expect.objectContaining({
key: "email-agent", purpose: "channel", setupPath: `/AGG/apps/chat/connect?provider=agentmail&purpose=chat&agentId=${claims.sub}`,
})]);
expect(result.instruction).toContain("connection_request");
const requested = await service.request(claims, "agentmail");
expect(requested).toMatchObject({ state: "needs_user_action", interactionId: expect.any(String) });
expect(await service.request(claims, "agentmail")).toMatchObject({ interactionId: requested.interactionId });
const options = await service.setupOptions(requested.interactionId!);
expect(options.interaction.payload).toMatchObject({ purpose: "channel", serviceSlug: "agentmail", requestingAgentId: claims.sub });
expect(options.emailSetup).toEqual({ credentialConnectionId: null, readyConnectionId: null });
await service.decline(requested.interactionId!, "responsible-user");
await expect(service.request(claims, "agentmail")).rejects.toThrow(/already been resolved/);
const browse = await service.search(claims, "");
expect(browse.results.some(item => item.service === "agentmail")).toBe(true);
expect(browse.results.some(item => item.service === "anthropic")).toBe(true);
} finally {
await instanceSettingsService(db).updateExperimental({ enableChatConnectors: false });
}
});
it("keeps useful capability matches even when other query words do not occur in the catalog", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(claims,
"I need something that can search documents and spreadsheets for our quarterly planning discussion");
expect(result.results.some(item => item.service === "google-drive")).toBe(true);
expect(result.results.some(item => item.service === "google-sheets")).toBe(true);
expect(result.instruction).not.toContain("search its exact name");
});
it("keeps installed capability matches when a generic query resembles an external app name", async () => {
await resetQuestions();
const installed = await seedProvider("Studio library", "notion:list_pages");
await db.update(toolCatalogEntries).set({ description: "Read recent pages and return their titles and verification code" })
.where(eq(toolCatalogEntries.connectionId, installed.id));
const service = connectionIntentService(db);
const result = await service.search(claims,
"connected page service that can find or list recent pages and return page titles plus a verification code");
expect(result.results).toEqual(expect.arrayContaining([expect.objectContaining({
service: `connection:${installed.id}`, connectionId: installed.id, state: "ready",
})]));
expect(result.providerQuestion).toBeUndefined();
expect(result.instruction).not.toContain("Ask the responsible user with providerQuestion");
// A deliberate app name still reaches the existing governed provider choice.
const named = await service.search(claims, "Page X");
expect(named.providerQuestion?.id).toBe("connection-provider:page-x");
expect(named.results[0]?.service).toBe("via:composio:page-x");
});
it("keeps an external typo as a discovery suggestion until its app is selected", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(claims,
"Find Circlebak meeting transcripts and action items from yesterday");
expect(result.results.some(item => item.service === "via:composio:circleback-mcp")).toBe(true);
expect(result.providerQuestion).toBeUndefined();
expect(result.instruction).toContain("aggregator.targetService");
});
it.each([
["help me find tools for circle back", "circleback-mcp"],
["Can you connect Circleback MCP to get all our meeting notes?", "circleback-mcp"],
["Find Attio tools to review all our customer contacts before next week's meeting", "attio"],
["Help me find a ClickUp connection to organize our team's projects", "clickup"],
])("finds verified aggregator apps in natural-language queries: %s", async (query, target) => {
await resetQuestions();
const result = await connectionIntentService(db).search(claims, query);
expect(result.results).toEqual(expect.arrayContaining([expect.objectContaining({
service: `via:composio:${target}`, source: "aggregator",
aggregator: expect.objectContaining({ evidenceUrl: expect.stringContaining("composio.dev/"), targetService: target }),
})]));
expect(result.providerQuestion?.options.map(option => option.id)).toContain(`via:composio:${target}`);
await expect(connectionIntentService(db).request(claims, `via:composio:${target}`)).rejects.toThrow();
});
it.each([false, true])("prefers an exact Motion match over fuzzy Notion (Notion denied: %s)", async denied => {
await resetQuestions();
if (denied) await seedProvider("notion", "notion_search", "responsible-user", false);
const result = await connectionIntentService(db).search(claims, "Help me find Motion tools");
expect(result.results[0]?.service).toBe("via:composio:motion");
expect(result.providerQuestion?.id).toBe("connection-provider:motion");
expect(result.instruction).not.toContain("administratively restricted");
});
it("finds namespaced installed aggregator tools with extra query words", async () => {
await resetQuestions();
await seedProvider("executor", "heliotrope:list_records");
const result = await connectionIntentService(db).search(claims, "Please find heliotrope tools for our weekly report");
expect(result.results[0]?.service).toBe("via:executor:heliotrope");
});
it("returns multiple named services so the agent can choose the relevant result", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(claims, "Find Linear or Notion for our project planning");
expect(result.results.map(item => item.service)).toEqual(expect.arrayContaining(["linear", "notion"]));
});
it("returns multiple aggregator apps without inventing a combined route or choosing consent", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(claims, "Find Circleback and Attio tools");
expect(result.results.map(item => item.service)).toEqual(expect.arrayContaining(["via:composio:circleback-mcp", "via:composio:attio"]));
expect(result.providerQuestion).toBeUndefined();
expect(result.instruction).toContain("aggregator.targetService");
});
it("keeps native and aggregator app matches from the same query", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(claims, "Find Notion and Circleback tools");
expect(result.results.map(item => item.service)).toEqual(expect.arrayContaining(["notion", "via:composio:circleback-mcp"]));
expect(result.providerQuestion).toBeUndefined();
});
it("prefers the built-in Jira connection and returns a direct instruction", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(
claims,
"atlassian jira",
);
expect(result.results[0]?.service).toBe("jira");
expect(result.providerQuestion).toBeUndefined();
expect(result.instruction).toContain("connection_request");
});
it("returns ranked verified routes and does not create questions or connections during search", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(
claims,
"HubSpot recent contacts",
);
expect(result.results.map((item) => item.service)).toEqual([
"via:composio:hubspot",
"via:arcade:hubspot",
"via:zapier:hubspot",
]);
expect(result.providerQuestion?.prompt).toContain("external service");
const ajv = new Ajv2020({ allErrors: true, allowUnionTypes: true, strict: false });
const input = { idempotencyKey: "provider-choice", interactionKind: "questions", title: "Choose a provider",
prompt: result.providerQuestion!.prompt, continuationPolicy: "wake_assignee" };
for (const payload of [{ version: 1, questionSet: result.providerQuestionSet }, { version: 1, questions: [result.providerQuestion] }]) {
expect(ajv.validate(requestHumanInputAction.live.descriptor.inputSchema, { ...input, payload }), JSON.stringify(ajv.errors)).toBe(true);
}
expect(questionSetToAskUserQuestionsPayload(result.providerQuestionSet!).questions).toEqual([result.providerQuestion]);
expect(result.instruction).toContain("questionSet:providerQuestionSet");
expect(result.instruction).not.toContain("do not add questionSet");
expect(ajv.validate(requestHumanInputAction.live.descriptor.inputSchema, { ...input, payload: { version: 1 } })).toBe(false);
expect(ajv.validate(requestHumanInputAction.live.descriptor.inputSchema, { ...input, payload: { version: 1,
questionSet: { ...result.providerQuestionSet, questions: [{ ...result.providerQuestionSet!.questions[0], answerMode: "text" }] } } })).toBe(false);
expect(result.results.every((item) => item.state !== "ready")).toBe(true);
expect(result.results.every((item) => item.aggregator?.evidenceUrl)).toBe(
true,
);
expect(
await db
.select()
.from(issueThreadInteractions)
.where(eq(issueThreadInteractions.companyId, claims.company_id)),
).toHaveLength(0);
});
it("requires a real saved provider answer before requesting setup", async () => {
await resetQuestions();
await expect(
connectionIntentService(db).request(claims, "via:composio:hubspot"),
).rejects.toMatchObject({ status: 403 });
await expect(
connectionIntentService(db).request(claims, "via:composio:hubspot", {
selectionInteractionId: randomUUID(),
}),
).rejects.toMatchObject({ status: 403 });
});
it("uses the second selected provider and preserves target disclosure and idempotency", async () => {
await resetQuestions();
const answer = await selectProvider("via:arcade:hubspot");
const service = connectionIntentService(db);
const search = await service.search(claims, "hubspot");
expect(search.results.map((item) => item.service)).toEqual([
"via:arcade:hubspot",
]);
expect(search.selectionInteractionId).toBe(answer.id);
const first = await service.request(claims, "via:arcade:hubspot", {
selectionInteractionId: answer.id,
});
const again = await service.request(claims, "via:arcade:hubspot", {
selectionInteractionId: answer.id,
});
expect(again.interactionId).toBe(first.interactionId);
const [row] = await db
.select()
.from(issueThreadInteractions)
.where(eq(issueThreadInteractions.id, first.interactionId!));
expect(row!.payload).toMatchObject({
serviceSlug: "arcade",
serviceName: "HubSpot through Arcade",
upstreamService: { slug: "hubspot", selectionInteractionId: answer.id },
});
await expect(
service.request(claims, "via:composio:hubspot", {
selectionInteractionId: answer.id,
}),
).rejects.toMatchObject({ status: 403 });
});
it("remembers None and refuses setup after reloading the service", async () => {
await resetQuestions();
const answer = await selectProvider("none");
const result = await connectionIntentService(db).search(
claims,
"hubspot",
);
expect(result.results).toEqual([]);
expect(result.providerQuestion).toBeUndefined();
expect(result.instruction).toContain("declined");
const retry = await connectionIntentService(db).search(
claims,
"hubspot",
{ retryProviderChoice: true },
);
expect(retry.providerQuestion).toBeDefined();
expect(retry.selectionInteractionId).toBeUndefined();
await expect(
connectionIntentService(db).request(claims, "via:arcade:hubspot", {
selectionInteractionId: answer.id,
}),
).rejects.toMatchObject({ status: 403 });
});
it("waits on an existing unanswered question instead of asking twice", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(
claims,
"hubspot",
);
const [run] = await db
.select()
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, runId));
const interaction = await issueThreadInteractionService(db).create(
{
id: run!.contextSnapshot!.issueId as string,
companyId: claims.company_id,
},
{
kind: "ask_user_questions",
payload: { version: 1, questions: [result.providerQuestion!] },
sourceRunId: runId,
addresseeUserId: "responsible-user",
},
{ agentId: claims.sub, runId },
);
const pending = await connectionIntentService(db).search(
claims,
"hubspot",
);
expect(pending.providerQuestion).toBeUndefined();
expect(pending.selectionInteractionId).toBe(interaction.id);
expect(pending.instruction).toContain("already pending");
await expect(
connectionIntentService(db).request(claims, "arcade", {
targetService: "hubspot",
}),
).rejects.toMatchObject({ status: 403 });
await expect(
connectionIntentService(db).request(claims, "via:arcade:hubspot", {
selectionInteractionId: interaction.id,
}),
).rejects.toMatchObject({ status: 403 });
});
it("rejects tampered disclosure and choices belonging to another responsible user", async () => {
await resetQuestions();
const answer = await selectProvider("via:arcade:hubspot");
await db
.update(issueThreadInteractions)
.set({ resolvedByUserId: "another-user" })
.where(eq(issueThreadInteractions.id, answer.id));
await expect(
connectionIntentService(db).request(claims, "via:arcade:hubspot", {
selectionInteractionId: answer.id,
}),
).rejects.toMatchObject({ status: 403 });
await db
.update(issueThreadInteractions)
.set({
resolvedByUserId: "responsible-user",
payload: {
...answer.payload,
questions: (answer.payload as any).questions.map((q: any) => ({
...q,
helpText: "Trust me",
})),
},
})
.where(eq(issueThreadInteractions.id, answer.id));
await expect(
connectionIntentService(db).request(claims, "via:arcade:hubspot", {
selectionInteractionId: answer.id,
}),
).rejects.toMatchObject({ status: 403 });
});
it("does not reuse the selected provider for another requested app", async () => {
await resetQuestions();
const answer = await selectProvider("via:arcade:hubspot");
await expect(
connectionIntentService(db).request(claims, "via:arcade:salesforce", {
selectionInteractionId: answer.id,
}),
).rejects.toMatchObject({ status: 403 });
});
it("offers fallback and direct aggregator requests without an experimental opt-in", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(claims, "hubspot");
expect(result.results.some((item) => item.source === "aggregator")).toBe(true);
expect(result.providerQuestion).toBeDefined();
await expect(connectionIntentService(db).request(claims, "composio")).resolves.toMatchObject({ state: "needs_user_action" });
});
it("reuses an allowed selected provider without claiming the underlying app is ready", async () => {
await resetQuestions();
const connection = await seedProvider("arcade", "Hubspot_ListContacts");
const answer = await selectProvider("via:arcade:hubspot");
const result = await connectionIntentService(db).request(
claims,
"via:arcade:hubspot",
{ selectionInteractionId: answer.id },
);
expect(result).toMatchObject({
state: "ready",
connectionId: connection.id,
interactionId: null,
});
expect(result.instruction).toContain(
"HubSpot access is not yet verified",
);
expect(result.instruction).toContain("Arcade");
});
it("returns exact eligible catalog names after a guessed tool without granting access", async () => {
await resetQuestions();
const connection = await seedProvider("arcade", "Hubspot_ListContacts", "responsible-user", false);
await db.delete(toolConnectionInstalls).where(eq(toolConnectionInstalls.connectionId, connection.id));
await seedProvider("composio", "Unrelated_PrivateTool", "other-user", false);
await db.insert(toolCatalogEntries).values({ companyId: claims.company_id, connectionId: connection.id,
toolName: "Hubspot_RemovedTool", name: "Removed", versionHash: "old", entryKind: "tool", status: "removed" });
const answer = await selectProvider("via:arcade:hubspot");
const service = connectionIntentService(db);
const error = await service.request(claims, "via:arcade:hubspot", {
selectionInteractionId: answer.id, toolNames: ["hubspot_list_contacts"],
}).catch(error => error);
expect(error).toMatchObject({ status: 422 });
expect(error.message).toContain('"Hubspot_ListContacts"');
expect(error.message).toContain("Request only the needed exact names");
expect(error.message).not.toMatch(/Unrelated_PrivateTool|Hubspot_RemovedTool/);
expect(await db.select().from(toolProfiles).where(eq(toolProfiles.companyId, claims.company_id))).toEqual([]);
const interactions = await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.companyId, claims.company_id));
expect(interactions.map(row => [row.id, row.status])).toEqual([[answer.id, "answered"]]);
const requested = await service.request(claims, "via:arcade:hubspot", {
selectionInteractionId: answer.id, toolNames: ["Hubspot_ListContacts"],
});
expect(requested.state).toBe("needs_user_action");
const loaded = await service.loadIntent(requested.interactionId!);
expect(loaded.interaction.payload.accessRequest?.tools.map(tool => tool.toolName)).toEqual(["Hubspot_ListContacts"]);
expect(loaded.interaction.status).toBe("pending");
expect(await db.select().from(toolProfiles).where(eq(toolProfiles.companyId, claims.company_id))).toEqual([]);
});
it("bounds catalog-name recovery and does not disclose an ineligible connection", async () => {
await resetQuestions();
const connection = await seedProvider("arcade", "Read_00", "responsible-user", false);
await db.insert(toolCatalogEntries).values(Array.from({ length: 24 }, (_, index) => ({
companyId: claims.company_id, connectionId: connection.id, toolName: `Read_${String(index + 1).padStart(2, "0")}`,
name: `Read_${String(index + 1).padStart(2, "0")}`, versionHash: "v1", entryKind: "tool" as const, status: "active" as const,
})));
const service = connectionIntentService(db);
const error = await service.request(claims, "arcade", { connectionId: connection.id, toolNames: ["guessed"] }).catch(error => error);
expect(error).toMatchObject({ status: 422 });
expect(error.message).toContain("first 20 of 25");
expect(error.message.match(/Read_\d{2}/g)).toHaveLength(20);
await db.update(connectionGrants).set({ status: "revoked" }).where(eq(connectionGrants.connectionId, connection.id));
const denied = await service.request(claims, "arcade", { connectionId: connection.id, toolNames: ["guessed"] }).catch(error => error);
expect(denied.message).not.toContain("Read_");
expect(denied.message).toContain("not eligible");
});
it("does not switch providers when the chosen route loses permission", async () => {
await resetQuestions();
const answer = await selectProvider("via:arcade:hubspot");
await seedProvider(
"arcade",
"Hubspot_ListContacts",
"responsible-user",
false,
);
const result = await connectionIntentService(db).search(
claims,
"hubspot",
);
expect(result.results).toEqual([]);
expect(result.providerQuestion).toBeUndefined();
expect(result.instruction).toContain("no longer available");
await expect(
connectionIntentService(db).request(claims, "via:composio:hubspot", {
selectionInteractionId: answer.id,
}),
).rejects.toMatchObject({ status: 403 });
});
it("keeps a decline effective when disclosure wording has changed", async () => {
await resetQuestions();
const answer = await selectProvider("none");
await db
.update(issueThreadInteractions)
.set({
payload: {
...answer.payload,
questions: (answer.payload as any).questions.map((q: any) => ({
...q,
prompt: "Older disclosure wording",
})),
},
})
.where(eq(issueThreadInteractions.id, answer.id));
expect(
(await connectionIntentService(db).search(claims, "hubspot"))
.instruction,
).toContain("declined");
});
it("persists app authorization guidance in the completed setup outcome", async () => {
await resetQuestions();
const answer = await selectProvider("via:composio:hubspot");
const service = connectionIntentService(db);
const requested = await service.request(claims, "via:composio:hubspot", {
selectionInteractionId: answer.id,
});
const connection = await seedProvider(
"composio",
"COMPOSIO_SEARCH_TOOLS",
);
const completed = await service.complete(
requested.interactionId!,
connection.id,
"responsible-user",
);
expect(completed).toMatchObject({
status: "accepted",
result: {
outcome: "connected",
connectionId: connection.id,
instruction: expect.stringContaining(
"HubSpot access is not yet verified",
),
},
});
expect((completed.result as any).instruction).toContain(
"COMPOSIO_MANAGE_CONNECTIONS",
);
});
it("uses only authorized Executor tool evidence and preserves the actual observation date", async () => {
await resetQuestions();
const connection = await seedProvider(
"executor",
"heliotrope:list_records",
"someone-else",
);
expect(
(await connectionIntentService(db).search(claims, "heliotrope"))
.results,
).toEqual([]);
await db
.update(connectionGrants)
.set({ subjectUserId: "responsible-user" })
.where(eq(connectionGrants.connectionId, connection.id));
const result = await connectionIntentService(db).search(
claims,
"heliotrope",
);
expect(result.results[0]).toMatchObject({
service: "via:executor:heliotrope",
aggregator: { verifiedAt: "2026-09-20T00:00:00.000Z" },
});
await db
.update(toolConnections)
.set({ status: "archived" })
.where(eq(toolConnections.id, connection.id));
expect(
(await connectionIntentService(db).search(claims, "heliotrope"))
.results,
).toEqual([]);
});
it("does not bypass an administratively denied built-in connection", async () => {
await resetQuestions();
await seedProvider("jira", "jira_read", "responsible-user", false);
const result = await connectionIntentService(db).search(
claims,
"Jira recent issues",
);
expect(result.results[0]).toMatchObject({
service: "jira",
state: "unavailable",
});
expect(result.providerQuestion).toBeUndefined();
expect(result.instruction).toContain("Do not bypass");
});
it("does not invent support for unknown apps", async () => {
await resetQuestions();
const result = await connectionIntentService(db).search(
claims,
"nimbuscrm-not-real",
);
expect(result.results).toEqual([]);
expect(result.instruction).toContain("could not be verified");
});
it("preserves indexed-only display names when a saved choice is requested by slug", async () => {
await resetQuestions();
await seedProvider("executor", "heliotrope:list_records");
const answer = await selectProvider(
"via:executor:heliotrope",
"responsible-user",
"HELIOTROPE",
);
const result = await connectionIntentService(db).request(
claims,
"via:executor:heliotrope",
{ selectionInteractionId: answer.id },
);
expect(result.state).toBe("ready");
expect(result.instruction).toContain(
"Heliotrope access is not yet verified",
);
});
it("honors explicitly named providers and preserves target app context", async () => {
await resetQuestions();
await userRequest("Connect HubSpot through Arcade");
const service = connectionIntentService(db);
const result = await service.search(claims, "HubSpot through Arcade");
expect(result.results.map((item) => item.service)).toEqual(["arcade"]);
expect(result.providerQuestion).toBeUndefined();
expect(result.instruction).toContain("targetService hubspot");
const request = await service.request(claims, "arcade", {
targetService: "hubspot",
});
const intent = await service.loadIntent(request.interactionId!);
expect(intent.interaction.payload).toMatchObject({
serviceName: "HubSpot through Arcade",
upstreamService: { slug: "hubspot", name: "HubSpot" },
});
expect(
(await service.search(claims, "HubSpot through Executor")).results,
).toEqual([]);
await expect(
service.request(claims, "executor", { targetService: "hubspot" }),
).rejects.toThrow("cannot connect");
await expect(
service.request(claims, "github", { targetService: "hubspot" }),
).rejects.toThrow("direct external-provider");
});
it("permits explicit provider preference for a supported native app but not a native denial", async () => {
await resetQuestions();
await userRequest("Connect Jira via Arcade, not Zapier");
const service = connectionIntentService(db);
expect(
(await service.search(claims, "Jira via Arcade, not Zapier")).results.map(
(item) => item.service,
),
).toEqual(["arcade"]);
await seedProvider("jira", "jira_read", "responsible-user", false);
expect((await service.search(claims, "Jira via Arcade, not Zapier")).results).toEqual(
[expect.objectContaining({ service: "jira", state: "unavailable" })],
);
});
it("does not let an agent-supplied explicit query or direct target override a saved choice", async () => {
await resetQuestions();
const service = connectionIntentService(db);
const unproven = await service.search(claims, "HubSpot via Arcade");
expect(
unproven.providerQuestion?.options.map((option) => option.id),
).toEqual(["via:arcade:hubspot", "none"]);
await expect(
service.request(claims, "arcade", { targetService: "hubspot" }),
).rejects.toThrow("cannot connect");
await userRequest("Connect HubSpot via Arcade is just an example; do not connect yet");
await expect(service.request(claims,"arcade",{targetService:"hubspot"})).rejects.toThrow("cannot connect");
await userRequest("Connect HubSpot via Arcade");
await selectProvider("none");
expect(
(await service.search(claims, "HubSpot via Arcade")).results,
).toEqual([]);
await expect(
service.request(claims, "arcade", { targetService: "hubspot" }),
).rejects.toThrow("cannot connect");
await userRequest("Connect HubSpot via Arcade", "someone-else");
await expect(
service.request(claims, "arcade", { targetService: "hubspot" }),
).rejects.toThrow("cannot connect");
await userRequest("Please connect HubSpot through Arcade");
expect(
(await service.request(claims, "arcade", { targetService: "hubspot" }))
.state,
).toBe("needs_user_action");
await resetQuestions();
const chosen = await selectProvider("via:composio:hubspot");
await expect(
service.request(claims, "arcade", { targetService: "hubspot" }),
).rejects.toThrow("cannot connect");
const accepted = await service.request(claims, "composio", {
targetService: "hubspot",
});
expect(
(await service.loadIntent(accepted.interactionId!)).interaction.payload,
).toMatchObject({
upstreamService: { selectionInteractionId: chosen.id },
});
});
it("asks for a choice when human and agent messages name alternatives", async () => {
await resetQuestions();
await userRequest("Connect HubSpot via Arcade or Composio");
const result = await connectionIntentService(db).search(
claims,
"HubSpot via Arcade or Composio",
);
expect(
result.providerQuestion?.options.map((option) => option.id),
).toEqual([
"via:composio:hubspot",
"via:arcade:hubspot",
"via:zapier:hubspot",
"none",
]);
await expect(
connectionIntentService(db).request(claims, "arcade", {
targetService: "hubspot",
}),
).rejects.toThrow("cannot connect");
});
},
);