Files
PaperClipAI/tests/e2e/chat-adapters-ui.shared.ts
T
Nicky LeachandClaude Opus 5.5 83c6832ccb fix(build): typecheck package, Playwright, and e2e test files in CI (#18839)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Runner, plugin, and Playwright tests verify how those agents run,
connect, and appear in the board
> - Most package tsconfigs exclude tests, and Vitest strips types, so a
type error in those tests fails nothing
> - The tests drifted to 803 real type errors, and some of those errors
hid assertions that checked nothing
> - paperclipai/paperclip#18739 and paperclipai/paperclip#18738 check
the server test tree and four test projects, but not these files
> - This pull request adds a required check for these test files, fixes
every error, and fails CI if a test file is in no program
> - The benefit is that a type error in any test file outside
`server/src` now fails CI

## Linked Issues or Issue Description

Refs #18739
Refs #18738

**What happened?**

517 test files were in no program that required CI checks. A type error
in those files did not fail CI.

**Expected behavior**

CI fails on a type error in any test file.

**Steps to reproduce**

Add `export const value: number = "text";` to a test file under
`packages/paperclip-runner/src` on master. CI stays green, because
`tsconfig.json` excludes test files and Vitest strips types.

**Measured gap**

Each row is a program with its tests included, on one thread. File and
error counts are for head `7d4b5e907a`. Memory peaks were measured
before the two latest merges of master.

| Group | Test files | Real errors | Peak |
| --- | ---: | ---: | ---: |
| `paperclip-runner` Node source | 290 | 644 | 1.09 GiB |
| `paperclip-runner` surfaces | 9 | 12 | 0.42 GiB |
| `paperclip-runner` devtools and examples | 14 | 0 | 0.68 GiB |
| `sandbox-providers/kubernetes` | 21 | 65 | 0.44 GiB |
| 8 other sandbox providers and `bridge-template` | 32 | 4 | 0.34 GiB |
| `plugins/sdk` | 22 | 15 | 0.41 GiB |
| `plugin-workspace-diff` | 5 | 0 | 0.37 GiB |
| `plugin-orchestration-smoke-example` | 1 | 1 | 0.15 GiB |
| `kv-demo-mcp-server` | 3 | 0 | 0.13 GiB |
| `packages/db` scripts | 1 | 0 | 1.01 GiB |
| 8 Playwright suites under `tests/` | 52 | 25 | 1.72 GiB |
| `tests/runner-acceptance` | 3 | 0 | 0.58 GiB |
| `tests/e2e`, through layer declarations | 63 | 37 | at most 1.59 GiB |
| `ui` `AttentionInteractionResolver.test.tsx` | 1 | 0 | n/a |
| **Total** | **517** | **803** | |

27 of the 803 errors came from master while this pull request was open.
Each merge of master brought new test code that no check had read.

Errors that were config errors, not real errors:

- One TS2688 for each provider, because the root install does not
install provider dependencies.
- 9 in the Runner Node program (TS7016 and TS6059).
- 2 in `tsconfig.browser.json`. TypeScript 7 removed `baseUrl`, so that
config was invalid.
- 213 in the Playwright suites (`@/*` aliases and Vite types).
- About 60 in `tests/e2e` (import extensions and aliases).

`packages/plugins/create-paperclip-plugin` has no gap. Its tsconfig
excludes its test, but `cli/tsconfig.json` reaches that test through an
import. `server/test-baselines` also has no gap, because
`tests/lifecycle-baseline` reads it.

## What Changed

- Added `scripts/typecheck-test-programs.mjs` and its tests. The
`Typecheck + Release Registry` job runs it. It checks each program in
`PROGRAMS` on one thread.
- The script fails if a tracked `*.test.ts(x)` or `*.spec.ts(x)` file
outside `server/src` is in no program. The compiler file list decides
this, because `include` drops `foo.test.tsx` when `foo.test.ts` exists.
- Added a `tsconfig.test.json` to each uncovered package, and
`tests/tsconfig.playwright.json`.
- Added `tests/e2e` to `PROJECTS` in
`scripts/typecheck-test-projects.mjs`, with a `tsconfig.json` and a
`tsconfig.layers.json`.
- Added `scripts/typecheck-sandbox-providers`. It is a small workspace
that installs provider SDKs from one lockfile. I did not add a lockfile
inside each provider, because
`scripts/build-standalone-public-packages.mjs` uses a frozen install
when one exists, and that changes what a release resolves.
- Corrected `packages/paperclip-runner/tsconfig.browser.json` for
TypeScript 7.
- Set `rootDir` to `..` in
`packages/paperclip-runner/tsconfig.test.json`, because one Runner test
imports from `packages/adapter-utils/src`. The program does not emit
files, so this changes no build output.
- Moved the Kubernetes provider from `vitest ^3.2.4` to `^4.1.8`, the
version the other providers use. Vitest 3 requires `tinypool ^1.1.1`,
which has two critical advisories (GHSA-85c8-ppgw-ccpr and
GHSA-5gmw-xhrv-c9v3). The first patched version is 2.1.2. Its tests have
the same result on both versions: 189 passed, and the same 12
`file-sync` tests failed.
- Moved the Kubernetes provider from `zod ^3.24.2` to `^4.6.6`, and gave
its two `z.record` calls a key schema. This is the only production
change, and it stays in this pull request. See "Production change for
review".
- Renamed `AttentionInteractionResolver.test.ts` to
`AttentionInteractionResolver.replace.test.ts`, so TypeScript reads the
`.test.tsx` beside it.
- Fixed all 803 errors, with changes to test files and test support
only. The fix for `contracts/native-execution.test.ts` is the same as
the fix in paperclipai/paperclip#18739, so those two changes merge
cleanly.
- Updated `doc/DEVELOPING.md`.

This pull request does not edit `server/scripts/typecheck-layers.mjs`.
paperclipai/paperclip#18739 and paperclipai/paperclip#18738 have both
merged, and this branch includes them.

### Production change for review

The new lockfile made dependency review flag `zod@3.25.76` as a
high-severity advisory (GHSA-mrhx-c7xc-m3w5). The first patched version
is 4.6.6, so 3.x has no fix. The published Kubernetes plugin uses zod to
validate operator config. The zod 3 source does not compile against zod
4, so a typecheck-only override cannot fix this.

The existing tests do not prove this change, because they also passed
with the invalid one-argument `z.record`. So I parsed 45 inputs with the
same schema on zod 3.25.76 and zod 4.6.7:

| Result | Inputs | Detail |
| --- | ---: | --- |
| Identical | 23 | Includes defaults, valid config, and a `__proto__`
key |
| Same outcome, new error wording | 21 | For example, `Invalid url` is
now `Invalid URL`. `validateConfig` shows this text to operators. |
| Different outcome | 1 | `jobTtlSecondsAfterFinished: 2 ** 60` was
valid and is now rejected. The new limit is `Number.MAX_SAFE_INTEGER`. |

I did not test this against a real cluster.

The decision is to keep this change in this pull request. Since then,
paperclipai/paperclip#17364 has moved the rest of the repository to zod
4.6.x, so the Kubernetes plugin now matches it.

### Behavior change for review

`reducedMotion` under `use` is not a Playwright option. It had no effect
in 11 configs and 2 `test.use` calls, so those suites ran with
`no-preference`. The options are now under `contextOptions`, so reduced
motion now takes effect. `app-shell-design.spec.ts` asserts a real
transition, so it now asks for `no-preference` itself.

The baseline manifest has `snapshotCount: 0`, so this invalidates no
published baseline. A frozen trial reference PNG can differ.

### Findings

These are product questions. I did not change them, and I did not hide
them with casts:

1. **`PrpEvent` and its schema disagree.** The TypeScript type in
`src/protocol/replay-contract.ts` requires `normalizedSessionId`. The
`required` list in `protocol/schemas/event.schema.json` does not include
it. `validatePrpEvent` accepts events that the type forbids.
2. **`requestCheckboxConfirmation` is stricter than its runtime.** It
uses the output type, so plugin authors must pass `continuationPolicy`
and `allowDeclineReason`, although the schema gives them defaults.
`z.input` would fix this.
3. **`RetainedRunnerdMaintenanceInput.backend.kind` is a free string.**
The quarantine domain comes from `descriptor.kind`, but the release uses
`proof.backend.kind`. If they differ,
`completeRetainedNativeSessionCleanup` returns 0 and the quarantine
stays. The production caller passes `"runner"` today, so this is not a
live bug.
4. **`CodexHarnessSession.close(input?)` is optional, but the contract
requires `{ reason }`.**

These are tests that verified nothing. I corrected them, and they pass:

- `backends/native-instruction-measurement.test.ts` filtered on
`placement === "core"`, which is not a valid placement. It measured an
empty tool list. It now measures 14 tools.
- `drivers/acpx/codex-acpx-driver.test.ts` checked for `usage.updated`,
which is not an event type.
- The `hostFixture()` helper in `drivers/acpx/runtime-host.test.ts`
ignored two overrides, so
`expect(prepareSandbox).not.toHaveBeenCalled()` could not fail.
- `live/live-session.test.ts` set `agent: "pi"` before a spread that
overwrote it.
- `native-distribution-integrity.test.ts` passed an AbortSignal where
`openCommand` expects `{ signal }`.

These are gaps that this pull request does not close:

- **The root lockfile still has some old zod versions.**
paperclipai/paperclip#17364 moved the direct zod dependencies to 4.6.x.
`pnpm-lock.yaml` still resolves `zod@3.25.76`, `4.1.11`, and `4.4.3`,
which are below the patched 4.6.6.
`packages/paperclip-runner/scripts/pi-distribution/package.json` pins
`zod` to `3.25.76`, and some agent SDKs resolve `4.4.3`. Dependency
review does not flag them, because this pull request does not add them.
- **Dead GitHub coverage.** `chat-adapters-ui-providers.spec.ts`
returned early for GitHub, so four later `provider === "github"`
branches never ran. I removed them. They included the private key file
and paste match, and the reconnect flow. Nothing asserts those behaviors
in that spec now.
- **Provider tests do not run in CI.** The root `vitest.config.ts` lists
only Daytona. The other 8 providers, `bridge-template` included, are
typechecked now, but CI does not run their tests.
- **One `packages/db` test never runs.**
`scripts/clean-poisoned-claude-sessions.test.ts` is outside the
`include` of `packages/db/vitest.config.ts`.
- **`plugin-orchestration-smoke-example` does not collect.**
`@paperclipai/shared` is not linked in its `node_modules`. It passes 3
of 3 with an alias.

## Verification

Run these commands:

```sh
node scripts/typecheck-test-programs.mjs --low-memory
pnpm --filter @paperclipai/server typecheck
node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations
node --test scripts/typecheck-test-programs.test.mjs
```

Results on macOS (arm64), Node 26.12. CI uses Node 24, and its
`Typecheck + Release Registry` job also passes. At head `7d4b5e907a`,
the test programs, `tests/e2e`, and the script tests pass again with 0
errors. The times and peaks below come from the run before the two
latest merges of master:

| Check | Result |
| --- | --- |
| Test programs, `--low-memory` | Exit 0, 0 errors, 42 seconds, peak
1.78 GiB |
| Server layers, low memory | Exit 0, 0 errors, 38 seconds, peak 3.41
GiB |
| Test projects, including `tests/e2e` | Exit 0, 0 errors, 13 seconds,
peak 1.59 GiB |
| Script tests | 12 passed |
| Injected type error in each of 23 places | Each one failed its program
with TS2322 |
| Coverage check | 0 unchecked files, 10 seconds |

I did not run this in a 4 GiB container. The peaks above are `maximum
resident set size` for the largest single process.

**Runtime check of the fixes.** Types do not prove that a test still
works, so I ran the suites before the two latest merges of master:

| Suite | Result |
| --- | --- |
| Runner, full suite | 4,885 passed, 3 failed, 13 skipped |
| Sandbox providers, 10 packages | 939 passed, 12 failed, 36 skipped |
| Plugin SDK | 200 passed |
| `kv-demo-mcp-server` | 12 passed |
| Renamed UI test and its sibling | 4 passed |

All 15 failures exist without these fixes or come from the local
machine:

- **3 Cursor Cloud timeouts.** `artifact-delivery.test.ts` has 1 and
`cursor-cloud-driver.test.ts` has 2. They time out in a full local run,
and all 133 tests in those files with `codex-security-config.test.ts`
pass when run alone. They fail in the same way with those two files at
their unedited versions.
- **12 Kubernetes `file-sync.test.ts` failures.** They fail in the same
way before and after the fixes. The first failure is that
`/proc/self/fd` does not exist on macOS.

After each of the two latest merges of master, I ran only the Runner
test files that I changed:

| Merge | Result |
| --- | --- |
| `885c249cb9`, 8 files | 534 passed |
| `7d4b5e907a`, 7 files | 672 passed, 16 failed |

The 16 failures are in the two `source-reconciliation` test files. They
fail in the same way with master's unedited versions of those files. 15
of them fail with `ACPX sidecar exited at initialize`, so I think my
local build of the Rust and sidecar binaries is out of date. I did not
rebuild them to check. CI builds them from scratch, and all Runner jobs
pass in CI at `7d4b5e907a`.

**Not run:**

- `tests/e2e` and the 8 Playwright suites need a server, so most of them
have only compile checks. `playwright test --list` loads 308 tests from
the main e2e config.
- `packages/plugins/sandbox-providers/kubernetes/test/integration` needs
a cluster, so it has only compile checks.
- I did not run the full `pnpm test:run` or root `pnpm build`. Required
CI runs them.

**Reduced motion.** I ran 6 motion-sensitive Storybook specs in system
Chrome with a temporary config, once with `reduce` and once with
`no-preference`. Both runs had 45 passed and the same 5 failed. So
reduced motion did not change those results. The 5 failures do not
depend on motion. At least one comes from the browser, because system
Chrome prints `oklch(55.6% 0 0)` where the test expects `oklch(0.556 0
0)`. I did not run the other Storybook specs. The `storybook-visual`
label runs that workflow.

## Risks

- **A new lockfile to maintain.** A change to a provider `package.json`
now fails the typecheck job until someone refreshes
`scripts/typecheck-sandbox-providers/pnpm-lock.yaml`. The failure
message gives the command.
- **The job has a new install.** The step downloads provider SDKs, so it
depends on the npm registry. It uses `--frozen-lockfile` and
`--ignore-scripts`.
- **The local install replaces provider `node_modules`.** A contributor
with a standalone provider install must reinstall that provider after
running the script.
- **Reduced motion can change Storybook results.** See "Behavior change
for review". That workflow is not part of required CI.
- **Kubernetes plugin validation changed.** A release of this plugin
ships zod 4. Error wording changes, and very large integers are now
rejected. See "Production change for review".
- **Master moves fast.** Each merge of master brought new test code with
type errors, 27 so far. After this merges, the check reports those
errors in the pull request that adds them.

## Model Used

- Provider: Anthropic Claude, through Claude Code
- Model: Claude Opus 5.5 (`claude-opus-5-5`)
- Capabilities: tool use, code execution, and subagents for the type
fixes in separate files

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived
details. The app generated the branch name.
- [ ] I have run tests locally and they pass. See the failures explained
in Verification.
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 07:33:59 -08:00

1279 lines
51 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { buildSlackAppManifest } from "../../packages/shared/src/slack-app-manifest";
import type { SlackAppConfiguration, SlackRegistrationState } from "../../packages/shared/src/index";
import { defaultGitHubReviewPolicy } from "../../packages/shared/src/types/chat-github";
import {
expect,
test,
type APIRequestContext,
type BrowserContext,
type Page,
type Route,
} from "@playwright/test";
/**
* Deterministic browser coverage for the native chat-connector UI.
*
* Provider APIs are deliberately not contacted here. The shared Paperclip
* server supplies the company, agent, and connector catalog, while a small
* stateful route fixture emulates the chat-control-plane responses. Live
* provider webhook and credential qualification belongs in the manual runbook
* because those checks require real accounts and publicly reachable ingress.
*/
export type Provider = "slack" | "github" | "discord" | "microsoft-teams" | "telegram" | "imessage-photon";
export const GITHUB_PRIVATE_KEY_FIXTURE =
"-----BEGIN PRIVATE KEY-----\nlocal-e2e-key\n-----END PRIVATE KEY-----\n";
export const GITHUB_PRIVATE_KEY_PASTE_FIXTURE =
"-----BEGIN PRIVATE KEY-----\nlocal-e2e-pasted-key\n-----END PRIVATE KEY-----\n";
export type ProviderCase = {
provider: Provider;
slug: string;
name: string;
accountLabel: string;
botLabel: string;
botUsername: string;
resourceLabel: string;
secondaryResourceLabel: string;
resourceType: string;
externalUrl: string;
setupHeading: RegExp;
setupButton: string;
chatAndTool: boolean;
};
export const PROVIDERS: ProviderCase[] = [
{
provider: "slack",
slug: "slack",
name: "Slack",
accountLabel: "Acme Workspace",
botLabel: "Maya",
botUsername: "maya-paperclip",
resourceLabel: "#product",
secondaryResourceLabel: "#support",
resourceType: "channel",
externalUrl: "https://app.slack.com/client/T-E2E/C-E2E/thread/C-E2E-1",
setupHeading: /Create a Slack app/i,
setupButton: "Connect Slack app",
chatAndTool: true,
},
{
provider: "github",
slug: "github-code-review-bot",
name: "GitHub",
accountLabel: "paperclip-ai",
botLabel: "Maya",
// GitHub's API returns the actor login with this suffix, while people
// invoke the App with the bare slug.
botUsername: "maya-paperclip[bot]",
resourceLabel: "paperclip-ai/paperclip",
secondaryResourceLabel: "paperclip-ai/chat-e2e",
resourceType: "repository",
externalUrl: "https://github.com/paperclip-ai/paperclip/issues/123",
setupHeading: /Create or connect a GitHub App/i,
setupButton: "Connect and verify",
chatAndTool: false,
},
{
provider: "microsoft-teams",
slug: "microsoft-teams",
name: "Microsoft Teams",
accountLabel: "Acme Tenant",
botLabel: "Maya",
botUsername: "maya-paperclip",
resourceLabel: "Product / General",
secondaryResourceLabel: "Product / Incidents",
resourceType: "channel",
externalUrl: "https://teams.microsoft.com/l/message/19:e2e@thread.tacv2/1",
setupHeading: /Connect Maya to Microsoft Teams/i,
setupButton: "Verify Microsoft credentials",
chatAndTool: false,
},
{
provider: "discord",
slug: "discord",
name: "Discord",
accountLabel: "Clawd",
botLabel: "Maya",
botUsername: "maya-paperclip",
resourceLabel: "#general",
secondaryResourceLabel: "#support",
resourceType: "channel",
externalUrl:
"https://discord.com/channels/1457808928258658549/1457808933082108089",
setupHeading: /Connect Maya to Discord/i,
setupButton: "Connect Discord bot",
chatAndTool: false,
},
{
provider: "telegram",
slug: "telegram",
name: "Telegram",
accountLabel: "@maya_paperclip_bot",
botLabel: "Maya",
botUsername: "maya_paperclip_bot",
resourceLabel: "Maya test chat",
secondaryResourceLabel: "Maya group chat",
resourceType: "direct_message",
externalUrl: "https://t.me/maya_paperclip_bot",
setupHeading: /Create Maya in Telegram/i,
setupButton: "Connect bot",
chatAndTool: false,
},
];
export const PROVIDER_LIFECYCLE_COPY: Record<
Provider,
{ reconnect: string; remove: string }
> = {
"imessage-photon": { reconnect: "Reconnect the same dedicated Photon number", remove: "does not delete the Photon project" },
slack: {
reconnect:
"Reconnect verifies or replaces credentials for this same Slack app. It does not reinstall the app or change its workspace or channel membership.",
remove: "It does not uninstall the Slack app",
},
github: {
reconnect:
"Reconnect verifies this same App and installation, then updates its webhook URL, secret, and secure delivery settings. It does not reinstall the App or change repository access.",
remove: "It does not uninstall the GitHub App",
},
discord: {
reconnect:
"Reconnect verifies this same Discord application and server installation. It does not add or remove the bot from the server.",
remove: "It does not uninstall the bot",
},
"microsoft-teams": {
reconnect:
"Reconnect verifies this same Microsoft app, tenant, and bot identity. It does not upload or reinstall the Teams app.",
remove: "It does not uninstall the Teams app",
},
telegram: {
reconnect:
"Reconnect verifies this same BotFather bot and automatically refreshes its Paperclip webhook and command menu.",
remove:
"queues durable removal of its Telegram webhook and command menu. After Telegram confirms that cleanup, Paperclip retires the saved token",
},
};
export type Seed = {
companyId: string;
prefix: string;
agentId: string;
otherAgentId: string;
};
export async function json<T>(
response: Awaited<ReturnType<APIRequestContext["get"]>>,
label: string,
): Promise<T> {
expect(
response.ok(),
`${label} failed ${response.status()}: ${await response.text()}`,
).toBe(true);
return (await response.json()) as T;
}
export async function seedCompanyAndAgent(request: APIRequestContext): Promise<Seed> {
const company = await json<{ id: string; issuePrefix: string }>(
await request.post("/api/companies", {
data: { name: `Chat adapters browser E2E ${Date.now()}` },
}),
"create company",
);
const agent = await json<{ id: string }>(
await request.post(`/api/companies/${company.id}/agents`, {
data: {
name: "Maya",
role: "qa",
title: "Chat connector test agent",
capabilities: "Exercises deterministic chat connector browser flows.",
adapterType: "process",
adapterConfig: {
command: process.execPath,
args: ["--input-type=module", "-e", "process.exit(0)"],
},
},
}),
"create agent",
);
const otherAgent = await json<{ id: string }>(
await request.post(`/api/companies/${company.id}/agents`, {
data: {
name: "Nora",
role: "qa",
title: "Second chat connector test agent",
capabilities: "Proves a chat connection keeps its chosen agent.",
adapterType: "process",
adapterConfig: {
command: process.execPath,
args: ["--input-type=module", "-e", "process.exit(0)"],
},
},
}),
"create second agent",
);
return {
companyId: company.id,
prefix: company.issuePrefix,
agentId: agent.id,
otherAgentId: otherAgent.id,
};
}
export function bodyOf(route: Route): Record<string, unknown> {
return route.request().postDataJSON() as Record<string, unknown>;
}
export async function fulfill(route: Route, body: unknown, status = 200) {
await route.fulfill({
status,
contentType: "application/json",
body: JSON.stringify(body),
});
}
export function endpointFixture(provider: ProviderCase, seed: Seed) {
const now = new Date().toISOString();
return {
id: `endpoint-${provider.provider}`,
companyId: seed.companyId,
connectionId: `connection-${provider.provider}`,
provider: provider.provider,
publicId: `public-${provider.provider}`,
status: "draft",
deploymentMode: "direct",
assignedAgentId: seed.agentId,
assignedAgentName: "Maya",
sponsorUserId: null,
providerAccountId: null,
providerAccountLabel: null,
botExternalId: null,
botUsername: null,
botLabel: null,
allowDirectMessages: false,
allowGroupChats: false,
allowUnlinkedPeople: false,
replyMode: "subscribed",
capabilities: {
threads: provider.provider !== "telegram",
directMessages: provider.provider !== "github",
nativeStreaming:
provider.provider === "slack" || provider.provider === "telegram",
messageEdits: true,
messageDeletes:
provider.provider === "slack" ||
provider.provider === "github" ||
provider.provider === "discord",
reactions: true,
files: true,
cards:
provider.provider === "slack" ||
provider.provider === "microsoft-teams",
actions: true,
modals: false,
slashCommands:
provider.provider !== "github" && provider.provider !== "discord",
ephemeralMessages:
provider.provider === "slack" ||
provider.provider === "microsoft-teams",
proactiveDirectMessages: false,
},
setup: {
slackSetupMethod: undefined as "managed" | "automatic" | "manual" | "existing" | undefined,
slackRegistration: undefined as SlackRegistrationState | undefined,
slackApp: undefined as SlackAppConfiguration | undefined,
slackAccount: undefined as Record<string, unknown> | undefined,
slackAvatar: undefined as Record<string, unknown> | undefined,
github: undefined as Record<string, unknown> | undefined,
slackOAuthCallbackUri: "https://paperclip.example.test/api/chat-slack/oauth/callback",
step: "provider_setup",
authorizationUrl:
provider.provider === "telegram"
? "https://t.me/BotFather"
: provider.provider === "microsoft-teams"
? "https://dev.teams.microsoft.com/apps"
: provider.provider === "github"
? "https://github.com/settings/apps"
: provider.provider === "discord"
? "https://discord.com/developers/applications"
: "https://api.slack.com/apps",
providerUrl: provider.externalUrl,
webhookUrl: `https://paperclip.example.test/api/chat-webhooks/public-${provider.provider}/${provider.provider}`,
messagingEndpoint: `https://paperclip.example.test/api/chat-webhooks/public-${provider.provider}/microsoft-teams`,
command: provider.provider === "slack" ? "/maya-public" : undefined,
testStartedAt: null as string | null,
webhookVerifiedAt: null as string | null,
webhookSecretConfigured: false,
},
healthMessage: null,
lastActivityAt: now,
lastPublicationAt: now,
activatedAt: null,
createdAt: now,
updatedAt: now,
};
}
export type ChatMock = {
slackCreations: number;
slackInstallations: number;
setSlackInstalled: () => void;
setSlackUncertain: () => void;
chatEndpointListReads: number;
createdWithAgentId: string | null;
configuredCredentialKeys: string[];
githubPrivateKeyMatchedFile: boolean | null;
githubPrivateKeyMatchedPaste: boolean | null;
githubSetupSecretRequests: number;
githubRepositoryRefreshes: number;
githubIdentityConfirmed: boolean;
setupAttempts: number;
updatedResource: boolean;
resourceUpdates: Array<Array<{ id: string; enabled: boolean }>>;
allowDirectMessages: boolean | null;
allowGroupChats: boolean | null;
allowUnlinkedPeople: boolean | null;
linkIntentPrincipalId: string | null;
revokedPrincipalId: string | null;
lifecycleActions: string[];
replayedDelivery: boolean;
liveActivitySummary: string | null;
conversationState: "active" | "waiting";
removed: boolean;
setStatus: (status: string) => void;
setWebhookVerified: () => void;
};
export async function installChatControlPlaneMock(
page: Page | BrowserContext,
provider: ProviderCase,
seed: Seed,
{
enableChatConnectors,
enableGitHubReviewBots = false,
resourceCount = 2,
photonShared = false,
automaticSlack = false,
managedSlack = false,
}: { enableChatConnectors: boolean; enableGitHubReviewBots?: boolean; resourceCount?: number; photonShared?: boolean; automaticSlack?: boolean; managedSlack?: boolean },
): Promise<ChatMock> {
const endpoint = endpointFixture(provider, seed);
if (automaticSlack || managedSlack) endpoint.setup.slackSetupMethod = managedSlack ? "managed" : "automatic";
let slackIdentityLinked = false;
let githubAppConnected = false;
let githubConfiguration = { revision: 0, configuration: {
version: 1, toolsEnabled: false, responsibleUserId: "local-board", memberAccess: "all_linked", people: [], defaults: defaultGitHubReviewPolicy(), repositories: {},
}};
const state: ChatMock & {
created: boolean;
failNextGitHubEndpointRead: boolean;
} = {
created: false,
slackCreations: 0, slackInstallations: 0,
setSlackInstalled: () => {
slackIdentityLinked = true;
endpoint.setup.slackAccount = { externalUserId: "UINSTALLER", paperclipUserId: "local-board", status: "linked", welcomeStatus: "sent", dmChannelId: "DE2E" };
endpoint.setup.slackRegistration = { status: "configured", appId: "AE2E", managementUrl: "https://api.slack.com/apps/AE2E" };
Object.assign(endpoint, { status: "verifying", providerAccountId: "TE2E", botExternalId: "UE2E", botUsername: "maya-paperclip" });
},
setSlackUncertain: () => { endpoint.setup.slackRegistration = { status: "uncertain", errorCode: "slack_creation_uncertain", managementUrl: "https://api.slack.com/apps" }; },
failNextGitHubEndpointRead: false,
chatEndpointListReads: 0,
createdWithAgentId: null,
configuredCredentialKeys: [],
githubPrivateKeyMatchedFile: null,
githubPrivateKeyMatchedPaste: null,
githubSetupSecretRequests: 0,
githubRepositoryRefreshes: 0,
githubIdentityConfirmed: false,
setupAttempts: 0,
updatedResource: false,
resourceUpdates: [],
allowDirectMessages: null,
allowGroupChats: null,
allowUnlinkedPeople: null,
linkIntentPrincipalId: null,
revokedPrincipalId: null,
lifecycleActions: [],
replayedDelivery: false,
liveActivitySummary: null,
conversationState: "active",
removed: false,
setStatus: (status) => {
endpoint.status = status;
},
setWebhookVerified: () => {
endpoint.setup.webhookVerifiedAt = new Date().toISOString();
},
};
const resource = {
id: `resource-${provider.provider}`,
companyId: seed.companyId,
endpointId: endpoint.id,
type: provider.resourceType,
providerResourceId: `provider-resource-${provider.provider}`,
label: provider.resourceLabel,
detail:
provider.provider === "github" ? "Repository" : "Available at provider",
providerUrl: provider.externalUrl,
availability: "available",
enabled: false,
createdAt: endpoint.createdAt,
updatedAt: endpoint.updatedAt,
};
const secondaryResource = {
...resource,
id: `resource-${provider.provider}-secondary`,
providerResourceId: `provider-resource-${provider.provider}-secondary`,
label: provider.secondaryResourceLabel,
detail:
provider.provider === "github" ? "Repository" : "Available at provider",
};
const resources = [
resource,
secondaryResource,
...Array.from({ length: Math.max(0, resourceCount - 2) }, (_, index) => ({
...resource,
id: `resource-${provider.provider}-extra-${index}`,
providerResourceId: `provider-resource-${provider.provider}-extra-${index}`,
label: `Extra destination ${index + 1}`,
})),
];
await page.route("**/api/**", async (route) => {
const request = route.request();
const pathname = new URL(request.url()).pathname;
const method = request.method();
if (provider.provider === "github" && pathname.startsWith(`/api/chat-endpoints/${endpoint.id}/github/`)) {
const operation = pathname.split("/github/")[1];
const body = method === "GET" ? {} : bodyOf(route);
if (operation === "setup") {
if (!githubAppConnected) { await fulfill(route, { endpointId: endpoint.id, state: "create" }); return; }
if (!state.githubIdentityConfirmed) { await fulfill(route, { endpointId: endpoint.id, state: "identity", identityMethod: "existing_connection" }); return; }
githubConfiguration.configuration.toolsEnabled = true;
resource.enabled = true;
Object.assign(endpoint, { status: "active", botExternalId: "123456", resources: [resource], setup: { ...endpoint.setup, step: "complete" } });
await fulfill(route, { endpointId: endpoint.id, state: "connected", identityLinked: true, verification: { ready: true, checks: [] } }); return;
}
if (operation === "draft") {
Object.assign(endpoint.setup, { github: { ...endpoint.setup.github, appName: body.name, ownerType: body.ownerType, ownerLogin: body.ownerLogin } });
await fulfill(route, { saved: true }); return;
}
if (operation === "configuration") {
if (method === "PUT") { expect(body.expectedRevision).toBe(githubConfiguration.revision); githubConfiguration = { revision: githubConfiguration.revision + 1, configuration: body.configuration as typeof githubConfiguration.configuration }; }
await fulfill(route, githubConfiguration); return;
}
if (operation === "progress") {
Object.assign(endpoint.setup, { github: { appSlug: githubAppConnected ? "maya-paperclip" : undefined, stage: body.stage, managementUrl: "https://github.com/settings/installations/2468" } });
await fulfill(route, endpoint); return;
}
if (operation === "app") {
state.setupAttempts++;
state.configuredCredentialKeys = Object.keys(body).sort();
if (state.setupAttempts === 1) { await fulfill(route, { error: "GitHub rejected the supplied App credentials." }, 422); return; }
githubAppConnected = true;
Object.assign(endpoint, { status: "attention", botExternalId: "123456", botUsername: "maya-paperclip[bot]", botLabel: "Maya" });
Object.assign(endpoint.setup, { github: { stage: "install", appSlug: "maya-paperclip", installationUrl: "https://github.com/apps/maya-paperclip/installations/new", managementUrl: "https://github.com/settings/installations/2468" } });
await fulfill(route, endpoint); return;
}
if (operation === "repositories") {
const url = new URL(request.url());
const offset = Number(url.searchParams.get("offset") ?? 0), limit = Number(url.searchParams.get("limit") ?? 20);
const search = (url.searchParams.get("search") ?? "").toLowerCase();
const found = resources.filter(row => (row.label ?? row.providerResourceId).toLowerCase().includes(search));
await fulfill(route, { items: found.slice(offset, offset + limit), nextOffset: offset + limit < found.length ? offset + limit : null,
totalCount: resources.length, enabledCount: resources.filter(row => row.enabled).length, availableCount: resources.length }); return;
}
if (operation === "repositories/refresh") { state.githubRepositoryRefreshes++; await fulfill(route, resources); return; }
if (operation === "verify") {
const ready = githubConfiguration.configuration.toolsEnabled;
await fulfill(route, { ready, checks: [{ key: "tools", label: "Assigned agent’s effective GitHub tools", ok: ready, detail: ready ? "The bot tools are assigned." : "Assign this bot’s GitHub tools." }] }); return;
}
if (operation === "personal-connections") { await fulfill(route, [{ connectionId: "personal-github", name: "My GitHub", login: "octocat", enabled: true, status: "active" }]); return; }
if (operation === "identity") {
expect(body.connectionId).toBe("personal-github");
if (body.confirmedGithubUserId) { expect(body.confirmedGithubUserId).toBe("42"); state.githubIdentityConfirmed = true; }
await fulfill(route, { githubUserId: "42", login: "octocat", connectionId: "personal-github", avatarUrl: null }); return;
}
if (operation === "reviews") { await fulfill(route, []); return; }
}
if (pathname === `/api/companies/${seed.companyId}/chat-endpoints`) {
if (method === "GET") {
state.chatEndpointListReads += 1;
await fulfill(route, { endpoints: state.created ? [endpoint] : [] });
return;
}
if (method === "POST") {
const body = bodyOf(route);
expect(body).toMatchObject({
provider: provider.provider,
assignedAgentId: seed.agentId,
});
state.createdWithAgentId = String(body.assignedAgentId);
if (body.slackSetupMethod) endpoint.setup.slackSetupMethod = body.slackSetupMethod as "managed" | "automatic" | "manual" | "existing";
if (provider.provider === "slack" && body.slackApp) {
endpoint.setup.slackApp = body.slackApp as SlackAppConfiguration;
endpoint.setup.command = endpoint.setup.slackApp.command;
}
state.created = true;
await fulfill(route, endpoint, 201);
return;
}
}
if (pathname === `/api/chat-endpoints/${endpoint.id}`) {
if (method === "GET") {
if (
provider.provider === "github" &&
state.failNextGitHubEndpointRead
) {
state.failNextGitHubEndpointRead = false;
await fulfill(route, { error: "Temporary read failure." }, 503);
return;
}
await fulfill(route, endpoint);
return;
}
if (method === "PATCH") {
const body = bodyOf(route);
if (body.slackApp) endpoint.setup.slackApp = body.slackApp as SlackAppConfiguration;
if (body.slackSetupMethod) endpoint.setup.slackSetupMethod = body.slackSetupMethod as "managed" | "automatic" | "manual" | "existing";
if (typeof body.allowDirectMessages === "boolean")
state.allowDirectMessages = body.allowDirectMessages;
if (typeof body.allowGroupChats === "boolean")
state.allowGroupChats = body.allowGroupChats;
if (typeof body.allowUnlinkedPeople === "boolean")
state.allowUnlinkedPeople = body.allowUnlinkedPeople;
Object.assign(endpoint, body, {
updatedAt: new Date().toISOString(),
});
await fulfill(route, endpoint);
return;
}
}
if (
pathname === "/api/instance/settings/experimental" &&
method === "GET"
) {
await fulfill(route, {
enableChatConnectors,
enableGitHubReviewBots,
enableIsolatedWorkspaces: false,
});
return;
}
if (pathname === `/api/companies/${seed.companyId}/chat-slack/setup-options`) {
await fulfill(route, { managedAvailable: managedSlack, defaultMethod: managedSlack ? "managed" : "automatic", workspaces: managedSlack ? [{ grantId: "11111111-1111-4111-8111-111111111111", workspaceId: "TE2E", workspaceName: "Paperclip", userId: "UINSTALLER" }] : [] }); return;
}
if (pathname === `/api/chat-endpoints/${endpoint.id}/slack/managed/provision`) {
const body = bodyOf(route);
expect(body.grantId).toBe("11111111-1111-4111-8111-111111111111");
expect(body).not.toHaveProperty("credentials");
state.slackCreations++; state.slackInstallations++;
state.setSlackInstalled(); await fulfill(route, {}); return;
}
if (pathname === `/api/chat-endpoints/${endpoint.id}/slack/registration`) {
const body = bodyOf(route);
expect(body.credentials).toEqual({ configurationToken: "fixture-config-token" });
expect(body).not.toHaveProperty("manifest");
state.slackCreations++;
endpoint.setup.slackRegistration = { status: "install", appId: "AE2E", managementUrl: "https://api.slack.com/apps/AE2E" };
endpoint.setup.slackAvatar = { status: "uploaded", uploadedAt: new Date().toISOString() };
await fulfill(route, endpoint); return;
}
if (pathname === `/api/chat-endpoints/${endpoint.id}/slack/install`) {
state.slackInstallations++;
await fulfill(route, { authorizationUrl: "https://slack.test/consent", expiresAt: new Date(Date.now() + 600_000).toISOString() }); return;
}
if (pathname === `/api/chat-endpoints/${endpoint.id}/slack/resume`) {
state.setSlackInstalled(); await fulfill(route, endpoint); return;
}
if (
pathname === `/api/chat-endpoints/${endpoint.id}/setup-secret` &&
method === "POST"
) {
state.githubSetupSecretRequests += 1;
endpoint.setup.webhookSecretConfigured = true;
endpoint.setup.step = "provider_setup";
endpoint.setup.webhookVerifiedAt =
state.githubSetupSecretRequests === 1 ? new Date().toISOString() : null;
if (state.githubSetupSecretRequests > 1) {
state.failNextGitHubEndpointRead = true;
}
await fulfill(route, { webhookSecret: "github-webhook-secret" }, 201);
return;
}
if (
pathname === `/api/chat-endpoints/${endpoint.id}/setup` &&
method === "POST"
) {
const body = bodyOf(route);
const action = String(body.action);
if (action === "pause" || action === "resume" || action === "remove") {
state.lifecycleActions.push(action);
if (action === "pause") endpoint.status = "paused";
if (action === "resume") endpoint.status = "active";
if (action === "remove") {
endpoint.status = "archived";
state.created = false;
state.removed = true;
}
await fulfill(route, endpoint);
return;
}
expect(["configure", "verify", "reconnect"]).toContain(action);
if (body.action === "configure") {
state.setupAttempts += 1;
if (body.credentials) state.configuredCredentialKeys = Object.keys(body.credentials as Record<string, string>).sort();
if (provider.provider === "github" && !githubAppConnected) {
const privateKey = (
(body.credentials ?? {}) as Record<string, string>
).privateKey;
state.githubPrivateKeyMatchedFile =
state.githubPrivateKeyMatchedFile === true ||
privateKey === GITHUB_PRIVATE_KEY_FIXTURE;
state.githubPrivateKeyMatchedPaste =
state.githubPrivateKeyMatchedPaste === true ||
privateKey === GITHUB_PRIVATE_KEY_PASTE_FIXTURE;
if (state.setupAttempts === 1) {
await fulfill(
route,
{ error: "GitHub rejected the supplied App credentials." },
422,
);
return;
}
}
if (provider.provider === "telegram" && state.setupAttempts === 1) {
const submittedToken = String(
((body.credentials ?? {}) as Record<string, string>).botToken ?? "",
);
await fulfill(
route,
{
error: `Telegram rejected bot token ${submittedToken}. Confirm the token in BotFather and try again.`,
},
422,
);
return;
}
} else if (action !== "reconnect") {
expect(provider.provider).toBe("slack");
}
if (provider.provider === "imessage-photon") {
expect(body.photon).toEqual(photonShared ? {allocation:"shared",projectId:"project-e2e"} : {allocation:"dedicated",projectId:"project-e2e",lineId:"line-one"});
}
// Verification advances setup without replacing the identity captured by installation.
if (provider.provider === "slack" && action === "verify") {
endpoint.setup.step = "test";
endpoint.setup.testStartedAt = new Date().toISOString();
await fulfill(route, endpoint);
return;
}
Object.assign(endpoint, {
...(provider.provider === "imessage-photon" ? {photonAllocation: photonShared ? "shared" : "dedicated", allowGroupChats: !photonShared} : {}),
status: "verifying",
providerAccountId: `account-${provider.provider}`,
providerAccountLabel: provider.accountLabel,
botExternalId: provider.provider === "imessage-photon" ? photonShared ? "photon-project:project-e2e" : "+15555550100" : `bot-${provider.provider}`,
botUsername: photonShared ? null : provider.botUsername,
botLabel: provider.botLabel,
setup: {
...endpoint.setup,
testStartedAt: body.action === "verify" ? new Date().toISOString() : endpoint.setup.testStartedAt,
step:
provider.provider === "slack" && body.action === "configure"
? "provider_setup"
: "test",
},
});
await fulfill(route, endpoint);
return;
}
if (pathname === `/api/chat-endpoints/${endpoint.id}/photon/inspect` && method === "POST") {
expect(bodyOf(route)).toEqual({projectId:"project-e2e",projectSecret:"photon-test-secret"});
await fulfill(route,{projectId:"project-e2e",projectName:"Photon Test",allocation:photonShared ? "shared" : "dedicated",eligible:true,lines:photonShared ? [] : [{lineId:"line-one",phoneNumber:"+15555550100",eligible:true},{lineId:"line-two",phoneNumber:"+15555550102",eligible:true}]}); return;
}
if (
(pathname === `/api/chat-endpoints/${endpoint.id}/test` || pathname === `/api/chat-endpoints/${endpoint.id}/finish`) &&
method === "POST"
) {
Object.assign(endpoint, {
status: "active",
activatedAt: new Date().toISOString(),
setup: { ...endpoint.setup, step: "complete" },
});
await fulfill(route, endpoint);
return;
}
if (pathname === `/api/chat-endpoints/${endpoint.id}/test-status`) {
await fulfill(route, { messageReceivedAt: null });
return;
}
if (pathname === `/api/chat-endpoints/${endpoint.id}/principals/principal-slack-self/link-intent` && method === "POST") {
await fulfill(route, { confirmationUrl: `https://paperclip.example.test/${seed.prefix}/chat-identity/confirm?token=e2e-self` });
return;
}
if (pathname === "/api/chat-identity-links/confirm" && method === "POST") {
slackIdentityLinked = true;
await fulfill(route, { endpointId: endpoint.id, companyId: seed.companyId });
return;
}
if (pathname === `/api/chat-endpoints/${endpoint.id}/resources`) {
if (method === "GET") {
await fulfill(route, { resources });
return;
}
if (method === "PUT") {
const updates = (bodyOf(route).resources ?? []) as Array<{
id: string;
enabled: boolean;
}>;
state.resourceUpdates.push(updates);
// Match the real request-size boundary without contacting a provider.
if (updates.length > 500) {
await fulfill(route, { error: "Too many resource updates." }, 400);
return;
}
for (const update of updates) {
const target = resources.find((item) => item.id === update.id);
if (target) target.enabled = update.enabled;
}
state.updatedResource = resource.enabled;
await fulfill(route, resources);
return;
}
}
if (
pathname === `/api/chat-endpoints/${endpoint.id}/principals` &&
method === "GET"
) {
await fulfill(route, {
principals: [
...(provider.provider === "slack" && (slackIdentityLinked || endpoint.setup.step === "test") ? [{
id: "link-slack-self", principalId: "principal-slack-self", externalLabel: "Test operator",
externalDetail: "operator@slack", lastConnectAt: new Date().toISOString(),
paperclipUserId: slackIdentityLinked ? "local-board" : null,
status: slackIdentityLinked ? "linked" : "pending",
}] : []),
{
id: `link-${provider.provider}`,
principalId: `principal-${provider.provider}`,
externalLabel: "Ada Lovelace",
externalDetail: `ada@${provider.provider}`,
paperclipUserId: null,
paperclipUserLabel: null,
status: "pending",
},
{
id: `link-${provider.provider}-linked`,
principalId: `principal-${provider.provider}-linked`,
externalLabel: "Grace Hopper",
externalDetail: `grace@${provider.provider}`,
paperclipUserId:
state.revokedPrincipalId ===
`principal-${provider.provider}-linked`
? null
: "paperclip-user-grace",
paperclipUserLabel:
state.revokedPrincipalId ===
`principal-${provider.provider}-linked`
? null
: "Grace Hopper",
status:
state.revokedPrincipalId ===
`principal-${provider.provider}-linked`
? "revoked"
: "linked",
},
],
});
return;
}
if (
pathname ===
`/api/chat-endpoints/${endpoint.id}/principals/principal-${provider.provider}/link-intent` &&
method === "POST"
) {
state.linkIntentPrincipalId = `principal-${provider.provider}`;
await fulfill(route, {
confirmationUrl: `https://paperclip.example.test/${seed.prefix}/chat-identity/confirm?token=e2e-redacted`,
});
return;
}
if (
pathname ===
`/api/chat-endpoints/${endpoint.id}/principals/principal-${provider.provider}-linked/link` &&
method === "DELETE"
) {
state.revokedPrincipalId = `principal-${provider.provider}-linked`;
await route.fulfill({ status: 204, body: "" });
return;
}
if (
pathname === `/api/chat-endpoints/${endpoint.id}/conversations` &&
method === "GET"
) {
await fulfill(route, {
conversations: [
{
id: `conversation-${provider.provider}`,
companyId: seed.companyId,
endpointId: endpoint.id,
resourceId: resource.id,
issueId: `issue-${provider.provider}`,
issueIdentifier: "CHAT-123",
issueTitle: `Investigate ${provider.name} delivery`,
externalConversationId: `external-conversation-${provider.provider}`,
externalThreadId: `external-thread-${provider.provider}`,
externalLabel: provider.resourceLabel,
externalUrl: provider.provider === "imessage-photon" ? null : provider.externalUrl,
isDirectMessage: provider.provider === "telegram",
state: state.conversationState,
lastPublicationStatus: "published",
createdAt: endpoint.createdAt,
updatedAt: endpoint.updatedAt,
},
],
});
return;
}
if (
pathname === `/api/chat-endpoints/${endpoint.id}/activity` &&
method === "GET"
) {
await fulfill(route, {
items: [
...(state.liveActivitySummary
? [
{
id: `live-reaction-${provider.provider}`,
kind: "delivery",
status: "processed",
summary: state.liveActivitySummary,
createdAt: endpoint.createdAt,
replayable: false,
},
]
: []),
{
id: `delivery-${provider.provider}`,
kind: "delivery",
status: "failed",
summary: `Inbound ${provider.name} delivery could not be processed`,
detail: "Credential values and request bodies are redacted.",
createdAt: endpoint.createdAt,
replayable: true,
},
{
id: `publication-${provider.provider}`,
kind: "publication",
status: "published",
summary: `Published safe output to ${provider.name}`,
createdAt: endpoint.createdAt,
replayable: false,
},
],
});
return;
}
if (
pathname ===
`/api/chat-endpoints/${endpoint.id}/deliveries/delivery-${provider.provider}/replay` &&
method === "POST"
) {
state.replayedDelivery = true;
await fulfill(route, {});
return;
}
await route.continue();
});
return state;
}
export async function selectMaya(page: Page) {
await page.getByRole("button", { name: "Choose an active agent" }).click();
await page.getByRole("button", { name: "Select Maya" }).click();
await page.getByRole("button", { name: "Continue", exact: true }).click();
}
export async function fillProviderSetup(page: Page, provider: ProviderCase) {
if (provider.provider === "slack") {
await page.getByRole("button", { name: "I already created the app" }).click();
await expect(page.getByLabel("Bot User OAuth Token")).toHaveAttribute("type", "password");
await expect(page.getByLabel("Signing Secret")).toHaveAttribute("type", "password");
await page.getByLabel("Bot User OAuth Token").fill("xoxb-e2e-redacted");
await page.getByLabel("Signing Secret").fill("slack-signing-secret");
} else if (provider.provider === "github") {
await page.getByRole("button", { name: "Generate webhook secret" }).click();
await expect(page.getByLabel("Generated webhook secret")).toHaveValue(
"github-webhook-secret",
);
await page.getByLabel("GitHub App ID").fill("123456");
const privateKeyFile = page.getByLabel(
"Choose GitHub App private key file",
);
await privateKeyFile.setInputFiles({
name: "paperclip-test.pem",
mimeType: "application/x-pem-file",
buffer: Buffer.alloc(64 * 1024 + 1, "x"),
});
await expect(page.getByRole("alert")).toContainText(
"That file is too large. Choose a GitHub App private key smaller than 64 KB.",
);
await page.getByLabel("GitHub App ID").focus();
await page.getByLabel("GitHub App ID").press("Tab");
await expect(page.getByRole("alert")).toBeVisible();
const fileChooserPromise = page.waitForEvent("filechooser");
await page.getByRole("button", { name: "Choose .pem file" }).click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles({
name: "paperclip-test.pem",
mimeType: "application/x-pem-file",
buffer: Buffer.from(GITHUB_PRIVATE_KEY_FIXTURE),
});
await expect(page.getByRole("alert")).toHaveCount(0);
await expect(page.getByRole("status")).toContainText(
"Private key loaded. It stays in this form until you connect.",
);
await expect(
page.getByRole("button", { name: "Show private key" }),
).toBeVisible();
await expect(page.getByLabel("Private key (PEM)")).toHaveAttribute(
"type",
"password",
);
await expect(page.locator("textarea#github-private-key")).toHaveCount(0);
await expect(page.locator("body")).not.toContainText(
GITHUB_PRIVATE_KEY_FIXTURE,
);
await expect(page.locator("body")).not.toContainText("paperclip-test.pem");
await page.getByRole("button", { name: provider.setupButton }).click();
await expect(page.getByRole("alert")).toContainText("Connection failed");
await page
.getByLabel("Private key (PEM)")
.evaluate((element, privateKey) => {
const clipboardData = new DataTransfer();
clipboardData.setData("text/plain", privateKey);
element.dispatchEvent(
new ClipboardEvent("paste", {
bubbles: true,
cancelable: true,
clipboardData,
}),
);
}, GITHUB_PRIVATE_KEY_PASTE_FIXTURE);
await expect(page.locator("textarea#github-private-key")).toHaveCount(0);
await expect(page.locator("body")).not.toContainText(
GITHUB_PRIVATE_KEY_PASTE_FIXTURE,
);
await page.getByRole("button", { name: "Show private key" }).click();
await expect(page.locator("textarea#github-private-key")).toHaveValue(
GITHUB_PRIVATE_KEY_PASTE_FIXTURE,
);
await page.getByRole("button", { name: "Hide private key" }).click();
await expect(page.locator("textarea#github-private-key")).toHaveCount(0);
await expect(page.getByLabel("Private key (PEM)")).toHaveAttribute(
"type",
"password",
);
await expect(page.locator("body")).not.toContainText(
GITHUB_PRIVATE_KEY_PASTE_FIXTURE,
);
} else if (provider.provider === "microsoft-teams") {
const clientId = "00000000-0000-4000-8000-000000000001";
await page.getByLabel("Application / Client ID").fill(clientId);
await page
.getByLabel("Directory / Tenant ID")
.fill("00000000-0000-4000-8000-000000000002");
await page.getByLabel("Client secret").fill("teams-client-secret");
const manifest = JSON.parse(
await page.getByLabel("Required Teams app manifest block").inputValue(),
) as { webApplicationInfo?: { id?: string; resource?: string } };
expect(manifest.webApplicationInfo).toEqual({
id: clientId,
resource: "https://paperclip.ing",
});
await expect(
page.getByRole("button", { name: "Copy manifest settings" }),
).toBeEnabled();
} else if (provider.provider === "discord") {
await page.getByLabel("Application ID").fill("1457808928258658549");
await page.getByLabel("Server ID").fill("1457808928258658549");
await page.getByLabel("Bot token").fill("discord-e2e-redacted");
} else {
await page.getByLabel("Bot token").fill("123456:e2e-redacted");
}
await page.getByRole("button", { name: provider.setupButton }).click();
}
export function expectedCredentialKeys(provider: Provider): string[] {
if (provider === "slack") return ["botToken", "signingSecret"];
if (provider === "github") return ["appId", "privateKey"];
if (provider === "microsoft-teams")
return ["clientId", "clientSecret", "tenantId"];
if (provider === "discord") return ["applicationId", "botToken", "guildId"];
return ["botToken"];
}
export async function expectSetupRail(page: Page, newDraft = false) {
const rail = page.getByRole("navigation", { name: "Connection setup progress" });
await expect(rail).toBeVisible();
const labels = new URL(page.url()).searchParams.get("provider") === "slack"
? newDraft
? ["Choose agent", "App configuration access token", "Install Slack app", "Send a message"]
: ["Choose agent", "Create Slack app", "Add credentials", "Verify Slack connection", "Connect your Slack account", "Try it"]
: ["Choose agent", "Connect provider", "Try it"];
await expect(rail.getByRole("listitem")).toHaveCount(labels.length);
for (const label of labels) {
await expect(rail.getByText(label, { exact: true })).toBeVisible();
}
}
export function expectedSlackManifest(webhookUrl: string) {
return JSON.stringify(buildSlackAppManifest({ app: { appName: "maya-paperclip", botName: "maya", command: "/maya" }, agentName: "Maya", webhookUrl }), null, 2);
}
export async function expectMinimumProviderSetup(page: Page, provider: ProviderCase) {
const webhookUrl = `https://paperclip.example.test/api/chat-webhooks/public-${provider.provider}/${provider.provider}`;
if (provider.provider === "slack") {
const steps = page.getByRole("navigation", { name: "Connection setup progress" });
await steps.getByRole("button", { name: /Choose agent/ }).click();
await page.locator("summary:visible").filter({ hasText: "Advanced" }).click();
await expect(page.getByLabel("Slack app name", { exact: true })).toBeEditable();
await expect(page.getByLabel("Bot display name", { exact: true })).toBeEditable();
await expect(page.getByLabel("Slash command", { exact: true })).toBeEditable();
await page.getByRole("button", { name: "Continue", exact: true }).click();
await page.getByRole("button", { name: "View Slack App Manifest" }).click();
const manifest = page.getByRole("textbox", { name: "Slack app manifest", exact: true });
await expect(manifest).toHaveValue(expectedSlackManifest(webhookUrl));
await expect(manifest).toHaveAttribute("readonly", "");
await page.keyboard.press("Escape");
await expect(page.getByRole("button", { name: "Create Slack app", exact: true })).toBeVisible();
return;
}
if (provider.provider === "github") {
await expect(page.getByText(webhookUrl, { exact: true })).toBeVisible();
await expect(page.getByText(/Metadata remains read-only/)).toBeVisible();
await expect(page.getByText(/issue_comment/)).toBeVisible();
await expect(page.getByText(/pull_request_review_comment/)).toBeVisible();
await expect(page.getByText(/generate one private key/)).toBeVisible();
await expect(page.getByText(/Enable SSL verification/)).toBeVisible();
await expect(page.getByText(/Only on this account/)).toBeVisible();
await expect(page.getByLabel("GitHub App ID")).toHaveAttribute(
"type",
"text",
);
await expect(
page.getByRole("button", { name: "Generate webhook secret" }),
).toBeVisible();
await expect(
page.getByRole("button", { name: "Open new GitHub App form" }),
).toBeVisible();
return;
}
if (provider.provider === "microsoft-teams") {
const messagingEndpoint =
"https://paperclip.example.test/api/chat-webhooks/public-microsoft-teams/microsoft-teams";
await expect(
page.getByText(messagingEndpoint, { exact: true }),
).toBeVisible();
await expect(
page.getByText(/single-tenant app registration/),
).toBeVisible();
await expect(
page.getByText(/Microsoft 365 work or school organization/),
).toBeVisible();
await expect(page.getByText(/teams\.live\.com/)).toBeVisible();
await expect(
page.getByText(/^In Azure, create an Azure Bot/),
).toBeVisible();
await expect(page.getByText("Microsoft portal field map")).toBeVisible();
await expect(
page.getByText(/Accounts in this organizational directory only/),
).toBeVisible();
await expect(page.getByText(/Use existing app registration/)).toBeVisible();
await expect(
page.getByText(/Configure · App features · Bot/),
).toBeVisible();
await expect(
page.getByText(/Upload an app · Upload a custom app/),
).toBeVisible();
const manifest = await page
.getByLabel("Required Teams app manifest block")
.inputValue();
expect(manifest).toContain("ChannelMessage.Read.Group");
expect(manifest).toContain("ChatMessage.Read.Chat");
expect(manifest).toContain('"personal"');
expect(manifest).toContain('"team"');
expect(manifest).toContain('"groupChat"');
expect(JSON.parse(manifest)).toMatchObject({
bots: [
{
commandLists: [
{
scopes: ["personal", "groupChat"],
commands: [
{ title: "/status" },
{ title: "/new" },
{ title: "/close" },
],
},
],
},
],
webApplicationInfo: {
id: "<application-client-id>",
resource: "https://paperclip.ing",
},
});
expect(manifest).not.toContain("api://paperclip-chat/");
expect(manifest).not.toContain("supportsTargetedMessages");
await expect(
page.getByRole("button", { name: "Copy manifest settings" }),
).toBeDisabled();
await expect(page.getByText(/not a complete app package/)).toBeVisible();
await expect(
page.getByText(/does not use Teams single sign-on/),
).toBeVisible();
await expect(
page.getByText(/only associates the RSC permissions/),
).toBeVisible();
await expect(page.getByText(/receive every message/).first()).toBeVisible();
await expect(
page.getByText(/One team install covers its standard channels/),
).toBeVisible();
await expect(
page.getByRole("link", { name: "Open Microsoft Entra" }),
).toHaveAttribute("href", /entra\.microsoft\.com/);
await expect(
page.getByRole("link", { name: "Create Azure Bot" }),
).toHaveAttribute("href", /portal\.azure\.com/);
await expect(
page.getByRole("link", { name: "Open Teams Developer Portal" }),
).toHaveAttribute("href", /dev\.teams\.microsoft\.com/);
await expect(page.getByLabel("Application / Client ID")).toHaveAttribute(
"type",
"text",
);
await expect(page.getByLabel("Directory / Tenant ID")).toHaveAttribute(
"type",
"text",
);
await expect(page.getByLabel("Client secret value")).toHaveAttribute(
"type",
"password",
);
return;
}
if (provider.provider === "discord") {
await expect(page.getByText(/enable Message Content Intent/)).toBeVisible();
await expect(page.getByText(/copy its Server ID/)).toBeVisible();
await expect(page.getByLabel("Application ID")).toHaveAttribute(
"type",
"text",
);
await expect(page.getByLabel("Server ID")).toHaveAttribute("type", "text");
await expect(page.getByLabel("Bot token")).toHaveAttribute(
"type",
"password",
);
await expect(
page.getByRole("button", { name: "Open Discord Developer Portal" }),
).toBeVisible();
await page.getByLabel("Application ID").fill("1457808928258658549");
await page.getByLabel("Server ID").fill("1457808928258658549");
await expect(
page.getByRole("link", { name: "Install bot in this server" }),
).toHaveAttribute(
"href",
/client_id=1457808928258658549&permissions=309237763136&scope=bot&guild_id=1457808928258658549&disable_guild_select=true/,
);
await expect(page.getByText(/Create Public Threads/)).toBeVisible();
return;
}
await expect(page.getByText("/newbot", { exact: true })).toBeVisible();
await expect(page.getByText(/username ending in/)).toBeVisible();
await expect(page.getByText(/\/task@bot_username/)).toBeVisible();
await expect(
page.getByText(/ordinary mentions are not delivered/),
).toBeVisible();
await expect(page.getByLabel("Bot token")).toHaveAttribute(
"type",
"password",
);
await expect(
page.getByRole("button", { name: "Open BotFather" }),
).toBeVisible();
}
export async function expectProviderTryInstructions(
page: Page,
provider: ProviderCase,
) {
if (provider.provider === "slack") {
for (const text of ["Open a channel and invite @maya-paperclip if needed.", "@maya-paperclip you there?", "Continue the conversation in the thread."]) {
await expect(page.getByText(text, { exact: true })).toBeVisible();
}
await expect(page.getByRole("button", { name: "Copy message" })).toBeVisible();
await expect(page.getByRole("link", { name: "Open Slack", exact: true })).toHaveCount(0);
return;
}
const expected =
provider.provider === "github"
? [
"Open an installed issue or pull request.",
"Mention @maya-paperclip in a comment.",
"Add another comment to continue the same task.",
]
: provider.provider === "microsoft-teams"
? [
"Open an installed channel and start a new post.",
"Mention @maya-paperclip in the post.",
"Reply once beneath the post.",
]
: provider.provider === "discord"
? [
"Open a text channel where the bot is installed.",
"Mention @maya-paperclip in a new root message.",
"Reply once inside Maya's new Discord thread.",
]
: [
"Open the bot's private chat.",
"Tap Start.",
"Send “Help me test this”.",
];
for (const instruction of expected) {
await expect(page.getByText(instruction, { exact: true })).toBeVisible();
}
await expect(
page.getByRole("link", { name: `Open ${provider.name}` }),
).toBeVisible();
await expect(
page.getByRole("link", { name: `Open ${provider.name}` }),
).toHaveAttribute("href", provider.externalUrl);
}