Files
PaperClipAI/ui/storybook/stories/github-chat.stories.tsx
DottaandPaperclip 8813a50105 feat: run GitHub review bots through Paperclip agents (#13717)
## Thinking Path

> - Paperclip manages agent work as tasks and runs.
> - GitHub chat brings repository conversations into those tasks.
> - A review bot needs the assigned agent, its authority, and governed
provider tools.
> - The existing channel connection did not supply that review workflow
or a complete setup journey.
> - This pull request adds GitHub App setup, account access, event
prompts, task-bound review tools, and exact-commit checks.
> - Operators can inspect each review through the same task, run, and
activity systems.

## Linked Issues or Issue Description

**Subsystem affected**

GitHub chat, governed connection tools, task execution, shared/database
contracts, and connector setup UI.

**Problem or motivation**

Operators need a GitHub review bot that runs their assigned Paperclip
agent. Mentions and PR events must preserve task ownership and requester
authority. Provider publication must use the bot App identity and
enforce the configured permissions.

**Proposed solution**

Extend the existing GitHub chat connector with resumable App onboarding,
linked-member and sponsored-guest access, editable event prompts, and
governed review operations. Validate structured assessments on the
server and compute a stable Paperclip Review check for the exact head
commit.

**Alternatives considered**

A separate review scheduler would duplicate Paperclip execution and
permissions. Reusing personal GitHub credentials would change the bot
identity and credential boundary.

**Roadmap alignment**

This extends the existing Connected Apps and governed-tool
infrastructure. The project owner requested and approved this design.
Related PR #8645 imports external Codex review feedback; this change
runs an assigned Paperclip agent and publishes its results through the
existing chat connector.

## What Changed

- Include the current Paperclip instance origin in the copied setup
prompt. Storybook uses its configured Paperclip origin; callback
parameters and URL credentials are excluded.

- Add a Claude/Codex copy button in the real setup and Storybook opening
step. Its detailed prompt asks four setup questions and guides
embedded-browser setup, verification, and optional required checks.
Clipboard failure exposes selectable instructions.
- Add a tutorial that explains why App installation, review scheduling,
and required checks are separate choices.

- Add manifest registration, an existing-App path, separate installation
and repository selection, repository refresh, and explicit account
confirmation.
- Add low-trust agent guidance, effective capability verification,
member selection, and explicit restricted guests with a sponsor.
- Add configurable PR events, prompts, repository overrides, rating
thresholds, and separate formal-review permissions.
- Give the assigned agent governed App tools to read PRs, comment, begin
an assessment, submit findings, and optionally submit a formal review.
- Bind review history, root PR events, and inline replies to ordinary
tasks. Deduplicate deliveries/findings and reject stale publication.
- Link check Details to the underlying task on the current trusted
hostname, or to Reviews before task creation.
- Add schema migration 0283, API contracts, production UI, and 49
interactive Storybook states.
- Repair local lease recovery. Keep the Cloud Dockerfile identical to
master; no provider-pack layer or runtime-default environment variable
is added.
- Retry only rolled-back wake-admission transactions after transient
endpoint-lock contention. A deterministic held-lock regression proves
one accepted wake.

## Verification

- Current head: `7ba761fe007bb798400d3e62346fa964f607f0f8`, rebased on
master `d9b3a5653e41f2ee5a1345b97c86a238f7a5c8e9`. Dockerfile has zero
diff against master. Final workspace typecheck and build passed. The new
PostgreSQL migration regression passed and preserves existing relation
and constraint identities after replay.
- Greptile reviewed this exact head at 5/5. There are zero unresolved
review threads and no merge conflicts.
- All current-head checks are green: 54 passed and two conditional
Storybook jobs skipped. This includes complete server/workspace test
suites, build, typechecks, policy checks, Runner suites, browser suites,
and security status. One timing-sensitive callback-ordering test passed
in isolation and its CI shard passed one retry. The duplicate local
full-suite run was stopped after CI completed; it is not counted as a
local full-suite pass.
- Before the final Slack rebase and migration renumbering, 186 focused
GitHub tests, 14 native bootstrap cases, token gates, and Storybook
build passed. The final rebase retained the new Slack communication
guidance.
- The embedded-browser setup test copied the full detailed prompt,
including the configured Paperclip instance URL. Desktop and narrow
layouts were checked. Component tests cover successful copying and
clipboard failure with selectable text and retry.
- Live local and hosted GitHub acceptance evidence refers to application
revision `cb703ac959876a07ebf3d7a295847f9f351eb6fc`. Real agent tasks
exercised issue mentions, automatic PR reviews, inline findings,
repeated mentions, task continuation, and failing-to-passing checks
after a push. The Storybook agent generated, built, and browser-rendered
pages; missing acceptance text failed, matching text passed, and broken
JSX produced an incomplete result.
- Live cases also covered independently disabled push events, prompt
injection, duplicate signed deliveries, rapid pushes, stale-result
rejection, finding deduplication, and restart recovery. Formal reviews
were denied while disabled and published only after explicit enablement.
Check Details links pointed to the underlying task on the trusted
hostname.
- Those hosted native Claude runs used the provider-pack layer now
removed from this PR. They do not prove native Claude works on the
standard Cloud image. A replacement hosted native Codex run is not yet
verified: the disposable QA tenant has only an Anthropic AI connection.
No new staging or production deployment was made for the packaging
removal.
- Required-check merge enforcement could not be tested because the
private disposable repository's GitHub plan rejected the rules
configuration. Published success/failure/incomplete check states were
verified directly.

## Risks

- Latest master allocated migration 0282 to Slack. The GitHub migration
is regenerated as 0283 with replay-safe table/index/constraint creation;
a PostgreSQL regression verifies existing relations and constraints are
preserved. Existing preview tenants remain subject to the fleet
migration-history compatibility preflight; no bypass is introduced.

- Migration 0283 adds company-scoped configuration, registration,
review, and publication records. Existing connections retain their
behavior until reviews/tools are enabled.
- Signed webhooks and expiring registration state remain required.
Hosted installations also need the companion narrow Cloud gateway
exemptions.
- Agent assessments can be incomplete or wrong. The server enforces
coverage/result structure, current-head publication, rating policy, and
separate formal-review permission; it does not replace code-review
judgment.
- No Cloud image packaging changes are included. Remote native
ACPX/Claude and OpenCode retain their existing operator-supplied
provider-pack prerequisite. Native Codex and Codex with managed MCP
tools do not require that pack. Earlier staging deployment evidence
refers to its stated revision, not this packaging-removal head.
Production rollout and merging remain outside this change.

## Model Used

OpenAI GPT-6 through Codex, with repository, code execution, API, and
embedded-browser tools. The exact serving model ID and context-window
size were not exposed by the environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-21 14:41:19 -05:00

442 lines
16 KiB
TypeScript

import type { Meta, StoryObj } from "@storybook/react-vite";
import { expect, userEvent, within } from "storybook/test";
import { GitHubChatPreview } from "../prototypes/github-chat/GitHubChatPreview";
const meta = {
title: "Apps/GitHub chat & reviews",
component: GitHubChatPreview,
parameters: {
layout: "fullscreen",
docs: {
description: {
component:
"Design approval preview. Provider handoffs, tool checks, identity events, tasks, and reviews are simulated. No backend behavior or live GitHub actions are implemented by these stories. Use Start here for the complete journey, then inspect alternate and management states.",
},
},
},
tags: ["autodocs"],
} satisfies Meta<typeof GitHubChatPreview>;
export default meta;
type Story = StoryObj<typeof meta>;
export const StartHere: Story = {
name: "00 · Start here — interactive setup",
args: { persistKey: "github-chat-design-draft-v3" },
};
export const ChooseAgent: Story = { name: "01 · Choose agent" };
export const StandardTrustWarning: Story = {
name: "01 · Agent without low-trust review",
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.selectOptions(canvas.getByRole("combobox", { name: /Paperclip agent/ }), "Atlas");
await expect(canvas.getByRole("alert")).toHaveTextContent("Atlas is not configured for low-trust review");
await expect(canvas.getByRole("link", { name: /Learn about low-trust agents/ })).toHaveAttribute("href", "https://docs.paperclip.ing/administration/trust-and-low-trust-review/");
},
};
export const CreateApp: Story = {
name: "02 · Create GitHub App",
args: { initialStep: 1 },
};
export const ExistingApp: Story = {
name: "02 · Existing App credentials",
args: { initialStep: 1, scenario: "existing" },
};
export const Reconnect: Story = {
name: "02 · Reconnect saved App",
args: { initialStep: 1, scenario: "reconnect" },
};
export const ExpiredRegistration: Story = {
name: "02 · Registration expired",
args: { initialStep: 1, scenario: "expired" },
};
export const PrivateAddress: Story = {
name: "02 · Public HTTPS prerequisite",
args: { initialStep: 1, scenario: "private-url" },
};
export const SelectRepositories: Story = {
name: "04 · Choose allowed repositories",
args: { initialStep: 3 },
};
export const Verify: Story = {
name: "05 · Connection and agent tools verified",
args: { initialStep: 4 },
};
export const MissingTools: Story = {
name: "05 · Agent missing GitHub tools",
args: { initialStep: 4, scenario: "tools" },
};
export const MissingPermissions: Story = {
name: "05 · Installation needs more permissions",
args: { initialStep: 4, scenario: "permissions" },
};
export const WebhookFailure: Story = {
name: "05 · Webhook not verified",
args: { initialStep: 4, scenario: "webhook" },
};
export const UnsupportedRuntime: Story = {
name: "05 · Agent environment unavailable",
args: { initialStep: 4, scenario: "runtime" },
};
export const ConnectIdentity: Story = {
name: "06 · Use your existing GitHub connection",
args: { initialStep: 5 },
};
export const ConfirmIdentity: Story = {
name: "06 · Confirm existing GitHub account",
args: { initialStep: 5, scenario: "identity" },
};
export const LinkedIdentity: Story = {
name: "06 · Account linked",
args: { initialStep: 5, scenario: "linked" },
};
export const Configure: Story = {
name: "07 · Review triggers, prompts, and tools",
args: { initialStep: 6 },
};
export const TryIt: Story = {
name: "08 · Optional first review",
args: { initialStep: 7 },
};
export const SaveError: Story = {
name: "09 · Save failure and retry",
args: { initialStep: 6, scenario: "save-error" },
};
export const Settings: Story = {
name: "Management · Settings",
args: { initialStep: 7, section: "Settings" },
};
export const RepositoryOverride: Story = {
name: "Management · Repository overrides",
args: { initialStep: 7, section: "Settings", initialOverride: true },
};
export const Access: Story = {
name: "Management · Linked member access",
args: { initialStep: 7, section: "Access" },
};
export const GuestAccess: Story = {
name: "Management · Sponsored guest permissions",
args: { initialStep: 7, section: "Access", scenario: "guest" },
};
export const Reviews: Story = {
name: "Reviews · Below minimum rating",
args: { initialStep: 7, section: "Reviews", reviewState: "failed" },
};
export const Passed: Story = {
name: "Reviews · Passing current head",
args: { initialStep: 7, section: "Reviews", reviewState: "passed" },
};
export const Running: Story = {
name: "Reviews · Agent running",
args: { initialStep: 7, section: "Reviews", reviewState: "running" },
};
export const Queued: Story = {
name: "Reviews · Task queued",
args: { initialStep: 7, section: "Reviews", reviewState: "queued" },
};
export const Incomplete: Story = {
name: "Reviews · Incomplete coverage",
args: { initialStep: 7, section: "Reviews", reviewState: "incomplete" },
};
export const ManualNeeded: Story = {
name: "Reviews · Authorized mention needed",
args: { initialStep: 7, section: "Reviews", reviewState: "manual" },
};
export const Conversations: Story = {
name: "Management · Task-bound conversations",
args: { initialStep: 7, section: "Conversations" },
};
export const Activity: Story = {
name: "Management · Activity pagination",
args: { initialStep: 7, section: "Activity" },
};
export const MobileSetup: Story = {
name: "Mobile · Setup",
args: { initialStep: 1 },
globals: { viewport: { value: "mobile", isRotated: false } },
};
export const MobileReviews: Story = {
name: "Mobile · Review and task links",
args: { initialStep: 7, section: "Reviews" },
globals: { viewport: { value: "mobile", isRotated: false } },
};
export const VerifiedJourney: Story = {
name: "Verification · Complete setup journey",
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const body = within(canvasElement.ownerDocument.body);
await userEvent.click(canvas.getByRole("button", { name: "Continue" }));
await userEvent.click(
canvas.getByRole("button", { name: "Create GitHub App" }),
);
await userEvent.click(
await body.findByRole("button", {
name: "Simulate App created and return",
}),
);
await userEvent.click(
canvas.getByRole("button", { name: "Install on GitHub" }),
);
await userEvent.click(
await body.findByRole("button", {
name: "Simulate installation completed",
}),
);
await userEvent.click(
canvas.getByRole("button", { name: "Choose repositories" }),
);
await userEvent.click(
canvas.getByRole("button", { name: "Check connection and tools" }),
);
await userEvent.click(
canvas.getByRole("button", { name: "Connect your account" }),
);
await userEvent.click(
canvas.getByRole("button", { name: "Use this GitHub account" }),
);
await userEvent.click(
canvas.getByRole("button", { name: "Configure behavior" }),
);
await expect(canvas.getByLabelText("Minimum passing rating")).toHaveValue(
"5",
);
await userEvent.click(
canvas.getByRole("button", { name: "Continue to try it" }),
);
await userEvent.click(canvas.getByRole("button", { name: "Finish setup" }));
await expect(
canvas.getByRole("heading", { name: "Settings" }),
).toBeVisible();
},
};
export const VerifiedToolGate: Story = {
name: "Verification · Missing tools block progress",
args: { initialStep: 4, scenario: "tools" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await expect(
canvas.getByRole("button", { name: "Connect your account" }),
).toBeDisabled();
await userEvent.click(
canvas.getByRole("button", { name: "Simulate repair and recheck" }),
);
await expect(
canvas.getByRole("button", { name: "Connect your account" }),
).toBeEnabled();
},
};
export const VerifiedOverrides: Story = {
name: "Verification · Repository settings stay isolated",
args: { initialStep: 7, section: "Settings" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.selectOptions(
canvas.getByLabelText("Configuration scope"),
"acme/platform",
);
await userEvent.selectOptions(
canvas.getByLabelText("Minimum passing rating"),
"4",
);
await userEvent.click(
canvas.getByRole("switch", { name: "Review draft PRs" }),
);
await userEvent.selectOptions(
canvas.getByLabelText("Configuration scope"),
"defaults",
);
await expect(canvas.getByLabelText("Minimum passing rating")).toHaveValue(
"5",
);
await expect(
canvas.getByRole("switch", { name: "Review draft PRs" }),
).toHaveAttribute("aria-checked", "false");
await userEvent.selectOptions(
canvas.getByLabelText("Configuration scope"),
"acme/platform",
);
await expect(canvas.getByLabelText("Minimum passing rating")).toHaveValue(
"4",
);
await userEvent.click(
canvas.getByRole("button", { name: "Reset to defaults" }),
);
await expect(canvas.getByLabelText("Minimum passing rating")).toHaveValue(
"5",
);
},
};
export const VerifiedSaveRetry: Story = {
name: "Verification · Failed save retains draft",
args: { initialStep: 1, scenario: "save-error" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.clear(canvas.getByLabelText("App name"));
await userEvent.type(canvas.getByLabelText("App name"), "custom-reviewer");
await userEvent.click(canvas.getByRole("button", { name: "Save & exit" }));
await expect(canvas.getByRole("alert")).toBeVisible();
await expect(canvas.getByLabelText("App name")).toHaveValue(
"custom-reviewer",
);
await userEvent.click(canvas.getByRole("button", { name: "Save & exit" }));
await userEvent.click(canvas.getByRole("button", { name: "Resume setup" }));
await expect(canvas.getByLabelText("App name")).toHaveValue(
"custom-reviewer",
);
},
};
export const VerifiedGuestAccess: Story = {
name: "Verification · Guest access is explicit",
args: { initialStep: 7, section: "Access" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const body = within(canvasElement.ownerDocument.body);
await userEvent.click(
canvas.getByRole("button", { name: "Allow GitHub user" }),
);
await userEvent.click(
await body.findByRole("button", { name: "Look up account" }),
);
await expect(body.getByLabelText("Responsible sponsor")).toHaveValue(
"Dotta",
);
await userEvent.click(
body.getByRole("button", { name: "Allow this GitHub user" }),
);
await expect(
canvas.getByRole("switch", {
name: "Automatically review PRs by @external-contributor",
}),
).toHaveAttribute("aria-checked", "false");
await expect(
canvas.getByLabelText("Responsible sponsor for @external-contributor"),
).toHaveValue("Dotta");
await userEvent.click(
canvas.getByRole("button", {
name: "Remove access for @external-contributor",
}),
);
await expect(
canvas.queryByRole("switch", {
name: "Automatically review PRs by @external-contributor",
}),
).toBeNull();
},
};
export const InstallApp: Story = {
name: "03 · Install GitHub App",
args: { initialStep: 2 },
};
export const NoPersonalConnection: Story = {
name: "06 · No personal GitHub connection",
args: { initialStep: 5, scenario: "no-account" },
};
export const ExpiredPersonalConnection: Story = {
name: "06 · Reconnect personal GitHub",
args: { initialStep: 5, scenario: "identity-expired" },
};
export const NoRepositories: Story = {
name: "04 · No repositories available",
args: { initialStep: 3, scenario: "repositories-empty" },
};
export const RepositoryRefreshFailure: Story = {
name: "04 · Repository refresh failed",
args: { initialStep: 3, scenario: "repositories-error" },
};
export const VerifiedRepositoryRefresh: Story = {
name: "Verification · New installation access stays disabled",
args: { initialStep: 3 },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const body = within(canvasElement.ownerDocument.body);
await expect(
canvas.queryByRole("switch", { name: "acme/mobile" }),
).toBeNull();
await userEvent.click(
canvas.getAllByRole("button", { name: "Configure access on GitHub" })[0],
);
await userEvent.click(
await body.findByRole("button", {
name: "Simulate adding a repository on GitHub",
}),
);
await expect(
canvas.queryByRole("switch", { name: "acme/mobile" }),
).toBeNull();
await userEvent.click(
canvas.getByRole("button", { name: "Refresh access" }),
);
await expect(
await canvas.findByRole("switch", { name: "acme/mobile" }),
).toHaveAttribute("aria-checked", "false");
await expect(
canvas.getByRole("switch", { name: "acme/platform" }),
).toHaveAttribute("aria-checked", "true");
},
};
export const VerifiedPersonalConnection: Story = {
name: "Verification · Personal sign-in requires confirmation",
args: { initialStep: 5, scenario: "no-account" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const body = within(canvasElement.ownerDocument.body);
await userEvent.click(
canvas.getByRole("button", { name: "Connect GitHub" }),
);
await userEvent.click(
await body.findByRole("button", {
name: "Simulate GitHub sign-in completed",
}),
);
await expect(
canvas.queryByRole("button", { name: "Configure behavior" }),
).toBeNull();
await userEvent.selectOptions(
canvas.getByLabelText("Your GitHub connection"),
"dotta-work",
);
await userEvent.click(
canvas.getByRole("button", { name: "Use this GitHub account" }),
);
await expect(
canvas.getByText("@dotta-work is linked to Dotta."),
).toBeVisible();
await expect(
canvas.getByRole("button", { name: "Configure behavior" }),
).toBeEnabled();
},
};
export const MobileAccess: Story = {
name: "Mobile · People and sponsored access",
args: { initialStep: 7, section: "Access", scenario: "guest" },
globals: { viewport: { value: "mobile", isRotated: false } },
};
export const VerifiedMemberAccess: Story = {
name: "Verification · Selected member access and separate automation",
args: { initialStep: 7, section: "Access" },
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
const body = within(canvasElement.ownerDocument.body);
await userEvent.click(
canvas.getByRole("button", { name: "Add Paperclip member" }),
);
await userEvent.selectOptions(await body.findByLabelText("Member"), "Sam");
await expect(
body.getByRole("button", { name: "Add member" }),
).toBeDisabled();
await userEvent.selectOptions(body.getByLabelText("Member"), "Alex");
await userEvent.click(body.getByRole("button", { name: "Add member" }));
await expect(canvas.getByLabelText("Paperclip members")).toHaveValue(
"selected",
);
await expect(
canvas.getByRole("switch", { name: "Automatically review PRs by @alex" }),
).toHaveAttribute("aria-checked", "false");
await userEvent.click(
canvas.getByRole("switch", { name: "Automatically review PRs by @alex" }),
);
await expect(
canvas.getByRole("switch", { name: "Automatically review PRs by @alex" }),
).toHaveAttribute("aria-checked", "true");
},
};