Files
PaperClipAI/packages/shared/src/app-definitions.ts
DottaandPaperclip 0ffc091473 feat(connections): add durable GitHub identities and webhooks (#12843)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents need source control access for repository work
> - A shared token cannot preserve the responsible person's identity or
an agent's dedicated identity
> - GitHub App tokens also need durable refresh, repository access
checks, and webhook delivery
> - Paperclip already has managed connections, encrypted grants, run
secret leases, and merge-confirmation behavior
> - This pull request extends those systems with GitHub identities
instead of adding a parallel credential system
> - The benefit is durable GitHub access with explicit identity,
repository, runtime, and webhook boundaries

## Linked Issues or Issue Description

No public GitHub issue describes this connection change. This
description follows the feature request template.

**Subsystem affected**

Connected Apps, connection grants, secret resolution, native Git runtime
setup, webhook processing, and the Apps UI.

**Problem or motivation**

Users need to connect GitHub once and let agents use the correct GitHub
identity. A run should use a dedicated agent account when one exists.
Otherwise, it should use the responsible person's account. The
connection must survive token expiry, repository access changes, and
temporary instance downtime.

**Proposed solution**

Add user-owned and agent-owned GitHub grants to the existing connection
model. Resolve one identity for MCP, Git, `gh`, health checks, and
webhook bindings. Store provider tokens in the existing encrypted secret
system. Refresh expiring token pairs under the existing lease and
compare-and-swap path. Register signed Cloud webhook bindings and
process normalized pull request and installation events through a
durable local inbox.

**Alternatives considered**

An organization-wide GitHub token would lose person and agent
attribution. Environment variables alone would bypass the managed
connection and grant model. A new GitHub-only credential store would
duplicate the existing secret and access systems. GitHub App
installation tokens and private-key custody remain outside this first
version.

**Roadmap alignment**

This change implements the Connected Apps direction. It also extends the
shipped MCP Tool Gateway, per-agent secret access, and
action-attribution systems. It does not add a repository catalog. The
open repository catalog work in
[#11234](https://github.com/paperclipai/paperclip/pull/11234) is related
and complementary.

## What Changed

- Added agent-owned connection grants and a per-agent credential policy
with company and subject constraints.
- Added a managed GitHub App method while keeping the personal access
token method as an advanced fallback.
- Added durable access-token and refresh-token handling with proactive
rotation and one automatic recovery after a provider `401`.
- Added GitHub identity and installation summaries without storing
repository-name lists.
- Added signed Cloud webhook binding, event lease, acknowledgement,
local idempotency, pull request merge processing, and installation
access handling.
- Added one identity resolver for MCP, native Git, `gh`, checkout,
health checks, and webhook bindings.
- Added a class-3 run projection for `GH_TOKEN`, `GITHUB_TOKEN`, a
`github.com`-only credential helper, SSH-to-HTTPS rewrite, and GitHub
noreply commit attribution.
- Added personal and dedicated-agent setup choices plus identity,
repository, continuity, and webhook status in the Apps UI.
- Added schema migrations, tests, and connection documentation.

## Verification

- The current head is fully green in GitHub CI, including build,
typecheck, all serialized/general server shards, all browser shards,
policy, canary dry run, review, and security checks.
- Live staging proof completed with a non-expiring GitHub App user
token, selected-repository installation, repository add/remove refresh,
managed MCP, native `gh`, HTTPS clone/push/delete, GitHub noreply commit
attribution, signed merged-PR webhook acceptance, durable
Cloud-to-instance delivery, and installation-access event processing.
Temporary branches and temporary repository access were removed
afterward.
- `pnpm check:token-gates` passed.
- `pnpm -r typecheck` passed before and after the rebase onto
`origin/master`.
- `pnpm build` passed.
- The focused connector suite passed 285 tests after the rebase.
- The full stable suite passed 5,790 tests and failed 22 tests across 8
general server files. The failures reproduced as shared-runner
environment issues. They included `/tmp` versus `/private/tmp`, closed
database connections, and invalid high ephemeral ports. The focused
connection tests pass in isolation.

## Risks

- Migrations add agent grant subjects and a durable connection-event
inbox. Migration numbering and safety checks pass.
- A raw GitHub user token enters the agent process for Git and `gh`.
Per-tool Ask-first controls cannot limit those shell operations. The UI
warns users about this boundary.
- GitHub App user tokens can be non-expiring. Paperclip performs a
continuity check every 30 days, but provider revocation still requires a
reconnect.
- The webhook path accepts only signed and bounded payloads. It stores a
minimal normalized record and no raw provider payload.
- GitHub repository permissions remain authoritative. Removed access can
make a cached repository count temporarily stale, but runtime access
fails immediately.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, `gpt-5.6-sol`, extended reasoning, tool use, code
execution, browser control, and multi-file repository editing. The
context window size was not provided.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-04 18:02:52 -05:00

252 lines
8.7 KiB
TypeScript

import { APP_DEFINITIONS } from "./app-definitions.generated.js";
import { SELF_SERVE_MCP_CANDIDATES } from "./self-serve-mcp-research.js";
import type { AppDefinition, ConnectionMethodDef, FieldDef } from "./types/app-definition.js";
import type { ToolConnectionOwnership } from "./types/tool-access.js";
export const CONNECTABLE_APP_SLUGS = new Set([
...SELF_SERVE_MCP_CANDIDATES.map((entry) => entry.slug),
"zapier",
"slack",
"notion",
"posthog",
"linear",
"google-sheets",
"context7",
"shopify",
"composio",
"gmail",
"google-drive",
"google-docs",
"google-slides",
"google-calendar",
"google-chat",
"google-people",
"google-workspace-search",
"github",
]);
export const CONNECTABLE_APP_DEFINITIONS = APP_DEFINITIONS.filter((app) =>
CONNECTABLE_APP_SLUGS.has(app.slug)
);
/**
* Definitions retained for existing connections and later verification, but
* intentionally withheld from the customer-facing store. Keeping visibility
* separate from recognition avoids breaking saved connections when a provider
* is pulled from Browse or reserved for a future first-party experience.
*/
export const APP_STORE_HIDDEN_SLUGS = new Set([
"beehiiv",
"bitly",
"brex",
"candid",
"coda",
"composio",
"context7",
"egnyte",
"embat",
"kernel",
"local-falcon",
"make",
"manufact",
"oreilly",
"planetscale",
"razorpay",
"sanity",
"similarweb",
"slack",
"ticket-tailor",
"ticktick",
"xero",
]);
export const APP_STORE_DEFINITIONS = CONNECTABLE_APP_DEFINITIONS.filter((app) =>
!APP_STORE_HIDDEN_SLUGS.has(app.slug)
);
export const DEFAULT_OWNERSHIP_AVAILABILITY: Record<ToolConnectionOwnership, boolean> = {
platform_shared: false,
platform_provisioned: false,
customer: true,
dcr: true,
};
export function getConnectableAppDefinition(slug: string): AppDefinition | null {
return CONNECTABLE_APP_DEFINITIONS.find((app) => app.slug === slug) ?? null;
}
export function getAppStoreDefinition(slug: string): AppDefinition | null {
return APP_STORE_DEFINITIONS.find((app) => app.slug === slug) ?? null;
}
export function isAppStoreVisibleSlug(slug: string | null | undefined): boolean {
return Boolean(slug && !APP_STORE_HIDDEN_SLUGS.has(slug) && CONNECTABLE_APP_SLUGS.has(slug));
}
function wildcardPatternToRegExp(pattern: string): RegExp {
const escaped = pattern.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, ".*");
return new RegExp(`^${escaped}$`, "i");
}
export function getAppDefinitionForUrl(
link: string,
definitions: readonly AppDefinition[] = CONNECTABLE_APP_DEFINITIONS,
): AppDefinition | null {
let normalized: string;
try {
normalized = new URL(link.trim()).toString();
} catch {
return null;
}
return definitions.find((app) =>
app.urlPatterns.some((pattern) => wildcardPatternToRegExp(pattern).test(normalized))
) ?? null;
}
export function getAvailableConnectionMethods(app: AppDefinition): ConnectionMethodDef[] {
const availability = app.ownershipAvailability ?? DEFAULT_OWNERSHIP_AVAILABILITY;
return app.methods.filter((method) =>
method.ownershipModes.some((ownership) => availability[ownership] !== false)
);
}
/**
* Pick the method that gives a new connection the app's useful write surface.
*
* Google Workspace publishes separate read and write capability groups. The
* read method is intentionally listed first for documentation, but treating
* array order as a product default silently created read-only connections and
* left every write action Off. Capability metadata is the durable signal; apps
* without an explicit write/draft capability retain their declared order.
*/
export function getRecommendedConnectionMethod(
methods: readonly ConnectionMethodDef[],
): ConnectionMethodDef | null {
const recommendedCapability = (candidates: readonly ConnectionMethodDef[]) => candidates.find((method) => {
const capabilityKey = method.capabilityProfile?.key;
return capabilityKey === "write" || capabilityKey === "draft";
});
const managedMethods = methods.filter((method) =>
method.oauthStrategy === "paperclip_cloud_connector"
|| method.oauthStrategy === "paperclip_id_connector"
);
// When a managed pilot advertises only read access, defaulting to a
// customer-owned write method would turn the available one-click path into
// an OAuth client setup form. Capability-specific callers pass only the
// selected group, so explicit write/draft choices keep their own fallback.
return recommendedCapability(managedMethods)
?? managedMethods[0]
?? recommendedCapability(methods)
?? methods[0]
?? null;
}
export function getAvailableConnectionMethod(
app: AppDefinition,
methodKey?: string | null,
): ConnectionMethodDef | null {
const methods = getAvailableConnectionMethods(app);
return methodKey
? methods.find((method) => method.key === methodKey) ?? null
: getRecommendedConnectionMethod(methods);
}
export function connectionMethodSupportsAutomaticOAuth(method: ConnectionMethodDef | null | undefined): boolean {
return method?.auth === "oauth" && (
(method.oauthStrategy === "paperclip_cloud_connector" || method.oauthStrategy === "paperclip_id_connector")
|| method.ownershipModes.includes("dcr")
);
}
export function connectionMethodAcceptsCustomerOAuthClient(method: ConnectionMethodDef | null | undefined): boolean {
return method?.auth === "oauth"
&& !method.oauthStrategy
&& method.ownershipModes.includes("customer");
}
export function connectionMethodSupportsCatalogSetup(method: ConnectionMethodDef | null | undefined): boolean {
if (!method) return false;
if (method.auth === "none" || method.auth === "api_key") return true;
return connectionMethodSupportsAutomaticOAuth(method)
|| connectionMethodAcceptsCustomerOAuthClient(method);
}
export function connectionMethodRequiresConfiguration(method: ConnectionMethodDef | null | undefined): boolean {
if (!method) return false;
const visibleTenantFields = method.tenantFields?.filter((field) => !field.hidden) ?? [];
const visibleExtensionFields = method.extensionFields?.filter((field) => !field.hidden) ?? [];
return Boolean(
method.credentialFields?.length
|| visibleTenantFields.length
|| visibleExtensionFields.length
|| method.configRequirements?.atLeastOneOf?.length
// "Use your own OAuth app" is an advanced alternative when DCR/CIMD is
// available, not a required setup field. Only customer-client-only methods
// must stop on the configuration screen.
|| (
connectionMethodAcceptsCustomerOAuthClient(method)
&& !connectionMethodSupportsAutomaticOAuth(method)
),
);
}
export function appSupportsCatalogSetup(app: AppDefinition | null | undefined): boolean {
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodSupportsCatalogSetup));
}
export function isConnectableAppSlug(slug: string | null | undefined): boolean {
return Boolean(slug && CONNECTABLE_APP_SLUGS.has(slug));
}
export function appSupportsAutomaticOAuth(app: AppDefinition | null | undefined): boolean {
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodSupportsAutomaticOAuth));
}
export function appAcceptsCustomerOAuthClient(app: AppDefinition | null | undefined): boolean {
return Boolean(app && getAvailableConnectionMethods(app).some(connectionMethodAcceptsCustomerOAuthClient));
}
export function credentialConfigPath(field: FieldDef): string {
return `credentials.${field.key}`;
}
export function resolveConnectionMethodServerUrl(
method: ConnectionMethodDef,
configValues: Record<string, string | boolean>,
): string | null {
const template = method.defaults?.serverUrlTemplate;
if (!template) return method.defaults?.serverUrl ?? null;
let missingValue = false;
const resolved = template.replace(/\{([a-zA-Z0-9_-]+)\}/g, (_placeholder, key: string) => {
const value = configValues[key];
if (value === undefined || String(value).trim().length === 0) {
missingValue = true;
return "";
}
return encodeURIComponent(String(value).trim());
});
if (missingValue) return null;
try {
return new URL(resolved).toString();
} catch {
return null;
}
}
export function recommendedDefaultsForApp(app: AppDefinition, methodKey?: string | null): Record<string, unknown> {
// Keep the parameters in the public contract: callers resolve defaults for a
// concrete app/method even though the initial policy is now uniform. This is
// an open default, not an approval bypass: connection finalization remains a
// configure-authorized, audited operation, and Ask first stays available as
// an operator-selected policy for any action after the connection is made.
void app;
void methodKey;
return {
access: "all_agents",
askFirstRiskLevels: [],
};
}