Files
PaperClipAI/packages/shared/src/feature-catalog.test.ts
T
Devin FoleyandPaperclip ad74fb5450 Add a feature catalog build artifact derived from the experimental settings schema (#10055)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Instances expose ~23 experimental feature settings, all declared in
one shared zod schema and toggled per instance
> - Deployment tooling and hosting control planes have no
machine-readable list of those feature keys for a given release — the
schema is only reachable from code that imports the package
> - Any external system that references feature keys therefore does so
as free text, and typos drift silently
> - This pull request derives a versioned `feature-catalog.json` build
artifact from the schema, with a compiler-checked metadata map so the
schema stays the single source of truth
> - The benefit is a stable contract external tooling can validate
feature-key references against, with zero runtime behavior change

## Linked Issues or Issue Description

No public issue exists; `feature_request` template fields:

**Problem or motivation:**
External deployment tooling cannot enumerate or validate an instance's
feature keys per release; free-text references fail silently when keys
are renamed or removed.

**Proposed solution:**
A metadata map keyed by the settings schema's own keys (compiler flags
drift) plus a build step emitting `feature-catalog.json` (keys, tiers,
defaults, `catalogVersion`) as a release artifact.

**Alternatives considered:**
A hand-maintained catalog file (drifts from the schema); serving the
schema from a runtime API (requires a running instance at validation
time — a build artifact works offline and pins to a release).

**Roadmap alignment:**
Supports the in-progress "Cloud deployments" milestone in `ROADMAP.md`.

## What Changed

Adds a metadata map (title, description, tier, cloud/self-hosted
defaults) keyed by the keys of `instanceExperimentalSettingsSchema`, so
the schema stays the single source of truth and the compiler flags any
drift. A new build step (`build:feature-catalog --version <v>`) emits
`feature-catalog.json` — all 23 feature keys, their tiers, and a
`catalogVersion` — as a release artifact that managed-hosting control
planes can validate feature-flag writes against. No runtime behavior
changes.

- New `packages/shared/src/feature-catalog.ts`: per-flag metadata map
keyed by a type derived from the settings schema
(adding/removing/renaming a flag without updating the map is a compile
error), plus `featureCatalogArtifactSchema` and
`buildFeatureCatalogArtifact`/`renderFeatureCatalogArtifact` for the
artifact
- New `scripts/generate-feature-catalog.ts` wired as `pnpm
build:feature-catalog --version <v>`
- `scripts/create-github-release.sh` generates the artifact and uploads
it as a GitHub Release asset (with a dry-run preview line)
- Tests in `packages/shared/src/feature-catalog.test.ts`

## Verification

- `vitest run packages/shared/src/feature-catalog.test.ts` — 9 tests:
schema-key coverage, drift detection, artifact shape
- `pnpm --filter @paperclipai/shared typecheck`
- Artifact generation run end-to-end: `pnpm build:feature-catalog
--version 0.0.0-test` emits 23 keys with `catalogVersion`

## Risks

Low risk — no runtime behavior changes; the change is metadata, a build
script, and a release-artifact emission step only.

## Model Used

Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use;
independently peer-reviewed by a second AI agent before push

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-22 18:12:56 -07:00

112 lines
4.1 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { z } from "zod";
import {
FEATURE_TIERS,
INSTANCE_FEATURE_CATALOG,
INSTANCE_FEATURE_KEYS,
buildFeatureCatalogArtifact,
featureCatalogArtifactSchema,
renderFeatureCatalogArtifact,
} from "./feature-catalog.js";
import { instanceExperimentalSettingsSchema } from "./validators/instance.js";
function schemaBooleanFlagKeys(): string[] {
return Object.entries(instanceExperimentalSettingsSchema.shape)
.filter(([, fieldSchema]) => {
let current: z.ZodTypeAny = fieldSchema as z.ZodTypeAny;
for (;;) {
if (current instanceof z.ZodDefault) {
current = current._def.innerType as z.ZodTypeAny;
} else if (current instanceof z.ZodOptional || current instanceof z.ZodNullable) {
current = current.unwrap() as z.ZodTypeAny;
} else {
break;
}
}
return current instanceof z.ZodBoolean;
})
.map(([key]) => key)
.sort();
}
describe("INSTANCE_FEATURE_CATALOG", () => {
it("covers exactly the boolean flag keys of the experimental settings schema", () => {
expect([...INSTANCE_FEATURE_KEYS]).toEqual(schemaBooleanFlagKeys());
});
it("keeps selfHostedDefault in sync with the schema defaults", () => {
const schemaDefaults = instanceExperimentalSettingsSchema.parse({});
for (const key of INSTANCE_FEATURE_KEYS) {
expect(INSTANCE_FEATURE_CATALOG[key].selfHostedDefault, key).toBe(schemaDefaults[key]);
}
});
it("has a non-empty title, description, and valid tier for every flag", () => {
for (const key of INSTANCE_FEATURE_KEYS) {
const entry = INSTANCE_FEATURE_CATALOG[key];
expect(entry.title.trim().length, key).toBeGreaterThan(0);
expect(entry.description.trim().length, key).toBeGreaterThan(0);
expect(FEATURE_TIERS, key).toContain(entry.tier);
}
});
});
describe("buildFeatureCatalogArtifact", () => {
it("emits catalogVersion plus one tier entry per flag key", () => {
const artifact = buildFeatureCatalogArtifact("2026.720.0");
expect(artifact.catalogVersion).toBe("2026.720.0");
expect(Object.keys(artifact.features)).toEqual([...INSTANCE_FEATURE_KEYS]);
for (const key of INSTANCE_FEATURE_KEYS) {
expect(artifact.features[key]).toEqual({ tier: INSTANCE_FEATURE_CATALOG[key].tier });
}
});
it("produces output that validates against featureCatalogArtifactSchema", () => {
const artifact = buildFeatureCatalogArtifact("2026.720.0");
expect(featureCatalogArtifactSchema.parse(artifact)).toEqual(artifact);
});
it("rejects an empty catalogVersion", () => {
expect(() => buildFeatureCatalogArtifact("")).toThrow(/catalogVersion/);
expect(() => buildFeatureCatalogArtifact(" ")).toThrow(/catalogVersion/);
});
});
describe("featureCatalogArtifactSchema", () => {
it("rejects unknown top-level or per-feature properties", () => {
const valid = buildFeatureCatalogArtifact("2026.720.0");
expect(
featureCatalogArtifactSchema.safeParse({ ...valid, extra: true }).success,
).toBe(false);
expect(
featureCatalogArtifactSchema.safeParse({
catalogVersion: "2026.720.0",
features: { enableApps: { tier: "managed", extra: true } },
}).success,
).toBe(false);
});
it("rejects unknown tiers and a missing catalogVersion", () => {
expect(
featureCatalogArtifactSchema.safeParse({
catalogVersion: "2026.720.0",
features: { enableApps: { tier: "mystery" } },
}).success,
).toBe(false);
expect(
featureCatalogArtifactSchema.safeParse({ features: {} }).success,
).toBe(false);
});
});
describe("renderFeatureCatalogArtifact", () => {
it("is deterministic, sorted, and newline-terminated", () => {
const rendered = renderFeatureCatalogArtifact("2026.720.0");
expect(rendered).toBe(renderFeatureCatalogArtifact("2026.720.0"));
expect(rendered.endsWith("\n")).toBe(true);
const parsed = featureCatalogArtifactSchema.parse(JSON.parse(rendered));
expect(Object.keys(parsed.features)).toEqual([...Object.keys(parsed.features)].sort());
expect(parsed).toEqual(buildFeatureCatalogArtifact("2026.720.0"));
});
});