Files
PaperClipAI/packages/shared/src/settings-visibility.test.ts
T
Devin FoleyandPaperclip e2f1a66aa7 feat: support default-hidden experimental settings (#13980)
## Thinking Path

> - Paperclip is the open source control plane for AI-agent companies.
> - Operators can hide settings that their users must not change.
> - The server shares the effective restrictions with the UI and
settings API.
> - An explicit list must change whenever a new experimental flag is
added.
> - This pull request adds a wildcard with named exceptions to the
existing setting.
> - Core applies the policy to its current catalog, so new flags stay
hidden automatically.

## Linked Issues or Issue Description

Related: #11823 introduced settings visibility. #13907 added workspace
isolation visibility. I searched related PRs and issues and found no
duplicate wildcard implementation.

**What existing behavior does this improve?**

Operator control of experimental setting visibility through
`PAPERCLIP_HIDDEN_SETTINGS`.

**Subsystem affected**

Shared settings policy and its existing server health and mutation
consumers.

**Current behavior**

Operators must name every hidden experimental toggle. A new Core flag
can become visible until the operator updates that list.

**Proposed behavior**

`instance.experimental.*` hides current and future experimental toggles.
Entries such as `!instance.experimental.enableEnvironments` leave named
controls available. Explicit hidden keys and the hidden parent page take
precedence over exceptions.

**Reason and benefit**

Operators can maintain a short list of allowed controls instead of a
second copy of Core's full feature catalog.

**Breaking changes**

Existing explicit lists and unset configuration keep their behavior. The
new syntax is opt-in. Older images ignore it, so operators must retain
explicit restrictions until those images are upgraded. Visibility does
not change feature values.

## What Changed

- Expand the wildcard into concrete catalog keys in the shared parser.
- Limit exceptions to known experimental controls and preserve explicit
restrictions in either input order.
- Test a synthetic future catalog addition, duplicate and invalid
entries, API rejection, same-value echoes, and the effective health
payload.
- Document the syntax and the transition for deployments with mixed
image versions.

## Verification

- Targeted parser, future-catalog, health, and settings-route tests
pass: 95 tests across four files.
- `pnpm -r typecheck` passes, including Rust checks, with the installed
Cargo directory on PATH.
- `pnpm build` passes.
- All current-head CI gates pass, including the full test shards, Rust,
build, browser E2E, and canary dry run:
https://github.com/paperclipai/paperclip/actions/runs/36083292578. One
unchanged runtime-exposure cold-start test passed on its first retry.
- The full local `pnpm test:run` did not pass on macOS/Node 25: the
first server group reported 13,356 passed, 18 failed, and 99 skipped,
with six failed files (including two failed suite setups). Failures were
in unchanged runtime/company skill cache, chat/email connector fixtures,
embedded-Postgres setup, and workspace cleanup tests. A standalone
filesystem probe reproduced the read-only-directory rename permission
failure. Missing connector fixture paths, database startup failures, and
two integration assertions also occurred; the remaining local groups
were not reached after this group failed. The corresponding CI lanes all
pass. These local failures are not claimed as fixed by this PR.
- Browser suites were not run because this changes the shared policy,
not UI rendering or browser workflows. Health payload and route tests
cover the shared UI/API contract.

## Risks

A malformed exception remains hidden and is reported as unknown.
Exceptions cannot override an explicit hidden toggle or parent page.
Older images ignore wildcard syntax; keep their explicit list during a
mixed-version rollout. No schema or feature-value changes are included.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, tool use, and code
execution. The exact runtime variant and context-window size are not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-25 08:07:50 -07:00

159 lines
7.2 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { INSTANCE_FEATURE_KEYS } from "./feature-catalog.js";
import {
HIDEABLE_COMPANY_PAGES,
HIDEABLE_COMPANY_SECTIONS,
HIDEABLE_GENERAL_SECTIONS,
HIDEABLE_SETTING_KEYS,
UI_ONLY_GENERAL_SECTIONS,
experimentalSettingKey,
hidesCompanyPage,
hidesCompanySection,
hidesExperimentalSetting,
hidesGeneralSection,
hidesInstancePage,
parseHiddenSettingsList,
} from "./settings-visibility.js";
import { instanceGeneralSettingsSchema } from "./validators/instance.js";
describe("hideable setting keys", () => {
it("derives one key per experimental flag and has no duplicates", () => {
const experimental = HIDEABLE_SETTING_KEYS.filter(
(key) => key.startsWith("instance.experimental."),
);
expect(experimental).toEqual(INSTANCE_FEATURE_KEYS.map(experimentalSettingKey));
expect(new Set(HIDEABLE_SETTING_KEYS).size).toBe(HIDEABLE_SETTING_KEYS.length);
});
it("covers every top-level company settings page except the General root", () => {
expect(HIDEABLE_COMPANY_PAGES).toEqual([
"company.members",
"company.invites",
"company.secrets",
"company.export",
"company.import",
]);
for (const page of HIDEABLE_COMPANY_PAGES) {
expect(HIDEABLE_SETTING_KEYS).toContain(page);
}
});
it("maps field-backed general sections onto real general-settings fields", () => {
const generalFields = Object.keys(instanceGeneralSettingsSchema.shape);
const uiOnly = new Set<string>(UI_ONLY_GENERAL_SECTIONS);
for (const section of HIDEABLE_GENERAL_SECTIONS) {
if (uiOnly.has(section)) continue;
expect(generalFields).toContain(section.slice("instance.general.".length));
}
});
});
describe("parseHiddenSettingsList", () => {
it("expands the experimental wildcard into concrete keys without hiding the page", () => {
const parsed = parseHiddenSettingsList("instance.experimental.*");
expect(parsed).toEqual({ hidden: INSTANCE_FEATURE_KEYS.map(experimentalSettingKey), unknown: [] });
expect(parsed.hidden).not.toContain("instance.experimental");
});
it("allows only named exceptions, independent of order or duplicates", () => {
const exception = "!instance.experimental.enableEnvironments";
for (const raw of [`instance.experimental.*, ${exception}, ${exception}`, `${exception},instance.experimental.*`]) {
const parsed = parseHiddenSettingsList(raw);
expect(parsed).toEqual({
hidden: INSTANCE_FEATURE_KEYS.filter((key) => key !== "enableEnvironments").map(experimentalSettingKey),
unknown: [],
});
}
});
it("keeps explicit hides and parent page restrictions stronger than exceptions", () => {
for (const restriction of ["instance.experimental.enableEnvironments", "instance.experimental"]) {
for (const raw of [
`instance.experimental.*,!instance.experimental.enableEnvironments,${restriction}`,
`${restriction},!instance.experimental.enableEnvironments,instance.experimental.*`,
]) {
const { hidden } = parseHiddenSettingsList(raw);
expect(hidesExperimentalSetting(new Set(hidden), "enableEnvironments")).toBe(true);
expect(new Set(hidden).size).toBe(hidden.length);
}
}
});
it("ignores unknown or out-of-scope exceptions without opening any controls", () => {
const parsed = parseHiddenSettingsList("instance.experimental.*,!instance.experimental.enableTypo,!instance.plugins,!instance.experimental,!instance.experimental.*");
expect(parsed.hidden).toEqual(INSTANCE_FEATURE_KEYS.map(experimentalSettingKey));
expect(parsed.unknown).toEqual(["!instance.experimental.enableTypo", "!instance.plugins", "!instance.experimental", "!instance.experimental.*"]);
});
it("does not use exceptions to override individual restrictions without a wildcard", () => {
expect(parseHiddenSettingsList("!instance.experimental.enableEnvironments").hidden).toEqual([]);
expect(parseHiddenSettingsList("instance.experimental.enableEnvironments,!instance.experimental.enableEnvironments").hidden)
.toEqual(["instance.experimental.enableEnvironments"]);
});
it("accepts workspace controls independently of experimental flags", () => {
expect(parseHiddenSettingsList("workspaces.isolation")).toEqual({ hidden: ["workspaces.isolation"], unknown: [] });
expect(hidesExperimentalSetting(new Set(["workspaces.isolation"]), "enableIsolatedWorkspaces")).toBe(false);
});
it("returns nothing hidden for undefined or empty input", () => {
expect(parseHiddenSettingsList(undefined)).toEqual({ hidden: [], unknown: [] });
expect(parseHiddenSettingsList(" , ,")).toEqual({ hidden: [], unknown: [] });
});
it("splits known from unknown keys, trims, and deduplicates", () => {
const parsed = parseHiddenSettingsList(
"instance.plugins, instance.general.backupRetention ,instance.bogus,instance.plugins,instance.experimental.enableEnvironments",
);
expect(parsed.hidden).toEqual([
"instance.plugins",
"instance.general.backupRetention",
"instance.experimental.enableEnvironments",
]);
expect(parsed.unknown).toEqual(["instance.bogus"]);
});
});
describe("membership helpers", () => {
const hidden = new Set(
parseHiddenSettingsList(
"instance.plugins,instance.general.censorUsernameInLogs,instance.experimental.enableEnvironments",
).hidden,
);
it("answers company-page membership", () => {
const companyHidden = new Set(parseHiddenSettingsList("company.import,company.secrets").hidden);
expect(hidesCompanyPage(companyHidden, "company.import")).toBe(true);
expect(hidesCompanyPage(companyHidden, "company.secrets")).toBe(true);
expect(hidesCompanyPage(companyHidden, "company.export")).toBe(false);
});
it("answers company-section membership independently of the parent page", () => {
const sectionHidden = new Set(parseHiddenSettingsList("company.secrets.vaults").hidden);
expect(hidesCompanySection(sectionHidden, "company.secrets.vaults")).toBe(true);
expect(hidesCompanySection(sectionHidden, "company.secrets.proposals")).toBe(false);
expect(hidesCompanyPage(sectionHidden, "company.secrets")).toBe(false);
});
it("keeps every company section key parseable and prefixed by its page", () => {
for (const key of HIDEABLE_COMPANY_SECTIONS) {
expect(parseHiddenSettingsList(key).hidden).toEqual([key]);
expect(key.startsWith("company.")).toBe(true);
}
});
it("answers page, section, and experimental membership", () => {
expect(hidesInstancePage(hidden, "instance.plugins")).toBe(true);
expect(hidesInstancePage(hidden, "instance.adapters")).toBe(false);
expect(hidesGeneralSection(hidden, "instance.general.censorUsernameInLogs")).toBe(true);
expect(hidesGeneralSection(hidden, "instance.general.backupRetention")).toBe(false);
expect(hidesExperimentalSetting(hidden, "enableEnvironments")).toBe(true);
expect(hidesExperimentalSetting(hidden, "enableIsolatedWorkspaces")).toBe(false);
});
it("treats a hidden Experimental page as hiding every toggle", () => {
const pageHidden = new Set(parseHiddenSettingsList("instance.experimental").hidden);
expect(hidesExperimentalSetting(pageHidden, "enableEnvironments")).toBe(true);
});
});