Files
PaperClipAI/ui/src/lib/new-agent-runtime-config.test.ts
T
DottaandPaperclip 1426494ab8 fix(agents): disable cheap model profiles by default (#10019)
## Thinking Path

> - Paperclip is the control plane people use to create and govern
AI-agent companies
> - Agent creation persists runtime configuration that controls which
model profiles future runs may select
> - Adapters can expose a `cheap` profile, and existing creation paths
implicitly left that profile available when operators made no choice
> - That made a newly created agent eligible for a lower-cost model
without an explicit operator opt-in
> - The UI also dropped an explicit opt-in when the operator selected
the adapter's default cheap model rather than a custom model ID
> - Codex additionally hardcoded `gpt-5.3-codex-spark` into its cheap
profile and static fallback model list, making Paperclip choose an
auth-dependent model rather than requiring an operator choice
> - This pull request makes new-agent creation disable an available
cheap profile by default while preserving explicit opt-in from the UI or
API
> - The Codex cheap profile now remains available for explicit
configuration but supplies no model default, so an unconfigured cheap
request stays on the primary model
> - The benefit is predictable model quality for new agents and an
intentional, auditable choice before lower-cost routing is enabled

## Linked Issues or Issue Description

**Problem**

New agents created with an adapter that exposes a `cheap` model profile
can inherit that profile without the operator explicitly enabling it. In
the UI, enabling the adapter-default cheap model is also omitted because
runtime configuration is only written when a custom model ID is present.

**Expected behavior**

- New agents default an available `cheap` model profile to `{ enabled:
false }` when the caller does not specify it.
- Explicit API configuration remains authoritative.
- UI opt-in persists even when the adapter default model is used.
- Codex does not advertise or automatically select
`gpt-5.3-codex-spark`; operators must explicitly configure any
lower-cost Codex model.

**Related public work**

- Refs #4881, which introduced cheap model profiles for local adapters.
- Supersedes the default-selection portions of #8032 and #10004 by
removing the Codex model default instead of replacing it with another
hardcoded model.

## What Changed

- Detect whether the selected adapter exposes a `cheap` model profile
during agent creation and hiring.
- Persist `runtimeConfig.modelProfiles.cheap.enabled = false` only when
the caller did not explicitly configure the profile.
- Preserve UI cheap-profile opt-in when using the adapter's default
model by writing an empty adapter config.
- Remove `gpt-5.3-codex-spark` from the Codex static model list.
- Keep the Codex `cheap` profile explicitly configurable while giving it
an empty adapter config, so Paperclip never chooses a cheap Codex model
automatically.
- Verify that a Codex cheap request without an explicit model leaves the
primary model unchanged.
- Extend server route and UI runtime-config tests for default-disable
and explicit-opt-in behavior.

## Verification

- `env -u PAPERCLIP_IN_WORKTREE -u PAPERCLIP_WORKTREE_NAME -u
PAPERCLIP_CONFIG -u PAPERCLIP_HOME -u PAPERCLIP_INSTANCE_ID -u
PAPERCLIP_CONTEXT pnpm exec vitest run
packages/adapters/codex-local/src/index.test.ts
packages/adapters/codex-local/src/server/codex-args.test.ts
server/src/__tests__/adapter-models.test.ts
server/src/__tests__/adapter-registry.test.ts
server/src/__tests__/heartbeat-model-profile.test.ts
server/src/__tests__/agent-permissions-routes.test.ts
ui/src/lib/new-agent-runtime-config.test.ts`
- Result: 7 test files passed, 105 tests passed.
- GitHub `Typecheck + Release Registry` check passed on the final head.
- `git diff --check public-gh/master...HEAD`

## Risks

- Low behavioral risk: only newly created or hired agents are
normalized; existing agents are unchanged.
- Explicit `cheap` profile settings remain untouched, including explicit
opt-in.
- Codex users who explicitly opt into the cheap lane must choose a
model; requests without a configured override intentionally continue on
the primary model.
- Adapter profile discovery is now awaited during creation, adding a
small amount of adapter metadata lookup work.
- The source branch name is automation-provided and retained as required
by the task, so it does not satisfy the preferred public branch naming
convention.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI `gpt-5.4` via Codex CLI, with reasoning, repository editing,
terminal execution, and GitHub/Paperclip tool access. The runtime did
not expose a context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-27 19:11:51 -05:00

81 lines
2.3 KiB
TypeScript

// @vitest-environment node
import { describe, expect, it } from "vitest";
import { AGENT_DEFAULT_MAX_CONCURRENT_RUNS } from "@paperclipai/shared";
import { buildNewAgentRuntimeConfig } from "./new-agent-runtime-config";
describe("buildNewAgentRuntimeConfig", () => {
it("defaults new agents to no timer heartbeat", () => {
expect(buildNewAgentRuntimeConfig()).toEqual({
heartbeat: {
enabled: false,
intervalSec: 300,
wakeOnDemand: true,
skipTimerWhenNoActionableWork: true,
cooldownSec: 10,
maxConcurrentRuns: AGENT_DEFAULT_MAX_CONCURRENT_RUNS,
},
});
});
it("preserves explicit heartbeat settings", () => {
expect(
buildNewAgentRuntimeConfig({
heartbeatEnabled: true,
intervalSec: 3600,
}),
).toEqual({
heartbeat: {
enabled: true,
intervalSec: 3600,
wakeOnDemand: true,
skipTimerWhenNoActionableWork: true,
cooldownSec: 10,
maxConcurrentRuns: AGENT_DEFAULT_MAX_CONCURRENT_RUNS,
},
});
});
it("stores cheap model under modelProfiles.cheap, not primary adapterConfig", () => {
const config = buildNewAgentRuntimeConfig({
heartbeatEnabled: true,
intervalSec: 600,
cheapModel: "claude-sonnet-4-6",
cheapModelEnabled: true,
});
expect(config.modelProfiles).toEqual({
cheap: {
enabled: true,
adapterConfig: { model: "claude-sonnet-4-6" },
},
});
// primary heartbeat config still present
expect(config.heartbeat).toMatchObject({ enabled: true, intervalSec: 600 });
});
it("omits modelProfiles when no cheap model is configured", () => {
const config = buildNewAgentRuntimeConfig({ heartbeatEnabled: false });
expect(config.modelProfiles).toBeUndefined();
});
it("persists explicit cheap-profile opt-in when using the adapter default", () => {
const config = buildNewAgentRuntimeConfig({
cheapModelEnabled: true,
});
expect(config.modelProfiles).toEqual({
cheap: {
enabled: true,
adapterConfig: {},
},
});
});
it("omits modelProfiles when cheap model is set but explicitly disabled", () => {
const config = buildNewAgentRuntimeConfig({
cheapModel: "claude-sonnet-4-6",
cheapModelEnabled: false,
});
expect(config.modelProfiles).toBeUndefined();
});
});