0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-02 16:58:52 +01:00

Files

T

6874fed05c feat(adapters): refresh model catalogs and sandbox CLI pins (Oct 23 audit) (#17809 )

<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/26/changes/paperclip/26
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents run on coding harnesses. Each adapter has a model catalog,
and sandboxes contain pinned CLIs.
> - Upstream providers released new CLI versions and new models. OpenAI
retired some Codex models for ChatGPT sign-in.
> - A stale catalog hides valid models. A misleading error makes
operators upgrade a CLI that is already current.
> - This pull request adds verified OpenCode model IDs, names Codex
ChatGPT retirements in the environment Test, and refreshes the sandbox
CLI pins.
> - The benefit is that operators can select current models and get the
correct reason when a retired model fails.

## Linked Issues or Issue Description

Refs #16708 (adds Claude Haiku 5.5 to `claude_local`; this PR does not
duplicate it).
Refs #17245 (moves the Codex runtime to 0.163.0; this PR does not change
Codex pins).

**What happened?**

OpenAI retired `gpt-5.5` (October 14, 2026) and `gpt-5.4` /
`gpt-5.4-mini` (August 31, 2026) from Codex with ChatGPT sign-in. A
saved agent with one of these models and ChatGPT sign-in now fails. The
legacy environment Test says that the plan, the rollout, or an old CLI
can be the cause. The Paperclip Runner Test gives generic account and
runtime advice. The real cause is the retirement.

**Expected behavior**

Both Tests give the retirement date and the documented replacement.
API-key users keep the models.

**Proposed change**

Record the verified retirements. Add the new verified OpenCode IDs.
Refresh the sandbox CLI pins with verified digests. Record the audit.

## What Changed

- Codex: add a table of models retired from Codex with ChatGPT sign-in
(`gpt-5.5` → `gpt-6.1-sol`, or `gpt-6-luna` on Free and Go; `gpt-5.4` →
`gpt-6-sol`; `gpt-5.4-mini` → `gpt-6-luna`). The shared
`codexChatGptRetirementHint` builds the hint. The legacy CLI Test
(`codexHelloProbeModelRejectionCheck`) and the Paperclip Runner Test
(`testNativeRunnerAuthentication`) both use it when the backend rejects
one of these models. For other failures, the Paperclip Runner Test hint
stays the same. The picker keeps the models for API-key users.
- OpenCode: add `anthropic/claude-haiku-5-5` and `google/gemini-4-argon`
to the remote fallback catalog. The Paperclip Runner OpenCode picker
also uses this catalog.
- Evaluation image sandbox layer (`docker/daytona-runner`):

  | CLI | Before | After |
  | --- | --- | --- |
  | Grok CLI | 1.0.46 | 1.0.70 |
  | Gemini CLI | 0.62.0 | 0.65.0 |
  | Kimi Code | 2.1.1 | 2.3.3 |
  | Cursor CLI | 2026.10.01-e373342 | 2026.10.22-27dcf5f |
  | GitHub CLI | 2.102.0 | 2.103.3 |

- Add `doc/adapter-model-audit-2026-10-23.md` with sources,
compatibility findings, and deferred items.

No model in this PR needs a newer Codex, Claude, or OpenCode runtime
than the current pins. Claude Haiku 5.5 needs Claude Code 2.1.293 or
later; #16708 records that floor, and this PR does not add it.

## Verification

- `vitest run src/index.test.ts src/server/test.remote.test.ts
src/server/acp.cli-version.test.ts` in `packages/adapters/codex-local`:
41 passed.
- `vitest run src/services/native-runtime/setup-readiness.test.ts` in
`server`: 98 passed. These include the two new Paperclip Runner Test
cases: a retired model shows the retirement hint, and a model that is
not retired keeps the generic hint. The retirement case fails on the
earlier code. Run with the host's empty `ANTHROPIC_BASE_URL` unset; see
the baseline note in the audit doc.
- `vitest run` in `packages/adapters/opencode-local`: 87 passed.
- `vitest run src/__tests__/adapter-models.test.ts` in `server`: 27
passed.
- `tsc --noEmit` in `packages/adapters/codex-local` and
`packages/adapters/opencode-local`: passed.
- Cursor archive SHA-256
`85296944e4600a83c4b764f578d1502bf9c416db200d33a593e8f3ecabfdc294` is
computed from the versioned Linux x64 tarball. The extracted binary
reports `2026.10.22-27dcf5f`. On October 26, 2026,
`https://cursor.com/install` still resolves this version.
- GitHub CLI SHA-256
`4c806978ffce1cec70853302a56d70ac6da4f19bae7adca4e1a8ab00633dd91d`
matches `gh_2.103.3_checksums.txt`.
- Grok CLI 1.0.70 and Kimi Code 2.3.3 installed from npm report those
versions. The flags that Paperclip uses (`-m`, `--effort`, `grok
models`; `-m`, `-p`, `--output-format`, `acp`) are present in `--help`.
- `git merge-tree` with #16708 (head `3a06740e9`) and #17245 (head
`41f9c611f`): this PR shares no files with either PR. The two PRs can
merge in any order.
- Not run in the authoring environment: the full `pnpm -r typecheck` and
`pnpm build` (no Rust toolchain), the full server `tsc` (memory limit),
and the image build. CI runs these.

## Risks

- Low risk for catalogs: the IDs are additive suggestions. No saved
selection, default, or spending setting changes.
- The Paperclip Runner Test changes only the hint text, and only for a
ChatGPT rejection of a retired model. Check codes and pass/fail results
do not change.
- Sandbox CLI pins:
- Kimi Code 2.2.0 and 2.3.0 remove the `kaos` MCP executor and require
qualified plugin skill names. Paperclip uses neither. Kimi Code 2.3.2
stops automatic swarms in `kimi -p` and ACP sessions.
- Gemini CLI 0.65.0 makes workspace settings read-only in untrusted
folders. Paperclip writes home settings and sets
`GEMINI_CLI_TRUST_WORKSPACE` for remote runs.
- xAI publishes no Grok CLI changelog. Grok 1.0.70 behavior changes are
verified only against the CLI reference and `--help`.
- The pins apply only after a new image is built and promoted. The fleet
sandbox image definition must get the same pins.
- Rollback: revert this PR.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell,
web research, repository search).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-26 16:32:13 -07:00

8.0 KiB

Raw Blame History

Adapter model and harness audit: October 23, 2026

This audit follows the October 2, 2026 audit. It covers model selection in the coding-agent adapters and the CLI pins in the evaluation sandbox image. Catalog entries identify models; provider accounts and installed CLIs determine access. No agent defaults or saved model selections are migrated.

Model changes

Adapter	Changes from the audit
Codex	No new selectable model IDs. Codex 0.160.0 through 0.164.0 ship the same bundled model set, which the picker already lists (including GPT-6 Astra, GPT-5.6 Terra, and GPT-5.6 Luna). OpenAI retired `gpt-5.5` (October 14, 2026) and `gpt-5.4` / `gpt-5.4-mini` (August 31, 2026) from Codex with ChatGPT sign-in. The API and API-key Codex sessions still serve them, so they stay in the picker. When the backend rejects one of them for a ChatGPT account, the environment Test (legacy CLI and Paperclip Runner) now names the retirement and the documented replacement instead of suggesting a stale CLI, a plan restriction, or generic account checks.
OpenCode	Add `anthropic/claude-haiku-5-5` and `google/gemini-4-argon` to the static fallback used by remote environments and the Paperclip Runner picker. Both IDs are in the OpenCode model registry (models.dev).
Claude Code (direct and Bedrock)	Claude Haiku 5.5 is covered by #16708; this audit does not duplicate it. Haiku 5.5 needs Claude Code 2.1.293 or later and supports all five effort levels.
Gemini CLI	Gemini 4 Argon (`gemini-4-argon`) is generally available on the Gemini API, but Gemini CLI 0.65.0 does not register it. Deferred.
Grok Build, Kimi Code, Cursor	No new verified model IDs. Grok 4.7 and Kimi K3 remain the newest documented models.

Harness changes

Harness	Before	After
Grok CLI (`@xai-official/grok`)	1.0.46	1.0.70
Gemini CLI	0.62.0	0.65.0
Kimi Code CLI	2.1.1	2.3.3
Cursor CLI (sandbox layer)	2026.10.01-e373342	2026.10.22-27dcf5f
GitHub CLI	2.102.0	2.103.3
Codex CLI (shared provider pack)	0.160.0	Unchanged here; #17245 moves it to 0.163.0
Claude Agent SDK / Claude Code	0.3.286 / 2.1.286	Unchanged; see "Deferred items"
OpenCode (shared provider pack)	1.18.34	Unchanged; see "Deferred items"
Hermes	0.19.0	0.19.0 (newest on PyPI)

These pins change only the evaluation image's sandbox layer. They must stay aligned with the fleet sandbox image definition.

Sources and verification

Research date: October 23, 2026. Harness pins rechecked October 26, 2026.

Codex models states the GPT-5.5 retirement ("Effective October 14, 2026 ... GPT-5.5 retires from Codex for users signed in with ChatGPT"), its replacements (GPT-6.1 Sol on paid plans, GPT-6 Luna on Free and Go), the GPT-5.4 retirement and its replacements (gpt-6-sol, gpt-6-luna), and that API-key sessions are not affected.
The bundled codex-rs/models-manager/models.json is unchanged in model set from rust-v0.160.0 to rust-v0.164.0, the npm latest release on October 26, 2026.
Claude Haiku 5.5 was released October 7, 2026. The Claude Code changelog entry for 2.1.293 adds claude-haiku-5-5.
models.dev lists anthropic/claude-haiku-5-5 and google/gemini-4-argon. The OpenCode 1.18.34 executable's bundled snapshot predates both. Paperclip Runner binds the selected model into the OpenCode provider configuration, so the snapshot does not limit it.
Gemini 4 Argon is generally available since October 13, 2026. Gemini CLI 0.65.0 (packages/core/src/config/models.ts) has no entry for it, and pull requests adding it to the CLI are still open.
Grok CLI 1.0.70, Gemini CLI 0.65.0, and Kimi Code 2.3.3 are the npm latest tags on October 26, 2026. xAI publishes no Grok CLI changelog; -m, --effort, and grok models are unchanged in the CLI reference and in grok --help from 1.0.70. Kimi Code 2.2.0 and 2.3.0 remove the kaos MCP executor and require qualified plugin skill names; Paperclip uses neither. Kimi Code 2.3.2 stops the agent from launching swarms by itself, so kimi -p and ACP sessions no longer start swarms; 2.3.1 and 2.3.3 are fixes and prompt changes. -m, -p, --output-format, and acp are unchanged in kimi --help from 2.3.3. Gemini CLI 0.65.0 makes workspace settings read-only in untrusted folders; Paperclip writes only home settings and sets GEMINI_CLI_TRUST_WORKSPACE for remote runs.
Cursor CLI 2026.10.22-27dcf5f is the version https://cursor.com/install resolves. The archive SHA-256 was computed from the versioned Linux x64 tarball, and the extracted executable reports that version.
GitHub CLI 2.103.3: the pinned digest matches the release checksums.txt. 2.103.1 through 2.103.3 are fixes on top of 2.103.0.
Hermes on PyPI remains at 0.19.0. Newer Hermes releases (v0.22.4) are published only through GitHub, the installer, and Docker.

Deferred items

Claude runtime 2.1.286 → 2.1.308. Claude Code's stable channel is 2.1.308 (latest is 2.1.309), with Agent SDK 0.3.308. The range changes headless defaults that Paperclip runs depend on: fork subagents are on by default in -p and the Agent SDK (2.1.302), sessions with no permission mode start in default mode on third-party providers (2.1.302), and MCP allow rules match whole server names (2.1.299). This needs a separate runner qualification together with the patched claude-agent-acp bridge. Until then, Haiku 5.5 on the CLI lane needs a 2.1.293+ CLI. Once #16708 records that minimum, sandbox runs on an older image prepare a private CLI at the model minimum before launch.
OpenCode 1.19.x and 2.x. OpenCode 1.19.3 generates reasoning variants from model options instead of catalog variants, which affects the variants Paperclip Runner binds. OpenCode 2 changes the SDK, event, and plugin contracts. Both need qualification.
ACP bridges. ACPX 0.20.0, claude-agent-acp 0.94.0 (ACP SDK 2.0), and codex-acp 2.2.2 (breaking tool-call contract in 2.0.0) need a qualification campaign rather than a pin bump.
GitHub Copilot CLI 1.0.103. The qualified Copilot profile binds Copilot 1.0.88 by immutable identity; a new profile version needs its own qualification. Copilot discovers models from the ACP session, so new Copilot models need no catalog change.
Pi 1.4.0 and Hermes 0.22.x need runtime qualification; Hermes no longer publishes to PyPI.
Gemini 4 Argon in the Gemini CLI and Cursor catalogs: the CLI does not register it yet, and Cursor documents display names only.
Codex default model. Codex 0.161.0 makes gpt-6.1-sol its default. Paperclip keeps gpt-5.6-sol so saved agents and spending do not change.
Hidden Codex aliases (gpt-daybreak-*) are restricted to eligible accounts.
Claude Sonnet 4.5 is deprecated and retires November 30, 2026. It stays in the picker until then.
Claude Mythos 5.1 remains restricted to verified organizations.

Host-environment baseline failures

Recorded once for this audit on the unchanged base (c5ca29d), Node 25.9.0:

server/src/__tests__/agent-hire-ai-connections.test.ts fails at import with AI_InvalidArgumentError: baseURL must be a non-empty string. because the host injects an empty AI provider URL. Unrelated to this change.

8.0 KiB Raw Blame History