Files
PaperClipAI/tests/canary-onboarding
Devin FoleyandPaperclip c2163b3ecf test(release): verify paused onboarding setup records (#18102)
## Thinking Path

> - Paperclip manages agents and controls when their work starts.
> - Onboarding saves an unfinished agent before account connection.
> - The setup hold keeps that record paused until Connect completes.
> - The published canary smoke still required an empty agent list at
that point.
> - That old assertion rejected the documented setup state and blocked
release work.
> - These tests now check the setup hold and the absence of early work.
> - The nightly smoke also checks that Connect completes the same
record.

## Linked Issues or Issue Description

**What happened?**

Published canary onboarding failed at the empty-agent-list assertion
after reaching Connect a model. The API returned the expected paused
setup record. The same failure affected recent canaries and prevented
their promotion to a nightly.

**Expected behavior**

Require exactly one named setup record before Connect, paused with the
`setup` hold and heartbeat disabled. Require no tasks or runs. After
successful Connect, complete that same agent record.

**Steps to reproduce**

Run the published canary onboarding smoke with
`PAPERCLIPAI_VERSION=2026.1028.0-canary.15`. Enter the organization and
agent names, then open Connect a model. The old assertion fails on the
held setup record.

**Paperclip version or commit**

The published artifact is `2026.1028.0-canary.15` at
`743698e700638cddc4d3fc4c0ad89e6c132fab40`. The test branch starts at
`65fa1cd617b43a45c9d22e10480f4fb6e52dbd21`.

**Deployment mode**

The canary check installs the exact public npm package in a fresh
local-trusted instance. The authenticated nightly smoke uses the
existing Docker harness and its provider mock.

Refs #17976 and #12667. The existing agent setup contract is in
`doc/SPEC-implementation.md` section 10.3.

## What Changed

- Replace the stale empty-agent assertion with a bounded check for one
paused setup record, disabled heartbeats and no tasks or runs.
- Check the same setup state in the authenticated nightly smoke. Require
Connect to complete that record without creating a second agent.
- Update the onboarding verification recipe with the scope of both
release checks.

## Verification

- Red: the unchanged smoke failed locally against the exact published
`.15` artifact at the empty-agent assertion. This matches the [release
failure](https://github.com/paperclipai/paperclip/actions/runs/40761030399/job/123112508486).
- Green: the revised smoke passed against the same `.15` artifact, with
a real browser and fresh data. No model credential was required.
- Full `pnpm build` and `node scripts/check-maps.mjs` pass.
- Full recursive `pnpm -r typecheck` passes.
- Full local `pnpm test:run` hit embedded PostgreSQL initialization
failures. A standalone probe of the same bundled `initdb` reproduced
`could not create shared memory segment: No space left on device`; this
Mac had exhausted its shared-memory IDs. The local suite was stopped
after those failures. No full local-suite pass is claimed.
- The additional old-nightly artifact probe stopped before the browser
journey with the same PostgreSQL bootstrap failure. Its failure is
retained separately from the canary red/green result.
- All 54 current-head checks pass; two optional checks are skipped. [CI
run](https://github.com/paperclipai/paperclip/actions/runs/40779358071).
- [Apex review is
5/5](https://github.com/paperclipai/paperclip/pull/18102#issuecomment-6482273653)
for `40c63db4e9bf66e915843a9f325763d449aee049`, with no unresolved
review threads.
- The first [authenticated Linux canary
smoke](https://github.com/paperclipai/paperclip/actions/runs/40779405110)
stopped before the browser because the canary tag moved to `.16` before
its matching server dependency was published. That dependency is now
available and the failed job has been retried. Both the canary and
[nightly
smoke](https://github.com/paperclipai/paperclip/actions/runs/40780591068)
passed background-service installation. Their authenticated Docker
browser jobs are still queued for GitHub-hosted Ubuntu runners. Full
authenticated browser acceptance remains unverified pending runner
allocation.

## Risks

These are test and verification-document changes. Unexpected agent
states, extra records, early tasks and early execution still fail the
checks. The authenticated smoke uses the existing mocked provider
acceptance; it does not qualify real provider authentication.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, tool use and code execution.
The session does not expose a more specific model ID or the
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass — targeted canary smoke;
full local-suite limitation is recorded above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 16:03:08 -07:00
..