Commit Graph
2 Commits
Author SHA1 Message Date
Devin FoleyandPaperclip 4cc387f907 fix(ci): remove npm propagation from cloud readiness (#13456)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud needs a verified image and matching database
migrator before it can deploy a merge.
> - New npm package versions can take minutes to become downloadable
after the package build finishes.
> - The direct producer now publishes signed archives and a complete
dependency lockfile for each master commit.
> - This pull request makes readiness verify those artifacts and removes
the duplicate automatic npm migrator run.
> - Deployment still requires all source checks, exact image identity,
migration compatibility, and pinned dependencies.

## Linked Issues or Issue Description

Refs: #13455, #13454, #13192

**What existing behavior does this improve?**
The time from a master merge to the `Cloud deployable v1` signal.

**Current behavior**
Readiness polls npm metadata for the new DB and shared versions. An
automatic dispatcher also starts a separate npm-only migrator workflow.
A measured source built its packages at 06:22:41 UTC on 2026-09-15, but
both npm archives were not downloadable until 06:31:56 UTC.

**Proposed behavior**
Wait for the successful exact-source direct producer, verify its signed
manifest and all pinned downloads, and publish readiness only after the
existing source and image jobs pass. Keep manual npm migrators and
branch previews available.

**Reason and benefit**
Remove new-version npm propagation from merge-to-deployable time. The
gain depends on whether image building or source verification finishes
later; it is not a fixed subtraction from every run.

## What Changed

- Require a successful producer from the canonical repository, exact
commit, master ref, expected workflow, and approved event.
- Verify the manifest's GitHub attestation with the hosted GitHub CLI.
Enforce the exact source SHA, master workflow identity, and hosted
runner.
- Download and validate both archives and the complete dependency
lockfile after publication succeeds. Reject invalid signatures,
inaccessible objects, corrupt bytes, and source mismatches.
- Remove automatic npm-only migrator dispatch. Retain manual release and
branch-preview publication.
- Document the cloud feature-switch prerequisite and coordinated
rollback.

## Verification

- `node --test .github/scripts/tests/*.test.mjs`: 405 pass.
- Focused readiness, routing, preview, and artifact tests: 249 pass.
- Workflow lint and `git diff --check`: pass.
- `pnpm test:release-registry`: 139 pass after installing this
worktree's dependencies.
- All latest-head GitHub CI checks passed. Greptile is 5/5 with no
unresolved comments.
- Application source is unchanged. Common-source local typecheck and
build passed. The full local application suite has the documented macOS
read-only-directory rename limitation from #13454 (13 failures in two
unchanged suites); Linux CI is the final application gate.
- Live readiness verification of master
da77a0c28c passed in 6.52 seconds,
including the real GitHub CLI signature policy and all artifact
downloads.
- Cloud consumer resolution with the certificate encoding fix passed in
5.45 seconds with zero npm metadata requests or npm processes. The
consumer is deployed and enabled in staging and production; their live
resolution APIs passed in 2.34 and 2.38 seconds. Both report the
expected fixed harness commit. A fresh tenant deployment follows this
cutover merge.

## Risks

- `Cloud deployable v1` no longer promises npm preview availability.
Enable the cloud direct-artifact consumer in staging and production
before merging this change.
- Artifact storage and GitHub attestations become required services for
new direct releases. Missing or invalid evidence fails explicitly.
- Restore the old npm dispatcher and readiness gate together before
disabling the consumer switch. Retain artifacts referenced by existing
releases.
- This change does not expand AWS runner access. The producer and
readiness bookkeeping use GitHub-hosted runners. Existing PR allowlists
and source verification gates remain enforced.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused checks; full
application host limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-15 01:14:43 -07:00
Devin FoleyandPaperclip da77a0c28c feat(ci): publish immutable cloud migrator artifacts (#13455)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud deploys images and a matching database migrator.
> - New migrator versions must currently become available on npm before
cloud can use them.
> - npm can serve package metadata while the named archive still returns
404.
> - This pull request publishes immutable migrator archives and a
complete dependency lockfile through the existing artifact store.
> - Cloud can install these exact packages without waiting for their new
npm versions.
> - This producer change prepares a separate cloud consumer and
readiness cutover.

## Linked Issues or Issue Description

Refs: #13454

**What happened?**
A recent master run built both packages by 06:22:41 UTC on 2026-09-15.
Both archives became downloadable from npm at 06:31:56 UTC. Fresh
metadata requests did not remove the delay.

**What did you expect to happen?**
Cloud should be able to install the verified migrator as soon as its
package build and artifact upload finish.

**Steps to reproduce**
Compare package build completion, npm publication, version metadata
availability, and tarball download availability for a fresh full commit
SHA.

**Version**
Master commit `08adcc70d5ec45b7ced9619a3dc10c1d1bec397d`.

## What Changed

- Add a master-only workflow that builds the DB and shared archives
without publication credentials.
- Resolve the dependency lockfile from local archives, then pin those
archives to content-addressed URLs.
- Publish the complete bundle to a separate prefix in the existing
S3/CloudFront artifact store. Write the commit manifest last and verify
public downloads.
- Add a dedicated OIDC role policy. Only canonical master can assume it.
Writes require `If-None-Match: *`; the role cannot overwrite or delete
objects.
- Add source, integrity, lockfile, publication, and real npm install
tests. Document the format and staged rollout.
- Attest the validated manifest with GitHub/Sigstore before S3
publication. The signature binds every package and lockfile hash to the
exact master workflow and source commit.

## Verification

- `node --test scripts/cloud-migrator-artifacts.test.mjs`: 7 tests pass,
including real `npm ci` with an empty cache and no new-version metadata
lookup.
- `pnpm test:release-registry`: 136 tests pass.
- `actionlint .github/workflows/cloud-migrator-artifacts.yml` and `git
diff --check`: pass.
- Ran the workflow's filtered install and package build against the
exact master source. Built and validated the dependency lockfile from
those real archives.
- Latest-head application tests passed, including reruns of two failures
in unchanged application tests. The final CI aggregate passed. The
application source is unchanged. Common-source local typecheck and build
passed; the full local suite has the same documented macOS
read-only-directory rename limitation as #13454 (13 failures in two
unchanged suites).
- The dedicated role and additive bucket read permission are configured.
IAM simulation allows only conditional writes in the intended prefix;
overwrite without the condition, other prefixes, and deletion are
denied.
- Published the verified master 08adcc70d5
bundle with the operator session and verified all public downloads.
GitHub OIDC publication is still pending the master workflow run.
- Cloud resolved the real bundle and checked all 278 SQL migrations in
2.9 seconds with zero npm metadata requests or npm processes. The
existing migration runner applied it to a disposable local PostgreSQL
database and succeeded again on repeat.
- The producer now requires an empty-cache smoke install of the actual
package archives and their full dependency graph before upload. That
check and imports of both installed packages passed locally.

## Risks

- This is an additive producer rollout. It does not yet change the cloud
resolver or the deployable marker.
- The dedicated role and bucket read statement must be installed before
the workflow can publish. Existing bucket policy statements and
public-access blocks must be preserved.
- Referenced artifacts must be retained for rollback. No expiry rule
applies to this prefix.
- Existing external dependencies still download from npm, with SHA-512
pins. New DB and shared versions do not require npm metadata.
- The workflow uses GitHub-hosted runners and has no PR trigger. It adds
no AWS compute routing or PR access.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The exact serving model ID and context window are not exposed
by this environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused release and
real-artifact tests; full-suite host limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-15 00:46:05 -07:00