Build isolated preview artifacts for exact-source deployments (#13041)

## Thinking Path > - Paperclip manages AI agents and their work. > - Managed deployments need a cloud image and a database migration package. > - Branch commits can lack both artifacts until a normal release runs. > - Operators need to test an exact commit without advancing release aliases. > - This pull request adds a preview build mode to the existing release workflow. > - Builds use an immutable source SHA and publish isolated, reusable artifacts. ## Linked Issues or Issue Description **Subsystem affected** Release automation, cloud Docker images, and shared/database npm packages. **Problem or motivation** An operator cannot deploy an unpublished branch with new migrations using only the normal release artifacts. Publishing it through a normal lane would also advance shared release aliases. **Proposed solution** Dispatch the trusted release workflow on master with a full source SHA and a request UUID. Build missing SHA images and, when needed, deterministic preview shared/DB packages. Publish packages under the preview dist-tag with exact workspace pins. Reuse matching artifacts on retries. **Roadmap alignment** This extends release tooling for operator validation. It does not add a core product feature or duplicate a planned product capability. Related PR searches found no duplicate preview deployment workflow. ## What Changed - Add the preview channel, request correlation, artifact checks, and result artifact. - Compile source packages in a separate job from the npm publisher. The publisher uses trusted master code and disables package lifecycle scripts. - Publish only SHA cloud image tags. Preserve release aliases. Use full-SHA tags and no shared build cache. - Verify full source identity for reused packages and images. Both image and npm publishers use isolated jobs and the externally master-restricted npm-canary environment. Fail on registry authentication errors, outages, or artifact identity mismatches. - Let bundled-package preparation use patches from the requested source checkout. - Document publishing configuration, artifact contracts, and deployment order. ## Verification - Passed `pnpm -r typecheck` and `pnpm build`. - Passed `pnpm test:release-registry`: 107 tests, including eight preview tests. - Passed `actionlint -shellcheck= .github/workflows/release.yml`. - Built real shared and DB preview tarballs from an isolated exact-SHA checkout. Verified package source identity and all 244 SQL files and journal entries. - Verified the full revision behind an existing published SHA cloud image. - `pnpm test:run` exposed missing local embedded PostgreSQL library symlinks. The package's postinstall repair restored initdb; all 12 previously affected suites passed on rerun (95 tests). Additional local matrix reruns are in progress. The complete PR CI matrix is green, including general/serialized tests, e2e, typecheck, build, release registry, canary dry run, and the required verify gate. - Live preview publication and staging deployment require this workflow on master and the compatible control-plane backend. They have not run yet. No production deployment was performed. ## Risks Preview npm versions are immutable public artifacts. Both packages must retain their trusted publisher for release.yml in environment npm-canary. Source builds must remain separated from privileged npm publishing. The deploying control plane must verify source identity, integrity, and migration compatibility before use. Normal release jobs retain their existing conditions. Roll back by stopping preview dispatches and reverting the workflow/tooling. Published preview versions remain isolated from normal release tags. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution, and test runs. The session does not expose a more specific model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in-PR following the feature template - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [ ] I have run the full tests locally and they pass - [x] I have added tests for the new behavior - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open recommendations or follow-ups - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-07 07:23:08 +02:00 · 2026-09-08 09:21:58 -05:00
1 parent b97101893f
commit 0cc796b7bd
@@ -1,4 +1,5 @@
 name: Release
+run-name: ${{ inputs.channel == 'preview' && format('Stack deploy {0} build', inputs.request_id) || 'Release' }}

 on:
  push:
@@ -17,12 +18,21 @@ on:
          - stable
          - beta
          - nightly
+          - preview
        default: stable
      source_ref:
-        description: (stable) Commit SHA, branch, or tag to publish as stable
+        description: Stable source ref, or full immutable SHA for a preview build
        required: true
        type: string
        default: master
+      request_id:
+        description: (preview) CLI correlation UUID
+        type: string
+        default: ""
+      preview_migrator:
+        description: (preview) Publish isolated shared and database packages if missing
+        type: boolean
+        default: false
      stable_date:
        description: Enter a UTC date in YYYY-MM-DD format, for example 2026-03-18. Do not enter a version string. The workflow will resolve that date to a stable version such as 2026.318.0, then 2026.318.1 for the next same-day stable.
        required: false
@@ -46,7 +56,7 @@ on:
        default: false

 concurrency:
-  group: release-${{ github.event_name }}-${{ github.ref }}
+  group: ${{ inputs.channel == 'preview' && format('preview-{0}', inputs.source_ref) || format('release-{0}-{1}', github.event_name, github.ref) }}
  cancel-in-progress: false

 env:
@@ -64,6 +74,219 @@ env:
  NPM_PUBLISH_VERIFY_DELAY_SECONDS: "10"

 jobs:
+  plan_preview:
+    name: Check preview artifacts
+    if: github.ref == 'refs/heads/master' && github.event_name == 'workflow_dispatch' && inputs.channel == 'preview' && !inputs.dry_run
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    outputs:
+      image: ${{ steps.plan.outputs.image }}
+      packages: ${{ steps.plan.outputs.packages }}
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+      - name: Validate immutable source and inspect existing artifacts
+        id: plan
+        env:
+          SOURCE_SHA: ${{ inputs.source_ref }}
+          REQUEST_ID: ${{ inputs.request_id }}
+          PREVIEW_MIGRATOR: ${{ inputs.preview_migrator }}
+        run: node scripts/preview-artifacts.mjs plan "$SOURCE_SHA" "$REQUEST_ID" "$PREVIEW_MIGRATOR"
+
+  package_preview:
+    name: Build preview migrator
+    needs: plan_preview
+    if: needs.plan_preview.outputs.packages == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          path: trusted
+          persist-credentials: false
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          ref: ${{ inputs.source_ref }}
+          path: source
+          persist-credentials: false
+      - uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+          run_install: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+      - name: Install build dependencies without lifecycle scripts
+        working-directory: source
+        run: pnpm install --ignore-scripts --no-frozen-lockfile
+      - name: Build and pack exact-source preview packages
+        env:
+          SOURCE_SHA: ${{ inputs.source_ref }}
+        run: node trusted/scripts/preview-artifacts.mjs pack source packages "$SOURCE_SHA"
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+        with:
+          name: preview-packages
+          overwrite: true
+          path: packages/*.tgz
+          if-no-files-found: error
+          retention-days: 7
+
+  publish_preview:
+    name: Publish preview migrator
+    needs: [plan_preview, package_preview]
+    if: github.ref == 'refs/heads/master' && needs.plan_preview.outputs.packages == 'true' && needs.package_preview.result == 'success'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    # Reuse release.yml's established npm trusted-publisher identity. This job
+    # publishes only isolated preview versions; it cannot advance lane tags.
+    environment: npm-canary
+    permissions:
+      contents: read
+      id-token: write
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+      - name: Install npm with trusted publishing support
+        run: npm install --global npm@11.18.0 --ignore-scripts
+      - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
+        with:
+          name: preview-packages
+          path: preview-packages
+      - name: Publish immutable preview packages without running package code
+        env:
+          SOURCE_SHA: ${{ inputs.source_ref }}
+        run: node scripts/preview-artifacts.mjs publish preview-packages "$SOURCE_SHA"
+
+  image_preview:
+    name: Build preview cloud image
+    needs: plan_preview
+    if: needs.plan_preview.outputs.image == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          ref: ${{ inputs.source_ref }}
+          persist-credentials: false
+      - uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+          run_install: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+      - name: Prepare locked image context
+        env:
+          SOURCE_SHA: ${{ inputs.source_ref }}
+        run: |
+          set -euo pipefail
+          test "$(git rev-parse HEAD)" = "$SOURCE_SHA"
+          pnpm install --resolution-only --ignore-scripts --ignore-pnpmfile --no-frozen-lockfile
+          echo "TOOLS_EPOCH=$(date -u +%G-W%V)" >> "$GITHUB_ENV"
+      - uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
+      - name: Build the immutable cloud image without registry credentials
+        uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7
+        with:
+          context: .
+          target: cloud
+          platforms: linux/amd64
+          push: false
+          provenance: false # Docker archives cannot carry registry attestations.
+          outputs: type=docker,dest=${{ runner.temp }}/preview-image.tar
+          tags: ghcr.io/paperclipai/paperclip:sha-${{ inputs.source_ref }}-cloud
+          build-args: |
+            CLOUD_BUNDLED_PLUGINS=daytona
+            CLOUD_BUNDLED_SERVER_DEPS=@sentry/node
+            PAPERCLIP_BUILD_COMMIT=${{ inputs.source_ref }}
+            PAPERCLIP_BUILD_VERSION=0.0.0-preview.g${{ inputs.source_ref }}
+            CLI_TOOLS_CACHE_EPOCH=${{ env.TOOLS_EPOCH }}
+          labels: |
+            org.opencontainers.image.revision=${{ inputs.source_ref }}
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+        with:
+          name: preview-image
+          overwrite: true
+          path: ${{ runner.temp }}/preview-image.tar
+          compression-level: 0
+          if-no-files-found: error
+          retention-days: 1
+
+  publish_image_preview:
+    name: Publish preview cloud image
+    needs: [plan_preview, image_preview]
+    if: github.ref == 'refs/heads/master' && needs.plan_preview.outputs.image == 'true' && needs.image_preview.result == 'success'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    # This existing environment has an external master-only branch policy.
+    environment: npm-canary
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+      - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
+        with:
+          name: preview-image
+          path: preview-image
+      - uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Verify image identity and publish without executing image code
+        env:
+          SOURCE_SHA: ${{ inputs.source_ref }}
+        run: node scripts/preview-artifacts.mjs publish-image preview-image/preview-image.tar "$SOURCE_SHA"
+
+  result_preview:
+    name: Verify preview artifacts
+    needs: [plan_preview, image_preview, publish_image_preview, package_preview, publish_preview]
+    if: >-
+      always() && needs.plan_preview.result == 'success' &&
+      (needs.publish_image_preview.result == 'success' || needs.plan_preview.outputs.image == 'false') &&
+      (needs.publish_preview.result == 'success' || needs.plan_preview.outputs.packages == 'false')
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+      - name: Confirm exact artifacts are visible
+        env:
+          SOURCE_SHA: ${{ inputs.source_ref }}
+          REQUEST_ID: ${{ inputs.request_id }}
+          PREVIEW_MIGRATOR: ${{ inputs.preview_migrator }}
+        run: node scripts/preview-artifacts.mjs result "$SOURCE_SHA" "$REQUEST_ID"
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
+        with:
+          name: stack-deploy-result
+          overwrite: true
+          path: stack-deploy-result/result.json
+          if-no-files-found: error
+          retention-days: 30
+
  verify_canary:
    if: github.event_name == 'push'
    uses: ./.github/workflows/release-verify.yml