mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-07 07:23:08 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The CI workflow is responsible for producing and publishing the Docker images that power Paperclip deployments > - The cloud image publish was previously coupled to the stock publish job, so a failure or delay in one path could gate the other > - That coupling makes the release pipeline less resilient than it needs to be > - This pull request gives the cloud publish its own top-level job so both publishes can run in parallel without a `needs:` dependency > - The benefit is better failure isolation and less wasted time when one publish path is slow or broken ## Linked Issues or Issue Description No public GitHub issue was found for this change. Problem statement: - The cloud image publish was implemented as trailing work inside the stock publish job. - That setup meant the cloud publish could be delayed or skipped if the stock job failed early. - The desired behavior is for the cloud publish to run independently so a failure in one publish path does not gate the other. Proposed solution: - Split the cloud publish into its own top-level workflow job. - Keep the same cloud-specific build settings and cache behavior. - Preserve the existing top-level concurrency behavior. Alternatives considered: - Keeping both publishes in one job with conditionals or later steps. Rejected because it still couples success and runtime between the two publish paths. ## What Changed - Split the cloud image publish into a separate top-level Docker workflow job. - Removed the dependency coupling so the cloud job does not need the stock job. - Expanded the drift-guard test to assert the two-job structure and the absence of `needs:` on the cloud job. ## Verification - The workflow YAML was parsed successfully and confirmed to contain two jobs: `build-and-push` and `build-and-push-cloud`. - The cloud job was confirmed to have no `needs:` entry. - The drift-guard assertions were reproduced in a dependency-free harness and passed. - PR #10408 completed GitHub Actions with all required checks green, including the e2e shards. - Greptile review completed at 5/5 with no unresolved comments. - No documentation files changed because this is a workflow/test-only change. ## Risks - The workflow now duplicates the prep steps across two runners, so any shared setup change must be kept in sync between both jobs. - The new job increases workflow surface area slightly, which can make future maintenance more verbose. - Overall risk is low because the change is limited to CI orchestration and test coverage. ## Model Used OpenAI Codex (GPT-5, tool-using code assistant) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
304 lines
11 KiB
YAML
304 lines
11 KiB
YAML
name: Docker
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- "master"
|
|
tags:
|
|
- "v*"
|
|
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
|
|
# Serialise builds per ref without killing an in-flight one: a newer push
|
|
# supersedes only the pending slot, so the image build that is already
|
|
# running always finishes and publishes.
|
|
concurrency:
|
|
group: docker-${{ github.ref }}
|
|
cancel-in-progress: false
|
|
|
|
jobs:
|
|
build-and-push:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 60
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v7
|
|
with:
|
|
# Full history and tags so `git describe` below can compute the
|
|
# release version to stamp into the image.
|
|
fetch-depth: 0
|
|
|
|
# `.git` is dockerignored, so a running image cannot derive its own
|
|
# version and otherwise reports the source package.json placeholder in
|
|
# analytics and the debug panel. Compute it here from the pristine
|
|
# checkout (real CalVer drift from the nearest release tag) and pass it
|
|
# into both builds. Empty when no release tag is reachable — the server
|
|
# then keeps its existing fallbacks.
|
|
- name: Compute build version
|
|
id: build-version
|
|
run: |
|
|
set -euo pipefail
|
|
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
|
|
echo "version=${version}" >> "$GITHUB_OUTPUT"
|
|
echo "Stamping build version: ${version:-<none>}"
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
run_install: false
|
|
|
|
# No dependency cache here: this workflow publishes release images, and
|
|
# restoring a shared Actions cache into the build inputs would let a
|
|
# poisoned cache entry reach the published artifact.
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v7
|
|
with:
|
|
node-version: 20
|
|
|
|
- name: Refresh lockfile for Docker build context
|
|
run: |
|
|
set -euo pipefail
|
|
pnpm install --lockfile-only --ignore-scripts --no-frozen-lockfile
|
|
|
|
changed="$(git status --porcelain)"
|
|
if [ -z "$changed" ]; then
|
|
echo "Lockfile already matches package metadata."
|
|
exit 0
|
|
fi
|
|
|
|
if printf '%s\n' "$changed" | grep -Fvq ' pnpm-lock.yaml'; then
|
|
echo "Unexpected files changed during lockfile refresh:"
|
|
echo "$changed"
|
|
exit 1
|
|
fi
|
|
|
|
echo "Using refreshed pnpm-lock.yaml in the Docker build context."
|
|
|
|
- name: Free runner disk
|
|
run: |
|
|
set -euo pipefail
|
|
echo "Disk before cleanup:"
|
|
df -h
|
|
|
|
pnpm store prune || true
|
|
sudo apt-get clean || true
|
|
sudo rm -rf \
|
|
/usr/share/dotnet \
|
|
/usr/share/swift \
|
|
/usr/local/lib/android \
|
|
/usr/local/share/boost \
|
|
/usr/local/share/powershell \
|
|
/opt/ghc \
|
|
/opt/hostedtoolcache/CodeQL \
|
|
/opt/hostedtoolcache/PyPy \
|
|
/opt/hostedtoolcache/Ruby || true
|
|
docker system prune -af || true
|
|
|
|
echo "Disk after cleanup:"
|
|
df -h
|
|
|
|
- name: Login to GitHub Container Registry
|
|
uses: docker/login-action@v4
|
|
with:
|
|
registry: ghcr.io
|
|
username: ${{ github.repository_owner }}
|
|
password: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- name: Set up Docker Buildx
|
|
uses: docker/setup-buildx-action@v4
|
|
|
|
# Deployment tooling reads these labels from the registry to verify an
|
|
# image's schema expectations against a migrator before deploying it,
|
|
# without pulling the image. The server refuses to start when the
|
|
# database is missing bundled migrations, so orchestrators need a cheap
|
|
# way to check image/migrator compatibility up front.
|
|
- name: Compute schema migration labels
|
|
id: schema
|
|
run: |
|
|
set -euo pipefail
|
|
last=$(ls packages/db/src/migrations/*.sql | sed 's|.*/||' | LC_ALL=C sort | tail -1)
|
|
count=$(ls packages/db/src/migrations/*.sql | wc -l | tr -d ' ')
|
|
echo "last=${last}" >> "$GITHUB_OUTPUT"
|
|
echo "count=${count}" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Docker meta
|
|
id: meta
|
|
uses: docker/metadata-action@v6
|
|
with:
|
|
images: ghcr.io/${{ github.repository }}
|
|
tags: |
|
|
type=raw,value=latest,enable={{is_default_branch}}
|
|
type=semver,pattern={{version}}
|
|
type=semver,pattern={{major}}.{{minor}}
|
|
type=sha
|
|
labels: |
|
|
io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }}
|
|
io.github.paperclipai.schema.migration-count=${{ steps.schema.outputs.count }}
|
|
|
|
- name: Build and push
|
|
uses: docker/build-push-action@v7
|
|
with:
|
|
context: .
|
|
# Pin the self-hosted image to the production stage explicitly:
|
|
# the Dockerfile now declares a later `cloud` stage, and without a
|
|
# target the default would silently become that stage.
|
|
target: production
|
|
build-args: |
|
|
PAPERCLIP_BUILD_VERSION=${{ steps.build-version.outputs.version }}
|
|
platforms: linux/amd64,linux/arm64
|
|
push: true
|
|
cache-from: type=gha
|
|
cache-to: type=gha,mode=max
|
|
tags: ${{ steps.meta.outputs.tags }}
|
|
labels: ${{ steps.meta.outputs.labels }}
|
|
|
|
# The cloud variant carries built bundled plugins for managed deployments
|
|
# (see the `cloud` stage in the Dockerfile). It runs as its own job with no
|
|
# `needs:` on the stock publish above, so the two builds run in parallel and
|
|
# a failure or slow build in one never gates, delays, or skips the other.
|
|
# Both jobs share only the single top-level concurrency slot. Each job is a
|
|
# separate runner, so this one carries its own copy of the prep steps
|
|
# (checkout through schema labels) — the accepted cost of that isolation.
|
|
build-and-push-cloud:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 60
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v7
|
|
with:
|
|
# Full history and tags so `git describe` below can compute the
|
|
# release version to stamp into the image.
|
|
fetch-depth: 0
|
|
|
|
# `.git` is dockerignored, so a running image cannot derive its own
|
|
# version and otherwise reports the source package.json placeholder in
|
|
# analytics and the debug panel. Compute it here from the pristine
|
|
# checkout (real CalVer drift from the nearest release tag) and pass it
|
|
# into the build. Empty when no release tag is reachable — the server
|
|
# then keeps its existing fallbacks.
|
|
- name: Compute build version
|
|
id: build-version
|
|
run: |
|
|
set -euo pipefail
|
|
version="$(git describe --tags --match 'v*' --long --dirty 2>/dev/null || true)"
|
|
echo "version=${version}" >> "$GITHUB_OUTPUT"
|
|
echo "Stamping build version: ${version:-<none>}"
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
run_install: false
|
|
|
|
# No dependency cache here: this workflow publishes release images, and
|
|
# restoring a shared Actions cache into the build inputs would let a
|
|
# poisoned cache entry reach the published artifact.
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v7
|
|
with:
|
|
node-version: 20
|
|
|
|
- name: Refresh lockfile for Docker build context
|
|
run: |
|
|
set -euo pipefail
|
|
pnpm install --lockfile-only --ignore-scripts --no-frozen-lockfile
|
|
|
|
changed="$(git status --porcelain)"
|
|
if [ -z "$changed" ]; then
|
|
echo "Lockfile already matches package metadata."
|
|
exit 0
|
|
fi
|
|
|
|
if printf '%s\n' "$changed" | grep -Fvq ' pnpm-lock.yaml'; then
|
|
echo "Unexpected files changed during lockfile refresh:"
|
|
echo "$changed"
|
|
exit 1
|
|
fi
|
|
|
|
echo "Using refreshed pnpm-lock.yaml in the Docker build context."
|
|
|
|
- name: Free runner disk
|
|
run: |
|
|
set -euo pipefail
|
|
echo "Disk before cleanup:"
|
|
df -h
|
|
|
|
pnpm store prune || true
|
|
sudo apt-get clean || true
|
|
sudo rm -rf \
|
|
/usr/share/dotnet \
|
|
/usr/share/swift \
|
|
/usr/local/lib/android \
|
|
/usr/local/share/boost \
|
|
/usr/local/share/powershell \
|
|
/opt/ghc \
|
|
/opt/hostedtoolcache/CodeQL \
|
|
/opt/hostedtoolcache/PyPy \
|
|
/opt/hostedtoolcache/Ruby || true
|
|
docker system prune -af || true
|
|
|
|
echo "Disk after cleanup:"
|
|
df -h
|
|
|
|
- name: Login to GitHub Container Registry
|
|
uses: docker/login-action@v4
|
|
with:
|
|
registry: ghcr.io
|
|
username: ${{ github.repository_owner }}
|
|
password: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- name: Set up Docker Buildx
|
|
uses: docker/setup-buildx-action@v4
|
|
|
|
# Deployment tooling reads these labels from the registry to verify an
|
|
# image's schema expectations against a migrator before deploying it,
|
|
# without pulling the image. The server refuses to start when the
|
|
# database is missing bundled migrations, so orchestrators need a cheap
|
|
# way to check image/migrator compatibility up front.
|
|
- name: Compute schema migration labels
|
|
id: schema
|
|
run: |
|
|
set -euo pipefail
|
|
last=$(ls packages/db/src/migrations/*.sql | sed 's|.*/||' | LC_ALL=C sort | tail -1)
|
|
count=$(ls packages/db/src/migrations/*.sql | wc -l | tr -d ' ')
|
|
echo "last=${last}" >> "$GITHUB_OUTPUT"
|
|
echo "count=${count}" >> "$GITHUB_OUTPUT"
|
|
|
|
# Published under the same tag set with a `-cloud` suffix
|
|
# (sha-<short>-cloud, latest-cloud, <version>-cloud).
|
|
- name: Docker meta (cloud)
|
|
id: meta-cloud
|
|
uses: docker/metadata-action@v6
|
|
with:
|
|
images: ghcr.io/${{ github.repository }}
|
|
flavor: |
|
|
suffix=-cloud,onlatest=true
|
|
tags: |
|
|
type=raw,value=latest,enable={{is_default_branch}}
|
|
type=semver,pattern={{version}}
|
|
type=semver,pattern={{major}}.{{minor}}
|
|
type=sha
|
|
labels: |
|
|
io.github.paperclipai.schema.last-migration=${{ steps.schema.outputs.last }}
|
|
io.github.paperclipai.schema.migration-count=${{ steps.schema.outputs.count }}
|
|
|
|
- name: Build and push (cloud)
|
|
uses: docker/build-push-action@v7
|
|
with:
|
|
context: .
|
|
target: cloud
|
|
# Space-separated sandbox-provider directory names to build into
|
|
# the variant; add here when managed deployments need another.
|
|
build-args: |
|
|
CLOUD_BUNDLED_PLUGINS=daytona
|
|
PAPERCLIP_BUILD_VERSION=${{ steps.build-version.outputs.version }}
|
|
platforms: linux/amd64,linux/arm64
|
|
push: true
|
|
cache-from: type=gha
|
|
cache-to: type=gha,mode=max
|
|
tags: ${{ steps.meta-cloud.outputs.tags }}
|
|
labels: ${{ steps.meta-cloud.outputs.labels }}
|