Files
PaperClipAI/scripts/draft-stable-notes.sh
T
Devin FoleyandPaperclip 8148fc0b1e fix(release): fetch draft-notes PR summaries in batches (#20602)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The release workflow drafts the next stable's release notes when a
beta publishes (`draft_stable_notes`)
> - The draft nests each referenced PR's summary under its commit
subject, and it fetches each summary with its own `gh pr view` call
> - A beta range now references more than 1,500 PRs, so these serial
calls do not finish inside the job's 10-minute timeout
> - This pull request fetches the summaries in batches of 50 per GraphQL
query
> - The benefit is that the job finishes in under a minute and pushes
the notes branch again

## Linked Issues or Issue Description

**What happened?**

The `draft_stable_notes` job of [run
42763789753](https://github.com/paperclipai/paperclip/actions/runs/42763789753)
(beta `2026.1110.0-beta.0`) reached its 10-minute timeout. GitHub
cancelled it during "Draft stable notes from the published beta", while
a `gh` process was still running. The beta itself published and its
smoke tests passed. The `release-notes/v2026.1110.0-beta.0` branch was
not pushed.

**Expected behavior**

The job writes the notes draft and pushes the branch for every beta,
whatever the size of the commit range.

**Steps to reproduce**

1. Publish a beta whose range from the last stable has many
PR-referencing commits. This range has 1,586 commits, and 1,583
different PRs.
2. `draft-stable-notes.sh` calls `gh pr view <n>` once for each commit,
one after another.
3. The job reaches `timeout-minutes: 10` before the loop ends.

**Paperclip version or commit**

`09e24dd12c36251dd12af2d73dbc715ce990f922` (beta `2026.1110.0-beta.0`).
This was the first beta to run the job after #12124 added the PR
summaries; earlier betas skipped it.

## What Changed

- `scripts/draft-stable-notes.sh`:
- Collects the unique PR numbers first. Fetches their bodies 50 per `gh
api graphql` query (aliased `pullRequest(number:)` fields), into a
temporary cache that `enrich_pr` reads.
- Parses the raw response with `jq`, not with `gh --jq`. When any number
in a batch is not a PR (for example, an issue reference), gh exits
non-zero and does not apply its `--jq` filter. The other PRs in the
batch still have usable data.
- The output format does not change. A missing `gh` or `jq`, a failed
batch, a cache directory that `mktemp` cannot create, or
`DRAFT_NOTES_SKIP_PR_ENRICHMENT=1` still gives bare subjects.
- No code comments are added (per `CONTRIBUTING.md`); this description
holds the explanation.
- `scripts/draft-stable-notes.test.mjs`:
- The enrichment tests now use a fake `gh` that answers GraphQL like
GitHub.
- New tests: batching (120 PRs make 3 queries, and a repeated PR is
fetched once); a batch with a non-PR number; a failed batch; a failed
`mktemp`; prose-only and empty bodies; the opt-out.
- Range tests now run with enrichment disabled, so an authenticated `gh`
on the test machine cannot reach GitHub.
- `package.json`: adds the test file to `test:release-registry`. These
tests did not run in CI before.

## Verification

- `node --test scripts/draft-stable-notes.test.mjs`: 17 pass. They also
pass with macOS bash 3.2 first on `PATH`.
- `node scripts/ci-profile/node-test.mjs
scripts/draft-stable-notes.test.mjs scripts/release-lib.test.mjs`: 31
pass.
- Red check: the new tests against the old script fail 5 cases (all
batching and enrichment cases). The `mktemp` test fails against an
unguarded `mktemp` call.
- Live run on the real range: `./scripts/draft-stable-notes.sh
2026.1110.0-beta.0` finished in 47 seconds. It wrote 4,596 excerpt lines
for 1,583 PRs.
- Parity: for 40 randomly sampled PRs from that range, the new excerpts
are byte-identical to the old per-PR `gh pr view` output.
- `shellcheck`: no new findings. The two remaining notices were there
before this change.

## Risks

- Low. The change touches only the post-publish draft job, and every
failure still gives the bare-subject draft.
- `jq` is now needed for the PR summaries. GitHub's Ubuntu runners
include it. Without `jq`, the draft has no summaries, the same as
without `gh`.
- I did not change `release.yml`. The job already checks out `master`,
so after this merges, re-running the failed job of run 42763789753
should push the missing notes branch with the fixed script.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) via Claude Code, with tool use
(shell, file editing, test runs, read-only GitHub API calls).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-10 11:48:55 -08:00

222 lines
8.0 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
# shellcheck source=./release-lib.sh
. "$REPO_ROOT/scripts/release-lib.sh"
beta_version=""
out_file=""
repo_dir="$REPO_ROOT"
usage() {
cat <<'EOF'
Usage:
./scripts/draft-stable-notes.sh <beta-version> [--out PATH] [--repo-dir PATH]
Examples:
./scripts/draft-stable-notes.sh 2026.817.1-beta.0
./scripts/draft-stable-notes.sh 2026.817.1-beta.0 --out /tmp/draft.md
Notes:
- Generates a stable release-notes draft for the given published beta:
the commit range from the newest stable tag (v*) to the beta's source
commit, grouped by conventional-commit type.
- Written to releases/beta/v<beta-version>.md by default. The draft is
meant to be committed to master via PR and edited during the beta
soak; the stable promotion reads it from master.
- With no stable tag yet, the range falls back to the previous beta
tag, and failing that to the full history of the source commit.
EOF
}
while [ $# -gt 0 ]; do
case "$1" in
--out)
shift
[ $# -gt 0 ] || release_fail "--out requires a path."
out_file="$1"
;;
--repo-dir)
shift
[ $# -gt 0 ] || release_fail "--repo-dir requires a path."
repo_dir="$1"
;;
-h|--help)
usage
exit 0
;;
*)
if [ -n "$beta_version" ]; then
release_fail "only one beta version may be provided."
fi
beta_version="$1"
;;
esac
shift
done
if [ -z "$beta_version" ]; then
usage
exit 1
fi
if [[ ! "$beta_version" =~ ^[0-9]+\.[0-9]+\.[0-9]+-beta\.[0-9]+$ ]]; then
release_fail "beta version must look like 2026.318.1-beta.0, got: $beta_version"
fi
beta_tag="beta/v${beta_version}"
if ! git -C "$repo_dir" rev-parse --verify "refs/tags/${beta_tag}" >/dev/null 2>&1; then
release_fail "beta tag ${beta_tag} does not exist. Draft notes are generated from a published beta."
fi
source_sha="$(git -C "$repo_dir" rev-parse "${beta_tag}^{commit}")"
source_short="$(git -C "$repo_dir" rev-parse --short "$source_sha")"
if [ -z "$out_file" ]; then
out_file="${repo_dir}/releases/beta/v${beta_version}.md"
fi
# Range start: walk stable tags newest-first and take the first whose
# merge-base with the source is a proper ancestor of the source. A stable
# cut from a candidate branch is not itself an ancestor of master, but
# its merge-base with the source is the promoted source commit — exactly
# the point the shipped stable's content diverges from. A stable that
# already contains the source (promoting an older commit) is skipped so
# the range never collapses to empty. Before the first stable, fall back
# to the nearest beta tag strictly before the source; with no marker at
# all, cover the source commit's full history.
range_start=""
range_label=""
range_cmd_start=""
while IFS= read -r stable_tag; do
[ -n "$stable_tag" ] || continue
mb="$(git -C "$repo_dir" merge-base "$stable_tag" "$source_sha" 2>/dev/null || true)"
[ -n "$mb" ] || continue
if [ "$mb" != "$source_sha" ]; then
range_start="$mb"
if [ "$mb" = "$(git -C "$repo_dir" rev-parse "${stable_tag}^{commit}")" ]; then
range_label="$stable_tag"
range_cmd_start="$stable_tag"
else
# rev-parse --short picks an abbreviation that is unambiguous in
# this repository, unlike a fixed nine-character truncation.
mb_short="$(git -C "$repo_dir" rev-parse --short "$mb")"
range_label="${stable_tag} (merge-base ${mb_short})"
range_cmd_start="$mb_short"
fi
break
fi
done < <(git -C "$repo_dir" tag -l 'v[0-9]*' --sort=-v:refname)
if [ -z "$range_start" ]; then
range_start="$(git -C "$repo_dir" describe --tags --match 'beta/v*' --abbrev=0 "${source_sha}^" 2>/dev/null || true)"
range_label="$range_start"
range_cmd_start="$range_start"
fi
if [ -n "$range_start" ]; then
range="${range_start}..${source_sha}"
else
range="$source_sha"
range_label="the beginning of history"
range_cmd_start="the beginning of history"
release_info "No stable or prior beta tag found; drafting from full history."
fi
subjects="$(git -C "$repo_dir" log --no-merges --format='%s' "$range")"
# Best-effort thoroughness: nest each referenced PR's own summary under its
# subject line, so the skeleton is a genuinely thorough raw document at
# creation time instead of a bare commit list. Prefers the PR template's
# "What Changed" bullets, falls back to the first prose lines. Degrades
# silently when gh or the network is unavailable (tests, offline runs);
# set DRAFT_NOTES_SKIP_PR_ENRICHMENT=1 to disable explicitly.
PR_BATCH_SIZE=50
pr_cache=""
fetch_pr_batch() {
local query="query(\$owner: String!, \$name: String!) { repository(owner: \$owner, name: \$name) {" pr_num num encoded
for pr_num in "$@"; do
query+=" pr${pr_num}: pullRequest(number: ${pr_num}) { body }"
done
query+=" } }"
(cd "$repo_dir" && gh api graphql -F owner='{owner}' -F name='{repo}' -f query="$query" 2>/dev/null) \
> "$pr_cache/batch.json" || true
while IFS=$'\t' read -r num encoded; do
[[ "$num" =~ ^[0-9]+$ ]] || continue
printf '%s' "$encoded" | base64 -d > "$pr_cache/$num" 2>/dev/null || rm -f "$pr_cache/$num"
done < <(jq -r '.data.repository // {} | to_entries[]
| select((.value | type) == "object" and (.value.body | type) == "string")
| "\(.key | ltrimstr("pr"))\t\(.value.body | @base64)"' "$pr_cache/batch.json" 2>/dev/null || true)
}
prefetch_pr_bodies() {
local pr_nums=() pr_num
[ "${DRAFT_NOTES_SKIP_PR_ENRICHMENT:-0}" = "1" ] && return 0
command -v gh >/dev/null 2>&1 || return 0
command -v jq >/dev/null 2>&1 || return 0
while IFS= read -r pr_num; do
[ -n "$pr_num" ] && pr_nums+=("$pr_num")
done < <(printf '%s\n' "$subjects" | grep -oE '\(#[0-9]+\)$' | tr -dc '0-9\n' | sort -un || true)
[ "${#pr_nums[@]}" -gt 0 ] || return 0
pr_cache="$(mktemp -d)" || { pr_cache=""; return 0; }
trap 'rm -rf "$pr_cache"' EXIT
local start=0
while [ "$start" -lt "${#pr_nums[@]}" ]; do
fetch_pr_batch "${pr_nums[@]:$start:$PR_BATCH_SIZE}"
start=$((start + PR_BATCH_SIZE))
done
}
enrich_pr() {
local pr_num="$1" pr_body excerpt
[ -n "$pr_cache" ] && [ -s "$pr_cache/$pr_num" ] || return 0
pr_body="$(cat "$pr_cache/$pr_num")"
[ -n "$pr_body" ] || return 0
if printf '%s\n' "$pr_body" | grep -q '^## What Changed'; then
excerpt="$(printf '%s\n' "$pr_body" | sed -n '/^## What Changed/,/^## /p' | grep -E '^- ' | head -3 || true)"
else
excerpt="$(printf '%s\n' "$pr_body" | grep -vE '^[[:space:]]*$|^#|^>|^<!--' | head -2 || true)"
fi
[ -n "$excerpt" ] || return 0
printf '%s\n' "$excerpt" | sed 's/^/ > /'
}
section() {
local title="$1" pattern="$2" invert="${3:-false}" body
if [ "$invert" = true ]; then
body="$(printf '%s\n' "$subjects" | grep -Ev "$pattern" || true)"
else
body="$(printf '%s\n' "$subjects" | grep -E "$pattern" || true)"
fi
[ -n "$body" ] || return 0
printf '## %s\n\n' "$title"
while IFS= read -r subject_line; do
[ -n "$subject_line" ] || continue
printf -- '- %s\n' "$subject_line"
pr_ref="$(printf '%s' "$subject_line" | grep -oE '\(#[0-9]+\)$' | tr -dc '0-9' || true)"
if [ -n "$pr_ref" ]; then
enrich_pr "$pr_ref"
fi
done <<< "$body"
printf '\n'
}
prefetch_pr_bodies
conventional='^(feat|fix)(\([^)]*\))?!?: '
mkdir -p "$(dirname "$out_file")"
{
printf '# Paperclip stable draft — from beta %s\n\n' "$beta_version"
printf '> Auto-generated at beta publish from `git log %s..%s` (baseline: %s).\n' "${range_cmd_start}" "${source_short}" "${range_label}"
printf '> Edit freely during the soak: rewrite for release-notes voice,\n'
printf '> fold noise, and call out anything a self-hoster must act on.\n'
printf '> The stable promotion reads this file from master and publishes\n'
printf '> it as the GitHub Release body under the stable version.\n\n'
section 'Features' '^feat(\([^)]*\))?!?: '
section 'Fixes' '^fix(\([^)]*\))?!?: '
section 'Other changes' "$conventional" true
} > "$out_file"
release_info "Draft stable notes written to $out_file"
release_info " Source beta: $beta_version ($source_sha)"
release_info " Range start: $range_label"