fix(runner): default qualification builds to hosted cloud runners

Keep the EC2 option explicit, preserve maintainer authorization and immutable
source selection, and prepare only disposable hosted build disk. Record the
Claude/OpenAI API-only qualification milestone and deferred connection work.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed 2026-10-09 09:06:25 -05:00
1 parent fe43a4b934
commit 8b5d075f0c
5 files changed
+64 -11

No files matched your search

+24 -7
View File
@@ -7,8 +7,15 @@ on:
description: "Branch in this repository to build; resolved to one immutable commit before checkout"
type: string
required: false
execution_host:
description: "Cloud execution host; EC2 requires this workflow ref in the runner-group allowlist"
type: choice
options:
- github-hosted
- ec2
default: github-hosted
publish_eval_image:
description: "Publish an immutable Daytona qualification image on the EC2 fleet (no provider credentials)"
description: "Publish an immutable Daytona qualification image in the cloud (no provider credentials)"
type: boolean
default: false
candidate_provider:
@@ -19,15 +26,15 @@ on:
- hermes
default: none
verify_source:
description: "Run broad source checks on EC2"
description: "Run broad source checks in the cloud"
type: boolean
default: false
verify_public_install:
description: "Build and verify clean public npm installation on EC2 (no provider credentials)"
description: "Build and verify clean public npm installation in the cloud (no provider credentials)"
type: boolean
default: false
build_eval_viewer:
description: "Build the canonical eval report viewer on EC2"
description: "Build the canonical eval report viewer in the cloud"
type: boolean
default: false
pull_request:
@@ -105,9 +112,9 @@ jobs:
echo "target_sha=$target_sha" >> "$GITHUB_OUTPUT"
manual_image:
name: Build and verify on EC2
name: Build and verify in cloud
needs: authorize_manual
runs-on: runs-on/fleet=paperclip-public-pr-x64/env=public-ci
runs-on: ${{ inputs.execution_host == 'ec2' && 'runs-on/fleet=paperclip-public-pr-x64/env=public-ci' || 'ubuntu-24.04' }}
timeout-minutes: ${{ inputs.publish_eval_image && !inputs.verify_source && !inputs.verify_public_install && !inputs.build_eval_viewer && 45 || 90 }}
permissions:
contents: read
@@ -142,6 +149,16 @@ jobs:
with:
node-version: 24
package-manager-cache: false
- name: Prepare disposable GitHub-hosted build disk
if: inputs.execution_host != 'ec2'
run: |
set -euo pipefail
test "$RUNNER_ENVIRONMENT" = github-hosted
test "$RUNNER_OS" = Linux
# These preinstalled SDKs are unrelated to this build. This VM is
# owned by this job and discarded afterward; no developer files exist.
sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc
df -h "$GITHUB_WORKSPACE"
- name: Resolve source dependencies without lifecycle scripts
run: |
pnpm install --resolution-only --ignore-scripts --no-frozen-lockfile
@@ -223,7 +240,7 @@ jobs:
' "$CANDIDATE_PROVIDER" "$SOURCE_SHA" > remote-verification/provider-pack-identity.json
echo "$immutable" > remote-verification/image.txt
echo "Image: $immutable" >> "$GITHUB_STEP_SUMMARY"
- name: Verify repository on EC2
- name: Verify repository in cloud
if: inputs.verify_source
# Leave time for artifact retention before the fleet's one-hour lifetime.
timeout-minutes: 38
+4 -2
View File
@@ -110,8 +110,10 @@ single successful browser task do not establish that qualification.
## Remote verification
Do not run Docker on a developer laptop when using remote verification. The
`Docker Runner check` workflow offers maintainer-authorized manual EC2 image
builds and broad source checks without provider credentials. It records the
`Docker Runner check` workflow offers maintainer-authorized cloud image
builds and broad source checks without provider credentials. Standard
GitHub-hosted Linux is the default; select `execution_host=ec2` only when the
workflow ref is authorized by the EC2 runner group. It records the
source revision, resolved lock digest and immutable image reference. Paid
Product E2E remains behind the protected default-branch workflow and environment.
@@ -2080,3 +2080,26 @@ and Linux closure
Only the small evidence files are extracted locally. The native controls and
question-budget changes above follow that run and need their own fresh CI.
This remains deterministic transport evidence, not paid or Daytona acceptance.
### 2026-10-09 API-only delivery milestone and hosted cloud builds
The user narrowed the immediate delivery milestone to Hermes with managed
Anthropic and OpenAI API-key connections. Subscription login, Grok and Bedrock
qualification are deferred. This changes the current qualification scope;
earlier outcomes and the deferred connection requirements remain recorded.
The remaining milestone covers real API-backed product behavior on Mac arm64
and Linux/Daytona: streaming, tools and edits, image input, native questions,
steering/queue/stop, strict restart recovery, persistent managed memory and
learned skills, permissions, routine firing, token/cost attribution, and clean
consumer installation. Successful scripted/native transport fixtures do not
replace those live proofs. Existing unknown-cost reservations remain held.
Read-only inspection of the AWS runner-group policy established that its Docker
workflow allowlist names the Grok branch, with no Hermes branch entry. The
Hermes image job was therefore not eligible for that group. The manual image
workflow now defaults to standard GitHub-hosted Linux, preserving the explicit
EC2 option, maintainer identity checks and immutable target checkout. Model
credentials are excluded from both build paths. Cloud disk preparation is
restricted to the disposable GitHub-hosted Linux job and unrelated preinstalled
SDK directories. No local Docker or Rust build is required.
+5
View File
@@ -1439,6 +1439,11 @@ credential boundaries and repetition requirements.
### Grok branch qualification on EC2
Credential-free image and clean-install preparation can use the manual
`Docker Runner check` workflow's default `execution_host=github-hosted`.
Choose `execution_host=ec2` only for an allowlisted workflow ref. Both paths
build remotely and retain the immutable target revision and image evidence.
The trusted default-branch workflow can run the explicit `grok-qualification`
suite from a selected target branch. Store `XAI_API_KEY` only in the protected
`runner-e2e-paid` environment. The paid step delivers it only to a profile whose
+8 -2
View File
@@ -124,12 +124,18 @@ describe("public repository paid workflow security", () => {
expect(workflow).not.toMatch(/path: runner-e2e-(?:build|provider-pack)\n/u);
expect(workflow).not.toMatch(/--file runner-e2e-(?:build|provider-pack)\//u);
});
it("keeps the manual EC2 image build credential-free and pins the authorized target", async () => {
it("keeps manual cloud builds credential-free, defaults to hosted compute, and pins the authorized target", async () => {
const workflow = await readFile(path.join(repositoryRoot, ".github/workflows/docker-runner-check.yml"), "utf8");
const manual = workflow.slice(workflow.indexOf(" authorize_manual:"));
expect(manual.match(/AWS_CI_TRUSTED_USER_IDS/gu)).toHaveLength(2);
expect(manual.match(/test "\$REPOSITORY_ID" = 1170821064/gu)).toHaveLength(2);
expect(manual).toContain('runs-on: runs-on/fleet=paperclip-public-pr-x64/env=public-ci');
expect(workflow).toMatch(/execution_host:\s+description: [^\n]+\s+type: choice\s+options:\s+- github-hosted\s+- ec2\s+default: github-hosted/u);
expect(manual).toContain("runs-on: ${{ inputs.execution_host == 'ec2' && 'runs-on/fleet=paperclip-public-pr-x64/env=public-ci' || 'ubuntu-24.04' }}");
const diskPreparation = manual.slice(manual.indexOf(" - name: Prepare disposable GitHub-hosted build disk"), manual.indexOf(" - name: Resolve source dependencies"));
expect(diskPreparation).toContain("if: inputs.execution_host != 'ec2'");
expect(diskPreparation).toContain('test "$RUNNER_ENVIRONMENT" = github-hosted');
expect(diskPreparation).toContain('test "$RUNNER_OS" = Linux');
expect(diskPreparation).not.toMatch(/\$HOME|\$RUNNER_TEMP|\.secrets/);
expect(manual).toContain('repos/$REPOSITORY/git/ref/heads/$TARGET_BRANCH');
expect(manual).toContain('ref: ${{ needs.authorize_manual.outputs.target_sha }}');
expect(manual).toContain('SOURCE_SHA: ${{ needs.authorize_manual.outputs.target_sha }}');