Commit Graph
5 Commits
Author SHA1 Message Date
DottaandPaperclip 322fa6b65c fix(ui): restore task document authoring and add persistence journey (#18356)
## Thinking Path

> - Paperclip helps people manage AI agents and their work.
> - Task documents keep versioned work beside each conversation.
> - The default chat interface could read general documents but hid
their editor.
> - People also need unfinished edits to survive a tab switch or reload.
> - This pull request restores the shared editor in Artifacts.
> - A portable browser journey checks exact saved content and revision
history.
> - The benefit is a usable document flow with independent deployment
checks.

## Linked Issues or Issue Description

**What happened?**

A person using the default task interface could read a general task
document but could not create or edit one. The existing editor was
available in Classic mode. A pending create could also leave newer text
as a new form after its editor unmounted, causing a conflict against the
person's own successful creation.

**Expected behavior**

A person can create and edit general documents through Artifacts.
Titles, bodies and revision history persist after reopening and
reloading the task. Unfinished text survives tab switches. A successful
create makes retained newer text an edit of that confirmed revision.
Later writes from another actor still cause a conflict.

**Steps to reproduce**

1. Open an ordinary task in the default chat interface, then open
Artifacts.
2. Try to create a general document. The previous interface has no
authoring control.
3. With authoring restored, submit a new document, keep typing, and
switch to Properties before the request finishes.
4. Reopen Artifacts. The previous draft path restores a new form despite
the successful create.

Related public work: #17598 adds document copy/download and #18157 adds
image galleries. The selected actual merged parent is
`f1d780838218f975064f38333946805e0ec103a5` from #18427. It includes the
stable profile-entry ordering repair and the exact recovery-fixture
prerequisite. The resulting PR diff contains 32 document and reporting
files.

## What Changed

- Reuse `IssueDocumentsSection` for general documents in default-chat
Artifacts. Keep Classic authoring in its existing task body. Ordinary
tasks, conversations and Coordinator mode share the same editor options
and signed-in permission wrapper.
- Preserve Plans, file-review documents, attachments, uploads and work
products. Hide duplicate general-document cards. Use the same
general-document key rules for visible rows, validation and refused
writes.
- Forward image uploads, mentions, references, author maps and feedback
to both editor mounts. Keep empty mention/project values stable while
queries load.
- Back up unfinished forms within their signed-in account, task and
editor scope. Preserve text entered before its key. Record a successful
create on its matching retained newer buffer, including after unmount or
an early reopen. Preserve newer title/body edits, real remote conflicts
and matching cancellation cleanup.
- Add the model-free `task-document-persistence` journey. Setup creates
only an owned empty backlog task. The browser creates and edits the
document; normal APIs independently check company, task, actor, exact
content and both revision records.
- Reopen the owned task through the explicit Backlog view, then reload
it. Require exact UI readback and zero native runs. Add only the Doc
registration, definition-hash inputs, report mapping and four closed
checks to the shared harness.
- Count explicit zero-run browser receipts after private journey
metadata is removed for publication. Preserve missing billing evidence,
positive usage and native/protocol accounting.
- Update browser assertions for the restored inline editor while
retaining exact download bytes, revision collisions, paused-state and
no-run checks. Update the feature map, usage docs and shared editor
stories.

## Verification

- Latest focused checks use official Node 24.22.2: 568 journey, catalog,
report, billing and attempt-accounting tests across 25 suites pass. All
four admitted journeys remain registered. The seven relevant UI suites
pass 529 tests, including the task-page container after the parent
merge. Maps, token gates, the PR-scope diff check and the owned
credential-pattern scan pass. The normal QA import preflight also passes
with an empty environment and network access blocked.
- Billing regression: before the repair, the explicit empty public
receipt counted one run, and two positive-usage receipts were discarded
by browser metadata. Those three cases fail before and pass after the
repair. Two additional red regressions cover aggregate usage beside an
empty run list. That ambiguous shape now keeps an unavailable receipt
and a minimum count of one. Missing receipts retain unavailable billing,
and native/protocol one-or-more run totals retain their measurements.
- The previous fresh CI run failed its unchanged snapshot-only recovery
fixture: one global sweep did not call its callback. The original
failure remains recorded. An isolated PGlite probe with all canonical
migrations exercises the real reconciler: it processes two earlier
sources, then reaches the owned third source on the next sweep. It
preserves source evidence, files and run IDs. This proves bounded
pagination, not full native snapshot repair.
- Local native recovery tests could not start PostgreSQL because the
host semaphore limit was reached. That run executed zero cases.
Canonical Linux CI later passed all 143 fixture cases, including the
original snapshot-only case and the new competing-candidate control. The
exact fixture repair is now inherited from #18427 and is absent from
this PR diff.
- The portable oracle rejects 19 stale, missing, foreign or incorrect
outcomes. Result validation also rejects model/provider metadata and
native runs for this case.
- Five real Artifacts-to-Properties unmount regressions fail before the
repair and pass after it. They cover both editor modes, reopening before
and after completion, later typing, and a foreign revision that must
still conflict. The edit-storage failure case also checks that Cancel
clears the promoted fallback copy.
- UI coverage includes default chat, Classic, conversations,
Coordinator, Plan rendering, document-card deduplication, reserved keys,
pending-query stability, permissions and rich editor options.
- Earlier real browser checks passed five upload cases and two
artifact-arrival cases. They retain exact CSV bytes, original/revised
content, collision, paused-state and no-run assertions. Current-head
normal CI runs the updated browser suites again.
- The branch merges only the selected actual parent
`f1d780838218f975064f38333946805e0ec103a5`. The merge has no conflicts
or manual resolution. The SQL service and recovery fixture match that
parent exactly. The own document editor, persistence flow, saved-state
oracle and billing repair retain their previously qualified source
blobs. Follow-up, clarification, question/resumption, screenshot helpers
and the Steering map are byte-identical to the merged parent. Existing
project/search/question/follow-up registrations remain present.
- Required document source blobs remain exact: Doc UI
`22e80663b053363e5f75e1432dd9043dc0e70243`, its unmount tests
`16f3f4ed0dc9b11aaff129e06e6cf2bbf4522713`, readiness helper
`800a747284db19b8484650b6759e8bd1f356839d`, and budget service
`3a15d489f0aa55c22e8f7fe6a6bc6a0e14e5914e`.
- Local typing is limited by borrowed declarations from another
checkout. They omit current `skill_test` and sandbox creation-operation
fields. No dependency declarations or repository config were changed.
The authoritative clean CI Typecheck + Release Registry gate passes on
exact head `2ffce29827ae170420d740940c1c069c7225a74c`.
- Full monorepo typecheck, tests and build were not run locally.
Borrowed build declarations from another checkout limit local typing.
Authoritative [clean
CI](https://github.com/paperclipai/paperclip/actions/runs/41171717008)
passes on exact head `82d15946c5844d3ae41c6ade62fa224053f20a9a`: 55
successful checks, 3 skipped and 1 neutral, with zero pending or failed
checks. Typecheck, build, Runner, server and browser lanes pass.
Greptile gives that same head a fresh 5/5. All 15 review threads are
resolved, with no further thread page. The PR is mergeable and clean.
- Failure history remains recorded. The earlier recovery failure was
repaired with a bounded, owned-source fixture. A later CI worker shut
down after 2,375 Runner unit tests passed. Its one authorized narrow
retry passed those tests and all five native integration suites. The
separate AgentConfig browser failure was reproduced and fixed by the
merged #18427 parent. These earlier passes do not replace current-head
CI.
- Final-source live qualification passed on exact harness and served
application `82d15946c5844d3ae41c6ade62fa224053f20a9a`. QA run
`41177696858` completed all four closed checks and all five phases with
zero native runs. Cleanup passed; the durable manifest is complete with
no pending evidence. The coordinator verified the original grade and
manifest.
- The [public qualification
report](https://pages.paperclip.ing/evals-qa-smoke-41177696858-1/)
records zero agent runs, empty run IDs and empty run usage. Pricing
remains unavailable and billing remains incomplete; the report does not
claim measured zero cost.
- The earlier harness `36c8d9ddd76d0b7ddb28a9a6c674d9e538f6df88` pass
against application `2ffce29827ae170420d740940c1c069c7225a74c` remains
historical evidence. It is separate from the final-source measurement
above.
- The selected parent brings profile-entry ordering, proactive chat, a
database migration, Runner updates, and task-page chat/read behavior.
Its new task-page container is included in the final application
measurement. The coordinator retains the original evidence and owns the
primary merge and daily-plan admission.

## Risks

Artifacts now mounts the existing shared editor. Its autosave, conflict,
revision, annotation and server access contracts remain in place. Tests
cover both interface modes, shared scopes, later remote writes and
unavailable edit storage. Plans and generated file-review documents keep
their existing flows. The PR adds no database schema or API change. The
shared billing repair requires explicit captured empty run IDs and an
unambiguous usage object containing only an empty run list for public
browser-only receipts. Other fields beside an empty list keep
conservative unavailable billing. Missing measurements remain
unavailable. Contradicting positive usage remains counted. The inherited
recovery fixture keeps the production sweep limit and its source,
readback and no-agent-work assertions.

The live journey must fail when authoring controls or saved outcomes are
absent. It must not create the expected document through setup APIs or
switch to another interface to pass. Final deployment verification must
retain the actual app and harness revisions separately.

## Model Used

OpenAI Codex, based on GPT-6, with tool use and code execution. The
exact runtime model ID and context window were not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-31 01:49:24 -05:00
e6ec781a1a fix(e2e): verify review outcomes and preserve report measurements (#17556)
## Thinking Path

> - Paperclip's Product E2E journeys verify user outcomes in the real
app and GitHub.
> - The review journey must show the score and failed check for its
exact reviewed commit, then the visible task status before and after
reload.
> - GitHub includes a hidden “Re-run failed checks” menu item before the
visible check result.
> - The journey selected that hidden item and failed even after native
work and publication completed. A subsequent attempt exposed the same
first-text-match problem with Done in the task UI.
> - This change reads the selected check's status header and binds its
URL, score, and commit.
> - Browser fixtures retain this failure and reject plausible wrong
projections.

## Linked Issues or Issue Description

Refs: #17457, #17513, #17551.

**What happened?**

The bot completed its native run and task, then published its summary,
inline precision finding, and failed 2/5 check. The browser assertion
selected a hidden “Re-run failed checks” menu item before the visible
result, so the journey still failed. After that was corrected, the next
live attempt passed the check projection but selected hidden Done text
before the visible task status.

**Expected behavior**

Verify the actual selected GitHub check. A hidden menu item or a word in
the model's narrative cannot prove its conclusion.

**Steps to reproduce**

1. Render a hidden “Re-run failed checks” item before a visible failed
check.
2. Use the old first text match for “failed”.
3. Observe the false visibility failure.

**Paperclip version or commit**

The base includes the journey changes from #17457 and #17513.

## What Changed

- Validate the check URL against the configured repository and select
its exact check-run ID.
- Require the expected score and full reviewed commit inside that check.
- Read failure status only from that check's header.
- Give each projection step its own phase so failures identify the
correct boundary.
- Stage the verified persisted screenshot as the standard report
final-state artifact, rejecting a passing report if it is absent.
- Preserve measured billing summaries during regeneration when raw run
usage is absent; newer raw usage takes precedence.
- Extend the catalog-wide history fixture to include the new journey
evidence contract.
- Read task completion from the visible Status property picker before
and after reload, with separate phase labels.
- Add eighteen isolated browser fixtures to the existing browser-support
suite and document both boundaries.

## Verification

- Reproduced the old selector failure in the live GitHub page and a
credential-free browser fixture before fixing it.
- All 60 browser-support cases pass, including eighteen focused
projection cases. They reject the wrong check status, score, commit,
page, and repository reference, plus hidden or unrelated Done text.
- All 163 focused billing, history, accounting, report, and journey
tests pass, as does runner E2E typechecking.
- A regression reproduced discarded saved token totals. Retained totals
now survive per-cell and campaign aggregation while missing dollar
prices stay unknown; fresh raw usage still overrides stale summaries.
- Diff and credential-value checks pass.
- The earlier failed attempts remain failed. A fresh complete QA journey
passed all 18 checkpoints and cleanup with a successful native run, a
below-threshold review and inline precision finding, the correctly
failed check, visible Done status, and reload persistence. The later
reference-validation commit changes only browser fixtures; the live
driver bytes are unchanged.
- A report regression reproduced the missing standard final-state
artifact. The packager now aliases only the hash-verified persisted
screenshot for passing results; missing final-state proof is rejected.
Presentation regeneration preserves the original live grade and
dispatches no provider work.

## Risks

This depends on GitHub's check-run header structure. A markup change
fails the journey instead of accepting narrative text as a result. The
product runtime and behavioral oracle stay unchanged. Usage-price
coverage remains a separate evidence dimension. A saved billing summary
is reused only when raw usage is absent, preserving the recorded
coverage and unknown prices.

The separate advisory Paperclip Review remains queued after its prior
assigned attempt timed out. Its normal Retry control rejected the
obsolete failed chat request as unsafe to retry. Required repository
gates `ci / verify` and `ci / e2e` pass, all 55 completed CI checks are
green, and final-head Greptile is 5/5 with no unresolved findings. The
advisory queue is not represented as a completed assessment.

## Model Used

OpenAI Codex, based on GPT-6. The exact runtime model ID and
context-window size are not exposed in this session. Used for code,
browser fixtures, and live evidence analysis.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <paperclip@paperclip.ing>
Co-authored-by: Paperclip <bot@paperclip.ing>
2026-10-24 23:54:27 -07:00
DottaandPaperclip 3bd709df8b Operationalize Product E2E accounting, tracing, and scheduling (#17009)
## Thinking Path

> - Paperclip manages agents and the work they perform.
> - Product E2E tests exercise real agents, tools, setup and cleanup.
> - Repeated runs need retained costs and traces, including failed
attempts.
> - A passing retry must not hide earlier spending or feedback.
> - Recurring work also needs admission limits, private reports and
failure detection.
> - This change supplies those controls while keeping paid activation
behind existing gates.

## Linked Issues or Issue Description

Refs #16996 and #17158. This is the consolidated landing of the reviewed
eval operations stack: #17019, #17024, #17025, #17027, #17030, #17033,
#17035, #17038, #17043, #17044, #17048, #17050, #17051, #17092, #17096,
#17100, #17107, #17109, #17112, #17115, #17117, #17121, #17123, #17127,
#17130, #17133, #17134, #17137, #17140, #17143, #17154. The repository
permits squash merges only. A local rehearsal showed that squashing each
slice into master causes repeated conflicts in dependent slices. This PR
preserves their commits and lands their combined implementation in one
reviewed integration. Superseded slices will close after this merge.

## What Changed

- Retain every attempt's cost receipt, including setup, failures,
interruptions and retries. Separate reported amounts, estimates, missing
costs and reserved capacity in reports and dashboards.
- Add bounded request admission, exact model pricecards, gateway error
accounting, cache policy and reconciliation evidence. Refuse unsupported
routes and requests that cannot fit the allowance.
- Integrate Braintrust gateway traces and OTLP correlation, with secret
filtering and experiment imports. Record phase timing, including setup,
feedback capture and cleanup.
- Collect complaint and suggestion evidence after each attempt,
including recovery after failures. Retain explicit review dispositions
independently from the product grade.
- Add gated daily and weekly selections, measured forecasts,
setup/readiness checks, wall-time limits and retained monitor
notifications. Include declared non-model overhead in admission while
reporting the existing subscription separately.
- Keep recurring history private, bound uploads and retention, and use
isolated archive environments. Keep public publication an explicit
operation. Report and publication jobs install only the committed
lockfile, with scripts disabled and isolated stores; stale pins fail
closed.
- Integrate current master, preserve its Product E2E journey billing and
privacy checks, and test that shared setup receipts do not double-count
estimated agent usage. Preserve upstream fixture repairs.

## Verification

- Complete integration baseline
`0649a5ce7544ab20825c2183d2f421157db49f24` includes master
`a3e1d915a7110a4e115b46396a3f79e6dfb4d05e` and every original stack head
as ancestors.
- On that integration baseline, `pnpm test:e2e:runner:unit`: 4,012
Vitest tests passed, two skipped, across 193 files; all 137 Node
source-contract tests passed.
- Focused billing and Product E2E journey contract checks: 24 tests
passed. They cover estimates, unknown receipts, single-owner setup
costs, browser-only execution and private publication guards.
- Monitor and workflow wiring: all 96 Node tests passed. `node
scripts/check-token-gates.mjs`: all gates clean.
- `pnpm test:e2e:runner:typecheck` passes after the standard Runner
TypeScript build refreshed local declarations. The first attempt
rejected the new master `runner_prepared_launch` union member because
this checkout retained an older generated declaration; the source union
already contained that member.
- Current head `492b0708a1c531a023d9ac10f0f3dc6383cc1754` removes
dependency re-resolution from archive-authorized reporting jobs after
the security scan flagged it. All 15 workflow-security tests pass after
updating the old two-install assertion.
- Hosted [CI run
39965460180](https://github.com/paperclipai/paperclip/actions/runs/39965460180)
passes on the current repaired head, including typecheck, build, general
tests, Runner checks, browser shards and canary dry run. Greptile is 5/5
on this exact head with no unresolved threads. The security scan passes
after the fixed lockfile-resolution thread was resolved.
- Paperclip Review returned a terminal neutral check but an incomplete
0-file assessment on the repaired head: its platform blocked both diff
responses as prompt-injection content. This is not a successful code
review. No bypass or fabricated pass was attempted; current-head
Greptile, security and hosted CI are independent gates.
- No paid test, reservation release or recurring-run activation
accompanies this landing. Live route qualification and invoice
reconciliation remain separate evidence.

<details>
<summary>Retained earlier validation failures</summary>

Earlier broad local runs had two 15-second test timeouts (CLI poll
cadence and Retry validation), one embedded-database setup timeout, and
later PostgreSQL semaphore exhaustion. The exact timed-out tests passed
separately without larger timeouts; the broad-run causes were not
established. Those local failures remain distinct from subsequent hosted
CI results. The independent #17158 fix records its Muse fixture
collision, its UUID correction and hosted verification. Local Muse
execution was also blocked by macOS embedded-PostgreSQL startup. No
historical failure is treated as a successful paid qualification.

</details>

## Risks

- Admission is a local control, not a provider invoice cap. Missing
provider receipts or resource charges retain unknown amounts and holds;
estimates do not become billed dollars.
- The bounded path admits only documented local suites and provider
routes. Ordinary legacy or separate Runner protocol campaigns have
different controls. Merging this code does not qualify additional
routes.
- Recurrence still needs the existing enable gates and current
qualification/allowance evidence. Archive permissions and resources must
match the documented private-environment contract.
- Legacy reports may lack complete attempt or phase evidence. Reports
preserve that limitation and do not infer zero usage from missing data.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools and code
execution. The exact deployment variant and context-window size are not
exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (targeted checks;
full-suite host limitation disclosed above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-23 07:18:37 -07:00
DottaandPaperclip 83076d7e7c feat: return completed handoffs to Agent Chat (#14408)
Return completed Agent Chat handoffs through a durable outbox and scope each generated update to its supplied tasks. Add recovery, browser delivery, result access, and calibrated quality coverage.

Validated with two consecutive ten-case Claude/Codex campaigns, all CI checks, and a 5/5 review.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-09-29 10:25:52 -05:00
Dotta 5716fe907e test(runner): add full-stack acceptance and eval gates (#12700)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The runner subsystem executes agent work across local and managed
provider backends.
> - The lower pull requests restore the task runtime, provider backends,
and managed-provider control plane.
> - The restored system needs repeatable full-stack checks before it can
ship safely.
> - Paid live checks also need clear access, cost, and secret controls.
> - This pull request adds acceptance, live evaluation, chaos, and
release gates for the restored runner stack.
> - The benefit is measurable runner parity with safer release
decisions.

## Linked Issues or Issue Description

**Subsystem affected**

Cross-cutting. This change covers runner tests, release workflows,
server contracts, and evaluation tools.

**Problem or motivation**

The runner stack did not have one complete acceptance surface for native
Codex, ACPX, Claude Managed, and AWS AgentCore. Release checks could
miss provider drift, task-view regressions, cost-policy errors, and
destructive cleanup errors.

**Proposed solution**

Add a 57-cell full-stack catalog, a Daytona image, and opt-in paid
workflows. Add live evaluation, chaos, cost-limit, redaction, and
release contract checks. Add AWS AgentCore infrastructure and guarded
provisioning tools. Keep the native runner experimental flag off by
default.

**Alternatives considered**

We considered manual smoke tests only. They do not give repeatable
evidence and they do not protect release branches. We also considered
one large pull request. The stacked pull requests keep each review below
the Greptile file limit.

**Roadmap alignment**

This work supports the shipped Cloud / Sandbox agents milestone and the
shipped Agent evals & feedback milestone in `ROADMAP.md`.

Related stack:

- #12699 adds managed provider backends and lifecycle support.
- #12691 adds qualified OpenCode and ACPX provider backends.
- #12685 restores task runtime rendering and steering.

## What Changed

- Add the runner full-stack harness with 57 catalog cells and 60 unit
tests.
- Add a Daytona runner image with digest-pinned base images and
base-aware image-content checks.
- Add guarded live evaluation and chaos workflows with a fixed
40-execution matrix; live and full-stack paid schedules now run only on
Sundays or by manual dispatch.
- Add in-flight reported-usage cost stops, post-turn cost caps,
exact-threshold failure classification, secret redaction, retry
classification, and actor authorization.
- Reattach stream and hard-budget listeners before restart-recovery
continuations so restored paid sessions cannot bypass in-flight
interruption.
- Preserve OpenCode usage and cost across tool-loop messages and turns
while exposing an explicit current-run delta to durable accounting.
- Keep PNG/WebM evidence in access-controlled artifacts only, reject
SVG, and publish only pruned inert structured per-attempt evidence.
- Add AWS AgentCore infrastructure, provisioning checks, and smoke
tools; reject unsafe model identifiers, require exact stack ownership
markers, and make failed-stack replacement explicit.
- Add evaluation-session contracts and capability reports.
- Add release workflow checks for immutable action pins, frozen
dependency installs, exact weekly cron shape, paid-run guards,
provider-secret isolation, and chaos test paths.
- Reauthorize the original and triggering numeric actor IDs as the first
step of every provider-secret job, including partial reruns, before
checkout or provider access.
- Give each full-stack matrix cell only its matching provider
credential, expose Daytona only to Daytona cells, and disable shared
dependency caches anywhere paid credentials or OIDC write access are
present.
- Protect the legacy manual E2E workflow with the same default-branch,
allowlist, environment, and per-job authorization boundary.
- Rotate live-eval candidates by week and retain 120 days of compatible
history so the seven-week trend window remains viable.
- Restore the root runner-acceptance commands and reconcile reported
snapshots,
raw receipts, and terminal usage without double counting or losing late
usage.
- Mark ACPX token deltas exact only when every budget field is present,
keep
cumulative cost/request authority separate, reject non-USD cost
labeling,
  and include thought tokens in output-token budgets.
- Keep `enableNativeRunner` off by default. The acceptance harness
enables it only in its isolated test instance.

## Verification

Passed locally:

- `pnpm --filter @paperclipai/paperclip-runner typecheck`
- `pnpm test:runner-acceptance:typecheck`
- `pnpm test:runner-acceptance` (19 tests)
- focused OpenCode proxy, driver, runnerd transport, live-session, and
turn-stream tests (106 tests)
- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/clean-room-server.test.ts` (22 tests)
- `pnpm test:e2e:runner:typecheck`
- `pnpm test:e2e:runner:unit` (62 tests)
- `node --test scripts/__tests__/release-verify-workflow.test.mjs`
- `pnpm --filter @paperclipai/paperclip-runner
test:runner-workflow-evals` (22 tests)
- `pnpm -r typecheck`
- `pnpm build`
- `node --test
packages/paperclip-runner/scripts/aws-agentcore-provisioning.test.mjs`
(6 tests)
- `git diff --check`
- `cargo test --manifest-path
packages/paperclip-runner/runner/Cargo.toml -p paperclip-runner-core
--lib --locked` (161 tests)
- focused ACPX provider-event tests (10 tests)
- The rebased PR changes 92 files. `pnpm-lock.yaml` is unchanged.

I did not run paid live provider jobs or provision AWS resources. Those
checks need credentials and can create cost.

## Risks

The paid workflows can create provider cost. They require an allowlisted
original and triggering actor, the protected `runner-e2e-paid`
environment, explicit opt-in variables, and cost limits. The four
provider credentials exist only in that master-only environment, which
requires allowlisted reviewer approval and disables administrator
bypass; repository and organization Actions scopes contain no copies.

Provider usage arrives after a billable request, so the live guard
cannot prevent one request from crossing a threshold. It interrupts
immediately on the first reported threshold hit and permits no
continuation.

Visual evidence can contain secrets rendered as pixels. PNG/WebM remain
only in access-controlled workflow artifacts; SVG and per-attempt XML
are excluded, and S3/Pages receive a pruned structured dashboard.

The AWS scripts can create cloud resources. They use explicit commands,
least-privilege roles, KMS encryption, saved nonsecret metadata, and
explicit teardown.

This pull request does not enable the experimental native runner for
existing instances.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex with GPT-5. The model used extended reasoning, tool use,
code execution, and parallel subagents.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-09-02 08:55:08 -05:00