Keep manual receipt cursor progress when chat is disabled. Follow saved cleanup deadlines after actual child-process and PostgreSQL work settles, and preserve the exact failure-backoff boundary.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Keep virtual time fixed while actual filesystem and database work settles, then advance the coordinator deadline. Preserve the three-second page-gap limit and snapshot grace assertions; starvation mutation remains detected.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Observe the target adapter-start event, then await its existing run-specific completion barrier. Preserve succeeded, execution-count and review-promotion assertions, including deliberately slow completion beyond the old terminal poll.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
The deadline worker retains its incomplete-subtree error and classifier tests. Remove the obsolete reporter and its two isolated tests after its aggregate callers were removed.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Observe the specific succeeded retry before closing its task and draining all execution. Once execution has settled, advance bounded fake cleanup time while joining worker passes; retain the existing timeout, exact retry identity, output and zero-SQL assertions.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Separate retry-schedule notifications from ordinary queued-run eligibility. Preserve exact deadlines and prove native producer commit/rollback behavior without global retry scans on ordinary agent or issue edits.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Lock and re-read project pause eligibility before publishing budget intent. Cover actual project Retry parking/release and concurrent pause admission without no-op hints.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Distinguish a budget or agent refusal saved on the existing receipt from genuine recovery failure. Preserve HTTP409 messages and details, grant each new wait reason its existing finite grace, and retain failure backoff for unsaved or unexpected errors. Prove actual Retry admission,25-minute SQL quiet, independent warm producer-to-process release, missing/cancelled receipts and real PostgreSQL write failure.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Preserve company reconstruction for graph, run and eligibility changes. A real PostgreSQL producer regression verifies one assignment check among1000 open tasks, with broader eligibility reconstruction still intact.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Reconcile failed ephemeral Cloud leases through a signed cleanup-only absence observation when local historical inspection is unavailable. Preserve failure outcomes, exact resource bindings, retention and row-version fences; grant no run-continuation authority.
All 54 current-head checks/statuses passed. Greptile is 5/5, Medium, with no findings or unresolved threads. The 38 focused tests pass; both new reconciliation cases fail without the fallback. Isolated baseline and branch idle suites each pass all 225 cases. The original broader local teardown timeout is retained in the PR, and the full local run remains in progress.
Cloud protocol support is already serving in staging. Exact candidate composition and hosted provider sleep/wake qualification follow this merge. Production has not been changed.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Budget views show saved policies, incidents and spend for each
company.
> - A policy can remain after its agent or project has been deleted.
> - The overview currently stops when it cannot resolve that old
resource.
> - A current agent's budget then becomes unreadable during normal
setup.
> - This pull request preserves the historical row in display summaries.
> - Admission and policy changes still require a real company-owned
resource.
## Linked Issues or Issue Description
**What happened?**
The budget overview throws `Agent not found` when a saved policy
references a deleted agent. One old resource prevents the company from
reading its current budgets. The same problem affects historical project
policies and incident summaries.
**Expected behavior**
The overview should retain historical policy and incident IDs and spend.
It should display `Deleted agent` or `Deleted project` for a missing
resource. Read access must not grant permission to execute work or
change a budget for that resource.
**Steps to reproduce**
1. Create an agent and a budget policy for it.
2. Permanently delete the agent after its other dependencies have been
cleared.
3. Read the company's budget overview. The existing service throws
instead of returning its remaining current and historical rows.
**Paperclip version or commit**
The isolated repair starts from merged commit
`d9354bbfc35ef74333e90e596ce21dc8e59149b9` and integrates the exact
merged baseline test fix `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57`. It
also integrates the actual merged Antigravity auth fix
`18b1637b6371a6a6adade857e2a73654ee626fef` (#18254). The diff against
the selected merged parent remains two budget files.
**Related work**
Earlier reports and implementations: #11058 and #4333. This extracts the
product prerequisite from #18351 so it can qualify independently of that
journey's other dependencies. The change contains only the budget
service repair and four database regression cases.
## What Changed
- Resolve deleted agent and project scopes only when building display
summaries. Preserve their original IDs, policies, incidents and
accounting records.
- Reject a saved policy or incident that resolves to another company's
resource. Propagate other read failures.
- Keep missing-company resolution, work admission and policy changes
strict. Validate company, policy and incident scope bindings before
either incident action. A budget raise requires a live company-owned
resource. Historical dismissal may proceed only for a genuinely deleted
owned agent/project, and must reject live foreign or mismatched scopes.
- Add four canonical PostgreSQL regression cases: deleted agent history,
deleted project history, missing-scope admission and budget raises with
safe historical dismissal, and foreign/mismatched references. The
history cases also verify current ledger spend and unchanged stored
rows.
## Verification
- Real-service RED/GREEN against official PGlite with every canonical
migration: the original overview throws for deleted history; the repair
returns the historical IDs, incidents and spend while retaining current
budgets.
- Mutation regression RED/GREEN: missing-resource budget raises must
reject before writes; owned deleted-scope dismissal must reject its
linked pending approval, retain policy/agent/ledger data and incident
history, and remove the incident from the active list. Live foreign and
mismatched scopes reject both actions without mutations.
- Current integrated head `b3434f5ad05fd3067379352700c36bfe826179c4`:
all four canonical PostgreSQL regressions pass in [server
2/12](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124361194077),
and all five existing budget-service tests pass in [server
7/12](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124361194080).
Both use the CI merge against selected parent `18b1637`.
- Clean-build declaration typing passes in the current CI typecheck
gate. The earlier local source-alias no-emit check also passed,
including the canonical actor argument. The local worktree borrows
compiled dependencies from a sibling checkout; a later real-service
import was blocked by its stale `sandboxCreationOperations` export, so
clean-build CI is authoritative for the final integration. No production
workaround was made.
- `git diff --check` passes. Broad repository typecheck, tests and build
run in current-head CI rather than repeated locally. Native local
PostgreSQL could not start because this host exhausted its semaphore
pool; no global IPC settings or unrelated processes were changed.
- One unchanged Runner lane retry was authorized after a probable
intermittent close-settlement fixture failure. The [original
failure](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124360234111)
occurred in `omitted-context` cleanup. A [fresh passing Linux
lane](https://github.com/paperclipai/paperclip/actions/runs/41142552195/job/124360110386)
passed the same case and all 1,998 tests with byte-identical entire
Runner and adapter-utils trees, lockfile, root package, scripts and
workflows, against the same merged base. The fixture has a five-second
close allowance with a 2.5-second suspension reserve; a host stall is
not proven. No budget path or changed close input was found. The [single
targeted
retry](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124365812609)
passed the same case in 1,034 ms and all 1,998 lane tests; dependent
verification also passed. All 57 current PR checks are terminal and
green.
- The product helper requires all current CI and clean review before
merge. Actual QA budget-overview proof will run on the combined final
app before hiring qualification. No deployed success is claimed. The
older `b4863dd` artifact was never deployed; selected parent `18b1637`
adds sandbox/lifecycle changes and migration 0418, so whole-app identity
is not claimed. The budget module and relevant budget contracts remain
unchanged, and final hiring requires the combined final app with
explicit source identities.
## Risks
Budget summaries now return historical rows whose resources have been
deleted. These rows use explicit display names and preserve their
accounting data. They do not create a resource or relax work admission,
authorization or budget grants. Historical incident dismissal changes
only that incident and its linked approval; it does not resume work or
raise a budget. Cross-company references still fail closed. There is no
schema migration and no new background work.
## Model Used
OpenAI Codex, GPT-6 family. The session does not expose a more specific
backend model ID or fixed context window. Used reasoning, repository
tools, TypeScript and database test execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (the
PR description documents the display-only history and strict-mutation
contract)
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
* codex/sleep-run-task-recovery:
fix: normalize run dispatch hints to company identity
# Conflicts:
# server/src/services/run-dispatch-notifications.ts
* codex/sleep-run-task-recovery:
test: prove timer deadlines survive settled drain holds
fix: back off expired deadlines during domain drain
perf(heartbeat): let the dispatch worker own startup retry promotion
fix(server): scope run retention hints to affected companies
fix(heartbeat): scope and batch dispatch wakeups by owner
fix(evals): support official TestOps job metadata (#18385)
fix: release settled deadline work holds during drain
fix(agents): revalidate error-clear authority before write intent
test(skills): verify Cloud CLI reads through shared object storage
# Conflicts:
# server/src/services/issues.ts
# server/src/services/run-dispatch-notifications.ts
Expect committed producers to release idle admission while retaining a resume deadline, then verify actual dispatch and final quiet after resume.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip manages companies and the agents that do their work.
> - Remote execution creates provider resources that need a durable
owner.
> - The server could create a sandbox before it saved its lease.
> - A failed lease write could leave cleanup data only on the old web
server disk.
> - This change saves a company-scoped operation before creation and
transfers ownership atomically.
> - Recovery finds the original Daytona resource or records an explicit
manual reconciliation outcome.
## Linked Issues or Issue Description
**What happened?**
A sandbox could exist without a lease after a database failure or
company rebind. Failed cleanup recording could then create a local
fallback file. Replacing that web server could lose the cleanup
reference.
**Expected behavior**
A failed admission write makes no provider call. An interrupted creation
retains a committed operation. Recovery must not create a second sandbox
or destroy an attached lease.
**Steps to reproduce**
1. Run the disposable PostgreSQL cases in
`server/src/__tests__/sandbox-creation-operations.test.ts`.
2. Reject the lease insert after the controlled provider creates its
resource.
3. Start a new receiver process with a different empty web disk. It must
find the same resource and create one cleanup owner.
Related ownership report: Refs #14672. This change addresses new
allocations before lease persistence. It does not replace existing lease
recovery.
## What Changed
- Add an idempotent migration and a small creation-operation table.
Retain original company, environment and run identities across deletion.
- Commit creation intent before run, ad-hoc lease, and plugin task
replacement calls. Fence duplicate requests and ambiguous lease commits.
- Retain a known returned task resource ID in the existing operation
before attachment. Preserve manual reconciliation and never delete an
ambiguously attached resource.
- Keep company and run authority checks and atomic lease transfer with
the lease owner.
- Reuse Daytona names, labels, read-only inspection and claimed cleanup.
Keep provider payload handling in the provider.
- Retain nonsecret credential identity hashes. Refuse recovery lookup
under a changed credential value.
- Record manual reconciliation for missing/deleted or explicitly
inactive saved credentials, versions, and provider vaults. Keep
temporary resolution failures retryable. Never reactivate a disabled
credential.
- Use transaction signals and concrete deadlines. Failed records back
off independently. Manual reconciliation stays durable and quiet.
- Remove new orphan-file fallback writes from the covered paths. Keep
legacy file replay and its tests for cutover.
- Replace obsolete fallback-producer assertions with normal-entrypoint
durability tests. Keep cleanup, diagnostics and warm handoff coverage.
## Verification
- [Independent current-head
verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6534744320)
passed 190/190 combined tests, 40/40 incoming TestOps tests, full
workspace typecheck, and build. PR head
`958af986130f25e136065ad30d1fdd843c46255c` has tree
`e6c3e5b02182d5941a70ccf5bf55638df3c2e28d`, identical to tested combined
commit `b9d7a9c2e9b30f0153685fb24a5f70b56132201f`. Real PostgreSQL,
compiled Rust Runner and controlled object providers were used;
external-S3 integration remains outside this local proof.
- Current head `958af986130f25e136065ad30d1fdd843c46255c` normally
merges master `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57`. The sole
conflict was the maintenance test: a parameterized case now preserves
both legacy-local `local_storage` and valid shared-S3
`provider_mismatch` checks, including the parent local receipt, exact
version identity, missing-snapshot/no-seed refusal, and unchanged
persisted row. The three master TestOps files remain intact. No
production, schema, migration, or package changes were made after the
fully qualified prior head `0b4d05505e`. Independent review approved the
resolution; the full managed-skill-content file and Cloud guard controls
pass 58/58 (105.11 seconds). [Required PR CI passed on this exact merge
head, first
attempt](https://github.com/paperclipai/paperclip/actions/runs/41136987412):
54 successful checks and three intended skips. The additional Paperclip
review is neutral at 4/5; two external Superagent checks (Contributor
trust and Security Scan) remain in progress with zero annotations and no
assessment, so they are not passing scan evidence. Greptile reviewed
exact head `958af986130f25e136065ad30d1fdd843c46255c` at 5/5; the final
thread audit has zero unresolved threads. All completed evidence below
retains its recorded revision scope. The additional Paperclip Review on
`958af98613` is neutral at 4/5 with a Ship verdict and no must-fix or
should-fix item. Its optional observation is that a generic create
rejection remains pending/manual even when an operator believes
allocation did not occur. This change preserves uncertainty because the
generic create-error contract does not prove provider absence; no new
error classification or provider-specific compensation was added.
- [Independent final-source
verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6534328247)
passed all 189 real-PostgreSQL/Runner and controlled-fixture cases, full
workspace typecheck, and build. The integration commit
`c8411a8528386f4824b340ec5fdaaed1efb9ea69` has the same source tree as
final PR head `0b4d05505e` (tree
`bda65efc14b4c904d0f589154db77bb160641c8c`). Controlled object-provider
coverage passed; the separately guarded external-S3 suite was not run
because the bounded local MinIO pull was unavailable. This includes the
two final compatibility fixes.
- The first final-head CI attempt exposed one E2E timing failure in
`disconnect-write-recovery.spec.ts:181`: the title stayed unchanged
through the 15-second session-recovery poll. The same original case
passed in 13.8 seconds on immediately prior head `7ec06f77b7`, and all
UI/session/E2E source is identical across those heads. The failure log
and CI artifacts are retained; no cause is claimed. One failed-job-only
retry is authorized on this evidence, without changing a timeout,
assertion, or production behavior. [The single failed-job retry
completed
successfully](https://github.com/paperclipai/paperclip/actions/runs/41133619710/attempts/2):
all 35 E2E shard cases passed (6.6 minutes), including the original
session-recovery case in 14.0 seconds. The initial failure remains
disclosed and its cause remains unproven.
- Final head `0b4d05505ee0aa72f2981cdff74761406960ec73` adds two
independently reviewed integration fixes after the complete 7ec CI
matrix exposed two deterministic failures. Server 9/12 failed legacy
local-storage maintenance because the CLI reached the new Cloud server
startup guard; server 5/12 found the Daytona package version did not
match its bumped manifest. The failed run is retained at
https://github.com/paperclipai/paperclip/actions/runs/41131204203. It is
not passing evidence and was not blindly retried.
- Maintenance now constructs the existing explicit storage dependency
after loading the required persisted Cloud snapshot, and passes it to
both maintenance owners. It can verify legacy local storage before
provider cutover while normal Cloud server startup remains guarded. The
original case reproduced red locally, then the complete
managed-skill-content and managed-cloud-prerequisites files passed 57/57
(32.05 seconds), including missing-snapshot refusal, unchanged stored
configuration, and existing server guard controls. Daytona package
version now matches manifest 0.1.10; the complete bundled package
identity file passed 14/14 (3.98 seconds), and explicit
manifest/lifecycle controls passed 2/2 (328 intentionally filtered; 3.42
seconds). An initial workspace-filter command matched no project and is
not counted as pass evidence. The final diff changes no sandbox runtime
or schema behavior. [Full CI is green on this exact final
head](https://github.com/paperclipai/paperclip/actions/runs/41133619710/attempts/2):
56 successful checks, three intended skips, one additional
incomplete/neutral Paperclip Review, and no pending or failed checks.
The neutral review is not a passing assessment. [Greptile reviewed the
exact final head at
5/5](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6529396969),
with no new actionable issues and all review threads resolved.
- Final combined head `7ec06f77b7c14a745c569d18401f16df3d26b626`
includes master managed-skills migration 0417 and shared-storage
prerequisites from `198218bf55` through normal merges. Sandbox creation
moves to generated `0418_curved_lady_mastermind.sql`; its idempotent SQL
is byte-for-byte unchanged. The regenerated 0418 snapshot contains both
sandbox creation operations and the master skill mutation receipts, and
points to the master 0417 snapshot. Master SQL and journal history
remain intact. Exactly five snapshots remain, 0414 through 0418. The
only sandbox-specific changes since the approved credential policy are
migration metadata/number and the matching test/document references; the
sandbox runtime files are unchanged.
- [Independent final combined
verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6533960026)
passed on integration revision `e6f05b4cf0cd`, containing final sandbox
head `7ec06f77b7` and actual master `198218bf55`: migration/declaration
checks, all 158 real-PostgreSQL/Runner and controlled-fixture cases (251
seconds), full workspace typecheck (81 seconds), and build (35 seconds).
- Final combined migration proof: fresh real PostgreSQL has both tables.
A separate original-0417 sandbox table with a retained pending row
survives two applications of the renamed 0418 SQL unchanged, with the
expected primary/index and foreign-key structure. Migration
numbering/safety checks pass. The normal runtime migration reapply case
passes 1/1 (20 intentionally filtered cases; 46.69 seconds). The first
temporary qualification harness stopped before any database assertion
because its external `.ts` file selected CommonJS; the `.mts` harness
corrected that setup and passed. Independent review approved the exact
metadata and reference diff. Earlier full suite and live-provider
results below retain their stated revision scope.
- Credential-policy head `c606d473b03709f27577a55a1dfc0b0d3b4359fd`
handles the remaining inactive-credential review warning. The existing
boundary accepts only concrete secret-owner 422 codes and exact
no-details disabled/coming-soon vault messages. Unknown 422,
authorization errors, provider wrappers, and temporary database/provider
failures still propagate. Independent source review approved this
three-file correction. No generic resolver, schema, provider protocol,
retry cap, or activation behavior changed.
- Creation/cleanup qualification at `c606d473b0`: all 39 cases pass
against real PostgreSQL (81.27 seconds). The new cases use actual
disabled/archived saved secrets, a valid disabled version, and provider
vaults created ready before normal API updates disable them or mark them
coming soon. Each remains manual through restart/repeated scans, with
zero provider lookup and no retry deadline. The independent plugin idle
blocker remains intact. The temporary 503 control retains retry
ownership and later recovers one cleanup owner. Before this correction,
all five inactive-state cases remained pending; that failed evidence is
retained. The prior 18-case known-ID task proof below belongs to
`237144fcf6`, whose helper and task path are unchanged by this
correction.
- Known-ID head `237144fcf69a249c0e0310563428c0f55ed6d55c` also
addresses the independent task-replacement reliability warning. The
existing creation operation now saves only the known nonsecret returned
resource ID before expiry validation and attachment. The write is scoped
to operation/company/original environment and allows only a missing or
matching ID. It changes no ownership state and copies no resolved
configuration, credentials, or provider payload. Independent source
review approved this four-file fix.
- The complete real-PostgreSQL task suite passes 18/18 (9.52 seconds).
Rejected attachments retain the ID through manual reconciliation and
repeated scans. A real committed attachment followed by an injected lost
response retains the same active lease and attached operation, with no
release or second create. Same-ID retries work; conflicting IDs and
foreign companies cannot change it. The initial three regression cases
failed because the ID was absent, before the fix.
- The shipped Daytona manifest does not declare task support, so it
cannot enter this generic replacement path. The dispatcher requires both
declared task support and an environmentTask worker method. Generic
drivers without creation lookup still use manual reconciliation, with a
known returned ID retained when the database write can commit. No
automatic lookup capability or unconditional compensation was added.
- Previous boundary-fix head `8ba77def151ea85eeae2db7693254db7f87673b0`
fixes the fresh review finding about missing credentials. Only known
secret-owner 404 results at the recovery secret-resolution boundary
select the existing manual outcome. Provider inspection and transient
database/provider errors remain retryable. Independent source review
approved this change.
- Secret-boundary focused acceptance at `8ba77def15`: all 34 cases pass
against isolated real PostgreSQL. The complete creation suite passes
16/16 (100.75 seconds); the complete cleanup coordinator suite passes
18/18 (99.83 seconds). The new cases use normal saved-secret creation
and deletion, confirm zero provider lookup, restart/repeated scans, no
automatic deadline, preserved independent plugin idle blocking, and no
additional idle blocker after the controlled worker supplies its quiet
proof. A temporary 503 resolution failure preserves pending ownership
and later recovers one cleanup owner. No additional live provider
resource was created.
- Failed attempts remain part of the evidence. The initial red run
reproduced the pending-operation bug and also exposed a numeric-deadline
test harness mistake, which was corrected. A later combined run reported
an existing replacement-receiver failure and a new idle-fixture
assertion failure; it was interrupted while the next test file remained
in imports. An isolated diagnostic confirmed that the manual state was
correct but the installed fixture plugin correctly blocked sleep without
its independent quiet proof. The corrected test retains that blocker
assertion before supplying the controlled worker witness. The unchanged
replacement-receiver case passes in the final complete 16-case run; its
earlier failure is not counted as a pass or assigned a proven cause. No
timeout or production idle guard was relaxed.
- Greptile reviewed previous combined head
`7ec06f77b7c14a745c569d18401f16df3d26b626` at 5/5, with no unresolved
review threads. Its CI produced the two concrete integration failures
addressed above. The earlier CI and live evidence below remain
historical evidence, not a claim that the current full gates have
completed.
- Master integration update: merged `dbff3747b8` and then latest master
`76fde891cc` without rewriting branch history. Only `doc/DATABASE.md`
conflicted; the sandbox ownership, object-backed run evidence, and cloud
configuration sections all remain. The master merges changed no sandbox
implementation, migration, or provider code. Migration numbering/safety
checks pass and exactly five schema snapshots remain. At merge head
`f5d23d4a59828835d01c150f83fa2645fb2d3bec`, the two focused
creation/coordinator suites pass all 32 cases against isolated real
PostgreSQL (64.57 seconds). Those merge-head checks precede the
missing-credential fix below. The earlier live Daytona proof still
covers the unchanged normal creation/lookup/teardown path; the new
missing-credential branch is qualified with real PostgreSQL and
controlled provider hooks.
The optional Superagent Security Scan was inconclusive on prior head
`7ec06f77b7`: its pinned PR merge-base checkout failed or timed out,
with a neutral conclusion and zero annotations. An earlier scan could
not start within 120 seconds; a bounded rerun request returned HTTP 404.
These scanner failures are not passing security-scan evidence. On final
head `0b4d05505e`, the additional Paperclip Review check is neutral: its
assigned review execution did not complete and produced no validated
assessment. Fresh Greptile 5/5 and independent source review are
separate evidence. Previous-head required CI, independent review, and
the live Daytona qualification below passed.
- [Complete previous-head
CI](https://github.com/paperclipai/paperclip/actions/runs/41105726052)
passed at `0b5cbb763ab8faa25aa0ff1d8a405cdc0d3db409`: 53 successful
checks and three intended skips, including typecheck, full tests, build
and E2E. Greptile is 5/5 on this head and both original findings are
resolved.
- Initial head: 200 PostgreSQL runtime, creation and coordinator tests
passed. The corrected head also passes all 32 creation and coordinator
tests, including deletion before/during lookup, a failed lookup after
deletion, restart, settled polling and actual idle-safety checks. Cases
cover failed admission, failed lease writes, concurrent recovery, lost
provider replies, ambiguous transfer retries, company rebind,
environment deletion, stopped runs, lookup backoff, restart, drain and
settled SQL quiet.
- Separate receiver processes: passed. A controlled HTTP provider
retains the resource while three independent receiver processes use two
empty web disks. Recovery creates one cleanup owner and never repeats
creation.
- Fresh PostgreSQL migration and two additional applications: passed.
The pending creation row survives unchanged.
- Server TypeScript and SDK/shared build checks pass after the
corrections. Migration numbering/safety checks passed. Exactly five
schema snapshots remain.
- Daytona adapter: all 330 tests passed, including stable-name lookup,
ownership checks, deletion and the updated manifest version. An earlier
run exposed the version expectation and it was corrected. Previous-head
CI passed.
- Live Daytona acceptance passed on `0b5cbb763a`: isolated real
PostgreSQL, normal runtime acquisition through independent receiver
processes, actual Daytona plugin/SDK 0.203.0, one disposable Ubuntu
24.04 sandbox (1 CPU / 1 GiB RAM / 1 GiB disk). A forced lease insert
failure retained the committed operation; a fresh receiver with empty
disk found the same sandbox, repeated recovery retained one cleanup
owner, and the actual claimed cleanup sweep expired the lease and
resolved the operation. Independent SDK lookup returned typed not-found;
ordinary recorded-ID inspection returned destroyed. This successful
attempt made one create, one recovery lookup, and one teardown. Two
earlier failed harness/provider-confirmation attempts are disclosed in
the [live acceptance
record](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6530313054).
The coordinator independently queried all three exact resource IDs
through the SDK and confirmed typed not-found for each.
- Complete CI-affected heartbeat/plugin warm-task suites passed 21/21
after test corrections. An unchanged workspace exposure case passed in
isolation after the prior CI shard reported an unhandled readiness
rejection. Complete previous-head CI and refreshed review passed.
## Risks
- Deleted companies retain an explicit manual reconciliation outcome.
Recovery does not retry missing credentials or insert a lease for a
deleted company.
- Only Daytona declares automatic creation lookup. Other uncertain
providers retain `needs_reconciliation`. A missing creation name alone
cannot prove that a late allocation will never appear.
- Pending operations use the existing process-local signal and startup
recovery model. This does not add cross-replica wake delivery or prove
deployed sleep.
- Legacy direct provider probes and custom-image setup use separate
lifecycles. They are outside this replacement of the run-lease
orphan-file fallback. The normal Test path that acquires a runtime lease
is covered.
- Existing orphan files still require the separate cutover inventory and
settlement. This change does not delete them.
- The schema change is additive. The new SQL can be reapplied.
Historical SQL and journal entries are preserved; only the oldest
generated snapshot is pruned.
## Model Used
OpenAI `gpt-6.1-sol`, with tool use, code execution and repository
review. The harness did not report the context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
The required branch status checks (`ci / verify`, `ci / e2e`) and all
test/build jobs passed on this head. The non-required Superagent
Contributor trust and Security Scan checks are still running without
findings; they are not claimed as passes. No required check or branch
rule is overridden.
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Failed lifecycle verification could allocate another test environment every minute indefinitely. Count failed host verification tests separately from claims, persist a repair wait for confirmed configuration failures or three generic failed tests, and clear it through authorized retry or execution configuration changes.
Preserve typed temporary execution-target failures through adapter checks so timeouts, transport failures and interrupted probes keep automatic recovery. Retain existing readiness, authorization, termination and cleanup gates. Document the policy and add real PostgreSQL persistence and actual OpenCode adapter regressions.
Verified 398 focused tests (one Windows-only test skipped on macOS), repository-wide typecheck and build, structural checks, fully green current-head CI and Greptile 5/5 with no unresolved comments.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Keep a future in-memory resume wake when domain dispatch is paused independently of coordinator admission. Preserve the saved obligation without rescheduling its expired deadline into an immediate skipped-attempt loop.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip manages AI agents and verifies real work through Product
E2E tests.
> - The fixed Chat feature workflow uses TestOps to select and report
one backlog case.
> - Official registration exposes a workflow-query URL and a Branch
parameter.
> - The existing reader accepts only workflow-file or numeric URLs, and
the admin launcher requires an operation parameter.
> - These assumptions reject valid official metadata before the feature
can run.
> - This pull request supports the exact observed form while preserving
source, selection, and once-only execution checks.
## Linked Issues or Issue Description
Refs: #18322, #18375.
**What happened?**
The second zero-model bootstrap stopped with a generic job-mismatch
code. Ordinary TestOps UI inspection then showed an official
workflow-query URL outside the accepted literals and a Branch-only job.
The exact first failed predicate in that bootstrap remains unknown.
**Expected behavior**
Accept that exact fixed-workflow URL. Preserve the existing master
Branch value when the job has no operation parameter. Keep rejecting
foreign URLs, unknown parameters, wrong branches, and conflicting
operation IDs before mutation.
**Steps to reproduce**
Pass the official query URL and Branch-only metadata through the
existing bootstrap, selection, and admin-launch owners. The new pure
controls exercise those actual helpers without a CLI, backend, or model
call.
**Paperclip version or commit**
This change starts from `0e15c92073a09e50df99190a37f69a86286b989b` after
#18375.
**Deployment mode**
GitHub Actions and the protected TestOps reporting environment.
## What Changed
- Share one exact URL predicate across bootstrap, evaluation selection,
and admin preparation. Add only the observed official workflow-query
literal.
- Support Branch-only, operation-only, and Branch-plus-operation jobs
with strict definition IDs, names, values, and duplicate checks.
- Preserve the exact master Branch value. Set an existing operation
parameter to `evaluate`; otherwise use the fixed workflow's existing
default.
- Use the same checked parameter vector for job and plan writes and
their readbacks.
- Report distinct fixed bootstrap predicate codes without response
values.
- Document when the protected operation-parameter ID is required.
- Align one existing Cloud-maintenance test with the inherited
managed-Cloud S3 policy. Preserve the same-version local-storage proof
in the parent, use valid synthetic Cloud settings in the child, and
check the expected provider mismatch without S3 requests. Retain the
no-seed and unchanged persisted-row assertions.
## Verification
- URL controls: three failures on the parent; all pass with the
correction.
- Branch controls: two failures and one correct rejection on the
predecessor; all pass with the correction.
- `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs`: 40
owning pure/mock controls pass on Node 26.4.0.
- Two MJS syntax checks and `git diff --check` pass. TypeScript noEmit
was not run for this MJS/documentation change.
- The first [normal CI
run](https://github.com/paperclipai/paperclip/actions/runs/41131094974)
failed only in the managed-skill Cloud fixture and its dependent
verification gate: 46 jobs passed, two failed, and one skipped. The
tested merge included a new upstream policy that rejects the test's
local-disk Cloud configuration. The exact caught runtime exception
remains unknown.
- Three pure controls using the actual policy validator pass: the old
invalid Cloud fixture rejects, the new synthetic S3 fixture admits, and
an invalid master key rejects. An initial tsx wrapper IPC permission
error executed zero controls and remains a separate setup failure.
- The PostgreSQL/maintenance-child test was not run locally. On new-head
AWS CI, `managed-skill-content.test.ts` passed all 17 tests in server
shard 9.
- [Normal
CI](https://github.com/paperclipai/paperclip/actions/runs/41133876636)
is green on exact head `b76d5dd40d7711f1b5f6b6316b1114c0aae02d57`: 48
jobs passed and one skipped. Both required `ci / e2e` and `ci / verify`
pass.
- Greptile reviewed this exact head at 5/5 with no actionable findings
or unresolved review threads.
- No actual CLI, backend, model, provider, app, database, build, or
bootstrap was run to validate this correction locally.
## Risks
The exact original bootstrap predicate is still unknown. This fixes a
demonstrated metadata false negative and preserves the prior failed run.
Actual backend parameter types and integration dispatch still need
verification. Unknown metadata stops before the once marker and writes.
This PR changes no workflow, credential, case, model, grader, clock, or
private-evidence policy. It does not claim a passing feature result.
## Model Used
OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool
use, and pure test review. The runtime does not expose the exact
context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Preserve dirty generations with an in-memory resume deadline while allowing settled transaction holds to end during drain. Keep unresolved database intent fenced by the coordinator.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
