Teach migration reconciliation to recognize the audited Runner D schema and data invariants, and retain user-defined functions and triggers in transformed logical backups.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Record the review-only 307-file snapshot as an ancestor without changing the canonical delivery tree. This makes the remaining 299-file Greptile partition terminate at the exact delivery head.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Review-only snapshot of the runtime and App-integration half of canonical PR #11634. Do not merge this partition independently.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Resolve the migration sequence against current master while preserving the audited Runner D data migration. Type the native backend factory mock so the merged TypeScript checks retain its two-argument contract.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Scheduled routines track each dispatch in `routine_runs` and link it
to an execution issue
> - Moving an execution issue to `blocked` or `cancelled` correctly
records a failed run state for operator visibility
> - When that issue later resumes or completes, the run can retain the
earlier failure reason and completion timestamp
> - That stale state makes an active or successfully completed routine
appear failed
> - This pull request reconciles the run back to a live state on resume
and preserves cleared failure details as completion context
> - The benefit is that routine run status consistently reflects the
current execution issue lifecycle without losing useful recovery history
## Linked Issues or Issue Description
Refs #9201
### What happened?
A routine execution issue that temporarily moved to `blocked` or
`cancelled` caused its linked routine run to become `failed`. If the
issue later returned to an active status or reached `done`, the routine
run could keep the stale failure reason and terminal timestamp.
### Expected behavior
Active execution issues should have an `issue_created` run with no
failure or completion timestamp. Completed execution issues should have
a `completed` run with no active failure reason, while retaining any
earlier transient failure in structured trigger context for diagnosis.
### Steps to reproduce
1. Create a routine run linked to a routine execution issue.
2. Move the issue to `blocked` and synchronize the run state.
3. Move the issue back to `in_progress` or forward to `done` and
synchronize again.
4. Observe that the run previously retained stale failed-state fields.
### Environment
- Reproduced on `master` at `da549123cc`.
- Core server behavior; not adapter-specific.
- Covered with the embedded PostgreSQL routines service test harness.
## What Changed
- Load the linked routine run while synchronizing execution issue
status.
- Restore transiently failed runs to `issue_created` when their
execution issue resumes active work.
- Clear stale failure state when an execution issue completes and retain
the earlier failure under `triggerPayload.transientFailure`.
- Add regression coverage for both resumed and completed execution
issues.
## Verification
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/routines-service.test.ts` — 57 tests passed.
- `pnpm --filter @paperclipai/server typecheck` — passed.
## Risks
- Low risk: the change is limited to routine execution issue/run
reconciliation.
- A completed run now stores a prior failed-state reason as structured
transient context instead of leaving `failureReason` populated.
- No schema, migration, API contract, or UI behavior changes are
included.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex using GPT-5.4 with reasoning, repository tools, GitHub
CLI access, code execution, and focused test execution. The runtime did
not expose a context-window size.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip provides a control plane for companies that run AI agents.
> - Sandboxed agents need a safe execution path for persistent command
streams.
> - The existing callback transport does not provide a bounded, generic
duplex route.
> - The host must control capability access, route identity, protocol
limits, and close behavior.
> - This pull request adds an opt-in duplex command-stream foundation
across the sandbox layers.
> - The feature stays inert because no current provider declares the
capability.
## Linked Issues or Issue Description
**Subsystem affected**
Cross-cutting (multiple of the above)
**Problem or motivation**
Sandbox command execution needs a persistent host-to-sandbox stream. The
current callback bridge uses a file transport and does not provide this
generic route.
**Proposed solution**
Add a fail-closed provider capability, generic worker protocol messages,
a host-owned bounded route, cross-layer service mediation, and a
versioned newline-delimited frame codec.
**Alternatives considered**
Keep the file transport and add feature-specific commands. This does not
provide one reusable duplex contract or host-owned route bounds.
**Roadmap alignment**
This work supports the completed Cloud / Sandbox agents roadmap area and
the safe autonomy goal in the product definition.
**Additional context**
The change passed a two-stage security review. The final code review
verdict was approve after fixes for active-stream bounds and
service-layer capability mediation.
## What Changed
- Add the opt-in `duplexCommandStream` provider capability with
fail-closed narrowing.
- Add duplex open, write, stop, and close requests and data and exit
notifications to the plugin worker protocol.
- Add a host-owned route with bounds for chunk size, cumulative bytes,
lifetime, protocol errors, pending requests, and pre-bind buffering.
- Add close acknowledgement handling with worker retirement when the
close remains unconfirmed.
- Wire `openDuplexChannel` through the execution target, runtime
service, and plugin worker.
- Add a versioned frame codec with shared wire-compatibility vectors and
split UTF-8 handling.
## Verification
- `server/src/__tests__/plugin-worker-manager-duplex.test.ts` passes 18
tests.
- `server/src/__tests__/environment-execution-target-duplex.test.ts`
passes 11 tests.
- `packages/adapter-utils/src/duplex-frame-codec.test.ts` passes 38
tests.
- `server/src/__tests__/sandbox-capability-contract.test.ts` passes 15
tests.
- Setup-token pseudo-terminal regression tests pass 47 tests.
- Server TypeScript check passes.
- Continuous integration will run the full required test, typecheck,
build, and policy checks.
## Risks
- Providers that opt into the capability must implement the complete
worker protocol.
- Route limit defaults can close a stream when a workload exceeds the
configured bounds.
- The capability remains disabled for current providers, so current
production behavior does not change.
## Model Used
OpenAI GPT-5 (`gpt-5`), with tool use and code execution. The model
reviewed and prepared this pull request from the supplied implementation
and verification record.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Execution workspaces need isolated databases, ports, and runtime
services
> - Concurrent workspaces could reuse ports or lose service ownership
after a restart
> - A markerless worktree also needed seed recovery, but normal
markerless instances still needed to boot
> - This pull request makes seed, port, and service ownership state
explicit and recoverable
> - It also checks live process and listener identity before it reclaims
shared resources
> - The benefit is reliable workspace startup, restart, adoption, and
concurrent provisioning
## Linked Issues or Issue Description
**What happened?**
Managed workspaces could lose runtime service ownership after a
control-plane restart. Concurrent worktrees could also reuse a port when
their parent paths differed. A seed recovery change made every
markerless instance resolve a worktree seed source, so normal instances
without a source could not start.
**Expected behavior**
Paperclip must preserve healthy managed services across restarts. It
must reserve unique ports across worktree parents. It must provision a
registered markerless worktree, but it must skip seed work for a normal
markerless instance.
**Steps to reproduce**
1. Start two managed worktrees under different parent paths at the same
time.
2. Restart the control plane while a managed service stays alive.
3. Start Paperclip with a config that has no seed markers and no
registered worktree source.
4. Observe duplicate port selection, lost service adoption, or a
seed-source startup error.
**Paperclip version or commit**
Current `master` plus the workspace runtime reliability changes in this
pull request.
**Deployment mode**
Local development with managed execution workspaces and embedded
Postgres.
## What Changed
- Added a shared port registry with lease heartbeats, process identity
checks, and live listener probes.
- Reserved worktree ports across custom parent paths and repaired
duplicate legacy assignments.
- Preserved and adopted healthy managed services across control-plane
restarts.
- Reconciled guest bind modes and verified listener ownership before
termination or reuse.
- Provisioned registered markerless worktree databases and kept normal
markerless instance startup as a no-op.
- Added CLI, shared, server, and shell regression tests for seed, port,
listener, restart, and adoption behavior.
- Updated the worktree development documentation.
## Verification
- `pnpm exec vitest run cli/src/__tests__/worktree.test.ts
--reporter=verbose` — 63 tests passed.
- `pnpm exec vitest run
packages/shared/src/worktree-port-registry.test.ts --reporter=verbose` —
5 tests passed.
- Focused runtime Vitest set — 199 tests passed across 37 suites.
- `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`
— 10 tests passed.
- `git diff --check` passed.
## Risks
- Port reservation now depends on lease and process identity data. The
fallback listener probe prevents early reclamation when process metadata
is incomplete.
- Runtime adoption is stricter about bind and owner identity. The tests
cover healthy adoption, stale records, PID reuse, and unrelated
listeners.
- Markerless seed detection now separates registered worktrees from
normal instances. The tests cover both paths.
- There are no database schema migrations.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- OpenAI Codex with the `gpt-5` model family. The serving snapshot and
context-window size are not exposed. The agent used reasoning,
repository tools, code execution, and test execution.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Dev Agent <dev@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip uses issue dependencies to pause work until blockers reach
a ready state.
> - A blocked issue with several blockers can miss its wake when the
final blocker completes.
> - The wake deduplication used a historical per-edge key, so an old
completed wake hid the current ready state.
> - This pull request adds a level-triggered key for the sorted set of
blocker issue ids and uses one helper for all wake paths.
> - The benefit is that the final blocker wake can repair a missed wake,
while repeated reconciliation stays bounded.
## Linked Issues or Issue Description
Refs #8009, #7853, and #6719. These public pull requests cover related
dependency-wake and deduplication behavior. This pull request fixes a
separate multi-blocker state-key gap.
**What happened?**
A blocked issue with multiple blockers received no
`issue_blockers_resolved` wake when the final blocker completed. An
earlier completed per-edge wake suppressed the wake for the current
all-ready state.
**Expected behavior**
The final blocker completion must emit one wake for the current ready
state. A later reconciliation pass must not emit a second wake for the
same state.
**Steps to reproduce**
1. Create a blocked issue with at least two blocker issues.
2. Complete one blocker and record its completed per-edge wake.
3. Complete the final blocker.
4. Run the route-time or reconciliation wake path.
5. Confirm that one level-triggered wake exists for the sorted blocker
set.
**Paperclip version or commit**
`eed1e5cad91a37547e1b521232da04b9ddb316f0`
**Deployment mode**
Local dev from source.
## What Changed
- Add a SHA-256 level-triggered idempotency key from the sorted blocker
issue ids.
- Share one deduplication helper across route-time, finalize-time, and
periodic wake paths.
- Treat state-key rows with idempotent statuses as duplicates.
- Treat legacy per-edge rows as duplicates only while they remain in
flight.
- Record skipped route-time wakes without suppressing later
finalize-time or periodic wakes.
- Add regression coverage for a completed earlier-blocker wake and a
second reconciliation pass.
## Verification
- Run `npx vitest run
server/src/__tests__/issue-dependency-wakeups-routes.test.ts`.
- Run `npx vitest run
server/src/__tests__/heartbeat-issue-liveness-escalation.test.ts`.
- Run `npx vitest run
server/src/__tests__/heartbeat-dependency-scheduling.test.ts`.
- Run `npx vitest run server/src/__tests__/issue-rewake-throttle.test.ts
server/src/__tests__/recovery-stale-issue-lock-sweep.test.ts`.
- Run `tsc --noEmit` on the touched files.
## Risks
The change alters wake deduplication for dependency reconciliation. The
new key uses the full sorted blocker set, so a change in that set
permits a new wake. The regression tests cover the missed-final-blocker
case and repeated reconciliation.
## Model Used
OpenAI Codex, GPT-5, tool-use model with code execution and repository
review support.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Co-authored-by: Paperclip <noreply@paperclip.ing>
Dotta