3 Commits

Include renames

Author	SHA1	Message	Date
Devin FoleyandPaperclip	4415665d67	fix(server): allow proved idle waits and verify historical cleanup (#16633) ... ## Thinking Path > - Paperclip manages agents, task continuations, integrations, and their execution resources. > - Automatic idle sleep needs durable evidence that accepted work and provider cleanup are settled. > - Response-driven questions, passive board escalations, and completed connection receipts currently look like unfinished work. > - Legacy ephemeral leases and terminal image sessions need actual deletion proof before their history can become idle. > - This PR adds narrow evidence-based exceptions and read-only provider verification while preserving queues, outcomes, and ownership fences. > - Idle instances can sleep without losing responses, automatic reconciliation, or cleanup obligations. ## Linked Issues or Issue Description **What happened?** Instances with no executable work can remain awake because pending human responses, passive recovery records, outbound integration configuration, or completed delivery history are counted as background work. Older failed ephemeral leases and finished setup sessions also lack the evidence needed to distinguish retained resources from deleted allocations. **Expected behavior** Permit idle sleep only for proved response-driven waits and settled history. Continue blocking real queued work, inbound integrations without a wake contract, unfinished accounting/retries, retained resources, and uncertain provider cleanup. **Steps to reproduce** Finish an issue's producing run, leave a pending question or confirmation, and request an owned scheduled idle report. Separately retain a processed connection receipt or a failed ephemeral lease whose provider allocation has already been deleted. The old checks keep those instances awake even after executable work has finished. Related public groundwork: #15522, #15599, #15761, #15897, #15907, and #16325. Searched existing public idle-sleep PRs; this addresses additional blockers in the current predicates. ## What Changed - Recognize supported response-driven waits and passive board recoveries without changing task state, response audiences, or continuation policies. Bind passive issue generation into routine-only alarms too. Keep automatically polled merge-confirmation cards awake. - Treat processed events, delivered intents, disabled drafts, and outbound MCP configuration as history/configuration where proved safe. Preserve pending deliveries, downstream wake requests, and inbound GitHub bindings. - Permit only pristine Slack setup drafts with a matching disabled channel connection, no provisioning registration, and no prior provider identity/activity. A saved slash-command name remains inert configuration. - Repair missed routine execution-issue status hooks in bounded awake-only maintenance. Lock and re-read exact owners, apply existing completion/transient-failure rules, and preserve normal continuation after an issue resumes. - Allow human-only connection setup to wait only when no reusable active connection of its purpose exists; available access reblocks sleep for reconciliation. The delivery worker shares this proof, releases its local idle hold, and retains maintenance only while already awake. Passive-only ticks pause before admission during an owned drain, allowing slow backups to finish; new producer notifications still reconcile and invalidate the backup. - Add optional `environmentInspectLease` SDK discovery/dispatch and read-only Daytona/fake-provider implementations. Old workers remain conservative. Daytona status inspection uses a fresh provider GET without reconnecting or poisoning unused-worker eligibility. - Bind historical credential reads to an unrotated initial secret version and its immutable resolved-value hash; reject ambiguous histories and ambient Daytona credential/API-endpoint fallbacks before querying provider state. - Verify bounded rotating batches of historical resources on already-awake Cloud instances, outside readiness. Store host-owned deletion evidence bound to exact row identity/version and reject concurrent ownership changes. Preserve failure outcomes, cleanup/export intents, and run continuation authority. - Preserve cancellation provenance, defer contended routine rows individually, and cap actual inspection RPCs across caller timeouts and worker exits. Add database, SDK, provider, configuration-scope, and race regression coverage and update idle protocol documentation. ## Verification - `pnpm -r typecheck` and `pnpm build` passed on `24fd1c0f5ec`; server typecheck/build passed again after the final coordinator-only fix in `60f23e6e4b6`. - Final server regressions: 258 sleep-safety/connection cases and all 17 coordinator cases passed. Earlier focused checks also covered 331 combined idle/cleanup cases, 34 custom-image cases, 39 credential/cleanup cases, 39 connection-history cases, and 80 existing routine cases. Counts overlap and are not additive. - SDK: 153 passed. Daytona: 406 passed with 14 existing opt-in skips. The provider implementation also typechecks against the locked SDK used by the Cloud wrapper. - Dependency boundaries were exercised with real child workers holding provider inspection replies beyond caller timeouts, a real PostgreSQL row lock delaying one routine owner, and real connection creation → queue notification → worker hold release → resumption after access is enabled without another notification. A controlled six-second backup goes through the real coordinator, task drain, work signals, and idle-safety generation checks; maintenance permits completion, while a new committed transaction invalidates it. - Full local `pnpm test:run` completed its general-server phase with 21,122 passing, 29 failing, and 109 skipped tests; later phases did not run. That long run spanned source edits and is not a final-revision pass. The affected Slack draft assertion passes on current source. Restoring `/usr/sbin` for `lsof` produced 233 passing tests and two Git-fixture failures across the other five files (3 existing skips). Both remaining assertions pass with command-scoped `init.defaultBranch=master`; this host defaults to main and leaves both branches in their fixture. No source/global Git setting changed for that comparison. Original failure logs are retained. - A parallel typecheck/build attempt raced generated Runner outputs and failed an import. Running the build sequentially passed with no source change. - Local database checks use the maintained test-constructor seam with native PostgreSQL 16.14 because bundled PostgreSQL 18 is unreliable on this host. No database assertions were skipped in the focused runs. An interrupted combined run was superseded by the separate passing coordinator and safety/connection runs; no owned test database processes remain running. - Apex reviewed final head `60f23e6e4b615a08591e19d86443a6c35c091ce1`: **5/5**, zero unresolved threads. All current-head CI gates are green, with expected policy skips. The branch is conflict-free and GitHub reports a clean merge state. - Current-head CI initially failed `plan-task-fixture.spec.ts:66` during fixture deletion: the final company DELETE hit `activity_log_company_id_companies_id_fk` after the feature assertions completed. The deletion path is unchanged here; the final diff only changes drain-gated passive maintenance. The preceding revision passed this shard. Failure evidence is retained in [the original CI job](https://github.com/paperclipai/paperclip/actions/runs/39006936642/job/117321244396); the targeted retry passed on the unchanged commit ([retry job](https://github.com/paperclipai/paperclip/actions/runs/39006936642/job/117322535013)). This shows recovery, not that the underlying company-deletion race is resolved. - These checks do not claim staging/production deployment or live provider cleanup proof. Provider tests simulate responses. Full Cloud staging verification requires a compatible Core/private-plugin composition after the Core revision is on master. ## Risks Sleep eligibility changes require the existing owned drain, final generation validation, local-work checks, and plugin drain. Unsupported states fail closed. Queued agent input and routine execution remain blocked. Outbound MCP exemptions exclude inbound GitHub grants; other unsupported inbound integrations remain awake. The inspection RPC is optional and requires a matching live worker method. Its actual outstanding calls are capped at two across workers even after the caller times out. Historical lease inspection requires its recorded plugin/config; setup inspection requires an unchanged recorded config fingerprint. Not-found under replacement account credentials is not historical deletion proof: ambiguous/rotated secret histories defer verification, and initial-version reads must match their recorded value hashes. Missing provenance (including legacy image sessions without a recorded config fingerprint), live/stopped resources, provider authorization failures, and concurrent changes defer verification. Routine repair replays existing completion/transient-failure behavior and timestamps at reconciliation time; it does not execute or discard deferred comments. No migration or destructive provider operation is added. The new marker does not grant run-resume authority. Full Cloud staging validation requires a compatible Core/private-plugin composition; no production deployment is included. ## Model Used OpenAI GPT-6 Codex, with repository/tool access and code execution. Exact model identifier and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run relevant tests locally; passing focused checks and full-run limitations are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-17 07:01:32 -07:00
Nicky LeachandPaperclip	db65a0863e	Deliver durable resource lifecycle events to plugins without polling (#16203) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Plugins keep external resources aligned with Paperclip projects and agents. > - The durable resource journal stores changes, but its plugin interface requires polling. > - Project deletion has no journal entry, so consumers must scan for missing projects. > - This PR delivers committed journal events through a native plugin callback and records deletion. > - Plugins can recover unfinished work without periodic change scans. ## Linked Issues or Issue Description Related: #15343 introduced the durable inbox. #15631 added required agent lifecycle participation. This change keeps both contracts and adds automatic delivery for the resource inbox. **What happened?** Plugins must schedule reads of the durable inbox. A deleted project has no durable deletion event. Consumers must scan saved bindings to detect its removal. **Expected behavior** Core should deliver committed resource events and replay unfinished work. Empty queues should have no polling timer. Project removal should leave a durable event. **Steps to reproduce** 1. Install a plugin that consumes project lifecycle events. 2. Create or update a project. The plugin must call `listLifecycle` to discover it. 3. Delete the project. The journal has no deletion entry. ## What Changed - Add `resourceLifecycle` to the manifest and `onResourceLifecycle` to the SDK. Require `events.subscribe` and a configured, enabled company scope. - Reuse transaction-aware work signals and the delivery coordinator. Wake after commit, on startup and plugin activation, and after configuration changes. - Acknowledge only a matching successful result. Retry unfinished or interrupted work. Retain deferred events without an idle retry timer. - Wake again after successful worker configuration application, and retain failed scans through idle drain (both Apex findings fixed). - Keep ordering per resource and deliver past failed or deferred resources. Keep the manual inbox methods for existing consumers. - Record project deletion in the removal transaction. Extend the journal constraint with a generated migration. ## Verification - Apex follow-up: 129 focused worker, delivery, coordinator, and application tests passed; server typecheck passed. Regression coverage verifies configuration-application races and failed scans across idle drain. - Focused lifecycle, transaction coordinator, manifest, and SDK dispatch suites: 105 tests passed, including single-connection saves, redacted failure diagnostics, wakeups retained during idle drain, and per-event retry deadlines. - Complete SDK suite plus delivery database tests: 128 tests passed. - Shared, database, and SDK typechecks passed. Direct server `tsc --noEmit` passed against rebuilt SDK declarations. - Full local `pnpm test:run` finished with 19,491 passed, 148 skipped, and 6 failed tests across 5 files (plus two suite setup errors). This run overlapped branch updates and is not final-head evidence: failures included missing Cargo, local Git default-branch assumptions, and stale in-process lifecycle modules. Final Apex-fix commit `75d028788` passed all CI gates, including the full test shards, and received Apex 5/5 with no unresolved comments. - `pnpm -r typecheck` and `pnpm build` were attempted. Both stopped because Cargo is unavailable locally. CI supplies the Rust toolchain. - Local installation used the repository's documented lockfile fallback because master has dependencies missing from its lockfile. No dependency changes are included here. ## Risks - Delivery is at least once. Plugins must re-read current state and make external effects idempotent. - Signals are process-local, like the other delivery queues. Startup recovers committed rows. Independent database writers or multiple active server processes require an explicit ownership and wake protocol. - Apply migration `0340_open_warlock.sql` before running code that records project deletion. Earlier deletions cannot be reconstructed. - Automatic delivery selects plugins with company configuration rows. `deferred` waits for a later startup, configuration change, or other work notification. - This does not replace required agent lifecycle participation or establish that external mounts are unused. ## Model Used OpenAI Codex (GPT-6), with tool-assisted code editing, code inspection, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-13 12:00:30 -07:00
Devin FoleyandPaperclip	b24c6909e8	Harden remote sandbox runtime probes, timeouts, and installs (#5685) ... ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent runs inside a sandbox environment so its CLI is isolated from the host > - Sandbox-backed adapter runs go through a small set of shared helpers — `ensureAdapterExecutionTargetCommandResolvable`, the sandbox callback bridge runner, and per-adapter `SANDBOX_INSTALL_COMMAND` strings > - When standing up new sandbox provider plugins, the existing helpers timed out, missed install fallbacks, or leaned on assumptions that only held for E2B > - Local adapters (`claude-local`, `codex-local`, `gemini-local`, `opencode-local`) needed slightly hardened probes so they could install themselves and validate inside *any* remote sandbox transport, not just E2B > - This pull request bundles those runtime fixes so future sandbox provider plugins inherit a working baseline > - The benefit is that adding a new sandbox provider plugin no longer requires touching adapter-utils or each local-adapter probe — the supporting infra is already correct ## What Changed - `packages/adapter-utils/src/execution-target.ts`: introduce `DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC = 1800` and `resolveAdapterExecutionTargetTimeoutSec(...)`. Local and SSH adapters keep the historical "0 means no adapter timeout" behavior; sandbox-backed runs without an explicit `timeoutSec` get an explicit 30-minute default so remote installs and warm-up don't time out at the per-RPC default. Plumbed `timeoutSec` through `ensureAdapterExecutionTargetCommandResolvable` so install probes inside a sandbox honor adapter-level overrides instead of the bridge's 5-minute default. - `packages/adapters/opencode-local/src/index.ts`: switch `SANDBOX_INSTALL_COMMAND` from `npm install -g opencode-ai` to `curl -fsSL https://opencode.ai/install | bash`. The npm package reifies four large prebuilt-binary subpackages in parallel even though only one matches the host arch; on bandwidth-constrained sandboxes that blew through the 240s install budget. The official installer fetches one arch-specific binary and adds `$HOME/.opencode/bin` to PATH via `~/.bashrc`, which the sandbox-callback-bridge login-shell script already sources. - `packages/adapters/{claude,codex,gemini,opencode}-local/`: harden remote-target probes — pass `--skip-git-repo-check` for Codex when probing outside a repo, normalize permission flags for Claude, and add `*.remote.test.ts` coverage that exercises the remote-sandbox path explicitly for each adapter. - `packages/adapter-utils/src/sandbox-install-command.{ts,test.ts}` (new): add `buildSandboxNpmInstallCommand` helper. `server/src/adapters/registry.ts` + new `server/src/__tests__/adapter-registry.test.ts`: wire adapter install commands so they fall back to a writable `$HOME/.local` prefix when global install isn't available. - `server/src/__tests__/plugin-worker-manager.test.ts` + new `server/src/__tests__/fixtures/plugin-worker-delayed.cjs`: pin per-call timeout overrides so plugin worker exec calls honor the caller's timeout instead of the worker's default. ## Verification - `pnpm typecheck` - `pnpm exec vitest run --no-coverage packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-install-command.test.ts` - `pnpm exec vitest run --no-coverage server/src/__tests__/plugin-worker-manager.test.ts server/src/__tests__/adapter-registry.test.ts server/src/__tests__/claude-local-adapter-environment.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/gemini-local-adapter-environment.test.ts` - `pnpm exec vitest run --no-coverage packages/adapters/codex-local/src/server/test.remote.test.ts packages/adapters/opencode-local/src/server/test.remote.test.ts packages/adapters/codex-local/src/server/codex-args.test.ts packages/adapters/codex-local/src/server/execute.remote.test.ts packages/adapters/gemini-local/src/server/execute.remote.test.ts` All passing locally. ## Risks - Touches shared `adapter-utils` and several `*-local` adapters. The 30-minute default applies only when both (a) the target is `remote+sandbox` and (b) no `timeoutSec` is configured — local + SSH paths are unchanged. New test coverage was added alongside each behavior change to pin the contracts. - Switching OpenCode's install command to the official installer is a behavior change for any operator running OpenCode inside a remote sandbox. Local installs are unaffected (the `SANDBOX_INSTALL_COMMAND` only runs when an adapter is being installed inside a sandbox). - Low risk overall — no migrations, no API surface change. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (1M context) - Capabilities used: extended reasoning, tool use (Read/Edit/Bash/Grep), no code execution beyond local repo commands ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A, no UI change - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-05-11 00:31:54 -07:00