mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 09:37:19 +01:00
c0fe0feffbb6b309a73a02f9a68595451f61baec
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
929e840595 |
fix(server): schedule routines without heartbeat polling (#17840)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/47/changes/paperclip/47 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Routines create work at saved cron deadlines. > - The heartbeat still wakes the routine worker when nothing is due. > - Its old next-deadline query also drops overdue work left by capped catch-up. > - Removing the heartbeat alone can strand that work and miss company reactivation. > - This pull request uses saved deadlines and committed producer changes. > - Idle routine scheduling stops querying until a change, retry, or real deadline. ## Linked Issues or Issue Description Refs #16514. Merged prerequisite: #17836. Related Cloud wake contract: #16215. **What happened?** Every heartbeat wakes the routine scheduler. The next-deadline query excludes overdue triggers. A 27-tick hourly backlog loses its deadline after the first 25-run pass. Company and budget lifecycle changes also lack routine schedule signals. **Expected behavior** Use the actual next saved deadline. Retain overdue work. Refresh after supported producer transactions commit. Keep idle schedules quiet between obligations. **Steps to reproduce** 1. Create an hourly routine with capped catch-up and 27 overdue ticks. 2. Run the registered worker once. The base processes 25 ticks and reports no next wake. 3. Create a future schedule and pause or reactivate its company. The base does not refresh its registered deadline until another wake. ## What Changed - Remove only the routine worker wake from the heartbeat tick. - Put routine eligibility and the next-deadline read in `routineService`. Include overdue triggers. Keep trigger/routine company scope consistent and require an active company. - Reuse `registerDeadlineWork` for retries, request generations, and passive drain behavior. Narrow its coordinator type to the `register` method it uses. Existing consumers keep the same runtime behavior. - Signal routine work from manual schedule invocation, company pause/resume/archive/delete, and budget-induced company pause/resume. Existing routine and trigger producers retain their transaction signals. - Stop between trigger batches during drain/shutdown. Finish a claimed batch so its advanced timestamp does not discard admitted work. - Add real PostgreSQL lifecycle tests and extend startup/query fixtures. Add a validated external database option to the company suite, including migrations and client closure. - Record producer coverage and limits in `doc/plans/2026-10-27-routine-schedule-sleep.md`. ## Verification - **417 passing unique cases across 14 complete files**, no skips, on accepted source `4659ef76d398193b15e7812d4a97f3da3b743226`. Coverage includes the real deadline worker, database eligibility queries, routine services/HTTP/routes/plugins, budget/company producers, git/device fixtures, startup and shared coordinator. - The first combined run passed 416 cases and had one UUID HTTP authorization timeout at its unchanged 20-second limit. One fresh complete 13-case E2E diagnostic passed (the affected case took 425 ms). No assertions, timeouts or production source were changed; the initial timeout's cause is not established. - Database build, production server typechecking/build, module boundaries, target migration order and Runner profile identity pass. Fresh PostgreSQL 17 fixtures applied all 412 journal entries through index 413; all selected test hashes and temporary fixture imports were restored exactly. - Normal parent merges preserve the tested production and test source. Historical acceptance and review corrections remain in [the routine audit](doc/plans/2026-10-27-routine-schedule-sleep.md). - No full local repository-suite, deployed suspend/wake, external alarm, live provider or fleet-cost claim. Final-head CI, security, exact-head Greptile 5/5 and actionable Ponytail findings remain separate merge gates. ## Risks - Company pause now prevents due routine claims, matching the prior deadline query. Rows remain saved for reactivation. Existing routine reactivation, project-pause skip records, cron, timezone, catch-up, and concurrency policies remain with the routine owner. - Signals cover supported service writers in one process. Independent SQL writers and other replicas need their own integration. - This proves worker quiet, not host/database suspension or fleet savings. Cloud idle proof and external wake coverage are unchanged. - Issue-monitor and legacy-agent-timer polling remain separate duties. This slice does not finish the full migration stack or its CI/review work. ## Model Used OpenAI Codex (GPT-6-based coding agent), using repository inspection, ordinary Git merges, source-composition proofs, PostgreSQL fixtures and local lifecycle tests. No subagents were used in the current reconciliation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
DottaandPaperclip
|
30b753f614 |
perf(budgets): scope accounting and deliver enforcement asynchronously (#17046)
## Thinking Path > - Paperclip manages AI agents and their paid work. > - Budgets must prevent new work and stop reported overruns. > - Cost settlement also ran serial lifecycle reconciliation for every eligible company agent. > - That made request latency grow with unrelated idle agents and delayed results on stop delivery. > - This change separates atomic admission and accounting from durable asynchronous enforcement. > - Scoped live usage detects overruns without writing provisional ledger charges. > - The result is constant query scope for ordinary settlement and recoverable stop delivery. ## Linked Issues or Issue Description **What happened?** Settlement and budget checks entered company-wide lifecycle reconciliation. A company with 1,000 unrelated idle agents performed 1,001 accounting transactions for one ordinary cost evaluation. User requests could also wait for provider termination. Running usage checkpoints did not enforce reported monetary exposure. **Expected behavior** Keep the final reservation gate authoritative. Commit charges, scope blocks, and enforcement intent before returning. Deliver stop requests after commit. Read only applicable policies and execution scopes for ordinary charges. Stop long-running work when supported reported or priced usage establishes an overrun. **Steps to reproduce** 1. Create a disposable company with one company budget and idle agents. 2. Evaluate a charge below the budget with 0, 17, and 1,000 unrelated idle agents. 3. Compare query counts, company accounting transactions, and duration. 4. Use the controlled partial-usage and delayed-stop fixtures to check enforcement and request responsiveness. **Paperclip version or commit** The measured baseline is `6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837` from `master`. Related public work: Refs #16976, #2006, #3539, #13763, #14296, #16853, #13075. These cover the affected service charge, cancellation, and accounting paths. This change removes their shared reconciliation overhead. ## What Changed - Batch applicable policy exposure under the existing company accounting lock. Preserve exact fractional money and zero-reservation dispatch fences. - Commit a durable final budget dispatch claim before provider I/O. Retain a separate final issue/run ownership transaction through synchronous handoff, without holding the company accounting lock or awaiting provider work. Preserve native recovery lease checks and duplicate handoff fencing. Browser handoff rechecks current policy; it reuses a consumed rate slot only for the identical policy generation. - Separate ledger spend, live observed spend, and effective held capacity. Use the maximum of reservation and live observation for each unfinished operation. - Deliver budget cancellation through the existing coordinator. Revalidate generations and persist queued/running stop fences before external termination. Bound stop concurrency to four. - Apply agent holds only to the affected agent. Project budgets leave other projects running. Project real company transitions in bounded lifecycle batches. - Replace periodic broad reconciliation with durable pending work, startup repair, receipt replay, and concrete retry/window deadlines. Retain drain and idle-safety checks. - Enforce accepted cumulative usage checkpoints without provisional charges. Final accounting atomically replaces live exposure and settles reservations. - Add operator-gated budget spans, regression coverage, map updates, and a real-PostgreSQL comparison harness. No schema or first-party telemetry change. ## Verification - Full workspace `pnpm -r typecheck` passes on the final ownership fix, rebased onto `bbcf5dfe98`. Production `pnpm build` passes on final head `4c5f4de8b1`. Map, module-boundary, token, and Storybook catalog checks pass. - Focused real-PostgreSQL coverage includes the 292-case rebased accounting/dispatch run, 63 browser cases, 52 run-control/cancellation cases, 42 policy-generation cases, 77 lifecycle/commentary cases, and the exact Muse confirmation regression. Native executor unit coverage passes all 768 cases. Final ownership coverage passes: 74 adapter/use-case cases, nine public regression cases, and the additional deleted-project race. These exercise competing ownership, budget changes and charges between phases, zero/nonzero duplicate claims, rollback, recovery contention, read-only capacity rechecks, and controlled HTTP startup after commit. Another 74 pure/startup/tracing cases pass. The complete CI matrix and both 5/5 reviews passed on `1ac9b58`; its external security scan identified a stale-owner handoff race. The final fix restores synchronous issue/run ownership fencing after the accounting transaction commits. Final head `8e84f55ce8` has fresh Greptile 5/5 with zero new comments, Paperclip review 5/5, and a passing security scan. The complete latest-head [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39535189808) passes, including budget, browser, Runner, build, and canary jobs. The unchanged Runner close fixture passed on its sole same-head job retry. - Broad local `pnpm test:run` invocations were stopped before required source updates (rebase and review fixes). They are not claimed as full passes. Final full-suite coverage comes from the complete latest-head CI matrix; local focused coverage and the incomplete broad invocations are reported separately. - Reproduce performance with `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-budget-enforcement.ts --baseline-ref=6c9b87d0922f1ebfc43e8b0c36889ac5ba7b9837`; repeat with `--reverse`. The harness uses disposable real PostgreSQL, 20 warm samples per variant/count, and records raw timings, SQL/lock counts, agent reads, unchanged agent state, and query plans. Final production-source measurement revision: `eff713a4f72c7f4b7a4002a701cb481d4bfccd95`; the extended harness is committed in `4c5f4de8b1`. | Unrelated idle agents | Baseline median / p95 | New median / p95 | Baseline SQL / locks | New SQL / locks | New agent reads | | --- | ---: | ---: | ---: | ---: | ---: | | 0 | 1.80 / 2.33 ms | 2.99 / 3.88 ms | 5 / 1 | 7 / 1 | 0 | | 17 | 15.53 / 19.55 ms | 2.24 / 2.60 ms | 90 / 18 | 7 / 1 | 0 | | 1,000 | 978.35 / 1,790.43 ms | 2.16 / 2.52 ms | 5,005 / 1,001 | 7 / 1 | 0 | - These measurements use local PostgreSQL 18.1 and Node 26.4.0 arm64. They isolate the cost-evaluation core and complete admission path, without hosted-region or provider latency. Prior-revision reversed-order controls also preserved the constant query counts; the final measurements above use the final production source. - Complete reserve → final ownership gate → immediate offline provider result is measured separately, with one responsible agent and one active operation at each unrelated-agent count. Admission p95 baseline/new: 0 agents **11.54 / 29.77 ms**; 17 agents **9.31 / 27.62 ms**; 1,000 agents **24.51 / 74.05 ms**. Baseline admission was already scoped; the stronger final fence adds bounded overhead. New counts stay constant at 53 Drizzle statements, three transactions, two company-accounting locks, and five agent-reading statements bound to the responsible agent. Zero unrelated-agent reads, exactly one provider callback per sample, and no leftover runs or holds. All admission scenarios stay below 250 ms. No external provider calls are included. - The partial-usage/delayed-termination fixture passed 20/20: receipt persistence p95 13 ms, receipt return to observed durable Stop fence p95 28 ms, complete receipt-start-to-fence p95 50 ms, unrelated database request p95 1 ms while termination was blocked. After releasing the controlled termination fixture, acknowledgement p95 was 14 ms. These are separately measured phases; fixture acknowledgement is not live-provider termination latency. - Models and provider deadlines are unchanged. No deployment, live inference, or staging acceptance is claimed. A quiet 30-minute follow-up waits for the merged revision on staging, then repeats classifier and controlled budget-stop acceptance in staging, recording hosted bookkeeping, queue delay, and actual provider acknowledgement separately. - Development failures retained: finalizer notification regressions (fixed; 27/27 affected cases passed), invalid browser admission fixture assumptions (fixed; browser coverage now passes 63/63), and first-head lifecycle/dispatch CI regressions (reproduced and repaired). A prior-head Cases E2E test hit its unchanged announcement-dismiss handler race while opening a filter; the same journey passed on `1ac9b58`. The external security review then found a stale-owner race between accounting commit and provider entry; final verification covers a competing run in that gap while retaining provider I/O outside the accounting lock. Early PostgreSQL startup attempts hit local semaphore pressure; serial reruns succeeded. The final-head Runner lane had one failure among 1,823 tests: the unchanged `timeout-attachment` fixture failed at its first process close after 6.28 seconds. The sole same-head job retry passed all 1,819 active tests (four skipped), including that fixture in 1.85 seconds. Three isolated local probes could not run correctly because the Rust fake-provider fixture binary was absent; cleanup masked the original startup failure. CI builds that fixture through its normal lane. These probes are inconclusive and are not counted as passes. ## Risks - This changes accounting, lifecycle, and cancellation ordering. Durable generations, source/sequence checks, ownership leases, and company locking remain required for correctness. - Provider stop acknowledgement can be slow or fail. Stop intent remains pending until existing termination proof succeeds; cancelled row status alone does not acknowledge delivery. - Mid-run limits depend on provider usage reports or supported price calculations. Providers that report only at completion cannot provide exact mid-call detection. - Signals are process-local wake hints for the supported single-active-writer model. Durable state and startup recovery remain authoritative. - There are no migrations, new queue tables, or cached balances. Repeat the classifier and controlled budget-stop acceptance after normal rollout. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, code execution, and parallel coding agents. The host does not expose the exact backend model revision or context-window size, so those values are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
2dd9811e8b |
Own agent hiring and termination in a lifecycle module (#15631)
## Thinking Path > - Paperclip manages AI agents and their work. > - Agent creation, approval, pause, and termination change whether an agent can work. > - These changes currently use separate service and database paths. > - Some agents need setup after their record exists and before their harness can run. > - This PR adds one module that owns those state changes and their transactions. > - The module records progress and retries incomplete work after a server restart. ## Linked Issues or Issue Description **Problem or motivation** A hire can become active before its execution environment is ready. Direct agent state writes also bypass one common lifecycle boundary. Termination does not have a durable step for resource cleanup. **Proposed solution** Use agent lifecycle commands for hiring, approval, pause, resume, and termination. Keep proposed hires pending until approval. Prepare each approved hire, test its saved harness configuration, and then permit work. Wait for required plugin work before completing a step. **Alternatives considered** A general database commit hook would expose transaction details to callers. A timer alone would delay each new hire. This module owns each command transaction and starts its worker after commit. A periodic scan recovers work after a process stops. **Roadmap alignment** This change adds the lifecycle boundary for the existing agent and plugin systems. It does not add a user interface or a provider implementation. It replaces the lifecycle delivery approach in #15391. Related proposals include #11475 and #13099; this change does not add their external executor protocols. ## What Changed - Add durable approval, preparation, verification, pause, resume, stop, and cleanup states. - Keep agent creation, lifecycle state, configuration writes, and agent record deletion behind `modules/agent-lifecycle`. Keep validation, credential rules, reads, onboarding, invitations, general approvals, and company data deletion in their services. - Commit a hire before the remaining onboarding or invitation work. Serialize requests with the existing dedicated database connection helper. Reuse the hire after a failed attempt. - Keep accepted work in the run queue during setup. Require execution and cleanup evidence before completing pause or termination. - Retain the original credential owner when approving migrated hires. - Start lifecycle work after each command commits. Use database leases and revision checks for retries and late results. - Add a generic required plugin method and capability. Store the required plugin IDs for each agent. Track host and plugin completion separately. - Keep harness verification and credential rules in services. Supply their results to lifecycle through explicit ports. Reject failed logins and incomplete probes. - Remove reverse imports from lifecycle adapters into services. Share record queries and validation below both layers. - Use one hire decision path for agent IDs and approval IDs. Resolve the approval and agent change in one transaction. Expose policy reconciliation on the command object. - Save configuration and verification invalidation through one module operation. Ordinary commands own their transaction. Restrict the shared transaction integration to its listed callers. Reject identity, company, accounting, timestamp, and unknown fields. - Require completed termination before deleting agent records. The company deletion service calls the module through a separate deletion interface. - Save an approved secret before applying its agent binding. A failed binding leaves the secret approved. A retry reuses it. - Remove old permission and metadata export files. Move hire configuration rules out of lifecycle persistence. - Report failed worker steps and scans with operation context and known error codes. Omit provider messages, stacks, and configuration values. Document ordinary application use and extension rules. - Recover expired native owners only after execution stops. Retry cleanup of failed retained leases. Wait for setup in the live smoke script. - Keep manual, budget, and company pause reasons separate. Guard execution status writes with the ready state. - Add a migration, boundary checks, regression tests, and a lifecycle guide. Keep the current UI. - Limit periodic policy checks to 100 eligible agents per minute. Continue from the last agent ID. Exclude pending hires and agents in termination or a final state. Keep explicit company and budget updates immediate. ## Verification - Latest head `ccc3c86b0` aligns the shared status update schema with the PATCH route. All 15 shared tests, 104 server route/OpenAPI tests, 2 CLI tests, and shared/server/CLI TypeScript checks pass. The Greptile comment is fixed. CI is running; no new Apex review was requested for this small fix. - Rebase `75708ac94` rebases onto master at `2eb5187e4`. The backup test uses the shared 90-second timeout from master; the older timeout commit was removed. The affected backup test passes. - Agent budget edits now pass the target agent ID through enforcement. All 50 affected tests, server TypeScript, and 11 boundary tests pass. Both configuration update paths have regression coverage. CI is running on the latest head. No new Apex review was requested for these small fixes. - Rebase `9b568666a` preserves the run-control extraction and delivery-work startup changes from master. All 559 affected tests, server TypeScript, and 11 boundary tests pass. CI and Apex review are running on the final head. - The policy scan fix passes 56 lifecycle and diagnostic tests, 24 policy and caller tests, and 11 boundary tests. A regression test checks the page limit, interval, excluded states, and cursor wraparound. - Rebase `fd4698750` passes all 611 tests in eight affected lifecycle, route, and heartbeat suites. Server TypeScript and all 11 module boundary tests pass. The company deletion rollback and concurrent hire tests are included. - Follow-up `00aac7646` fixes the failed decision deletion test and the public update schema. All 126 affected server tests, 8 shared tests, 2 CLI tests, and 11 boundary tests pass. Shared, server, CLI, and UI TypeScript checks pass. - Server TypeScript passes: `pnpm --filter @paperclipai/server exec tsc --noEmit`. - All 11 boundary tests pass: `node --test scripts/check-module-boundaries.test.mjs`. The repository boundary scan also passes. - All 68 lifecycle, failure diagnostic, and harness verification tests pass. Tests reject forbidden fields through the root command, combined transition command, and restricted transaction integration. They also check invalid fields returned by configuration preparation. - Failure tests cover host work, plugin work, claims, renewals, deferrals, and scans. Log tests check known error codes and exclusion of private error contents. - All 355 tests in 10 existing caller suites pass. These tests cover secret approval, pairing, pending hires, credentials, company import, built-in agents, managed agents, instructions, and skills. - The branch is rebased onto `origin/master` at `2eb5187e4`. The queue guards and cancellation lock fix use the extracted heartbeat queue. Agent cancellation uses the extracted run-control file. The source diff scan found no secrets or private URLs. - Earlier checks cover approval retries, rollback, late verification results, policy holds, deletion rollback, concurrent company deletion, and queue admission during setup. - Before the rebase, CI passed on `00aac7646` and Greptile Apex rated it 5/5 with no open comments. CI and a new Apex review are pending for the rebased commit. The branch has no merge conflicts. - The full local build and native Dot suite were not run. The host has no Cargo. No live provider test was run. ## Risks - Background policy recovery takes more than one minute when more than 100 agents are eligible. Explicit company and budget updates still reconcile immediately. - The restricted configuration integration keeps an agent reference and its dependent records in one transaction. It cannot change a budget or invoke a lifecycle transition. A periodic scan starts pending setup after that transaction commits. - Secret approval and binding use separate transactions. A failed binding can leave an approved secret with no agent binding. - New approved hires remain unavailable until setup and verification complete. Existing clients see `paused` during these steps. This PR does not change the UI. - Migration `0320_daily_onslaught.sql` maps existing statuses and pause reasons. It does not create external resources or claim that an existing harness passed verification. - Use a coordinated server update for this migration. Older server versions can write legacy status without the new lifecycle checks. - A failed onboarding or invitation request can leave a committed hire. A retry reuses that hire and completes the remaining records. - Deletion requires completed termination. Company deletion can require another request while cleanup is pending. - Required plugins must make repeated calls safe and reject older revisions. A disabled or removed required plugin blocks completion until it is restored. - A new hire can run a second harness test after the existing UI test. Some adapters can charge for this test. - No live provider test was run. Provider implementation and resource backfill remain separate work. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code tools, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in this PR with the feature request field labels - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] Targeted local tests pass - [x] All Paperclip CI gates are green on the new commit - [x] Greptile Apex is 5/5 with no open comments on the new commit - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
8c6cc7dccf |
feat(cloud): sync primary-company archive state with the Cloud control plane (#13837)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip Cloud runs managed instances and controls their lifecycle
from a control plane.
> - Inside the app, a person can archive a company. On a managed
instance, the Cloud-pinned primary company is the whole organization.
> - Today that archive stays local. The control plane does not learn
about it, so it keeps the instance running and shows the organization as
live.
> - This pull request notifies the control plane when the primary
company crosses the archived boundary, and adds two control endpoints so
the control plane can verify the state and undo the archive during a
restore.
> - The benefit is that an archived organization stops running and shows
as archived, and a restore brings the company back without manual steps.
## Linked Issues or Issue Description
No public issue exists. Description follows the enhancement template:
**What existing behavior does this improve?**
Company archive on Cloud-managed instances. Archiving the primary
company pauses agents and cancels runs, but the hosting control plane
never learns about it.
**Subsystem affected**
Server: company service, cloud-control middleware, instance routes.
**Current behavior**
A person archives the primary company. The instance keeps running. The
Cloud portfolio still shows the organization as live. Unarchive after a
Cloud restore requires manual steps inside the product.
**Proposed behavior**
The company service rings a Cloud lifecycle doorbell when the primary
company is archived or unarchived. The control plane verifies the state
through `GET /api/instance/lifecycle` before it acts. During a restore,
the control plane calls `POST /api/instance/lifecycle/unarchive-primary`
to bring the company back. Self-hosted instances are not affected.
**Reason and benefit**
An archived organization should not keep running, and its hosting
console should show it as archived. The verified read-back keeps the
doorbell a hint: a forged or duplicated ring cannot change state.
## What Changed
- New `services/cloud-lifecycle-sync.ts`: fire-and-forget doorbell `POST
{cloudOrigin}/v1/tenant/lifecycle-changed` with bounded retries. It runs
only on cloud-managed instances, and only for the derived primary
company id. It never blocks or fails the company mutation.
- `services/companies.ts`: ring the doorbell after a committed archive
or unarchive transition, in both the `update()` status-patch path and
`archive()`.
- New `GET /api/instance/lifecycle`: reports the primary company id, its
status, and how many other companies are not archived. Bound to a
`lifecycle:read` Cloud control assertion; board members can also read
it.
- New `POST /api/instance/lifecycle/unarchive-primary`: idempotent
unarchive of the primary company as a system actor. Bound to
`lifecycle:unarchive-primary`; requires instance admin otherwise.
- `middleware/cloud-control.ts`: a closed endpoint→method→action table
replaces the single hardcoded endpoint. Each assertion still authorizes
exactly one action on one endpoint.
- `services/cloud-instance.ts`: the primary-company id derivation moves
here as the single definition; `middleware/auth.ts` delegates to it. The
derivation itself is unchanged.
## Verification
- `pnpm exec tsc --noEmit` in `server/` is clean.
- `pnpm exec vitest run src/__tests__/cloud-lifecycle-sync.test.ts
src/__tests__/cloud-control-task-drain.test.ts
src/__tests__/instance-settings-routes.test.ts
src/__tests__/companies-service.test.ts
src/__tests__/cloud-tenant-company-provisioning.test.ts` — all pass.
- New tests cover: doorbell env-gating, retry bounds, no-throw contract;
the read-back status and sibling count; idempotent unarchive and the
admin gate; non-cloud 404s; control-assertion action binding for the new
endpoints, including cross-action and wrong-method rejection; and a
companies-service test that proves the doorbell rings exactly on
archived-boundary transitions.
## Risks
- Self-hosted instances see no behavior change: without a Cloud signal
the doorbell is a no-op and the endpoints answer 404.
- The doorbell is advisory by design. The control plane verifies through
the read-back before it acts, so a lost or duplicated ring cannot
corrupt state.
- The unarchive endpoint reuses the existing `companyService.update`
path, so agent reactivation and activity logging behave exactly like an
in-product unarchive.
## Model Used
- Claude Fable 5 (`claude-fable-5`), via Claude Code CLI, extended
thinking and tool use enabled.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
|
||
|
Devin Foley
|
76f7019bdf |
feat(server): derive hosted-tenant issue prefixes from the company name and follow renames (#12292)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every company has an issue prefix. It is the visible half of each task and case identifier, and a self-hosted company derives it from the name it was created with > - A hosted or managed instance does not use the create-company flow. The trusted-header auth path claims the tenant company instead > - That path minted the prefix from a hash of the stack id, and it wrote a placeholder description that nobody chose > - So a hosted company showed opaque task IDs such as `PC7F2A-14`, and a rename never changed them > - This pull request derives the prefix from the company name on that path too. It re-derives the prefix when the name changes on a managed instance, and it rewrites the stored issue and case identifiers so existing tasks follow the rename > - It also repairs each company that an earlier build claimed. The repair runs once, on the next authenticated request > - The benefit is that task IDs on a hosted instance read like the ones on a self-hosted instance, and they stay correct after a rename ## Linked Issues or Issue Description No public issue exists. The description below follows `.github/ISSUE_TEMPLATE/enhancement.yml`. **What existing behavior does this improve?** The tenant company claim in `resolveCloudTenantActor` (`server/src/middleware/auth.ts`) and the company update in `companyService.update` (`server/src/services/companies.ts`). Both decide the `issue_prefix` and the `description` of a company on a hosted or managed instance. **Subsystem affected** `server/` — REST API and orchestration services. One small hint was also added in `ui/`. **Current behavior** A self-hosted company gets its issue prefix from its name. "Acme Robotics" becomes `ACM`, and its tasks read `ACM-14`. A hosted or managed instance claims the company through the trusted-header auth path. That path wrote a different prefix: `"PC"` plus the first four hex characters of the SHA-256 of the stack id. The same path also wrote a placeholder description, `"Provisioned by ... for stack <stack id>."`. The result is a task ID such as `PC7F2A-14`. It says nothing about the company. A later rename of the company does not change it, because nothing re-derives the prefix after creation. **Proposed behavior** The claim path derives the prefix from the company name, exactly as the create-company flow does. It writes no description. On a managed instance, a rename re-derives the prefix. The stored issue and case identifiers move with it, so `ACM-14` becomes `NOR-14` when "Acme Robotics" becomes "Northwind Traders". A rename that keeps the same three-letter base keeps the current prefix, including any disambiguating suffix. A self-hosted instance is unchanged. A rename there still keeps the prefix the company was created with. Companies that an earlier build already claimed get a one-time repair on their next authenticated request. The repair re-derives the prefix from the current name, re-keys the identifiers, and clears the placeholder description. **Reason and benefit** A task ID is the primary handle for a task. People type it, paste it into chat, and read it in a URL. On a hosted instance that handle was an opaque hash, and it disagreed with the company name that the same user chose during signup. The name is the only prefix source a hosted user ever supplies, so the prefix now follows it. **Breaking changes** Yes, on hosted and managed instances only. A company rename now rewrites the stored issue and case identifiers. Links that carry an old identifier stop resolving after the rename. The company settings page states this before the user saves. The one-time repair applies the same rewrite once to companies that carry the old hash prefix. Self-hosted behavior does not change. ## What Changed - Added `server/src/services/issue-prefix.ts`. It holds the prefix helpers that used to live inside the `companyService` closure: `ISSUE_PREFIX_FALLBACK`, `deriveIssuePrefixBase`, `issuePrefixSuffixForAttempt`, and `isIssuePrefixConflict`. The companies service now imports them. - Added `pickAvailableIssuePrefix` to that module. It reads the prefixes in one base family and returns the first free candidate. A standalone `INSERT` can retry on a unique violation, because each failed statement is its own implicit transaction. A caller that already holds a transaction cannot, because the violation aborts the whole transaction. Such a caller picks first, then writes. - Added `rekeyCompanyIssueIdentifiers` to that module. It rewrites the prefix of the stored `issues.identifier` and `cases.identifier` values of one company in the caller's transaction, and it returns the two row counts. - `companyService.update` re-derives the prefix when the name changes on a managed instance, re-keys both tables in the same transaction, and writes a `company.updated` activity entry after the commit. - `resolveCloudTenantActor` claims the company with a name-derived prefix and a null description. The claim retries with the next suffix when the prefix is taken. - `resolveCloudTenantActor` also runs a one-time repair for companies that carry the old hash prefix. An exact-match fence on the update lets a concurrent rename win. The repair is idempotent, because its guards stop matching after it lands. - The rename takes a row lock on the company before it compares anything against it, and it re-keys from the prefix it reads under that lock. Only patch and environment facts gate the lock, so no stale read can steer the decision. Two overlapping updates would otherwise leave a company whose prefix disagrees with its own identifiers, in either direction: two renames, where the second re-keys from a prefix the first already moved; or a rename plus a stale form that resubmits the original name, where the second sees an unchanged name, skips re-derivation, and restores the old name on top of the first rename's prefix. Only a managed instance takes the lock, and only for an update that carries a name. - Both helpers compare an exact identifier head instead of a LIKE pattern. A stored prefix is data, so it must never be read as a pattern. - The company settings page shows a hint under the name field on a managed instance: renaming can change the task ID prefix. ## Verification Automated tests: ``` pnpm --filter @paperclipai/server exec vitest run \ src/services/issue-prefix.test.ts \ src/__tests__/companies-service.test.ts \ src/__tests__/cloud-tenant-company-provisioning.test.ts \ src/middleware/cloud-tenant-actor.test.ts \ src/__tests__/auth-session-route.test.ts \ src/__tests__/cloud-routes.test.ts \ src/__tests__/cloud-instance.test.ts \ src/__tests__/company-branding-route.test.ts \ src/__tests__/company-cloud-floor.test.ts \ src/__tests__/companies-route-cross-company-authz.test.ts \ src/__tests__/companies-route-path-guard.test.ts \ src/__tests__/company-portability.test.ts pnpm --filter @paperclipai/ui exec vitest run pnpm --filter @paperclipai/ui typecheck ``` New coverage: - `server/src/services/issue-prefix.test.ts` covers the derivation, the suffix ladder, the cause-chain walk of the unique-violation detector, and `pickAvailableIssuePrefix` against a stubbed select. - `server/src/__tests__/companies-service.test.ts` covers a managed rename against a real Postgres database: the prefix moves, both identifier tables are re-keyed, and the activity entry is written. It also covers a same-base rename, a collision that takes the suffixed candidate, a non-name patch, and a self-hosted rename that leaves the prefix alone. Two more tests drive the overlap cases: two concurrent renames of the same company, and a rename racing a stale form that resubmits the original name. Both assert that the surviving name's base matches the company prefix and that the stored identifiers sit on that prefix. - `server/src/__tests__/cloud-tenant-company-provisioning.test.ts` covers the claim path and the repair against a real Postgres database: a name-derived prefix, a null description, a suffixed prefix on collision, the full repair, a second pass that changes nothing, a description-only repair, and an operator-written description that the repair leaves alone. - `ui/src/pages/CompanySettingsRenameHint.test.tsx` covers the hint on a managed instance and its absence on a self-hosted instance. The `substring` cast in `rekeyCompanyIssueIdentifiers` is load-bearing and the database tests prove it. The driver binds the offset as text. Without the `::int` cast Postgres resolves the SQL-regex overload of `substring`, and every identifier becomes NULL. ## Risks - **Re-keying changes existing identifiers and URLs.** This is deliberate, and it happens on hosted and managed instances only. After a rename, a link that carries an old task identifier stops resolving. The settings page warns about this before the user saves. - **Identifiers inside comment text are not rewritten.** Only the `identifier` columns of `issues` and `cases` move. A task ID that someone typed into a comment, a description, or a document keeps the old prefix. - **A lost prefix race inside the rename transaction surfaces as a conflict.** The rename picks a free prefix and then writes, because a unique violation inside a transaction aborts the whole transaction. Two *different* companies renamed onto the same base at the same moment can still collide. The loser sees its PATCH fail with the unique violation. The write is retryable by the client, and the window is a single statement wide. Two renames of the *same* company no longer race: the row lock serializes them, and the second one re-keys from what the first committed. - **The rename holds a row lock.** A managed rename takes `SELECT ... FOR UPDATE` on its own company row for the rest of the transaction. It is one row, and no other path in the transaction locks a company row, so there is no lock-order cycle. A self-hosted instance and every non-rename company update never reach the lock. - **The one-time repair is best effort.** It runs inside a try/catch and logs a warning on failure, so it never blocks authentication. A failed pass is retried on the next request, because its guards still match. - No schema change and no migration. ## Model Used - Provider: Anthropic (Claude) - Model: Claude Opus, model id `claude-opus-5[1m]` - Context window: 1M - Reasoning mode: extended thinking - Capabilities used: agentic tool use through Claude Code (file edits, shell, test runs against an embedded Postgres database) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Christian Lappin
|
fc5c6ffed2 |
fix(server): return 404 instead of 500 for non-UUID company refs (#9959)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The board REST API is how operators and integrations read company
state; `GET /api/companies/:companyId` is one of its most basic reads
> - The route passes the raw path param into `companyService.getById`,
which queries the uuid-typed `companies.id` column directly
> - Any non-UUID ref — a slug, a typo, a stale bookmark — makes Postgres
throw `invalid input syntax for type uuid`, which surfaces as an HTTP
500 with a stack trace in the server log instead of a clean client error
> - A 500 for malformed client input is miscategorized: it pages
operators, pollutes error budgets, and hides the actual problem ("that
ref doesn't exist") from the caller
> - This pull request guards `getById` with a UUID check so non-UUID
refs resolve to `null` and the route returns its existing 404 path
> - The benefit is correct HTTP semantics for bad input, quieter logs,
and one less misleading 500 for self-hosters to chase
## Linked Issues or Issue Description
Fixes #9962 — `GET /api/companies/:companyId` returns 500 (`invalid
input syntax for type uuid`) for non-UUID refs instead of 404. Full
repro and log excerpt in the issue.
## What Changed
- `server/src/services/companies.ts`: `getById` returns `null` early for
non-UUID refs instead of passing them to the uuid-typed query.
- `server/src/__tests__/companies-service.test.ts`: regression test —
non-UUID refs (`"tumbly-haus-creative"`, `"not-a-uuid"`, `""`) resolve
to `null` without a query error.
## Verification
- `npx vitest run src/__tests__/companies-service.test.ts` — 12/12 pass
(new test included, embedded-postgres suite).
- Manual: `curl -i /api/companies/not-a-uuid` → 404 (was 500); `curl -i
/api/companies/<real-uuid>` → 200 unchanged.
## Risks
- Low. Pure input-validation guard on one read path; UUID lookups are
byte-for-byte unchanged. Only behavioral shift is 500→404 for refs that
could never have matched a row.
## Model Used
Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — diagnosis
from server logs, patch, and test authored with extended thinking and
tool use; human-reviewed and submitted by @christianlappin.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes (n/a —
no doc references this error path)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending first CI run)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending)
- [x] I will address all Greptile and reviewer comments before
requesting merge
|
||
|
scotttong
|
815e49bb7c | feat: make chat-style tasks the default experience (#11101) | ||
|
|
1f1f545238 |
feat: add built-in summarizer and summary slots (#9713)
## Thinking Path > - Paperclip is the open source control plane people use to organize, govern, and understand AI-agent work > - Operators need concise, current status views across projects and execution workspaces without manually reading every issue and run > - Paperclip already has auditable issues, documents, built-in agents, routines, and live run events, but no first-class summary-slot workflow connecting those systems > - A built-in Summarizer can generate status prose through ordinary governed tasks while summary slots provide stable, revisioned destinations for that output > - The UI needs to show current summaries, generation progress, failures, revisions, and streaming draft status in the places operators already work > - This pull request adds the end-to-end summary-slot data, API, agent, orchestration, and UI surfaces behind an experimental setting > - The benefit is decision-oriented status context that remains company-scoped, auditable, retryable, and inexpensive by default ## Linked Issues or Issue Description No public GitHub issue exists for this feature. **Problem** Operators currently have to reconstruct project and workspace status by reading many issues, runs, and comments. This makes it hard to identify decisions, review queues, recent work, and the next event worth watching. **Proposed capability** Add an experimental summary system with revisioned summary slots for projects and workspaces, a paused-by-default built-in Summarizer agent, governed generation tasks, live draft status, and reusable UI cards. **Expected behavior** - Summary data remains company-scoped and revisions remain auditable. - Generation runs through normal issue/agent orchestration and deduplicates active requests. - Only the linked built-in Summarizer generation task can author a slot revision. - Operators can generate, retry, inspect revisions, and follow draft progress from project and workspace views. - The feature remains opt-in and background generation remains paused by default. ## What Changed - Added summary-slot schema, idempotent migrations, shared contracts, validators, API paths, and service tests. - Added company-scoped summary-slot routes for reading revisions, requesting generation, and guarded Summarizer writes with activity logging. - Added terminal generation finalization, failure reasons, assignment wakeups, and orchestration integration. - Added the paused-by-default built-in Summarizer bundle, low-cost runtime defaults, status-summarization skill, and stale-summary routine. - Added summary cards, revision selection, retry/configuration states, live draft streaming, transcript chunk handling, and project/workspace integrations. - Updated Claude local parsing for streamed status output and expanded server, adapter, shared, database, catalog, and UI coverage. ## Verification - `pnpm -r typecheck` - `pnpm exec vitest run packages/db/src/summary-slots-schema.test.ts packages/shared/src/summary-slot.test.ts server/src/__tests__/summary-slot-routes.test.ts server/src/__tests__/summary-slots.test.ts server/src/__tests__/built-in-agents.test.ts ui/src/components/SummarySlotCard.test.tsx ui/src/components/SummarySlotCard.status.test.tsx ui/src/components/useSummaryDraftStream.test.tsx ui/src/lib/summary-draft-stream.test.ts ui/src/lib/run-log-chunks.test.ts ui/src/context/LiveUpdatesProvider.hook.test.tsx` — 113 tests passed - `pnpm test:run` — server and UI suites passed; one CLI AWS doctor test was affected by inherited `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY`, and passed when those host credentials were removed - `pnpm exec vitest run cli/src/__tests__/secrets.test.ts` with inherited AWS credential variables removed — 8 tests passed - `pnpm build` - `pnpm check:token-gates` currently reports nine `#9627` comment references introduced by current `master`; none are in this PR diff ## Risks - Database risk is limited by incrementally ordered, idempotent migrations and migration safety checks. - Summary generation creates normal issues/runs, so misconfiguration can produce failed slots; the UI exposes retryable failure reasons and agent configuration entry points. - Streaming draft parsing depends on the documented `STATUS:` protocol; final persisted revisions remain the source of truth. - The feature is experimental, opt-in, and its built-in routine is paused with no background token spend by default. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5.3 Codex with reasoning, repository tool use, code execution, GitHub CLI, and Paperclip control-plane integration. Earlier branch commits also record Claude model co-authorship where applicable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
8b6a06ee25 |
[codex] Add built-in agents and Reflection Coach bundle (#9206)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators need first-party agent capabilities for repeatable company work, not just manually created one-off agents. > - Built-in agents need to behave like normal company-scoped agents while preserving approval gates, permissions, budgets, and audit trails. > - Reflection and coaching work also needs bundled instructions, skill content, and a routine so the feature can be installed and reset predictably. > - The API, database, UI, portability, and tests all need to agree on the built-in lifecycle from not provisioned through setup, approval, ready, paused, and reset. > - This pull request adds built-in agent provisioning and the Reflection Coach bundle end-to-end. > - The benefit is a safer first-party path for Paperclip-managed agents without bypassing the same governance model used for operator-created agents. ## Linked Issues or Issue Description No public GitHub issue was found for this exact built-in agent and Reflection Coach bundle work. Problem/motivation: - Paperclip did not have a first-party built-in agent lifecycle for product-owned agents. - Bundled agent resources such as default instructions, skills, and routines needed managed ownership and reset semantics. - Approval-gated companies needed built-in setup to preserve requested adapter, budget, manager, and permission state through board approval. - The board UI needed clear built-in badges, setup affordances, readiness state, and bundle status without exposing secrets. Proposed solution: - Add a company-scoped built-in agent registry, provisioning/reset/reconcile/status APIs, and Reflection Coach bundled resources. - Track bundled managed resources in the database with idempotent migration behavior. - Reuse existing agent approval, authorization, budget, and activity-log paths instead of creating a bypass. - Add UI setup, badges, gates, bundle panels, and route coverage for built-in agents. Duplicate search: - Searched GitHub PRs for `built-in agents Reflection Coach repo:paperclipai/paperclip`; only this PR was returned. - Searched GitHub issues for the same query; no public issues were returned. ## What Changed - Added built-in agent definitions, lifecycle state derivation, provisioning, reset, reconcile, status, and routine-control routes. - Added the `built_in_managed_resources` migration and schema exports for bundled instructions, skill, and routine ownership. - Added the Reflection Coach built-in bundle with default instructions, skill catalog content, routine template, default permissions, and managed-resource drift handling. - Added approval-aware provisioning behavior that preserves requested adapter config, budgets, manager assignment, and built-in permissions through hire approval. - Added authorization and mutation gates for built-in agent and skill changes, including consented Reflection Coach change paths. - Added UI surfaces for built-in agent setup, roster/detail badges, readiness gates, bundle status, routine controls, and route filtering. - Added company import/export and validator coverage for built-in managed resources and low-trust/red-team presets. - Addressed Greptile follow-ups for pending approval reconciliation, consent-gate error propagation, config-read authorization fallback, approval-path manager preservation, and non-model adapter provisioning. ## Verification Local verification: - `git diff --check public/master..HEAD` passed. - `pnpm check:token-gates` passed with all gates clean. - `pnpm exec vitest run ui/src/components/ConfigureBuiltInAgentModal.test.tsx` passed: 1 file, 4 tests. - `pnpm exec vitest run ui/src/components/EntityRow.test.tsx ui/src/pages/Agents.test.tsx ui/src/components/BuiltInAgentGate.test.tsx ui/src/components/ConfigureBuiltInAgentModal.test.tsx ui/src/components/BuiltInBundlePanel.test.tsx ui/src/pages/InstanceExperimentalSettings.test.tsx ui/src/pages/Routines.test.tsx` passed: 7 files, 64 tests. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/built-in-agents.test.ts src/__tests__/authorization-service.test.ts src/__tests__/company-skills-routes.test.ts` passed: 3 files, 91 tests. - `pnpm --filter @paperclipai/db check:migrations` passed. - `pnpm -r typecheck` passed after the rebase; `pnpm --filter ui typecheck` passed after the final UI review fix. Remote verification on latest head `1c61f693a4ec881d739022b0e75a8ca8bf8c2cd8`: - Merge state: `CLEAN`. - Greptile: `5/5`, zero unresolved Greptile threads. - PR check rollup: all checks successful, neutral, or skipped as expected. - Passing gates include Build, Typecheck + Release Registry, all server shards, all workspace shards, all serialized server suites, e2e, Canary Dry Run, policy, review, verify, Socket, Superagent, and Snyk. ## Risks - This adds a new managed-resource table and migration; the migration uses idempotent create/add/index guards and passed migration safety checks. - Built-in agent provisioning touches approval and authorization paths; tests cover pending approval preservation, stale retry rejection, consent gates, and config-read fallback behavior. - Reflection Coach creates managed instructions, skill, and routine resources; drift/reset behavior is covered by service tests and redacted API responses. - Non-model adapter setup now provisions a `needs_setup` built-in row before command/endpoint fields are complete; this matches the server lifecycle and is covered by the setup modal regression test. ## Model Used OpenAI Codex coding agent based on GPT-5. Exact hosted model ID, context-window size, and reasoning-mode labels are not exposed in this runtime; tool use, shell execution, GitHub CLI/API access, and local code editing were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
93206f73fa |
fix: Stop archived companies from waking agents (#7478)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - Each agent has a heartbeat scheduler that wakes it on timers and on
events; every wake spawns an adapter (Claude / Codex / …) run that bills
the operator's subscription
> - When an operator archives a company, the agents inside it remain in
invokable states — the heartbeat scheduler never consults company status
— so timers keep firing and event-driven wakes (comments, mentions,
blockers-resolved, etc.) keep cascading
> - On real deployments this silently drains the operator's
subscription: idle archived companies wake their CEOs hourly, plus any
cross-company event cascade
> - This pull request enforces "archived ⇒ never spawns a run" as a
structural invariant by guarding the wake path AND cascading agent state
on archive/reactivate
> - The benefit is that archived companies stop billing the operator,
and the UI/queue stays consistent with the invariant
## What Changed
- `server/src/services/heartbeat.ts`:
- `enqueueWakeup()` loads the company and short-circuits when status is
not `active`. Background sources (timer, automation, events) write a
`company.inactive` skipped wake and return `null`; explicit user invokes
throw a `conflict` so the UI surfaces the real reason.
- `tickTimers()` joins agents to active companies so the scheduler does
not iterate archived-company agents at all (no skip-row noise).
- `server/src/services/companies.ts`:
- `archive(id, actor?)` pauses runnable agents with `pauseReason =
"company_archived"` inside the transaction (preserving
`pending_approval`, `terminated`, and agents paused for unrelated
reasons), then cancels `queued`/`running` heartbeat runs after the
transaction commits.
- `update(id, data, actor?)` reverses the cascade only for agents whose
`pauseReason === "company_archived"` on the `archived → active`
transition; manually-paused agents stay paused.
- Both methods emit activity-log entries (`company.archived` with
`agentsPaused` + `runsCancelled`, `company.reactivated` with
`agentsRestored`) so the audit trail fires regardless of caller.
- `packages/shared/src/constants.ts` + `server/src/services/budgets.ts`:
add `company_archived` to the legal `PauseReason` union so the
restorable marker is a first-class value.
-
`packages/db/src/migrations/0094_backfill_archived_company_agent_pauses.sql`:
backfill so existing archived-company agents become `paused /
company_archived` (excludes `pending_approval`).
- `ui/src/lib/activity-format.ts`: add the `company.reactivated` label.
## Verification
- `npx vitest run src/__tests__/companies-service.test.ts` — archive
cascade, reactivate cascade, and activity-log entries (with counts) all
pass.
- `npx vitest run
src/__tests__/heartbeat-archived-company-guard.test.ts` — timer +
on-demand + event-wake paths all blocked for archived companies;
`company.inactive` skipped-wake row written; user-initiated wakes throw
`conflict`.
- `pnpm typecheck` — clean.
- Manual repro from the bug description: archive a company, wait an
interval / post a comment on one of its issues, observe zero new
heartbeat runs.
## Risks
- Migration `0094` is a single bulk UPDATE on `agents` joined to
archived `companies`. On large deployments it briefly holds row locks on
archived-company agent rows; should be quick because the predicate is
narrow (`status NOT IN (paused, terminated, pending_approval)` and
`companies.status = 'archived'`).
- New `pauseReason` value (`company_archived`) is opaque to older
clients that only know the previous union. Acceptable because the union
is read as plain text and the contract is sync'd in the same change.
- Behavior change for users: invoking an agent in an archived company
now fails with a conflict instead of silently spawning a run. Intended.
## Model Used
- Claude (Anthropic) — model `claude-opus-4-7` ("Opus 4.7"), Claude Code
CLI, with tool use (Read/Edit/Bash/Grep). No extended thinking mode.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots — N/A, no UI changes beyond an activity-log label string
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
## Related Work
Fixes #1348 (`[Bug] Archived companies still running heartbeats and
consuming tokens`).
Prior attempts and parallel work in this area:
- #1365 and #1429 by @hungdqdesign (March 2026) — both closed without
merging. Same three-layer shape (`tickTimers` / `enqueueWakeup` /
`resumeQueuedRuns` + archive-route cancellation) targeting #1348. Credit
for first publicly proposing the wake-path-guard approach.
- #5865 by @stubbi (May 2026, open) — adds the same `companies.status !=
'archived'` joins to `tickTimers`, `enqueueWakeup`, `resumeQueuedRuns`,
**and** routines `tickScheduledTriggers`, bundled with plugin-table
tenant isolation (`plugin_entities` / `plugin_job_runs` / `plugin_logs`
/ `plugin_webhook_deliveries` get a `companyId` FK with `ON DELETE
CASCADE`). This PR is narrower — it does not touch routines or plugin
tables — but adds the **archive cascade** (pause agents with
`pauseReason = "company_archived"`), the **reactivate reverse**
(un-pause only that subset), the **`company_archived` pause-reason
marker**, and a **backfill migration** for pre-existing archived
companies, which #5865 does not include. Happy to coordinate sequencing
or rebase if #5865 lands first.
|
||
|
Aron Prins
|
4811d8dd33 |
Fix wrapped company issue prefix conflicts (#6423)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - Company creation is the first control-plane object operators create, and the generated issue prefix becomes part of task identity. > - The company service already retries when a generated issue prefix collides with the `companies_issue_prefix_idx` unique constraint. > - Drizzle 0.45.x wraps PostgreSQL errors in `DrizzleQueryError`, leaving the real `23505` constraint error on the `.cause` chain. > - The existing retry detector only inspected the top-level error, so wrapped prefix collisions surfaced as 500s instead of retrying. > - This pull request walks the error cause chain for the exact prefix constraint and verifies the retry path against embedded Postgres. > - The benefit is company creation no longer fails when generated prefixes collide under Drizzle 0.45.x wrappers. ## What Changed - Walk the error `.cause` chain when detecting `companies_issue_prefix_idx` unique violations, with a cycle guard and support for `constraint` / `constraint_name` fields. - Added an embedded Postgres regression test that seeds `ARO`, creates `Aron & Sharon`, and verifies the retry produces `AROA`. - Stabilized existing async tests touched by full verification: instance sidebar plugin rendering now waits for React Query results, and Tailscale-unavailable CLI tests explicitly hide host `tailscale` detection. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/companies-service.test.ts` - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-stale-queue-invalidation.test.ts` - `pnpm --filter @paperclipai/ui exec vitest run src/components/InstanceSidebar.test.tsx` - `pnpm --filter paperclipai exec vitest run src/__tests__/network-bind.test.ts src/__tests__/onboard.test.ts` - `pnpm test:run` - `pnpm -r typecheck` - `pnpm build` ## Risks - Low runtime risk: the retry behavior only expands detection for the existing exact company issue-prefix unique constraint. - The cause-chain walk is bounded by visited objects to avoid cycles. - The sidebar and CLI changes are test-only stabilization and do not change production behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent in Codex desktop, with local shell/tool execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A: no UI behavior change) - [x] I have updated relevant documentation to reflect my changes (N/A: bug fix with no user-facing docs change) - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Closes #6350 |