Commit Graph
15 Commits
Author SHA1 Message Date
Dotta 2eb5187e43 Runner/Pi 2/5: document setup and fix CI fixtures (#14922)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The experimental Runner owns provider processes and durable
sessions.
> - Pi needs working task execution and human controls.
> - The five-PR stack must preserve changes already on master.
> - The runtime implementation has merged in #14921; this layer carries
setup documentation and a narrowly scoped CI fixture repair.
> - This PR belongs to native GitHub stack #15602, ending at #14956.

## Linked Issues or Issue Description

Refs #14436, #14631, #14743 and #14956.

Ship Pi 1.0 through the experimental Paperclip Runner. The five PRs are
#14921, #14922, #14923, #14924 and #14956. The user authorized the
complete merge after checks pass. Existing `pi_local` execution is
unchanged. Accounting and wider provider/platform qualification remain
deferred.

## What Changed

- Create the warm Pi fake sidecar and its directory with explicit
private modes, independent of the runner umask. Preserve the
qualified-asset permission guard and every warm-admission timing
assertion.

- Document explicit public Pi host setup and verified Linux companion
import for Daytona.
- Preserve historical installation and qualification checkpoints with
their original source identities.
- Use the repository's existing `EMBEDDED_POSTGRES_TEST_TIMEOUT_MS` for
the backup error fixture, which starts a real PostgreSQL cluster.
Preserve all error, archive, and retention assertions.

## Verification

- Full local `pnpm -r typecheck` passes, including Runner Rust, server
and UI. Focused integration checks pass: 194 Runner
admission/environment tests, 63 profile/credential tests with one
expected skip, 152 Dot/UI configuration tests, and Pi transcript/notice
tests.
- Full local `pnpm build` passes on the final source.
- Fresh final-source checks pass: all 698 Rust workspace tests (32
binaries), 156 credential/profile/controller tests with one expected
skip, Runner TypeScript typecheck, and 20 package/setup/sandbox tests.
- The profile-21 Pi materializer passes on the native host with the
official pinned Node 24.21.0 and its npm. It verifies all 150 locked
packages, the patched dependency and the exact closure. Setup/package
bundle tests and UI token gates pass.
- The old hashes were reproduced for all three supported targets before
calculating the patched graph. New closure hashes are darwin-arm64
`282022db10150c6632b3444df421342e7d534bdf5d5fb1097a2e79d0625a2bcf`,
darwin-x64
`64e251e19009f755c0b04f73ce2138246faab71a961b0f13d75ebfcc34bef12e`, and
linux-x64
`713b1fdff42fb56a1518bdc084f181d70bee8ebadc3e4b1d76321ed9108c8410`.
Independent native platform execution is separate from graph identity
reproduction.
- Historical cloud qualification remains unchanged: all seven core cases
pass on shipping source `10dc43c9ec65d88c2f782d62afb296d09494f215`,
harness `1a4408a48cfb5a1f094a311141c257c92cd7a893`, image
`sha256:5b3a775b383591bda1b0c1889e509acc70ce7f37c53f09733c81d59037f02280`,
and accepted Sonnet 4.6/low fixture. All 215 canonical files and all
seven cleanup checks pass independent verification. These are profile-19
results and are not relabeled as fresh profile-22 runs.
- Current Pi digest:
`sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`.
[The readiness
plan](https://github.com/paperclipai/paperclip/blob/codex/pi-production-readiness/doc/plans/2026-10-02-pi-production-readiness.md)
preserves campaign and failed-attempt provenance.
- Merge only after every PR's current-head CI and fresh review pass.
Linux CI covers the full suites, build and browser tests. The local
embedded Postgres API-authority suite cannot start on this macOS/Node 26
host, so Linux CI must confirm that suite.

### Fresh profile-22 core qualification — 2026-10-08

All seven accepted core cases pass canonically on Pi profile 22, with
`openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low
thinking. This model is a fixture; production accepts the caller's
explicit Pi provider/model.

Runtime/install source: `3241a992f2a7703e59e97ed0fd3e5d6405de4401`.
Frozen accepted harness: `1a4408a48cfb5a1f094a311141c257c92cd7a893`.
Immutable cloud image:
`ghcr.io/paperclipai/paperclip-daytona-runner@sha256:506f22db7edd78f37c0c40bec1cc084af1850455026dbf467194bfbb8fcef141`.
Pi digest:
`sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`.

[Hosted Linux image and clean-install
verification](https://github.com/paperclipai/paperclip/actions/runs/37868328023)
passes, including all 20 source-bound archives, normal CLI/Pi setup,
companion import and the production pack reader. This exact installation
source includes the latest master integration and the corrected Pi warm
instruction-root fence. Full local typecheck/build and current-head
hosted CI verify the final stack. All 13 focused real-root regressions
pass. The full local executor suite passed 662 tests; 15 database tests
could not start the Mac embedded PostgreSQL service. Hosted Linux CI
passes the full required verification and E2E checks. These fresh
results keep their own source identity; profile-19 results remain
historical.

| Core path | Canonical campaign | Retained archive SHA-256 |
| --- | --- | --- |
| File edit, validation, download and Done |
`pi-core22-replyfix-0-1791511228` | 23 files;
`a473e8603a3dd4737863291f8d3d1e392391f0b16d433c3e0e0e9d8baf7a97b0` |
| Pending question and controller restart |
`pi-core22-replyfix-1-1791511376` | 33 files;
`6b829c4eb74e1f32a89c692a4ae7130dbfc1c6d3cf13915effe2103d9e242c8e` |
| Three-turn session/process/workspace continuity |
`pi-core22-replyfix-2-1791511587` | 23 files;
`7a87021f8f9a3fdd3c58bb4467f8d82c635e3ea4795d6e75f144d9aa14818df8` |
| Four typed questions and browser reconnects |
`pi-core22-replyfix-3-1791511881` | 42 files;
`9e31755252be1f4f9cb0626c984c142d4d1ae5f5bee3a7af08444db8d12c280a` |
| Plan approval and completion | `pi-core22-replyfix-4-1791512031` | 22
files;
`a0383ce1aab38e7b5a25ce0e9dd3bebea5c037ebd96ae6b29dae19015da2ae2c` |
| Same-turn steering and permission denial |
`pi-core22-replyfix-5-1791512261` | 39 files;
`c929b8c7070f0b66aedc17e65ca46e6beab1e363926ac9f7e2a75fb250f05949` |
| Stop during pending permission | `pi-core22-replyfix-6-1791512390` |
33 files;
`7f0a58ae0f4d5bfc76149435f4e322537089c5bd16e7ffe9b5ad71f10a621a07` |

All 215 canonical files (28714587 bytes) are independently
hash-verified. All seven cleanup grades pass, with no owned runtime
process or temporary root after each case. Automatic retries are zero.
The owned cloud host stopped normally after retention. The prior
profile-22 warm attempt remains failed and separately retained: archive
SHA-256
`1e54eba5ec72b50cee1534b23d1d1d4f21a090006b8a64501ba70db972abfde5`. Its
original canonical classification is preserved. Diagnosis reproduced a
product bug comparing an agent-files root against an unset
checkpoint-only field. The fix stores the admitted physical root
separately from the adopted per-run collection capability. The real-root
regression fails before the fix and passes afterward, including
rejection of a changed physical root. Fixture, grader, model and all
seven accepted case IDs are unchanged; this fresh campaign tests
final-reply publication after file registration first. The intermediate
restart attempt also remains failed and retained: archive SHA-256
`5dcaefdf1d17cf4cd54fd4cf810f45e736667392339b8ce7caf08bb4e225277f`. Its
original canonical classification is preserved. Pi resumed, wrote the
verified answer and completed its task; exact runner suspension was
proven, but idle stop consumed about 5.2s and left under 3s for the
drain acknowledgement. The Pi-only default shutdown grace is now 15s,
preserving a full 5s drain round trip and a finite suspension reserve.
Explicit caller deadlines, other provider defaults, literal drain
receipts and exact suspension identity checks remain unchanged. The
timing regression fails before this correction and passes afterward; all
18 focused settlement tests and Runner typecheck pass. The final-source
file attempt is also preserved as failed (`candidate_failure`), archive
SHA-256
`db6767b6773ea618997927ac77bdb005a5ac81492c7b9c0ffbc900449f829bc9`.
Native edit, validation, exact downloadable artifact and Done/succeeded
all passed, and the exact final reply was durably recorded. A workspace
recovery owner completed before the live heartbeat reached presentation,
leaving that reply absent from task chat. Recovery now materializes only
a completed final reply from the accepted turn of an ordinary internal
Done task, preserving issue/run/contract binding, suppression,
external-chat authorization and same-run deduplication. The database
regression covers the generated file-preparation receipt, suppression,
unapproved external continuation and replay. Server typecheck and all 49
response-selection tests pass; hosted Linux verifies the database
regression because embedded PostgreSQL cannot start on this Mac.

The delayed-final-answer database regression passes on [the final
root-source Linux server
shard](https://github.com/paperclipai/paperclip/actions/runs/37868262553/job/113628594152),
alongside 1,108 passing tests. The first root Runner shard had one
unchanged durable-resume test exceed its 5-second timeout; the identical
top-source shard and the isolated exact test passed. One rerun of that
failed job and its required aggregate passed without source or test
changes. The original failed job log and the single-rerun receipt remain
retained.

### October 9 merge verification

Current merge head: `18744bf27490d6f064f9d5bac2c2af202c590a99`. All
current-head checks pass, including `ci / verify` and `ci / e2e`;
exact-head Greptile review is 5/5 with no unresolved threads. Current
master conflicts are resolved. The user authorized the maintainer
override of the code-owner review gate after these checks. The seven
retained live core cases remain bound to source
`3241a992f2a7703e59e97ed0fd3e5d6405de4401` and its recorded cloud image.

## Risks

- The security correction changes the dependency closure and profile
identity. Old sessions must reopen on the new profile. Exact identities
and credential bindings fail closed.
- The runner remains experimental and requires explicit selection.
Legacy Pi Local is unchanged. Caller model IDs pass through; the E2E
model is a fixture.
- Accounting and the broad platform/provider matrix remain deferred.
This merge does not publish a release or deploy a service.

## Model Used

OpenAI GPT-6 through Codex assisted with reasoning, repository
inspection, editing and tool use. The exact serving ID and context
window are not exposed in this session. Final live qualification uses Pi
1.0.0 with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed
low thinking.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-10-09 10:19:13 -05:00
Devin FoleyandPaperclip 9b624a110e fix: checkpoint database backups before idle sleep (#15629)
## Thinking Path

> - Paperclip coordinates agent work and preserves company state.
> - Hosted instances can stop compute when admission and all work
sources are quiet.
> - The periodic database backup timer currently blocks every automatic
sleep.
> - Disabling backups would remove recovery points.
> - This change offers an explicit final-backup checkpoint before sleep.
> - A verified archive and restart marker preserve recovery while
compute is stopped.

## Linked Issues or Issue Description

**What happened?**

An otherwise idle instance with automatic backups enabled always reports
background work. Operators cannot reclaim idle compute without disabling
those backups. The work inventory also queries the database before
checking known local blockers.

**Expected behavior**

An operator can enable checkpoint mode. The server may authorize sleep
only after all other work is quiet and a fresh backup is verified under
the same owned hold. Backups resume on restart with the existing
retention policy.

**Steps to reproduce**

Acquire a bounded idle drain on an instance with no application work and
automatic backups enabled. Read its owned idle safety report. It reports
present even when all other work is clear. With this change and
`PAPERCLIP_DB_BACKUP_IDLE_CHECKPOINT_ENABLED=1`, a successful final
backup can permit sleep; backup failures still refuse it.

**Paperclip version or commit**

Reproduced from master at 4265cb3a2b.

**Deployment mode**

Hosted single-process instances with persistent backup storage and
serialized sleep/wake operations.

Searched open backup and sleep PRs and issues. Related: #15522 and
#15599 establish owned idle holds and plugin drains. #15358 proposes
general backup health/retention validation; this change verifies only
opted-in sleep checkpoints and their restart catch-up.

## What Changed

- Add an off-by-default checkpoint flag. Keep automatic backups enabled.
- Reject known local blockers before database work.
- Finish a new backup during each owned safety check, including final
revalidation.
- Verify the complete gzip archive and sync its file and directory
before pruning older recovery points.
- Use unique checkpoint filenames so repeated checks cannot replace an
earlier verified archive.
- Keep backup work counted through dump, validation and persistence.
Changed owners, expired holds, concurrent work and errors refuse sleep.
- Persist a restart marker before the dump. Attempt a fresh backup on
restart before idle eligibility, then resume the normal timer. Only a
verified fresh backup clears the marker.
- Restrict managed checkpoint reads to verified Cloud control actors.
Tenant instance-admin elevation does not grant backup authority.
- Keep archive verification outside engine fallback so failures retain
their original cause.
- Document that recurring duplicate archives pause while asleep. The
final checkpoint and historical archives remain on persistent storage;
retention does not prune while asleep.

## Verification

- Focused idle, spool, backup and route checks: 137 tests passed,
including restoration into a separate PostgreSQL database.
- Backup library and checkpoint tests: 15 passed, including rejection
before historical backups can be pruned.
- `pnpm -r typecheck` and `pnpm build` passed. Follow-up server
typecheck and database build cover the verification callback.
- Review follow-up: 51 backup/restore/startup tests, 122 route/safety
tests, 32 signed-control tests, and server typecheck passed. Startup
cases exercise both flag states and retry after failure. The final
test-cleanup adjustment passes all 35 startup tests.
- The full local run hit four ancestor-directory skill-fixture failures
and one tool-route failure. All five passed from an isolated checkout at
949210 (3 targeted chat/tool checks and the complete 39-test email
suite). The redundant local run was stopped after final-head hosted CI
passed. No full local pass is claimed.
- Added-line secret/private-reference review and `git diff --check`
passed.
- [Final-head hosted
CI](https://github.com/paperclipai/paperclip/actions/runs/37853380365)
passed, including typecheck, build, all general and serialized test
shards, all eight E2E shards and the canary dry run. Apex is 5/5 on
5b1de388ef, with no new findings and all review threads resolved. The
branch is conflict-free. Live staging activation awaits merge and a
canonical image; no production configuration or deployment changed.

## Risks

- The flag defaults off. This is for a single process whose host
controls every writer and preserves its backup volume across sleep.
Independent database writers require a different backup owner.
- Safety reads perform a backup only after a valid owned drain and all
other checks pass. Slow backups can outlive the caller timeout; they
remain counted until their work settles and cannot authorize sleep after
hold expiry.
- Initial and final reads each take a fresh backup. Hosts must allow
sufficient request time while retaining shutdown headroom.
- Retention settings are unchanged. Sleeping instances stop producing
duplicate hourly archives and retain their last checkpoint. Filesystem
data, encryption keys and off-provider disaster recovery remain separate
responsibilities.
- No schema migration. Disable the checkpoint flag to restore the
blanket backup blocker; automatic backups and an existing restart marker
remain active.

## Model Used

OpenAI Codex, GPT-6. Used reasoning, repository inspection, code editing
and command execution. The runtime did not expose an exact model
revision or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the bug report template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
identifier
- [x] Focused and isolated local checks pass; final-head hosted CI is
green (local full-run limitation documented above)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all review comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-08 15:58:55 -07:00
Nicky LeachandPaperclip 1cebdd4c4f Add plugin lifecycle delivery and an initial resource baseline (#15343)
## Thinking Path

> - Paperclip manages agents and projects for work.
> - Their lifecycle changes already commit to a durable journal.
> - Plugins need to read these records and track completed work.
> - Existing resources need a one-time baseline before delivery is
enabled.
> - Worker crashes must leave unfinished records available for retry.
> - Each plugin needs its own progress and company access checks.
> - This PR adds a pull inbox through the existing plugin SDK and job
system.

## Linked Issues or Issue Description

**Problem or motivation**

Plugins cannot consume the durable resource lifecycle journal.
In-process notifications can disappear during a restart and cannot
record successful completion. Resource plugins need retries, company
boundaries, and ordered transitions.

**Proposed solution**

Add `ctx.events.listLifecycle(companyId, limit?, afterId?)` and
`ctx.events.acknowledgeLifecycle(companyId, eventId)` under
`events.subscribe`. Seed a one-time baseline from current resource
state. Deliver creation first, then the remaining transitions in ID
order. Store acknowledgments for each plugin. Use existing plugin jobs
to poll configured companies.

**Alternatives considered**

A global sequence cursor can skip lower IDs that commit later.
Fire-and-forget subscriptions cannot record completion. A new dispatcher
is unnecessary because plugin jobs support polling. Consumers serialize
polling and use provider idempotency keys.

**Roadmap alignment**

This extends the existing plugin system and builds on #15280 and #15306.
Searches found no duplicate resource inbox work. Related #13306 exposes
decision events on the in-process bus. This PR includes the initial
journal baseline. Provider provisioning remains separate work.

## What Changed

- Add company-scoped lifecycle reads and acknowledgments to the SDK and
worker RPC host.
- Gate both methods by capability, invocation or proactive company
scope, plugin readiness, and company enablement.
- Seed hired agents and all projects once. Preserve paused/terminated
agent state and archived project state. Keep pending hires behind
approval.
- Preserve existing history and deliver backfilled creation before
partial transition histories.
- Deliver project archive events from #15371 through the SDK. Seed
archive intents for archived projects and update intents for active
projects with a partial archive history.
- Store acknowledgments per plugin and reject acknowledgments that skip
earlier resource events.
- Page past failed resources while retaining their pending records.
Reset the page cursor each sweep to include late commits.
- Add acknowledgment storage, an index for resource ordering, tests, and
authoring guidance.
- Preserve native identity definitions and sequence progress in
JavaScript backups. Repair journal ID generators lost by older backups
before seeding the baseline, without changing existing IDs.

## Verification

- `pnpm -r typecheck` passed after the final origin/master rebase.
- `pnpm build` passed before the final metadata rebase. The final CI
build also passed.
- All 46 focused lifecycle, SDK harness/RPC, migration snapshot, and
legacy restore checks passed again with migration 0309.
- Checks cover retries, per-plugin progress, resource order,
capability/company boundaries, baseline idempotency, approval gates,
archive delivery, restored projects with partial histories, and atomic
migration rollback.
- Full CI passed on final commit `0e674b87fc`. One unrelated Cursor
fixture hit a 10-second timeout; it passed locally in under one second,
and the failed server shard passed on retry.
- Greptile scored the final commit 5/5 with no unresolved review
threads.
- The branch is rebased onto origin/master and is conflict-free.
- Earlier full local runs were stopped as scope changed. CI ran the
complete repository suite.
- `git diff --check` and a local secret/PII scan passed.

## Risks


- Apply migration `0309_loving_the_hood.sql` before starting the new
server. It creates acknowledgment storage and an index, then seeds the
baseline in the same transaction. Agent/project writes wait for the
migration to commit. Keep these writes quiesced through the migration
and activation of the new capture-capable server; do not resume an older
runtime that lacks capture after the baseline.
- The baseline records current desired state, not historical
transitions. It includes archived projects and terminated-agent cleanup
intents. It runs once with the delivery migration; no later or runtime
journal backfill is planned.
- Delivery is at least once. Concurrent reads can repeat an event.
Consumers must serialize polling, use stable company/event idempotency
keys, and acknowledge successful operations only.
- Reset `afterId` at the start of every polling sweep. It is a page
cursor, not a persisted high-water mark.
- Deleting a plugin removes its acknowledgment records. A new
installation may replay existing journal events.
- Records contain identity and action. Consumers must load current
authorized data before acting. Cleanup and retention policy belong to
the provider plugin.
- Provider calls, VM/volume provisioning, and journal retention are
outside this PR.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, repository inspection,
code execution, and tool use. The exact deployment model ID and context
window are not exposed in this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-06 12:27:09 -07:00
DottaandPaperclip b3343dbd64 feat(connections): add self-serve intent runtime (#12345)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agents need a governed way to request app connections during issue
work.
> - The catalog now describes the available providers and setup methods.
> - A request must become a durable, company-scoped intent before an
operator acts on it.
> - This pull request adds that intent runtime across server, agent,
CLI, and shared contracts.
> - The benefit is a safe bridge from agent need to operator-approved
setup.

## Linked Issues or Issue Description

Refs #11965

This is stack 7 of 11. It depends on stack 6 and replaces another
reviewable part of #11965.

## What Changed

- Add connection intent types, validation, service logic, and routes.
- Add agent runtime tools and CLI support for connection requests.
- Add issue-thread interaction support for connection intents.
- Add runtime, route, adapter, and contract tests.
- Hold the final resolved-continuation row lock through asynchronous
adapter preparation until an actual process spawn, so parking or
reassignment cannot cross that boundary.
- Report Hermes Gateway's first remote run request through the shared
dispatch hook so the resolved-intent lock is released at the true
dispatch boundary.
- Revalidate the addressed user's live non-viewer membership and
connection-management authority for every intent mutation, including
OAuth completion.

## Verification

- `pnpm --filter @paperclipai/server typecheck`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/tool-access-service.test.ts`
- Result: 176 tests passed.
- `pnpm build`
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/heartbeat-stale-queue-invalidation.test.ts` (32 passed;
includes non-process dispatch lock-release coverage)
- `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/connection-intents-service.test.ts -t
"addressed-user mutation"` (1 passed)
- `pnpm exec vitest run --project @paperclipai/server
server/src/__tests__/tool-access-service.test.ts -t "binds OAuth
callback completion to the initiating board session"` (1 passed)
- `pnpm --filter @paperclipai/hermes-paperclip-adapter test --
src/gateway/server/execute.test.ts` (23 passed; includes dispatch-hook
ordering and exactly-once coverage)
- `pnpm --filter @paperclipai/hermes-paperclip-adapter typecheck`

## Risks

- A malformed intent could create an unusable operator request.
- Validators and company checks reject invalid or cross-company
requests.
- The final continuation gate holds the issue row lock through adapter
preparation until process or remote dispatch; later operator changes use
the normal active-run interruption path.
- The change does not add a database migration.

> I checked `ROADMAP.md`. This stack continues the existing app
connection work from #11965 and does not duplicate another planned item.

## Model Used

OpenAI Codex, GPT-5. The runtime model ID and context window were not
exposed. The model used reasoning, tool use, and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked the public source pull request with `Refs #`
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-29 12:08:34 -05:00
DottaandPaperclip 4d2af732ae feat(runner): add native persistence contracts (#12169)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent runs need durable records so Paperclip can explain results and
final status changes.
> - The current heartbeat tables support direct adapters, but they do
not model native runner evidence.
> - The runner transport and server coordinator must share a strict
finalization contract before they write production data.
> - This pull request adds that contract and its additive database
boundary.
> - It does not select the Paperclip Runner or change any existing
adapter execution path.
> - The benefit is a reviewable persistence layer that preserves all
current behavior and supports later guarded integration.

## Linked Issues or Issue Description

Refs #11962

Refs #12129

## What Changed

- Add native run result, finalization, completion, assessment, status
decision, and status effect tables.
- Add inert native metadata to heartbeat runs and events. Keep `legacy`
as the default runtime mode.
- Bind each evidence relationship to one company, issue, run, contract,
result, assessment, and decision with composite constraints.
- Add a strict `paperclip.native_finalization.v1` shared type and
validator.
- Preserve database functions, triggers, and the unique indexes required
by foreign keys in JavaScript backups.
- Add migration, backup, mixed-owner denial, validator, and
direct-adapter compatibility tests.
- Document the new records and their ownership rules.

## Verification

- Run `pnpm -r typecheck`.
- Run `pnpm build`.
- Run `pnpm db:generate`. The schema output and migration safety checks
remain current.
- Run
`PAPERCLIP_PSQL_PATH=/Applications/Postgres.app/Contents/Versions/latest/bin/psql
pnpm exec vitest run
packages/shared/src/validators/native-finalization.test.ts
packages/db/src/client.test.ts packages/db/src/backup-lib.test.ts
server/src/__tests__/heartbeat-workspace-busy.test.ts
server/src/__tests__/heartbeat-comment-wake-batching.test.ts`. All 52
tests pass.
- The full local `pnpm test:run` run completed 4,688 tests. It found 30
existing macOS test-environment failures. A serial rerun with the
canonical `/private/tmp` path reduced those failures to six existing
listener-diagnostics and skill-browser cases. None of those suites use
files in this change.
- The full Linux GitHub Actions matrix passes. This includes all
general-server, serialized-server, workspace, browser, build, typecheck,
canary, and aggregate verification jobs.
- Greptile passes at 5/5. Contributor trust, Superagent, Socket, and
Snyk pass with no finding from this change.
- Storybook visual regression skips by path because this pull request
has no UI or Storybook change.
- Confirm that the diff contains 25 files. Confirm that it contains no
workflow or `pnpm-lock.yaml` changes.

## Risks

- The migration adds tables, columns, indexes, a function, a trigger,
and ownership constraints. It does not remove or rename existing data.
- Composite foreign keys reject mixed-company, mixed-issue, and
mixed-run evidence even when each ID exists.
- The status-version trigger runs only when an issue status changes.
Backup tests confirm that restore retains this trigger and its
dependencies.
- Native source identifiers are unique when present. Legacy event rows
remain unchanged.
- This change does not add a unique run sequence constraint. The later
native writer must allocate its sequence atomically before that
invariant can be safe.
- Existing adapters keep their current execution and finalization paths.
New heartbeat runs default to `legacy` mode.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex with GPT-5. The exact deployment ID and context-window
size are not exposed. The model used agentic reasoning, repository
tools, code execution, and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-25 13:08:39 -05:00
LeonSGPandAndrew Aymeloglu b083b173ea fix(db): use calendar month retention for backups (#3718)
## Thinking Path

> - Paperclip includes built-in database backup and retention behavior
as part of its operational reliability surface.
> - The retention implementation lives in
`packages/db/src/backup-lib.ts`.
> - Monthly pruning is intended to keep the newest backup per retained
calendar month.
> - The current cutoff uses a fixed 30-day approximation, which deletes
January backups too early when the current month has 31 days.
> - This pull request switches the monthly cutoff to calendar-month
boundaries instead of a fixed day multiplier.
> - The benefit is that monthly retention now matches the documented
calendar-month behavior and does not prune valid backups prematurely.

## Linked Issues or Issue Description

Fixes #3713

Two other pull requests implemented the same fix and have already been
closed as duplicates of this one:

- #3798 — same author's later take. Anchors the cutoff to the 1st
correctly, but mutates the date in local time and leaves `monthKey` on
local time, and unit-tests the helper in isolation rather than end to
end.
- #4031 — decrements the month without anchoring to the 1st, so
partial-month drift and a `setMonth` day-overflow edge case remain. No
tests.

## What Changed

- Replaced the fixed `30 * 24h` monthly retention cutoff with a
calendar-month cutoff anchored to the first day of the earliest retained
month.
- Added a regression test that freezes `Date.now()` at March 31 and
proves the newest January backup is retained when `monthlyMonths=2`.
- Kept the rest of the pruning behavior unchanged: daily and weekly
tiers still use their existing windows and bucket selection rules.

## Verification

- `pnpm --filter @paperclipai/db exec vitest run src/backup-lib.test.ts`
- `pnpm --filter @paperclipai/db exec tsc --noEmit`
- Note: `pnpm --filter @paperclipai/db typecheck` hits an
environment-specific `check:migrations` runtime failure on this host
(`Cannot find module ./cjs/index.cjs from ` via Bun), so I used plain
`tsc --noEmit` to validate the code changes themselves.

## Risks

- Low risk. This only changes the monthly retention cutoff calculation.
- The pruning buckets are still selected the same way; the fix only
widens the retained month window to align with calendar-month semantics.

## Model Used

- OpenAI Codex GPT-5 coding agent with terminal tool use and code
execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---

<sub>Edited by commitperclip during triage: added the **Linked Issues**
section (`Fixes #3713`) and the duplicate-PR search line the PR template
requires. The duplicate search was performed by the triage pipeline,
which grouped this PR with #3798 and #4031 and selected this one as the
canonical fix. Everything else is the author's original
description.</sub>

---------

Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>
2026-07-24 13:12:43 -05:00
DottaandPaperclip 1de0a3bb1e feat(mcp) [split 2/8]: add governed access contracts (#9557)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Governed MCP access spans contracts, runtime enforcement, adapters,
UI surfaces, and operator verification
> - The parity reference PR #9534 is too large for effective automated
or human review
> - The feature therefore needs a linear stack whose individual diffs
stay below the 100-file review limit
> - This pull request is split 2/8 and focuses on database schema and
shared governance contracts
> - The benefit is a standalone, testable review boundary while
preserving byte-for-byte parity at the top of the stack

## Linked Issues or Issue Description

- Related parity reference: #9534
- Problem: The governed access model needs additive persistence and
synchronized shared types before server enforcement can compile.
- Proposed solution: Adds migrations 0148–0169, tool-access and Smoke
Lab schema, shared types/validators/gallery helpers, and the minimal
compile-required contract consumers identified by boundary testing.
- Alternatives considered: keeping #9534 as one 403-file review, or
rewriting the feature to manufacture seams; both were rejected in favor
of path extraction plus compile-driven boundary moves.
- Roadmap alignment: this advances the existing governed MCP/tool-access
work already represented by #9534; it does not introduce a separate
roadmap initiative.
- Stack position: base branch is `pap10341-split/01-demo-servers`.
- Merge policy: merge bottom-up, in order, only after the complete
eight-PR stack has been reviewed and the top-of-stack parity gate
remains empty.
- Requested review: QA for migrations/validators; Greptile on every PR.

## What Changed

- Adds migrations 0148–0169, tool-access and Smoke Lab schema, shared
types/validators/gallery helpers, and the minimal compile-required
contract consumers identified by boundary testing.
- Keeps this PR below 100 changed files and independently typecheckable.
- Preserves the final tree from #9534 when combined with the other seven
stack levels.

## Verification

- `pnpm typecheck` — passed, including migration numbering and safety
checks
- `pnpm --filter @paperclipai/db test` — passed
- `pnpm --filter @paperclipai/shared test` — passed

## Risks

- Migration or contract mistakes could affect every upper layer; all
migrations are additive/idempotent and compile consumers are included in
this boundary.
- Stack risk: merging out of order can expose incomplete layers;
mitigate by following the documented bottom-up merge policy.
- Parity risk: later edits to an intermediate branch can drift from
#9534; mitigate by re-running the empty top-of-stack diff before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context
window; medium reasoning with repository, shell, Git, GitHub CLI, and
code-execution tools enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] Internal references are omitted except the execution-plan link
explicitly required for this coordinated split stack
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge


## Stack Coordination

- Internal execution plan:
[PAP-13874](/PAP/issues/PAP-13874#document-plan)
- Parity reference: #9534
- Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563
- Merge bottom-up only after full-stack review and an empty parity diff
at #9563.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-14 12:57:20 -05:00
DottaandPaperclip 8b6a06ee25 [codex] Add built-in agents and Reflection Coach bundle (#9206)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Operators need first-party agent capabilities for repeatable company
work, not just manually created one-off agents.
> - Built-in agents need to behave like normal company-scoped agents
while preserving approval gates, permissions, budgets, and audit trails.
> - Reflection and coaching work also needs bundled instructions, skill
content, and a routine so the feature can be installed and reset
predictably.
> - The API, database, UI, portability, and tests all need to agree on
the built-in lifecycle from not provisioned through setup, approval,
ready, paused, and reset.
> - This pull request adds built-in agent provisioning and the
Reflection Coach bundle end-to-end.
> - The benefit is a safer first-party path for Paperclip-managed agents
without bypassing the same governance model used for operator-created
agents.

## Linked Issues or Issue Description

No public GitHub issue was found for this exact built-in agent and
Reflection Coach bundle work.

Problem/motivation:
- Paperclip did not have a first-party built-in agent lifecycle for
product-owned agents.
- Bundled agent resources such as default instructions, skills, and
routines needed managed ownership and reset semantics.
- Approval-gated companies needed built-in setup to preserve requested
adapter, budget, manager, and permission state through board approval.
- The board UI needed clear built-in badges, setup affordances,
readiness state, and bundle status without exposing secrets.

Proposed solution:
- Add a company-scoped built-in agent registry,
provisioning/reset/reconcile/status APIs, and Reflection Coach bundled
resources.
- Track bundled managed resources in the database with idempotent
migration behavior.
- Reuse existing agent approval, authorization, budget, and activity-log
paths instead of creating a bypass.
- Add UI setup, badges, gates, bundle panels, and route coverage for
built-in agents.

Duplicate search:
- Searched GitHub PRs for `built-in agents Reflection Coach
repo:paperclipai/paperclip`; only this PR was returned.
- Searched GitHub issues for the same query; no public issues were
returned.

## What Changed

- Added built-in agent definitions, lifecycle state derivation,
provisioning, reset, reconcile, status, and routine-control routes.
- Added the `built_in_managed_resources` migration and schema exports
for bundled instructions, skill, and routine ownership.
- Added the Reflection Coach built-in bundle with default instructions,
skill catalog content, routine template, default permissions, and
managed-resource drift handling.
- Added approval-aware provisioning behavior that preserves requested
adapter config, budgets, manager assignment, and built-in permissions
through hire approval.
- Added authorization and mutation gates for built-in agent and skill
changes, including consented Reflection Coach change paths.
- Added UI surfaces for built-in agent setup, roster/detail badges,
readiness gates, bundle status, routine controls, and route filtering.
- Added company import/export and validator coverage for built-in
managed resources and low-trust/red-team presets.
- Addressed Greptile follow-ups for pending approval reconciliation,
consent-gate error propagation, config-read authorization fallback,
approval-path manager preservation, and non-model adapter provisioning.

## Verification

Local verification:
- `git diff --check public/master..HEAD` passed.
- `pnpm check:token-gates` passed with all gates clean.
- `pnpm exec vitest run
ui/src/components/ConfigureBuiltInAgentModal.test.tsx` passed: 1 file, 4
tests.
- `pnpm exec vitest run ui/src/components/EntityRow.test.tsx
ui/src/pages/Agents.test.tsx ui/src/components/BuiltInAgentGate.test.tsx
ui/src/components/ConfigureBuiltInAgentModal.test.tsx
ui/src/components/BuiltInBundlePanel.test.tsx
ui/src/pages/InstanceExperimentalSettings.test.tsx
ui/src/pages/Routines.test.tsx` passed: 7 files, 64 tests.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/built-in-agents.test.ts
src/__tests__/authorization-service.test.ts
src/__tests__/company-skills-routes.test.ts` passed: 3 files, 91 tests.
- `pnpm --filter @paperclipai/db check:migrations` passed.
- `pnpm -r typecheck` passed after the rebase; `pnpm --filter ui
typecheck` passed after the final UI review fix.

Remote verification on latest head
`1c61f693a4ec881d739022b0e75a8ca8bf8c2cd8`:
- Merge state: `CLEAN`.
- Greptile: `5/5`, zero unresolved Greptile threads.
- PR check rollup: all checks successful, neutral, or skipped as
expected.
- Passing gates include Build, Typecheck + Release Registry, all server
shards, all workspace shards, all serialized server suites, e2e, Canary
Dry Run, policy, review, verify, Socket, Superagent, and Snyk.

## Risks

- This adds a new managed-resource table and migration; the migration
uses idempotent create/add/index guards and passed migration safety
checks.
- Built-in agent provisioning touches approval and authorization paths;
tests cover pending approval preservation, stale retry rejection,
consent gates, and config-read fallback behavior.
- Reflection Coach creates managed instructions, skill, and routine
resources; drift/reset behavior is covered by service tests and redacted
API responses.
- Non-model adapter setup now provisions a `needs_setup` built-in row
before command/endpoint fields are complete; this matches the server
lifecycle and is covered by the setup modal regression test.

## Model Used

OpenAI Codex coding agent based on GPT-5. Exact hosted model ID,
context-window size, and reasoning-mode labels are not exposed in this
runtime; tool use, shell execution, GitHub CLI/API access, and local
code editing were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-09 16:29:30 -05:00
DottaandPaperclip c91a062326 [codex] Runtime control-plane fixes (#6380)
## Thinking Path

> - Paperclip orchestrates AI agents through a server-side control plane
> - That control plane depends on reliable issue state transitions,
plugin lifecycle behavior, import limits, and startup/shutdown handling
> - Several small runtime fixes had accumulated on the working branch
and were mixed with larger feature work
> - Keeping them separate makes the correctness fixes reviewable and
mergeable without waiting for cloud-sync UI work
> - This pull request groups the server/runtime control-plane fixes into
one standalone branch
> - The benefit is a tighter, safer runtime baseline for retries,
imports, plugin migrations, feedback flushing, and trusted cloud import
handling

## What Changed

- Fixed updated issue list pagination sorting and scheduled retry
comment handling.
- Re-applied pending plugin migrations during hot reload and fixed
plugin-schema worktree seed restore.
- Hardened public tenant DB startup, portable import body limits,
trusted cloud import errors, and trusted cloud tenant import mutation
access.
- Expired stale request confirmations after user comments.
- Added feedback export shutdown hardening so database-unavailable flush
loops stop cleanly.
- Guarded plugin worker `error` event emission when no listener is
registered.

## Verification

- `pnpm install --frozen-lockfile --ignore-scripts`
- `pnpm --filter @paperclipai/plugin-sdk build`
- `npm run install --prefix
node_modules/.pnpm/sqlite3@5.1.7/node_modules/sqlite3`
- `pnpm exec vitest run server/src/__tests__/issues-service.test.ts
server/src/__tests__/plugin-lifecycle-restart.test.ts
server/src/__tests__/server-startup-feedback-export.test.ts
server/src/__tests__/issue-comment-reopen-routes.test.ts
server/src/__tests__/issue-thread-interactions-service.test.ts
server/src/__tests__/issue-thread-interaction-routes.test.ts
server/src/__tests__/body-limits.test.ts
server/src/__tests__/feedback-flush-controller.test.ts
server/src/__tests__/error-handler.test.ts
server/src/__tests__/board-mutation-guard.test.ts
packages/db/src/backup-lib.test.ts` initially exposed local setup issues
and two 5s test timeouts.
- Rerun after local prereq build: `pnpm exec vitest run --testTimeout
15000 server/src/__tests__/issue-comment-reopen-routes.test.ts
server/src/__tests__/issue-thread-interaction-routes.test.ts
server/src/__tests__/feedback-flush-controller.test.ts
server/src/__tests__/server-startup-feedback-export.test.ts` passed.
- Some embedded Postgres-backed tests skipped on this host because local
Postgres init was unavailable.

## Risks

- Runtime-touching branch: startup/shutdown and issue interaction
behavior should be reviewed carefully.
- The feedback export change disables repeated flush attempts only for
database connection-refused failures; other upload failures still log
normally.
- The plugin worker error guard avoids process crashes from unhandled
EventEmitter errors but may hide errors from code paths that expected an
emitted listener.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5-based coding agent with local shell/git/tool use.
Exact hosted model ID and context-window size are not exposed by the
local Paperclip adapter runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-20 10:37:11 -05:00
DottaandPaperclip cd606563f6 Expand database backups to non-system schemas (#4859)
## Thinking Path

> - Paperclip is the control plane for autonomous AI companies.
> - Reliable backups are part of operating that control plane safely.
> - The previous backup path was public-schema oriented and did not
clearly cover plugin-owned schemas or migration history.
> - Paperclip now has plugin database namespaces and Drizzle migration
state that must survive backup/restore.
> - This pull request expands logical database backups to non-system
schemas and documents the backup boundary.
> - The benefit is safer restore behavior for core and plugin-owned
database state without implying full filesystem disaster recovery.

## What Changed

- Include non-system database schemas in JavaScript and pg_dump backup
paths.
- Preserve enum, table, sequence, index, constraint, migration, and
plugin-schema objects across backup/restore.
- Add restore coverage for plugin-owned schemas and Drizzle migration
history.
- Clarify docs that DB backups are logical database backups, not full
instance filesystem backups.

## Verification

- `pnpm install --frozen-lockfile`
- `pnpm exec vitest run packages/db/src/backup-lib.test.ts`
- Result: 1 test file passed, 4 tests passed.
- Confirmed this PR does not include `pnpm-lock.yaml` or
`.github/workflows/*` changes.

## Risks

- Medium: backup generation touches schema discovery and restore
ordering, so unusual database objects may need additional coverage
later.
- No migrations are included.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5 coding agent, tool use enabled, medium reasoning
effort. Exact hosted context-window details are not exposed in this
runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

Note: no UI changes are included in this PR, so screenshots are not
applicable.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-30 12:54:35 -05:00
DottaandPaperclip e89d3f7e11 [codex] Add backup endpoint and dev runtime hardening (#4087)
## Thinking Path

> - Paperclip is a local-first control plane for AI-agent companies.
> - Operators need predictable local dev behavior, recoverable instance
data, and scripts that do not churn the running app.
> - Several accumulated changes improve backup streaming, dev-server
health, static UI caching/logging, diagnostic-file ignores, and instance
isolation.
> - These are operational improvements that can land independently from
product UI work.
> - This pull request groups the dev-infra and backup changes from the
split branch into one standalone branch.
> - The benefit is safer local operation, easier manual backups, less
noisy dev output, and less cross-instance auth leakage.

## What Changed

- Added a manual instance database backup endpoint and route tests.
- Streamed backup/restore handling to avoid materializing large payloads
at once.
- Reduced dev static UI log/cache churn and ignored Node diagnostic
report captures.
- Added guarded dev auto-restart health polling coverage.
- Preserved worktree config during provisioning and scoped auth cookies
by instance.
- Added a Discord daily digest helper script and environment
documentation.
- Hardened adapter-route and startup feedback export tests around the
changed infrastructure.

## Verification

- `pnpm install --frozen-lockfile`
- `pnpm exec vitest run packages/db/src/backup-lib.test.ts
server/src/__tests__/instance-database-backups-routes.test.ts
server/src/__tests__/server-startup-feedback-export.test.ts
server/src/__tests__/adapter-routes.test.ts
server/src/__tests__/dev-runner-paths.test.ts
server/src/__tests__/health-dev-server-token.test.ts
server/src/__tests__/http-log-policy.test.ts
server/src/__tests__/vite-html-renderer.test.ts
server/src/__tests__/workspace-runtime.test.ts
server/src/__tests__/better-auth.test.ts`
- Split integration check: merged after the runtime/governance branch
and before UI branches with no merge conflicts.
- Confirmed this branch does not include `pnpm-lock.yaml`.

## Risks

- Medium risk: touches server startup, backup streaming, auth cookie
naming, dev health checks, and worktree provisioning.
- Backup endpoint behavior depends on existing board/admin access
controls and database backup helpers.
- No database migrations are included.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5.4 tool-enabled coding model, agentic
code-editing/runtime with local shell and GitHub CLI access; exact
context window and reasoning mode are not exposed by the Paperclip
harness.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 06:08:55 -05:00
Aron Prins fcbae62baf feat(backups): tiered daily/weekly/monthly retention with UI controls
Replace single retentionDays with a three-tier BackupRetentionPolicy:
- Daily: keep all backups (presets: 3, 7, 14 days; default 7)
- Weekly: keep one per calendar week (presets: 1, 2, 4 weeks; default 4)
- Monthly: keep one per calendar month (presets: 1, 3, 6 months; default 1)

Pruning sorts backups newest-first and applies each tier's cutoff,
keeping only the newest entry per ISO week/month bucket. The Instance
Settings General page now shows three preset selectors (no icon, matches
existing page design). Remove Database icon import.
2026-04-08 14:40:05 +02:00
Aron Prins cc44d309c0 feat(backups): gzip compress backups and add retention config to Instance Settings
Compress database backups with gzip (.sql.gz), reducing file size ~83%.
Add backup retention configuration to Instance Settings UI with preset
options (7 days, 2 weeks, 1 month). The backup scheduler now reads
retention from the database on each tick so changes take effect without
restart. Default retention changed from 30 to 7 days.
2026-04-08 14:40:05 +02:00
dottaandPaperclip 0a9a8b5a44 Limit isolated workspace memory spikes
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-06 21:23:21 -05:00
dottaandPaperclip bd6d07d0b4 fix(ui): polish issue detail timelines and attachments
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 11:51:40 -05:00