Files
PaperClipAI/doc/database-backups.md
T
DottaandClaude Fable 5.2 ac114d9479 refactor: cut DEVELOPING.md to a setup and workflow guide (#21670)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - `AGENTS.md` tells every contributor and agent to read
`doc/DEVELOPING.md` for setup, commands, testing, and worktrees
> - That file grew to 5,495 lines. Most of it was notes on how one
subsystem behaves, added one pull request at a time
> - Readers could not find the commands. Agents also read behavior notes
as rules
> - This pull request cuts `doc/DEVELOPING.md` to a 660-line setup and
workflow guide. It moves the subsystem notes to focused docs
> - The benefit is a guide that a person or an agent can read in full,
and a clear place for each kind of note

## Linked Issues or Issue Description

**Issue type**

Outdated or confusing documentation.

**Where is the issue?**

`doc/DEVELOPING.md`

**What's wrong?**

The file was 5,495 lines. It opened with 112 lines about case handoffs
before it reached Prerequisites. Unrelated sections sat below the
"OpenClaw Docker UI One-Command Script" heading. Statements about
current behavior read like policy.

**Suggested fix**

Keep setup, commands, and workflow in `doc/DEVELOPING.md`. Move each
subsystem note to a doc for that subsystem. Add an index.

## What Changed

- `doc/DEVELOPING.md` is now 660 lines: prerequisites, start dev, health
checks, local run, test drive, Docker, instance layout, database,
storage, secrets, worktrees, test commands, Storybook, CLI, and an index
of the other docs.
- 15 new docs hold the moved text without wording changes:
`ci-workflows`, `test-infrastructure`, `worktrees`,
`hot-restart-deploys`, `heartbeat-internals`, `runner-internals`,
`workspace-repositories`, `workspace-storage`,
`skills-and-teams-catalog`, `ui-internals`, `case-handoffs`,
`chat-connector-development`, `instance-configuration`,
`integration-api-notes`, `database-backups`.
- "Publish a branch Storybook" moved to `doc/STORYBOOK-DEPLOYMENT.md`.
The privacy rollout notes moved to `doc/ISSUE-PRIVACY.md`.
- Deleted about 280 lines: the story-by-story Storybook notes, "Agent
Artifact Uploads" (a copy of `doc/AGENT-ARTIFACTS.md`), two pointer-only
sections, "Workspace feature checks (2026-10-09)", and "Chat and task
navigation caching". The test-drive build-lock paragraph is now four
lines.
- Repointed every inbound link and every script or workflow comment that
named a moved section.
- `AGENTS.md` now says where a behavior note goes, so the guide does not
grow again.

## Verification

- A script split the file by line range. It asserts that each of the
5,495 source lines is kept, moved, or dropped exactly one time.
- A link check on every changed Markdown file found 0 missing files and
0 missing anchors.
- `node scripts/check-maps.mjs` passes.
- `node --test .github/scripts/tests/*.test.mjs`: 808 pass, 0 fail.
- `node --check` passes on the three scripts with changed comments.
- `pnpm verify` was not run. This worktree has no installed
dependencies, and the change has no runtime code.

## Risks

- Low risk. The change is documentation and comments only.
- A link to a moved section from outside this repository will land on
the top of `doc/DEVELOPING.md`.
- The new docs are not in the audited prose list in
`.github/scripts/ci-inputs.mjs`. An edit to one of them runs full CI. An
edit to `doc/DEVELOPING.md` does not. This PR does not change that list.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`), Claude Code desktop, with tool
use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [ ] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 18:02:27 -06:00

7.5 KiB

Automatic database backups

How automatic logical backups run and how to restore one. Other database guidance is in DATABASE.md.

Paperclip can run automatic logical database backups on a timer. These backups cover non-system database schemas, including migration history and plugin-owned database schemas. Defaults:

  • enabled
  • every 60 minutes
  • retain 30 days
  • backup dir: ~/.paperclip/instances/default/data/backups

Automatic backups are disabled for isolated worktree instances created with paperclipai worktree init or paperclipai worktree:make. Existing worktree configs are migrated to the disabled setting when their server next starts. The main/default instance keeps the normal enabled-by-default behavior.

Configure these in:

pnpm paperclipai configure --section database

Run a one-off backup manually:

pnpm paperclipai db:backup
# or:
pnpm db:backup

Environment overrides:

  • PAPERCLIP_DB_BACKUP_ENABLED=true|false
  • PAPERCLIP_DB_BACKUP_INTERVAL_MINUTES=<minutes>
  • PAPERCLIP_DB_BACKUP_IDLE_CHECKPOINT_ENABLED=1 enables verified final backups for owned idle sleep, with restart catch-up. Off by default; see idle sleep safety for the hosting and storage contract.
  • PAPERCLIP_DB_BACKUP_RETENTION_DAYS=<days>
  • PAPERCLIP_DB_BACKUP_DIR=/absolute/or/~/path
  • PAPERCLIP_DB_BACKUP_MAX_AGE_HOURS=<hours> controls the /api/health stale-backup warning threshold
  • PAPERCLIP_DB_BACKUP_ALERT_FILE=/path/to/failure-marker lets external cron wrappers surface the last failed backup in /api/health
  • PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS=<days> sets how long the terminal-workspace reaper waits after an issue tree becomes terminal before it archives the execution workspace and deletes the worktree. A person can reopen the work inside this window. The default is 1. A value of 0 disables the cooldown and restores immediate reaping. A negative or non-numeric value falls back to the default.
  • PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS=<hours> sets how long the reaper waits after an issue tree becomes terminal before it removes regenerable files from the workspace (see Terminal workspace compaction). The default is 1. A value of 0 compacts on the next sweep. A negative, empty, or non-numeric value falls back to the default.
  • PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false turns compaction off. Any other value keeps it on.
  • PAPERCLIP_SKILL_PACKAGE_STORAGE_ENABLED=false disables external writes and automatic package maintenance for a coordinated rolling upgrade. Default: enabled. Legacy conversion runs automatically in bounded batches, outside turn preparation. Existing external packages remain readable with the override. See package storage and acquisition.
  • PAPERCLIP_SKILL_VERSION_RETENTION_KEEP=<count> sets how many of the newest materialized versions of each skill stay on disk (see Skill Version Snapshot Retention). The default is 0. A negative, empty, or non-numeric value falls back to the default.
  • PAPERCLIP_SKILL_VERSION_RETENTION_GRACE_MINUTES=<minutes> sets how long a materialized skill version stays after its last use. The default is 60. A negative, empty, or non-numeric value falls back to the default.
  • PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_DAYS=<days> sets how long an issue stays closed before the failover backups of its native sessions expire (see doc/recoverable-disk-cleanup.md). The default is 7 and the minimum is 1. A negative, empty, or non-numeric value falls back to the default. PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_ENABLED=false turns expiry off, and PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_DRY_RUN=true only reports candidates. A resumed session restores its failover backup from durable storage before the provider starts. Each read has a five-minute idle limit instead of a short total limit: a large or slow backup keeps restoring while bytes flow. A read that makes no progress for five minutes fails with native_harness_storage_download_stalled and publishes no partial backup. The whole restore, including local decrypt, digest and extraction, has a 30-minute outer limit (native_harness_storage_restore_deadline_exceeded). Stop ends the restore.
  • PAPERCLIP_NATIVE_RUNNER_EVIDENCE_RETENTION_DAYS=<days> sets how long native session continuity-break archives and quarantined session state are kept (see doc/recoverable-disk-cleanup.md). The default is 7 and the minimum is 1. A negative, empty, or non-numeric value falls back to the default. The backup expiration enabled and dry-run switches apply to it as well.
  • PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DAYS=<days> sets how long a settled run's native workspace sync folder (descriptor, manifest and seed archives under <instance>/native-workspace-sync/<run>) is kept (see doc/recoverable-disk-cleanup.md). The default is 7 and the minimum is 1. A negative, empty, or non-numeric value falls back to the default. PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_ENABLED=false turns retention off, and PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DRY_RUN=true only reports candidates.
  • PAPERCLIP_GIT_RESTORE_POINTS_ENABLED=true lets git cache expiration remove closed worktrees with unpushed or uncommitted work after saving an encrypted restore point (see doc/recoverable-disk-cleanup.md). It is off by default. PAPERCLIP_GIT_RESTORE_POINT_TERMINAL_DELAY_DAYS=<days> sets how long the issues that use the workspace (the source issue and descendants bound to it) must be closed first. The default is 3. A negative, empty, or non-numeric value falls back to the default. PAPERCLIP_GIT_RESTORE_POINT_IDLE_DAYS=<days> sets how long an open issue tree's worktree must go untouched before it is removed the same way; runs restore it automatically. The default is 7, and 0 turns this off.
  • PAPERCLIP_TASK_DIRECTORY_RETENTION_ENABLED=true lets the terminal workspace reaper save and remove plain task directories under <instance>/isolated-workspaces/ once their issue tree has been closed for PAPERCLIP_GIT_RESTORE_POINT_TERMINAL_DELAY_DAYS, and remove orphaned ones (see doc/recoverable-disk-cleanup.md). It is off by default and runs as a dry run until PAPERCLIP_TASK_DIRECTORY_RETENTION_DRY_RUN=false. Removal needs PAPERCLIP_GIT_RESTORE_POINTS_ENABLED=true, because the files are saved in an encrypted restore point first and restored on reopen. PAPERCLIP_TASK_DIRECTORY_ORPHAN_GRACE_DAYS=<days> sets how long an orphaned directory's issue tree must be closed and its contents unchanged. The default is 7 and the minimum is 1. A negative, empty, or non-numeric value falls back to the default.
  • PAPERCLIP_GIT_RESTORE_POINT_RETENTION_DAYS=<days> sets how long a git workspace restore point is kept after its issue tree closes (see doc/recoverable-disk-cleanup.md). The default is 90 and the minimum is 1. A negative, empty, or non-numeric value falls back to the default.

Without PAPERCLIP_DB_BACKUP_ALERT_FILE, health checks look for db-backup-to-s3.failure in the backup directory, beside the backup directory, and in the default sibling health/ directory.

DB backups are not full instance filesystem backups. For full local disaster recovery, also back up local storage files and the local encrypted secrets key if those providers are enabled.