feat: add cloud S3 bulk storage defaults

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip committed 2026-08-14 07:18:28 +00:00
1 parent a323bdf0af
commit 84c03f2dee
10 files changed
+184 -28

No files matched your search

+17
View File
@@ -0,0 +1,17 @@
# S3 bulk storage operations
Cloud runtimes (`PAPERCLIP_K8S_IN_CLUSTER=true`) default attachments to the configured S3 provider. Explicit `PAPERCLIP_STORAGE_PROVIDER` or `config.json` storage settings still win. Completed run logs use the same bucket/settings by default and are stored under `<storage-prefix>/run-logs/`; dedicated `RUN_LOG_S3_*` variables remain an override.
Encrypted state snapshots keep durable state at `instance-state/<instance>/...`. Manifest entries with retention are written separately at `retention/<days>-days/instance-state/<instance>/...`, so transcript/run-log retention cannot delete Codex sqlite snapshots. The approved transcript and run-log retention is 90 days.
Apply the lifecycle policy with:
```sh
PAPERCLIP_STORAGE_S3_BUCKET=paperclip \
PAPERCLIP_STORAGE_S3_PREFIX=production \
scripts/apply-s3-bulk-lifecycle.sh
```
The script uses `aws s3api put-bucket-lifecycle-configuration`. Set `AWS_ENDPOINT_URL` for an S3-compatible provider. Re-running it is idempotent but replaces the bucket lifecycle configuration, so merge additional bucket rules into the script payload before applying on a shared bucket.
Litestream was evaluated for Codex sqlite continuous point-in-time restore. It improves recovery point objectives, but adds a runtime binary/sidecar, per-database replica configuration, monitoring, restore orchestration, and ongoing version/security maintenance. P4 ships scheduled `sqlite3 .backup` copies without that dependency; adopting Litestream remains an operations decision for a later cloud phase.