mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 16:27:18 +02:00
Devin FoleyandPaperclip
63fbb714e6
## Thinking Path > - Paperclip runs work for AI agents. > - Hosts must prove that accepted work has finished before idle sleep. > - HTTP request tracking keeps unfinished writes active after a client leaves. > - Static file middleware has no returned completion promise. > - An aborted UI download can leave its request active forever. > - This PR completes only a matched file response and keeps other work tracked. ## Linked Issues or Issue Description Refs #15522. Found while qualifying #16679. A search found no duplicate static-download idle fix. **What happened?** Closing a client during a static UI download leaves one tracked request active. The response has closed, but the file middleware never calls the wrapped response end method. The instance cannot report idle after that request. **Expected behavior** A closed, matched static file response must not leave a permanent work hold. Unfinished application writes must remain tracked. A missing file must keep the normal fallback and error paths. **Steps to reproduce** 1. Serve a large local file through the static UI middleware with idle admission enabled. 2. Start the download over HTTP. 3. Disconnect the client after its first data chunk. 4. Observe that the tracked request remains active with the original middleware. **Paperclip version or commit** Reproduced on the behavior in `d30648dee89720358271c276b9fc28cf83f18be7`. This branch starts at `cfa9dce295970a85b3d8ee14cab0eccf50bc72e2`. ## What Changed - Mark the completion of a known read-only response without dropping pending middleware promises. - Wrap static UI file serving. A successful file match is required before a closed response can complete its tracking. A late match rechecks an already closed response. Missing files keep the normal fallback. - Use the wrapper for hashed assets, other static UI files, and public development files. Preserve caller headers and cache options. - Test both HTTP abort timings, nested routers, conservative mounted-application holds, parent writes, mutating GET fallbacks, and callback-only work. - Update the idle contract and its verification map. Correct the paused-message recipe from hosted qualification: an explicit Pause cancels older queued work, so test new accepted input after the pause and await the normal resume lifecycle. ## Verification - The original two real stream-abort regressions fail with the original Express static behavior. - Three review regressions fail on `e260f4694afab8447b0cb203f439cb3ccad254e8`: a client closing before file lookup, an unfinished parent in a mounted application, and an unfinished parent in a root without installed handler tracking. Fixed sources were restored byte for byte after the comparison. - Two focused suites: 16 passed. They cover nested routers, both abort timings, hidden parent writes, late missing-file fallbacks, contents, HEAD, conditional responses, and caller headers. - Module and map checks pass. Map checks are structural only. - `pnpm -r typecheck` and `pnpm build`: passed on `f242ee739344484995d3addf81635512370778c8`. - Full CI matrix: 54 successful checks and 2 intentional Storybook skips. - Apex: 5/5 on the exact current head, with zero unresolved threads. - Full local maintained test groups are still running before final handoff. - The initial local broad run was stopped before changing source for the P1 review findings. Two file-mode fixture failures in that run came from the scratch runner's inherited `0077` mask; the same suite passes with the normal child test mask `0022`, without source or host configuration changes. The replacement run uses `0022` for test children and command-scoped `init.defaultBranch=master` for existing Git fixture assumptions. - This PR has not been deployed to a hosted environment. ## Risks Only a matched file served by the admitted application with installed handler tracking can complete this way. Routers share that scope. Mounted Express applications and roots without handler tracking keep the existing conservative response hold; this change does not claim their hidden writes are tracked. An unfinished tracked parent promise still blocks sleep after a file client disconnects. Redirects, missing files, application callbacks, and other GET requests retain their guards. No authentication, company scope, provider cleanup, or scheduling gate is removed. No schema or API response changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, source editing, and shell tools. The session does not expose a more specific serving model ID or its context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
96 lines
6.6 KiB
Markdown
96 lines
6.6 KiB
Markdown
---
|
|
navigation: {"section":"Oversight and operation","sectionOrder":70,"order":40}
|
|
---
|
|
|
|
# Instance operations
|
|
|
|
An operator can install and run Paperclip, diagnose the instance, configure supported deployment settings, manage its service, and produce a recoverable database backup.
|
|
|
|
Implementation: [CLI entry point](../../cli/src/index.ts), [instance settings](../../ui/src/pages/InstanceExperimentalSettings.tsx), [backup command](../../cli/src/commands/db-backup.ts).
|
|
|
|
Agent access: [companion](../agent-access-map/instance-operations.md).
|
|
|
|
## Sub-features
|
|
|
|
- `install-update`: inspect install channel/version and apply supported update or rollback operations.
|
|
- `server-service`: start/stop the intended instance and inspect its service health.
|
|
- `configuration`: configure database, server binding, storage, secrets, and available experimental settings.
|
|
- `diagnostics`: use doctor/health output to distinguish readiness from a listening port.
|
|
- `backup`: create and inspect backups and verify restore only into a separate disposable target.
|
|
|
|
## How to get to it (user POV)
|
|
|
|
### `operator-cli`
|
|
|
|
Use `paperclipai install`, `run`, `service`, `doctor`, `configure`, `channels`, or `update --help` for the intended operation.
|
|
|
|
### `settings-backup`
|
|
|
|
Use currently visible instance settings for experiments/access and `paperclipai db:backup --help` for one-off backup. Some legacy general-settings URLs redirect.
|
|
|
|
## Driving it with Playwright and the Paperclip CLI
|
|
|
|
Preconditions: follow the [baseline](./README.md#before-driving-a-journey). Use a disposable data directory/service identity. Do not update, restart, or restore over an unrelated live instance as a documentation test.
|
|
|
|
### `operator-cli`
|
|
|
|
Automated: [doctor](../../cli/src/__tests__/doctor.test.ts), [update command](../../cli/src/__tests__/update-command.test.ts), and [service manager](../../cli/src/__tests__/service-manager.test.ts) cover command contracts.
|
|
|
|
Manual: Inspect the selected installation/channel and configuration, start the isolated instance, confirm health and browser access, then stop that instance. For update changes, use the dry-run/check path first and verify installed version and rollback state on an owned disposable installation.
|
|
|
|
Routine HTTP monitoring may use `GET /api/liveness` (or `HEAD`). It returns only
|
|
process startup status and the public build commit without SQL or authentication.
|
|
It remains reachable during an idle drain. Use `/api/health` for explicit database
|
|
readiness, signed bootstrap, and authorized diagnostics; liveness does not
|
|
establish database health or claim a workspace.
|
|
|
|
Automated: [static UI idle completion](../../server/src/__tests__/static-ui-idle.test.ts)
|
|
covers real aborted file streams, disconnects before file lookup, nested routers,
|
|
and unfinished parent/fallback writes. Mounted Express applications retain their
|
|
conservative hold. It does not qualify provider cleanup or a hosted sleep/wake cycle.
|
|
|
|
Manual: On an otherwise idle disposable instance, close a static UI download
|
|
served by the admitted application before it finishes. Repeat with a disconnect
|
|
before headers arrive. Confirm that neither leaves a permanent request hold. An
|
|
unfinished application write must still prevent sleep after its client leaves.
|
|
|
|
### `settings-backup`
|
|
|
|
Automated: [experimental settings](../../ui/src/pages/InstanceExperimentalSettings.test.tsx) and [database backup routes](../../server/src/__tests__/instance-database-backups-routes.test.ts) cover their UI/API contracts.
|
|
|
|
Manual: Change a harmless experimental setting, reload, and confirm the affected route honors it. Back up a disposable instance, inspect the reported file and metadata, and restore using the documented procedure into a second isolated target. Verify a known record after restoration.
|
|
|
|
## Gotchas
|
|
|
|
- Source-checkout update/check commands report manual update guidance; they do
|
|
not qualify a managed installation. Rollback restores retained CLI payloads,
|
|
not database migrations. Service status applies to its selected service
|
|
identity, not automatically to an owned foreground dev server.
|
|
|
|
- A successful backup write does not prove restoration works.
|
|
- Settings visibility and mutability differ for cloud-managed and self-hosted instances.
|
|
- Keep the three data paths distinct: first-party Telemetry, operator-configured OpenTelemetry, and local run logs.
|
|
|
|
### Storybook verification
|
|
|
|
#### `backup`
|
|
|
|
Scope: ui
|
|
Storybook: Mount shipped InstanceGeneralSettings in CompanySettings with daily/weekly/monthly retention presets, selected values, saving/error/loading and cloud-hidden controls. Backup creation and restore remain separate CLI/operator checks.
|
|
Production: PATCH /api/instance/settings/general must enforce instance settings authority and managed hidden fields, persist retention and survive reload. Separately inspect a real backup and restore a known record into a second disposable target; a successful write does not prove restoration.
|
|
Sources: Source: [InstanceGeneralSettings.tsx](../../ui/src/pages/InstanceGeneralSettings.tsx), [CompanySettings.tsx](../../ui/src/pages/CompanySettings.tsx), [instance-settings.ts](../../server/src/routes/instance-settings.ts), [db-backup.ts](../../cli/src/commands/db-backup.ts). Tests: [InstanceGeneralSettings.test.tsx](../../ui/src/pages/InstanceGeneralSettings.test.tsx), [instance-settings-routes.test.ts](../../server/src/__tests__/instance-settings-routes.test.ts), [instance-settings-managed-overlay.test.ts](../../server/src/__tests__/instance-settings-managed-overlay.test.ts), [instance-database-backups-routes.test.ts](../../server/src/__tests__/instance-database-backups-routes.test.ts)
|
|
|
|
#### `install-update`
|
|
|
|
Scope: cli
|
|
Storybook: No authored story required for install, channel selection, update or rollback command execution.
|
|
Production: Use an owned managed installation: check/dry-run, installed version, retained payload rollback and failure. Source-checkout guidance is a separate case; rollback does not undo database migrations.
|
|
Sources: Source: [index.ts](../../cli/src/index.ts), [install.ts](../../cli/src/commands/install.ts), [update.ts](../../cli/src/commands/update.ts). Tests: [install-command.test.ts](../../cli/src/__tests__/install-command.test.ts), [update-command.test.ts](../../cli/src/__tests__/update-command.test.ts)
|
|
|
|
#### `diagnostics`
|
|
|
|
Scope: cli
|
|
Storybook: No authored story required for doctor/health command results.
|
|
Production: Run doctor against an owned data directory; check readiness and nonzero failures. A listening port alone does not prove readiness; repair requires observing the repaired result.
|
|
Sources: Source: [doctor.ts](../../cli/src/commands/doctor.ts). Tests: [doctor.test.ts](../../cli/src/__tests__/doctor.test.ts)
|