DottaandPaperclip
ebceb2a744
## Thinking Path > - Paperclip manages agents and their work. > - The controller prepares workspaces before it starts a Runner. > - A slow run can spend 30–80 seconds inside the combined workspace phase. > - Startup recovery can also delay queued work for minutes. > - Native runs also have gaps before Runner startup and after the provider result. > - Existing traces do not identify the awaited steps inside those intervals. > - This change adds closed timing boundaries around the existing operations. > - Operators can locate the wait without changing execution or exposing content. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operator-configured OpenTelemetry diagnosis of controller preparation and startup recovery. Related: #16178, #16236 and #16285 introduced the existing preparation timing and sampling contract. #16306 and #16312 reduce separate configuration and copyback work. **Current behavior** The workspace interval combines source resolution, trust checks, provisioning, database persistence, binding and repository admission. Startup awaits accounting and native recovery before queued work resumes, but neither has a separate duration. Native backend setup and post-turn cleanup also contain unmeasured awaits. CPU profiles and a fast database probe cannot identify which await is slow. **Proposed behavior** Record workspace steps and native controller setup and cleanup with the existing hashed run ID. Record startup phases as independent, operator-sampled root spans without custom attributes. Keep the same operation order, results, failures and authorization. **Reason and benefit** These boundaries distinguish source work, database writes, repository admission and restart recovery. This is a diagnosis change. It does not claim a response-time improvement or a solved latency budget. ## What Changed - Add 14 closed workspace step names around 16 existing awaits. Repeated updates and bindings produce separate spans. - Time startup tool reviews, gateway sessions, delivery-worker readiness, accounting, plugin readiness, native recovery, hot restart, orphan reaping, retries, queued work and goal recovery. - Time native GitHub setup, wake binding, backend construction, connector assignments, MCP discovery, tool authority, companion selection and binary resolution. Time selected post-turn GitHub, coordinator and instruction cleanup awaits. - Move preparation timing to a pure shared module. Keep the existing heartbeat import as an alias. Both heartbeat and native timing skip tracer lookup when the endpoint is unset or blank, including when another library registered a provider. - Tracing lookup, callback, status and end faults cannot repeat work or replace its value or error. - Add real PostgreSQL workspace timing coverage and silent no-op/fault controls. Update the tracing contract and paired run maps. - Keep span attributes closed. Startup spans add none; workspace steps use only the existing run hash. No commands, paths, returned records or exception messages are recorded. ## Verification - Diagnostic-gap baseline: the new real workspace test fails against the unchanged workspace implementation because the expected step spans are absent. It passes with the wrappers. - Final affected coverage: **804 distinct cases pass** across seven complete suites: 698 native-executor cases, 39 real PostgreSQL workspace cases, 61 timing/context/startup cases, and six execution-configuration integration cases. The final native and unit invocations pass with no selected skips. The workspace suite passed in the initial six-file invocation on the same final workspace source. - Preserve the initial six-file attempt as failed: **7 failed and 789 passed**. All seven errors occurred during embedded PostgreSQL startup. The log reports local macOS shared-memory exhaustion. A fresh 15-case database group passes, followed by the complete 698-case native suite. These runs overlap; do not add their counts. - The new native timing assertion fails against unchanged native code because the expected spans are absent. It passes with the wrappers and retains the credential-isolation assertions. - Full `pnpm -r typecheck` and `pnpm build`: passed on the final production source. Earlier nullable-closure errors are fixed with erased assertions inside existing guards. The failed attempts remain recorded. - Normalized AST comparison: removing only timing wrappers, timing imports/local handles and erased assertions restores the original workspace, startup and native executor programs. This checks call order and policy expressions; it is not live acceptance. - The expanded head first failed two configuration integration assertions because their test providers had no endpoint. Configure the endpoint explicitly in those two tracing tests, and restore it after each test. All six cases then pass with the original credential cleanup, redaction and exact span assertions. Production is unchanged by this repair. - The first PR head failed one unchanged attachment test at its 15-second deadline in CI. The complete 32-case attachment suite passes locally with that deadline. Do not label the failed CI attempt green. The expanded head needs fresh full CI. - `node scripts/check-maps.mjs`: passed, 35 areas and 176 capability IDs. `node scripts/storybook-catalog.mjs`: 283 files and 2,728 stories. Structural checks only; no browser, play or Storybook build claim. - `git diff --check`: passed. No added prose comments, lockfile changes or new 1,000-line crossings. - A duplicate full local test run was not started. Fresh exact-head full CI and E2E pass on `15a683bb2419960f611584f8c07cdcfc9ce3af0b`. All 56 check runs are successful or optional skipped. Superagent and legacy Snyk pass. Completed Apex review is 5/5 on this same head, with zero unresolved review threads. Commands: ```sh pnpm exec vitest run --project @paperclipai/server --maxWorkers=1 --fileParallelism=false server/src/services/native-runtime/native-session-executor.test.ts server/src/services/execution-preparation-timing.test.ts server/src/startup-recovery-timing.test.ts server/src/services/heartbeat/preparation-timing.test.ts server/src/services/heartbeat/preparation-timing-context.test.ts server/src/services/heartbeat/workspace-preparation.test.ts pnpm -r typecheck pnpm build node scripts/check-maps.mjs node scripts/storybook-catalog.mjs ``` ## Risks - Configured operators receive more spans. They still follow the configured sampler and exporter. - Startup spans measure the awaited phase, including remaining readiness waits. They do not measure earlier worker construction or a provider turn. - Workspace and native setup intervals overlap their containing phases. Do not add them to the parent duration. Native timing covers named awaits, not every statement or every error path. - These spans are not yet deployed or tested against a live latency run. No first-party Telemetry event, run-log write, schema or API changes are included. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The exact served variant and context-window size are not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing>
Paperclip agent access map
This map accompanies the feature map. It records APIs, agent entry points, permission boundaries and verification for the same capability IDs. The shared contract defines the format and maintenance rules.
Browse access contracts
The same-named feature recipes define navigation and capability IDs. Run
node scripts/storybook-catalog.mjs for the generated
maps/generated/agent-access-map.md index, or browse the
published feature guide.
Add or update the affected companion; there is no central registration or count.
Existing baseline source references describe the original inspection, not current verification. Record new executed evidence in the PR, issue, or CI artifact with its revision and scope. Do not refresh unrelated headers as bookkeeping.
Using an entry
Read its Policy and Gaps before selecting an agent path. Native fallback, legacy
REST/CLI, stdio MCP and a personal assistant connection can have different
authority and availability. Use a real admitted production binding; a scenario
declaration is not an installed tool. A restricted entry records current
enforcement and leaves any unresolved delegation decision visible.
For API catalog disagreements, follow the route/service authorization and record
discovery drift. For not-applicable, retain the concrete exception and local
path. Split shared business operations from client-only presentation.
Use maintain-verification-skill to reconcile changed source and both maps.
Use verify-paperclip and existing API/runner/MCP fixtures to collect the
permitted-agent and denial evidence; report blocked/not-run coverage honestly.
Run node scripts/check-maps.mjs after editing. The structural check does not
certify permissions, API catalog coverage, tool admission or live behavior.
Native REST transport
The call_api parity change reconciles affected baselines against
58e16dbddfc9c8d0cf60867f9d9847f0feda055b and the changes in this PR. Ordinary
REST operations use the same route authorization as legacy agent REST. Dedicated
tools remain optional for those operations; protocol endpoints still need their
existing clients. Board-only routes remain restricted for agent identity.
Native search_api / call_api requires operator rollout admission, a running
authorized native run, current task assignment and execution-run binding, company
and budget authority, task privacy, and a permitted work mode. Ask and Plan retain
mutation restrictions, and runs retain their admitted JWT scope during restart
recovery even when task mode changes. A waiting
agent does not gain independent native control just because a REST route exists.
Credential-bearing replies, including company exports and their legacy alias, stay inline under a 10 MiB limit and use encrypted mutation receipts. Other saved API snapshots are limited to the issuing actor and responsible user, plus authorized administrators. Artifact uploads enforce the same access checks as downloads; private snapshots are excluded from company exports and public company logos. Explicit deliverables use the existing sharing path. Native event history, checkpoints and completion records omit API bodies while preserving receipt metadata and replay digests. Restored webhook secrets use the same encrypted receipt and inline response policy as newly created webhook secrets. Task image access requires the company boundary and the task permission; skill-test and low-trust agents do not need company-wide browsing authority. Completion replay applies the same idempotent projection to stored and expected payloads; the run-log guide describes its fields.
Sources: API client,
catalog,
native authority,
credential storage classification,
snapshot access,
and history redaction. The affected entries remain
baseline and source-only; this reconciliation does not certify every operation.
Targeted real-route proof lives in the
API integration suite,
including owned routines, skill tests, scoped secrets, proposals and private export replay.