mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 06:42:06 +01:00
Nicky LeachandPaperclip
1d571de5f9
## Thinking Path > - Paperclip manages work performed by AI agents. > - Native execution crosses the plugin worker, Runner, sidecar, and durable event boundaries. > - Existing outer spans leave repeated preparation and admission work unattributed. > - Optimizations need a common baseline with accurate ownership and event meanings. > - This change extends the existing opt-in tracing contract with bounded timing receipts. > - It preserves execution policy and keeps provider internals opaque where the API provides no timing proof. ## Linked Issues or Issue Description **What existing behavior does this improve?** Attribution of native task preparation, retained/cold Runner handoff, and completion-to-text work. **Current behavior** Worker spans use a flat provider contract. Workspace calls lack a consistent dispatch envelope. Submitted-to-started timing is named as a provider queue even though it includes other work. The transport does not expose a shared breakdown of its waits. **Proposed improvement** Add invocation-scoped nesting, bounded numeric remote receipts, current-run transport timing, durable receipt intervals, and distinct completion/text milestones. Reuse the existing exporter. No latency optimization is included. Related public work: #17842 consolidates existing startup tracing tests; it does not add this runtime coverage. ## What Changed - Extend the optional SDK tracer with nested async scopes and completed timing receipts. Keep the host responsible for parent identity, capability checks, attributes, clocks, depth, and count limits. - Add dispatch and bounded host-read timings for environment and workspace providers. - Resolve transport parents on each call so retained sessions do not retain an earlier run's trace. - Split cold/resume/attach/prepare/registration/command/started-event waits. Add opt-in numeric ACPX Runner/sidecar status and prompt-start receipts. - Distinguish first completion request, validated tool receipt, final result commit, first nonempty text, and event persistence. Rename the submitted-to-started interval to match its meaning. - Link completed earlier remote lifecycles to their host-owned observer rather than placing them under a later polling request. Preserve sampling, timestamp limits and budgets. - Document the versioned coverage map, causal clock limits, and baseline acceptance requirements. ## Verification - Current head: GitHub CI is fully green. Standard Greptile and Apex are both 5/5, with no unresolved review threads. Full-suite validation was performed in CI; follow-up local validation was limited to affected packages and tests. - Review follow-up: early rejection, missing provider capability and startup-parent regressions were reproduced and fixed. The affected worker/SDK/timing set passes 128 tests; 28 timing/context tests, SDK typecheck and direct server TypeScript checking pass. The server typecheck wrapper reached an unrelated Runner rebuild and stopped because Cargo was absent from PATH; full Runner/image validation remains in CI. - CI failure accounting: the earlier Turbo cache/force conflict and Linux OpenCode shell assertion were reproduced and diagnosed. A direct proc probe ruled out permissions; a portable shell comparison identified Bash skipping BASH_ENV with stripped SHLVL and socket stdin. Upstream #18230 now supplies the CI and ignored-stdin fixture fixes; this PR adopts them without duplicate CI changes. The strict Linux OOM assertion passed before this source-equivalent merge. - Retry follow-up: the three insertion/replay/failure timing cases and server typecheck pass. The existing database regression now asserts a single marker across identical result replay; the current-head CI integration gate passes. - Upstream CI alignment: all three real-Turbo 2.11.10 integration cases pass. The repository policy gate rejected committing a generated lockfile; the final PR contains no lockfile edit and uses the maintained inline stale-lockfile recovery. - Linked-lifecycle follow-up: 44 focused host/export tests pass (three optional SDK-dependent cases skip); SDK and server typechecks pass. The API contract test verifies an explicit root context, host-owned observer link and unsampled suppression. - SDK typecheck and focused worker/invocation tests: 30 passed, including clock changes, bounded legacy IPC, and manual-span creation-scope parentage. - Focused reviewed host/native/transaction-boundary checks: 44 passed. The 86-test worker-manager suite passes after updating its exact assertions for the host-owned timing budget found by CI. Earlier preparation/first-text checks also passed. - Affected server and Runner TypeScript checks pass after building local workspace declarations. - Merge follow-up: the profile/candidate startup subset passes 94 tests, the four Grok source-identity checks pass, and Runner TypeScript checking passes. A subsequent execution-factory merge passes 21 focused timing/trace tests and server TypeScript checking. - Pinned Rust build and the ACPX Rust subset pass. The hostile receipt projection test passes. Source profile compatibility checks: 127 passed, one existing skip. Grok identity checks: four passed. - Broad transport/ACPX suite: 410 passed after building all fake provider binaries. Earlier attempts failed on missing Runner/fake-provider outputs; building the required fixtures resolved those failures. - The first CI run correctly rejected stale source attestations. After integrating current master, Grok advances 13→14 and Copilot 84→85; master’s historical fixtures and Pi 35 stay byte-identical. Provider dependencies, models and permission defaults are unchanged. Identity generation/checks now pass. - The added real-Postgres result lock regression could not initialize its local embedded cluster. The insertion/replay/failure unit regressions pass. The current-head CI integration gate passes. - Comprehensive repository checks and image builds are delegated to CI, as requested for this instrumentation pass. No deployment, model run, collector-ingestion proof, or latency improvement is claimed. ## Risks The new SDK methods are optional. Legacy span calls and endpoint-off behavior remain supported. The host accepts only closed timing names and bounded attributes. Remote durations are anchored at local receipt; transport/scheduling uncertainty remains explicit. Trace trees have count/depth limits. The first rollout must inspect actual collector exports before treating the baseline as accepted. Fresh live qualification for the new source identities remains a rollout requirement. Revert these commits to remove the new measurements; no schema migration is required. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and local code/tool execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Standard Greptile and Apex are both 5/5, with no unresolved findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
40 lines
2.7 KiB
TypeScript
40 lines
2.7 KiB
TypeScript
import { afterEach, describe, expect, it, vi } from "vitest";
|
|
const mocks=vi.hoisted(()=>({parent:undefined as unknown, spans:[] as any[]}));
|
|
vi.mock("@opentelemetry/api",()=>({SpanStatusCode:{ERROR:2},trace:{
|
|
getTracer:()=>({startSpan:(name:string,options:unknown,parent:unknown)=>{const span={name,options,parent,end:vi.fn(),setStatus:vi.fn()};mocks.spans.push(span);return span;}}),
|
|
setSpan:(parent:unknown,span:unknown)=>({parent,span}),
|
|
}}));
|
|
vi.mock("@paperclipai/adapter-utils/acpx-engine/startup-timing",()=>({
|
|
getActiveStepContext:()=>mocks.parent?{parentContext:mocks.parent}:null,
|
|
runWithRuntimeParent:(_parent:unknown,work:()=>unknown)=>work(),
|
|
}));
|
|
import { measureRunnerOperation, recordRunnerTimingReceipt } from "./runner-operation-timing.js";
|
|
afterEach(()=>{vi.unstubAllEnvs();mocks.spans=[];mocks.parent=undefined;});
|
|
describe("Runner operation ownership",()=>{
|
|
it("does no tracer work when disabled and preserves the operation",async()=>{
|
|
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT","");mocks.parent={run:"a"};let count=0;
|
|
expect(await measureRunnerOperation("runner.transport.resume",async()=>++count)).toBe(1);
|
|
expect(mocks.spans).toEqual([]);
|
|
});
|
|
it("reads each current run parent and closes concurrent failures",async()=>{
|
|
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT","http://collector.test");
|
|
const a={run:"a"},b={run:"b"};mocks.parent=a;
|
|
let done!:()=>void;const pending=measureRunnerOperation("runner.transport.resume",()=>new Promise<void>(resolve=>done=resolve));
|
|
await Promise.resolve();mocks.parent=b;
|
|
const error=Error("private");await expect(measureRunnerOperation("runner.transport.attach",async()=>{throw error;})).rejects.toBe(error);
|
|
done();await pending;
|
|
expect(mocks.spans.map(span=>span.parent)).toEqual([a,b]);
|
|
expect(mocks.spans.every(span=>span.end.mock.calls.length===1)).toBe(true);
|
|
expect(mocks.spans[1].setStatus).toHaveBeenCalledWith({code:2});
|
|
});
|
|
it("validates numeric receipt bounds and rejects names or content",()=>{
|
|
vi.stubEnv("OTEL_EXPORTER_OTLP_ENDPOINT","http://collector.test");mocks.parent={run:"a"};
|
|
recordRunnerTimingReceipt({runnerMs:100,roundTripMs:120,sidecarMs:80,statusMs:30,promptMs:40,statusOffsetMs:0,promptOffsetMs:40,secret:"private"});
|
|
expect(mocks.spans.map(span=>span.name)).toEqual(["runner.remote.turn_start","runner.sidecar.turn_start","runner.sidecar.status_read","runner.sidecar.prompt_started"]);
|
|
expect(JSON.stringify(mocks.spans)).not.toContain("private");
|
|
recordRunnerTimingReceipt({runnerMs:10,roundTripMs:120,sidecarMs:80,statusMs:30,promptMs:40,statusOffsetMs:0,promptOffsetMs:40});
|
|
recordRunnerTimingReceipt({runnerMs:100,roundTripMs:120,sidecarMs:80,statusMs:NaN,promptMs:40,statusOffsetMs:0,promptOffsetMs:40});
|
|
expect(mocks.spans).toHaveLength(4);
|
|
});
|
|
});
|