mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-31 22:24:16 +01:00
## Thinking Path > - Paperclip manages work performed by AI agents. > - Native execution crosses the plugin worker, Runner, sidecar, and durable event boundaries. > - Existing outer spans leave repeated preparation and admission work unattributed. > - Optimizations need a common baseline with accurate ownership and event meanings. > - This change extends the existing opt-in tracing contract with bounded timing receipts. > - It preserves execution policy and keeps provider internals opaque where the API provides no timing proof. ## Linked Issues or Issue Description **What existing behavior does this improve?** Attribution of native task preparation, retained/cold Runner handoff, and completion-to-text work. **Current behavior** Worker spans use a flat provider contract. Workspace calls lack a consistent dispatch envelope. Submitted-to-started timing is named as a provider queue even though it includes other work. The transport does not expose a shared breakdown of its waits. **Proposed improvement** Add invocation-scoped nesting, bounded numeric remote receipts, current-run transport timing, durable receipt intervals, and distinct completion/text milestones. Reuse the existing exporter. No latency optimization is included. Related public work: #17842 consolidates existing startup tracing tests; it does not add this runtime coverage. ## What Changed - Extend the optional SDK tracer with nested async scopes and completed timing receipts. Keep the host responsible for parent identity, capability checks, attributes, clocks, depth, and count limits. - Add dispatch and bounded host-read timings for environment and workspace providers. - Resolve transport parents on each call so retained sessions do not retain an earlier run's trace. - Split cold/resume/attach/prepare/registration/command/started-event waits. Add opt-in numeric ACPX Runner/sidecar status and prompt-start receipts. - Distinguish first completion request, validated tool receipt, final result commit, first nonempty text, and event persistence. Rename the submitted-to-started interval to match its meaning. - Link completed earlier remote lifecycles to their host-owned observer rather than placing them under a later polling request. Preserve sampling, timestamp limits and budgets. - Document the versioned coverage map, causal clock limits, and baseline acceptance requirements. ## Verification - Current head: GitHub CI is fully green. Standard Greptile and Apex are both 5/5, with no unresolved review threads. Full-suite validation was performed in CI; follow-up local validation was limited to affected packages and tests. - Review follow-up: early rejection, missing provider capability and startup-parent regressions were reproduced and fixed. The affected worker/SDK/timing set passes 128 tests; 28 timing/context tests, SDK typecheck and direct server TypeScript checking pass. The server typecheck wrapper reached an unrelated Runner rebuild and stopped because Cargo was absent from PATH; full Runner/image validation remains in CI. - CI failure accounting: the earlier Turbo cache/force conflict and Linux OpenCode shell assertion were reproduced and diagnosed. A direct proc probe ruled out permissions; a portable shell comparison identified Bash skipping BASH_ENV with stripped SHLVL and socket stdin. Upstream #18230 now supplies the CI and ignored-stdin fixture fixes; this PR adopts them without duplicate CI changes. The strict Linux OOM assertion passed before this source-equivalent merge. - Retry follow-up: the three insertion/replay/failure timing cases and server typecheck pass. The existing database regression now asserts a single marker across identical result replay; the current-head CI integration gate passes. - Upstream CI alignment: all three real-Turbo 2.11.10 integration cases pass. The repository policy gate rejected committing a generated lockfile; the final PR contains no lockfile edit and uses the maintained inline stale-lockfile recovery. - Linked-lifecycle follow-up: 44 focused host/export tests pass (three optional SDK-dependent cases skip); SDK and server typechecks pass. The API contract test verifies an explicit root context, host-owned observer link and unsampled suppression. - SDK typecheck and focused worker/invocation tests: 30 passed, including clock changes, bounded legacy IPC, and manual-span creation-scope parentage. - Focused reviewed host/native/transaction-boundary checks: 44 passed. The 86-test worker-manager suite passes after updating its exact assertions for the host-owned timing budget found by CI. Earlier preparation/first-text checks also passed. - Affected server and Runner TypeScript checks pass after building local workspace declarations. - Merge follow-up: the profile/candidate startup subset passes 94 tests, the four Grok source-identity checks pass, and Runner TypeScript checking passes. A subsequent execution-factory merge passes 21 focused timing/trace tests and server TypeScript checking. - Pinned Rust build and the ACPX Rust subset pass. The hostile receipt projection test passes. Source profile compatibility checks: 127 passed, one existing skip. Grok identity checks: four passed. - Broad transport/ACPX suite: 410 passed after building all fake provider binaries. Earlier attempts failed on missing Runner/fake-provider outputs; building the required fixtures resolved those failures. - The first CI run correctly rejected stale source attestations. After integrating current master, Grok advances 13→14 and Copilot 84→85; master’s historical fixtures and Pi 35 stay byte-identical. Provider dependencies, models and permission defaults are unchanged. Identity generation/checks now pass. - The added real-Postgres result lock regression could not initialize its local embedded cluster. The insertion/replay/failure unit regressions pass. The current-head CI integration gate passes. - Comprehensive repository checks and image builds are delegated to CI, as requested for this instrumentation pass. No deployment, model run, collector-ingestion proof, or latency improvement is claimed. ## Risks The new SDK methods are optional. Legacy span calls and endpoint-off behavior remain supported. The host accepts only closed timing names and bounded attributes. Remote durations are anchored at local receipt; transport/scheduling uncertainty remains explicit. Trace trees have count/depth limits. The first rollout must inspect actual collector exports before treating the baseline as accepted. Fresh live qualification for the new source identities remains a rollout requirement. Revert these commits to remove the new measurements; no schema migration is required. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and local code/tool execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Standard Greptile and Apex are both 5/5, with no unresolved findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
69 lines
3.6 KiB
TypeScript
69 lines
3.6 KiB
TypeScript
import { AsyncLocalStorage } from "node:async_hooks";
|
|
import { describe, expect, it, vi } from "vitest";
|
|
import { createInvocationTracer } from "../src/invocation-tracer.js";
|
|
import type { PluginSpanRecord } from "../src/protocol.js";
|
|
|
|
describe("invocation timing receipts",()=>{
|
|
it("keeps concurrent roots and nested failures separate",async()=>{
|
|
const records:PluginSpanRecord[]=[];
|
|
const tracer=createInvocationTracer(()=>true,span=>records.push(span));
|
|
let release!:()=>void;
|
|
const waiting=new Promise<void>(resolve=>release=resolve);
|
|
const first=tracer.withSpan("first",async()=>{await waiting;return tracer.withSpan("child",async()=>42);});
|
|
const error=new Error("private");
|
|
await expect(tracer.withSpan("second",()=>tracer.withSpan("failed",async()=>{throw error;}))).rejects.toBe(error);
|
|
release();expect(await first).toBe(42);
|
|
expect(records.map(span=>span.name)).toEqual(["second","first"]);
|
|
expect(records[0].children?.[0].status).toEqual({code:2});
|
|
expect(records[1].children?.[0].name).toBe("child");
|
|
for(const span of records) expect(span.endTimeMs!).toBeGreaterThanOrEqual(span.startTimeMs!);
|
|
expect(JSON.stringify(records)).not.toContain("private");
|
|
});
|
|
it("is inert without an invocation trace and isolates recording failures",async()=>{
|
|
const records:PluginSpanRecord[]=[];
|
|
let count=0;
|
|
const disabled=createInvocationTracer(()=>false,span=>records.push(span));
|
|
expect(await disabled.withSpan("disabled",async()=>++count)).toBe(1);
|
|
expect(records).toEqual([]);
|
|
const broken=createInvocationTracer(()=>true,()=>{throw Error("export");});
|
|
expect(await broken.withSpan("broken",async()=>++count)).toBe(2);
|
|
});
|
|
it("bounds children and excludes detached work after its parent closes",async()=>{
|
|
const records:PluginSpanRecord[]=[];const tracer=createInvocationTracer(()=>true,span=>records.push(span));
|
|
let detached!:()=>Promise<void>;
|
|
await tracer.withSpan("root",async()=>{
|
|
for(let n=0;n<300;n++)await tracer.withSpan("child",async()=>{});
|
|
detached=AsyncLocalStorage.bind(()=>tracer.withSpan("late",async()=>{}));
|
|
});
|
|
expect(records[0].children).toHaveLength(255);
|
|
await detached();expect(records).toHaveLength(1);
|
|
});
|
|
});
|
|
|
|
it("uses one monotonic origin across children even if the epoch clock changes",async()=>{
|
|
const records:PluginSpanRecord[]=[];const tracer=createInvocationTracer(()=>true,span=>records.push(span));
|
|
const clock=vi.spyOn(Date,"now").mockReturnValueOnce(1000).mockReturnValue(999999);
|
|
try {await tracer.withSpan("root",async()=>{await tracer.withSpan("child",async()=>{});});}
|
|
finally {clock.mockRestore();}
|
|
const root=records[0],child=root.children![0];
|
|
expect(child.startTimeMs!).toBeGreaterThanOrEqual(root.startTimeMs!);
|
|
expect(child.endTimeMs!).toBeLessThanOrEqual(root.endTimeMs!);
|
|
expect(root.endTimeMs!).toBeLessThan(999999);
|
|
});
|
|
it("consumes excess legacy records instead of falling back to standalone export",async()=>{
|
|
const tracer=createInvocationTracer(()=>true,()=>{});
|
|
await tracer.withSpan("root",async()=>{
|
|
for(let n=0;n<300;n++)expect(tracer.recordSpan({name:"pack"})).toBe(true);
|
|
});
|
|
});
|
|
|
|
it("manual spans retain their creation scope when ended inside a nested operation",async()=>{
|
|
const records:PluginSpanRecord[]=[];const tracer=createInvocationTracer(()=>true,span=>records.push(span));
|
|
await tracer.withSpan("root",async()=>{
|
|
const record=tracer.captureRecorder(),startTimeMs=tracer.now();
|
|
await tracer.withSpan("nested",async()=>{record({name:"manual",startTimeMs,endTimeMs:tracer.now()});});
|
|
});
|
|
expect(records[0].children?.map(span=>span.name)).toEqual(["nested","manual"]);
|
|
expect(records[0].children?.[0].children).toEqual([]);
|
|
});
|