Files
PaperClipAI/server/src/__tests__/redaction.test.ts
T
DottaandClaude Opus 5.5 a9ac2e2039 feat(native): split local runner startup into stage, launch and session-load spans (#19852)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - When a person messages an agent, the agent must start its turn fast.
Time to first token (TTFT) and agent boot are first-class metrics.
> - The Phase 0 baseline (#19846) found that local runner startup costs
1.6–17 s at p50 and up to 50 s at p95. It is the largest part of TTFT.
> - The run log records that startup as one span
(`runner.session.bootstrap` or `runner.session.resume`). We cannot see
if the time goes to staging, to the `paperclip-runnerd` process launch,
or to the provider session load.
> - This pull request splits local runner startup into three sub-spans
in the run log.
> - The benefit: the follow-up work to cut follow-up-turn TTFT can
measure each step before and after.

## Linked Issues or Issue Description

Refs #19846 (TTFT baseline, finding 1 and gap "No sub-spans exist inside
local runner startup"). Refs #19838.

**What existing behavior does this improve?**
The run-log performance spans for local native runner startup.

**Current behavior**
The local path records one `runner.session.bootstrap` or
`runner.session.resume` span that covers runtime staging, the runner
process launch and the provider handshake together. The remote path
already records `runner.runtime.stage` and `runner.process.launch`; the
local path does not.

**Proposed behavior**
The local path records three sequential child spans of
`runner.session.startup`: `runner.runtime.stage`,
`runner.process.launch` and `provider.session.load`.

**Reason and benefit**
The TTFT work needs to know which step dominates before it changes the
runner lifecycle. Without these spans, a change to warm runner reuse
cannot show which cost it removes.

## What Changed

- `RunnerdCodexTransport` has a new optional `onStartupStep` callback.
`#start` and `#resume` time three milestones and call it synchronously.
A failing callback never changes startup. A nested startup (exact
adoption from `thread/start`) keeps the outer clock.
- `runner.runtime.stage`: launch preparation before the runner process
exists.
- `runner.process.launch`: spawn until runnerd accepts the run
(`run.prepare`, or the recovery attachment on resume). An adopted live
runner records no launch. This keeps the existing meaning in
`summarizeStartupReuse`: a launch span means a fresh runner.
- `provider.session.load`: `session.open` (or the recovery probe) until
the provider identity is confirmed.
- `recordNativeRunnerStartupStep` (`native-run-trace.ts`) records each
step as a `run.performance.span` row with `mode` and `target: "local"`.
It uses `trace.defer`, so the run-log write never delays `turn.start`.
- The native executor passes the callback only for a local target.
Remote launchers keep their own spans, so there are no duplicates.
- `redaction.ts`: allowlist `runner.process.launch` and
`provider.session.load` as native span names.
- `doc/run-log-events.md`: new section "Local runner startup sub-steps".

Data path (AGENTS.md §5.7): run log only, plus Observability spans when
an operator sets an OTLP endpoint. The attributes (`mode`, `target`,
`outcome`) are already in the closed span-attribute allowlist. No
Telemetry change.

## Verification

- `pnpm --filter @paperclipai/paperclip-runner exec vitest run
src/live/runnerd-codex-transport-startup-steps.test.ts`: a real runnerd
and the fake Codex fixture. Bootstrap and a cold resume each report
stage, launch and session load in order, with `ok` outcomes and no
overlap. A throwing sink does not change startup.
- `cd server && npx vitest run
src/services/native-runtime/native-run-trace.test.ts
src/__tests__/redaction.test.ts`: the rows have the correct parent,
offsets, durations and failed level. A stalled run-log sink does not
block the caller. The rows settle before
`task.provider_session.measured`. The names survive redaction.
- The existing `runnerd-codex-transport*` suites pass (412 tests). Under
host load (load average 16–27), the first full run had 7 timing failures
in suites that do not set the callback. Each failing file passed when I
ran it again alone. `runnerd-codex-transport-settlement` passed 3/3 for
the failing case alone.
- `tsc --noEmit` passes for `packages/paperclip-runner` and `server`.

## Risks

- Low risk. Without `onStartupStep`, the transport behaves as before:
the clock is not created and each milestone call is an optional-chain
no-op.
- More run-log rows: three per local runner startup.
- The step boundaries are controller-side observations.
`runner.process.launch` includes the runnerd authentication and the
`run.prepare` acknowledgement, not only the OS spawn.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with
tool use (shell, file edit, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 10:57:31 -06:00

975 lines
37 KiB
TypeScript

import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
describe("explicit diagnostic credential forms", () => {
it("masks qualified credential fields while preserving metadata", () => {
expect(redactEventPayload({ authorizationHeader: "abcdefghijkl", apiKeyProduction: "sensitivevalue", credentialHandling: "harness", tokenPolicy: "least privilege" }))
.toEqual({ authorizationHeader: "***REDACTED***", apiKeyProduction: "***REDACTED***", credentialHandling: "harness", tokenPolicy: "least privilege" });
});
it("masks standalone bearer values and whitespace-prefixed JSON token headers", () => {
const jwt = `${Buffer.from(' {"alg":"HS256","typ":"JWT"}').toString("base64url")}.abcdefghijk.abcdefghijkl`;
expect(redactSensitiveText("provider said Bearer abcdefghijkl")).not.toContain("abcdefghijkl");
expect(redactEventPayload({ diagnostic: jwt })?.diagnostic).toBe("***REDACTED***");
expect(redactSensitiveText("Use bearer authentication and bearer tokens.")).toBe("Use bearer authentication and bearer tokens.");
});
});
import {
PRP_V1_EVENT_TYPES,
PRP_V2_EVENT_TYPES,
REDACTED_EVENT_VALUE,
redactAgentAdapterConfig,
redactEventPayload,
redactSensitiveText,
sanitizeRecord,
} from "../redaction.js";
import { createCopilotToolEvidence } from "../../../packages/paperclip-runner/src/drivers/acpx/copilot-tool-evidence.js";
import { appendSemanticToolReceipt } from "../../../packages/paperclip-runner/src/drivers/semantic-tool-receipt.js";
function receiptNotice(version: 1 | 2 = 1): Record<string, any> {
return {
schema: "paperclip.provider.notice.v1", noticeId: `copilot-evidence-${"a".repeat(24)}-1`,
severity: "info", category: `paperclip_semantic_tool_receipt_v${version}`, scope: "turn",
recoverable: true, userActionable: false, summary: "Paperclip returned a semantic tool result.",
provenance: { method: "paperclip/semantic_tool_result", eventType: "semantic_result", sessionId: "session", turnId: "turn" },
details: Object.entries({ stage: "semantic_result", schema: `paperclip.semantic_tool_receipt.v${version}`,
operationId: "finish_task", callIdentitySha256: "a".repeat(64), inputSha256: "b".repeat(64),
resultSha256: "c".repeat(64), outcome: "returned", ...(version === 2 ? { normalizedInputSha256: "d".repeat(64) } : {}),
}).map(([name, value]) => ({ name, value })),
};
}
const receiptSchemaValue = (notice: Record<string, any>) => notice.details.find((detail: any) => detail.name === "schema").value;
describe("redaction", () => {
it.each([1, 2, 3] as const)("preserves structured panel activation and redacts adjacent secrets in PRP v%s", version => {
const event = { schema: `paperclip.prp.event.v${version}`, schemaVersion: version,
eventType: "runtime.activity.recorded", payload: { namespace: "example.runtime", activityKey: "desktop.used" } };
expect(redactEventPayload({ prpEvent: event })).toEqual({ prpEvent: event });
expect(redactEventPayload({ prpEvent: { ...event, payload: { ...event.payload, apiKey: "secret-sentinel" } } }))
.toEqual({ prpEvent: { ...event, payload: { ...event.payload, apiKey: REDACTED_EVENT_VALUE } } });
});
it.each([1, 2, 3] as const)("preserves credential update metadata and masks adjacent secrets in PRP v%s", version => {
const event = { schema: `paperclip.prp.event.v${version}`, schemaVersion: version,
eventType: "auth.credentials.updated", payload: { schema: "paperclip.auth.credentials.updated.v1",
provider: "codex", bindingId: "harness-auth", revision: 1, detection: "credential_store" } };
expect(redactEventPayload({ prpEvent: event })).toEqual({ prpEvent: event });
expect(redactEventPayload({ prpEvent: { ...event, payload: { ...event.payload, refresh_token: "secret-sentinel" } } }))
.toEqual({ prpEvent: { ...event, payload: { ...event.payload, refresh_token: REDACTED_EVENT_VALUE } } });
});
it("preserves the actual Copilot receipt producer discriminator through nested durable redaction", () => {
const events: Array<Record<string, any>> = [];
const projector = createCopilotToolEvidence({ sessionId: "session", turnId: "turn", workingDirectory: "/workspace",
active: () => true, emit: event => events.push(event) });
const receipt = appendSemanticToolReceipt({ tool: "finish_task", callId: "call", arguments: {} },
{ content: [{ type: "text", text: "accepted" }] }).receipt;
expect(receipt.schema).toBe("paperclip.semantic_tool_receipt.v2");
projector.captureSemanticReceipt()!(receipt);
expect(events).toHaveLength(1);
expect(events[0]!.payload.category).toBe("paperclip_semantic_tool_receipt_v2");
expect(events[0]!.payload.details).toHaveLength(8);
const input = { prpEvent: { schema: "paperclip.prp.event.v1", schemaVersion: 1,
eventType: "provider.notice.recorded", payload: events[0]!.payload } };
expect(redactEventPayload(input)).toEqual(input);
expect(redactEventPayload(redactEventPayload(input))).toEqual(input);
});
it.each([1, 2] as const)("preserves public v%s receipt identifiers as text", version => {
const notice = receiptNotice(version);
expect(redactEventPayload(notice)).toEqual(notice);
// Master's JWT detector recognizes encoded headers, so a dotted public
// identifier needs no receipt-shaped exemption to survive redaction.
expect(redactEventPayload({ value: `paperclip.semantic_tool_receipt.v${version}` }))
.toEqual({ value: `paperclip.semantic_tool_receipt.v${version}` });
});
it("redacts credentials in receipt-shaped data and every adjacent field", () => {
const jwt = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.signature12345678";
const notice = receiptNotice();
notice.details.find((d: any) => d.name === "operationId").value = jwt;
notice.provenance.sessionId = jwt;
const redacted = redactEventPayload(notice)! as Record<string, any>;
expect(receiptSchemaValue(redacted)).toBe("paperclip.semantic_tool_receipt.v1");
expect(redacted.details.find((d: any) => d.name === "operationId").value).toBe(REDACTED_EVENT_VALUE);
expect(redacted.provenance.sessionId).toBe(REDACTED_EVENT_VALUE);
expect(redactEventPayload({ notice, password: "canary", arbitrary: jwt }))
.toMatchObject({ password: REDACTED_EVENT_VALUE, arbitrary: REDACTED_EVENT_VALUE });
notice.details.find((d: any) => d.name === "schema").value = jwt;
notice.summary = "Authorization: Bearer canary-token";
const hostile = redactEventPayload(notice)!;
expect(receiptSchemaValue(hostile)).toBe(REDACTED_EVENT_VALUE);
expect(JSON.stringify(hostile)).not.toContain("canary-token");
});
it("keeps the discriminator allowlist in exact PRP v1 schema parity", () => {
const schema = JSON.parse(
readFileSync(
new URL(
"../../../packages/paperclip-runner/protocol/schemas/event.schema.json",
import.meta.url,
),
"utf8",
),
) as { properties: { eventType: { enum: string[] } } };
expect([...PRP_V1_EVENT_TYPES]).toEqual(schema.properties.eventType.enum);
});
it("keeps the v2 additions in parity with the canonical v2 schema", () => {
const v1 = JSON.parse(
readFileSync(
new URL(
"../../../packages/paperclip-runner/protocol/schemas/event.schema.json",
import.meta.url,
),
"utf8",
),
) as { properties: { eventType: { enum: string[] } } };
const v2 = JSON.parse(
readFileSync(
new URL(
"../../../packages/paperclip-runner/protocol/schemas/event-v2.schema.json",
import.meta.url,
),
"utf8",
),
) as { properties: { eventType: { enum: string[] } } };
const v1Types = new Set(v1.properties.eventType.enum);
expect([...PRP_V2_EVENT_TYPES]).toEqual(
v2.properties.eventType.enum.filter((eventType) => !v1Types.has(eventType)),
);
});
it("preserves every discriminator in the cross-language replay stream", () => {
const fixture = JSON.parse(
readFileSync(
new URL(
"../../../packages/paperclip-runner/protocol/fixtures/replay/duplicate-event.json",
import.meta.url,
),
"utf8",
),
) as { events: Array<Record<string, unknown>> };
for (const event of fixture.events) {
const sanitized = redactEventPayload({ prpEvent: event });
const envelope = sanitized?.prpEvent as Record<string, unknown>;
expect(envelope.eventType).toBe(event.eventType);
expect(envelope.sourceEventId).toBe(event.sourceEventId);
expect(envelope.payload).toEqual(event.payload);
}
});
it("preserves current PRP v2 session discriminators", () => {
const schema = JSON.parse(readFileSync(
new URL("../../../packages/paperclip-runner/protocol/schemas/event-v2.schema.json", import.meta.url),
"utf8",
)) as { properties: { eventType: { enum: string[] } } };
for (const eventType of schema.properties.eventType.enum) {
const event = {
schema: "paperclip.prp.event.v2",
schemaVersion: 2,
eventType,
payload: { safe: true },
};
const sanitized = redactEventPayload({ prpEvent: event });
expect(sanitized?.prpEvent).toEqual(event);
}
});
it("preserves diagnostic PRP discriminators while redacting their credential payloads", () => {
const unknown = redactEventPayload({
prpEvent: {
schema: "paperclip.prp.event.v2",
schemaVersion: 2,
eventType: "session.not-a-real.event",
},
});
expect((unknown?.prpEvent as Record<string, unknown>).eventType).toBe(
"session.not-a-real.event",
);
const mismatched = redactEventPayload({
prpEvent: {
schema: "paperclip.prp.event.v1",
schemaVersion: 2,
eventType: "session.capabilities.updated",
},
});
expect((mismatched?.prpEvent as Record<string, unknown>).eventType).toBe(
"session.capabilities.updated",
);
const secretPayload = redactEventPayload({
prpEvent: {
schema: "paperclip.prp.event.v2",
schemaVersion: 2,
eventType: "session.capabilities.updated",
payload: { authorization: "Bearer secret-value" },
},
});
expect(
((secretPayload?.prpEvent as Record<string, unknown>).payload as Record<string, unknown>)
.authorization,
).toBe(REDACTED_EVENT_VALUE);
});
it("redacts sensitive keys and nested secret values", () => {
const input = {
apiKey: "abc123",
nested: {
AUTH_TOKEN: "token-value",
safe: "ok",
},
env: {
OPENAI_API_KEY: "sk-openai",
OPENAI_API_KEY_REF: {
type: "secret_ref",
secretId: "11111111-1111-1111-1111-111111111111",
version: "aaa.bbb.ccc",
projectionAllowlistKey: "aaa.bbb.ccc",
token: "must-not-survive-reference-shape",
},
USER_API_KEY_REF: {
type: "user_secret_ref",
key: "OPENAI_API_KEY",
password: "must-not-survive-user-reference-shape",
},
OPENAI_API_KEY_PLAIN: {
type: "plain",
value: "sk-plain",
},
PAPERCLIP_API_URL: "http://localhost:3100",
},
};
const result = sanitizeRecord(input);
expect(result.apiKey).toBe(REDACTED_EVENT_VALUE);
expect(result.nested).toEqual({
AUTH_TOKEN: REDACTED_EVENT_VALUE,
safe: "ok",
});
expect(result.env).toEqual({
OPENAI_API_KEY: REDACTED_EVENT_VALUE,
OPENAI_API_KEY_REF: {
type: "secret_ref",
secretId: "11111111-1111-1111-1111-111111111111",
},
USER_API_KEY_REF: {
type: "user_secret_ref",
key: "OPENAI_API_KEY",
},
OPENAI_API_KEY_PLAIN: {
type: "plain",
value: REDACTED_EVENT_VALUE,
},
PAPERCLIP_API_URL: "http://localhost:3100",
});
});
it("redacts jwt-looking values even when key name is not sensitive", () => {
const jwt =
"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c";
const input = {
session: jwt,
opaque: "aaa.bbb.ccc",
normal: "plain",
};
const result = sanitizeRecord(input);
expect(result.session).toBe(REDACTED_EVENT_VALUE);
expect(result.opaque).toBe("aaa.bbb.ccc");
expect(result.normal).toBe("plain");
});
it("preserves Paperclip protocol schema identifiers", () => {
expect(
sanitizeRecord({
schema: "paperclip.question_set.v1",
nested: {
schema: "paperclip.question_response.v1",
runtimeSchema: "paperclip.runtime_request.v2",
arbitraryProviderValue: "paperclip.question_set.v1",
},
}),
).toEqual({
schema: "paperclip.question_set.v1",
nested: {
schema: "paperclip.question_response.v1",
runtimeSchema: "paperclip.runtime_request.v2",
arbitraryProviderValue: "paperclip.question_set.v1",
},
});
});
it("preserves native execution v5 and model envelope v3 schema identifiers", () => {
const input = {
schema: "paperclip.native-execution-input.v5",
nested: {
runtimeSchema: "paperclip.native-model-envelope.v3",
providerValue: "paperclip.native-execution-input.v5",
},
};
expect(sanitizeRecord(input)).toEqual({
schema: input.schema,
nested: {
runtimeSchema: input.nested.runtimeSchema,
providerValue: input.nested.providerValue,
},
});
expect(sanitizeRecord({ schema: "paperclip.native-execution-input.v9" })).toEqual({ schema: "paperclip.native-execution-input.v9" });
expect(sanitizeRecord({ schema: "paperclip.native-execution-input.v7", runtimeSchema: "paperclip.native-execution-input.v8" })).toEqual({
schema: "paperclip.native-execution-input.v7",
runtimeSchema: "paperclip.native-execution-input.v8",
});
expect(sanitizeRecord({ schema: "paperclip.native-execution-input.v4", runtimeSchema: "paperclip.native-model-envelope.v2" })).toEqual({
schema: "paperclip.native-execution-input.v4",
runtimeSchema: "paperclip.native-model-envelope.v2",
});
});
it("preserves only known PRP v1 event discriminators inside validated envelopes", () => {
const payload = {
prpEvent: {
schema: "paperclip.prp.event.v1",
schemaVersion: 1,
eventType: "tool.execution.started",
payload: {
eventType: "run.result.accepted",
credential: "aaa.bbb.ccc",
},
},
unrelated: {
eventType: "workspace.file.referenced",
},
};
const sanitized = redactEventPayload(payload);
expect(sanitized).toEqual({
prpEvent: {
schema: "paperclip.prp.event.v1",
schemaVersion: 1,
eventType: "tool.execution.started",
payload: {
eventType: "run.result.accepted",
credential: REDACTED_EVENT_VALUE,
},
},
unrelated: {
eventType: "workspace.file.referenced",
},
});
expect(redactEventPayload(sanitized)).toEqual(sanitized);
});
it("preserves unknown dotted event values instead of treating them as JWTs", () => {
expect(
redactEventPayload({
schema: "paperclip.prp.event.v1",
schemaVersion: 1,
eventType: "attacker.supplied.token",
})?.eventType,
).toBe("attacker.supplied.token");
});
it("preserves dotted schema identifiers without interpreting their authority", () => {
expect(
redactEventPayload({
schema: "paperclip.attacker.control.v1",
runtimeSchema: "paperclip.attacker.runtime.v1",
schemaVersion: 1,
eventType: "tool.execution.started",
}),
).toEqual({
schema: "paperclip.attacker.control.v1",
runtimeSchema: "paperclip.attacker.runtime.v1",
schemaVersion: 1,
eventType: "tool.execution.started",
});
});
it("preserves native run span identities and ordinary hostnames", () => {
const spanNames = [
"environment.workspace.realize",
"native.coordinator.claim",
"runner.transport.selected",
"runner.prp.authenticate",
"runner.prp.route.register",
"runner.transport.connect",
"runner.session.bootstrap",
"runner.runtime.stage",
"runner.process.launch",
"provider.session.load",
"runner.turn.submit",
"runner.session.startup",
"provider_pack.select",
"provider_pack.discover",
"provider_pack.verify_uploaded",
"provider_pack.upload",
"provider.turn.queue",
"question_response.to_run_created",
"native.session.execute",
"native.result.finalize",
"task.run.measured",
];
for (const span of spanNames) {
const input = {
schema: "paperclip.run-performance-span.v1",
span,
parentSpan: "native.session.execute",
providerHostname: "api.openai.com",
};
const sanitized = redactEventPayload(input);
expect(sanitized).toMatchObject({
schema: input.schema,
span,
parentSpan: "native.session.execute",
providerHostname: "api.openai.com",
});
expect(redactEventPayload(sanitized)).toEqual(sanitized);
}
expect(
redactEventPayload({
schema: "paperclip.run-performance-span.v1",
span: "api.openai.com",
})?.span,
).toBe("api.openai.com");
expect(
redactEventPayload({
schema: "paperclip.run-performance-span.v1",
span: "runner.example.com",
})?.span,
).toBe("runner.example.com");
});
it("redacts payload objects while preserving null", () => {
expect(redactEventPayload(null)).toBeNull();
expect(redactEventPayload({ password: "hunter2", safe: "value" })).toEqual({
password: REDACTED_EVENT_VALUE,
safe: "value",
});
});
it("redacts credentials in ordinary nested diagnostic strings", () => {
const sanitized = redactEventPayload({
message: "request failed with Authorization: Basic dXNlcjpwYXNz",
diagnostic: {
reason: "upstream returned Bearer live-provider-token",
details: [
"safe context",
"proxyAuthorization Basic nested-proxy-secret",
"aaa.bbb.ccc",
["Bearer nested-array-secret"],
],
},
});
expect(sanitized).toEqual({
message: `request failed with Authorization: ${REDACTED_EVENT_VALUE}`,
diagnostic: {
reason: `upstream returned Bearer ${REDACTED_EVENT_VALUE}`,
details: [
"safe context",
`proxyAuthorization ${REDACTED_EVENT_VALUE}`,
"aaa.bbb.ccc",
[`Bearer ${REDACTED_EVENT_VALUE}`],
],
},
});
expect(redactEventPayload(sanitized)).toEqual(sanitized);
});
it("preserves authorization decision reasons in audit payloads", () => {
expect(
redactEventPayload({
authorizationReason: "allow_scoped_agent_write",
authorization: "Bearer secret",
surface: "issue.comment.create",
}),
).toEqual({
authorizationReason: "allow_scoped_agent_write",
authorization: REDACTED_EVENT_VALUE,
surface: "issue.comment.create",
});
});
/**
* A removal receipt (PAP-17119) has to show what it revoked, so a fixed set of
* count keys is exempt from the secret-key guard — but only while the value is
* a number. The second half of this test is the point: the same key carrying
* anything else is still blanked, so the exemption cannot be used to smuggle
* material out under a familiar name.
*/
it("keeps numeric removal-receipt counts but still redacts non-numeric values on the same keys", () => {
expect(
sanitizeRecord({
secretsRevoked: 2,
secretsRetainedShared: 0,
credentialRefsCleared: 3,
secretBindingsRemoved: 3,
tokenIssuanceHashesCleared: 1,
gatewayTokensRevoked: 0,
appProfile: "deleted",
}),
).toEqual({
secretsRevoked: 2,
secretsRetainedShared: 0,
credentialRefsCleared: 3,
secretBindingsRemoved: 3,
tokenIssuanceHashesCleared: 1,
gatewayTokensRevoked: 0,
appProfile: "deleted",
});
expect(
sanitizeRecord({
secretsRevoked: "pasted-api-key-value",
secretBindingsRemoved: { name: "tool_app.abc.headers_authorization" },
tokenIssuanceHashesCleared: Number.NaN,
gatewayTokensRevoked: ["pcgw_live_token"],
}),
).toEqual({
secretsRevoked: REDACTED_EVENT_VALUE,
secretBindingsRemoved: REDACTED_EVENT_VALUE,
tokenIssuanceHashesCleared: REDACTED_EVENT_VALUE,
gatewayTokensRevoked: REDACTED_EVENT_VALUE,
});
});
it("redacts common secret shapes from unstructured text", () => {
const jwt =
"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c";
const githubToken = "ghp_1234567890abcdefghijklmnopqrstuvwxyz";
const input = [
"Authorization: Bearer live-bearer-token-value",
`payload {"apiKey":"json-secret-value"}`,
`paperclip {"PAPERCLIP_API_KEY":"paperclip-json-secret"}`,
`escaped {\\"apiKey\\":\\"escaped-json-secret\\"}`,
`export PAPERCLIP_API_KEY='paperclip-shell-secret'`,
`GITHUB_TOKEN=${githubToken}`,
`session=${jwt}`,
].join("\n");
const result = redactSensitiveText(input);
expect(result).toContain(REDACTED_EVENT_VALUE);
expect(result).not.toContain("live-bearer-token-value");
expect(result).not.toContain("json-secret-value");
expect(result).not.toContain("paperclip-json-secret");
expect(result).not.toContain("escaped-json-secret");
expect(result).not.toContain("paperclip-shell-secret");
expect(result).not.toContain(githubToken);
expect(result).not.toContain(jwt);
});
it("redacts authorization variants and standalone bearer credentials from diagnostic text", () => {
const input = [
"Authorization: Basic dXNlcjpwYXNz",
"Authorization Basic uncolonized-secret",
"proxyAuthorization Basic compound-secret",
'Authorization Bearer abc"embedded-tail',
'Authorization "Bearer quoted-secret"',
'request failed with Bearer standalone"embedded-tail',
].join("\n");
const result = redactSensitiveText(input);
expect(result).toContain(`Authorization: ${REDACTED_EVENT_VALUE}`);
expect(result).toContain(`Authorization ${REDACTED_EVENT_VALUE}`);
expect(result).toContain(`proxyAuthorization ${REDACTED_EVENT_VALUE}`);
expect(result).toContain(`Authorization "${REDACTED_EVENT_VALUE}"`);
expect(result).toContain(`Bearer ${REDACTED_EVENT_VALUE}`);
expect(result).not.toContain("dXNlcjpwYXNz");
expect(result).not.toContain("uncolonized-secret");
expect(result).not.toContain("compound-secret");
expect(result).not.toContain("embedded-tail");
expect(result).not.toContain("quoted-secret");
expect(result).not.toContain("standalone-secret");
expect(redactSensitiveText(result)).toBe(result);
});
it("redacts raw and escaped quote-delimited authorization values without leaking nested content", () => {
const cases = [
{
input: 'prefix Authorization: Bearer "two word secret" suffix',
expected: `prefix Authorization: ${REDACTED_EVENT_VALUE} suffix`,
},
{
input: "prefix Authorization: Basic 'two word secret' suffix",
expected: `prefix Authorization: ${REDACTED_EVENT_VALUE} suffix`,
},
{
input: String.raw`prefix {\"Authorization\":\"Basic dXNlcjpwYXNz\"} suffix`,
expected: String.raw`prefix {\"Authorization\":\"***REDACTED***\"} suffix`,
},
{
input: String.raw`prefix Authorization: \"Bearer \\\"nested two word secret\\\"\" suffix`,
expected: String.raw`prefix Authorization: \"***REDACTED***\" suffix`,
},
];
for (const { input, expected } of cases) {
const result = redactSensitiveText(input);
expect(result).toBe(expected);
expect(result).not.toContain("two word secret");
expect(result).not.toContain("dXNlcjpwYXNz");
expect(redactSensitiveText(result)).toBe(result);
}
});
it("redacts raw and serialized standalone bearer values without leaking quoted content", () => {
const cases = [
{
input: 'prefix Bearer "two word secret" suffix',
expected: `prefix Bearer "${REDACTED_EVENT_VALUE}" suffix`,
},
{
input: "prefix Bearer 'two word secret' suffix",
expected: `prefix Bearer '${REDACTED_EVENT_VALUE}' suffix`,
},
{
input: String.raw`prefix Bearer \"two word secret\" suffix`,
expected: String.raw`prefix Bearer \"***REDACTED***\" suffix`,
},
{
input: String.raw`prefix Bearer \"outer \\\"nested secret\\\" value\" suffix`,
expected: String.raw`prefix Bearer \"***REDACTED***\" suffix`,
},
];
for (const { input, expected } of cases) {
const result = redactSensitiveText(input);
expect(result).toBe(expected);
expect(result).not.toContain("two word secret");
expect(result).not.toContain("nested secret");
expect(redactSensitiveText(result)).toBe(result);
}
});
it("redacts token tails attached to raw and escaped closing credential quotes", () => {
const cases = [
{
input: 'Authorization: Basic "abc"defg retry',
expected: `Authorization: ${REDACTED_EVENT_VALUE}`,
},
{
input: String.raw`Authorization: Basic \"abc\"defg retry`,
expected: `Authorization: ${REDACTED_EVENT_VALUE}`,
},
{
input: 'Bearer "abc"defg retry',
expected: `Bearer "${REDACTED_EVENT_VALUE}"`,
},
{
input: String.raw`Bearer \"abc\"defg retry`,
expected: String.raw`Bearer \"***REDACTED***\"`,
},
{
input: 'Authorization: Basic "abc"defg retry\nsafe context',
expected: `Authorization: ${REDACTED_EVENT_VALUE}`,
},
{
input:
String.raw`Authorization: Basic \"abc\"defg retry` + "\nsafe context",
expected: `Authorization: ${REDACTED_EVENT_VALUE}`,
},
{
input: 'authorization="Bearer abc\ndef" status=401',
expected: `authorization="${REDACTED_EVENT_VALUE}" status=401`,
},
{
input: String.raw`authorization=\"Bearer abc
def\" status=401`,
expected: String.raw`authorization=\"***REDACTED***\" status=401`,
},
{
input: 'authorization="Bearer abc"\ndef" status=401',
expected: `authorization="${REDACTED_EVENT_VALUE}" status=401`,
},
{
input: String.raw`authorization=\"Bearer abc\"
def\" status=401`,
expected: String.raw`authorization=\"***REDACTED***\" status=401`,
},
{
input: 'authorization="Bearer abc"\ndef status=401',
expected: `authorization="${REDACTED_EVENT_VALUE}"`,
},
{
input: String.raw`authorization=\"Bearer abc\"
def status=401`,
expected: String.raw`authorization=\"***REDACTED***\"`,
},
{
input:
'authorization="Bearer first-line"\nrequest failed with Bearer standalone"embedded-tail',
expected: `authorization="${REDACTED_EVENT_VALUE}"\nrequest failed with Bearer ${REDACTED_EVENT_VALUE}`,
},
{
input: String.raw`authorization=\"Bearer first-line\"
request failed with Bearer standalone\"embedded-tail`,
expected: String.raw`authorization=\"***REDACTED***\"
request failed with Bearer ***REDACTED***`,
},
{
input: String.raw`{\"authorization\":\"Bearer a\"b c\"} suffix`,
expected: String.raw`{\"authorization\":\"***REDACTED***\"} suffix`,
},
{
input: `{"authorization":"Bearer a"b c"} suffix`,
expected: `{"authorization":"${REDACTED_EVENT_VALUE}"} suffix`,
},
{
input: String.raw`{\"authorization\":\"Bearer a\" b c\"} suffix`,
expected: String.raw`{\"authorization\":\"***REDACTED***\"} suffix`,
},
{
input: `{"authorization":"Bearer a" b c"} suffix`,
expected: `{"authorization":"${REDACTED_EVENT_VALUE}"} suffix`,
},
];
for (const { input, expected } of cases) {
const result = redactSensitiveText(input);
expect(result).toBe(expected);
expect(result).not.toContain("abc");
expect(result).not.toContain("defg");
expect(result).not.toContain('a"b c');
expect(result).not.toContain('a" b c');
expect(redactSensitiveText(result)).toBe(result);
}
const sanitized = redactEventPayload({
message: 'Authorization: Basic "abc"defg retry',
malformedProviderText: '{"authorization":"Bearer a" b c"} suffix',
authorization: 'Bearer a"b c"',
nested: {
diagnostics: [
String.raw`Bearer \"abc\"defg retry`,
String.raw`{\"authorization\":\"Bearer a\"b c\"} suffix`,
String.raw`authorization=\"Bearer first-line
second-line\" status=401`,
String.raw`authorization=\"Bearer first-line\"
second-line\" status=401`,
],
},
});
expect(sanitized).toEqual({
message: `Authorization: ${REDACTED_EVENT_VALUE}`,
malformedProviderText: `{"authorization":"${REDACTED_EVENT_VALUE}"} suffix`,
authorization: REDACTED_EVENT_VALUE,
nested: {
diagnostics: [
String.raw`Bearer \"***REDACTED***\"`,
String.raw`{\"authorization\":\"***REDACTED***\"} suffix`,
String.raw`authorization=\"***REDACTED***\" status=401`,
String.raw`authorization=\"***REDACTED***\" status=401`,
],
},
});
expect(redactEventPayload(sanitized)).toEqual(sanitized);
});
it("redacts inline secrets from command metadata without hiding safe command text", () => {
const input = {
command:
"custom-acp --token ghp_example_secret env OPENAI_API_KEY=sk-live-example custom-acp",
commandArgs: [
"--safe",
"ok",
"--token",
"ghp_arg_secret",
"--api-key=sk-inline-example",
],
env: {
PAPERCLIP_RESOLVED_COMMAND:
"env OPENAI_API_KEY=sk-live-example custom-acp --token ghp_example_secret",
SAFE_VALUE: "visible",
},
};
const result = redactEventPayload(input);
expect(result?.command).toBe(
`custom-acp --token ${REDACTED_EVENT_VALUE} env OPENAI_API_KEY=${REDACTED_EVENT_VALUE} custom-acp`,
);
expect(result?.commandArgs).toEqual([
"--safe",
"ok",
"--token",
REDACTED_EVENT_VALUE,
`--api-key=${REDACTED_EVENT_VALUE}`,
]);
expect(result?.env).toEqual({
PAPERCLIP_RESOLVED_COMMAND: `env OPENAI_API_KEY=${REDACTED_EVENT_VALUE} custom-acp --token ${REDACTED_EVENT_VALUE}`,
SAFE_VALUE: "visible",
});
});
it("redacts non-string command args after secret flags", () => {
const result = redactEventPayload({
commandArgs: ["--api-key", { nested: "secret-value" }, "safe-next"],
});
expect(result?.commandArgs).toEqual([
"--api-key",
REDACTED_EVENT_VALUE,
"safe-next",
]);
});
it("does not treat bare args payloads as command args", () => {
const result = redactEventPayload({
args: ["--api-key", "not-a-command-secret"],
argv: ["--api-key", "command-secret"],
});
expect(result?.args).toEqual(["--api-key", "not-a-command-secret"]);
expect(result?.argv).toEqual(["--api-key", REDACTED_EVENT_VALUE]);
});
it("redacts every plaintext agent env binding while preserving secret references", () => {
const plaintextValue = "adapter-env-value-must-not-leak";
const result = redactAgentAdapterConfig({
command: "pnpm agent:run",
env: {
EXISTING_VALUE: plaintextValue,
NEW_VALUE: { type: "plain", value: plaintextValue },
SECRET_REFERENCE: {
type: "secret_ref",
secretId: "55555555-5555-4555-8555-555555555555",
version: "latest",
},
USER_SECRET_REFERENCE: {
type: "user_secret_ref",
key: "GITHUB_TOKEN",
},
},
});
expect(result).toEqual({
command: "pnpm agent:run",
env: {
EXISTING_VALUE: { type: "plain", value: REDACTED_EVENT_VALUE },
NEW_VALUE: { type: "plain", value: REDACTED_EVENT_VALUE },
SECRET_REFERENCE: {
type: "secret_ref",
secretId: "55555555-5555-4555-8555-555555555555",
version: "latest",
},
USER_SECRET_REFERENCE: {
type: "user_secret_ref",
key: "GITHUB_TOKEN",
},
},
});
expect(JSON.stringify(result)).not.toContain(plaintextValue);
});
it("redacts non-env adapter keys while leaving env binding shapes intact", () => {
const result = redactAgentAdapterConfig({
command: "pnpm agent:run",
apiKey: "adapter-level-secret",
env: {
API_KEY: "env-level-secret",
AUTH_TOKEN: { type: "plain", value: "another-env-secret" },
},
});
// Non-env keys still go through the shared payload sanitizer.
expect(result.apiKey).toBe(REDACTED_EVENT_VALUE);
expect(result.command).toBe("pnpm agent:run");
// Env bindings keep their binding shape rather than collapsing to a bare
// sentinel string, which is what a second sanitizer pass would produce for
// these sensitive-looking key names.
expect(result.env).toEqual({
API_KEY: { type: "plain", value: REDACTED_EVENT_VALUE },
AUTH_TOKEN: { type: "plain", value: REDACTED_EVENT_VALUE },
});
});
it("redacts adapter configs that have no env block", () => {
expect(redactAgentAdapterConfig({ command: "pnpm agent:run", apiKey: "secret" })).toEqual({
command: "pnpm agent:run",
apiKey: REDACTED_EVENT_VALUE,
});
});
});
describe("redaction of long letter/digit runs", () => {
const typescriptSource = [
"export function compactRunLogChunk(chunk: string, maxChars = 65536) {",
" const normalized = redactSensitiveText(chunk);",
" if (normalized.length <= maxChars) return normalized;",
" return normalized.slice(0, maxChars);",
"}",
"",
].join("\n");
function base64OfTypeScriptSource(length: number) {
const repeatedSource = typescriptSource.repeat(Math.ceil(length / typescriptSource.length));
return Buffer.from(repeatedSource).toString("base64").slice(0, length);
}
function redactWithinTimeBudget(input: string) {
const startedAt = performance.now();
const output = redactSensitiveText(input);
expect(performance.now() - startedAt).toBeLessThan(1_000);
return output;
}
it("redacts secret field names that carry a prefix in plain and escaped JSON", () => {
const cases = [
[`{"myApiKey": "plain-prefixed-secret"}`, `{"myApiKey": "${REDACTED_EVENT_VALUE}"}`],
[`{"x_auth_token":"snake-prefixed-secret","status":"ok"}`, `{"x_auth_token":"${REDACTED_EVENT_VALUE}","status":"ok"}`],
[`GITHUB_TOKEN: "upper-prefixed-secret"`, `GITHUB_TOKEN: "${REDACTED_EVENT_VALUE}"`],
[`{'db_password': 'single-quoted-secret'}`, `{'db_password': '${REDACTED_EVENT_VALUE}'}`],
[`text"apiKey":"glued-quote-secret"`, `text"apiKey":"${REDACTED_EVENT_VALUE}"`],
[`aGVsbG8apiKey:"glued-run-secret"`, `aGVsbG8apiKey:"${REDACTED_EVENT_VALUE}"`],
[String.raw`{\"myApiKey\":\"escaped-prefixed-secret\"}`, String.raw`{\"myApiKey\":\"${REDACTED_EVENT_VALUE}\"}`],
[
String.raw`{\"x_auth_token\":\"escaped-snake-secret\",\"status\":\"ok\"}`,
String.raw`{\"x_auth_token\":\"${REDACTED_EVENT_VALUE}\",\"status\":\"ok\"}`,
],
[String.raw`aGVsbG8\"GITHUB_TOKEN\":\"escaped-glued-secret\"`, String.raw`aGVsbG8\"GITHUB_TOKEN\":\"${REDACTED_EVENT_VALUE}\"`],
];
for (const [input, expected] of cases) {
expect(redactSensitiveText(input)).toBe(expected);
expect(redactSensitiveText(expected)).toBe(expected);
}
});
it("keeps base64 of source text intact and fast while redacting a secret glued to its end", () => {
const blob = base64OfTypeScriptSource(48 * 1024);
expect(blob).toMatch(/^[A-Za-z0-9]+$/);
const shellCommand = `printf '%s' '${blob}' | base64 -d > src/compact.ts`;
expect(redactWithinTimeBudget(shellCommand)).toBe(shellCommand);
const toolCallLine = JSON.stringify({ type: "tool_use", input: { command: shellCommand } });
expect(redactWithinTimeBudget(toolCallLine)).toBe(toolCallLine);
const serializedToolCallLine = JSON.stringify(toolCallLine);
expect(redactWithinTimeBudget(serializedToolCallLine)).toBe(serializedToolCallLine);
const gluedSecret = redactWithinTimeBudget(`${blob}apiKey:"run-glued-secret"`);
expect(gluedSecret).toBe(`${blob}apiKey:"${REDACTED_EVENT_VALUE}"`);
});
it("scans long dash-separated runs in linear time", () => {
for (const run of ["a-".repeat(60_000), "a--".repeat(40_000)]) {
expect(redactWithinTimeBudget(`${run} Authorization: Bearer dash-run-secret`))
.toBe(`${run} Authorization: ${REDACTED_EVENT_VALUE}`);
expect(redactWithinTimeBudget(`${run}Authorization: Bearer glued-run-secret`))
.toBe(`${run}Authorization: ${REDACTED_EVENT_VALUE}`);
}
});
});