mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 13:20:54 +01:00
## Thinking Path > - Paperclip manages agents and tasks. > - Runner owns each authenticated run and its provider process. > - Runtime clients need to report successful activity in that run. > - A retained process can outlive a run, so an operation needs current authority. > - This PR captures that authority before the operation and records activity afterward. > - Core can consume the event through the existing run stream. ## Linked Issues or Issue Description Builds on merged #18129 and targets master. The dependent consumer is #18131. Test stability fixes were separated into merged #18128. Earlier combined-head verification remains in [the archive](https://github.com/paperclipai/paperclip/pull/18050#issuecomment-6489369676). **Subsystem affected** Runner activity recording and provider command environments. **Problem or motivation** A retained provider process can complete an operation after its original run ends. Runtime clients need a host-attributed event and a helper that works inside the provider sandbox. **Proposed solution** Capture opaque Runner context before use. Publish a structured notification after success. Runner checks current authority and records `runtime.activity.recorded`. ## What Changed - Add `notification-context` and `record-activity CONTEXT NAMESPACE ACTIVITY_KEY`. Rotate context on warm attachment and process recovery. Accept no caller run ID. - Use one private named pipe. Reuse Runner's bounded JSON-line reader. Serialize concurrent publishers with a read-only lock. A blocking reader queues at most 128 records; the existing control loop handles at most 16 per pass. - Grant Codex access to the exact helper, descriptor, lock and pipe. Preserve other filesystem and network limits. Preserve host discovery values through provider launch boundaries and reject configured-environment overrides. - Update source identities to Pi 34, Copilot 81 and Grok 12. Keep historical identity fixtures and recovery behavior. ## Verification - Targeted release notification and sandbox-policy tests: seven passed. They cover run rotation, recovery, Unicode limits, concurrent writers, malformed and partial records, bounded queues, failed persistence and fixed-path cleanup. - Real CLI integration and pinned Codex 0.160.0 sandbox checks: passed. The sandbox check uses no model, credential or network call. It verifies activity recording while unrelated state reads and parent-directory writes remain denied. - ACPX profile generator check, formatting and diff checks: passed. - Rebased onto master `31b927e30` at head `64d0248a8`. Master used the same profile versions, so the resolution preserves its Pi 33, Copilot 79 and Grok 11 fixtures byte-for-byte and creates new identities. All identity checks pass. Five targeted provider/environment/recovery files pass 168 tests with one existing skip; the Rust Pi admission regression passes. The notification module and Codex sandbox implementation are byte-identical to the tested pipe head `671373fe0`. - Removed the mailbox traversal reported by security review. Its regression replaces the pipe path with a 100-level tree and an unreadable leaf. Shutdown preserves the leaf, performs no traversal and completes promptly, including with a partial record. - An initial targeted test failed because its run-rotation fixture updated config without updating durable state. The corrected fixture passes. Runner also rejects inconsistent run bindings. - Previous head `26abbf62b` passed Runner, build and browser CI. Server shard 11/12 failed in unchanged `attention-service.test.ts`: cleanup/test timeouts were followed by a duplicate `ATQ` fixture key. The final head needs fresh CI and review. - CI on `64d0248a8` completed with 52 successful checks and four intentional skips. Runner Vitest shard 1/2 failed when retained-state copies encountered the live FIFO; the aggregate also failed. Standard review scored 4/5 and found relative pipe paths plus late context publication. Follow-up `5c3f14d32` places IPC in a private canonical temporary directory and publishes context before welcome commands. Its relative-state CLI test, two authenticated reconnect cases, seven notification/policy cases and real Codex sandbox pass. One affected retained-settlement case passes with the matching release binary; the other 18 cases remain for CI. The sandbox denies sibling-directory and lock writes. All build, test and security checks pass on `5c3f14d32`; standard review is 5/5 with no unresolved threads. The requested Apex review is 5/5 on this exact head, with no findings or unresolved threads. - Earlier broad test results in the archive apply to their recorded source revisions. Per the requested workflow, final local checks stay targeted; GitHub CI supplies broad coverage. - After Apex completed, master advanced again and conflicted on Copilot identity. Head `e94a9359d` is rebased on master `8ac5a128e`; it preserves the published Copilot 80 fixture and uses Copilot 81. The notification module, Codex sandbox and durable control-loop code are byte-identical to Apex-reviewed `5c3f14d32`. All 131 targeted installation/compatibility/recovery cases and identity generator checks pass. All 55 checks pass with four intentional skips on the rebased head. Standard review is 5/5 with no unresolved threads. Apex is 5/5 on `e94a9359d`, with zero findings and no unresolved threads. GitHub confirms this head is conflict-free and ready for review. ## Risks Activity remains an optional hint and grants no access. A successful helper exit means publication to the pipe; Runner persists accepted events through its existing durable outbox. Full queues, unavailable helpers or failed writes can drop a hint without failing the operation. Context reads are local and do not wait for the control loop. Publishers have a 500 ms lock deadline and a 2 KiB record limit. IPC lives outside durable state. Idle transport blocks; cleanup removes its fixed files and empty private directory. The host must provide `/usr/bin/mkfifo`. Rebuild matching Runner/provider artifacts for the updated source identities. No dependency, migration, live provider call or deployment is included. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools and code execution. The exact serving snapshot is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
114 lines
7.6 KiB
JavaScript
114 lines
7.6 KiB
JavaScript
// Regenerate after changing any bound implementation; live qualification must
|
|
// restart whenever the resulting identity changes. Historical identities remain.
|
|
import { createHash } from "node:crypto";
|
|
import { execFileSync } from "node:child_process";
|
|
import { readFileSync, writeFileSync } from "node:fs";
|
|
import { fileURLToPath } from "node:url";
|
|
import { resolve, dirname } from "node:path";
|
|
|
|
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
|
|
const fixture = resolve(root, "test/fixtures/copilot-profile-v81-identity.json");
|
|
const historical = JSON.parse(readFileSync(resolve(root, "test/fixtures/copilot-profile-v12-identity.json"), "utf8"));
|
|
const declaration = { ...historical.declaration, agentProfileVersion: 81 };
|
|
delete declaration.permissionIdentitySourceSha256;
|
|
const sources = {
|
|
displayEventSourceSha256: "src/drivers/acpx/copilot-events.ts",
|
|
displayProjectionSourceSha256: "src/drivers/acpx/copilot-extension-adapter.ts",
|
|
configuredEnvironmentSourceSha256: "src/configured-environment.ts",
|
|
configuredEnvironmentRustSourceSha256: "runner/crates/runner-core/src/configured_environment.rs",
|
|
profileActivitySourceSha256: "src/drivers/acpx/profile-activity.ts",
|
|
profileExtensionsSourceSha256: "src/drivers/acpx/profile-extensions.ts",
|
|
profileInstallationSourceSha256: "src/drivers/acpx/profile-installation.ts",
|
|
attachmentPolicySourceSha256: "runner/crates/runner-core/src/acpx_provider_capabilities.rs",
|
|
singleReadEvidenceSourceSha256: "src/drivers/acpx/single-read-evidence.ts",
|
|
nativeSessionRuntimeSourceSha256: "src/native-session-runtime.ts",
|
|
nativeRuntimeContextSourceSha256: "src/contracts/runtime-context.ts",
|
|
nativeRuntimePromptsSourceSha256: "src/contracts/runtime-context-prompts.ts",
|
|
nativeSessionContractSourceSha256: "src/contracts/native-session-backend.ts",
|
|
harnessSessionContractSourceSha256: "src/contracts/harness-driver.ts",
|
|
harnessBackendSourceSha256: "src/backends/harness-driver-backend.ts",
|
|
harnessSessionSourceSha256: "src/drivers/codex/codex-harness-session.ts",
|
|
transportContractSourceSha256: "src/drivers/codex/app-server-transport.ts",
|
|
nativeBackendSourceSha256: "src/backends/codex-native-backend.ts",
|
|
nativeBackendFactorySourceSha256: "src/backends/native-backend-factory.ts",
|
|
nativeInstructionsSourceSha256: "src/backends/runtime-context.ts",
|
|
providerLaunchSourceSha256: "src/live/runner-provider-launch.ts",
|
|
launchPreparationStoreSourceSha256: "src/live/runner-launch-preparation-store.ts",
|
|
runtimeRequestBridgeSourceSha256: "src/live/runnerd-codex-transport.ts",
|
|
runtimeRequestFacadeSourceSha256: "src/drivers/codex/codex-session-notifications.ts",
|
|
runtimeRequestNotificationSourceSha256: "src/drivers/codex/codex-thread-normalization.ts",
|
|
guardedInstallationSourceSha256: "src/drivers/acpx/installation-integrity.ts",
|
|
nativeExecutionSourceSha256: "src/contracts/native-execution.ts",
|
|
profileCompatibilitySourceSha256: "src/drivers/acpx/profile-compatibility.ts",
|
|
installationSourceSha256: "src/drivers/acpx/copilot-installation.ts",
|
|
assetOwnershipSourceSha256: "src/drivers/acpx/provider-assets-root.ts",
|
|
assetInstallerSourceSha256: "scripts/install-copilot-assets.mjs",
|
|
credentialBindingSourceSha256: "src/drivers/acpx/environment.ts",
|
|
admissionDiagnosticsSourceSha256: "src/drivers/acpx/admission-diagnostics.ts",
|
|
grokAdmissionSourceSha256: "src/drivers/acpx/grok-effort.ts",
|
|
sidecarErrorSourceSha256: "src/cli/acpx-sidecar-input.ts",
|
|
profileSourceSha256: "src/drivers/acpx/copilot-profile.ts",
|
|
metadataProbeSourceSha256: "src/drivers/copilot-metadata-probe.ts",
|
|
acpxPatchSha256: "../../patches/acpx@0.13.1.patch",
|
|
policySha256: "src/drivers/acpx/copilot-policy.ts",
|
|
distributionSourceSha256: "scripts/materialize-copilot-binary.mjs",
|
|
innerDistributionSourceSha256: "scripts/copilot-inner-distribution.mjs",
|
|
permissionContextSourceSha256: "src/drivers/acpx/copilot-permission-context.ts",
|
|
permissionAdapterSourceSha256: "src/drivers/acpx/acp-permission-adapter.ts",
|
|
permissionToolIdentitySourceSha256: "src/drivers/acpx/cursor-plan-tool-identity.ts",
|
|
permissionLocationsSourceSha256: "src/drivers/acpx/safe-locations.ts",
|
|
permissionRedactionSourceSha256: "src/semantic-tools/redaction.ts",
|
|
permissionClassifierSourceSha256: "src/drivers/acpx/generated-sidecar-contract.ts",
|
|
semanticReceiptSourceSha256: "src/drivers/semantic-tool-receipt.ts",
|
|
semanticBridgeSourceSha256: "src/drivers/runner-tool-bridge.ts",
|
|
toolEvidenceSourceSha256: "src/drivers/acpx/copilot-tool-evidence.ts",
|
|
semanticSidecarSourceSha256: "src/cli/acpx-runtime-sidecar.ts",
|
|
semanticContinuationSourceSha256: "src/drivers/completion-continuation.ts",
|
|
semanticDirectDriverSourceSha256: "src/drivers/acpx/codex-acpx-driver.ts",
|
|
semanticValidationSourceSha256: "src/protocol/replay-contract.ts",
|
|
semanticNormalizationSourceSha256: "src/protocol/result-normalization.ts",
|
|
semanticCompletionContractSourceSha256: "src/contracts/completion-result.ts",
|
|
semanticValidatorsSourceSha256: "src/protocol/generated/standalone-validators.ts",
|
|
semanticSchemaBundleSourceSha256: "src/protocol/generated/schema-bundle.ts",
|
|
semanticSidecarProtocolSourceSha256: "src/drivers/acpx/sidecar-protocol.ts",
|
|
preparedHarnessHomeSourceSha256: "src/drivers/preconfigured-home.ts",
|
|
runtimeHostSourceSha256: "src/drivers/acpx/runtime-host.ts",
|
|
runtimeAdapterSourceSha256: "src/drivers/acpx/codex-runtime-adapter.ts",
|
|
runtimeSandboxSourceSha256: "src/drivers/acpx/runtime-sandbox.ts",
|
|
cancellationSourceSha256: "src/drivers/acpx/turn-cancellation.ts",
|
|
agentFilesSourceSha256: "src/drivers/acpx/agent-files-binding.ts",
|
|
nativeDistributionSourceSha256: "src/drivers/acpx/native-distribution-integrity.ts",
|
|
};
|
|
for (const [field, path] of Object.entries(sources)) {
|
|
declaration[field] = createHash("sha256").update(readFileSync(resolve(root, path))).digest("hex");
|
|
}
|
|
const sorted = Object.fromEntries(Object.entries(declaration).sort(([a], [b]) => a.localeCompare(b)));
|
|
const commandDigest = `sha256:${createHash("sha256").update(JSON.stringify(sorted)).digest("hex")}`;
|
|
const identity = `${JSON.stringify({ commandDigest, declaration }, null, 2)}\n`;
|
|
// The shared manifest now generates both TypeScript and Rust admission. Update
|
|
// only Copilot; other providers retain their upstream release attestations.
|
|
const manifestPath = resolve(root, "acpx-profiles.json");
|
|
const manifest = JSON.parse(readFileSync(manifestPath, "utf8"));
|
|
if (process.argv.includes("--check")) {
|
|
if (manifest.profiles.copilot.commandDigest !== commandDigest
|
|
|| manifest.profiles.copilot.agentProfileVersion !== declaration.agentProfileVersion) {
|
|
throw new Error("Copilot identity is stale in acpx-profiles.json");
|
|
}
|
|
} else {
|
|
manifest.profiles.copilot.commandDigest = commandDigest;
|
|
manifest.profiles.copilot.agentProfileVersion = declaration.agentProfileVersion;
|
|
writeFileSync(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`);
|
|
}
|
|
// The builder reads the active manifest; no duplicate digest literal is allowed.
|
|
const distributionSource = readFileSync(resolve(root, "scripts/build-copilot-distribution.mjs"), "utf8");
|
|
if (!distributionSource.includes("profiles.profiles.copilot.commandDigest")) {
|
|
throw new Error("Copilot distribution builder must use the active manifest identity");
|
|
}
|
|
// The shared generator validates this exact attestation before emitting admission.
|
|
if (process.argv.includes("--check")) {
|
|
if (readFileSync(fixture, "utf8") !== identity) throw new Error("Copilot source identity is stale");
|
|
} else writeFileSync(fixture, identity);
|
|
execFileSync(process.execPath, [resolve(root, "scripts/generate-acpx-profiles.mjs"),
|
|
...(process.argv.includes("--check") ? ["--check"] : [])], { stdio: "inherit" });
|
|
console.log(commandDigest);
|