mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 10:48:12 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents receive task constraints and completion tools from Paperclip. > - Completion tools already define the procedure for reporting a result. > - Repeated procedure text adds instructions to each full task turn. > - The final reply must still explain a blocker and link a saved document. > - This pull request removes repeated procedure text and keeps these visible outcome requirements explicit. > - A document receipt supplies the exact link, and stricter evals check the persisted reply and browser navigation. ## Linked Issues or Issue Description Refs: #14961. Related: #14948 and #15007. **What happened?** Native task envelopes repeat completion procedure text. A reduced envelope needs explicit final-reply requirements. The `write_document` receipt also lacks a canonical document link. **Expected behavior** Keep the completion tools as the source of procedure details. Require one accepted completion result before the final reply. A blocked reply must explain the reason, owner and unblock action. A document reply must contain a working link to the saved document. **Steps to reproduce** 1. Run the native assigned-skill document case and native blocker case. 2. Inspect the run-attributed provider final and its persisted comment. 3. Check the blocker explanation or open the final reply's document link. ## What Changed - Remove repeated completion procedure text from the native task constraints and backend instructions. - Keep explicit blocker and document-link requirements in full task turns. - Return a company/task-scoped `documentHref` from `write_document`. Preserve the link in the idempotent mutation receipt. - Repeat canonical links for this run's current saved revisions in accepted completion feedback. Give blocked providers final-response guidance for the cause, owner and unblock action. - Keep internal document/comment anchors when Markdown issue links load cached issue details. - Add a manual six-cell comparison suite with strict source, build, default-instruction and budget admission. - Capture eighteen shared runnerd RPC projections and six direct OpenCode HTTP projections across start, resume and continuation phases, using scripted local transports and no provider execution. - Apply v3 checks only to the manual instruction comparison; preserve v2 checks for the existing native completion suite. Check the actual persisted blocker reason and exact saved-document link. Click the rendered document link and check the original content marker in the classic document card or the new document tab. - Forward exact OpenCode finishing calls through the controller. Wait for acceptance, keep accepted feedback and concrete rejection text, and reject malformed responses. Preserve ordinary dynamic-tool response handling. - Settle the completion decision and tool response before mapping a racing idle/error/abort event or handling explicit close/interruption. Reject a concurrent finishing call before controller admission. - Add a provider-free regression through real runnerd, the OpenCode proxy and a fake provider. Reject the first completion, accept the corrected report in the same turn, and propose one result. - Keep all original verdicts unchanged. Treat replay under new checks as separate diagnostics. ## Verification - `pnpm -r typecheck` and `pnpm build` pass locally. - Native document-authority tests pass, including company/run authorization and idempotent replay. - Native runtime-context, backend and measurement tests pass. - Final-answer calibration, protocol scoring, source-admission and catalog tests pass. Wrong reasons, absent links and wrong link targets fail. - `pnpm test:e2e:runner:typecheck` passes. Discovery lists exactly six single-attempt local cells with the declared models. - Exported `prepareNativeInstructionPreflight` then `verifyNativeInstructionPreflight` pass on this clean committed source. They build locally and make zero provider calls. - Corrective live confirmation is incomplete. Source3a7349dpassed both Claude and both Codex cases. OpenCode saved the correct document but omitted its final link; its blocker case was canceled before paid execution. Preserve this failure. Thee171282confirmation was stopped during build after fresh review found a completion-settlement race; it executed zero providers. Sourcec3e0cb303fixes that race. Two affected OpenCode cases await fresh review and one bounded confirmation; earlier results remain attributed to their original source. - OpenCode proxy parsing, driver, factory and input tests: 81 pass across retained focused runs, including six settlement races. Evaluator/scoring/admission checks: 122 pass. The real proxy regression passes. Fresh local prepare then verify passes with 18 shared and 6 direct scripted captures, fresh SDK/Rust builds and zero providers. - The full local suite recorded two failures: a webhook timeout and a Git-scan load count mismatch. Both files pass in isolation with unchanged assertions/time budgets; preserve the original failure log. Freshc3e0cb303CI and review are pending. This PR remains draft. ## Risks - Final-answer wording can vary by provider. The checks cover the declared release-access blocker and saved document fixture, not general answer quality. - A single trial does not establish general equivalence, cause, speed, cost or live resume behavior. - `documentHref` is an additive receipt field. It points to the current saved document, not an immutable historical revision. Replaying an older receipt does not fabricate a new link. - The correction adds four production paths for document receipts, accepted completion feedback and UI navigation, plus four OpenCode controller/proxy paths, beyond the original three instruction paths. Completion rejection must remain repairable; the production-boundary regression covers it. - Preserve the frozen comparison context for live measurement. A merge-tree check against current master is clean. Do not relabel earlier live results as results from a later source tree. ## Model Used - OpenAI Codex, GPT-6 family. The exact serving model ID and context window are unavailable in this session. Capabilities used: reasoning, code editing, shell execution, test authoring and evidence review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
71 lines
3.9 KiB
TypeScript
71 lines
3.9 KiB
TypeScript
import { createHash } from "node:crypto";
|
|
import { lstat, readFile, readdir, readlink } from "node:fs/promises";
|
|
import path from "node:path";
|
|
import type { RunnerProfileFixture } from "./types.js";
|
|
|
|
export const NATIVE_COMPLETION_BUDGET_CENTS = 1_000;
|
|
// Independent, immutable current-master default; not the reduced #14948 manual.
|
|
export const NATIVE_MASTER_DEFAULT_SHA256 = "e4d2375d722602cd744403292d99f6e6c9b7e8014d9cdfa6f9811ace03428e5f";
|
|
|
|
/** Scoped blocker proof: hash symlink identity without following any target. */
|
|
export async function nativeCompletionWorkspaceDigest(root: string): Promise<string> {
|
|
const hash = createHash("sha256");
|
|
async function visit(relative: string) {
|
|
const entries = await readdir(path.join(root, relative), { withFileTypes: true });
|
|
for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) {
|
|
const file = path.join(relative, entry.name), absolute = path.join(root, file), stat = await lstat(absolute);
|
|
hash.update(JSON.stringify([file, stat.isSymbolicLink() ? "symlink" : stat.isDirectory() ? "directory" : stat.isFile() ? "file" : "other"]));
|
|
if (stat.isSymbolicLink()) hash.update(await readlink(absolute));
|
|
else if (stat.isDirectory()) await visit(file);
|
|
else if (stat.isFile()) hash.update(await readFile(absolute));
|
|
}
|
|
}
|
|
await visit(""); return hash.digest("hex");
|
|
}
|
|
|
|
export function nativeCompletionProfile(profile: RunnerProfileFixture): RunnerProfileFixture {
|
|
if (profile.generation !== "native") throw new Error("Native completion requires a native profile");
|
|
return { ...profile, buildAgent(input) {
|
|
const { instructionsBundle: _fixtureBundle, ...agent } = profile.buildAgent(input);
|
|
return { ...agent, budgetMonthlyCents: NATIVE_COMPLETION_BUDGET_CENTS };
|
|
} };
|
|
}
|
|
|
|
export interface NativeDefaultReceipt {
|
|
schema: "paperclip.native-completion-default.v1";
|
|
agentId: string;
|
|
companyId: string;
|
|
entryFile: unknown;
|
|
files: Array<{ path: string; sha256: string; bytes: number }>;
|
|
budgets: { company: unknown; agent: unknown };
|
|
}
|
|
|
|
export function gradeNativeDefault(receipt: NativeDefaultReceipt, expectedDefaultSha256 = NATIVE_MASTER_DEFAULT_SHA256) {
|
|
const checks = [
|
|
{ id: "production-default-bundle", passed: receipt.entryFile === "AGENTS.md"
|
|
&& receipt.files.length === 1 && receipt.files[0]?.path === "AGENTS.md"
|
|
&& receipt.files[0]?.sha256 === expectedDefaultSha256,
|
|
detail: "The served public hire bundle matches the declared production default, with no fixture instruction injection." },
|
|
{ id: "bounded-company-and-agent", passed: receipt.budgets.company === NATIVE_COMPLETION_BUDGET_CENTS
|
|
&& receipt.budgets.agent === NATIVE_COMPLETION_BUDGET_CENTS,
|
|
detail: "Both public budgets enforce the declared 1,000-cent hard stop." },
|
|
];
|
|
return { passed: checks.every(check => check.passed), checks };
|
|
}
|
|
|
|
export async function captureNativeDefault(input: {
|
|
api: { get<T>(path: string): Promise<T> }; agentId: string; companyId: string;
|
|
}): Promise<NativeDefaultReceipt> {
|
|
const [bundle, company, agent] = await Promise.all([
|
|
input.api.get<{ entryFile: unknown; files: Array<{ path: string }> }>(`/api/agents/${input.agentId}/instructions-bundle`),
|
|
input.api.get<{ budgetMonthlyCents: unknown }>(`/api/companies/${input.companyId}`),
|
|
input.api.get<{ budgetMonthlyCents: unknown }>(`/api/agents/${input.agentId}`),
|
|
]);
|
|
const files = await Promise.all(bundle.files.map(async file => {
|
|
const detail = await input.api.get<{ content: string }>(`/api/agents/${input.agentId}/instructions-bundle/file?path=${encodeURIComponent(file.path)}`);
|
|
return { path: file.path, sha256: createHash("sha256").update(detail.content).digest("hex"), bytes: Buffer.byteLength(detail.content) };
|
|
}));
|
|
return { schema: "paperclip.native-completion-default.v1", agentId: input.agentId, companyId: input.companyId,
|
|
entryFile: bundle.entryFile, files, budgets: { company: company.budgetMonthlyCents, agent: agent.budgetMonthlyCents } };
|
|
}
|