mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-31 06:04:11 +01:00
test: require real onboarding result links (#18288)
## Thinking Path > - Paperclip lets people assign work to agents and use their saved results. > - The Cloud onboarding journey tests this handoff through a real parent and child task. > - The child can publish a correct file while the parent names that file without linking it. > - The existing journey correctly rejects that reply because the user cannot download the named result from it. > - This pull request makes the scripted parent request explicit about real Markdown links and returned attachment paths. > - The independent artifact and parent-link checks remain unchanged. ## Linked Issues or Issue Description Refs #18266 and #18258. **What happened?** A Cloud onboarding attempt produced a correct child artifact and saved document. The parent then named the child, document key, and filename in inline code. It omitted the attachment URL. The attempt failed with `onboarding_parent_result_links_missing`. **Expected behavior** The parent's final reply must link the saved child results. The file link must use the real published attachment reference. **Steps to reproduce** Run the existing explicit Cloud onboarding journey. A reply that only names the file must remain a failure, even when the child artifact is correct. **Paperclip version or commit** The branch starts from `965184d2968f40b1183803529b88f288264c832f`. ## What Changed - Ask the parent for real Markdown links in its final reply. Use the published attachment's returned content or download path. Do not invent IDs or URLs. - Add a filename-only reply to the existing negative oracle cases. - Document this scripted fixture requirement. ## Verification - Focused onboarding tests: 79 passed. - Shared task, campaign, and contract tests: 66 passed. - Existing artifact and native-completion Node checks: 137 passed. - `git diff --check`: passed. - Harness typecheck, `pnpm -r typecheck`, and `pnpm build`: passed after building the existing workspace dependencies. The initial missing-declaration and local-IPC setup failures remain recorded. - The full local database test suite was not rerun. Required repository CI runs the broader suites on Linux. - Required repository CI passed on the exact PR head. No paid model calls or Cloud operations were run for this patch. - [Initial CI](https://github.com/paperclipai/paperclip/actions/runs/41027635545/attempts/1) failed one unchanged retention timing test (3310 ms versus the strict 3000 ms bound). The unchanged case passed once on an isolated local Postgres17 fixture. One failed-jobs-only rerun passed in [attempt 2](https://github.com/paperclipai/paperclip/actions/runs/41027635545/attempts/2). The original failure remains visible. No timing threshold or retention source changed. - The previous live attempt stays failed. Local tests do not prove that a model follows the revised request. A new bounded live attempt must establish that result. ## Risks - The extra instruction can still be ignored by a model. The unchanged oracle will reject missing links. - This changes the fixture definition hash. Historical attempts retain their original source and result. - No product runtime, API, approval policy, model profile, budget, or cleanup behavior changes. ## Model Used OpenAI `gpt-6.1-sol`, ultra reasoning, with code execution and repository tools. Model metadata was confirmed in the parent task. The context window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
965184d296
commit
7a7ade4b93
3 files changed
+8
-3
No files matched your search
@@ -54,6 +54,9 @@ it does not retry the journey. The model still computes the result, publishes
|
||||
the file, saves the document, and completes the tasks. Independent download,
|
||||
math, provenance, document, and parent-link checks remain authoritative.
|
||||
Historical failed attempts retain their original fixture and outcome.
|
||||
The parent's final reply must use Markdown links to the saved child results,
|
||||
including the attachment's real content or download path. Naming a file or
|
||||
document key in inline code does not satisfy the existing parent-link check.
|
||||
|
||||
Save this credential-free local target config outside the repository:
|
||||
|
||||
|
||||
@@ -237,7 +237,7 @@ describe("Cloud canonical onboarding first task", () => {
|
||||
} });
|
||||
await expect(onboardingFirstTask(f.input)).rejects.toMatchObject({ code: mode === "no-child" ? "onboarding_completed_without_child" : "onboarding_approved_child_mismatch" });
|
||||
});
|
||||
it.each(["parent-run", "wrong-connection", "bad-hash", "missing-document", "missing-parent-links"])("does not pass an unverified deliverable: %s", async mode => {
|
||||
it.each(["parent-run", "wrong-connection", "bad-hash", "missing-document", "missing-parent-links", "filename-only-parent-result"])("does not pass an unverified deliverable: %s", async mode => {
|
||||
const f = fixture({ mutate(phase, state) {
|
||||
if (phase !== "accepted") return;
|
||||
if (mode === "parent-run") state.attachments[0]!.originatingRunId = "acceptance-run";
|
||||
@@ -245,8 +245,9 @@ describe("Cloud canonical onboarding first task", () => {
|
||||
if (mode === "bad-hash") state.attachments[0]!.sha256 = "0".repeat(64);
|
||||
if (mode === "missing-document") state.documents = [];
|
||||
if (mode === "missing-parent-links") state.comments.pop();
|
||||
if (mode === "filename-only-parent-result") state.comments.at(-1)!.body = "Done: `QA-2` completed. Saved document: `connection-proof`. Deliverable published: `connection-proof.json`.";
|
||||
} });
|
||||
await expect(onboardingFirstTask(f.input)).rejects.toMatchObject({ code: mode === "bad-hash" ? "artifact_independent_check_failed" : mode === "missing-document" ? "onboarding_child_document_missing" : mode === "missing-parent-links" ? "onboarding_parent_result_links_missing" : "artifact_run_attribution_mismatch" });
|
||||
await expect(onboardingFirstTask(f.input)).rejects.toMatchObject({ code: mode === "bad-hash" ? "artifact_independent_check_failed" : mode === "missing-document" ? "onboarding_child_document_missing" : ["missing-parent-links", "filename-only-parent-result"].includes(mode) ? "onboarding_parent_result_links_missing" : "artifact_run_attribution_mismatch" });
|
||||
expect(f.events).not.toContain("visible:download");
|
||||
});
|
||||
it("counts proposal and parent runs toward the campaign cap", async () => {
|
||||
|
||||
@@ -188,7 +188,8 @@ export async function onboardingFirstTask(input: OnboardingFirstTaskInput): Prom
|
||||
].join("\n");
|
||||
await page.getByTestId("question-other-answer-composer").last().locator('[contenteditable="true"],textarea').first().fill([
|
||||
`I have one task in mind. Propose exactly one child task named "${title}" and wait for my approval before creating it or doing its work.`,
|
||||
"After approval, assign it to yourself and let its own Paperclip execution produce the result. Then verify and link its saved document and downloadable file in this onboarding conversation and complete both tasks.",
|
||||
"After approval, assign it to yourself and let its own Paperclip execution produce the result. After it finishes, verify its saved document and published attachment.",
|
||||
"In your final reply in this onboarding conversation, include actual Markdown links to the child task, its saved document, and connection-proof.json. Copy the published attachment's returned contentPath or downloadPath into the file link URL, including its actual attachment ID. A backticked filename, document key, or path is not a clickable link. Obtain the real references from the child's saved results; do not invent IDs or URLs. Then complete both tasks.",
|
||||
"When you create the child, copy the entire brief between BEGIN CHILD BRIEF and END CHILD BRIEF VERBATIM into create_task.description. Preserve every step, tool name, parameter instruction, and exact encoded input. Do not summarize or paraphrase it.",
|
||||
"BEGIN CHILD BRIEF",
|
||||
childBrief,
|
||||
|
||||
Reference in new issue
Block a user