Files
PaperClipAI/tests/runner-e2e/journey-dependencies.ts
T
DottaandPaperclip 6d46851920 test(e2e): add attended GitHub review bot journey (#17457)
## Thinking Path

> - Paperclip manages agents and the work they perform.
> - Product E2E checks verify real user paths through those agents.
> - A recorded GitHub review-bot walkthrough had no repeatable
registered case.
> - App setup, automatic triggering, publication, and task completion
need separate evidence.
> - This pull request adds an explicit attended journey to the existing
catalog and report tools.
> - Reusable setup profiles and independent assertions preserve the same
outcome across compatible targets.

## Linked Issues or Issue Description

Refs: #17262. This case remains an explicit attended attached-target
flow. It does not replace the shared deployment-target or
paid-operations framework.

**What existing behavior does this improve?**

The Product E2E catalog can now select and retain evidence for GitHub
review-bot setup and its first automatic review.

**Current behavior**

Recorded walkthroughs cannot be rerun as a fixed registered case. A
published review alone can hide a failed run or unfinished task.

**Proposed behavior**

Create and install an owned App, save automatic new-PR settings, create
a defective PR, and require correlated review delivery, an inline
finding, a consistent check, a successful run, and a Done task. Retain
failures and confirm owned-resource cleanup.

**Reason and benefit**

The case makes a useful external integration reproducible without
treating setup, unit checks, or report rendering as proof of a completed
live journey.

## What Changed

- Add two explicit attended cells for an existing configured agent in
local or Daytona execution.
- Declare hosting, account, connection, and credential prerequisites
through private setup bindings.
- Add fixed browser controls, ownership checks, independent grading, and
cleanup.
- Package declared screenshots through the existing sanitizer, manifest,
validator, and dashboard.
- Reject operations-managed execution until budgeted routing is
qualified.
- Document hosted QA credential setup and evidence limits.

## Verification

- Focused catalog, oracle, report, connection, and ownership checks: 127
tests passed.
- Inherited-environment preflight regression failed against the previous
commit and passed after the fix. Actual-run environment verification
remains required.
- Harness TypeScript check and full workspace typecheck passed. Full
build passed. The local full-test attempt was interrupted after repeated
embedded PostgreSQL startup failures; its exit code of zero does not
establish a pass; Exact-head Linux CI passed the full test, typecheck,
build, Runner, serialized, browser, and canary gates on
`abbddeade848edefc8de75f11cc9008b2cd52764`.
- A real hosted attempt published a rated review and inline finding. It
failed when its native session timed out before the task reached Done.
Cleanup passed. Full live qualification remains open.
- Credential scan found no known credential values in the authored files
or retained report corpus.

## Risks

- The full case creates external test resources and uses the configured
provider. It requires an explicit selection and current task authority.
It is excluded from default and CI matrices.
- Time and run limits are not a hard dollar cap. Missing dollar cost
stays unknown. Fixture setup does not weaken budget policy.
- Fresh personal OAuth, unattended authentication, runtime skill-read
proof, and cross-host qualification are separate claims.
- The selected live fixture needs environment/lifecycle recovery before
it can adopt a longer provider deadline.

## Model Used

OpenAI GPT-6 family through Codex, with reasoning, code execution, and
browser tools. The exact deployment model ID and context-window size are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used, including the available version
and capability details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the enhancement
template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket ID
- [x] I have run focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open findings
- [x] I will address all Greptile and reviewer comments before merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-24 12:16:26 -07:00

63 lines
3.5 KiB
TypeScript

import { execFile } from "node:child_process";
/** Public requirements use aliases; deployment profiles bind these to private references. */
export type JourneyCredentialBinding =
| { source: "environment"; variable: string }
| { source: "onepassword"; reference: string };
export class JourneyDependencyError extends Error {
constructor(readonly code: "invalid_dependency_binding" | "service_account_unavailable" | "credential_unavailable") {
super(code);
}
}
export function parseJourneyCredentialBinding(input: unknown): JourneyCredentialBinding {
if (!input || typeof input !== "object" || Array.isArray(input)) throw new JourneyDependencyError("invalid_dependency_binding");
const value = input as Record<string, unknown>;
if (value.source === "environment" && Object.keys(value).sort().join() === "source,variable"
&& typeof value.variable === "string" && /^[A-Z][A-Z0-9_]{1,100}$/.test(value.variable)
&& value.variable !== "OP_SERVICE_ACCOUNT_TOKEN") return { source: value.source, variable: value.variable };
// Resolve one selected field, never a vault/item dump or shell expression.
if (value.source === "onepassword" && Object.keys(value).sort().join() === "reference,source"
&& typeof value.reference === "string" && value.reference.length < 1024
&& /^op:\/\/[^/?#\s]+\/[^/?#\s]+\/(?:[^/?#\r\n]+\/)?[^/?#\r\n]+(?:\?attribute=otp)?$/.test(value.reference)
&& !/[`$\x00-\x1f]/.test(value.reference)) return { source: value.source, reference: value.reference };
throw new JourneyDependencyError("invalid_dependency_binding");
}
export type SecretReader = (reference: string, env: NodeJS.ProcessEnv, signal?: AbortSignal) => Promise<string>;
const readOnePassword: SecretReader = (reference, env, signal) => new Promise((resolve, reject) => {
execFile("op", ["read", "--no-newline", reference], {
env, encoding: "utf8", timeout: 30_000, maxBuffer: 64 * 1024, signal,
}, (error, stdout) => {
// CLI errors can contain private paths, item references or provider responses.
if (error || !stdout.trim()) reject(new JourneyDependencyError("credential_unavailable"));
else resolve(stdout);
});
});
/** The service-account credential goes only to op, never the browser or product agent. */
export async function resolveJourneyCredential(binding: JourneyCredentialBinding, options: {
env?: NodeJS.ProcessEnv; signal?: AbortSignal; read?: SecretReader; registerSecret(value: string): void;
}): Promise<string> {
const checked = parseJourneyCredentialBinding(binding);
const env = options.env ?? process.env;
if (checked.source === "environment") {
const value = env[checked.variable];
if (!value?.trim()) throw new JourneyDependencyError("credential_unavailable");
options.registerSecret(value);
return value;
}
if (!env.OP_SERVICE_ACCOUNT_TOKEN?.trim()) throw new JourneyDependencyError("service_account_unavailable");
options.registerSecret(env.OP_SERVICE_ACCOUNT_TOKEN);
const isolatedEnv: NodeJS.ProcessEnv = Object.fromEntries(["PATH", "HOME", "USER", "TMPDIR", "LANG", "SSL_CERT_FILE", "NODE_EXTRA_CA_CERTS"]
.flatMap(key => env[key] ? [[key, env[key]!]] : []));
isolatedEnv.OP_SERVICE_ACCOUNT_TOKEN = env.OP_SERVICE_ACCOUNT_TOKEN;
try {
const value = await (options.read ?? readOnePassword)(checked.reference, isolatedEnv, options.signal);
if (!value.trim()) throw new JourneyDependencyError("credential_unavailable");
options.registerSecret(value);
return value;
} catch { throw new JourneyDependencyError("credential_unavailable"); }
}