mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 12:37:55 +01:00
docs(runner-e2e): describe the agent view of the eval harness
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b8ccd434ac
commit
79f71ae908
1 file changed
+35
@@ -323,6 +323,41 @@ checks it out only after authorization. Target-controlled code cannot replace
|
||||
the report sanitizer or the AWS history publisher. Fork refs and
|
||||
target-controlled workflow definitions do not enter this path.
|
||||
|
||||
## Agent view of the eval harness
|
||||
|
||||
The agent under test must not read the eval. A local cell runs the product from
|
||||
the checkout that also holds `tests/`, as the same OS user as the harness, so
|
||||
file modes cannot separate them. OpenCode's `external_directory` policy guards
|
||||
its own read and grep tools, but not arbitrary shell commands.
|
||||
|
||||
`agent-filesystem-sandbox.ts` closes that gap for local native cells on Linux.
|
||||
The launcher selects the hidden paths: `tests/`, the trusted
|
||||
`.manual-private-trusted/` copy, the private evidence directory and
|
||||
`RUNNER_TEMP`. The E2E server entry starts each `paperclip-runnerd` through
|
||||
`agent-filesystem-sandbox.py`, which applies a Landlock domain and then execs
|
||||
runnerd with its original PID, argv, cwd and environment. Runnerd, the provider,
|
||||
its tools and every shell command inherit the domain. The domain also sets
|
||||
`no_new_privs`, so `sudo` cannot widen it. The product server itself stays
|
||||
unrestricted because it loads its E2E entry from the harness.
|
||||
|
||||
Before the product starts an agent, a probe runs a sandboxed shell. The probe
|
||||
must be denied the harness and every hidden path, and it must read the product
|
||||
package and write a temporary file. The server log records the probe result and
|
||||
each wrapped runnerd launch. In CI a missing or ineffective sandbox stops the
|
||||
attempt. Landlock does not hide that a path exists (`stat` still works); it
|
||||
denies reading and listing.
|
||||
|
||||
Limits: Codex cells keep Codex's own bubblewrap sandbox, which cannot mount
|
||||
inside Landlock. Legacy adapters, installed-release launches and macOS run
|
||||
without this sandbox. A process that can reach the Docker daemon can still
|
||||
mount the checkout into a container, and `ps` shows the launcher's command
|
||||
line. Daytona cells run the agent in a remote sandbox that holds only the
|
||||
staged runnerd and provider pack, not the checkout.
|
||||
|
||||
Fixture names that an agent can read (company, agent, environment, secret,
|
||||
connection and project) use a neutral word and a digest suffix. They do not
|
||||
contain case ids, suite names, "E2E" or "test".
|
||||
|
||||
## AWS OIDC and S3
|
||||
|
||||
The AWS role trust policy should accept only GitHub's OIDC audience and the
|
||||
|
||||
Reference in new issue
Block a user