Commit Graph
7726 Commits
Author SHA1 Message Date
Dotta de6d29e025 test(evals): bind native timeline audit producer declaration 2026-11-03 00:39:51 -06:00
DottaandPaperclip 4dcd2a5c1e test(evals): join timeline milestones to native status decisions
Preserve raw system activity and prove its exact accepted worker outcome instead of requiring the legacy agent PATCH audit shape.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-03 00:39:39 -06:00
Dotta 40f4215186 test(evals): synchronize source5 fixture observer declarations 2026-11-03 00:30:57 -06:00
DottaandPaperclip 36ec1aa4fb test(evals): inspect authored checklist as rich text blocks
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-03 00:30:23 -06:00
DottaandPaperclip 166815c13a test(evals): bind multipart source observation to original deadline
Preserve the original case budget and require the saved question and canonical completed source run before human form actions.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-03 00:25:30 -06:00
DottaandPaperclip 075651b15c test(evals): match canonical controller secret keys
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-03 00:20:14 -06:00
Dotta a2d4980eb5 fix(chat): bound proposal metadata before trimming in candidate 2026-11-03 00:16:27 -06:00
DottaandPaperclip 0a76dbde9b test(evals): inspect announcements on normal company surfaces
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-03 00:16:27 -06:00
Dotta 2aeceb963d fix(evals): pin reviewed Source4 chat repair candidate (#18736)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The chat catalog checks real user journeys through a fixed
candidate.
> - The current candidate retains several known fixture and observer
defects.
> - A new source must bind each repaired declaration before paid work
starts.
> - This pull request pins the reviewed Source4 candidate and its exact
descriptor snapshot.
> - The next measurements can test those repairs without changing old
results or relaxing private evidence guards.

## Linked Issues or Issue Description

Refs #18726. Related core changes remain separate and unmerged: #18717
and #18734.

**What happened?**

The active lane still uses Source3. It cannot test the accepted primary,
app review, CLI, timeline, plan review, hire fixture and SDK input
repairs. Old results must remain historical.

**Expected behavior**

The trusted lane must bind the new immutable candidate, all 244 case
identities, exact current declarations and private source hashes. It
must retain the original definition provenance and explain changed
conditions.

**Steps to reproduce**

1. Compare the current manifest with candidate
`51ef450754370c8a8f2cbe4bd18e1536bbc93966`.
2. Check each of its 242 declarations against the checked descriptor
snapshot.
3. Run the focused manifest, contract and reconciliation controls. Run
the selected real cases only after normal source delivery and a fresh
execution guard.

## What Changed

- Pin candidate `51ef450754370c8a8f2cbe4bd18e1536bbc93966` and its exact
descriptor snapshot. Keep all 244 identities and aliases.
- Record cumulative original-to-current field overrides. Keep the
original `de094` definition provenance and case limits. Studio144
retains its earlier explicit 2..3 accounting revision.
- Record the reviewed SDK expected build-lock correction and the
candidate-only full proposal disclosure from unmerged #18734. Keep
compiler/parser/Zod identities, privacy guards and the other SDK holds.
- Update the static owning controls and identify current versus
historical sources in the lane guide. Runtime JavaScript, declarations,
workflow permissions and private transfer code are unchanged in this PR.

## Verification

- 47 focused Node controls passed: contract, manifest and idempotent
reconciliation.
- Candidate controls: 293 pure tests across 16 files passed. All 242
actual factory declarations and 42 admission controls passed without
fixture callbacks.
- The final candidate SDK registry check matched all 506 inputs. The
actual Git launcher, checked snapshot, three private protocol files and
all 244 singleton evidence policies matched.
- The candidate product disclosure is byte-identical to the reviewed
#18734 file. Its token gate passed. The separate core change retains 115
React controls.
- Independent full PR and integration source review passed before
publication.
- Full local workspace typecheck, application build and model execution
were not run for this pin. The existing workspace dependency limitations
remain recorded. Latest-head CI `41586212647` passed on attempt 1: 48
passed and 1 skipped. Both required checks passed. Greptile reviewed the
exact head at 5/5 with no open threads.

## Risks

- These are new candidate measurements. A source-qualified repair does
not prove a user journey passed.
- SDK115 must first pass real admission with its original
300-second/one-native limit. The other five SDK cases and plugin558
canary certificates remain held.
- The CLI parser rejects unresolved placeholders and foreign values. It
executes the actual returned commands without substitutions. The old
failed command remains failed.
- The core guidance and review-card PRs remain unmerged. This pin
changes neither core merge authority nor the previous full-catalog
acceptance gate.
- A running Source3 launch remains pinned to its immutable controller
and candidate. The next model wave must wait for terminal accounting and
cleanup guards.

## Model Used

OpenAI `gpt-6.1-sol`, with code execution, tool use and independent
agent source review. The host does not expose the exact context window.
No model eval was run to prepare this PR.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-11-03 00:04:17 -06:00
Nicky LeachandClaude Opus 5.5 32f93cf9dd fix(runner): set Runner OOM protection at launch (#18730)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The native Runner supervises the harness and tool processes for each
run
> - A launcher can start Runner with a negative `oom_score_adj`, and
Linux children inherit that value
> - Then a large tool process has the same protection as Runner
> - If that protection is -1000, the kernel cannot kill the process, and
a VM with no other candidate panics
> - The correct protection for Runner is different in each environment,
so one static value is not correct
> - This pull request adds an optional `--oom-score-adj` launch flag.
With the flag, Runner also sets supervised processes to 0 when Runner
has a negative score
> - The benefit is that the launcher controls Runner protection, and the
kernel can kill the large process and let Runner report the failure

## Linked Issues or Issue Description

Related: #14509 and #16844. These pull requests change scores for the
server and for sandbox tool commands. This pull request changes the
native Runner.

**What happened?**

A native Runner on a VM inherited `oom_score_adj` -1000 from the VM init
process. A typecheck under Runner used most of the VM memory. The kernel
could not select any process and panicked with "System is deadlocked on
memory". The VM restarted, and the run failed with a transport error.

**Expected behavior**

The kernel can kill the large tool process. Runner continues and reports
the result.

**Steps to reproduce**

Start Runner with a negative score. Start a supervised process. Read
`/proc/<pid>/oom_score_adj` for the supervised process. The value is the
same as the Runner value. After this change, the value is 0 when the
launcher gives the flag.

**Deployment mode**

Native Runner on Linux, when the launcher gives Runner a negative score.

## What Changed

- Add the optional `--oom-score-adj <-1000..=1000>` flag to
`paperclip-runnerd`. Runner has no default value.
- Stop Runner startup when the value is not valid. When the kernel
refuses a valid value, or the platform is not Linux, write a diagnostic
and continue, because Runner is then only less protected than the
launcher asked for.
- When the launcher gives the flag and the Runner score is negative, set
each supervised process and verified stdio MCP server to 0 after spawn.
- Without the flag, Runner does not change any score. Children inherit
the Runner score, as they do now.
- Add unit and integration tests.

## Verification

Tests ran in `rust:1-bookworm` (rustc 1.99.1) on Linux arm64 with
`CAP_SYS_RESOURCE`:

- `cargo fmt --check -p paperclip-runner-core`
- `cargo test -p paperclip-runner-core --no-fail-fast`, before the flag
gate commit: all targets pass except `codex_provider` and `stdio_mcp`.
- `provider_receives_the_isolated_codex_auth_home` and its subprocess
helper fail in this container. They fail in the same way on clean
`master`.
-
`receipt_limit_polls_an_authoritative_terminal_with_unacknowledged_events`
and `cancellation_waiting_behind_output_is_handled_before_new_dispatch`
failed once in the full run. They pass when they run alone, on this
branch and on clean `master`. I think that they are sensitive to load,
but I did not prove this. CI is the check for these tests.
- `--test oom_score` and `--test oom_score_without_flag` pass with and
without `CAP_SYS_RESOURCE`. It covers the real Runner binary, flags that
are not valid, a Runner that cannot apply a negative score, and a
supervised child of a Runner with a negative score. The tests for
children of a Runner with a negative score skip themselves without
`CAP_SYS_RESOURCE`. The unit test covers the same reset logic without
that capability.

I also ran the debug Runner binary through the agent computer `enter`
command in a container, as UID 1000, with a floor of -500:

- Runner stayed alive at -500.
- The `oom_score` integration test passed in the same setup. Its
supervised child had the score 0.

After the flag gate commit, I ran the library `oom_score` tests, `--test
oom_score`, `--test oom_score_without_flag`, and `--test
process_supervisor` again. CI runs the complete suite.

I did not run this on a real VM.

## Risks

- A fork in the short time after exec and before the write keeps the
Runner score. The kernel can still kill that process unless the score is
-1000.
- The write to the child can fail. Runner ignores this failure, because
the child has no more protection than Runner.
- A launcher without the necessary privilege gets a Runner at the
inherited score. Runner writes a diagnostic to standard error, and the
run continues. A launcher that discards standard error does not see the
diagnostic.
- A launcher that does not give the flag has no change, also when Runner
starts with a negative score. On Linux, each spawn has one more atomic
read and no system call. A `oom_score_without_flag` test binary checks
this.
- A new Runner under a launcher that does not give the flag and gives
Runner -1000 keeps its children at -1000. Update that launcher to give
the flag.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with
tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:02:45 -08:00
Dotta 51ef450754 test(evals): synchronize repaired current case declarations 2026-11-02 23:43:49 -06:00
Dotta 48cfd4a349 fix(evals): observe supported owned CLI profile guidance 2026-11-02 23:43:07 -06:00
DottaandPaperclip 80d7992590 fix(evals): bind SDK public proof to reviewed runtime lock
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 23:42:40 -06:00
Dotta fd69dde707 fix(chat): expose reviewed redacted tool proposal in candidate 2026-11-02 23:38:41 -06:00
DottaandPaperclip 332431de44 test(evals): inspect bound full app proposals before decisions
Pair the ordinary shared proposal disclosure with exact pending invocation arguments and actual provider dispatch. Keep the short preview valid, reject uninspectable or changed content, and retain the original three human/native permission phases.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 23:38:26 -06:00
DottaandPaperclip baa5479225 test(evals): read owned timeline bytes with fixture authority
Keep worker credentials limited to task validation and use the existing trusted fixture client for the fixed board-only file route after a live scope/checkout join. Preserve actual bounded command and receipts; reject changed custody before publishing the exit result.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 23:38:26 -06:00
DottaandPaperclip 636dec32fd test(evals): observe primary guidance within its original deadline
Bound only the initial reply wait to the existing remaining 300-second case clock, checking before and after. Preserve actual terminal reply, personal permission controls and all settled Chat final verification.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 23:38:26 -06:00
DottaandPaperclip 459c5d784f test(evals): bind app test evidence to normalized MCP result
Reconstruct the documented gateway result independently, retaining raw provider and source attribution checks. Preserve the original failure and mark the oracle v3 changed condition.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 23:38:26 -06:00
DottaandPaperclip e5713d95c6 test(e2e): add scheduled routine journey (#18728)
## Thinking Path

> - Paperclip manages agent work through tasks, projects, and routines.
> - Daily QA must exercise the staging app through normal browser and
API surfaces.
> - A schedule must create real work and retain its result and history.
> - Pausing that schedule must prevent the next invocation.
> - The routine journey in #18652 passed in QA, while its separate
completion journey has not passed.
> - This PR delivers the routine test independently and preserves its
measured flow and assertions.

## Linked Issues or Issue Description

Refs: #18652. This is the routine-only delivery from that PR. The
original completion implementation and its failed attempts remain on
#18652.

## What Changed

- Add the target-independent `routine-scheduled-run` journey with one
native run and seven required checks.
- Start from the measured routine source
`ac4efb7c6f4782ea806661683c656fd2b4cc3189`. Keep its request, seven
success assertions, one native-run limit, and 65-second pause check.
- Bound each observed GET by the original remaining journey deadline.
Preserve the normal API timeout for other callers. Keep the original
routine error if final trigger disabling also fails, save a closed
secondary receipt, and fail an otherwise successful journey when
disabling fails.
- Narrow the shared output-reader type to its one-argument GET
capability without changing its runtime code.
- Register only this case in the catalog, types, feature associations,
dispatcher, and definition hash. Keep the strict catalog expectation in
sync.
- Retain the real Routines component controls and the existing
project-flow coverage for the shared browser helper.
- Document resource cleanup and the 65-second pause observation.
Preserve the current routine map content.

## Verification

- The original source passed [QA staging run
41567574185](https://pages.paperclip.ing/evals-qa-smoke-41567574185-1/).
TestOps launch 1657 / result 3431 maps to case 718. All seven checks and
five phases passed. One scheduler-bound native run, saved output
attribution, the full pause window, and cleanup were verified.
- Current source `8f62137f130f59fd3e39276e3d60185ddff24517` passed
[fresh QA staging run
41582706162](https://pages.paperclip.ing/evals-qa-smoke-41582706162-1/)
against Core `09162fe893b464dd2ccdafd14bd3ef1611202a86`. TestOps launch
1855. All seven checks, five phases, and all three workflow jobs passed.
The run used one scheduler-bound native turn and observed 65.352 seconds
after pause and completion. Canonical evidence collection and normal
cleanup completed, with no active operations and the owned agent
terminal.
- This fresh pass covers the reviewed deadline and failure-reporting
changes in the current source. The earlier ac4 pass and the
byte-preserving b940 extraction remain historical records under their
original conditions; neither failed attempts nor prior source conditions
are regraded.
- Causal controls: six assertions failed on the old behavior, then all
62 focused controls passed after the repair. Local: 834 related support
tests, 41 shared chat/output-reader tests, and 26 unchanged real
component controls passed. Runner E2E typecheck, recursive typecheck,
build, maps, token gates, and `git diff --check` passed. The temporary
output-reader type mismatch exposed by typecheck was corrected by
narrowing the capability type.
- The first restricted build/typecheck invocations failed because local
tsx IPC was denied. A restricted support run also lacked the report CLI
output. The same source passed with normal owned local IPC allowed; the
original logs remain retained.
- The original head's hosted e2e shard 2/8 failed at the unchanged
mobile preference frame-count test: expected more than five frames,
observed five. The cause is unknown. Its original log remains retained;
it is not labeled a proven flake. No CI rerun was requested.
- Current-head full CI run 41580423918, attempt 1, passed. The fresh
review is 5/5 with zero unresolved threads. The broad local suite was
not repeated. The earlier full local run recorded in #18643 had four
failures and did not reach 164 serialized groups; this PR does not claim
an all-local pass.
- Prior failed routine attempts remain recorded. They exposed composer
focus, collapsed schedule controls, pending browser events, and Markdown
serialization problems. The measured later pass does not regrade those
attempts.

## Risks

- The shared browser-event helper also serves task creation and project
creation. Its regression tests and existing consumers remain included.
- The staging pass qualifies the captured source and configuration. The
coordinator retains source merging, daily pinning, and plan selection.
- The repository test and the unpassed completion test are not
registered by this PR.

## Model Used

OpenAI GPT-6 through Codex, with tool use and code execution. The exact
deployment ID and context window are not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub issue/PR references)

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
canary/v2026.1103.0-canary.5
2026-11-02 23:25:56 -06:00
DottaandPaperclip 4461905b92 fix(evals): bind repair launcher and Source3 candidate contracts (#18726)
## Thinking Path

> - Paperclip manages AI agents and their work.
> - The private chat catalog tests real user requests through TestOps
and AWS.
> - Each test must bind its immutable candidate and private evidence
owner before it receives model credentials.
> - The previous source pin changed the candidate launcher but kept the
old launcher digest in the full-catalog evidence policy.
> - The private guard rejected the 23-case launch before any original
test or model-key step began.
> - This pull request binds the full-catalog launcher to its reviewed
candidate and selects the next reviewed repair source.
> - The result is an explicit source contract for new measurements, with
the old grades and privacy gates preserved.

## Linked Issues or Issue Description

Refs #18721. Related: #18710 and #18717. The core guidance PR #18717
remains unmerged. A duplicate search found no open PR for this launcher
binding defect.

**What happened?**

The full-catalog evidence policy used the legacy pilot launcher digest
after the candidate launcher changed. All first eight case jobs failed
the private binding step. Their original Execute and model-key steps
were skipped. The operator cancelled the remaining jobs once. This was
an operational failure, not a feature result.

**Expected behavior**

Every singleton policy must use the launcher digest from the same
reviewed manifest as the immutable full-catalog target. A final source
check must compare the digest and private protocol files with the exact
Git target before submission.

**Steps to reproduce**

Compare the old evidencePolicy digest with launch.ts from target
36dff2e. The old owning regression fails. The corrected full-lane policy
emits the digest of launch.ts from target 43c44ccc. The legacy pilot
digest remains unchanged.

## What Changed

- Pin the full catalog to candidate
`43c44ccc823a8c972b247fa7159281887e6f5255` and its exact launcher
digest. Keep the strict private prepare guard and pilot digest
unchanged.
- Bind the actual 242-row candidate descriptor snapshot and the explicit
changed definition fields. Preserve all 244 case IDs, aliases and
original definition provenance.
- Select the reviewed source repairs for skill import guidance,
immutable skill versions, app review navigation, authorized app testing,
adapter prerequisites, the isolated announcement fixture and bounded
diagnostics. These are new candidate measurements.
- Record the one native-accounting change: Studio144 has a 2..3 envelope
and requires three genuine settled source-bound turns. Its query, model,
human count and 540-second case limit stay fixed. All other native
bounds and all original case clocks stay fixed.
- Keep the SDK scanner, compiler/parser identities and canary
certificates unchanged. The public SDK source-input registry is
synchronized; this does not certify the runtime or release held SDK
cases.
- Update the current-target documentation and preserve dated source
history.

## Verification

- The old full-catalog policy fails the new regression with the stale
launcher digest. The corrected policy passes for all 244 singleton
declarations.
- Current CI tests hash the checked exact candidate launcher source
bytes and bind their target and Git blob OID. The previous test accepted
an advanced target with a stale launcher; the corrected test rejects
that mutation. The candidate remains unchanged.
- 62 focused Node controls pass across the manifest, API adapter,
reconciliation and trusted contract owners.
- The source-bound final-GO helper reads the exact Git target launcher
and all three private protocol files. It verifies every singleton policy
against those bytes. All 244 pass.
- The unchanged actual private prepare command passes with the exact
candidate bytes. Mutated launcher bytes, a missing launcher and a
wrong-target policy fail before outputs. No service or provider
credentials are passed in these controls.
- Candidate verification: 294 pure controls, eight Git/FS updater
controls, actual 242 factory declarations with zero fixture callbacks,
and 42 admission/legacy controls pass. SDK registry check reports no
change on the final candidate.
- Prior head6bda80f passed PR CI41577494646 on attempt1:48 jobs passed
and1 intentionally skipped. Both required checks and Greptile5/5 passed.
Two late review findings then required declaration synchronization and
removal of an unused exported helper. Final head09719566 then passed PR
CI41579310890 on attempt1:48 jobs passed and1 intentionally skipped.
Both required checks and exact-head Greptile5/5 pass. The whole ten-file
diff received independent review after both late findings were fixed.
- One actual TypeScript declaration consumer reproduces
missing-export/property errors on the prior declarations and passes
after synchronization (installed TypeScript6.0.3; isolated consumer
only). The finalGO keeps direct exact-source comparisons; no unused
guard export remains.
- Local full graph typecheck/build and native browser/model execution
were not run for this trusted metadata change. Workspace dependency
output prevents a clean local full typecheck claim. Latest-head CI is
required. No new feature PASS is claimed from these controls.

## Risks

- This advances a private evaluation candidate. It does not merge the
candidate's core guidance changes to master. Existing result history and
the catalog qualification gate remain unchanged.
- The new Studio144 accounting and app492 context/outcome are explicit
changed conditions. They must not be credited as passes under the
original definitions.
- SDK input synchronization preserves all scanner guards. Runtime canary
certificates still need real qualification; the other five SDK cases
remain held.
- The isolated announcement fixture contains one public synthetic JSON
object. Production announcement feeds and the global index are
unchanged.
- The cancelled 23-case launch must be reported as operational BROKEN
rows with unknown original grades. It must not be retried under the
unchanged source.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) assisted with source integration,
code, tests and review. The model used tool execution and repository
inspection. The host did not report the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-02 22:54:16 -06:00
DottaandPaperclip 37f27fe999 test(evals): synchronize reviewed price and plan source4 declarations
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 22:17:05 -06:00
DottaandPaperclip d272fcff5a test(evals): bind plan decisions to actual source deliveries
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 22:16:25 -06:00
DottaandPaperclip 64da56f835 test(evals): use a genuinely pricier hire proposal fixture
Use the public Pro configuration as the incidental starting proposal, verified against current official prices and every selected 1017 rate variant. Keep all comparison, catalog, identity, configuration and pending-approval guards unchanged. Record the starting-model change as definition v3, never a historical regrade.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 22:16:24 -06:00
Dotta fea6163811 test(evals): pin synchronized chat repair declarations and definitions (#18721)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The chat catalog runs a fixed candidate through the trusted TestOps
lane.
> - The first repair source exposed fixture, observation and
evidence-readiness defects.
> - Changed declarations also need matching static runtime descriptors.
> - This pull request pins one corrected candidate and records each
changed definition.
> - New measurements keep their source and original outcomes separate
from past results.

## Linked Issues or Issue Description

Refs: #18710, #18693, #18680, #18647.

**What happened?**

The first repair candidate left five static descriptors out of sync with
its actual declarations. The real no-callback admission check rejects
them. A five-case attempt was cancelled before any case or model step.
The earlier pin note overstated remote coverage: importing the catalog
did not load every actual factory.

**Expected behavior**

Every repaired declaration must pass the real factory admission check
before a new batch. The trusted pin must record exact source and
definition changes. Existing case identities, prior measurements and the
full-catalog pass gate must remain intact.

**Steps to reproduce**

Run `node --import ./cli/node_modules/tsx/dist/loader.mjs
tests/runner-e2e/chat-catalog-consistency.ts` on the old candidate. It
rejects five declarations. The corrected candidate admits all 242
declarations without invoking a fixture. The existing legacy-cell
scoring and selection controls also pass.

## What Changed

- Pin candidate `36dff2eab3dfb6de6efb4fd2ce6c1a58f8a5f383`. Its
[candidate-only
delta](https://github.com/paperclipai/paperclip/compare/9ab0f951a5b5990c266ef4456610e0a0775aa334...36dff2eab3dfb6de6efb4fd2ce6c1a58f8a5f383)
remains separate from master application code.
- Record the real descriptor digest and all 19 declaration-field changes
across 18 cases. Check them against the exact candidate static
descriptor snapshot, including setup policies. The runtime loader still
rejects drift.
- Keep original definition values and record 19 manifest-field changes
across 15 cases. These include the clarified Fast response request,
reviewed goal/skill navigation outcomes, and scoped oracle/source
ownership changes.
- Keep unchanged all 244 ordered identities, model, case clocks,
native-run bounds, private evidence rules, workflows and reporting
authority. The original ledger and historical results are unchanged.
- Require actual factory admission and legacy controls before each
repair-batch GO. SDK audit capture remains private diagnosis only; it
does not waive scanner rejection or qualify the other SDK cases.

## Verification

- Candidate focused controls: 561 passed across 32 pure test files.
- Real lazy factory admission: 242 declarations passed, with zero
fixture callbacks. The prior source failed 18 declarations before
descriptor sync. The two-file admission and legacy scoring suite passed
all 42 controls.
- Trusted manifest, reconciliation and adapter controls: 38 passed.
Contract and dispatch controls: 23 passed.
- Independent source reviews cover the repair slices, descriptor sync
and this metadata pin.
- `git diff --check` passed.
- No local app, DB, native or model run was started for this metadata
PR. Full harness typecheck/build is not qualified: existing workspace
dependency errors remain outside these pure checks. The established
credential-free remote Runner build is still required before paid
queries. It does not typecheck the full harness graph.
- Initial head 90d4ecf passed first-attempt CI (48 passed, 1 skipped)
and Greptile 5/5. A later review found stale current-target prose and an
unchecked setup-policy provenance assertion. The corrected head adds the
exact candidate descriptor snapshot, computes its digest, and compares
every declared field. A wrong-policy mutant accepted by the parent is
now rejected, as is a stale digest. Correct metadata controls pass all
38 cases. Latest-head CI41572232196 passed48 jobs with1 skip after one
approved failed-only recovery. The first attempt failed one browser
cleanup cancellation with PostgreSQL row-lock timeout; the lock holder
remains unknown. No server or timeout change was made. All46 successful
executions kept their original full step and execution timestamps. Only
the failed browser shard and dependent e2e summary reran. Both required
checks passed, current-head Greptile is5/5 and all review threads are
resolved. An optional review may remain queued; no claim is made that
all optional checks finished. No corrected-candidate feature pass is
claimed.

## Risks

The candidate changes selected setup, observation and oracle conditions.
Each new result must retain its exact source and declared changes; it
cannot replace an old failed result. The Fast response request and
goal/skill outcome changes are explicit new definitions. Static TestOps
prose may still show the original definition until a separate supported
metadata reconciliation. SDK scanner findings remain unknown until the
private audit is captured; the scanner and evidence bounds remain
enforced.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, source editing and
tool use. The exact context window is not exposed by this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
following the issue template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All required Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
canary/v2026.1103.0-canary.4
2026-11-02 21:55:33 -06:00
Dotta 43c44ccc82 test(evals): bind app team and announcement repair declarations 2026-11-02 21:53:05 -06:00
DottaandPaperclip 6743055493 test(evals): reject explicit missing target access reports
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:49:34 -06:00
DottaandPaperclip a74bbc949b test(evals): reject contradictory app test reports
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:49:34 -06:00
DottaandPaperclip b23439902a test(evals): qualify authorized target-scoped app tests
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:49:34 -06:00
DottaandPaperclip 4837c96544 test(evals): scope announcement dismissal staging input
Enable only the singleton dismissal journey against a reviewed public synthetic staging feed. Require its real server publication before any chat and preserve actual human-only personal dismissal and audience checks. Publication is a separate explicit operator action.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:49:34 -06:00
DottaandPaperclip d12be2755b test(evals): recognize exact team account probe refusal
Accept the documented OpenCode auth-required warning as a missing-account prerequisite while keeping a distinct member strictly ready. Retain only the existing closed probe projection before setup refusal; no new probes, model calls or clock changes.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:49:33 -06:00
DottaandPaperclip 1e3ab690b0 fix(routines): prevent recursive Sentry run repairs (#18714)
Validate Sentry failed-run alerts before routine dispatch. Reject repair-generated provenance and private sources. Deduplicate source runs through stable primary-key receipts, including setup retries and suppressed deliveries.

Verified with 166 focused tests, server typecheck and build, generated-reference checks, passing required CI, and Greptile 5/5.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:37:11 -06:00
Dotta 7ff7bacadf test(evals): synchronize next repair declarations and Studio accounting 2026-11-02 21:35:30 -06:00
Dotta f8b35b3318 fix(ui): retain Slack conversation handoff after setup (#18722)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Slack setup connects a person with an agent in Slack.
> - Welcome delivery can complete setup before the person chooses Open
Slack.
> - Success and saved settings then offer no link to the conversation.
> - This pull request retains Open Slack in both views.
> - The benefit is that people can use the connected bot without finding
its URL elsewhere.

## Linked Issues or Issue Description

Refs: #18707. This is the next problem found during its deployed user
walkthrough.

**What happened?**

Managed setup completed after welcome delivery. The Open Slack link
disappeared. Success offered Done, and saved settings offered only Slack
app configuration. The bot could answer, but the handoff to its
conversation was lost.

**Expected behavior**

A person can open the connected bot from Success and from the saved
connection.

**Steps to reproduce**

1. Complete managed Slack setup with welcome delivery.
2. Wait for automatic Success before choosing Open Slack.
3. Choose Done and open the saved connection. No conversation link is
available.

## What Changed

- Retain Open Slack on Success while keeping the centered Done action.
- Add Open Slack to completed Slack connection pages. Keep app
configuration separate.
- Use a saved DM only when its linked owner matches the current
signed-in account. Other managers and unknown identities use the
provider-link or Slack fallback.
- Update the existing component regressions, three Storybook cases,
setup guidance and feature recipe.

The Slack UI owns the shared URL helper. This change adds no API or
permission contract and leaves unfinished setup on its existing path.

## Verification

- Focused Slack component and prompt suites: 40 passed. All related chat
UI suites: 269 passed across 19 files. The settings regression switches
from the installer to a second user and then to a signed-out state.
- Repository typecheck and production build: passed.
- Token gates, map structure and Storybook catalog checks: passed. The
map/catalog Node suite passed all 30 tests; these are structural checks.
- Storybook managed and own-app Success play assertions passed. The
managed mobile view retained readable Open Slack and Done controls after
scrolling. Saved settings play assertions passed after a fresh page
load. Fixtures simulate Slack; no live provider actions were sent from
Storybook.
- The first local typecheck rejected an unsupported Storybook assertion
option. It passed after using an exact-name regular expression. The
settings story first hit a stale Vite dynamic-import cache; a normal
reload fetched the current module and all settings play assertions
passed. Restarting the owned review server later emitted Storybook
manager state timeouts; all three production-page play cases passed
again on the corrected head.
- The preceding deployed fix produced real Slack DM and thread replies.
This follow-up still needs deployed acceptance through its visible Open
Slack link.
- Greptile first found that a second manager could receive the
installer's DM link. The correction checks the linked owner against the
resolved account and falls back while identity is unknown. The two
threads have replies and are resolved. Greptile reviewed corrected head
94a6946719 at 5/5 with no new findings.
The optional Paperclip agent check ended neutral without a complete
assessment; one fresh review was requested. No fresh secondary
assessment was available at merge.
- [All required CI suites
passed](https://github.com/paperclipai/paperclip/actions/runs/41571276944)
on corrected head 94a6946719. Full local
Vitest was not repeated for this UI-only follow-up.

## Risks

- Low risk. This adds navigation to existing Slack destinations.
- Old connections without a known DM use their provider link or Slack.
- No schema, credential, scope, identity-link or transport changes. The
UI consumes the existing shared account identity query.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository editing, browser
use and command execution. The runtime does not expose a more precise
model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-11-02 21:33:59 -06:00
Dotta 43813e06d0 test(evals): bind SDK public proof to the evaluated source tree 2026-11-02 21:33:04 -06:00
DottaandPaperclip 9ce8b0d274 test(evals): synchronize SDK public source pins offline
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:28:36 -06:00
DottaandPaperclip af3b925d04 test(evals): account for actual Studio result delivery
Read immutable pinned version detail and require the source-bound native completion turn. Preserve the original query, model, 540-second attempt and actual failing-sample output requirements.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:28:36 -06:00
DottaandPaperclip a5510186ae test(evals): retain closed packaging budget diagnostics
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:16:38 -06:00
DottaandPaperclip d432bc425b test(evals): qualify skill update reads before model admission
Require the actual pinned update-status prerequisite before the canonical request. Add an exact human-session GET reader that bounds error-body parsing and preserves only three source-owned reason codes and finite provider statuses; generic response-body withholding is unchanged. Fail closed on unknown, oversized, malformed or private error content. Stop retrying typed update-read refusals while retaining all real source, review, update, version and fork assertions. This improves diagnosis and eligibility, not feature grading.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:16:38 -06:00
DottaandPaperclip e55b3b025f test(evals): respect app review preview content
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:16:38 -06:00
DottaandPaperclip 481722349e fix(evals): open scoped app review in composer takeover
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 21:16:38 -06:00
Dotta 486cde8748 fix(candidate): clarify authored skills versus source imports 2026-11-02 21:14:43 -06:00
Dotta 36dff2eab3 fix(evals): synchronize repaired declarations before catalog admission 2026-11-02 21:02:29 -06:00
DottaandPaperclip 65a3e8eef8 fix(evals): retain SDK scan audit after bootstrap rejection
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 20:58:49 -06:00
DottaandPaperclip 462af87e84 test(evals): wait for settled primary Chat evidence
Require the actual recent CEO conversation, original visible human and native reply, and editable composer after navigation and final profile proof. Re-read the persisted Morgan preference and exact original chat/run set after readiness. Use only remaining original case budget and preserve all existing primary/privacy grading. Calibrate the actual final verifier with loading, late side-effect, wrong identity and exhausted-budget negatives.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 20:58:49 -06:00
DottaandPaperclip b7dfdbbefd test(evals): preserve consumed commentary identity response
Dispose only unread browser-session response bodies. Exercise the actual callback with real Node Responses so successful JSON is not followed by locked-stream cancellation. Preserve real reader and owner identity checks, restoration, denied-body disposal, malformed JSON failures, feedback privacy oracles, and original clocks.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 20:58:49 -06:00
DottaandPaperclip 0ecd43035a fix(evals): bound status-card setup by original case clock
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 20:58:49 -06:00
DottaandPaperclip fa644b9b20 test(evals): bind deferred question delivery to its actual successor
Accept a null-target durable wake receipt only with the unique succeeded answer-bound continuation from the server response path. Keep actual human answer, company/task/agent, ordering, duplicate conflict, saved output and full run accounting checks. Version the observer and retain failed history.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 20:58:49 -06:00
DottaandPaperclip 41743cb0ae test(evals): observe required question validation on Next
Replace the disabled-button assumption with real invalid-click validation, no page advance, no saved answer and no new work. Keep all subsequent exact answers, page retention and continuation checks. Calibrate the real QuestionForm with a pure React test; no product UI behavior changes.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 20:58:49 -06:00
DottaandPaperclip 12e6def200 test(evals): mutate readonly context only through fixture setup
Keep the exhausted-budget negative while respecting the readonly public context type. No production observer or grader changes.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 20:58:49 -06:00
DottaandPaperclip bd4dc1f10b test(evals): inspect authored skills through normal Studio navigation
For the reusable-skill request, prove ordinary library discovery, editable Studio checklist content and reload with the same saved version instead of prescribing a chat-link format. Keep current-version/content/attribution/assignment checks and all other skill contracts unchanged; record the explicit authoring v2 conditions.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-11-02 20:58:49 -06:00