Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-08 21:03:51 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
codex/plugin-task-execution
PaperClipAI/doc/connections/RAILWAY-REVIEW.md
T
Devin FoleyandPaperclip 6fe8e30625 feat(apps): add Railway connection and governed deployment tools (#13415) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Apps gives agents governed access to external resources.
> - Operators need to inspect Railway services, read logs, deploy code,
and run container commands.
> - Railway offers hosted MCP with OAuth, but broad remote actions hide
their internal operations.
> - This PR adds a branded connection and fixed direct operations
through the existing gateway.
> - Separate SSH keys enable container commands under the same grants
and policies.
> - Operators can require approval for an action and inspect the
resulting audit record.

## Linked Issues or Issue Description

**Subsystem affected**

Apps catalog, connection setup, gateway execution, and connection
documentation.

**Problem or motivation**

Agents need Railway access through Paperclip. Operators need to grant
and revoke that access, inspect available actions, and govern deployment
and container operations without giving agents provider credentials.

**Proposed solution**

Reuse hosted MCP OAuth, vault storage, catalog discovery, grants, and
the gateway. Probe the actual credential before enabling fixed GraphQL
operations. Use a dedicated grant-owned SSH key for bounded container
commands.

**Alternatives considered**

A catalog entry alone cannot execute the missing operations. The hosted
general agent has opaque internal effects. An unrestricted CLI runtime
can bypass action policy and inherit ambient credentials.

**Roadmap alignment**

This extends the existing MCP Tool Gateway & Apps path and the Connected
Apps direction in ROADMAP.md. It does not add a plugin or parallel
connection service.

Related PRs #311, #939, and #7861 concern hosting Paperclip on Railway.
They do not add this outbound Apps connection. The separate shared
agent-picker fix is #13414 and is not included here.

## What Changed

- Add the generated Railway catalog entry, official marks, provenance,
and OAuth setup guidance.
- Add fixed service/deployment status, bounded logs, and
redeploy/restart/rollback tools. Block source deployment until the
provider can atomically bind the approved repository and commit.
- Verify API access with an explicit workspace before exposing direct
tools.
- Add grant-owned SSH key setup and a bounded runner with host
verification, target checks, isolated state, and cleanup.
- Block the opaque hosted railway-agent and accept-deploy actions.
Preserve normal Allowed defaults and Ask-first policies for other
actions.
- Quarantine new or changed Railway schemas after initial discovery,
including reconnect.
- Add provider, lifecycle, gateway, SSH, UI, and browser fixtures.
Document setup, limitations, and the release checklist.

## Verification

- Security follow-up: removed the unsafe source-deployment mutation.
Direct calls and old active catalog entries are denied before any
upstream request, including normalized aliases. Refresh marks retired
entries disabled. All 386 focused Railway, catalog and gateway tests
passed, and server TypeScript checking passed. Full [GitHub
CI](https://github.com/paperclipai/paperclip/actions/runs/35139421144)
passed on d86530ab9, including typecheck, build, all tests, runner
checks, and browser tests. Superagent passed and confirmed the P2 fix.
Greptile reviewed the same commit at 5/5 with no findings.

- CI follow-up: fixed the missing Railway SSH operation in the OpenAPI
document, including its request schema, operator-only authentication,
and error responses. The failure reproduced locally before the fix; all
403 selected API, Railway, catalog, and artwork tests passed after it.
Synced current master and resolved the catalog/artwork conflicts.

- After rebase: 440 focused provider, lifecycle, gateway, catalog, and
container-panel tests passed. AppDetail and AppsConnect passed another
196 tests.
- Full typecheck, build, token gates, and the gallery browser check
passed after rebase.
- During implementation, full build and the gallery browser check
passed. Shared generic-MCP fixtures covered OAuth callback/state/issuer
binding and failure paths.
- Local live consent and tools/list succeeded. There were 44 active
hosted actions and two blocked actions. A workspace-bound API probe and
direct project/service/environment reads succeeded. The inspected
project had no deployed services. No provider mutation ran.
- Full GitHub CI passed on commit 303340f19, including all
server/workspace test groups, typecheck, build, runtime verification,
release dry run, and browser tests. The original local full-run attempt
was incomplete; the complete automated suite is now verified in CI.

Manual review: connect Railway, review the actual actions, install for
an agent, and run a resource read through the gateway. Choose Ask first
before testing a deployment mutation. Configure a dedicated key only
when container access is needed.

**Release qualification is still open.** Live agent gateway reads/logs,
rejected and approved deployment calls, refresh/revoke, public HTTPS
consent, and SSH enrollment/commands/cleanup need an authorized
disposable service. The passing API diagnostic does not replace those
tests. See doc/connections/RAILWAY.md and RAILWAY-REVIEW.md.

## Risks

Overall risk is medium. New runtime behavior is gated to Railway
connections, but the PR changes shared catalog, credential lifecycle,
and gateway code. A regression in those paths can affect other Apps
connections. The highest-impact operations are Railway deployments and
container commands.

- Provider consent can authorize an entire workspace. Catalog labels are
not local resource allowlists. Direct tools check target membership, and
provider permissions still apply.
- Shell commands have broad internal authority. Action policy cannot
approve each internal shell step. Timeouts close the local connection
but cannot guarantee remote child-process termination.
- Log and command output may contain application secrets that pattern
redaction cannot recognize.
- Source deployment is unavailable until the provider supports atomic
repository/commit binding. Existing deployments can still be redeployed,
restarted or rolled back.
- No database migration is required. Rollback can remove promotion and
direct dispatch while preserving connection data and the generic MCP
path.
- Live Railway qualification must still pass before release acceptance.

## Model Used

OpenAI Codex, based on GPT-6, with code execution and browser testing.
An independent read-only security agent reviewed the local
implementation. The exact serving model ID and context window were not
exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-16 13:54:44 -07:00

156 lines
8.4 KiB
Markdown
Raw Permalink Blame History

# Railway implementation verification
Updated: 2026-09-16. Implementation and local verification were performed on
2026-09-13. The change is published for review on a dedicated branch.
Live qualification remains open. Full GitHub CI passed on commit `303340f19`
before the source-deployment security follow-up below.
## Thinking Path
> - Paperclip controls the access that agents receive to external resources.
> - Apps already supplies remote MCP, OAuth, vault storage, grants and policies.
> - Operators need Railway service inspection, logs, deployments and container commands.
> - The hosted Railway server alone does not supply narrow governed tools for all these operations.
> - This change adds a branded connector and fixed direct operations inside the existing gateway.
> - Dedicated grant keys enable bounded commands in deployed containers.
> - Operators keep the existing action defaults and can require approval before execution.
## Linked Issues or Issue Description
**Subsystem affected**
Apps catalog, connection setup, gateway execution, shared contracts and connection documentation.
**Problem or motivation**
An operator needs to authorize a Railway account once, grant access to selected
agents, and let them inspect and operate Railway resources through Paperclip.
**Proposed solution**
Use hosted OAuth for connection setup. Expose direct status/log/deployment tools
only after an actual API credential probe. Add separate container-key setup and
a fixed OpenSSH runner behind the same grant and policy checks.
**Alternatives considered**
A manifest alone cannot execute missing operational tools. The hosted general
agent has opaque internal effects. An unrestricted CLI runtime would bypass
per-action review and could inherit ambient credentials.
## What Changed
- Added Railway's generated definition, curated entry, official marks and provenance.
- Added fixed GraphQL operations for service/deployment status, bounded logs,
and redeploy/restart/rollback. Source deployment is blocked pending atomic
provider repository/revision binding.
- Added grant-owned SSH key setup and container commands with host verification,
target checks, deadlines, output caps and cleanup.
- Blocked the hosted general agent and staged-change acceptance. Preserved normal
Allowed defaults and Ask-first policies. Kept changed-schema quarantine across reconnect.
- Added setup guidance, provider fixtures, lifecycle/SSH/gateway tests and browser verification.
## Verification
Security follow-up on 2026-09-16: removed the source-deployment schema and mutation.
The provider block applies to old active catalog entries and normalized aliases
before refresh; refresh marks them disabled. Regression tests cover direct-client
and gateway denial before any upstream request. A repository preflight is no
longer used as authorization for source deployment.
All 386 focused Railway, catalog and gateway tests passed for this follow-up.
After rebase onto master on 2026-09-14, 440 focused provider, connection, gateway,
catalog and container-panel tests passed. The AppDetail and AppsConnect suites
passed another 196 tests. The new Apps entries on master are preserved.
Passing checks observed during the original implementation:
- 284 tests across the final Railway API, SSH, lifecycle, tool-access service and shared-definition suites.
- 59 generic-MCP tests. These cover callback/state/issuer binding and OAuth error paths.
- 62 gateway tests and 3 container-panel tests.
- AppDetail and AppsConnect UI suites passed as part of a 224-test targeted run.
- The browser test `tests/e2e/railway-catalog.spec.ts` passed against a throwaway
instance. It checks the real gallery, logo, OAuth setup entry and visible limitations.
It does not complete Railway account consent.
- `pnpm -r typecheck`, `pnpm build`, and `pnpm check:token-gates` passed.
- Local port 3100 serves the dev checkout, health reports `ok`, bootstrap is ready,
the Railway brand asset returns 200, and the public OAuth metadata advertises
the loopback callback. An unauthenticated browser reaches the sign-in page.
The full `pnpm test:run` attempt was stopped after failures outside the focused
Railway coverage and repeated database-startup timeouts. One related gallery
count expectation was fixed and the complete tool-access suite subsequently
passed. Other failures included five chat integration timeouts, three company
skills cases, native session-resume fixtures, and the CLI guidance scan finding
an existing local `.claude/settings.local.json` command. Native and CLI failures
were reproduced separately; no unrelated source or private settings were changed.
The full suite is **not green**, and later runner shards did not complete.
The pinned Rust 1.97.1 toolchain was used for full typecheck/build. Browser
output and detailed logs are retained locally as ignored QA output. No provider
credentials, private configuration, or unsanitized live captures are committed.
Follow-up after the operator connected locally: loopback consent and actual
tools/list succeeded. The initial direct API probe incorrectly requested projects
without a workspace ID. Railway returned HTTP 200 with a `Not Authorized` GraphQL
error; the same token accepted a query bound to the selected workspace. The fix
discovers a workspace through the hosted read, requires a workspace ID for direct
project listing, and recognizes GraphQL authorization errors without exposing
provider details or requesting broader OAuth scopes.
The repaired local connection reports direct API access available. Its 44 hosted
actions remain active; the 12 newly discovered direct actions remain quarantined
for review. Direct project, service and environment reads succeeded; the inspected
project has no services. No deployment or container operation was attempted.
The follow-up Railway suites passed 30 tests, the shared tool-access/gateway suites
passed 293 tests, and server TypeScript checking passed.
The live evidence contains only status and resource counts; it remains local.
Agent gateway proof, disposable service/deployment targets, HTTPS/customer-client
registration, logs, deployment, SSH enrollment/host trust, refresh and provider
cleanup still require operator-assisted proof. See
[RAILWAY.md](RAILWAY.md) for the exact release checklist and setup.
## Risks
- Live provider qualification remains outstanding. Advertised DCR is not proof
that a particular account/client/callback combination works.
- OAuth authorization can cover more resources than one service. Metadata filters
are not local authorization allowlists.
- Container commands have broad internal authority; timeout cannot guarantee remote
child termination. Provider-side key removal is a separate operator action.
- Source deployment is unavailable until the provider can atomically bind the
approved repository and commit. Existing deployments can still be redeployed,
restarted or rolled back.
- No schema migration is needed. Rollback can remove promotion and direct dispatch
while retaining connection data and the generic MCP path.
- The full test suite must be resolved before claiming release readiness.
## Model Used
OpenAI Codex, based on GPT-6, with tool execution and a separate read-only security
review agent. The exact serving deployment ID and context window were not exposed
in this session. The independent reviewer found no remaining concrete blocker
for a local preview; this is not a claim of completed live qualification.
## Checklist
- [x] I have included a thinking path that traces from project context to this change
- [x] I have specified the model used, with available version and capability details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked them above
- [x] I have described the issue using the feature-request fields
- [x] I have not referenced internal Paperclip issues
- [x] My branch name describes the change
- [ ] I have run all tests locally and they pass
- [x] I have added or updated relevant tests
- [x] I have updated the connection documentation
- [x] I have documented the risks
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open follow-ups
- [x] I will address all reviewer comments before requesting merge
Unchecked release checks remain pending. Related Railway PRs #311, #939 and
#7861 concern hosting Paperclip on Railway, not governing Railway through Apps.
This implementation uses the existing governed Apps path described in ROADMAP.md.
Reference in new issue View Git Blame Copy Permalink