security(operator): fail closed without OPERATOR_API_TOKEN #190

Open
opened 2026-09-12 13:11:28 +00:00 by PlasticDigits · 0 comments

Summary

packages/operator/src/api.rs treats OPERATOR_API_TOKEN as optional. start_api_server loads it with std::env::var(...).ok().filter(|t| !t.is_empty()) and stores Option<Arc<str>>. check_auth returns true when the option is None or empty. GET /status and GET /pending therefore serve queue data with no Authorization header whenever the env var is missing.

That is fail-open. Production-like config (.env.example does not require the var; unit test test_check_auth_no_token_configured asserts the open path) still binds those routes.

Module docs state the same: /status and /pending are “auth-gated when OPERATOR_API_TOKEN is set”. Closed O2 in packages/operator/security_reviews/SECURITY_REVIEW_2026-02-12.md marked optional bearer gating as “Fixed”. Residual: unset still means unauthenticated.

Default listen is 127.0.0.1:9092. OPERATOR_API_BIND_ADDRESS can be 0.0.0.0. The API has no TLS (packages/operator/README.md). Rate limits (RATE_LIMIT_PER_SECOND / burst) do not replace auth.

This is not #115 (RPC quorum / /health staleness). This is not #178 (EVM guardBridge / rateLimitBridge zero-address). This is not #184 (EVM deposit nonce lookup). This is not #187 (CI skip fail-closed). Keyword overlap on “fail closed” / “operator” / “pending” is not this bug.

Internal review id: RS-M3 (medium). Still in source as of 2026-09-12 (packages/operator/src/api.rs on main).

Bundle (same ticket, do not split):

  1. Fail startup (do not bind the HTTP API) when OPERATOR_API_TOKEN is missing or empty, unless an explicit local/dev flag is set.
  2. Stop encoding optional-token as success in check_auth and in test_check_auth_no_token_configured.
  3. Tests that production-like config (no token, no dev flag) does not serve /status or /pending as unauthenticated JSON. Local/E2E must set a test token (preferred) or the documented dev flag — not leave the routes open.
  4. Docs / .env.example / security-review residual: token is required in production; optional O2 is not a complete fix.

Founder-required operator auth / keys. No community autoland. Do not add ready.

Impact (today vs hypothetical)

Auth and operational confidentiality at risk today in source, and on any live operator whose process started without OPERATOR_API_TOKEN. /pending returns up to 50 pending/submitted approvals and releases with id, nonce, recipient, amount, status. /status returns pending deposit/approval/release counts. That is enough to map in-flight bridges (who, how much, which nonce) without holding an operator key.

This is not a user-facing puzzle that needs an exploit recipe. Anyone who can reach the listen address can GET those paths. Default bind is loopback, so remote reach needs a non-local bind, a proxy, or a local process. Source still permits OPERATOR_API_BIND_ADDRESS=0.0.0.0 and still serves the privileged routes with no token. Docs already warn about plaintext when the token is set; they do not refuse to start when it is not.

Hypothetical-only if every production process already has a non-empty token and cannot bind without it. Source and unit tests still treat absence as open access. Sticky until startup refuses the empty path and tests prove production-like config never registers unauthenticated /status / /pending.

Do not publish a live-operator probe, bind inventory, or copy-paste request against a public listen address.

check_auth fail-open

fn check_auth(headers, required_token: Option<&str>) -> bool {
  match required_token {
    Some(t) if !t.is_empty() => t,  // then compare Bearer
    _ => return true,               // no token configured → allow
  }
}

status_handler / pending_handler call this before any DB read. Missing Authorization is 401 only when a token was loaded.

Startup never requires the var

start_api_server logs only when the token is set. Empty and unset both become None. Router always .route("/status", ...).route("/pending", ...). Bind happens either way.

.env.example documents DB, RPC, keys, and rate limits. It does not list OPERATOR_API_TOKEN. A production copy of the example starts with open privileged routes.

Unit tests lock in the vuln

test_check_auth_no_token_configured asserts check_auth(&empty, None) and check_auth(&empty, Some("")) are true. There is no test that production-like config refuses to listen, or that /status / /pending are unregistered without a token.

Invariants

  • INV-OP-A1 (new): Production-like operator start (no documented local-only unauthenticated flag) must not bind GET /status or GET /pending unless OPERATOR_API_TOKEN is non-empty. Missing/whitespace-only token → fail startup of the API server (do not listen).
  • INV-OP-A2 (new): When a token is configured, unauthenticated or wrong-Bearer GET /status and GET /pending are 401 and return no queue JSON (no nonce, recipient, or amount).
  • INV-OP-A3 (new): GET /health and GET /metrics may stay public liveness/scrape surfaces. They must not grow queue rows or recipient/amount fields.
  • INV-OP-A4 (new): An explicit local/dev flag may allow process start without a token for Anvil/E2E. That flag must be named, documented, and default-off. Production-like tests must not set it. Prefer giving E2E a test bearer over leaving routes open.
  • Do not weaken dest WithdrawApprove, watcher finality, or cancel windows. Do not put live tokens in fixtures, docs, or issue comments. Do not require TLS inside this ticket (README already requires a terminating proxy if bound beyond localhost).

Constraints / guardrails

  • Prefer fail-closed at start_api_server / main before TcpListener::bind. Do not keep a silent “open if unset” path.
  • Empty string, whitespace-only, and unset are all missing. Do not treat Some("") as configured.
  • Keep /health (liveness / idle 503) and /metrics public unless a later ticket scopes scrape auth. Do not dump PendingResponse onto /health.
  • Dev flag: one explicit env (name it in the PR; same class as DEV_ALLOW_HTTP). Document it as local/E2E only. Do not use “bind is 127.0.0.1” as the substitute for a token in production-like tests.
  • E2E (packages/frontend/src/test/e2e-infra/operator.ts) and QA env helpers must set a test token or the dev flag. Prefer a test token so CI exercises the 401 path.
  • Constant-time Bearer compare is optional hardening, not a substitute for fail-closed startup. Do not log the token or echo it in 401 bodies.
  • Do not change Solidity / CosmWasm / Solana programs. Do not retune rate-limit numbers as the “fix”.
  • Founder-required operator API / keys. No community autoland. Do not add ready. No public live-API recipe.

Relevant files

Path Why
packages/operator/src/api.rs Optional token load; check_auth returns true when unset; /status and /pending always routed; unit tests encode fail-open
packages/operator/src/main.rs OPERATOR_API_BIND_ADDRESS / port; calls start_api_server
packages/operator/.env.example Token not listed as required
packages/operator/README.md “when OPERATOR_API_TOKEN is set”; TLS note
packages/operator/security_reviews/SECURITY_REVIEW_2026-02-12.md O2 marked Fixed as optional gating
docs/operator.md (if present) Operator API contract
packages/frontend/src/test/e2e-infra/operator.ts E2E operator env; bind loopback, no token requirement today
scripts/qa/qa-host.env QA bind defaults
  1. Require a non-empty OPERATOR_API_TOKEN before TcpListener::bind. Return an error from start_api_server (and fail process start) when it is missing, unless the documented local-only flag is set.
  2. Change check_auth: configured token required for /status and /pending. Missing/wrong Bearer → 401, empty body. Invert or delete test_check_auth_no_token_configured.
  3. Production-like test (no token, no dev flag): start_api_server errors and does not accept connections on /status or /pending. Optionally: router built for that config does not register those routes.
  4. Configured-token tests: valid Bearer still returns JSON; missing/wrong Bearer is 401 with no queue fields.
  5. With only the local flag set and no token: either do not register /status//pending, or keep them local-debug only and not used by production-like tests. E2E should set a test token.
  6. Update README, .env.example, and the O2 residual note: optional gating is not production-complete.

Acceptance criteria

  • AC1. Production-like start without OPERATOR_API_TOKEN (and without the local-only flag) does not bind GET /status or GET /pending. API server start fails closed.
  • AC2. Empty and whitespace-only OPERATOR_API_TOKEN are treated as missing (AC1).
  • AC3. With a non-empty token: no header or wrong Bearer → 401 and no nonce / recipient / amount in the body for /status and /pending.
  • AC4. With a matching Bearer: /status queue counts and /pending lists still work. /health and /metrics remain reachable without the token.
  • AC5. test_check_auth_no_token_configured no longer asserts open access as success. New tests cover AC1–AC4 without a live operator host.
  • AC6. README / .env.example / security-review residual state that production requires the token. O2 optional gating is not the end state.
  • AC7. E2E/QA operator spawn sets a test token (or the documented flag). Default example env does not leave privileged routes open.

Test plan (functional paths)

# Path Expect
T1 Start API with non-empty token; GET /status + valid Bearer 200 JSON queue counts, no 401
T2 Same process; GET /pending + valid Bearer 200 JSON; fields as today
T3 Same process; GET /status with no Authorization 401; no queue JSON
T4 Same process; GET /pending with wrong Bearer 401; no recipient/amount
T5 Same process; GET /health with no token 200 or 503 liveness only; no pending rows
T6 Same process; GET /metrics with no token Prometheus text; no pending recipient list
T7 Start with token unset, no local flag start_api_server / process errors; no listener serving T3/T4 as 200
T8 Start with OPERATOR_API_TOKEN="" or whitespace Same as T7
T9 Local flag set, no token (if kept) Documented debug behavior only; production-like tests do not use this path
T10 Existing operator unit tests Stay green except the inverted fail-open auth test

Test plan (attack, hack, and abuse)

Non-exploitative. Operator unit / in-process router tests only. Do not use these as a live listen-address recipe.

# Vector Expect
A1 Production-like config, token unset, client omits auth No 200 queue JSON from /status or /pending (no bind or fail start)
A2 Token set; Authorization missing 401, empty privileged body
A3 Token set; Authorization: Bearer + wrong secret 401
A4 Token set; Authorization: Basic … or no Bearer prefix 401 (existing test_check_auth_no_bearer_prefix)
A5 Token set; bearer vs Bearer prefix Still only the matching secret succeeds; prefix case must not open a no-token path
A6 Bind 0.0.0.0 in a test config with token missing and no local flag Still fail start; bind address is not an auth substitute
A7 Local flag accidentally set in a test named production-like Fail the test; production-like fixtures must not set the flag

Verification criteria

  • Operator package tests: T1–T10 and A1–A7. Grep that check_auth no longer returns true for None / empty in production-like builds.
  • Grep that start_api_server (or main) errors when the token is missing unless the documented local flag is set.
  • Grep .env.example / README for required OPERATOR_API_TOKEN (or an explicit “required unless <dev flag>” sentence).
  • Invert or remove test_check_auth_no_token_configured as a success case for open access.
  • Do not verify by scanning a live operator URL or by posting real Bearer values.

Out of scope

  • #115 /health idle 503 / RPC quorum / FINALITY_BLOCKS.
  • #178 on-chain guard / rate-limit wiring.
  • #184 dest-approve nonce scoping.
  • Adding TLS to the operator process (proxy remains the documented control if bound beyond localhost).
  • Authenticating /metrics scrape (optional later; not this ticket).
  • Live operator redeploy / env rotation (ops).

First-pass model recommendation

Recommendation: grok-high

Rationale: Security class and founder-required operator API auth / keys. Composer is disallowed (High/security; not a low-risk first pass). Even if the production edit is likely api.rs plus startup in main.rs plus tests/docs, file count does not establish safety: fail-open auth on /pending leaks in-flight nonces, recipients, and amounts, and the existing unit test encodes the wrong default. Verify with in-process start/router fixtures (token missing vs set), not a live operator probe.

## Summary `packages/operator/src/api.rs` treats `OPERATOR_API_TOKEN` as optional. `start_api_server` loads it with `std::env::var(...).ok().filter(|t| !t.is_empty())` and stores `Option<Arc<str>>`. `check_auth` returns `true` when the option is `None` or empty. `GET /status` and `GET /pending` therefore serve queue data with no `Authorization` header whenever the env var is missing. That is fail-open. Production-like config (`.env.example` does not require the var; unit test `test_check_auth_no_token_configured` asserts the open path) still binds those routes. Module docs state the same: `/status` and `/pending` are “auth-gated **when** `OPERATOR_API_TOKEN` is set”. Closed O2 in `packages/operator/security_reviews/SECURITY_REVIEW_2026-02-12.md` marked optional bearer gating as “Fixed”. Residual: unset still means unauthenticated. Default listen is `127.0.0.1:9092`. `OPERATOR_API_BIND_ADDRESS` can be `0.0.0.0`. The API has no TLS (`packages/operator/README.md`). Rate limits (`RATE_LIMIT_PER_SECOND` / burst) do not replace auth. This is not [#115](https://git.cl8y.com/code/cl8y-bridge-monorepo/issues/115) (RPC quorum / `/health` staleness). This is not [#178](https://git.cl8y.com/code/cl8y-bridge-monorepo/issues/178) (EVM `guardBridge` / `rateLimitBridge` zero-address). This is not [#184](https://git.cl8y.com/code/cl8y-bridge-monorepo/issues/184) (EVM deposit nonce lookup). This is not [#187](https://git.cl8y.com/code/cl8y-bridge-monorepo/issues/187) (CI skip fail-closed). Keyword overlap on “fail closed” / “operator” / “pending” is not this bug. Internal review id: RS-M3 (medium). Still in source as of 2026-09-12 (`packages/operator/src/api.rs` on `main`). Bundle (same ticket, do not split): 1. Fail startup (do not bind the HTTP API) when `OPERATOR_API_TOKEN` is missing or empty, unless an explicit local/dev flag is set. 2. Stop encoding optional-token as success in `check_auth` and in `test_check_auth_no_token_configured`. 3. Tests that production-like config (no token, no dev flag) does not serve `/status` or `/pending` as unauthenticated JSON. Local/E2E must set a test token (preferred) or the documented dev flag — not leave the routes open. 4. Docs / `.env.example` / security-review residual: token is required in production; optional O2 is not a complete fix. Founder-required operator auth / keys. No community autoland. Do not add `ready`. ## Impact (today vs hypothetical) Auth and operational confidentiality at risk **today** in source, and on any live operator whose process started without `OPERATOR_API_TOKEN`. `/pending` returns up to 50 pending/submitted approvals and releases with `id`, `nonce`, `recipient`, `amount`, `status`. `/status` returns pending deposit/approval/release counts. That is enough to map in-flight bridges (who, how much, which nonce) without holding an operator key. This is not a user-facing puzzle that needs an exploit recipe. Anyone who can reach the listen address can `GET` those paths. Default bind is loopback, so remote reach needs a non-local bind, a proxy, or a local process. Source still permits `OPERATOR_API_BIND_ADDRESS=0.0.0.0` and still serves the privileged routes with no token. Docs already warn about plaintext when the token *is* set; they do not refuse to start when it is not. Hypothetical-only if every production process already has a non-empty token **and** cannot bind without it. Source and unit tests still treat absence as open access. Sticky until startup refuses the empty path and tests prove production-like config never registers unauthenticated `/status` / `/pending`. Do not publish a live-operator probe, bind inventory, or copy-paste request against a public listen address. ### `check_auth` fail-open ```text fn check_auth(headers, required_token: Option<&str>) -> bool { match required_token { Some(t) if !t.is_empty() => t, // then compare Bearer _ => return true, // no token configured → allow } } ``` `status_handler` / `pending_handler` call this before any DB read. Missing `Authorization` is `401` only when a token was loaded. ### Startup never requires the var `start_api_server` logs only when the token **is** set. Empty and unset both become `None`. Router always `.route("/status", ...).route("/pending", ...)`. Bind happens either way. `.env.example` documents DB, RPC, keys, and rate limits. It does not list `OPERATOR_API_TOKEN`. A production copy of the example starts with open privileged routes. ### Unit tests lock in the vuln `test_check_auth_no_token_configured` asserts `check_auth(&empty, None)` and `check_auth(&empty, Some(""))` are `true`. There is no test that production-like config refuses to listen, or that `/status` / `/pending` are unregistered without a token. ## Invariants - INV-OP-A1 (new): Production-like operator start (no documented local-only unauthenticated flag) must not bind `GET /status` or `GET /pending` unless `OPERATOR_API_TOKEN` is non-empty. Missing/whitespace-only token → fail startup of the API server (do not listen). - INV-OP-A2 (new): When a token is configured, unauthenticated or wrong-Bearer `GET /status` and `GET /pending` are `401` and return no queue JSON (no nonce, recipient, or amount). - INV-OP-A3 (new): `GET /health` and `GET /metrics` may stay public liveness/scrape surfaces. They must not grow queue rows or recipient/amount fields. - INV-OP-A4 (new): An explicit local/dev flag may allow process start without a token for Anvil/E2E. That flag must be named, documented, and default-off. Production-like tests must not set it. Prefer giving E2E a test bearer over leaving routes open. - Do not weaken dest `WithdrawApprove`, watcher finality, or cancel windows. Do not put live tokens in fixtures, docs, or issue comments. Do not require TLS inside this ticket (README already requires a terminating proxy if bound beyond localhost). ## Constraints / guardrails - Prefer fail-closed at `start_api_server` / `main` before `TcpListener::bind`. Do not keep a silent “open if unset” path. - Empty string, whitespace-only, and unset are all missing. Do not treat `Some("")` as configured. - Keep `/health` (liveness / idle 503) and `/metrics` public unless a later ticket scopes scrape auth. Do not dump `PendingResponse` onto `/health`. - Dev flag: one explicit env (name it in the PR; same class as `DEV_ALLOW_HTTP`). Document it as local/E2E only. Do not use “bind is 127.0.0.1” as the substitute for a token in production-like tests. - E2E (`packages/frontend/src/test/e2e-infra/operator.ts`) and QA env helpers must set a test token or the dev flag. Prefer a test token so CI exercises the `401` path. - Constant-time Bearer compare is optional hardening, not a substitute for fail-closed startup. Do not log the token or echo it in `401` bodies. - Do not change Solidity / CosmWasm / Solana programs. Do not retune rate-limit numbers as the “fix”. - Founder-required operator API / keys. No community autoland. Do not add `ready`. No public live-API recipe. ## Relevant files | Path | Why | | --- | --- | | `packages/operator/src/api.rs` | Optional token load; `check_auth` returns true when unset; `/status` and `/pending` always routed; unit tests encode fail-open | | `packages/operator/src/main.rs` | `OPERATOR_API_BIND_ADDRESS` / port; calls `start_api_server` | | `packages/operator/.env.example` | Token not listed as required | | `packages/operator/README.md` | “when `OPERATOR_API_TOKEN` is set”; TLS note | | `packages/operator/security_reviews/SECURITY_REVIEW_2026-02-12.md` | O2 marked Fixed as optional gating | | `docs/operator.md` (if present) | Operator API contract | | `packages/frontend/src/test/e2e-infra/operator.ts` | E2E operator env; bind loopback, no token requirement today | | `scripts/qa/qa-host.env` | QA bind defaults | ## Recommended direction 1. Require a non-empty `OPERATOR_API_TOKEN` before `TcpListener::bind`. Return an error from `start_api_server` (and fail process start) when it is missing, unless the documented local-only flag is set. 2. Change `check_auth`: configured token required for `/status` and `/pending`. Missing/wrong Bearer → `401`, empty body. Invert or delete `test_check_auth_no_token_configured`. 3. Production-like test (no token, no dev flag): `start_api_server` errors and does not accept connections on `/status` or `/pending`. Optionally: router built for that config does not register those routes. 4. Configured-token tests: valid Bearer still returns JSON; missing/wrong Bearer is `401` with no queue fields. 5. With only the local flag set and no token: either do not register `/status`/`/pending`, or keep them local-debug only and **not** used by production-like tests. E2E should set a test token. 6. Update README, `.env.example`, and the O2 residual note: optional gating is not production-complete. ## Acceptance criteria - AC1. Production-like start without `OPERATOR_API_TOKEN` (and without the local-only flag) does not bind `GET /status` or `GET /pending`. API server start fails closed. - AC2. Empty and whitespace-only `OPERATOR_API_TOKEN` are treated as missing (AC1). - AC3. With a non-empty token: no header or wrong Bearer → `401` and no `nonce` / `recipient` / `amount` in the body for `/status` and `/pending`. - AC4. With a matching Bearer: `/status` queue counts and `/pending` lists still work. `/health` and `/metrics` remain reachable without the token. - AC5. `test_check_auth_no_token_configured` no longer asserts open access as success. New tests cover AC1–AC4 without a live operator host. - AC6. README / `.env.example` / security-review residual state that production requires the token. O2 optional gating is not the end state. - AC7. E2E/QA operator spawn sets a test token (or the documented flag). Default example env does not leave privileged routes open. ## Test plan (functional paths) | # | Path | Expect | | --- | --- | --- | | T1 | Start API with non-empty token; `GET /status` + valid Bearer | 200 JSON queue counts, no 401 | | T2 | Same process; `GET /pending` + valid Bearer | 200 JSON; fields as today | | T3 | Same process; `GET /status` with no `Authorization` | 401; no queue JSON | | T4 | Same process; `GET /pending` with wrong Bearer | 401; no recipient/amount | | T5 | Same process; `GET /health` with no token | 200 or 503 liveness only; no pending rows | | T6 | Same process; `GET /metrics` with no token | Prometheus text; no pending recipient list | | T7 | Start with token unset, no local flag | `start_api_server` / process errors; no listener serving T3/T4 as 200 | | T8 | Start with `OPERATOR_API_TOKEN=""` or whitespace | Same as T7 | | T9 | Local flag set, no token (if kept) | Documented debug behavior only; production-like tests do not use this path | | T10 | Existing operator unit tests | Stay green except the inverted fail-open auth test | ## Test plan (attack, hack, and abuse) Non-exploitative. Operator unit / in-process router tests only. Do not use these as a live listen-address recipe. | # | Vector | Expect | | --- | --- | --- | | A1 | Production-like config, token unset, client omits auth | No 200 queue JSON from `/status` or `/pending` (no bind or fail start) | | A2 | Token set; `Authorization` missing | 401, empty privileged body | | A3 | Token set; `Authorization: Bearer ` + wrong secret | 401 | | A4 | Token set; `Authorization: Basic …` or no `Bearer` prefix | 401 (existing `test_check_auth_no_bearer_prefix`) | | A5 | Token set; `bearer` vs `Bearer` prefix | Still only the matching secret succeeds; prefix case must not open a no-token path | | A6 | Bind `0.0.0.0` in a test config **with** token missing and no local flag | Still fail start; bind address is not an auth substitute | | A7 | Local flag accidentally set in a test named production-like | Fail the test; production-like fixtures must not set the flag | ## Verification criteria - Operator package tests: T1–T10 and A1–A7. Grep that `check_auth` no longer returns `true` for `None` / empty in production-like builds. - Grep that `start_api_server` (or `main`) errors when the token is missing unless the documented local flag is set. - Grep `.env.example` / README for required `OPERATOR_API_TOKEN` (or an explicit “required unless `<dev flag>`” sentence). - Invert or remove `test_check_auth_no_token_configured` as a success case for open access. - Do not verify by scanning a live operator URL or by posting real Bearer values. ## Out of scope - [#115](https://git.cl8y.com/code/cl8y-bridge-monorepo/issues/115) `/health` idle 503 / RPC quorum / `FINALITY_BLOCKS`. - [#178](https://git.cl8y.com/code/cl8y-bridge-monorepo/issues/178) on-chain guard / rate-limit wiring. - [#184](https://git.cl8y.com/code/cl8y-bridge-monorepo/issues/184) dest-approve nonce scoping. - Adding TLS to the operator process (proxy remains the documented control if bound beyond localhost). - Authenticating `/metrics` scrape (optional later; not this ticket). - Live operator redeploy / env rotation (ops). ## First-pass model recommendation Recommendation: grok-high Rationale: Security class and founder-required operator API auth / keys. Composer is disallowed (High/security; not a low-risk first pass). Even if the production edit is likely `api.rs` plus startup in `main.rs` plus tests/docs, file count does not establish safety: fail-open auth on `/pending` leaks in-flight nonces, recipients, and amounts, and the existing unit test encodes the wrong default. Verify with in-process start/router fixtures (token missing vs set), not a live operator probe.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
code/cl8y-bridge-monorepo#190
No description provided.