Indexer: narrow fee-discount registry health API #373

New issue

2026-06-13T02:49:41Z

PlasticDigits commented

2026-06-13 02:49:41 +00:00

(Migrated from gitlab.com)

Parent

Follow-up from GitLab #365 — approved verification follow-up 1: narrow GET /api/v1/health/fee-discount with LCD config probe and sanitized errors.

Parent gap: GitLab #361 register item M2 (fee-discount registry Err → silent full fee on-chain; off-chain ops need a health signal).

Current codebase

Pair contract maps registry GetDiscount query Err(_) to full pair fee_bps with no discount metadata (smartcontracts/contracts/pair/src/discount_cache.rs:145,173). Integration test: swap_uses_full_fee_when_discount_registry_query_fails (smartcontracts/tests/src/lib.rs).
Indexer GET /health still returns only {"status":"ok"} (indexer/src/api/mod.rs) — no DB/LCD probe on the generic health surface (deep /health was rejected for #361).
Partial implementation on main: indexer/src/indexer/fee_discount_registry_health.rs (60s background LCD config probe, consecutive failure counter, tracing::error at ≥2 failures), indexer/src/api/fee_discount_health.rs (GET /api/v1/health/fee-discount), route wired in indexer/src/api/mod.rs, spawned from indexer/src/main.rs when FEE_DISCOUNT_ADDRESS is set. Integration tests: indexer/tests/api_fee_discount_health.rs (sanitized JSON, no LCD stack traces).
trader_tracker.rs still logs per-trader tier-sync LCD failures at warn but does not expose them via API (by design — no per-trader enumeration).

Why needed

When the fee-discount registry is paused, migrating, or LCD-unreachable, pairs fail-closed to full fee on-chain with no swap revert. Ops and integrators currently have no narrow, opt-in signal that the registry LCD surface is unhealthy — only scattered per-trader sync warnings in logs.

Constraints / guardrails

Do not extend generic GET /health with LCD/DB probes (rejected for #361). Keep this endpoint narrow and dedicated.
Do not expose per-trader registry errors or wallet-specific failure reasons in the public API (enumeration risk).
Sanitize all error surfaces per indexer/tests/security.rs patterns — no raw LCD URLs, stack traces, or upstream body text in JSON responses.
Do not change on-chain fail-open/fail-closed behavior without an explicit ADR.
No Prometheus /metrics dependency (#200).

Relevant files

indexer/src/indexer/fee_discount_registry_health.rs
indexer/src/api/fee_discount_health.rs
indexer/src/api/mod.rs (route registration, AppState.fee_discount_registry_health)
indexer/src/main.rs (probe loop spawn)
indexer/src/config.rs (FEE_DISCOUNT_ADDRESS)
indexer/tests/api_fee_discount_health.rs
indexer/tests/security.rs
docs/indexer-invariants.md (observability exception row)

Recommended direction

Confirm GET /api/v1/health/fee-discount returns FeeDiscountRegistryHealthSnapshot:
- configured: bool — FEE_DISCOUNT_ADDRESS set
- fee_discount_registry_ok: bool | null — null when unconfigured; otherwise latest probe result
- consecutive_lcd_failures: u32
Background loop probes registry config smart query every 60s via existing LcdClient; record success/failure in shared in-memory state.
Log repeated failures at error (≥2 consecutive) with guidance that pairs fail-closed to full fee; recovery at info.
Document the endpoint in docs/indexer-invariants.md as the narrow observability exception to the tracing-only rule.

Acceptance criteria

GET /api/v1/health/fee-discount returns structured JSON with configured, fee_discount_registry_ok, consecutive_lcd_failures.
When FEE_DISCOUNT_ADDRESS is unset, fee_discount_registry_ok is null and configured is false.
Simulated LCD failure increments consecutive_lcd_failures and sets fee_discount_registry_ok: false without leaking upstream error text in the HTTP body.
Repeated probe failures emit tracing::error (not only per-trader tier-sync warns).
Generic GET /health remains unchanged ({"status":"ok"}).

Test plan

Path	Command / action
Unit	`cd indexer && cargo test fee_discount_registry_health --lib`
Integration	`cd indexer && cargo test --test api_fee_discount_health -- --test-threads=1`
Security	`cd indexer && cargo test --test security` (sanitization spot-check)
Manual	With indexer running and registry LCD blocked → `curl /api/v1/health/fee-discount` shows `fee_discount_registry_ok: false`; body contains no LCD hostnames

Attack / abuse test plan

Response JSON must not include raw LCD error strings, contract query payloads, or internal URLs (api_fee_discount_health.rs asserts).
Endpoint must not accept trader address parameters or return per-wallet registry state.
Rate limiting: inherits existing Axum middleware on /api/v1/* if applicable; no unbounded LCD fan-out from public callers (probe is server-side only).

Verification criteria

Wiremock integration test: 500 from LCD → fee_discount_registry_ok: false, consecutive_lcd_failures ≥ 2, response text excludes cosmwasm, http://, All LCD endpoints failed.
Ops can poll /api/v1/health/fee-discount during registry outage without parsing per-trader logs.
On-chain swap_uses_full_fee_when_discount_registry_query_fails still passes unchanged.

## Parent Follow-up from [GitLab #365](https://gitlab.com/PlasticDigits/cl8y-dex-terraclassic/-/work_items/365) — approved verification follow-up **1**: narrow `GET /api/v1/health/fee-discount` with LCD `config` probe and sanitized errors. Parent gap: [GitLab #361](https://gitlab.com/PlasticDigits/cl8y-dex-terraclassic/-/work_items/361) register item **M2** (fee-discount registry `Err` → silent full fee on-chain; off-chain ops need a health signal). ## Current codebase - Pair contract maps registry `GetDiscount` query `Err(_)` to **full pair `fee_bps`** with no discount metadata (`smartcontracts/contracts/pair/src/discount_cache.rs:145,173`). Integration test: `swap_uses_full_fee_when_discount_registry_query_fails` (`smartcontracts/tests/src/lib.rs`). - Indexer `GET /health` still returns only `{"status":"ok"}` (`indexer/src/api/mod.rs`) — no DB/LCD probe on the generic health surface (deep `/health` was rejected for #361). - **Partial implementation on `main`:** `indexer/src/indexer/fee_discount_registry_health.rs` (60s background LCD `config` probe, consecutive failure counter, `tracing::error` at ≥2 failures), `indexer/src/api/fee_discount_health.rs` (`GET /api/v1/health/fee-discount`), route wired in `indexer/src/api/mod.rs`, spawned from `indexer/src/main.rs` when `FEE_DISCOUNT_ADDRESS` is set. Integration tests: `indexer/tests/api_fee_discount_health.rs` (sanitized JSON, no LCD stack traces). - `trader_tracker.rs` still logs per-trader tier-sync LCD failures at `warn` but does not expose them via API (by design — no per-trader enumeration). ## Why needed When the fee-discount registry is paused, migrating, or LCD-unreachable, pairs **fail-closed** to full fee on-chain with no swap revert. Ops and integrators currently have no narrow, opt-in signal that the registry LCD surface is unhealthy — only scattered per-trader sync warnings in logs. ## Constraints / guardrails - **Do not** extend generic `GET /health` with LCD/DB probes (rejected for #361). Keep this endpoint **narrow** and dedicated. - **Do not** expose per-trader registry errors or wallet-specific failure reasons in the public API (enumeration risk). - Sanitize all error surfaces per `indexer/tests/security.rs` patterns — no raw LCD URLs, stack traces, or upstream body text in JSON responses. - **Do not** change on-chain fail-open/fail-closed behavior without an explicit ADR. - No Prometheus `/metrics` dependency ([#200](https://gitlab.com/PlasticDigits/cl8y-dex-terraclassic/-/issues/200)). ## Relevant files - `indexer/src/indexer/fee_discount_registry_health.rs` - `indexer/src/api/fee_discount_health.rs` - `indexer/src/api/mod.rs` (route registration, `AppState.fee_discount_registry_health`) - `indexer/src/main.rs` (probe loop spawn) - `indexer/src/config.rs` (`FEE_DISCOUNT_ADDRESS`) - `indexer/tests/api_fee_discount_health.rs` - `indexer/tests/security.rs` - `docs/indexer-invariants.md` (observability exception row) ## Recommended direction 1. Confirm `GET /api/v1/health/fee-discount` returns `FeeDiscountRegistryHealthSnapshot`: - `configured: bool` — `FEE_DISCOUNT_ADDRESS` set - `fee_discount_registry_ok: bool | null` — `null` when unconfigured; otherwise latest probe result - `consecutive_lcd_failures: u32` 2. Background loop probes registry `config` smart query every 60s via existing `LcdClient`; record success/failure in shared in-memory state. 3. Log repeated failures at `error` (≥2 consecutive) with guidance that pairs fail-closed to full fee; recovery at `info`. 4. Document the endpoint in `docs/indexer-invariants.md` as the narrow observability exception to the tracing-only rule. ## Acceptance criteria - [ ] `GET /api/v1/health/fee-discount` returns structured JSON with `configured`, `fee_discount_registry_ok`, `consecutive_lcd_failures`. - [ ] When `FEE_DISCOUNT_ADDRESS` is unset, `fee_discount_registry_ok` is `null` and `configured` is `false`. - [ ] Simulated LCD failure increments `consecutive_lcd_failures` and sets `fee_discount_registry_ok: false` without leaking upstream error text in the HTTP body. - [ ] Repeated probe failures emit `tracing::error` (not only per-trader tier-sync warns). - [ ] Generic `GET /health` remains unchanged (`{"status":"ok"}`). ## Test plan | Path | Command / action | |------|------------------| | Unit | `cd indexer && cargo test fee_discount_registry_health --lib` | | Integration | `cd indexer && cargo test --test api_fee_discount_health -- --test-threads=1` | | Security | `cd indexer && cargo test --test security` (sanitization spot-check) | | Manual | With indexer running and registry LCD blocked → `curl /api/v1/health/fee-discount` shows `fee_discount_registry_ok: false`; body contains no LCD hostnames | ## Attack / abuse test plan - Response JSON must not include raw LCD error strings, contract query payloads, or internal URLs (`api_fee_discount_health.rs` asserts). - Endpoint must not accept trader address parameters or return per-wallet registry state. - Rate limiting: inherits existing Axum middleware on `/api/v1/*` if applicable; no unbounded LCD fan-out from public callers (probe is server-side only). ## Verification criteria - Wiremock integration test: 500 from LCD → `fee_discount_registry_ok: false`, `consecutive_lcd_failures ≥ 2`, response text excludes `cosmwasm`, `http://`, `All LCD endpoints failed`. - Ops can poll `/api/v1/health/fee-discount` during registry outage without parsing per-trader logs. - On-chain `swap_uses_full_fee_when_discount_registry_query_fails` still passes unchanged.

PlasticDigits commented

2026-06-13 02:49:56 +00:00

(Migrated from gitlab.com)

mentioned in issue #365

PlasticDigits commented

2026-06-13 03:00:41 +00:00

(Migrated from gitlab.com)

mentioned in commit a165749c4c

mentioned in commit a165749c4c8c61438cfac75edf40ef9369706c05

PlasticDigits commented

2026-06-13 03:01:01 +00:00

(Migrated from gitlab.com)

mentioned in merge request !891

PlasticDigits commented

2026-06-13 03:02:02 +00:00

(Migrated from gitlab.com)

mentioned in merge request !892

PlasticDigits commented

2026-06-13 03:17:07 +00:00

(Migrated from gitlab.com)

mentioned in commit 85b2af8e5f

mentioned in commit 85b2af8e5ffadddf89594895b1a40ed893904fcc

Brouie commented

2026-06-13 03:19:34 +00:00

(Migrated from gitlab.com)

Verified on my side on main edbe288.

The narrow endpoint is in and the ACs hold:

GET /api/v1/health/fee-discount returns the structured snapshot — configured, fee_discount_registry_ok, consecutive_lcd_failures.
FEE_DISCOUNT_ADDRESS unset → fee_discount_registry_ok null and configured false.
A probe failure increments consecutive_lcd_failures and flips fee_discount_registry_ok to false without putting upstream error text in the HTTP body.
Repeated failures log at error (≥2 consecutive), recovery at info — not just the per-trader tier-sync warns.
Generic GET /health is untouched, still {"status":"ok"}.

Tests green:

lib fee_discount_registry_health (snapshot reflects configured + failures, unconfigured → null)
integration api_fee_discount_health (reflects probe state with no LCD stack traces; unconfigured returns null ok)
security suite 25/25 (sanitization patterns)

Live endpoint is serving configured:true / fee_discount_registry_ok:true on the current deploy, and /health is unchanged alongside it.

Good to close from my side. @PlasticDigits

Verified on my side on main edbe288. The narrow endpoint is in and the ACs hold: - GET /api/v1/health/fee-discount returns the structured snapshot — configured, fee_discount_registry_ok, consecutive_lcd_failures. - FEE_DISCOUNT_ADDRESS unset → fee_discount_registry_ok null and configured false. - A probe failure increments consecutive_lcd_failures and flips fee_discount_registry_ok to false without putting upstream error text in the HTTP body. - Repeated failures log at error (≥2 consecutive), recovery at info — not just the per-trader tier-sync warns. - Generic GET /health is untouched, still {"status":"ok"}. Tests green: - lib fee_discount_registry_health (snapshot reflects configured + failures, unconfigured → null) - integration api_fee_discount_health (reflects probe state with no LCD stack traces; unconfigured returns null ok) - security suite 25/25 (sanitization patterns) Live endpoint is serving configured:true / fee_discount_registry_ok:true on the current deploy, and /health is unchanged alongside it. Good to close from my side. @PlasticDigits

PlasticDigits commented

2026-06-13 03:34:34 +00:00

(Migrated from gitlab.com)

mentioned in commit cccd4cf23e

mentioned in commit cccd4cf23e247efc3215242f320e58c7eb491353

PlasticDigits commented

2026-06-13 03:41:33 +00:00

(Migrated from gitlab.com)

mentioned in commit 44d73ee47c

mentioned in commit 44d73ee47ceefe87dd8a52fc469d5482d3f37332

PlasticDigits commented

2026-06-13 03:41:36 +00:00

(Migrated from gitlab.com)

mentioned in merge request !893

PlasticDigits (Migrated from gitlab.com) closed this issue

2026-06-13 03:51:26 +00:00

Brouie commented

2026-06-13 03:55:46 +00:00

(Migrated from gitlab.com)

Re-verified on main d013b61 after the health-API rework landed (the fee_discount_registry_health rework + main.rs from_config wiring), since my earlier note was against the pre-rework tree. Rebuilt + restarted the indexer off d013b61 first.

Still holds, with one behavior change worth noting:

Snapshot shape unchanged — {configured, fee_discount_registry_ok, consecutive_lcd_failures}.
New behavior: fee_discount_registry_ok now starts null even when configured, and only flips to true/false after the first background probe (previously it was optimistically true at startup). Live: right after restart it probes a healthy registry and the endpoint reports configured:true / ok:true / 0 failures.
record_probe_failure now takes the LcdError, but it's only used in the tracing::error/warn log line — it is NOT in the serialized snapshot, so the API body still carries no upstream LCD text. Sanitization holds.
Generic /health is still {"status":"ok"}.

Tests green on d013b61: lib fee_discount_registry_health 4/4 (incl. configured-starts-null + successful-probe-sets-true), integration api_fee_discount_health 5/5 (incl. unconfigured returns null ok), security 25/25.

Still good to close from my side. @PlasticDigits

Re-verified on main d013b61 after the health-API rework landed (the fee_discount_registry_health rework + main.rs from_config wiring), since my earlier note was against the pre-rework tree. Rebuilt + restarted the indexer off d013b61 first. Still holds, with one behavior change worth noting: - Snapshot shape unchanged — {configured, fee_discount_registry_ok, consecutive_lcd_failures}. - New behavior: fee_discount_registry_ok now starts null even when configured, and only flips to true/false after the first background probe (previously it was optimistically true at startup). Live: right after restart it probes a healthy registry and the endpoint reports configured:true / ok:true / 0 failures. - record_probe_failure now takes the LcdError, but it's only used in the tracing::error/warn log line — it is NOT in the serialized snapshot, so the API body still carries no upstream LCD text. Sanitization holds. - Generic /health is still {"status":"ok"}. Tests green on d013b61: lib fee_discount_registry_health 4/4 (incl. configured-starts-null + successful-probe-sets-true), integration api_fee_discount_health 5/5 (incl. unconfigured returns null ok), security 25/25. Still good to close from my side. @PlasticDigits

Sign in to join this conversation.