Production indexer (indexer.dex.cl8y.com) returns 503 'no available server' on all endpoints #491

New issue

Closed

opened 2026-07-14 19:33:18 +00:00 by leonardocolucci · 6 comments

leonardocolucci commented

2026-07-14 19:33:18 +00:00

(Migrated from gitlab.com)

Summary

The production indexer at https://indexer.dex.cl8y.com is returning HTTP 503 with body no available server on every endpoint, and has been for several hours (observed continuously from ~17:30 UTC to 19:33 UTC, 2026-07-14). The last known-good response we have is from 2026-07-13 (~21:00 UTC), when /api/v1/pairs served the full 10-pair soft-launch list normally.

The chain side is unaffected: LCD queries, pair contracts, swaps and the dApp static site all work. Only the indexer is down.

Reproduction

$ curl -s -o /dev/null -w '%{http_code}\n' https://indexer.dex.cl8y.com/api/v1/pairs
503
$ curl -s https://indexer.dex.cl8y.com/api/v1/pairs
no available server

Same result (503, no available server) on every route tested: /api/v1/pairs, /api/v1/tokens, /api/v1/overview, /cg/tickers. The body suggests the reverse proxy / load balancer has no healthy upstream, i.e. the indexer process itself is down or failing health checks rather than erroring per-request.

Impact

dApp market panels (charts, volume, trade history, route solve) are blind while the outage lasts — swaps still work since they go through the wallet + LCD directly.
Third-party integrations built on the public API (we run an independent trading bot against it) lose pair discovery and market data. We fall back to LCD factory queries, but the CMC/CG endpoints (/cmc/*, /cg/*) presumably matter for listing aggregators and have no fallback.

Context

Found while running an independent trading/market-making bot against the public API during the soft launch. Happy to provide more observations or re-test on request.

## Summary The production indexer at `https://indexer.dex.cl8y.com` is returning **HTTP 503** with body `no available server` on **every endpoint**, and has been for several hours (observed continuously from ~17:30 UTC to 19:33 UTC, 2026-07-14). The last known-good response we have is from 2026-07-13 (~21:00 UTC), when `/api/v1/pairs` served the full 10-pair soft-launch list normally. The chain side is unaffected: LCD queries, pair contracts, swaps and the dApp static site all work. Only the indexer is down. ## Reproduction ``` $ curl -s -o /dev/null -w '%{http_code}\n' https://indexer.dex.cl8y.com/api/v1/pairs 503 $ curl -s https://indexer.dex.cl8y.com/api/v1/pairs no available server ``` Same result (503, `no available server`) on every route tested: `/api/v1/pairs`, `/api/v1/tokens`, `/api/v1/overview`, `/cg/tickers`. The body suggests the reverse proxy / load balancer has no healthy upstream, i.e. the indexer process itself is down or failing health checks rather than erroring per-request. ## Impact - dApp market panels (charts, volume, trade history, route solve) are blind while the outage lasts — swaps still work since they go through the wallet + LCD directly. - Third-party integrations built on the public API (we run an independent trading bot against it) lose pair discovery and market data. We fall back to LCD factory queries, but the CMC/CG endpoints (`/cmc/*`, `/cg/*`) presumably matter for listing aggregators and have no fallback. ## Context Found while running an independent trading/market-making bot against the public API during the soft launch. Happy to provide more observations or re-test on request.

Rows
Columns