feat: /protocol global stats — total USD pair liquidity + 24h/30d % change #569
Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:implement
agent:implement
agent:open_issues
agent:ready
agent:research
agent:security_audit
agent:verify
architecture
backend
blocker:hybrid
blocker:launch
blocker:limit-orders
blocker:v2
block:log_only
block:security
bug
ci
contracts
correctness
deploy
dev
devops
docs
documentation
duplicate
e2e
enhancement
epic
feature
frontend
functional-completion
gas
good first issue
governance
help wanted
high-risk
hooks
hybrid
indexer
infra
infrastructure
integrators
invalid
launch-blocker
limit-orders
localnet
localterra
low priority
missing-implementation
needs-design
ops
performance
priority
high
priority
medium
product
qa
QA
question
ready
ready
research
scripts
security
security-hardening
smartcontracts
tech-debt
testing
ux
UX
v2
verification
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
code/cl8y-dex-terraclassic#569
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Add total USD liquidity in factory pairs (pool TVL) to
/protocolGlobal stats, plus % change vs 24 hours ago and % change vs 30 days ago. Bundle into one issue. Do not split “current TVL” vs “24h Δ%” vs “30d Δ%” vs “snapshot table” vs “overview JSON”.Related: #550 (Protocol USD census + overview rollup), #556 (DEX hub USD + humanized pair TVL for ranking), #548 (catalog
volume_usd), #522 (P522-Q quote USD), #515 (CEX USTC/LUNC; not vFDUSD for DEX conversion), #281 / #333 (/overview60s cache + O(1) rollup), #489 (short retail copy).Current codebase
/protocolis the DEX USD census page after #550 / #556. Global stats show volume (flow) and listing census. There is no protocol-wide pool liquidity (stock / TVL) headline, and no historical liquidity series to compute 24h / 30d % change.ProtocolGlobalStats.tsx: 24h / 7d / 30d USD volume, tokens, new tokens 30d, new pairs 30d, active pairs, 24h trades.formatProtocolUsd/formatProtocolCount→ em-dash on missing / non-finite. Copy: “USD volume uses the USTC reference feed.” No liquidity / TVL / Δ% boxes.ProtocolPage.tsx: title → Global stats → DEX hub prices → one CEX oracle card → audit contracts → hooks.GET /api/v1/overview(indexer/src/api/overview.rs): 60s whole-response cache. Cache miss readsglobal_stats_24h(volume windows,active_pairs_24h,unique_traders_24h) + cheapCOUNT(*)census + O(1)hub_prices. Nototal_liquidity_usd, no liquidity Δ%.SUM(volume_usd)over swaps (flow), P522-Q catalog, not vFDUSD (P550-4, X4). P550-8 already statesactive_pairs_24his not TVL.unique_traders_24his on the JSON for DoS safety (AC7) but is not a Protocol headline (dust-swap gaming; #489).pair_reservesis a current-state upsert (one row per pair;reserve_0/reserve_1rawNUMERIC(38,0)). Written bybook_snapshot.rs(~10s). No history. Swaps and LP add/remove both move reserves, soliquidity_eventsalone cannot reconstruct TVL.hub_usd.rsalready computes humanized USD TVL =human(reserve_0)×usd_0 + human(reserve_1)×usd_1to rank cUSTC/UST1/USTR source pools (H1–H8). Dust floor, stale/reserve_*=0, symbol-spoof natives skipped. That TVL is stored per hub ticker onhub_prices.tvl_usd, not summed across all factory pairs.pair_price_usd.rsP522-Q:uusd/USTC/cUSTC → USTC oracle;uluna/LUNC/cLUNC → LUNC; UST1/USTR →hub_prices(not$1/2.5×). Unknown / spoof native →None.humanize_raw_amount+fits_numeric_38_18.indexer/src/api/cg.rsliquidity_in_usdis currently 24hvolume_usd, not pool TVL. #550 already forbade inventing Protocol TVL from that field.volume_aggregator.rs~5 min + startup.refresh_global_statsmust not be a 24h-onlyINSERTthat leaves extra columns stale (runbook).ProtocolPage.test.tsx,formatProtocolStats.test.ts,e2e/protocol-page.spec.ts,indexer/tests/api_overview.rs,indexer/tests/indexer_overview_global_stats.rs,make verify-issue-550.skills/AGENTS_FRONTEND_PROTOCOL_STATS.md(P550-1–P550-12),docs/indexer-invariants.mdrow Protocol global stats (#550),docs/frontend.md§ Protocol,docs/runbooks/overview-global-stats-brin.md.Why this is needed
/protocolalso ask how much is locked in pools and whether that stock is growing or shrinking.pair_reservescannot answer “vs yesterday / vs last month.” Without snapshots, the UI would either lie (compute Δ from nothing) or omit the only numbers that make TVL useful.Constraints / guardrails
/overviewstays rollup + cheapCOUNT(*)+ O(1) hub read (P550-5). Do notSUMlive pair TVL, scanswap_events, or walk 30d snapshots on the GET path. 60s whole-response cache stays (V5).liquidity_in_usd. That field is mislabeled 24h volume. Do not usetotal_volume_*as liquidity. Do not use LP token supply as TVL.pair_reserves(constant-product legs). Do not add resting limit-order escrow, parked dust, or wallet balances. Document that book depth is not in this number.uusd→ USTC oracle; LUNC/cLUNC/uluna→ LUNC; UST1/USTR →hub_prices. Never convert with vFDUSD/FDUSD. Never$1UST1 or2.5×USTR. Oracle down → that handle is unpriced (omit / do not peg).symbol=USTRonugem) must not price. Hub CW20s need a contract address.human(reserve)×usd, never raw 18-dec vs 6-dec integers. Reusehumanize_raw_amount+fits_numeric_38_18. Overflow / non-positive → skip that pair (do not wrap or Inf).snapshot_at(same TTL as hub / book mirror),reserve_*=0, missing reserves, same-asset legs, unlisted pairs: omit from the sum. Omitted ≠$0(zeros would fake a drain).h0×usd0 + h1×usd1. If exactly one leg is catalogued →2 ×that leg’s USD (constant-product). If neither → omit. Do not invent gem/gem USD.pct = (now − then) / then × 100whenthen > 0. If no snapshot in the window,then = 0, or overflow → JSONnull, UI em-dash. NeverInfinity/NaN/"∞%". Cold start,--freshrebuild, and indexer younger than 24h/30d → null Δ% (copy must not say “0% change”).liquidity_events(swaps change reserves; prices move).IndexerOverviewuntil indexer ships. Existing volume/census/hub fields unchanged. Charts must keep working."0". Idle DEX with no priced reserves →total_liquidity_usd = "0". Unpriced-only activity does not force volume-stylenullunless no pair could be priced and priced-pair count is 0 — prefer"0"plus optionalpriced_pair_countrather than a third meaning ofnull. Δ% staysnulluntil a baseline exists.protocol-global-stats. Do not add a second stats card. Do not clone factory/routerAddressRow(#378). Light + dark; mobile grid wrap./ust1window. Same “reference” voice as volume.refresh_global_statsthat omits liquidity columns must not ship (same footgun as #550 7d/30d columns).liquidity_in_usdin this issue unless it is a one-line comment pointing at the new field — aggregators may already consume the mislabel. Protocol must not depend on it.Relevant files
indexer/src/api/overview.rsindexer/src/db/queries/volume.rsrefresh_global_stats/get_global_statsindexer/src/indexer/volume_aggregator.rsindexer/src/indexer/hub_usd.rspair_tvl/usd_from_reserves/ staleness / dustindexer/src/indexer/pair_price_usd.rsindexer/src/db/queries/hub_prices.rslist_reserve_pairs, hub marksindexer/src/db/queries/pair_reserves.rsindexer/migrations/global_stats_24hcolumns (pattern:20260818120000_global_stats_windows_and_census.sql)indexer/tests/api_overview.rsindexer/tests/indexer_overview_global_stats.rsfrontend-dapp/src/types/index.tsIndexerOverviewfrontend-dapp/src/components/protocol/ProtocolGlobalStats.tsxfrontend-dapp/src/utils/formatProtocolStats.tsfrontend-dapp/src/pages/ProtocolPage.test.tsxfrontend-dapp/e2e/protocol-page.spec.tsdocs/runbooks/overview-global-stats-brin.mddocs/indexer-invariants.mdskills/AGENTS_FRONTEND_PROTOCOL_STATS.mddocs/frontend.mdscripts/qa/verify-issue-550.shverify-issue-<iid>AGENTS.mdmake verify-issue-<iid>Recommended direction
Indexer — current TVL
Reuse hub math (
pair_tvl/usd_from_reserves) over all factory pairs with a usablepair_reservesrow. Price each leg via P522-Q +hub_prices. Apply one-sided2×when only one leg is catalogued. Sum priced pairs intototal_liquidity_usd(NUMERIC(38,18), clamp with the sameLEAST/overflow pattern as volume USD). Optionally persistpriced_pair_count/unpriced_pair_counton the rollup for operators (JSON ok; not Protocol headlines).Refresh in the aggregator (~5 min) and once at startup, after hub USD refresh so UST1/USTR marks exist. Do not LCD-query pools on
/overview.Indexer — history for Δ%
New table (names may vary):
Insert on each successful TVL refresh. Retain ≥ 35 days; prune older in the same loop. Lookup
thenas the snapshot nearest tonow() - 24h/now() - 30dwithin a documented tolerance (e.g. ±30 min). If none → Δ% null.Materialize on
global_stats_24h(additive columns) so GET stays O(1):total_liquidity_usdliquidity_change_24h_pctnullif no baselineliquidity_change_30d_pctnullif no baselineOptional (useful for tests / UI tooltips, not required as headlines):
total_liquidity_usd_24h_ago,total_liquidity_usd_30d_ago. Prefer indexer-computed % so clients do not divide by zero.Per-pair snapshot history is out of scope unless needed to debug; Protocol only needs the global series.
Frontend —
/protocolprotocol-global-stats(suggested testids):protocol-stat-liquidity,protocol-stat-liquidity-24h,protocol-stat-liquidity-30d.formatProtocolUsdfor total. NewformatProtocolPct: missing / non-finite → em-dash; finite → signed compact percent (noInfinity). XSS: text nodes only (existing StatBox).unique_traders_24h. Do not show mixed-unit reserves.Docs / verify
Extend Protocol skill + invariants + overview runbook +
docs/frontend.md+AGENTS.md. Addscripts/qa/verify-issue-<iid>.shandmake verify-issue-<iid>(Postgres-only indexer tests + frontend unit/RTL; Playwright e2e-smoke @ 5 workers whenfrontend-dapp/node_modulesexists).Acceptance criteria
/protocolGlobal stats shows total USD liquidity in factory pairs ($+ compact human fromtotal_liquidity_usd). Not volume, not CGliquidity_in_usd, not raw reserves.0%/Infinity.GET /api/v1/overviewgains the new fields without breaking existing keys. Charts still works.swap_eventsor walk 30d snapshots or recompute all pair TVL. EXPLAIN / tests prove GET reads the rollup row (and cheap census). 60s cache unchanged.$1UST1. Spoof natives omitted. Unpriced pairs omitted (not$0).make verify-issue-<iid>+ indexer overview/TVL tests + RTL + Playwright (5 workers).Test plan (all paths)
Global stats UI
$…; 24h/30d signed%; volume boxes still presentundefined/NaN/Infinitytotal_liquidity_usd = "0", Δ% null$0(or$0.00); Δ% em-dashprotocol-global-stats, above hub + oracletotal_volume_24hstill not a headline; liquidity ≠ volume string%Indexer TVL
$12.5×"0"total; Δ% nullIndexer snapshots / Δ%
(now-then)/then*100nullthen = 0, now > 0null(not Inf)refresh_global_statsIndexer overview / DoS budget
api_overview.rsswap_events30d aggregate; no sequential scan of snapshot history on GETOVERVIEW_GLOBAL_STATS_LIVE=1Unit / RTL
formatProtocolPct: null /NaN/Infinity/""→ em-dash;"12.5"/"-3"render as percent.ProtocolPage.test.tsx: new testids; factory/router still present; hub card still after stats.Playwright (5 workers,
e2e-smoke)/protocolprotocol-stat-liquidity+ 24h + 30d testids visible with volume statsTest plan (attack, hack, abuse)
/overviewcache miss does not join allpair_reserves×oracles or scan snapshot history. Burst within 60s hits cache.ORDER BY sampled_atover unbounded history.symbol=USTR/UST1with non-hub denom does not price (A1).$0$1UST1 /2.5×USTRtotal_liquidity_usd/ pct strings with<script>/javascript:render as text;formatProtocolUsd/formatProtocolPctem-dash on non-finite.then=0or overflow → JSON null + UI em-dash. Client must notnow/thenitself without a zero guard.fits_numeric_38_18skipped; response stays small; HTTP 200.--fresh, do not backfill 30d Δ% from zeros or fromliquidity_events. Δ% null until real snapshots accrue.liquidity_in_usd. Integrator tests may still see the old CG mislabel.?ticker=oracle allowlist unchanged.Verification criteria
Ship is done when all of the following hold:
make verify-issue-<iid>is green on a Cloud Agent / local machine withmake setup-indexer-postgres(no wasm deploy required for indexer + RTL). Playwright e2e-smoke is optional-but-run whenfrontend-dapp/node_modulesexists, 5 workers.api_overview/ new*_liquidity_*integration file,--test-threads=1).formatProtocolStats+ProtocolPageRTL for S1–S8; Charts types still compile with additive overview fields.docs/frontend.md,AGENTS.mdcrosslink./protocolshows total liquidity + 24h% + 30d%; young indexer shows em-dash for Δ%; volume / hub / oracle / audit unchanged.make verify-issue-550andmake verify-issue-556stay green (no regression of census, hub card, or cheap/overview).mentioned in merge request !1100
mentioned in merge request !1101
mentioned in merge request !1102
mentioned in commit
91aed331f1Merged via !1102.
Conflict with #576 hid liquidity StatBoxes when overview failed; that was reversed before merge so
/protocole2e still findsprotocol-stat-liquidity*. RTL + Playwright protocol-page (5 workers) passed. Remaining:/protocol: Total liquidity + 24h% + 30d% next to volume; young indexer Δ% is em-dash, not0%/ Infinityglobal_stats_pool_tvlmigration and runrefresh_protocol_liquiditymentioned in issue #583
marked as related to #583
mentioned in issue #586
mentioned in issue #589
mentioned in issue #631
marked as related to #631
mentioned in issue #652
mentioned in issue #653
mentioned in issue #655
marked as related to #655
mentioned in issue #664
mentioned in issue #667
marked as related to #667
mentioned in merge request !1173
mentioned in issue #677
marked as related to #677
mentioned in issue #684
mentioned in issue #685
marked as related to #685
mentioned in issue #689
marked as related to #689