fix: token/trader 24h rollups never decay; stale global_stats_24h freezes Charts volume #577
Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:implement
agent:implement
agent:open_issues
agent:ready
agent:research
agent:security_audit
agent:verify
architecture
backend
blocker:hybrid
blocker:launch
blocker:limit-orders
blocker:v2
block:log_only
block:security
bug
ci
contracts
correctness
deploy
dev
devops
docs
documentation
duplicate
e2e
enhancement
epic
feature
frontend
functional-completion
gas
good first issue
governance
help wanted
high-risk
hooks
hybrid
indexer
infra
infrastructure
integrators
invalid
launch-blocker
limit-orders
localnet
localterra
low priority
missing-implementation
needs-design
ops
performance
priority
high
priority
medium
product
qa
QA
question
ready
ready
research
scripts
security
security-hardening
smartcontracts
tech-debt
testing
ux
UX
v2
verification
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
code/cl8y-dex-terraclassic#577
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Charts 24h Volume (USD) is supposed to be a trailing 24h sum from
global_stats_24h. Pair list volume already zeros idle pairs. Two other rollups do not decay, and/overviewwill keep serving a frozen non-zero 24h total if the ~5 min aggregator stops.Retail report: 24h volume “never goes back to 0 / only goes up.” Copy/UX is a linked issue. This issue owns rollup decay + freshness so token, trader, pair, and global windows actually fall when
swap_eventsleave the cutoff.Current codebase
What already decays
refresh_global_statsrebuildsglobal_stats_24hfromswap_eventswithFILTER (WHERE block_timestamp >= $1)for 24h (and 7d/30d). A successful refresh can drop volume to 0.refresh_pair_volumesINSERTs pairs with 24h swaps, then:Idle pairs go to 0. There is no integration test that a pair with only 48h-old swaps is zero after refresh.
Pair stats (
get_24h_stats_for_pair) are a livenow − 24hquery — they decay without a rollup.What does not decay
1.
token_volume_stats—refresh_token_volumesINSERT … SELECT … WHERE block_timestamp >= $cutoff GROUP BY offer_asset_id ON CONFLICT DO UPDATE.If an asset has no swaps in that window, the SELECT emits no row, so the previous
24h/7d/30drow is left forever.GET /api/v1/tokens/{addr}returns that stalevolume_stats(tokens.rs). No tests callrefresh_token_volumes.Offer-side only (
GROUP BY offer_asset_id) is existing semantics — this issue must not silently start summing both legs.2.
traders.volume_24h/volume_7d/volume_30d—refresh_rolling_volumesUPDATE traders … FROM (SELECT sender … FROM swap_events WHERE block_timestamp >= $30d GROUP BY sender) sub WHERE t.address = sub.sender.Traders inside the 30d subquery get correct CASE zeros (24h can be 0 while 30d > 0). Traders whose last swap is older than 30d are not in the subquery, so
volume_30d(and any leftover 24h/7d if a refresh was skipped) sticks. Leaderboardsort=volume_30d/volume_24hcan rank ghosts (get_leaderboard). Charts Volume tab uses lifetimetotal_volume_usd(#553) — still fix the rolling columns for API / QA.Do not zero
total_volume/total_volume_usd(lifetime).3. Stale
global_stats_24hon/overviewget_global_statsreads the rollup and ignoresupdated_at. Live fallback only whentotal_trades == 0and recent swaps exist (uninitialized seed). If the aggregator dies after a non-zero refresh,/overviewkeeps the last 24h USD/trades forever (plus 60s response cache). That matches “only goes up / never comes back.”run_volume_refresh_loopsleeps 300s first.poller.rsdoes an initialrefresh_pair_volumes+refresh_global_statsonly — not token volumes or trader rolling windows.Existing test
global_stats_rollup_excludes_swaps_older_than_24hinserts a 48h swap alongside fresh seed swaps and asserts the count stays 5. It does not age a previously counted swap out and assert the total decreases.Why this is needed
global_stats_24h.updated_at, no live fallback for a stale non-zero row.Constraints / guardrails
SUM(swap_events)on every/overviewGET (#281 / #333 V5). Stale handling must not become a DoS. Prefer: keep serving rollup; log + metric onupdated_atage; optional one-shot refresh in the background loop (not on the request path); live fallback only behind the existing env flag or a tight stale threshold with a timeout/statement_timeout.Utc::now() − window. No calendar-day reset.volume_usd_for_swap/ P522-Q). Decay uses storedvolume_usd/ raw amounts as today.traders.total_volume,total_volume_usd,total_trades).LEASTnumeric caps on sums (overflow / NUMERIC(38) / USD 38,18).DELETE FROM swap_events.resolve_block_time— neverUtc::now()at ingest). Decay tests must use storedblock_timestamp, not wall-clock sleeps of 24h.#548$0vs—JSON contract.Relevant files
indexer/src/db/queries/volume.rsindexer/src/db/queries/traders.rsindexer/src/indexer/volume_aggregator.rs,indexer/src/indexer/poller.rsindexer/src/api/overview.rsindexer/src/api/tokens.rsindexer/tests/indexer_overview_global_stats.rs,indexer/tests/indexer_pair_volume_pagination.rs,indexer/tests/api_traders.rs,indexer/tests/common/mod.rsdocs/indexer-invariants.md,docs/runbooks/overview-global-stats-brin.md,skills/AGENTS_INDEXER_VOLUME_PAGINATION.mdRecommended direction
UPDATE token_volume_stats SET volume=0, volume_usd=0, trade_count=0, unique_traders=0for that"window"when noswap_eventsforoffer_asset_idin cutoff (same pattern aspair_volume_24h). Bind window/cutoff — no string-concat SQL.UPDATEsettingvolume_24h=volume_7d=volume_30d=0for addresses not in the 30d sender set (orNOT EXISTSrecent swaps). Do not touch lifetime columns.pollerinitial refresh also runsrefresh_token_volumes+refresh_rolling_volumes(failures = warn, same as pair/global).global_stats_24h.updated_atinget_global_stats. If older than a documented bound (e.g. 15 min): tracing warning + Prometheus/histogram if the indexer already has metrics; do not default to unbounded live 30d scan. Optional JSONstats_updated_atis integrator-only — not a retail Charts box.block_timestampinto the past (25h / 8d / 31d), call refresh, assert windows drop (including to 0). Add pair idle-zero test (behavior already implemented).make verify-issue-<iid>+ invariants/skill row.Acceptance criteria
token_volume_statswindow24his 0 after refresh (7d/30d analogously).volume_24h,volume_7d,volume_30dare 0;total_volume/total_volume_usdunchanged.pair_volume_24h.volume_quote = 0after refresh (tested).global_stats_24hvolume/trades (not only “extra old row ignored”).global_stats_24h.updated_atis observable (log/metric and/or documented operator check)./overviewGET must not grow into a production live 30dswap_eventsscan.make verify-issue-<iid>.Test plan (all paths)
now−25h;refresh_token_volumes24hrow zeros;7dstill counts if within 7d7d/30drows zero when past cutoffvolume_24h=0,volume_7d>0swap_eventspair_volume_24h.volume_quote=0; listsort=volume_24hdoes not rank it as livetotal_trades_24h=0, USD 0 / JSON"0"when idleOVERVIEW_GLOBAL_STATS_LIVE=1(asset_id, window)/ pair PK holdTest plan (attack, hack, abuse)
block_timestamp(clock skew / hostile LCD)Utc::now()at ingest. Document; decay tests use explicit past timestampswindowSQL injection (24hbind)LEAST(…, POWER(10,38)-1)/ USD cap; zero-out uses0not NULL unless column allows/overviewstale → live 30d scan on every GETtotal_volume_usd“to fix 24h”sort=volume_24hwith stale ghostsvolume_30dUPDATEmust use indexes (pair_id,(asset_id, window),traders.address); EXPLAIN on refresh statements in tests optionalVerification criteria
make verify-issue-<iid>:cd indexer && cargo test --test indexer_overview_global_stats --test indexer_pair_volume_pagination --test api_traders --test api_tokens -- --test-threads=1(plus any new decay test file) aftermake setup-indexer-postgresif needed.24hvolume 0.SELECT updated_at FROM global_stats_24h WHERE id = 1advances on a running indexer (document in the BRIN/overview runbook).$2.7Kon a live trailing window; it must not leave frozen or never-zeroed token/trader windows.marked as related to #576
mentioned in merge request !1099
mentioned in commit
39c6cb382amentioned in commit
704e5b9793mentioned in issue #576
Merged via !1099.
make verify-issue-577passed (docs, stale-bound unit tests, integration decay + overview + pair + traders + tokens). Remaining operator checks on a running indexer:SELECT updated_at FROM global_stats_24h WHERE id = 1advances; age > 15 min logs the stale-rollup warning without a live 30d scanmentioned in issue #583
marked as related to #583
mentioned in issue #586
mentioned in issue #589
mentioned in issue #613
mentioned in issue #652
mentioned in issue #655
mentioned in issue #682
mentioned in issue #683
mentioned in issue #692