feat: /protocol fee tracking — 24h/7d/30d USD, % change, token mix, source (swap/book/place/wrap) #586
Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:implement
agent:implement
agent:open_issues
agent:ready
agent:research
agent:security_audit
agent:verify
architecture
backend
blocker:hybrid
blocker:launch
blocker:limit-orders
blocker:v2
block:log_only
block:security
bug
ci
contracts
correctness
deploy
dev
devops
docs
documentation
duplicate
e2e
enhancement
epic
feature
frontend
functional-completion
gas
good first issue
governance
help wanted
high-risk
hooks
hybrid
indexer
infra
infrastructure
integrators
invalid
launch-blocker
limit-orders
localnet
localterra
low priority
missing-implementation
needs-design
ops
performance
priority
high
priority
medium
product
qa
QA
question
ready
ready
research
scripts
security
security-hardening
smartcontracts
tech-debt
testing
ux
UX
v2
verification
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
code/cl8y-dex-terraclassic#586
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Add protocol fee tracking to
/protocol: trailing 24h / 7d / 30d treasury fees (USD), % change vs the previous equal window, which tokens fees were earned in, and source (AMM swap, book take, limit place, wrap, unwrap). Bundle into one issue. Do not split “USD totals” vs “Δ%” vs “token table” vs “source table” vs “wrap ingest” vs “overview JSON”.Related: #550 (Protocol USD census + overview rollup), #569 (pool TVL + snapshot Δ%), #576 (trailing-window copy), #577 (window decay), #556 (hub USD), #548 / #522 (P522-Q catalog), #515 (CEX USTC/LUNC; not vFDUSD for DEX conversion), #196 (L7 pool vs book commission), #516 (wrap-mapper split fees), #281 / #333 (
/overview60s cache + O(1) rollup), #489 (short retail copy), #568 (do not rewrite historical USD from live hub).Current codebase
/protocolis the DEX USD census page after #550 / #556 / #569. Global stats show volume (flow) and pool TVL (stock). There is no protocol-wide treasury fee headline, no fee Δ%, no fee token mix, and no wrap/unwrap fee ingest.ProtocolGlobalStats.tsx: TVL + 24h/30d liquidity Δ%, Last 24h/7d/30d vol, tokens, new tokens/pairs, active pairs, 24h trades.formatProtocolUsd/formatProtocolPct→ em-dash on missing / non-finite. No fee boxes, source table, or token mix.ProtocolPage.tsx: title → Global stats → DEX hub prices → one CEX oracle card → audit contracts → hooks.GET /api/v1/overview(indexer/src/api/overview.rs): 60s whole-response cache. Cache miss readsglobal_stats_24h(volume windows, census, TVL / Δ%) + cheapCOUNT(*)+ O(1)hub_prices. No fee fields.swap_events.commission_amountis pool-leg only (Terraport baseline; L7 / #196). Parserparse_swapsdoes not persistbook_commission_amount. Amounts are raw ask-asset integer, not USD.limit_order_fills.commission_amountis the taker half per fill (bid: token1; ask: token0). Summing fills plus swapcommission_amountis the hybrid total ifbook_commission_amountis not also added.place_limit_orderemitsmaker_fee_amount(docs/limit-orders.md).limit_order_placementsstores owner/side/price — notmaker_fee_amount. ParserParsedLimitOrderPlacementhas no fee field.fee_wrap_bps/fee_unwrap_bpslive in ustr-cmm; DEX UI quotes them (#516). Indexer has no wrap/unwrap event table and does not expose mapper fees (AGENTS_WRAP_MAPPER_SPLIT_FEES.md).total_fees_paidtraders.total_fees_paidis a lifetime mixed-unit counter.position_tracker.rsaddsspread_amount + commission_amount(spread is not treasury). Not windowed, not USD, not protocol-earned. Do not headline this.uusd→ USTC oracle; LUNC/cLUNC/uluna→ LUNC; UST1/USTR →hub_prices. Never vFDUSD/FDUSD. Never$1UST1 or2.5×USTR (X4 / P522-Q / H1–H8). Stampfee_usdas-of ingest (#568) — do not rewrite history from the live hub.volume_aggregator.rs~5 min + startup. Cache-miss/overviewmust stay O(1) (P550-5, V5). Do notSUM30dswap_events/ fills / wrap rows on GET.Utc::now() − N, not calendar buckets (#576). Fee windows must match. Idle window →$0; unpriced + activity →—(same as volume C3 / W4).ProtocolPage.test.tsx,e2e/protocol-page.spec.ts,indexer/tests/api_overview.rs,make verify-issue-550/569/576/577.skills/AGENTS_FRONTEND_PROTOCOL_STATS.md(P550, P569),docs/indexer-invariants.md,docs/frontend.md§ Protocol,docs/runbooks/overview-global-stats-brin.md,skills/AGENTS_HOOK_COMMISSION.md(L7).Why this is needed
/protocolalso ask how much the protocol earned, in which tokens, and from which product (swap vs book vs wrap).commission_amountundercounts hybrid takes. Placementmaker_fee_amountand wrap-mapper fees never enter Postgres. A naiveSUM(commission_amount)would be wrong and mixed-unit.traders.total_fees_paidis the wrong object. It is trader-paid, lifetime, mixed decimals, and includes spread. Protocol fees are treasury inflows in USD, windowed, sourced.then = 0→ JSONnull, UI em-dash — neverInfinity.Constraints / guardrails
/overviewstays rollup + cheap census + O(1) hub. Do not live-scanswap_events,limit_order_fills, wrap rows, or 60d windows on GET. 60s response cache stays. Fee breakdown JSON must not turn overview into an unbounded payload.spread_amount, Terra Classic burn tax, gas,hook_fee_amount/ AfterSwap hook deductions, LP, book escrow, parked dust.commission_amount+ book taker (limit_order_fills.commission_amountor persistedbook_commission_amount, not both). Placement maker fee is additional. Wrap/unwrap is additional (different contract).SUMis forbidden.humanize_raw_amount+fits_numeric_38_18+ P522-Q / hub. Overflow / non-positive / unpriced → omit that row from USD (omit ≠$0when other priced fees exist; idle window with zero trades →$0).$1UST1 /2.5×USTR. Oracle/hub down → that handle unpriced. Identity is contract/denom (A1); spoof natives skipped.fee_usdat ingest (or at aggregator from stored ingest USD). Do not rewrite historical fee USD from the live hub snapshot (#568).now − 24h / 7d / 30d. Successful refresh zeros a window when events age out. Copy: Last 24h / 7d / 30d — not “today” / midnight reset.pct = (current_W − previous_W) / previous_W × 100whenprevious_W > 0. Previous 24h =[now−48h, now−24h); previous 7d =[now−14d, now−7d); previous 30d =[now−60d, now−30d). No events in prior window,then = 0, or overflow → JSONnull, UI em-dash. NeverInfinity/NaN/"∞%"/ fake0%. Cold start /--fresh/ indexer younger than 2×W → null Δ% (copy must not claim on-chain genesis fees).global_liquidity_snapshotsorformatProtocolPctsemantics that compare stock-to-stock without documenting flow-vs-stock. Reuse the formatter; labels must say the window is trailing fee flow.WRAP_MAPPER_ADDRESS(bech32 allowlist). Random contracts emittingaction=wrapmust not inflate totals. Unknown / empty env → wrap source omitted (not$0pretending the mapper is idle).other; fixed source enum). Oversized JSON is a DoS/UX bug.IndexerOverviewvolume/TVL fields.title/aria-labellike volume. Fees are reference / indexed, not settlement, not CMM bank balance, not “guaranteed USD.”uluna(#276). Per-pair fee pages. CSV export. Changing on-chain fee bps. CoinGeckoliquidity_in_usd. Headliningunique_traders_24h.indexer/into root Docker for cargo (workspace rule). Hostcargo test/make verify-issue-*.Relevant files
indexer/src/indexer/parser.rsbook_commission_amount; parsemaker_fee_amount; ingest wrap-mapper eventsindexer/src/db/queries/swap_events.rsindexer/src/db/queries/limit_order_lifecycle.rsindexer/src/db/queries/limit_order_fills.rsindexer/src/db/queries/volume.rsrefresh_global_statspattern; do not 24h-only INSERT that zeros new columnsindexer/src/indexer/volume_aggregator.rsindexer/src/indexer/pair_price_usd.rsindexer/src/api/overview.rsindexer/src/api/mod.rsindexer/migrations/book_commission_amount/fee_usdon swaps; placement fee; wrap_events; rollup columns; token/source rollup tablesindexer/tests/api_overview.rsindexer/tests/indexer_overview_global_stats.rsindexer/tests/swap_events_hybrid_columns.rsindexer/tests/indexer_volume_window_decay.rsindexer/tests/security.rsfrontend-dapp/src/types/index.tsIndexerOverviewfrontend-dapp/src/services/indexer/client.tsGET /api/v1/protocol/feesfrontend-dapp/src/pages/ProtocolPage.tsxfrontend-dapp/src/components/protocol/ProtocolGlobalStats.tsxfrontend-dapp/src/utils/formatProtocolStats.tsfrontend-dapp/src/utils/trailingWindowCopy.tsfrontend-dapp/src/pages/ProtocolPage.test.tsxfrontend-dapp/e2e/protocol-page.spec.tsdocs/runbooks/overview-global-stats-brin.mddocs/indexer-invariants.mddocs/frontend.mdskills/AGENTS_FRONTEND_PROTOCOL_STATS.mdAGENTS.mdmake verify-issue-<iid>scripts/qa/verify-issue-569.shverify-issue-<iid>Recommended direction
Fee event of record (ingest)
Introduce a canonical fee row (table name may vary:
protocol_fee_events) or additive columns plus a wrap table, as long as the aggregator canSUM(fee_usd)bysource×asset_id× trailing window without double-count.source(enum)swap_ammcommission_amount(pool leg)book_takebook_commission_amountor sum oflimit_order_fills.commission_amountfor that swaplimit_placemaker_fee_amountonplace_limit_orderlimit_order_placements; skip0wrapWRAP_MAPPER_ADDRESSunwrapPersist:
block_timestamp,tx_hash,source,asset_id,amount_raw,decimals,fee_usd(nullable), unique(tx_hash, source, ordinal)so replay / reorg does not double-count (ON CONFLICT DO NOTHING).Parse wrap-mapper wasm from LCD the same way as pair events. Document the exact attrs against current ustr-cmm (post-#9 split fees). Fail closed on missing amount/denom. Do not infer fee as
amount × bpsfrom config at query time (bps can change; historical txs used the then-config).Indexer — USD
At ingest (preferred) or in the aggregator from stored raw+asset: humanize × catalog/hub as of that block’s oracle/hub row when available, else latest-at-ingest snapshot used for
volume_usdtoday. Clamp with the sameLEAST/fits_numeric_38_18pattern as volume USD. Unpriced →fee_usdNULL; still keep the raw token amount for the token table (human units + symbol; USD—).Indexer — rollup (aggregator ~5 min + startup)
Materialize on
global_stats_24h(scalars) and small child tables for breakdowns:Scalars (overview-safe):
total_fees_24h_usdtotal_fees_7d_usdtotal_fees_30d_usdfees_change_24h_pctnullif no baselinefees_change_7d_pctfees_change_30d_pctJSON
"0"only when the window has no fee events (idle). Activity + all unpriced → JSONnull(UI—), same spirit astotal_volume_24h_usdC3.Breakdown (dedicated
GET /api/v1/protocol/feesrecommended):by_source[]:{ source, amount_usd, share_pct }for the 24h window (also 7d/30d as querywindow=24h\|7d\|30dallowlisted — unknown → 400).by_token[]:{ asset_id, symbol, contract_or_denom, amount_human, amount_usd, share_pct }sorted by USD desc, cap 8 +other.limitunused or clamped. No LCD.Keep overview additive scalars so
/protocolcan show three fee StatBoxes without a second payload; fetch breakdown only for the fee panel tables. Alternatively one dedicated endpoint for both if overview size is a concern — then Global stats must not block on it (fee panel owns the query). Prefer overview scalars + dedicated breakdown so volume/TVL cache stays stable.Do not run a 24h-only
INSERTthat leaves fee columns stale (same class of bug as #550/#569).OVERVIEW_GLOBAL_STATS_LIVE=1must not live-SUM 60d fees on GET.Frontend —
/protocolNew panel
protocol-fee-statsimmediately after Global stats and before DEX hub (P550-1 extended: title → global stats → fees → hub → oracle → audit → hooks). Do not merge factory/router into fees. Do not clone onto Swap confirmation.Headlines (StatBox, trailing copy):
Tables (compact,
#489):$0or show$0with em-dash USD when that source is unconfigured, e.g. no wrap mapper — pick one and document; unconfigured ≠ idle).—.Outage: include the fee query in
detectMarketDataOutage/ retry. Missing indexer fields → hide panel or em-dash, not$0invented.Testids (suggested):
protocol-fee-stats,protocol-stat-fees-24h,protocol-stat-fees-7d,protocol-stat-fees-30d,protocol-stat-fees-24h-chg,protocol-stat-fees-7d-chg,protocol-stat-fees-30d-chg,protocol-fees-by-source,protocol-fees-by-token.Docs / skills
New invariant block PFee-1–PFee-n in
AGENTS_FRONTEND_PROTOCOL_STATS.md(or a short sibling skill linked from AGENTS.md). Indexer-invariants row Protocol fees. Runbook: aggregator SQL, freshness (same 15 min stale log as #577 D6 — still serve rollup).make verify-issue-<iid>.Acceptance criteria
/protocolshows trailing 24h / 7d / 30d protocol fee USD and Δ% vs the previous equal window.limit_place, not as swap volume.swap_events/ fills / wrap tables.$0; unpriced activity →—; missing prior window → Δ%—; neverInfinity.--fresh/ young indexer → Δ% empty until 2×W fills; copy does not claim chain genesis.$1UST1; ingest-time USD not rewritten from live hub.make verify-issue-<iid>, e2e smoke testids.traders.total_fees_paidis not the Protocol headline.Test plan (all paths)
Indexer unit / integration (Postgres;
make setup-indexer-postgres)commission_amount> 0, no fills →swap_ammonly; USD = human(ask)×catalog.swap_amm+book_take= pool + fill commissions; not 2× book if bothbook_commission_amountand fills exist.book_takerow.maker_fee_amountindexed;limit_placeUSD; zero fee place skipped.$0(per documented choice).action=wrapfrom a non-allowlisted contract ignored.block_timestamp, no 24h sleep — V3).+50%; prior 0 → null; equal →0%.swap_events/protocol_fee_events/limit_order_fills.window=— allowlist;window=1';drop/javascript:/../→ 400.other;other.amount_usd= remainder.OVERVIEW_GLOBAL_STATS_LIVE=1still must not 60d-SUM fees on GET.updated_at> 15 min logs warning, still serves last fee rollup (D6).api_overview.rs).Frontend unit (Vitest)
nullUSD / pct → em-dash;"0"→$0/0%.titlediscloses trailing window (not calendar).?ticker=still allowlisted; fee panel ignores ticker.E2E smoke (Playwright, 5 workers; no chain required if mocked)
/protocolshowsprotocol-fee-statsafterprotocol-global-statsand beforeprotocol-dex-hub-prices.$0or$…or—acceptable on empty LocalTerra).Docs / verify script
scripts/qa/verify-issue-<iid>.sh: skill invariants, overview keys, EXPLAIN grep, cargo tests, vitest, optional e2e-smoke.docs/indexer-invariants.md+docs/frontend.md+AGENTS.md+ runbook updated.make verify-issue-550569576577still pass.Test plan (attack, hack, abuse)
/overviewand/protocol/feesmust notSUM30d/60d event tables. EXPLAIN + integration assert. Burst within 60s served from cache.window,tickerleftovers,format,sorton the new route: allowlist only; parameterized SQL (security.rspattern).WRAP_MAPPER_ADDRESSmatch (bech32, not substring).denom=ugemsymbol=USTRmust not price as USTR (A1).book_commission_amount+ N fills → USD equals pool + book once.spread_amount, InstantWithdraw tax, orhook_fee_amountdo not increase protocol fees.fits_numeric_38_18); no Inf JSON.symbol<img onerror>/javascript:rendered as text, not HTML.?ticker=for API paths; P550-2 unchanged.= + - @(text_csv.rs/ #432). Prefer no CSV in v1.OVERVIEW_GLOBAL_STATS_LIVE=1must not become a 60d fee scan gadget.RATE_LIMIT_RPSstill applies."Internal server error"; no sqlx text (internal_err).fee_usdfor old txs (#568 analogue).WRAP_MAPPER_ADDRESSwith spaces /terra1+ newline / wrong HRP rejected at ingest; noLIKE %match.sourcestrings from DB not interpolated intoeval; enum map in Rust + frontend allowlist.Verification criteria
Manual
/protocol(LocalTerra or prod indexer): fee panel under Global stats; 24h/7d/30d USD; Δ% em-dash until 2× windows exist; source + token tables; wrap rows only after mapper txs; hybrid swap increases AMM + book, not double book; unwrap does not include burn tax; hub/oracle still follow P550-1.Done when ACs 1–14 pass, verify script is green, and invariants are in skills + indexer-invariants + frontend.md.
mentioned in merge request !1112
mentioned in commit
d1d2222cccmentioned in commit
833687d003Post-merge note (!1112 on
main)Merged: https://gitlab.com/PlasticDigits/cl8y-dex-terraclassic/-/merge_requests/394
Sanity on
main(make verify-issue-586): docs, indexer lib (protocol_fees10/10), Protocol RTL, Playwright e2e-smoke (5 workers) passed. Indexer integration (indexer_protocol_fees/api_overview) and related 550/569 integration failed here withpool timed outto127.0.0.1:5432— composepostgreswas not running on this host. Not treated as a code defect in !1112; re-run aftermake setup-indexer-postgres.Still open vs AC / test plan (not blockers for the merge):
indexer/migrations/20260821120000_protocol_fees.sqlon QA/prod; set indexerWRAP_MAPPER_ADDRESSor wrap/unwrap sources stay omitted.limit_place, wrap/unwrap from pinned mapper only; unwrap must not count InstantWithdraw burn tax.protocol_fee_events, not LCD wasm attrs throughparser.rs.window=24h(useProtocolFeesQuery.ts); 7d/30d breakdown is API-only.Do not treat
/protocolfee USD as CMM bank balance. Windows are trailing flow (#576).mentioned in issue #590
marked as related to #590
Follow-up tracker: #590 (indexer migrate + LocalTerra fee smoke).
Update: after
make setup-indexer-postgres,cargo test --test indexer_protocol_fees --test api_overview -- --test-threads=1is green (11 + 1). The earlier verify-586 integration FAIL was Postgres down, not !1112.mentioned in issue #613
marked as related to #613
mentioned in issue #614
marked as related to #614
mentioned in commit
4de6d8395cmentioned in issue #631
marked as related to #631
mentioned in issue #652
mentioned in issue #653
mentioned in issue #667
marked as related to #667
mentioned in issue #682
mentioned in issue #683
mentioned in issue #687
mentioned in issue #689
marked as related to #689
ops-worker referenced this issue2026-09-06 12:05:09 +00:00