fix: DeFiLlama fees adapter throws on null — unblock dimension-adapters#8987 #687
Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:implement
agent:implement
agent:open_issues
agent:ready
agent:research
agent:security_audit
agent:verify
architecture
backend
blocker:hybrid
blocker:launch
blocker:limit-orders
blocker:v2
block:log_only
block:security
bug
ci
contracts
correctness
deploy
dev
devops
docs
documentation
duplicate
e2e
enhancement
epic
feature
frontend
functional-completion
gas
good first issue
governance
help wanted
high-risk
hooks
hybrid
indexer
infra
infrastructure
integrators
invalid
launch-blocker
limit-orders
localnet
localterra
low priority
missing-implementation
needs-design
ops
performance
priority
high
priority
medium
product
qa
QA
question
ready
ready
research
scripts
security
security-hardening
smartcontracts
tech-debt
testing
ux
UX
v2
verification
wontfix
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
code/cl8y-dex-terraclassic#687
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Llama maintainer review of dimension-adapters#8987 (
fees/cl8y-dex): recent UTC days staynulland the adapter throws. Volume for the same days can be a number. This leftover does not reopen closed #631 (listing + daily GET) or #683 (CL8Y / economicfee_usdstamps). Bundle indexer GET contract + vendored / upstream adapter + start-date / 404 backfill into one ticket.Live probe (2026-08-27): yesterday–6d now return numeric
daily_fees_usdafter economic stamps; 2026-05-01 still 404; ~2026-08-13 and older 404; 2026-08-17–21 return volume withdaily_fees_usd: "0". The throw-on-null path is still in the GitHub PR and inscripts/defillama/fees/index.ts. Any future unpriced fee event (or Llama backfill of a 404 day) failspnpm test fees cl8y-dexagain.Current codebase
Upstream PR (what Llama runs)
dimension-adapters#8987 adds
dexs/cl8y-dex/index.tsandfees/cl8y-dex/index.ts. Both are version 1, pinGET https://indexer.dex.cl8y.com/api/v1/defillama/daily?timestamp=${options.startOfDay}, and throw when the USD field is JSONnull:asNumberOrNull(data?.volume_usd)→ throwcl8y-dex dailyVolume unpriced or missingasNumberOrNull(data?.daily_fees_usd)→ throwcl8y-dex dailyFees unpriced or missinghttpGet(axios) also throws on HTTP 404. CI previously failed withRequest failed with status code 404before Coolify shipped the route. The fees review is the remaining merge block: maintainerpnpm test fees cl8y-dexover the last few UTC days hit 200 +daily_fees_usd: null+ throw.Fees mapping in the PR: labeled
METRIC.SWAP_FEES/DEPOSIT_WITHDRAW_FEES/MINT_REDEEM_FEESfromfees.*, residualdaily_fees_usd - labeled,dailyRevenuefromdaily_revenue_usd(fallback total),dailySupplySideRevenue: 0.Indexer daily GET (#631)
indexer/src/api/defillama.rsdefillama_daily_statsrow → 404.daily_usd_field_fail_closed(fee_events, fee_unpriced, &fee_usd)unpriced_count > 0across all sources → wholedaily_fees_usd/ revenue fields are JSONnull.fee_fieldswap_amm…ust1_redeem. Missing source →"0".indexer/src/db/queries/defillama.rsSUM(fee_usd)fromprotocol_fee_events; gem-pair txs excluded for pair sources; wrap/window always included. Inserts all sevenFeeSource::ALLrows.indexer/src/indexer/defillama.rsdaily_usd_field_fail_closedactivity_count <= 0) →"0". Else ifunpriced_count > 0or priced sum<= 0→None.DAILY_LOOKBACK_DAYS = 8defillama_daily_*; days never refreshed stay 404.ADAPTER_START_UTC_DAY = 1_777_593_600(2026-05-01)scripts/defillama/gems.jsADAPTER_START. Llama will request every day from that start.Tests already encode fail-closed:
indexer/tests/indexer_defillama.rsunpriced_active_is_null_idle_is_zeroassertsdaily_fees_usdis JSON null when a swap_amm row hasfee_usd = NULL.Conflict with #683 (closed)
#683 EFee-6: Protocol headlines are partial priced SUM; idle
"0"; all activity unpriced →null. Llama was documented as inheriting stampedfee_usd.Llama GET does not match that. One unpriced CL8Y
limit_place/ ask-sideswap_amm(the original #683 hole) nulls wrap + UST1 window + priced pair commission for the whole UTC day. Volume for the same day can stay a number becauseswap_events.volume_usdis offer/quote hub catalog, not ask-side CL8Y.Vendored copies (drift)
scripts/defillama/dexs/index.tsmapDaily.js+options.http.get; still throws on null.startis unixADAPTER_START, PR uses"2026-05-01".scripts/defillama/fees/index.tsmapped.dailyFeesthen adds each breakdown label again — double-counts if this file is copied upstream. PR adds labeled METRIC amounts + residual only.scripts/defillama/dimensions/mapDaily.jsasNumberOrNulltreats JSON null as null (correct for L631-7). Unit tests do not cover adapter throw / 404 / partial SUM.Docs still say #8987 is draft until Coolify ships the route (
docs/DEFILLAMA.md,scripts/defillama/README.md). The route is live; the leftover is null/404 contract vs Llama crawler.Why the new implementation is needed
pnpm test/ production fetch treat adapter exceptions as adapter breakage. JSONnullis our honest “unpriced” signal; the adapter must not turn a partial-price day into an exception once the GET headline is a number.start: "2026-05-01"+ lookback 8 days → Llama historical backfill 404s from May through mid-August. Axios throws. Same class of “adapter broken” as null fees.daily_fees_usd: "0"(no fee rows / ingest gap). That must stay documented idle-or-pre-ingest zero, not a silent$0covering unpriced CL8Y. After #683 stamps, refresh those days if events exist.Constraints and guardrails
liquidity_in_usd/overview.total_volume_24h_usdas Llama volume or TVL (A1 / L631-1).spread_amount, burn tax, gas, hooks, or community-tax extra-debit asdailyFees(L631-6).dailySupplySideRevenueto a residual. SSR stays0. Ignore review bots that reclassify leftover USD as LP commission.$1or USTR=2.5×USTC. Named wrap substitution only (cLUNC→uluna, cUSTC→uusd).from/torange dump onGET /defillama/daily. Single UTC-daytimestamponly (L631-7 / A12).swap_events/protocol_fee_eventson GET. Refresh stays the ~5 min volume loop.indexer/into root Docker for cargo.scripts/defillama/in lockstep.startearlier than the first day the daily API can return 200 with the documented idle/unpriced contract.daily_fees_usd/ revenue) must match EFee-6. Per-sourcefees.swap_ammetc. may still benullwhen that source has unpriced events.nullremains valid only when fee activity exists and the priced SUM is empty (all unpriced). Adapter may throw on that (fail closed) or skip the day — pick one, document it, and keep volume/fees consistent.startto the first 200 day over fabricating May–August zeros. Optional one-time backfill ofdefillama_daily_*from that start is OK; do not widen GET into a live scan.https://dex.cl8y.com. Llama is server-side. Host pin stayshttps://indexer.dex.cl8y.com(A18).Relevant files
indexer/src/api/defillama.rsindexer/src/indexer/defillama.rsindexer/src/db/queries/defillama.rsindexer/src/db/queries/defillama_assets.rsindexer/migrations/20260825150000_defillama_daily.sql,20260825160000_defillama_daily_assets.sqlindexer/tests/indexer_defillama.rsindexer/src/indexer/economic_usd.rs,indexer/src/indexer/protocol_fees.rs(overview_fee_usd_field)scripts/defillama/dexs/index.tsscripts/defillama/fees/index.tsscripts/defillama/dimensions/mapDaily.js,mapDaily.test.jsscripts/defillama/gems.jsdocs/DEFILLAMA.md,skills/AGENTS_DEFILLAMA.md,docs/indexer-invariants.md(DeFiLlama UTC-day + Economic fee USD)scripts/qa/verify-issue-631.sh,Makefileverify-issue-631/verify-issue-683dexs/cl8y-dex,fees/cl8y-dex)Recommended direction
daily_fees_usd/daily_revenue_usd/daily_protocol_revenue_usd= SUM of priced sourceamount_usd. JSONnullonly when totalevent_count > 0and priced SUM is 0. Do not ORunpriced_countacross sources into a whole-day null while wrap/window/hub sources are priced. Keep per-sourcefees.*fail-closed.refresh_defillama_daily(or a one-shot from adapter start) so Coolify rows pick up #683 NULL-onlyfee_usdbackfill. Days with volume and true zero fee events stay"0".start. Set GitHub +scripts/defillama/gems.jsADAPTER_STARTto the first UTC day that returns 200 (today that is ~2026-08-17, not 2026-05-01). Document indocs/DEFILLAMA.md. Version stays 1 (cannot split hourly)."0". After (1), last-few-days tests must return numericdailyFees. If a day is still all-unpriced (null), either throw with a stable message or omit the day — do not map unpriced →$0. Catch 404 only ifstartstill precedes coverage; preferred fix is (3), not swallow-404-as-zero.scripts/defillama/fees/index.tsso it does notaddUSDValue(total)then add breakdown again.startstring vs unix: one canonical form in both trees.pnpm test fees cl8y-dexandpnpm test dexs cl8y-dexsucceed for yesterday andstart, and the GitHub PR is re-tested.make verify-issue-<this-iid>that covers indexer partial-SUM tests + mapper/adapter-start pins +verify-issue-631/683related. KeepVERIFY_ISSUE_631_SKIP_RELATEDpattern if needed.Acceptance criteria
GET /api/v1/defillama/daily?timestamp=<yesterday_00:00_utc>returns 200 with numericdaily_fees_usdwhen any priced treasury fee exists that day (wrap/window/hub/economic). One unpriced source does not null the headline.nullonly when fee events exist and none havefee_usd > 0. Idle (no fee events) stays"0"even if volume is non-zero.fees.swap_amm(etc.) may benullindependently; SSR remains"0";daily_revenue_usdequalsdaily_fees_usd.startis the first 200 UTC day.GETforstartis 200;GETforstart - 86400is 404 (or start is the genesis row). Llama backfill does not walk May 2026 404s.fees/cl8y-dex+ in-repo copy: no throw on numeric /"0"days; no double-count of breakdown vs total; METRIC labels ⊆breakdownMethodology.pnpm test fees cl8y-dexandpnpm test dexs cl8y-dex(yesterday andstart) succeed; output pasted on #8987 (operator).make verify-issue-<this-iid>green on host Postgres.make verify-issue-631andmake verify-issue-683stay green.Test plan (all paths)
Indexer (Postgres) —
make setup-indexer-postgresswap_amm(hub token) + pricedwrap+ unpriced CL8Ylimit_place(fee_usdNULL) + gem-pairswap_amm(excluded).daily_fees_usd= priced swap_amm + wrap only (partial SUM).fees.limit_placeisnull. Gem pair not in any source.null(not"0"). Volume may still be a number.daily_fees_usd"0",volume_usdpriced or fail-closed on its own unpriced swaps.book_takefromlimit_order_fills.commission_amountonly (not swap-levelbook_commission_amount); volume still L10 parent swap.protocol_fee_events.fee_usdbackfill fixture, refresh flips headline fromnullto partial SUM; non-null stamps are not rewritten (C568-1).Mapper / vendored adapters (no Llama SDK)
node --test scripts/defillama/dimensions/mapDaily.test.js(and fees start pin).mapFees:"0"→ 0; JSON null → null; numeric string → number. SSR 0. Breakdown keys ⊆ methodology.daily_fees_usd10, labeled 10 → no residual. Labeled 8 + total 10 → residual 2 without double-counting 10+8.dailyUrl(ADAPTER_START)hits the first 200 day, not 2026-05-01 unless that day is 200.liquidity_in_usd/total_liquidity_usd.Upstream clone (operator)
pnpm test fees cl8y-dexyesterday UTC → numeric dailyFees, revenue = fees, SSR 0.pnpm test fees cl8y-dex <start>→ 200 path, no 404.pnpm test dexs cl8y-dexsame days → numeric dailyVolume.Live Coolify
startcurl as inscripts/defillama/README.md.start.Test plan (attack, hack, abuse)
liquidity_in_usdas Llama TVL/volumedailyVolumelimit_order_fills.$0so Llama stops throwingnullor skip, not zero.spread_amount/ community-tax extra-debit as residualdailyFeesdailySupplySideRevenue$0from May 2026startor backfill real rows.CL8Yon a native/gem to mint economicfee_usddaily_fees_usdtimestampinjection / unaligned / futurefrom/torange dumpindexer.dex.cl8y.comonly.createBalancesfee_usdwhen the mark jumpsSUM(protocol_fee_events)Verification criteria
Issue is done when all of the following are true:
GET /api/v1/defillama/daily?timestamp=<yesterday_utc>returns 200 with numericdaily_fees_usdwhenever priced fee events exist; maintainer-style “last few days” do not return headlinenullsolely because one source is unpriced.startmatches the first 200 UTC day; GitHub +scripts/defillama/gems.jsagree.pnpm test fees cl8y-dexandpnpm test dexs cl8y-dexlogs (yesterday + start). Maintainer can re-run without throw.make verify-issue-<this-iid>passes (make setup-indexer-postgres). Relatedverify-issue-631andverify-issue-683pass.Close is blocked if Llama fees tests still throw on recent days, if unpriced activity is published as
$0, if SSR becomes non-zero, iffrom/tois added, or if TVL/volume start using indexer USD / CGliquidity_in_usd.Related
fee_usdstamps (closed; EFee-6)marked as related to #631
marked as related to #683
mentioned in merge request !1187
mentioned in commit
8801b0fcf2mentioned in commit
6ae8b7270bmentioned in issue #688
marked as related to #688
mentioned in issue #684
Merged as !1187 onto
main(8801b0fc). In-repo AC for indexer partial-SUM headline,start2026-08-17/1786924800, METRIC+residual, andmake verify-issue-687passed. Makefile conflict with #686 leftover verify resolved (kept both targets).This issue stays open. Close gate is operator, not GitLab merge:
start: "2026-08-17", METRIC+residual; no throw on"0"; do not map JSONnull→$0; do not swallow 404 as zero).pnpm test fees cl8y-dexandpnpm test dexs cl8y-dex(yesterday + start) on #8987.scripts/defillama/README.md.Coolify + GitHub ops also listed on leftover #688. Do not reopen #631 / #683.
CI was
failed(ci_quota_exceeded); not waited; not a product signal.Ops 2026-08-27 — leftover #688 / GitHub #8987
Indexer GET on Coolify already returns priced-SUM headlines (
daily_fees_usdis a number or"0", notnull). Prior-start day stays 404.Patched upstream https://github.com/DefiLlama/dimension-adapters/pull/8987
6e87a48:start: "2026-08-17". METRIC groups + residual unchanged. Still throws on JSONnull. Does not map HTTP 404 to$0.pnpm teston that branch (also pasted on #8987):Live:
timestamp=1786924800→ 200, fees"0", volume ~716.51.timestamp=1786838400→ 404.Local
make verify-issue-687green (includes 631 + 683).Do not reopen #631 / #683. Close this issue after Llama merges #8987 (crawler test gate is green on our fork). Parent leftover: #688 (Coolify GT backfill still open).
/agent implement
/agent implement
/agent implement
/agent implement
Verified 2026-09-24.
make verify-issue-687passed 7/7 in an isolated Postgres database, including #631 and #683. Live GET: yesterday UTC and adapter start 1786924800 returned 200 (numeric fees / idle "0"); 1786838400 returned 404 as expected. GitHub dimension-adapters#8987 merged 2026-08-27. Newer ops tracker #688 is closed; no #687 verification remains. Related: #631, #683, #688. Documentation, invariant crosslinks, third-party playbook, and verifier assertion updated in local worktree branch verify/issue-687.