bug: CoinGecko oracle polls 403 without a descriptive User-Agent #579
Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:implement
agent:implement
agent:open_issues
agent:ready
agent:research
agent:security_audit
agent:verify
architecture
backend
blocker:hybrid
blocker:launch
blocker:limit-orders
blocker:v2
block:log_only
block:security
bug
ci
contracts
correctness
deploy
dev
devops
docs
documentation
duplicate
e2e
enhancement
epic
feature
frontend
functional-completion
gas
good first issue
governance
help wanted
high-risk
hooks
hybrid
indexer
infra
infrastructure
integrators
invalid
launch-blocker
limit-orders
localnet
localterra
low priority
missing-implementation
needs-design
ops
performance
priority
high
priority
medium
product
qa
QA
question
ready
ready
research
scripts
security
security-hardening
smartcontracts
tech-debt
testing
ux
UX
v2
verification
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
code/cl8y-dex-terraclassic#579
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Indexer CoinGecko polls for USTC, LUNC, and vFDUSD/FDUSD return HTTP 403 with
Please add a descriptive User-Agent to your request. The oracle HTTP client is built with timeout only — noUser-Agent. CoinGecko is a soft-fail fallback (alternate ticks); USTC/LUNC still average KuCoin+MEXC, but vFDUSD’s CEX ticker has no KuCoin pair, so a CG miss leaves MEXC-only (logs:from 1/2/from 1/1sources).Observed (columbus-5):
Related: #515 / #550 (ticker oracle), #520 (extra CEX sources — do not substitute for a User-Agent). CEX FDUSD labeled as vFDUSD/USD is a separate identity issue.
Current codebase
indexer/src/indexer/oracle.rsrun_oracle_loop:Client::builder().timeout(Duration::from_secs(10)).build(). Nodefault_headers, nouser-agent. reqwest’s default UA is empty orreqwest/x.y— CoinGecko free API now rejects that with 403 (not 429).fetch_coingecko_url: 429 or JSONerror_code: 429→OracleError::RateLimited(debug log). Any other non-success →OracleError::Parse("CoinGecko HTTP {status}: {body…}")→ warn. 403 therefore looks like a parse error and is not retried as rate-limit.tick_count % 2 == 0) to spare quota. KuCoin+MEXC every tick.kucoin_symbol() = None). Alternate tick: MEXC + CG = 2 slots; CG 403 → 1/2. Non-CG tick: MEXC only = 1/1.fetch_coingecko_maps_429_to_rate_limitedcovers 429.coingecko_body_is_rate_limitedis 429-only. No 403 / User-Agent test. Wiremock helpers do not assert request headers.Why this is needed
Constraints / guardrails
User-Agenton the oraclereqwestclient (all KuCoin/MEXC/CG requests on that client is OK). Map CoinGecko 403 whose body asks for User-Agent to a distinct, actionable log (not “Parse error”). Optional: treat 403-UA as a one-lineerroronce, thendebug, to avoid spam — do not hide a 403 that is not UA-related (key/ban).cl8y-dex-indexer/<version> (+https://gitlab.com/PlasticDigits/cl8y-dex-terraclassic). Do not impersonate browsers, wallets, or other products. Do not put emails, tokens, hostnames of the operator, or wallet addresses in the UA. Version may come fromCARGO_PKG_VERSION.RateLimited(debug). 403 UA-missing is not rate-limited success. After the fix, 403 should be rare; tests must still distinguish 429.$1hardcodes, orvolume_usdcatalog. Soft-fail: CG down still must not blank KuCoin/MEXC averages.fetch_coingecko_url— keep truncation. Do not put API keys in UA or query string unless a documented paid plan is added in a separate issue (this issue is UA-only; no CoinGecko API key required for the free simple/price endpoint).Relevant files
indexer/src/indexer/oracle.rsfetch_coingecko_url, error mapping, testsindexer/tests/api_oracle.rsdocs/runbooks/indexer-external-oracle.mdskills/AGENTS_INDEXER_EXTERNAL_ORACLE.mdscripts/qa/verify-issue-<iid>.shRecommended direction
Client::builder().user_agent(ORACLE_USER_AGENT).timeout(...)with aconst/CARGO_PKG_VERSIONstring as above.fetch_coingecko_url, if status is 403 and body mentionsUser-Agent(orerror_code403 with that message), return a dedicatedOracleErrorvariant (e.g.MissingUserAgent) or keepParsebut stable wordingCoinGecko HTTP 403: User-Agent requiredwithout dumping the whole JSON every tick.user-agentmatches the const; 403+UA message ≠RateLimited; 429 stillRateLimited.make verify-issue-<iid>.Acceptance criteria
reqwestclient sends a descriptive, non-browserUser-Agentincluding the indexer name and a repo URL or package name.RateLimited; 403 User-Agent missing is not classified as 429.$1peg. No ticker rename.make verify-issue-<iid>green without live CoinGecko.Test plan (all paths)
RateLimitedRateLimited; error distinguishable; body truncated.usdfetch_coingecko = falseTest plan (attack, hack, abuse)
GITLAB_TOKEN/ RPC URLcoingecko_id()is fixed perOracleTicker; no user string in URLVerification criteria
Ship is done when AC1–AC5 pass, T1–T12 and A1–A8 are recorded (or waived with reason), and a local wiremock CG 200 with UA matcher is green (
make setup-indexer-postgresnot required for--lib oracle).marked as related to #515
marked as related to #550
marked as related to #520
mentioned in issue #580
marked as related to #580
mentioned in merge request !1098
mentioned in merge request !1103
mentioned in merge request !1104
mentioned in commit
a30edba3c7Merged via !1098.
make verify-issue-579passed (docs +--lib oracleUser-Agent / 403 vs 429). Remaining: production indexer must be redeployed so live CoinGecko polls send the descriptive User-Agent. No code follow-up.mentioned in issue #583
marked as related to #583
mentioned in issue #619
mentioned in issue #682
mentioned in issue #683