Indexer: MegaETH miniBlocks WSS stream (additive) + frontend realtime head pill #237
Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:open_issues
agents
agent:security_audit
agent:verify
area
timecurve
arena
arena-v2
audit
block:feature_review
block:security
bug
contracts
design
devops
docs
documentation
enhancement
epic
feature
frontend
indexer
low priority
megaeth
operations
ops
performance
QA
qa checklist
referrals
removal
security
smart-contracts
tech-debt
testing
time-arena
timecurve
visual-qa
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
code/yieldomega#237
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Add additive MegaETH Realtime API ingestion over WebSocket (
eth_subscribe→miniBlocks) so yieldomega.com can show a live mini-block head (number + timestamp in ms) next to the existing “latest indexed block” pill on the agent card. RPC block polling, per-block transactional ingest, and reorg rollback must remain unchanged — they are the authoritative path into Postgres; WSS is a best-effort realtime lane that must never be assumed reliable.Why (MegaETH specifics)
eth_blockNumber/ ~1s EVM blocks are too coarse for “live” UX.miniBlocks— each notification includes the full set of transactions and receipts for that mini-block (Realtime API,eth_subscribeminiBlocks).eth_chainId~every 30s); limits: 50 connections / VIP endpoint, 10 subscriptions / connection (see megaeth-developer skill — WebSocket section).https://mainnet.megaeth.com/rpc(WSS URL from operator docs / env).Related MegaETH methods (not all in scope for this issue, but document for follow-ups):
eth_subscribelogswithfromBlock/toBlock="pending"eth_subscribestateChangeseth_callAftereth_getLogsWithCursorNon-negotiable constraints
ingestion::runRPC loop (indexer/src/ingestion.rs),rollback_after/find_common_ancestor(indexer/src/reorg.rs), orINV-INDEXER-140per-block DB transactions (#140).chain_pointeror event tables. If WSS ever drives writes, it must reconcile against RPC before commit (default: no WSS writes in Phase 1).confirmation: "indexed" | "wss_realtime") and/or separate endpoints — avoid overloadingmax_indexed_blocksemantics onGET /v1/status.Proposed architecture (phased)
Phase 1 — Realtime head fan-out (this issue’s demo goal)
Indexer
eth_subscribe("miniBlocks", …)(params per MegaETH docs).MiniBlockHead:{ mini_block_number, timestamp_ms, received_at_ms }— not written to Postgres event tables in Phase 1.eth_blockNumber/ latest mini-block query for lag metrics (operator-only).HTTP / streaming to frontend
GET /v1/realtime/mini-block-head(JSON snapshot), and/orGET /v1/realtime/stream(SSE) or WebSocket from indexer → browser (single upstream WSS per indexer process; do not open per-user MegaETH WSS from the browser — rate limits + secrets).x-schema-version; document indocs/indexer/design.md.Frontend (yieldomega.com agent card)
IndexerStatusBar“latest indexed block … · live” (IndexerStatusBar.tsx, fed byGET /v1/status/IndexerConnectivityContext):TimeCurveSimpleAgentCardfooter region if Simple hides global footer (#232).data-testidhooks for Playwright; manual QA checklist row.Phase 2+ (separate issues / subtasks — outline only)
provisionaluntil RPC catch-up).REORG_STRATEGY.mdrules — RPC parent-hash walk remains canonical (#140,indexer/REORG_STRATEGY.mdopen question on streaming vs polling).eth_callAfterfor approval → swap / buy flows (wallet RPC, not indexer).eth_getLogsWithCursorfor large ranges whereeth_getLogshits limits.Configuration
INDEXER_WSS_URL(or derive fromRPC_URLwithwss://scheme) — optional; when unset, realtime task idle, API returns explicit “wss disabled”.INDEXER_WSS_ENABLEDdefault off in local Anvil stack; document mainnet WSS URL for operators testing Phase 1.INDEXER_RPC_REQUEST_TIMEOUT_SECon HTTP RPC only; WSS ping/keepalive separate.Testing
MiniBlockHead; reconnect backoff; provenance fields on API DTOsAdd invariants:
INV-INDEXER-*-WSS-*,INV-FRONTEND-*-WSS-HEAD-*indocs/testing/invariants-and-business-logic.md.Acceptance criteria (Phase 1)
miniBlockson MegaETH mainnet WSS when enabled; keepalive documented and implemented.confirmation: wss_realtime(or equivalent) separate frommax_indexed_block/chain_pointer.indexer/README.md,docs/indexer/design.md— dual-path diagram + mainnet-only WSS testing note.References
indexer/REORG_STRATEGY.mdOut of scope (this issue)
eth_callAfter/eth_getLogsWithCursorimplementation (noted for follow-up)Discussion notes (architecture review + Q&A)
Captured from design review thread — supplements the issue body with pitfalls, protocol choices, and prod split (ingestion indexer vs HTTP indexer).
Q: SSE or WSS for browser ↔ indexer?
A: SSE on the HTTP indexer (MegaETH WSS only on ingestion ↔ node).
eth_subscribeminiBlocks)NOTIFY(or throttled upsert)EventSource+ auto-reconnect; we own both sidesWSS browser↔indexer only if we later need bidirectional/binary/high-frequency per-client protocols. For a head pill, throttled SSE (4–10 Hz display) is enough.
Also expose
GET /v1/realtime/mini-block-headsnapshot for agents and SSE connect/bootstrap.Q: Prod has separate HTTP indexer (client API) and ingestion indexer (same code, different env).
node WSS → ingestion → Postgres → HTTP → SSE— is Postgres fast enough? Separate SSE service?A:
miniBlockspayloads (txs + receipts) to Postgres at ~50–100 notifications/sec — wrong workload, WAL bloat, contends withINV-INDEXER-140block transactions.timestamp_ms+received_at_ms), throttled to ~4–10 writes/sec even if WSS fires faster. Postgres latency for one-row upsert is sub-ms; end-to-end tens of ms is fine for UI.broadcaston the HTTP tier; fan-out fromLISTENonpg_notify(or 250ms–1s poll fallback).INDEXER_WSS_ENABLEDthere; off on HTTP tier). Avoid duplicate MegaETH subscriptions.Recommended prod topology:
Scale-out: multiple HTTP replicas each
LISTEN+ SSE OK; only one ingestion instance should hold the MegaETH WSS subscription (leader lock / single replica).Phase 1 shortcut: skip SSE initially — throttled head row +
GETpoll @ 1s — then add SSE on HTTP when needed.Architecture pitfalls (from review)
Two different “heads” —
max_indexed_block/chain_pointer= EVM block (RPC, reorg-safe). WSS pill = mini-block (~10ms). Numbers will diverge; never label WSS lag as “indexer blocks behind” without units. Strong UX copy: realtime preview · not indexed.False freshness — WSS head can run ahead of anything in
/v1/timecurve/buys, podiums, etc. Phase 1 in-memory/head-only is safe for DB corruption but risky for product semantics.Indexer→browser streaming — SSE still needs heartbeats, stale detection (
last_wss_at_ms), proxy idle timeouts; many tabs = many SSE connections on HTTP tier (acceptable; not many MegaETH WSS).WSS SPOF — VIP disconnect → pill stale; RPC ingest may continue on HTTP RPC. Reconnect must resync head via RPC after gap; detect sequence regressions / gaps.
Phase 2 underestimate — Queue + “RPC confirm” before promoting WSS data to
idx_*is hard: EVM-block-oriented schema today, ~100 mini-blocks/s if processing full payloads, idempotency keys, reorg retraction if we ever stream logs from WSS.Reorg —
REORG_STRATEGY.mdtiers are EVM-block based. WSS tier must stay non-authoritative until reconcile design exists.Mainnet-only WSS testing — Anvil/testnet won’t exercise miniBlocks; CI needs golden notification fixtures + optional mainnet smoke.
Config —
INDEXER_WSS_URLmust matchCHAIN_ID; fail closed underINDEXER_PRODUCTION=1(same spirit as #156).Three clocks — RPC ingest pointer,
chain_timerpoller (#48 / #216), WSS mini-block head. Sale phase stays on/v1/timecurve/chain-timer(RPC-backed); WSS pill is display-only.Frontend — Don’t re-render agent card at 100 Hz; throttle display + mark stale if no event ~2s; a11y (constant announcements).
Scope creep —
eth_callAfter= wallet RPC (frontend), not indexer.eth_getLogsWithCursor= ingestion backfill, not realtime head.Decisions / open items for implementation
chain_pointer/ event tablesconfirmation: wss_realtimevs indexedNOTIFY; throttle writeseth_callAfter/eth_getLogsWithCursorAcceptance addendum (prod)
INDEXER_WSS_ENABLEDonly on ingestion deployment; HTTP deployment does not open MegaETH WSS.indexer_state.wss_mini_block_head), notchain_pointer.LISTEN+ SSE route;wss_alive/last_wss_at_mson status or realtime payload.Clarification — real goal (not just a mini-block # pill)
Follow-up discussion: the mini-block number in the UI is a pipeline smoke test. The actual product goals are:
block_timestamp_secticks).TimeCurveBuy, not when the EVM block is ingested into Postgres (~1s+ later).A throttled head-only Postgres row alone does not achieve either goal.
Current bottlenecks (repo today)
useTimecurveHeroTimer→GET /v1/timecurve/chain-timer~1s poll;block_timestamp_sec(whole seconds);setInterval(..., 1000)fetchTimecurveBuys~5s poll (backoff) fromidx_timecurve_buyGET /v1/timecurve/podiumslive prediction from Postgres;usePodiumScoreClockticks 1sPostgres is not “too slow” for queries — data arrives too late for ms UX.
Revised architecture (goal-aligned)
MegaETH guidance mapping:
miniBlockshead or dedicated clock field)chain_clock_msrow +NOTIFY; SSE is sufficient for fan-outlogswithfromBlock/toBlock="pending", filter TimeCurve +Buytopicidx_timecurve_buy; provisional not authoritativePrefer
logs(pending) over persisting fullminiBlocksJSON for buy latency — same decoder as RPC path, smaller payloads.Two kinds of “accurate”
deadlinestill from TimeCurve RPC (chain-timer~1s OK)Buyover SSE (confirmation: wss_realtime)confirmation: indexed)UI must label provisional vs indexed until RPC confirms; retract provisional rows on reorg.
Suggested implementation notes
Timer (ms)
chain_timestamp_ms,mini_block_number(optional),observed_at_ms.chainNowMsfrom stream;remainingMs = deadlineMs - chainNowMs; display format with sub-second digits; local tick 50–100ms between SSE events if needed.Buy, refreshdeadline(softchain-timerpoll or targetedeth_call) — timer resets onchain at buy; ms clock alone is insufficient.derivePhase(#48) from WSS ms alone without same sale-start/deadline/ended rules as today.Last buy / recent buys (ms)
Buywith existing decoder → push SSE payload (buyer, charm, tx_hash, log_index, timer fields,block_timestamp_ms).fetchTimecurveBuys+/v1/timecurve/podiumsas reconcile.observed_at_msor chain ts ms.Cross-process (prod split)
pg_notifysmall JSON or direct internal fan-out if co-located.LISTEN→broadcast→ SSE to browsers (no third “SSE indexer”).Pitfalls (goal-specific)
Buy, not only advance clock ms.chain-timer+ledgerSecIntForPhase(#48); WSS is display/latency layer.Issue scope split (suggested)
logs(pending) + decodeBuy+ SSE to client → recent buys + Last Buy row update in ms; merge with indexed poll; provisional vs indexed provenanceclock_ms→ hero timer ms + deadline refresh onBuyPhase A + B are the product; C is optional operator/debug chrome.
Acceptance criteria addendum (product)
Buyappears in Simple live buys / Last Buy beforemax_indexed_blockadvances.derivePhase/ hero timer sale rules remain consistent with #48 when indexerchain-timeris set.Decisions table (update)
logspending, not full miniBlocks in PGTiered realtime — different Hz per SSE stream
Principle: “Realtime” is not one global number. Each UX surface has a different freshness requirement, onchain driver, and cost (WSS decode, PG, SSE fan-out, React renders). Use separate logical streams (multiplexed on one SSE connection with
type+seq, or separate URLs) each with an explicit max emit Hz (server) and max render Hz (client).Stream catalog (proposed)
clockminiBlockshead ts or clock fieldwss_realtimefor clock; deadline from RPCtimer_paramsTimeCurveat a block tagchain-timerbatchbuy/endSale/ extension eventsbuyBuyvisible on executelogspending,Buytopicpodium_last_buyBuybuystream or indexer/podiumsbuy); full snapshot ≤2buyevent; full reconcile ≤0.5podium_snapshotwarbow_feedchain_statusGET /v1/status+ WSS healthminiblock_debugwss_realtimeWhy not one SSE at 100 Hz?
endSale, etc. — 1 Hz + triggers is enough.Multiplexed SSE envelope (sketch)
Server-side per-type throttle + monotonic
seqfor gap detection (client: ifseqjumps >1, soft-refetch snapshot endpoints).Client subscription policy
/timecurve: subscribeclock+buy+timer_params(+podium_last_buyif not folded intobuy).warbow_feedat 2–5 Hz; do not subscribe arena to 20 Hzclockunless arena shows ms timer (otherwise share oneclockconnection app-wide).chain_statusonly).Use one
EventSourceper tab with typed dispatch, not multiple MegaETH WSS.Ingestion → HTTP fan-out rates
clockBuylogbuychain-timerpolltimer_params1/s or on changebuyconfirmation: indexedPitfalls
buyevents while floodingclock.clockHz for podium — WarBow/streak need indexed rates, not mini-block rate.timer_paramson Buy — ms clock correct but deadline wrong after reset.tx_hash, not one global bucket.Suggested issue subtasks
docs/indexer/design.mdand OpenAPI/SSE specRealtimeSSEProviderwith typed reducers + per-stream render capssse_emit_dropped{stream},wss_staleness_ms{stream}Realtime tiers — per-stream Hz (implementation comment)
Principle: “Realtime” is not one global number. Each UX surface has different freshness needs, onchain drivers, and cost (WSS decode, PG, SSE fan-out, React). Use separate logical SSE streams (one
EventSource, multiplexed bytype+seq) with explicit server max emit Hz and client max render Hz.MegaETH can emit ~100 mini-blocks/s; browsers and Arena podiums must not subscribe to a single 100 Hz firehose.
Stream catalog
clockminiBlockshead timestampwss_realtime; deadline from RPCtimer_paramsended, timer capTimeCurveat block tagchain-timerbuy/endSale/ extensionbuyBuyvisible on executelogspending,Buytopicpodium_last_buyBuybuystream or/podiumsbuy; snapshot ≤2buy; reconcile ≤0.5podium_snapshotwarbow_feedchain_status/v1/status+ WSS healthminiblock_debugwss_realtimeWhy not one SSE at 100 Hz?
Multiplexed SSE envelope
seqper connection for gap detection → soft-refetch snapshot endpoints if gap detected.buyin the same bucket asclock).Client subscription policy
/timecurveclock+buy+timer_params(+podium_last_buyif not folded intobuy)warbow_feedat 2–5 Hz; share app-wideclockonly if Arena shows ms timerchain_statusonly)One
EventSourceper tab → typed dispatch. Never per-user MegaETH WSS from the browser.Ingestion → HTTP → browser rates
clock20/sBuylogtx_hash+log_index)buyimmediatechain-timertimer_params1/s or on changebuywithconfirmation: indexedRules of thumb
clock≠buy— never throttle buys behind the clock bucket.timer_paramson everybuy— ms clock without freshdeadlineis wrong after timer reset.chain-timer+ledgerSecIntForPhase; WSS is latency/display layer.tx_hash, not one global coalesce bucket.Pitfalls
buywhile floodingclock.timer_paramspush whenBuyarrives.Product context (this issue)
Real goals: ms hero timer + ms last-buy / live buys. Mini-block # in UI = pipeline test (
miniblock_debugstream only). Full miniBlocks in Postgres not required; uselogspending for buys + throttled clock for timer.Scope: Phase A =
buystream; B =clock+timer_params; C =miniblock_debug/chain_status.Implementation checklist
docs/indexer/design.md+ SSE specRealtimeSSEProvider— typed reducers + per-stream render capssse_emit_dropped{stream},wss_staleness_ms{stream}Outdated due to upgrade from timecurve to timearena. WSS and SSE should be investigated but above plan must be updated first
mentioned in commit
f50cc8e1cementioned in merge request !411
mentioned in commit
c930aa98dfmentioned in merge request !415
"## Verification report \u2014 agent:verify (2026-06-07)\n\nIssue: https://gitlab.com/PlasticDigits/yieldomega/-/work_items/237\n\nSummary: Phase 1 MegaETH
miniBlocksWSS + frontend realtime head pill is not implemented (deferred per issue comment 2026-05-30 \u2014 Arena v2 replan required). Deferred guardrails, docs, and RPC ingest path PASS. Issue acceptance criteria for shipped Phase 1 FAIL until implementation lands.\n\n---\n\n### Acceptance criteria (Phase 1)\n\n| Criterion | Result | Evidence |\n|-----------|--------|----------|\n| Indexer subscribes tominiBlockson MegaETH mainnet WSS when enabled; keepalive documented and implemented | FAIL | No WSS client inindexer/src\u2014rg 'miniBlocks\\|INDEXER_WSS\\|eth_subscribe' indexer/src\u2192 empty.INDEXER_WSS_URL/INDEXER_WSS_ENABLEDdocumented as not wired inindexer/README.md. |\n| RPC ingestion + reorg behavior unchanged | PASS |bash scripts/verify-issue-237-wss-deferred.sh(includescargo test\u2014 31 unit + 4 integration tests).cargo test --test integration_stage2\u2014 4 passed.cargo clippy --all-targets -- -D warnings\u2014 clean. |\n| Status/realtime API exposes WSS head withconfirmation: wss_realtimeseparate frommax_indexed_block/chain_pointer| FAIL | NoGET /v1/realtime/*routes.api.rshas nowss_realtime/mini_blockfields. |\n| Agent card shows two live readouts (indexed block + websockets mini-block head) | FAIL |IndexerStatusBar.tsxshows indexed block only \u2014 no WSS pill, nodata-testidhooks for WSS head. |\n| Docs:indexer/README.md,docs/indexer/design.md\u2014 dual-path diagram + mainnet-only WSS testing note | PASS (deferred docs) | Design \u00a7237 documents RPC vs WSS vs browser SSE table, mainnet-only (4326) note, Arena v2 replan pointer. InvariantsINV-INDEXER-237-*/INV-FRONTEND-237-WSS-HEAD-PILLindocs/testing/invariants-and-business-logic.md. Manual QA \u00a7237 indocs/testing/manual-qa-checklists.md. |\n\n---\n\n### Automated checks\n\n| Check | Command | Result |\n|-------|---------|--------|\n| Deferred guardrails |bash scripts/verify-issue-237-wss-deferred.sh| PASS |\n| Indexer clippy |cd indexer && cargo clippy --all-targets -- -D warnings| PASS |\n| Indexer integration |cargo test --test integration_stage2(native PG:5433) | PASS (4/4) |\n| Frontend |cd frontend && npm run typecheck && npm run lint && npm test| PASS (559 tests) |\n| Postgres toolchain |bash scripts/verify-cloud-postgres.sh| PASS |\n\n---\n\n### Manual QA (\u00a7237)\n\n| Step | Result |\n|------|--------|\n| Agent card \u2014 indexed pill | PASS |\n| Agent card \u2014 WSS pill | FAIL \u2014 not implemented |\n| Kill WSS upstream | FAIL \u2014 not implemented (blocked) |\n| Anvil local stack without WSS env | PASS |\n\nMainnet WSS smoke: SKIP \u2014 noINDEXER_WSS_URLsecret / feature not wired.\n\n---\n\n### Invariants (deferred state)\n\n-INV-INDEXER-237-WSS-NO-WRITE\u2014 PASS\n-INV-INDEXER-237-STATUS-SPLIT\u2014 PASS\n-INV-INDEXER-237-REALTIME-ROUTES\u2014 PASS\n-INV-FRONTEND-237-WSS-HEAD-PILL\u2014 PASS\n-INV-INDEXER-237-ANVIL-GRACEFUL\u2014 PASS\n\n---\n\n### Outcome\n\nIssue remains open. Phase 1 acceptance criteria are not met. Repo correctly documents deferral and keeps RPC ingest authoritative. No doc/invariant fixes required this pass.\n\nFollow-up foragent:implement: Replan WSS/SSE stream catalog against Arena v2 surfaces (issue comments 2026-05-22 / 2026-05-30), then implement Phase 1 per issue body \u2014 starting with ingestion WSS task +GET /v1/realtime/mini-block-head(or SSE) + second agent-card pill onIndexerStatusBar/ArenaSimpleAgentCard.\n"mentioned in commit
23d03125f3mentioned in commit
fd4fe72ab4mentioned in issue #306
mentioned in issue #309
mentioned in issue #331