Indexer: persist LastBuyEpochStarted and assign exact last_buy_epoch on arena buys #278
Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:open_issues
agents
agent:security_audit
agent:verify
area
timecurve
arena
arena-v2
audit
block:feature_review
block:security
bug
contracts
design
devops
docs
documentation
enhancement
epic
feature
frontend
indexer
low priority
megaeth
operations
ops
performance
QA
qa checklist
referrals
removal
security
smart-contracts
tech-debt
testing
time-arena
timecurve
visual-qa
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
code/yieldomega#278
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context
Parent: #238 (Arena v2 epic). Related: #246 (onchain
LastBuyEpochStarted+lastBuyEpoch), #255 (wallet stats HTTP), #273 (live podiums ingest onLastBuyEpochStarted).Current codebase
Onchain (source of truth)
TimeArena.lastBuyEpochis a global counter (not per-wallet). It increments only on Last Buy hard reset (remaining < threshold → snap to 900s), then emitsLastBuyEpochStarted(uint256 indexed epoch, uint256 deadline)before the same tx’sBuyevent.claimCred(epoch)all key off this global epoch at buy/claim time — seedocs/product/arena-v2.md,docs/product/time-arena.md.test_timer_hard_reset_increments_epoch,test_emits_LastBuyEpochStarted_on_hard_reset—INV-TIME-ARENA-EPOCH-EVENT(#246).Indexer today
LastBuyEpochStarted→DecodedEvent::ArenaLastBuyEpochStarted { epoch, deadline }inindexer/src/decoder.rs.DecodedEvent::ArenaLastBuyEpochStarted { .. } => {}inindexer/src/persist.rs(line ~327).arena_podium_live.rsalready reacts toArenaLastBuyEpochStartedfor cat-0 RPC snapshots (ingest path).idx_arena_buytimer_hard_resetbut nolast_buy_epochcolumn (migrations/20240601000000_arena_v2.up.sql).arena_wallet_stats.rsinfers epoch per buy via SQL window:SUM(CASE WHEN timer_hard_reset THEN 1 ELSE 0 END) OVER (ORDER BY block_number, log_index …)onidx_arena_buyrows only (per-wallet queries filterbuyer = $1).ARENA_INDEX_TABLESand covered byrollback_after.INV-INDEXER-112: every deployedTimeArenaevent must map to decode + persist + rollback —LastBuyEpochStartedis currently a gap.Why inference is wrong
lastBuyEpochis global; per-wallet cumulativetimer_hard_resetcounts only that wallet’s hard resets → mis-tags CHARM/CRED epoch for buyers who never reset but participate in later global epochs (#255 follow-up noted in QA).LastBuyEpochStartedis emitted beforeBuy— inference fromtimer_hard_reseton the buy row alone cannot assign the post-reset epoch without reading the event stream order.epochs_participatedandhighest_scores[].epoch,podium_win_rate, WarBow peak epoch in wallet stats all depend on correct per-buy epoch assignment.Why this implementation is needed
GET /v1/arena/wallet/{address}/stats, #255) and any future epoch-scoped indexer APIs.eth_callper buy at read time.INV-INDEXER-112gap and align docs/invariants with actual persist behavior.Constraints and guardrails
AGENTS.md, guardrails).persist_decoded_log_conninside block ingest.(block_number, tx_hash, log_index); include table inrollback_after/ARENA_INDEX_TABLES.ON CONFLICT DO NOTHINGon natural keys (match existingidx_arena_*patterns)./v1/timecurve/*(#266).x-schema-versiononly if HTTP response shapes change; prefer internal-only migration if wallet stats JSON fields stay stable but values become correct.Relevant files
contracts/src/arena/TimeArena.solLastBuyEpochStarted,lastBuyEpoch, hard-reset branchcontracts/test/TimeArena.t.solindexer/src/decoder.rsArenaLastBuyEpochStartedvariantindexer/src/persist.rsindexer/src/ingestion.rsindexer/src/reorg.rsindexer/src/arena_wallet_stats.rsindexer/src/arena_podium_live.rsindexer/src/warbow_score.rslast_buy_epoch_at_block— compare with persisted headindexer/tests/integration_stage2.rsindexer/migrations/idx_arena_buy.last_buy_epoch(recommended)docs/indexer/design.mddocs/testing/invariants-and-business-logic.mdINV-INDEXER-*row.cursor/skills/yieldomega-guardrails/SKILL.mdRecommended direction
1. Migration
idx_arena_last_buy_epoch_started:(block_number, block_timestamp, tx_hash, log_index, epoch, deadline)— PK(tx_hash, log_index); index(epoch),(block_number).idx_arena_buy.last_buy_epoch BIGINT NOT NULL DEFAULT 0(backfill strategy below).2. Ingest (ordered log processing)
Maintain a running
head_last_buy_epoch(starts at0) while processing logs in block order (same order aspersist_decoded_log_conntoday):ArenaLastBuyEpochStarted { epoch, .. }: insert row; sethead_last_buy_epoch = epoch.ArenaBuy: persist buy withlast_buy_epoch = head_last_buy_epoch(the onchain epoch after any earlierLastBuyEpochStartedlogs in the same block/tx).Do not use per-wallet
timer_hard_resetsums for epoch assignment.3. Backfill (deployed indexers)
idx_arena_buy+ decoded logs fromindexed_blocksor document “wipe and reindex” for dev only.deployBlock(document in ops note).4. Wallet stats / consumers
fetch_wallet_buys/fetch_all_buys_ordered/epochs_participatedquery withlast_buy_epochcolumn.epochs_participated=COUNT(DISTINCT last_buy_epoch)for wallet buys (still “epochs with ≥1 buy”).5. Verification script (optional but recommended)
bash scripts/verify-last-buy-epoch-anvil.sh: hard-reset buy → assert DB epoch rows matchcast call TimeArena.lastBuyEpoch()and wallet stats epochs.Acceptance criteria
LastBuyEpochStartedpersisted toidx_arena_last_buy_epoch_startedwith correctepochanddeadline(wei/sec as numeric).idx_arena_buyrow haslast_buy_epochequal to onchainTimeArena.lastBuyEpoch()at that buy’s log position (verified on Anvil fixture: pre-reset buys → epoch N; first buy after reset in same tx → epoch N+1).epochs_participatedfor wallet W matches distinctlast_buy_epochon W’s buys, not distinct per-wallet hard-reset count.highest_scores[].epochand CHARM-by-epoch peaks (#255).persist_decoded_log_connno longer no-opsArenaLastBuyEpochStarted;INV-INDEXER-112table list updated.INV-INDEXER-*-LAST-BUY-EPOCHin invariants + indexer design §; guardrails cross-link.Test plan (all paths)
Unit / module
LastBuyEpochStarted+ followingBuyin one synthetic block → buy row epoch matches event.Integration (
YIELDOMEGA_PG_TEST_URL)postgres_stage2_persist_all_events_and_rollback_after: assert row inidx_arena_last_buy_epoch_startedwhen fixture includesArenaLastBuyEpochStarted.arena_wallet_stats_two_epochs_and_bonus_fields: add case — wallet B buys in epoch 1 without ever hard-resetting;epochs_participatedandhighest_scoresepochs match global epoch (regression for inference bug).rollback_after(K), rows gone; re-persist restores.Anvil smoke (manual / script)
start-local-anvil-stack.sh→ hard-reset buy →psql/ HTTP wallet stats vscast call lastBuyEpoch().last_buy_epochon B’s row only for B.Regression
cargo test(indexer),arena_podium_liveingest still passes (#273).TimeArenatests unless contract event shape changes (not expected).Test plan — attack, hack, and abuse vectors
rollback_afterintegration; HTTP reads only above finalized pointer if applicableTimeArenaproxy (#259)ON CONFLICT DO NOTHING; idempotency testlast_buy_epochon buyLastBuyEpochStartedimmediately beforeBuyin same tx; assert order-sensitive assignmenteth_callto untrusted RPC(buyer, last_buy_epoch); drop globalSUM() OVERon full buy table where possibleNUMERIC(78,0)like other u256 fields; HTTP serializes as decimal stringVerification criteria (QA / Agent)
cd indexer && YIELDOMEGA_PG_TEST_URL=… cargo test— all integration tests green including new epoch fixtures.curl -s http://127.0.0.1:3100/v1/arena/wallet/0x…/stats | jq '.epochs_participated,.highest_scores'matches scenario script (two global epochs, non-resetting participant).psql:SELECT last_buy_epoch, timer_hard_reset FROM idx_arena_buy ORDER BY block_number, log_indexmatches event replay.INV-INDEXER-*-LAST-BUY-EPOCHwith file pointers.bash scripts/verify-last-buy-epoch-anvil.shexits 0.Leave issue open until QA signs off on Anvil + integration matrix above.
mentioned in commit d640b713fca7892f9bec98506284efec4616ba08
mentioned in commit 649b35b3a8b99bc6be83df49b16572f1484f3583
mentioned in commit
f9ba6052a6mentioned in commit
0899de7d27mentioned in merge request !407
mentioned in commit
ae9653fc5bImplementation complete (MR !407 merged)
Indexer now persists global Last Buy epoch boundaries and assigns the correct
last_buy_epochon every arena buy — replacing the incorrect per-wallettimer_hard_resetwindow inference.What changed
20260602120000_idx_arena_last_buy_epoch_gl278: newidx_arena_last_buy_epoch_startedtable +idx_arena_buy.last_buy_epochcolumn (index on(buyer, last_buy_epoch)).last_buy_epoch_head.rs: running global head loaded at block start; updated onLastBuyEpochStarted, applied to eachBuyin log order.persist.rs:LastBuyEpochStartedno longer a no-op; closesINV-INDEXER-112gap.arena_wallet_stats.rs:epochs_participatedand buy queries use storedlast_buy_epoch.reorg.rs: epoch table inARENA_INDEX_TABLES.postgres_stage2_persist_all_events_and_rollback_after; newlast_buy_epoch_global_assignment_non_resetting_participantregression.bash scripts/verify-last-buy-epoch-anvil.shINV-INDEXER-278-LAST-BUY-EPOCHin invariants, indexer design §278, guardrails + play skills cross-links.Agent verification (PASS)
cd indexer && cargo clippy --all-targets -- -D warningsYIELDOMEGA_PG_TEST_URL=postgres://yieldomega:password@127.0.0.1:5433/yieldomega_indexer_test cargo testbash scripts/verify-last-buy-epoch-anvil.shQA verification checklist
YIELDOMEGA_PG_TEST_URL=… cargo testinindexer/— all integration tests green.bash scripts/verify-last-buy-epoch-anvil.shexits 0.psqlshows row inidx_arena_last_buy_epoch_startedwithepochmatchingcast call TimeArena.lastBuyEpoch().idx_arena_buy.last_buy_epoch= global epoch (not 0).curl /v1/arena/wallet/{B}/stats→epochs_participatedmatches distinct stored epochs.Follow-ups
last_buy_epoch(documented in migration SQL).Requesting verification from the QA agent team. Issue left open pending Anvil + integration sign-off.
mentioned in merge request !406
Verification complete (Cloud Agent)
Re-verified GitLab #278 after MR !407 merge on
main(6d520e7). No repository changes required.What was verified
cargo clippy --all-targets -- -D warningscd indexer && cargo clippy …postgres_stage2_persist_all_events_and_rollback_after(epoch table + buylast_buy_epoch)YIELDOMEGA_PG_TEST_URL=postgres://yieldomega:password@127.0.0.1:5433/yieldomega_indexer_test cargo test— 35 tests totallast_buy_epoch_global_assignment_non_resetting_participant(wallet B in epoch 1 without hard-reset)arena_wallet_stats_two_epochs_and_bonus_fieldsbash scripts/verify-last-buy-epoch-anvil.shidx_arena_last_buy_epoch_startedepoch 1 after hard resetCOUNT(*) >= 1for epoch 1cast call TimeArena.lastBuyEpoch()= 1last_buy_epoch= 1psqlin scriptepochs_participated= 1curl /v1/arena/wallet/{bob}/statslast_buy_epoch = 0ArenaLastBuyEpochStartednot a no-opARENA_INDEX_TABLES; rollback integrationintegration_stage2::postgres_stage2_persist_all_events_and_rollback_afterINV-INDEXER-278-LAST-BUY-EPOCH, invariants §278, indexer design §278, guardrails §6gAcceptance criteria mapping (issue body)
LastBuyEpochStarted→idx_arena_last_buy_epoch_startedidx_arena_buy.last_buy_epochmatches onchain epoch at log position (same-tx reset → N+1 for following buy)epochs_participated= distinctlast_buy_epoch(non-resetting participant regression)highest_scores[].epoch/ global epoch correctness covered by integration + AnvilINV-INDEXER-112gap closed forLastBuyEpochStartedFollow-ups
20260602120000_idx_arena_last_buy_epoch_gl278and reindex from deploy block for historicallast_buy_epoch(noted in migration SQL).skills/README.mdplay-time-arena-doub row has a broken markdown link fragment for GitLab #258 (functional cross-links in guardrails/invariants are correct).Closing #278 — implementation and verification matrix are green on
main.mentioned in issue #285
mentioned in commit
05dac2661ementioned in merge request !414
mentioned in commit
0d408509b1mentioned in merge request !416
mentioned in commit
9baac170e5mentioned in commit
9f376dced2mentioned in commit
315455e43cmentioned in commit
9ce7a78686mentioned in issue #305
marked as related to #305
mentioned in issue #342
mentioned in issue #346
mentioned in merge request !521
mentioned in issue #364