Persist missing TimeArena events and harden ingest side-effects (#317) #462

Merged

PlasticDigits merged 1 commit from issue-317-timearena-events-ingest-harden into main 2026-06-13 09:29:41 +00:00

Conversation 4 Commits 1 Files changed 9 +422 -22

PlasticDigits commented 2026-06-13 08:12:58 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

Summary

Closes INV-INDEXER-112 gaps from #309 / #317:

• Decode + persist + reorg rollback for six missing TimeArena events: FirstBuyCredScheduled, LevelUp, FeatureUnlocked, PausedSet, WarbowPodiumFinalized, and WarBowFlagClaimed (was decoded but no-op).
• New migration 20260613120000_idx_arena_timearena_events_gl317 with dedicated idx_* tables.
• Ingest side-effect policy: WarBow BP and live podium eth_call snapshot failures now abort the block transaction (no warn-and-continue) per INV-INDEXER-317-INGEST-SIDE-EFFECTS.
• Extended postgres_stage2_persist_all_events_and_rollback_after to cover all new variants plus ArenaLastBuyEpochCharmAnchored.
• Documented event map and invariants in docs/testing/invariants-and-business-logic.md §317 and docs/indexer/design.md § ingest side-effects.

Acceptance checklist

Criterion	Verification	Result
All six events have idx_* rows after ingest	cd indexer && YIELDOMEGA_PG_TEST_URL=postgres://yieldomega:password@127.0.0.1:5433/yieldomega_indexer_test cargo test --test integration_stage2 postgres_stage2_persist_all_events	PASS
Reorg rollback clears new tables	Same test (rollback_after assertions for level/feature rows)	PASS
Side-effect failure no longer leaves silent stale state	Code review: ingestion.rs propagates snapshot errors; block tx rolls back	PASS
Stage 2 integration test covers new variants	integration_stage2.rs enumerates all six + charm anchor	PASS
INV-INDEXER-112 checklist in invariants doc	docs/testing/invariants-and-business-logic.md#indexer-timearena-events-gitlab-317	PASS
cargo clippy + cargo test	cd indexer && cargo clippy --all-targets -- -D warnings && cargo test	PASS
verify-podium-live-anvil.sh	bash scripts/verify-podium-live-anvil.sh (pre-build release indexer first)	PASS (podium/live ingest path; script reaches final integration gate)
Postgres integration (full suite)	YIELDOMEGA_PG_TEST_URL=... cargo test --test integration_stage2	SKIP — arena_wallet_stats_two_epochs_and_bonus_fields fails on main too (warbow_battle_points 1000 vs 750; pre-existing, unrelated to #317)

Third-party verification

bash scripts/bootstrap-cloud-postgres-native.sh
export YIELDOMEGA_PG_TEST_URL=postgres://yieldomega:password@127.0.0.1:5433/yieldomega_indexer_test
cd indexer && cargo clippy --all-targets -- -D warnings && cargo test
cd indexer && cargo test --test integration_stage2 postgres_stage2_persist_all_events_and_rollback_after
cd indexer && cargo build --release
bash scripts/verify-podium-live-anvil.sh

After ingest on Anvil, confirm rows exist:

SELECT COUNT(*) FROM idx_arena_first_buy_cred_scheduled;
SELECT COUNT(*) FROM idx_arena_level_up;
SELECT COUNT(*) FROM idx_arena_feature_unlocked;
SELECT COUNT(*) FROM idx_arena_paused_set;
SELECT COUNT(*) FROM idx_arena_warbow_podium_finalized;
SELECT COUNT(*) FROM idx_arena_warbow_flag_claimed;

Blockers

• None for #317 scope. Full integration_stage2 green requires fixing pre-existing arena_wallet_stats_two_epochs_and_bonus_fields on main.

Follow-ups

• Optional: expose PausedSet history via GET /v1/arena/* for AUDIT timeline (issue recommended direction).
• Optional: retry queue alternative to fail-fast side-effects if operators prefer eventual consistency over stall-on-RPC.

---

Note

Medium Risk
Schema migration plus a behavioral change to block ingestion (stall/retry on RPC snapshot failure) affects derived podium/WarBow read models; core decode/persist paths are well covered by integration tests but operators may see more ingest retries under RPC griefing.

Overview
Closes INV-INDEXER-112 gaps for six TimeArena events that were missing or no-op in Postgres: FirstBuyCredScheduled, LevelUp, FeatureUnlocked, PausedSet, WarbowPodiumFinalized, and WarBowFlagClaimed (previously decoded but not inserted). A new migration adds dedicated idx_* tables; decoder.rs, persist.rs, and reorg.rs wire decode, insert, and rollback for each.

Ingest side-effect policy changes in ingestion.rs: post-log eth_call snapshots for idx_warbow_epoch_score and idx_arena_podium_live now propagate errors and abort the per-block transaction instead of logging a warning and continuing—so derived tables cannot commit ahead of failed RPC work (INV-INDEXER-317-INGEST-SIDE-EFFECTS).

Docs and integration_stage2 extend the event map, invariants §317, and stage-2 persist/rollback coverage (including ArenaLastBuyEpochCharmAnchored in the integration fixture).

^{Reviewed by Cursor Bugbot for commit ac5f4330ed. Bugbot is set up for automated code reviews on this repo. Configure here.}

## Summary Closes **INV-INDEXER-112** gaps from [#309](https://gitlab.com/PlasticDigits/yieldomega/-/issues/309) / [#317](https://gitlab.com/PlasticDigits/yieldomega/-/issues/317): - Decode + persist + reorg rollback for six missing **`TimeArena`** events: `FirstBuyCredScheduled`, `LevelUp`, `FeatureUnlocked`, `PausedSet`, `WarbowPodiumFinalized`, and `WarBowFlagClaimed` (was decoded but no-op). - New migration `20260613120000_idx_arena_timearena_events_gl317` with dedicated `idx_*` tables. - **Ingest side-effect policy:** WarBow BP and live podium `eth_call` snapshot failures now **abort the block transaction** (no warn-and-continue) per `INV-INDEXER-317-INGEST-SIDE-EFFECTS`. - Extended `postgres_stage2_persist_all_events_and_rollback_after` to cover all new variants plus `ArenaLastBuyEpochCharmAnchored`. - Documented event map and invariants in `docs/testing/invariants-and-business-logic.md` §317 and `docs/indexer/design.md` § ingest side-effects. ## Acceptance checklist | Criterion | Verification | Result | |-----------|--------------|--------| | All six events have `idx_*` rows after ingest | `cd indexer && YIELDOMEGA_PG_TEST_URL=postgres://yieldomega:password@127.0.0.1:5433/yieldomega_indexer_test cargo test --test integration_stage2 postgres_stage2_persist_all_events` | **PASS** | | Reorg rollback clears new tables | Same test (`rollback_after` assertions for level/feature rows) | **PASS** | | Side-effect failure no longer leaves silent stale state | Code review: `ingestion.rs` propagates snapshot errors; block tx rolls back | **PASS** | | Stage 2 integration test covers new variants | `integration_stage2.rs` enumerates all six + charm anchor | **PASS** | | `INV-INDEXER-112` checklist in invariants doc | `docs/testing/invariants-and-business-logic.md#indexer-timearena-events-gitlab-317` | **PASS** | | `cargo clippy` + `cargo test` | `cd indexer && cargo clippy --all-targets -- -D warnings && cargo test` | **PASS** | | `verify-podium-live-anvil.sh` | `bash scripts/verify-podium-live-anvil.sh` (pre-build release indexer first) | **PASS** (podium/live ingest path; script reaches final integration gate) | | Postgres integration (full suite) | `YIELDOMEGA_PG_TEST_URL=... cargo test --test integration_stage2` | **SKIP** — `arena_wallet_stats_two_epochs_and_bonus_fields` fails on **main** too (`warbow_battle_points` 1000 vs 750; pre-existing, unrelated to #317) | ## Third-party verification ```bash bash scripts/bootstrap-cloud-postgres-native.sh export YIELDOMEGA_PG_TEST_URL=postgres://yieldomega:password@127.0.0.1:5433/yieldomega_indexer_test cd indexer && cargo clippy --all-targets -- -D warnings && cargo test cd indexer && cargo test --test integration_stage2 postgres_stage2_persist_all_events_and_rollback_after cd indexer && cargo build --release bash scripts/verify-podium-live-anvil.sh ``` After ingest on Anvil, confirm rows exist: ```sql SELECT COUNT(*) FROM idx_arena_first_buy_cred_scheduled; SELECT COUNT(*) FROM idx_arena_level_up; SELECT COUNT(*) FROM idx_arena_feature_unlocked; SELECT COUNT(*) FROM idx_arena_paused_set; SELECT COUNT(*) FROM idx_arena_warbow_podium_finalized; SELECT COUNT(*) FROM idx_arena_warbow_flag_claimed; ``` ## Blockers - None for #317 scope. Full `integration_stage2` green requires fixing pre-existing `arena_wallet_stats_two_epochs_and_bonus_fields` on main. ## Follow-ups - Optional: expose `PausedSet` history via `GET /v1/arena/*` for AUDIT timeline (issue recommended direction). - Optional: retry queue alternative to fail-fast side-effects if operators prefer eventual consistency over stall-on-RPC. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Schema migration plus a behavioral change to block ingestion (stall/retry on RPC snapshot failure) affects derived podium/WarBow read models; core decode/persist paths are well covered by integration tests but operators may see more ingest retries under RPC griefing. > > **Overview** > Closes **INV-INDEXER-112** gaps for six **TimeArena** events that were missing or no-op in Postgres: **`FirstBuyCredScheduled`**, **`LevelUp`**, **`FeatureUnlocked`**, **`PausedSet`**, **`WarbowPodiumFinalized`**, and **`WarBowFlagClaimed`** (previously decoded but not inserted). A new migration adds dedicated **`idx_*`** tables; **`decoder.rs`**, **`persist.rs`**, and **`reorg.rs`** wire decode, insert, and rollback for each. > > **Ingest side-effect policy** changes in **`ingestion.rs`**: post-log **`eth_call`** snapshots for **`idx_warbow_epoch_score`** and **`idx_arena_podium_live`** now **propagate errors** and **abort the per-block transaction** instead of logging a warning and continuing—so derived tables cannot commit ahead of failed RPC work (**`INV-INDEXER-317-INGEST-SIDE-EFFECTS`**). > > Docs and **`integration_stage2`** extend the event map, invariants §317, and stage-2 persist/rollback coverage (including **`ArenaLastBuyEpochCharmAnchored`** in the integration fixture). > > ^{Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit ac5f4330edd21a0160e8092531668b6bfb013f49. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).} <!-- /CURSOR_SUMMARY -->

project_80561744_bot_5118c68a88bff95eacb6ab1c47c19cae commented 2026-06-13 08:13:08 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

changed the description

changed the description

project_80561744_bot_5118c68a88bff95eacb6ab1c47c19cae commented 2026-06-13 08:13:17 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

changed the description

changed the description

PlasticDigits commented 2026-06-13 08:15:10 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

Security review — MR !462

Commit reviewed: ac5f4330edd21a0160e8092531668b6bfb013f49

Scope: Indexer ingest hardening for GitLab #317 — six missing TimeArena event decode/persist/rollback paths, new idx_* migration tables, and fail-fast propagation of post-log eth_call snapshot errors (idx_warbow_epoch_score, idx_arena_podium_live) inside the per-block Postgres transaction.

Method: Traced attacker-controlled inputs (onchain logs via eth_getLogs, RPC eth_call responses) through decoder.rs → persist.rs / ingestion.rs → Postgres sinks; checked SQL parameterization, reorg rollback coverage, side-effect failure handling, and API exposure of new tables.

Outcome: FINDINGS: 0 medium+

Summary

No medium, high, or critical vulnerabilities identified on this diff.

• SQL injection: New inserts in persist.rs use sqlx parameterized binds ($1…$N); table names in reorg.rs rollback come from a compile-time ARENA_INDEX_TABLES allowlist — no attacker-controlled SQL fragments.
• Log injection / forged events: Decoded logs are filtered to registry contract addresses and validated via alloy topic/signature decoding against TimeArena.sol event definitions; no new trust boundary or auth bypass.
• Ingest side-effect policy (ingestion.rs): Propagating snapshot RPC failures aborts the block transaction instead of warn-and-continue. This hardens integrity (prevents committed derived rows ahead of failed RPC work per INV-INDEXER-317-INGEST-SIDE-EFFECTS). Availability impact from RPC griefing is operational/infrastructure trust, not a new onchain exploit path — battlePoints / podium view calls used by snapshots do not expose attacker-triggered reverts under normal contract state.
• New tables: Persist-only audit/index rows; not yet wired into HTTP handlers. No XSS, SSRF, path traversal, or deserialization surface introduced.
• Smart contracts: Unchanged in this MR.

Inline threads: None (no findings to anchor).

Security review: no medium+ findings on this diff.

## Security review — MR !462 **Commit reviewed:** `ac5f4330edd21a0160e8092531668b6bfb013f49` **Scope:** Indexer ingest hardening for GitLab #317 — six missing `TimeArena` event decode/persist/rollback paths, new `idx_*` migration tables, and fail-fast propagation of post-log `eth_call` snapshot errors (`idx_warbow_epoch_score`, `idx_arena_podium_live`) inside the per-block Postgres transaction. **Method:** Traced attacker-controlled inputs (onchain logs via `eth_getLogs`, RPC `eth_call` responses) through `decoder.rs` → `persist.rs` / `ingestion.rs` → Postgres sinks; checked SQL parameterization, reorg rollback coverage, side-effect failure handling, and API exposure of new tables. **Outcome:** `FINDINGS: 0` medium+ ### Summary No medium, high, or critical vulnerabilities identified on this diff. - **SQL injection:** New inserts in `persist.rs` use sqlx parameterized binds (`$1`…`$N`); table names in `reorg.rs` rollback come from a compile-time `ARENA_INDEX_TABLES` allowlist — no attacker-controlled SQL fragments. - **Log injection / forged events:** Decoded logs are filtered to registry contract addresses and validated via alloy topic/signature decoding against `TimeArena.sol` event definitions; no new trust boundary or auth bypass. - **Ingest side-effect policy (`ingestion.rs`):** Propagating snapshot RPC failures aborts the block transaction instead of warn-and-continue. This **hardens integrity** (prevents committed derived rows ahead of failed RPC work per `INV-INDEXER-317-INGEST-SIDE-EFFECTS`). Availability impact from RPC griefing is operational/infrastructure trust, not a new onchain exploit path — `battlePoints` / `podium` view calls used by snapshots do not expose attacker-triggered reverts under normal contract state. - **New tables:** Persist-only audit/index rows; not yet wired into HTTP handlers. No XSS, SSRF, path traversal, or deserialization surface introduced. - **Smart contracts:** Unchanged in this MR. **Inline threads:** None (no findings to anchor). Security review: no medium+ findings on this diff.

PlasticDigits (Migrated from gitlab.com) merged commit a4660ac669 into main 2026-06-13 09:29:41 +00:00

PlasticDigits commented 2026-06-13 09:29:42 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

mentioned in commit a4660ac669

mentioned in commit a4660ac66995213ef94ff9236096b5a290b91217

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

agent:fix_bugfix

  

agent:fix_conflicts

  

agent:fix_security

  

agent:gap_analysis

  

agent:implement

  

agent:open_issues

  

agents

  

agent:security_audit

  

agent:verify

  

area

timecurve

  

arena

  

arena-v2

  

audit

  

block:feature_review

  

block:security

  

bug

  

contracts

  

design

  

devops

  

docs

Documentation

  

documentation

  

enhancement

  

epic

Epic / umbrella issue

  

feature

  

frontend

  

indexer

  

low priority

Low priority, investigate other issues first

  

megaeth

  

operations

  

ops

Deploy and operations

  

performance

  

QA

  

qa checklist

  

referrals

  

removal

Legacy removal

  

security

  

smart-contracts

  

tech-debt

  

testing

  

time-arena

TimeArena v2 revamp

  

timecurve

  

visual-qa

No labels

agent:fix_bugfix

agent:fix_conflicts

agent:fix_security

agent:gap_analysis

agent:implement

agent:open_issues

agents

agent:security_audit

agent:verify

area

timecurve

arena

arena-v2

audit

block:feature_review

block:security

bug

contracts

design

devops

docs

documentation

enhancement

epic

feature

frontend

indexer

low priority

megaeth

operations

ops

performance

QA

qa checklist

referrals

removal

security

smart-contracts

tech-debt

testing

time-arena

timecurve

visual-qa

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

code/yieldomega!462

No description provided.