Indexer: per-block Multicall3 batching for warbow_score + podium_live ingest RPC (#356) #530

2026-06-28T06:08:50Z

PlasticDigits commented

2026-06-28 06:08:50 +00:00

(Migrated from gitlab.com)

Summary

Implements GitLab #356: per-block Multicall3 aggregate3 batching for ingest-side warbow_score + arena_podium_live RPC reads.

New indexer/src/arena_ingest_rpc.rs: plans deduped sub-calls per block, executes one aggregate3 at BlockId::Number(block) (caller ingestion), applies DB writes in log order; sequential fallback on Multicall3 failure.
indexer/src/ingestion.rs: replaces per-log dual RPC paths with block batch orchestration (preserves INV-INDEXER-140 / INV-INDEXER-317).
indexer/src/multicall.rs: push_address_arg + subcalls() for battlePoints + fallback iteration.
Docs/invariants: INV-INDEXER-356-* in docs/testing/invariants-and-business-logic.md, docs/indexer/design.md, docs/indexer/rpc-load-benchmark.md.

RPC metrics choice: successful ingest batches record one eth_call under caller ingestion (not split warbow_score/podium_live). Sequential fallback attributes sub-calls to warbow_score / podium_live by selector.

Acceptance checklist

Item	Command / step	Result
AC-1: one Multicall3 per block with buys	`cargo test arena_ingest_rpc` + active-arena benchmark (`eth_call:ingestion` only)	PASS
AC-2: DB parity	`integration_stage2` + verify scripts	PASS
AC-3: HTTP unchanged	`bash scripts/verify-podium-live-anvil.sh`	PASS
AC-3: wallet stats	`bash scripts/verify-wallet-profile-anvil.sh`	PASS
AC-4: `allowFailure: false` / fail-closed	`multicall.rs` + `INV-INDEXER-356-FAIL-CLOSED`	PASS
AC-5: sequential fallback	`arena_ingest_rpc::execute_sequential_batch`	PASS (code path; no dedicated Anvil-without-Multicall3 harness)
AC-6: reorg rollback	`integration_stage2::postgres_stage2_persist_all_events_and_rollback_after`	PASS
AC-7: RPC reduction ≥80%	`BENCHMARK_SCENARIO_SEC=120 bash scripts/benchmark-indexer-rpc-anvil.sh`	PASS — active_arena cumulative `eth_call:warbow_score` + `eth_call:podium_live`: 297 → 0 vs `rpc-benchmark-20260615T021945Z`; batched as 33 `eth_call:ingestion` (`rpc-benchmark-20260628T055944Z`)
AC-8: invariants documented	`design.md` + `invariants-and-business-logic.md`	PASS
T-14: clippy + unit	`cd indexer && cargo clippy --all-targets -- -D warnings && cargo test`	PASS
RPC metrics smoke	`bash scripts/verify-indexer-rpc-metrics.sh`	PASS

Verification for third parties

export PATH="$HOME/.foundry/bin:$PATH"
source /usr/local/cargo/env 2>/dev/null || true
bash scripts/bootstrap-cloud-postgres-native.sh

cd indexer && cargo clippy --all-targets -- -D warnings && cargo test

bash scripts/verify-podium-live-anvil.sh
bash scripts/verify-wallet-profile-anvil.sh
bash scripts/verify-indexer-rpc-metrics.sh
BENCHMARK_SCENARIO_SEC=120 bash scripts/benchmark-indexer-rpc-anvil.sh

Compare active_arena by_method_caller in docs/indexer/benchmarks/rpc-benchmark-20260628T055944Z.json — expect no eth_call:warbow_score / eth_call:podium_live and nonzero eth_call:ingestion.

## Summary Implements GitLab #356: per-block Multicall3 `aggregate3` batching for ingest-side `warbow_score` + `arena_podium_live` RPC reads. - New [`indexer/src/arena_ingest_rpc.rs`](indexer/src/arena_ingest_rpc.rs): plans deduped sub-calls per block, executes one `aggregate3` at `BlockId::Number(block)` (caller `ingestion`), applies DB writes in log order; sequential fallback on Multicall3 failure. - [`indexer/src/ingestion.rs`](indexer/src/ingestion.rs): replaces per-log dual RPC paths with block batch orchestration (preserves `INV-INDEXER-140` / `INV-INDEXER-317`). - [`indexer/src/multicall.rs`](indexer/src/multicall.rs): `push_address_arg` + `subcalls()` for battlePoints + fallback iteration. - Docs/invariants: `INV-INDEXER-356-*` in [`docs/testing/invariants-and-business-logic.md`](docs/testing/invariants-and-business-logic.md), [`docs/indexer/design.md`](docs/indexer/design.md), [`docs/indexer/rpc-load-benchmark.md`](docs/indexer/rpc-load-benchmark.md). **RPC metrics choice:** successful ingest batches record **one** `eth_call` under caller **`ingestion`** (not split `warbow_score`/`podium_live`). Sequential fallback attributes sub-calls to `warbow_score` / `podium_live` by selector. ## Acceptance checklist | Item | Command / step | Result | |------|----------------|--------| | AC-1: one Multicall3 per block with buys | `cargo test arena_ingest_rpc` + active-arena benchmark (`eth_call:ingestion` only) | PASS | | AC-2: DB parity | `integration_stage2` + verify scripts | PASS | | AC-3: HTTP unchanged | `bash scripts/verify-podium-live-anvil.sh` | PASS | | AC-3: wallet stats | `bash scripts/verify-wallet-profile-anvil.sh` | PASS | | AC-4: `allowFailure: false` / fail-closed | `multicall.rs` + `INV-INDEXER-356-FAIL-CLOSED` | PASS | | AC-5: sequential fallback | `arena_ingest_rpc::execute_sequential_batch` | PASS (code path; no dedicated Anvil-without-Multicall3 harness) | | AC-6: reorg rollback | `integration_stage2::postgres_stage2_persist_all_events_and_rollback_after` | PASS | | AC-7: RPC reduction ≥80% | `BENCHMARK_SCENARIO_SEC=120 bash scripts/benchmark-indexer-rpc-anvil.sh` | PASS — active_arena cumulative `eth_call:warbow_score` + `eth_call:podium_live`: **297 → 0** vs `rpc-benchmark-20260615T021945Z`; batched as **33** `eth_call:ingestion` (`rpc-benchmark-20260628T055944Z`) | | AC-8: invariants documented | `design.md` + `invariants-and-business-logic.md` | PASS | | T-14: clippy + unit | `cd indexer && cargo clippy --all-targets -- -D warnings && cargo test` | PASS | | RPC metrics smoke | `bash scripts/verify-indexer-rpc-metrics.sh` | PASS | ## Verification for third parties ```bash export PATH="$HOME/.foundry/bin:$PATH" source /usr/local/cargo/env 2>/dev/null || true bash scripts/bootstrap-cloud-postgres-native.sh cd indexer && cargo clippy --all-targets -- -D warnings && cargo test bash scripts/verify-podium-live-anvil.sh bash scripts/verify-wallet-profile-anvil.sh bash scripts/verify-indexer-rpc-metrics.sh BENCHMARK_SCENARIO_SEC=120 bash scripts/benchmark-indexer-rpc-anvil.sh ``` Compare active_arena `by_method_caller` in `docs/indexer/benchmarks/rpc-benchmark-20260628T055944Z.json` — expect **no** `eth_call:warbow_score` / `eth_call:podium_live` and nonzero `eth_call:ingestion`.

PlasticDigits commented

2026-06-28 06:11:17 +00:00

(Migrated from gitlab.com)

Security review — MR !530

Commit reviewed: 41ca39eb1d8c2d439aa3a94e49ac663dec8cabe3
Scope: indexer/src/arena_ingest_rpc.rs (new), ingestion.rs batch orchestration, multicall.rs helpers, visibility tweaks in warbow_score.rs / arena_podium_live.rs, docs/benchmark artifacts.
Prior security threads: none on this MR (re-checked).

Method

Traced attacker-controlled inputs (on-chain arena logs → decoded events → Multicall3 sub-call planning → RPC results → SQL upserts) against sinks (RPC eth_call, aggregate3 decode, idx_warbow_epoch_score / idx_arena_podium_live writes). Compared semantics to pre-MR per-log snapshot_warbow_players_after_log / snapshot_live_podium_after_log paths.

Areas examined (no medium+ issues)

Injection / SSRF: RPC targets are config-bound arena + canonical MULTICALL3_ADDRESS; SQL remains parameterized via existing sqlx helpers.
Authn/authz: no new HTTP or admin surfaces.
Fail-closed / griefing: allowFailure: false + block-tx abort on RPC/decode failure preserved (INV-INDEXER-317, INV-INDEXER-356-FAIL-CLOSED); Multicall3 failure falls back to sequential calls with identical calldata/block tag.
State correctness: per-block BlockId::Number(next) matches prior per-log d.block_number semantics (end-of-block state); deduped sub-calls are safe because all reads share the same block tag; DB apply order (persist logs → batch RPC → per-log warbow then podium side-effects) preserves warbow-score visibility for rebuild_warbow_live_from_scores_conn.
EVM / oracle manipulation: indexer remains non-authoritative; no new trust assumptions beyond existing RPC provider model.
DoS: large batches fail the block transaction (no partial commit); not a new integrity bypass vs sequential path.

Outcome: FINDINGS: 0 medium+ — Security review: no medium+ findings on this diff.

No inline threads (nothing to anchor).

## Security review — MR !530 **Commit reviewed:** `41ca39eb1d8c2d439aa3a94e49ac663dec8cabe3` **Scope:** `indexer/src/arena_ingest_rpc.rs` (new), `ingestion.rs` batch orchestration, `multicall.rs` helpers, visibility tweaks in `warbow_score.rs` / `arena_podium_live.rs`, docs/benchmark artifacts. **Prior security threads:** none on this MR (re-checked). ### Method Traced attacker-controlled inputs (on-chain arena logs → decoded events → Multicall3 sub-call planning → RPC results → SQL upserts) against sinks (RPC `eth_call`, `aggregate3` decode, `idx_warbow_epoch_score` / `idx_arena_podium_live` writes). Compared semantics to pre-MR per-log `snapshot_warbow_players_after_log` / `snapshot_live_podium_after_log` paths. ### Areas examined (no medium+ issues) - **Injection / SSRF:** RPC targets are config-bound arena + canonical `MULTICALL3_ADDRESS`; SQL remains parameterized via existing `sqlx` helpers. - **Authn/authz:** no new HTTP or admin surfaces. - **Fail-closed / griefing:** `allowFailure: false` + block-tx abort on RPC/decode failure preserved (`INV-INDEXER-317`, `INV-INDEXER-356-FAIL-CLOSED`); Multicall3 failure falls back to sequential calls with identical calldata/block tag. - **State correctness:** per-block `BlockId::Number(next)` matches prior per-log `d.block_number` semantics (end-of-block state); deduped sub-calls are safe because all reads share the same block tag; DB apply order (persist logs → batch RPC → per-log warbow then podium side-effects) preserves warbow-score visibility for `rebuild_warbow_live_from_scores_conn`. - **EVM / oracle manipulation:** indexer remains non-authoritative; no new trust assumptions beyond existing RPC provider model. - **DoS:** large batches fail the block transaction (no partial commit); not a new integrity bypass vs sequential path. **Outcome: `FINDINGS: 0` medium+** — Security review: no medium+ findings on this diff. No inline threads (nothing to anchor).

PlasticDigits commented

2026-06-28 06:15:01 +00:00

(Migrated from gitlab.com)

mentioned in commit e1be645b69

mentioned in commit e1be645b69e7cdf54ea858e4ce785b60a209fc66

PlasticDigits (Migrated from gitlab.com) merged commit e1be645b69 into main

2026-06-28 06:15:01 +00:00

Sign in to join this conversation.

No reviewers

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

code/yieldomega!530

Rows
Columns