Indexer: parse per-maker limit_order_fill rows from merged wasm event streams (GitLab #254 follow-up) #269
Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:implement
agent:implement
agent:open_issues
agent:ready
agent:research
agent:security_audit
agent:verify
architecture
backend
blocker:hybrid
blocker:launch
blocker:limit-orders
blocker:v2
block:log_only
block:security
bug
ci
contracts
correctness
deploy
dev
devops
docs
documentation
duplicate
e2e
enhancement
epic
feature
frontend
functional-completion
gas
good first issue
governance
help wanted
high-risk
hooks
hybrid
indexer
infra
infrastructure
integrators
invalid
launch-blocker
limit-orders
localnet
localterra
low priority
missing-implementation
needs-design
ops
performance
priority
high
priority
medium
product
qa
QA
question
ready
ready
research
scripts
security
security-hardening
smartcontracts
tech-debt
testing
ux
UX
v2
verification
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
code/cl8y-dex-terraclassic#269
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Fix
parse_limit_order_fillsso hybrid swaps that fill multiple makers persist onelimit_order_fillsrow per maker when Terra LCD merges contract wasm emissions into grouped attribute streams.Current codebase
indexer/src/indexer/parser.rs—parse_limit_order_fills: iterateswasmevents and gates each event onwasm_attr_last(attrs, "action") == "limit_order_fill", then reads fill fields viawasm_attr_last(last duplicate key wins).process_block_txs: dispatches parsed fills toprocess_limit_order_fill→limit_order_fills::insert_fill(dedup viaUNIQUE (tx_hash, pair_id, order_id)andfill_exists).indexer/migrations/20260326000001_limit_order_fills.sql— per-maker rows linked to parentswap_eventsvia optionalswap_event_id.parse_swapscorrectly indexesswap_eventsbook/pool columns (book_return_amount,limit_book_offer_consumed, etc.) because the aggregateaction=swapattrs typically land last in their wasm group.actionoccurrence usingwasm_kv_map_after_action+wasm_contract_addr_before; batch paths use columnar zip for placements, cancellations, and claims in the same module.docs/indexer-invariants.mddescribe merged-stream behavior for lifecycle wasm; fill parsing was never aligned.Confirmed failure (LocalTerra QA during GitLab #254 verification):
swap_events.idlimit_order_filleventslimit_order_fillsrowsIngestion does not fail (
indexer_failed_blocks = 0); the fill parser silently returns an empty vec.Why this is needed
wasmevent with a flattened attribute stream. A hybrid swap emitting Nlimit_order_fillevents may collapse into ~few grouped events where the lastactionistransferorswap, notlimit_order_fill. The per-event gate never matches → zero rows inserted.limit_order_fill,wasm_attr_lastonorder_id/maker/ amounts would recover at most one fill per group, not all makers.GET /api/v1/traders/{addr}/limit-fills), maker analytics, order partial-fill visibility, and any tooling joininglimit_order_fillsto hybrid swaps. Aggregate volume reporting (swap_events/ L10) is unaffected but per-maker attribution is wrong or empty.Constraints and guardrails
parse_swaps/ aggregate hybrid column behavior.action=limit_order_fillindex; scope contract withwasm_contract_addr_before; segment attrs withwasm_kv_map_after_action.parse_limit_order_placements_columnardetection rules).wasm-wasmevents where lifecycle parsers already do (is_wasm_lifecycle_event_type) if LocalTerra emits fill attrs there.fill_existsandUNIQUE (tx_hash, pair_id, order_id); replays must not duplicate rows.swap_event_idwhen aswap_eventsrow exists for the same tx+pair (existing insert behavior).swap_events(L10).Relevant files
indexer/src/indexer/parser.rsindexer/src/db/queries/limit_order_fills.rsindexer/migrations/20260326000001_limit_order_fills.sqldocs/indexer-invariants.mdparser.rs(#141 lifecycle, columnar placements/cancellations/claims)indexer/tests/swap_events_hybrid_columns.rsindexer/src/api/traders.rs,indexer/tests/api_traders.rsskills/AGENTS_INDEXER_INGESTION_HARDENING.mdRecommended direction
parse_limit_order_fills_from_wasm_attrs(attrs)that loops allaction=limit_order_fillindices (same structure asparse_limit_order_expired_parked_from_wasm_attrs).order_id,side,maker,price,token0_amount,token1_amount,commission_amount; validateside ∈ {bid, ask}; skip malformed segments without failing the block.parse_limit_order_fillsto call the helper for eachwasm/wasm-wasmevent (stop gating onwasm_attr_last(action)).limit_order_fillactions followed bytransfer/swapin one attribute stream; assert N parsed fills and field correctness per segment.parser.rs#[cfg(test)].docs/indexer-invariants.mdindexing matrix with a Limit fill rows row cross-linking #141; mention indocs/limit-orders.mdif fill indexing is documented there.Acceptance criteria
limit_order_fillactions → K rows inlimit_order_fillsfor that tx+pair.actionisswaportransferstill parses all priorlimit_order_fillsegments.swap_eventsaggregate hybrid fields unchanged for the same txs.GET /api/v1/traders/{maker}/limit-fillsreturns indexed rows after hybrid multi-maker swap.Test plan (functional paths)
swap_event_idlinked when swap row existswasm-wasmevents (if emitted)wasmTest plan (attack / abuse / hack vectors)
limit_order_fillon non-pair wasmwasm_contract_addr_beforescopes contract; unknown pair → discover or skip without failing blockorder_idin same txUNIQUE (tx_hash, pair_id, order_id)+fill_existsmakerstring in attrsVerification criteria
cd indexer && cargo test limit_order_fill(and new merged-stream parser tests) green.cd indexer && cargo test --libgreen.cd indexer && cargo test --tests -j 1 -- --test-threads=1green (Postgres integration).swap_eventsid 324 →SELECT count(*) FROM limit_order_fills WHERE tx_hash = …equals on-chain fill count (20).indexer_failed_blocksremains 0 after ingest.docs/indexer-invariants.mdupdated.Related
mentioned in issue #254
mentioned in commit
d6701c4b00Implementation summary (merged to
main@d6701c4)Fixed
parse_limit_order_fillsso hybrid swaps that fill multiple makers persist onelimit_order_fillsrow per maker when Terra LCD merges contract wasm emissions into grouped attribute streams.Root cause
The old parser gated each wasm event on
wasm_attr_last(attrs, "action") == "limit_order_fill". In merged streams the last action is typicallyswaportransfer, so fill parsing returned an empty vec (confirmed during #254 QA: swap_events id 323/324 had 5/20 on-chain fills but 0 DB rows).Fix
parse_limit_order_fills_from_wasm_attrsfollowing the GitLab #141 pattern: scan everyaction=limit_order_fillindex, scope contract viawasm_contract_addr_before, segment fields viawasm_kv_map_after_action.wasm-wasmevents viais_wasm_lifecycle_event_type.fill_exists+UNIQUE (tx_hash, pair_id, order_id)).Docs / invariants
docs/indexer-invariants.md— new Limit fill rows indexing matrix row (#269, cross-links #141 / #254 / L10).docs/limit-orders.md— LCD merged-stream note extended tolimit_order_fill.skills/AGENTS_INDEXER_INGESTION_HARDENING.md— do-not-regress bullet.Tests run (all green)
cd indexer && cargo test limit_order_fillcd indexer && cargo test --lib(82 passed)cd indexer && cargo test --tests -j 1 -- --test-threads=1New unit fixtures: 5-fill and 20-fill merged streams ending with swap/transfer; wasm-wasm; malformed segment skip; pool-only regression.
Verification checklist
SELECT count(*) FROM limit_order_fills WHERE tx_hash = '<tx>'equals on-chainlimit_order_fillcount (expect 20 for id 324 fixture).swap_eventsaggregate hybrid columns unchanged for the same tx (pool_return_amount,book_return_amount,limit_book_offer_consumed).GET /api/v1/traders/{maker}/limit-fillsreturns rows for makers in that hybrid tx.indexer_failed_blocks = 0).Follow-ups
limit_order_fillsrows for merged-stream txs will remain missing until replayed.Requesting verification from the QA agent team when convenient.
mentioned in issue #264
Verified #269 on
d6701c4— and this one got a real-world live proof, not just fixtures.Unit (
cargo test --lib limit_order_fill— 6/6 green):parse_limit_order_fills_twenty_makers_merged_before_transfer(the exact #254 id-324 shape: 20 fills merged before a transfer),parse_limit_order_fills_merged_before_swap,..._skips_malformed_segment_keeps_valid(attack vector),..._pool_only_swap_has_zero_fills(K=0),..._from_wasm_wasm_event_type,..._extracts_events.cargo test --lib= 82 passed.LIVE merged-stream proof: my #262 100-maker bench fired a real hybrid swap filling 99 distinct makers in one merged wasm stream (tx
7B2F9A04F229D10D4407612BF3D2D0E118ACF1346AD6FF24B8596FEEB47F3BF8). Against the live indexer DB:SELECT count(*) FROM limit_order_fills WHERE tx_hash = '7B2F9A0…'= 99 (== on-chainlimit_order_fillcount).swap_event(swap_event_idNOT NULL = 99/99).indexer_failed_blocks = 0. The OLD parser would have inserted 0 rows here — this is the regression fixed.swap_eventsaggregate row for the same tx intact:book_return97226019 /pool_return21418348 /limit_book_offer_consumed97611919 — per-maker fills are separate attribution rows, no double-count (L10).GET /api/v1/traders/{maker}/limit-fillsreturns the maker's fill row (order_id 100, bid @1.0100, linked to swap_event 63).UNIQUE (tx_hash, pair_id, order_id)+fill_exists).Attack vectors covered: malformed-segment skip (test), duplicate/replay (UNIQUE + live no-dup), volume double-count (aggregate on
swap_events), contract scoping (wasm_contract_addr_before).docs/indexer-invariants.mdupdated.Note your own follow-up: environments that ingested hybrid txs before this fix need a one-time re-index for historical fill rows. Good to close from my side once !733 merges. @PlasticDigits