fix(ledger): make transfer accounting exactly once across registration and concurrent writers #37
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Scope and source
Priority: P0. Gap coverage: GAP-002, GAP-003, GAP-013, GAP-040. Derived from GAP_1791945837.md, reviewed code commit
f8dbe93df953a78b17cb3304b717dcf84c4cd517. Currente476a68da09645c2308cbf47b90211fcaffcb7afadds only the report, so implementation findings remain applicable. Report evidence is local reproduction/source inspection unless explicitly proposed; no production exploit or current live-readiness claim.Coverage and sequencing: gap backlog tracker. Workstream 02. Dependencies: 01.
Current codebase
Both transfer persistence paths ignore rows affected by ON CONFLICT DO NOTHING and apply deltas again; the audit reproduced 100→90→80 on replay. Active registration checks omit the snapshot boundary, so block 109 transfers can be counted again in a live block-110 baseline. Height and balance are obtained separately. Running checkpoints use read-then-upsert without a production writer lock; negative amounts are silently clamped.
Why this implementation is needed
Retries, replicas and catch-up can corrupt frozen power even when transfer rows are unique. A coherent registration boundary and atomic event/cursor commit are prerequisites for trustworthy voting.
Relevant files
ledger/src/db.rsledger/src/register.rsledger/src/bsc.rsledger/src/ingest.rsledger/src/parser/mod.rsledger/src/amount.rsledger/src/test_lock.rsledger/migrations/20260825000001_ledger.sqlledger/tests/ledger_integration.rsdocs/LEDGER_INVARIANTS.mdRecommended solution
Apply a delta only when a stable source event is newly inserted, atomically with corresponding balance checkpoints and cursor/hash progress. Terra identities must use original execution indices, not registration-filtered positions. Obtain live balance and its actual observed block coherently on one provider without archive dependency; reject/retry ambiguous observations. Serialize registration insertion with ingest and apply only events strictly after the end-of-block baseline. Enforce a production single-writer lease/advisory lock or explicit row-locked serializable processing with bounded retries. Unexpected underflow is a durable corruption/degraded signal, not a zero repair.
Constraints and guardrails
Preserve standalone offchain advisory voting, no new smart contracts or archive-node dependency, BSC as a core electorate, raw 18-decimal integer balances and separate Terra/BSC address identities. Keep Legal fail-closed transactional UI and the proven DEX Terra/Bridge EVM connectors; no browser BSC balance RPC, seed/key prompt or embedded admin/model credential. API must retain a restricted role separate from the ledger writer. Freeze at actual opening, never at draft creation or a client-supplied height. Agents own design, implementation, review and evidence under repository guidance; do not shift approvals/tests to the human or invent PASS evidence.
Acceptance criteria
Duplicate deliveries, overlap and crash/restart leave balances unchanged after first application. Before/at-boundary events never change the initial snapshot; after-boundary events apply once. Concurrent registration/ingest and concurrent writers preserve order and raw 18-decimal integers. No partially committed event/cursor state or silent negative-balance repair.
Test plan: supported paths, failure paths and concurrency
Replay debit/credit/self-transfer/mint/burn on both chains; multiple transfers at one height; registered/unregistered legs and later registration-set changes. Test moving chain tip and endpoint failover between snapshot calls, before/at/after registration, simultaneous register/ingest, overlapping ranges and two writers. Inject failures after insert, balance update and before cursor commit, then restart. Add conservation/property tests with a reference ledger and assert all-or-nothing state.
Test plan: attack, hack and abuse vectors
Repeated transfer replay, malicious conflicting event identities, reordered batches, registration during fetched ranges and replica concurrency attempting extra power; huge/negative/invalid amounts and underflow. Confirm unsafe boundaries fail closed and cannot manufacture voting balance.
Verification and completion criteria
Reproduce the original replay and registration-boundary failures as permanent regressions, then attach exact expected/actual balances and crash/concurrency results for both chains. Run unit plus isolated DB suites and validate forward migration preserves existing frozen proposals.
Record commit and pinned toolchain, commands/results and sanitized artifacts. Relevant Rust changes require
cargo test --workspace --libplus isolated Postgres integration (explicit guarded LEDGER_TEST_DATABASE_URL); frontend changes requirenpm testand production build plus relevant browser cases. Run secret scanning for new fixtures/config. Exercise fresh/upgrade migrations when schema/grants change; report skipped/live-only checks explicitly.Implementation status after main merge PR #53, commit
cd07f00f32f04ad02f8b760ed34588fd9645919e:Services are paused while existing storage is audited and the corrected pinned release is prepared. No production wallet transaction or hardware-wallet/Legal QA is claimed. This is an interim report; the issue remains open and the final comment will map the original criteria to actual live results and any unverified cases.
Implemented through PR53, 54, 55, 56, 57, 58, and PR59. Final code release
5cd9363(PR59; CI passed) is deployed across all three Coolify services, each finished and healthy. This draft records completed issue criteria for the parent to post after publishing the final evidence artifact.Verification checklist:
Transfer event, balance and cursor/hash state commit atomically under per-chain transaction advisory locks; duplicate payloads must match and overlaps apply distinct source events once.
Both chains preserve exact raw integers: before/at registration-baseline transfers leave the observed snapshot unchanged; later transfers apply once. Concurrent registration/ingestion retries overtaken snapshots without double credit.
Concurrent writers, reordered/multi-log deliveries and actual SIGKILL at event INSERT, balance update and COMMIT retain wholly old/new state and exact restart replay.
Underflow and invalid/out-of-width source amounts fail and roll back rather than silently repairing balances; token conservation and registered-only accounting expectations pass.
Forward upgrades/recovery preserve frozen proposal weights and original accepted history. The live inventory proves original registrations and transfers retain all original fields.
Pinned Rust1.88.0 complete fresh disposable release13 passed: 74 unit/supervisor tests, 121 integration tests (66 ledger hardening), 245 schema-validated API responses and 146 frontend tests, with production build/browser, nginx and Gitleaks checks. PR59 CI passed. Commands and results are recorded in the release report.
All three Coolify deployments of
5cd9363finished healthy. Private readiness was green in 240 + 240 + 600 samples (1080 total), with maximum Terra lag 2 and BSC lag 17; both checkpoints stayed complete/canonical and intents/baselines healthy. The final two public rounds passed 45/45 each through 14:07:21 UTC. Live smoke (13 checks), four canonical checks, both original expired-action exact receipt replays, both-chain offchain registration/receipt checks and runtime configuration passed. Grafana/Loki has no post-activation worker warnings; the direct Coolify RPC warning at 13:56:21 belongs to the old container before 13:58:22 activation. See final live evidence.Invariants, recovery/finality/freshness/rollback policy and third-party agent guidance are crosslinked in LEDGER_INVARIANTS.md, OPS.md and AGENTS_LEDGER_HARDENING.md.
Fresh pinned BSC range planning certifies only baselines through the actual selected end. A future unverified baseline with a divergent hash remains pending outside that end; post-SQL same-source end/head/network failures roll back all attempted certifications and accounting.
Regression evidence:
replay_boundary_and_concurrent_writers_both_chains,registration_cannot_insert_into_stale_registered_set_during_http_fetch_both_chains,deterministic_reference_ledger_conserves_raw_decimals_across_replay_both_chains,actual_ingest_cancellation_after_insert_balance_and_during_commit_retries_once_both_chains,concurrent_partially_overlapping_actual_ingest_ranges_apply_distinct_events_once_both_chains,process_sigkill_after_event_balance_and_during_commit_recovers_exactly_once_both_chains, andfinal_bsc_same_source_recheck_after_sql_rejects_fork_stale_header_wrong_height_and_network_atomically. See HTTP/process/SQL tests.Observed exception: the first public round had one unexplained 503 at 14:00:58.805988 UTC (44/45), recovering by 14:01:01; it fell outside the first private capture window. That earlier round is not the final acceptance window. The subsequent two public rounds and 1080 private samples passed; this is bounded observation evidence, not a promise of permanent availability or a reason to relax fail-closed readiness.
Limits: no blockchain transaction, new contract, archive dependency, chain-identity merge or browser BSC balance RPC. Funded human/hardware/mobile wallet and Legal QA, prolonged external CPU stress and production database downgrade/destructive attacks are not claimed by the isolated fixtures or unfunded live probes. Grafana currently provides Loki only; no Prometheus datasource was available. Legacy hash-only receipts and uncertified polls retain their documented limits.
The body acceptance/completion criteria and current comment requirements are satisfied by the named deterministic regressions, final release checks and recorded live evidence. Closing is supported after the parent publishes the final artifact; no remaining mandatory implementation or verification gap is identified.
Final evidence is published through PR #60. All mandatory body/comment criteria are satisfied by the checklist above; closing this issue.