Security: contract migration runbook does not document rollback limitations [SEC-H05] #443

Closed
opened 2026-06-30 10:19:09 +00:00 by totdking · 9 comments
totdking commented 2026-06-30 10:19:09 +00:00 (Migrated from gitlab.com)
No description provided.
totdking commented 2026-06-30 10:21:19 +00:00 (Migrated from gitlab.com)

Summary

smartcontracts/tests/src/migration_tests.rs provides migration state-preservation tests that simulate upgrades via cw2 version downgrade, covering factory, pair, and fee-discount. These satisfy the "rehearsed" component. However, docs/runbooks/wasm-admin-migration.md has no "rollback limitations" section. The runbook covers pre-flight, migration steps, automated regression, and admin rotation, but does not document what operators cannot do after a migration: e.g., that rolling back requires migrating to the prior code ID (if still stored on chain), that a failed migration with cleared admin cannot be undone, or that indexer DB migrations have separate down.sql files under indexer/migrations/revert/. The docs/runbooks/launch-checklist.md rollback section says only "CosmWasm upgrades/migrations are out of band for this runbook; document admin keys and wasm migration policy separately."


What Was Checked

  • smartcontracts/tests/src/migration_tests.rs lines 1-57: confirms migration tests exist, use cw2 version downgrade simulation (not archived prior wasm), and cover factory, pair, fee-discount. Tests run via make test-contracts.
  • docs/runbooks/wasm-admin-migration.md read in full: sections are Pre-flight, Migration/upgrade steps, Automated regression, Commands, Admin rotation, References. No Rollback or Rollback limitations section exists.
  • docs/runbooks/launch-checklist.md rollback section (lines 177-181): 2 lines only. Acknowledges CosmWasm migrations are out of band and says to pause via factory for incidents. No rollback limitation detail.
  • docs/testing.md: mentions indexer/migrations/revert/ down.sql files for indexer DB migrations. Not cross-linked from wasm-admin-migration.md.

Expected (per checklist)

The migration runbook includes a "rollback limitations" note covering:

(1) contract migration can be reversed by migrating back to the prior code ID if admin is retained and code ID is still on-chain;

(2) if admin is cleared or code ID is purged, the contract cannot be rolled back;

(3) indexer DB migrations have separate down.sql files under indexer/migrations/revert/ for manual rollback;

(4) what to do if a migration is partially applied (e.g., factory migrated but pair not yet).


Actual

No rollback limitations are documented in the wasm-admin-migration.md runbook or the launch-checklist.md rollback section. Operators encountering a failed or bad migration have no documented guidance on what can or cannot be reversed.


Evidence

  • docs/runbooks/wasm-admin-migration.md: no rollback or rollback-limitations section
  • docs/runbooks/launch-checklist.md lines 177-181: only "pause via factory" and "out of band" acknowledgement for CosmWasm migrations

Suggested Fix

Add a Rollback and limitations section to docs/runbooks/wasm-admin-migration.md covering:

(1) contract migration reversal path (migrate back to prior code ID using stored code, confirm code ID is still on chain with terrad query wasm code <id>);

(2) irrecoverable cases (admin cleared, code ID purged, or governance key lost);

(3) indexer DB migration rollback using indexer/migrations/revert/ down.sql files (reference docs/testing.md);

(4) partial-migration recovery steps when only some contracts have been migrated. Cross-link from docs/runbooks/launch-checklist.md rollback section.


Verification Checklist

  • docs/runbooks/wasm-admin-migration.md contains a rollback limitations section covering contract migration reversal and irrecoverable cases
  • The section references indexer DB migration down.sql files under indexer/migrations/revert/
  • docs/runbooks/launch-checklist.md rollback section cross-links to the wasm-admin-migration rollback limitations

Labels

security, pre-launch

Cc: @PlasticDigits

### Summary `smartcontracts/tests/src/migration_tests.rs` provides migration state-preservation tests that simulate upgrades via cw2 version downgrade, covering factory, pair, and fee-discount. These satisfy the "rehearsed" component. However, `docs/runbooks/wasm-admin-migration.md` has no "rollback limitations" section. The runbook covers pre-flight, migration steps, automated regression, and admin rotation, but does not document what operators cannot do after a migration: e.g., that rolling back requires migrating to the prior code ID (if still stored on chain), that a failed migration with cleared admin cannot be undone, or that indexer DB migrations have separate down.sql files under `indexer/migrations/revert/`. The `docs/runbooks/launch-checklist.md` rollback section says only "CosmWasm upgrades/migrations are out of band for this runbook; document admin keys and wasm migration policy separately." --- ### What Was Checked - `smartcontracts/tests/src/migration_tests.rs` lines 1-57: confirms migration tests exist, use cw2 version downgrade simulation (not archived prior wasm), and cover factory, pair, fee-discount. Tests run via `make test-contracts`. - `docs/runbooks/wasm-admin-migration.md` read in full: sections are Pre-flight, Migration/upgrade steps, Automated regression, Commands, Admin rotation, References. No Rollback or Rollback limitations section exists. - `docs/runbooks/launch-checklist.md` rollback section (lines 177-181): 2 lines only. Acknowledges CosmWasm migrations are out of band and says to pause via factory for incidents. No rollback limitation detail. - `docs/testing.md`: mentions `indexer/migrations/revert/` down.sql files for indexer DB migrations. Not cross-linked from wasm-admin-migration.md. --- ### Expected (per checklist) The migration runbook includes a "rollback limitations" note covering: (1) contract migration can be reversed by migrating back to the prior code ID if admin is retained and code ID is still on-chain; (2) if admin is cleared or code ID is purged, the contract cannot be rolled back; (3) indexer DB migrations have separate down.sql files under `indexer/migrations/revert/` for manual rollback; (4) what to do if a migration is partially applied (e.g., factory migrated but pair not yet). --- ### Actual No rollback limitations are documented in the wasm-admin-migration.md runbook or the launch-checklist.md rollback section. Operators encountering a failed or bad migration have no documented guidance on what can or cannot be reversed. --- ### Evidence - `docs/runbooks/wasm-admin-migration.md`: no rollback or rollback-limitations section - `docs/runbooks/launch-checklist.md` lines 177-181: only "pause via factory" and "out of band" acknowledgement for CosmWasm migrations --- ### Suggested Fix Add a **Rollback and limitations** section to `docs/runbooks/wasm-admin-migration.md` covering: (1) contract migration reversal path (migrate back to prior code ID using stored code, confirm code ID is still on chain with `terrad query wasm code <id>`); (2) irrecoverable cases (admin cleared, code ID purged, or governance key lost); (3) indexer DB migration rollback using `indexer/migrations/revert/` down.sql files (reference `docs/testing.md`); (4) partial-migration recovery steps when only some contracts have been migrated. Cross-link from `docs/runbooks/launch-checklist.md` rollback section. --- ### Verification Checklist - [ ] `docs/runbooks/wasm-admin-migration.md` contains a rollback limitations section covering contract migration reversal and irrecoverable cases - [ ] The section references indexer DB migration down.sql files under `indexer/migrations/revert/` - [ ] `docs/runbooks/launch-checklist.md` rollback section cross-links to the wasm-admin-migration rollback limitations --- ### Labels `security`, `pre-launch` Cc: @PlasticDigits
totdking commented 2026-06-30 10:24:12 +00:00 (Migrated from gitlab.com)

mentioned in issue #381

mentioned in issue #381
PlasticDigits commented 2026-06-30 11:00:22 +00:00 (Migrated from gitlab.com)

mentioned in merge request !975

mentioned in merge request !975
PlasticDigits commented 2026-06-30 11:15:56 +00:00 (Migrated from gitlab.com)

mentioned in commit 5058c61b95

mentioned in commit 5058c61b9595a1381fdd408b7a4b60efbd2c7a20
PlasticDigits commented 2026-06-30 11:16:44 +00:00 (Migrated from gitlab.com)

mentioned in commit c0bd454668

mentioned in commit c0bd454668363fb607795f1cc638ce3265785e7a
Brouie commented 2026-06-30 11:19:06 +00:00 (Migrated from gitlab.com)

Went through the SEC-H05 rollback-limitations MR (8c71c202, !975) — the runbook content holds up and the verification all passes for me.

make verify-issue-443 → 8/8 PASS, including the live make test-contracts (the SEC-C14 state-preservation rehearsal stays green; the migration_tests.rs change is a comment-only re-point to the new #rollback-and-limitations-sec-h05 anchor, so no behavior change there).

Walked the actual rollback section in wasm-admin-migration.md, not just the grep markers — it covers everything the issue asked:

  • contract migration reversal: confirm prior code_id still on chain (terrad query wasm code), confirm you still hold admin (contract_info.admin), migrate back to the prior code_id
  • irrecoverable cases table — admin cleared, prior code_id purged, governance/multisig key lost, bad-migration-with-retained-admin
  • indexer DB rollback pointing at indexer/migrations/revert/ .down.sql (and the three revert files exist exactly as named)
  • partial-migration recovery (stop / pause / revert the migrated set / keep the off-chain stack aligned / log to the incident timeline)
  • launch-checklist rollback section cross-links it

One thing before it lands: the MR is cannot_be_merged right now, but it's just a rebase — the conflict is the Makefile .PHONY/target-list line (it and !976 both add their verify-issue / check-docs targets to the same region, and main moved on since the branch was cut). The two SEC-H05/H09 MRs will also collide with each other on that line, so they want sequential rebasing. Content-wise, good to merge / good to close from my side once it's rebased.

@PlasticDigits

Went through the SEC-H05 rollback-limitations MR (8c71c202, !975) — the runbook content holds up and the verification all passes for me. `make verify-issue-443` → 8/8 PASS, including the live `make test-contracts` (the SEC-C14 state-preservation rehearsal stays green; the migration_tests.rs change is a comment-only re-point to the new `#rollback-and-limitations-sec-h05` anchor, so no behavior change there). Walked the actual rollback section in wasm-admin-migration.md, not just the grep markers — it covers everything the issue asked: - contract migration reversal: confirm prior code_id still on chain (`terrad query wasm code`), confirm you still hold admin (`contract_info.admin`), migrate back to the prior code_id - irrecoverable cases table — admin cleared, prior code_id purged, governance/multisig key lost, bad-migration-with-retained-admin - indexer DB rollback pointing at indexer/migrations/revert/ .down.sql (and the three revert files exist exactly as named) - partial-migration recovery (stop / pause / revert the migrated set / keep the off-chain stack aligned / log to the incident timeline) - launch-checklist rollback section cross-links it One thing before it lands: the MR is cannot_be_merged right now, but it's just a rebase — the conflict is the Makefile .PHONY/target-list line (it and !976 both add their verify-issue / check-docs targets to the same region, and main moved on since the branch was cut). The two SEC-H05/H09 MRs will also collide with each other on that line, so they want sequential rebasing. Content-wise, good to merge / good to close from my side once it's rebased. @PlasticDigits
PlasticDigits commented 2026-06-30 11:19:52 +00:00 (Migrated from gitlab.com)

Verification — GitLab #443 (SEC-H05)

Result: PASS — all acceptance criteria satisfied; no repo changes required.

Checklist

Criterion Result How verified
wasm-admin-migration.md rollback limitations section (reversal + irrecoverable cases) PASS make verify-issue-443 step "rollback section covers contract reversal path" + "rollback section documents irrecoverable cases"; manual read of docs/runbooks/wasm-admin-migration.md § Rollback and limitations (SEC-H05)
Section references indexer DB indexer/migrations/revert/ down.sql PASS make verify-issue-443 step "rollback section references indexer revert down.sql"; confirmed 3 .down.sql files exist under indexer/migrations/revert/
launch-checklist.md rollback section cross-links migration limitations PASS make verify-issue-443 step "launch runbook rollback cross-links migration limitations"; manual read of docs/runbooks/launch-checklist.md § Rollback / incident (line 199)
SEC-C14 migration rehearsal (related) PASS make verify-issue-443 — make test-contracts (388 tests passed)
Doc drift guard PASS make check-wasm-migration-rollback-docs via make verify-issue-443
Agent skill present PASS skills/AGENTS_WASM_MIGRATION_ROLLBACK.md exists

Command

make verify-issue-443
# PASS: 8   FAIL: 0

Coverage confirmed in docs

  1. Contract reversal — migrate back to prior code_id when admin intact and wasm still on chain (terrad query wasm code <id>).
  2. Irrecoverable cases — admin cleared, code ID purged, governance key lost.
  3. Indexer DB rollback — manual .down.sql under indexer/migrations/revert/, cross-linked to docs/testing.md.
  4. Partial migration recovery — stop fleet, pause pairs, revert migrated contracts, coordinate off-chain stack.

Closing as verified.

## Verification — GitLab #443 (SEC-H05) **Result: PASS** — all acceptance criteria satisfied; no repo changes required. ### Checklist | Criterion | Result | How verified | |-----------|--------|--------------| | `wasm-admin-migration.md` rollback limitations section (reversal + irrecoverable cases) | **PASS** | `make verify-issue-443` step "rollback section covers contract reversal path" + "rollback section documents irrecoverable cases"; manual read of `docs/runbooks/wasm-admin-migration.md` § Rollback and limitations (SEC-H05) | | Section references indexer DB `indexer/migrations/revert/` down.sql | **PASS** | `make verify-issue-443` step "rollback section references indexer revert down.sql"; confirmed 3 `.down.sql` files exist under `indexer/migrations/revert/` | | `launch-checklist.md` rollback section cross-links migration limitations | **PASS** | `make verify-issue-443` step "launch runbook rollback cross-links migration limitations"; manual read of `docs/runbooks/launch-checklist.md` § Rollback / incident (line 199) | | SEC-C14 migration rehearsal (related) | **PASS** | `make verify-issue-443` — `make test-contracts` (388 tests passed) | | Doc drift guard | **PASS** | `make check-wasm-migration-rollback-docs` via `make verify-issue-443` | | Agent skill present | **PASS** | `skills/AGENTS_WASM_MIGRATION_ROLLBACK.md` exists | ### Command ```bash make verify-issue-443 # PASS: 8 FAIL: 0 ``` ### Coverage confirmed in docs 1. **Contract reversal** — migrate back to prior `code_id` when admin intact and wasm still on chain (`terrad query wasm code <id>`). 2. **Irrecoverable cases** — admin cleared, code ID purged, governance key lost. 3. **Indexer DB rollback** — manual `.down.sql` under `indexer/migrations/revert/`, cross-linked to `docs/testing.md`. 4. **Partial migration recovery** — stop fleet, pause pairs, revert migrated contracts, coordinate off-chain stack. Closing as verified.
PlasticDigits (Migrated from gitlab.com) closed this issue 2026-06-30 11:19:53 +00:00
PlasticDigits commented 2026-06-30 11:22:04 +00:00 (Migrated from gitlab.com)

mentioned in commit 9313da2fb4

mentioned in commit 9313da2fb45d92a400c170de52c937961ef49082
PlasticDigits commented 2026-06-30 11:22:09 +00:00 (Migrated from gitlab.com)

mentioned in merge request !976

mentioned in merge request !976
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
code/cl8y-dex-terraclassic#443
No description provided.