Add launch-monitoring runbook + docs-check (#434) #964

Merged

Brouie merged 3 commits from issue-434-launch-monitoring-runbook into main 2026-06-30 03:52:23 +00:00

Conversation 9 Commits 3 Files changed 4 +247 -2

Brouie commented 2026-06-29 16:19:19 +00:00 (Migrated from gitlab.com)

Copy link

SEC-G01 / #434: the stack had no runbook listing monitoring commands/queries for launch signals — only reorg-halt alerting was documented.

What this adds

• docs/runbooks/launch-monitoring.md — for each launch signal: a tested command, its healthy baseline, and the escalation trigger. Covers: (1) contract/index error spikes, (2) indexer lag behind chain tip, (3) API 429/5xx, (4) large swaps, (5) large LP withdrawals, (6) blacklist hits, (7) pause-state changes, (8) the existing reorg halt. Plus a one-pass "quick sweep".
• scripts/check_launch_monitoring_docs.py + make check-launch-monitoring-docs (wired into make lint) — asserts the runbook covers all 8 signals so a future edit can't silently drop one.
• launch-checklist.md Phase 4 item referencing it.

Validated

Every command was run against the live LocalTerra/QA stack with real baselines baked into the doc — indexer lag 6, swap_events p99 return 442,197,580, indexer_failed_blocks 0, liquidity_events add/remove split, the blacklist endpoint shape, and the pause query. make check-launch-monitoring-docs passes; adversarially confirmed it fails when a signal section is removed.

Includes a log-grep caveat that bit me while validating: naive grep -ci error $LOG returned 7147 (substring "Parse error" in benign oracle WARNs), while the real ERROR-level count was 4 (all genuine reorg-halt events). The runbook tells operators to match the level token after stripping ANSI, not the bare word.

Note: touches the same .PHONY / lint: lines as !961 (#433) — if both merge, it's a trivial 2-word combine on those two lines.

For #434.

SEC-G01 / #434: the stack had no runbook listing monitoring commands/queries for launch signals — only reorg-halt alerting was documented. ## What this adds - **`docs/runbooks/launch-monitoring.md`** — for each launch signal: a **tested command**, its **healthy baseline**, and the **escalation trigger**. Covers: (1) contract/index error spikes, (2) indexer lag behind chain tip, (3) API 429/5xx, (4) large swaps, (5) large LP withdrawals, (6) blacklist hits, (7) pause-state changes, (8) the existing reorg halt. Plus a one-pass "quick sweep". - **`scripts/check_launch_monitoring_docs.py`** + **`make check-launch-monitoring-docs`** (wired into `make lint`) — asserts the runbook covers all 8 signals so a future edit can't silently drop one. - **launch-checklist.md** Phase 4 item referencing it. ## Validated Every command was run against the live LocalTerra/QA stack with real baselines baked into the doc — indexer lag `6`, `swap_events` p99 return `442,197,580`, `indexer_failed_blocks` `0`, `liquidity_events` add/remove split, the blacklist endpoint shape, and the pause query. `make check-launch-monitoring-docs` passes; adversarially confirmed it fails when a signal section is removed. Includes a **log-grep caveat** that bit me while validating: naive `grep -ci error $LOG` returned 7147 (substring "Parse error" in benign oracle WARNs), while the real `ERROR`-level count was 4 (all genuine reorg-halt events). The runbook tells operators to match the **level token** after stripping ANSI, not the bare word. Note: touches the same `.PHONY` / `lint:` lines as !961 (#433) — if both merge, it's a trivial 2-word combine on those two lines. For #434.

Brouie commented 2026-06-29 16:19:44 +00:00 (Migrated from gitlab.com)

Copy link

mentioned in issue #434

mentioned in issue #434

PlasticDigits commented 2026-06-29 16:22:03 +00:00 (Migrated from gitlab.com)

Copy link

Security review — MR !964

Commit reviewed: e3806af045e97d4cbf2f492b26dd9faab3fb767c

Scope: Added docs/runbooks/launch-monitoring.md, scripts/check_launch_monitoring_docs.py, Makefile lint target, and launch-checklist Phase 4 checkbox. No application/runtime code (indexer, contracts, frontend) changed.

Method: Diff review; traced operator runbook shell/SQL/curl snippets for injection, authz, secret exposure, and SSRF sinks; compared blacklist-check examples against indexer/src/api/compliance.rs (BlacklistCheckParams.wallet).

Outcome: FINDINGS: 0 medium+

This MR is documentation and a static doc-coverage checker only. The runbook commands are operator-run, use fixed SQL with operator-set thresholds, quote log paths, and source on-chain-address-shaped fields from the indexer DB. The checker reads a fixed repo path with no user input. No new attack surface in deployed services, dependencies, or CI beyond substring presence checks.

Inline threads: none (no medium+ findings).

Note (below threshold): Signal 6 curl examples use ?address= but the indexer API expects ?wallet= per BlacklistCheckParams — operators following the runbook literally would get wallet-less checks (blocked: false). That is an ops-doc correctness gap for SEC-G01 detection, not an exploitable vulnerability in this diff; recommend fixing in a follow-up doc pass.

## Security review — MR !964 **Commit reviewed:** `e3806af045e97d4cbf2f492b26dd9faab3fb767c` **Scope:** Added `docs/runbooks/launch-monitoring.md`, `scripts/check_launch_monitoring_docs.py`, `Makefile` lint target, and launch-checklist Phase 4 checkbox. No application/runtime code (indexer, contracts, frontend) changed. **Method:** Diff review; traced operator runbook shell/SQL/curl snippets for injection, authz, secret exposure, and SSRF sinks; compared blacklist-check examples against `indexer/src/api/compliance.rs` (`BlacklistCheckParams.wallet`). **Outcome:** `FINDINGS: 0` medium+ This MR is documentation and a static doc-coverage checker only. The runbook commands are operator-run, use fixed SQL with operator-set thresholds, quote log paths, and source on-chain-address-shaped fields from the indexer DB. The checker reads a fixed repo path with no user input. No new attack surface in deployed services, dependencies, or CI beyond substring presence checks. **Inline threads:** none (no medium+ findings). **Note (below threshold):** Signal 6 curl examples use `?address=` but the indexer API expects `?wallet=` per `BlacklistCheckParams` — operators following the runbook literally would get wallet-less checks (`blocked: false`). That is an ops-doc correctness gap for SEC-G01 detection, not an exploitable vulnerability in this diff; recommend fixing in a follow-up doc pass.

PlasticDigits commented 2026-06-29 16:26:11 +00:00 (Migrated from gitlab.com)

Copy link

Resolved merge conflicts with main in Makefile by keeping both targets from this MR and !961:

• check-launch-monitoring-docs (#434) — runbook coverage check
• lint-log-secrets (#433) — indexer log secret scan

Updated .PHONY, the individual targets, and the lint aggregate. Sanity: make check-launch-monitoring-docs and make lint-log-secrets both pass.

Resolved merge conflicts with `main` in **Makefile** by keeping both targets from this MR and !961: - `check-launch-monitoring-docs` (#434) — runbook coverage check - `lint-log-secrets` (#433) — indexer log secret scan Updated `.PHONY`, the individual targets, and the `lint` aggregate. Sanity: `make check-launch-monitoring-docs` and `make lint-log-secrets` both pass.

PlasticDigits commented 2026-06-29 16:26:23 +00:00 (Migrated from gitlab.com)

Copy link

added 5 commits

• e3806af0...de59858d - 4 commits from branch PlasticDigits:main
• 131058d1 - Merge origin/main into issue-434-launch-monitoring-runbook

Compare with previous version

added 5 commits <ul><li>e3806af0...de59858d - 4 commits from branch <code>PlasticDigits:main</code></li><li>131058d1 - Merge origin/main into issue-434-launch-monitoring-runbook</li></ul> [Compare with previous version](/PlasticDigits/cl8y-dex-terraclassic/-/merge_requests/246/diffs?diff_id=1876585898&start_sha=e3806af045e97d4cbf2f492b26dd9faab3fb767c)

PlasticDigits commented 2026-06-29 16:27:54 +00:00 (Migrated from gitlab.com)

Copy link

Security review — MR !964 (re-review)

Commit reviewed: 131058d111fdcae7f32be3d1ea9c52f8eded2e17

Scope: Re-reviewed after merge with main (Makefile conflict resolution with !961). Changes remain: docs/runbooks/launch-monitoring.md, scripts/check_launch_monitoring_docs.py, Makefile lint wiring (check-launch-monitoring-docs), and launch-checklist Phase 4 checkbox. No application/runtime code (indexer, contracts, frontend) in this diff.

Method: Full diff review; traced operator runbook shell/SQL/curl snippets for injection, authz bypass, secret exposure, and SSRF sinks; verified blacklist-check examples against indexer/src/api/compliance.rs (BlacklistCheckParams.wallet).

Outcome: FINDINGS: 0 medium+

This MR adds operator documentation and a static doc-coverage checker only. Runbook commands are operator-run locally: SQL uses fixed queries with operator-set numeric thresholds ($THRESH, $THRESH_LP), log paths are quoted ("$LOG"), and on-chain-address-shaped fields come from the indexer DB or operator env. The Python checker reads a fixed repo path with no external/user input. No new attack surface in deployed services, dependencies, or CI beyond substring presence checks.

Inline threads: none (no medium+ findings).

Note (below threshold, unchanged from prior review on e3806af): Signal 6 curl examples use ?address= but the indexer API expects ?wallet= per BlacklistCheckParams. Operators following the runbook literally would get wallet-less checks (blocked: false). That is an ops-doc correctness gap for SEC-G01 detection, not an exploitable vulnerability in this diff; recommend fixing in a follow-up doc pass.

## Security review — MR !964 (re-review) **Commit reviewed:** `131058d111fdcae7f32be3d1ea9c52f8eded2e17` **Scope:** Re-reviewed after merge with `main` (Makefile conflict resolution with !961). Changes remain: `docs/runbooks/launch-monitoring.md`, `scripts/check_launch_monitoring_docs.py`, `Makefile` lint wiring (`check-launch-monitoring-docs`), and launch-checklist Phase 4 checkbox. No application/runtime code (indexer, contracts, frontend) in this diff. **Method:** Full diff review; traced operator runbook shell/SQL/curl snippets for injection, authz bypass, secret exposure, and SSRF sinks; verified blacklist-check examples against `indexer/src/api/compliance.rs` (`BlacklistCheckParams.wallet`). **Outcome:** `FINDINGS: 0` medium+ This MR adds operator documentation and a static doc-coverage checker only. Runbook commands are operator-run locally: SQL uses fixed queries with operator-set numeric thresholds (`$THRESH`, `$THRESH_LP`), log paths are quoted (`"$LOG"`), and on-chain-address-shaped fields come from the indexer DB or operator env. The Python checker reads a fixed repo path with no external/user input. No new attack surface in deployed services, dependencies, or CI beyond substring presence checks. **Inline threads:** none (no medium+ findings). **Note (below threshold, unchanged from prior review on `e3806af`):** Signal 6 curl examples use `?address=` but the indexer API expects `?wallet=` per `BlacklistCheckParams`. Operators following the runbook literally would get wallet-less checks (`blocked: false`). That is an ops-doc correctness gap for SEC-G01 detection, not an exploitable vulnerability in this diff; recommend fixing in a follow-up doc pass.

PlasticDigits commented 2026-06-30 02:19:28 +00:00 (Migrated from gitlab.com)

Copy link

Resolved merge conflicts with main in Makefile only.

Combined both sides on .PHONY and lint::

• kept lint-indexer-log-secrets + lint-log-secrets from main (!961)
• kept check-launch-monitoring-docs from this branch

Sanity: make check-launch-monitoring-docs passes.

Resolved merge conflicts with `main` in **Makefile** only. Combined both sides on `.PHONY` and `lint:`: - kept `lint-indexer-log-secrets` + `lint-log-secrets` from main (!961) - kept `check-launch-monitoring-docs` from this branch Sanity: `make check-launch-monitoring-docs` passes.

PlasticDigits commented 2026-06-30 02:21:05 +00:00 (Migrated from gitlab.com)

Copy link

added 4 commits

• 131058d1...2a06048b - 3 commits from branch PlasticDigits:main
• a5bb1c32 - Merge origin/main into issue-434-launch-monitoring-runbook

Compare with previous version

added 4 commits <ul><li>131058d1...2a06048b - 3 commits from branch <code>PlasticDigits:main</code></li><li>a5bb1c32 - Merge origin/main into issue-434-launch-monitoring-runbook</li></ul> [Compare with previous version](/PlasticDigits/cl8y-dex-terraclassic/-/merge_requests/246/diffs?diff_id=1877278459&start_sha=131058d111fdcae7f32be3d1ea9c52f8eded2e17)

PlasticDigits commented 2026-06-30 02:22:49 +00:00 (Migrated from gitlab.com)

Copy link

Security review — MR !964

Commit reviewed: a5bb1c32c9a60873e2709b0035cf7702ac035ddb

Scope: Re-reviewed after latest merge with main (Makefile conflict resolution). Changes remain: docs/runbooks/launch-monitoring.md, scripts/check_launch_monitoring_docs.py, Makefile lint wiring (check-launch-monitoring-docs), and launch-checklist Phase 4 checkbox. No application/runtime code (indexer, contracts, frontend) in this diff.

Method: Full diff review; traced operator runbook shell/SQL/curl snippets for injection, authz bypass, secret exposure, and SSRF sinks; verified blacklist-check examples against indexer/src/api/compliance.rs (BlacklistCheckParams.wallet); confirmed prior review threads still do not apply as medium+ findings on current SHA.

Outcome: FINDINGS: 0 medium+

This MR adds operator documentation and a static doc-coverage checker only. Runbook commands are operator-run locally: SQL uses fixed queries with operator-set numeric thresholds ($THRESH, $THRESH_LP), log paths are quoted ("$LOG"), and on-chain-address-shaped fields come from the indexer DB or operator env. The Python checker reads a fixed repo path with no external/user input. No new attack surface in deployed services, dependencies, or CI beyond substring presence checks.

Inline threads: none (no medium+ findings).

Note (below threshold, unchanged): Signal 6 curl examples use ?address= but the indexer API expects ?wallet= per BlacklistCheckParams. Operators following the runbook literally would get wallet-less checks (blocked: false). That is an ops-doc correctness gap for SEC-G01 detection, not an exploitable vulnerability in this diff; recommend fixing in a follow-up doc pass.

## Security review — MR !964 **Commit reviewed:** `a5bb1c32c9a60873e2709b0035cf7702ac035ddb` **Scope:** Re-reviewed after latest merge with `main` (Makefile conflict resolution). Changes remain: `docs/runbooks/launch-monitoring.md`, `scripts/check_launch_monitoring_docs.py`, `Makefile` lint wiring (`check-launch-monitoring-docs`), and launch-checklist Phase 4 checkbox. No application/runtime code (indexer, contracts, frontend) in this diff. **Method:** Full diff review; traced operator runbook shell/SQL/curl snippets for injection, authz bypass, secret exposure, and SSRF sinks; verified blacklist-check examples against `indexer/src/api/compliance.rs` (`BlacklistCheckParams.wallet`); confirmed prior review threads still do not apply as medium+ findings on current SHA. **Outcome:** `FINDINGS: 0` medium+ This MR adds operator documentation and a static doc-coverage checker only. Runbook commands are operator-run locally: SQL uses fixed queries with operator-set numeric thresholds (`$THRESH`, `$THRESH_LP`), log paths are quoted (`"$LOG"`), and on-chain-address-shaped fields come from the indexer DB or operator env. The Python checker reads a fixed repo path with no external/user input. No new attack surface in deployed services, dependencies, or CI beyond substring presence checks. **Inline threads:** none (no medium+ findings). **Note (below threshold, unchanged):** Signal 6 curl examples use `?address=` but the indexer API expects `?wallet=` per `BlacklistCheckParams`. Operators following the runbook literally would get wallet-less checks (`blocked: false`). That is an ops-doc correctness gap for SEC-G01 detection, not an exploitable vulnerability in this diff; recommend fixing in a follow-up doc pass.

PlasticDigits commented 2026-06-30 03:52:24 +00:00 (Migrated from gitlab.com)

Copy link

mentioned in commit 32063409d0

mentioned in commit 32063409d0caea28c66b3a142e9199a10c972c88

PlasticDigits (Migrated from gitlab.com) merged commit 32063409d0 into main 2026-06-30 03:52:24 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

agent:fix_bugfix

  

agent:fix_conflicts

  

agent:fix_security

  

agent:gap_analysis

  

agent:implement

  

agent:implement

Queue implement gate

  

agent:implement

Queue implement gate

  

agent:open_issues

  

agent:ready

  

agent:research

  

agent:security_audit

  

agent:verify

  

architecture

Architecture

  

backend

API and services

  

blocker:hybrid

Blocks hybrid launch

  

blocker:launch

Blocks launch

  

blocker:limit-orders

Blocks limits launch

  

blocker:v2

Blocks v2

  

block:log_only

  

block:security

  

bug

  

ci

  

contracts

Smart contracts

  

correctness

  

deploy

  

dev

  

devops

  

docs

  

documentation

  

duplicate

  

e2e

  

enhancement

  

epic

Epic

  

feature

  

frontend

  

functional-completion

Functional

  

gas

  

good first issue

  

governance

  

help wanted

  

high-risk

  

hooks

  

hybrid

Hybrid swaps

  

indexer

Indexer

  

infra

CI/CD

  

infrastructure

  

integrators

  

invalid

  

launch-blocker

  

limit-orders

Limit orders

  

localnet

  

localterra

  

low priority

  

missing-implementation

Implementation

  

needs-design

Design decision required

  

ops

  

performance

  

priority

high

  

priority

medium

  

product

  

qa

  

QA

  

question

  

ready

Definition of Ready

  

ready

Definition of Ready

  

research

  

scripts

  

security

Security

  

security-hardening

Security hardening

  

smartcontracts

  

tech-debt

  

testing

Testing

  

ux

  

UX

  

v2

Pool-only swaps

  

verification

  

wontfix

No labels

agent:fix_bugfix

agent:fix_conflicts

agent:fix_security

agent:gap_analysis

agent:implement

agent:implement

agent:implement

agent:open_issues

agent:ready

agent:research

agent:security_audit

agent:verify

architecture

backend

blocker:hybrid

blocker:launch

blocker:limit-orders

blocker:v2

block:log_only

block:security

bug

ci

contracts

correctness

deploy

dev

devops

docs

documentation

duplicate

e2e

enhancement

epic

feature

frontend

functional-completion

gas

good first issue

governance

help wanted

high-risk

hooks

hybrid

indexer

infra

infrastructure

integrators

invalid

launch-blocker

limit-orders

localnet

localterra

low priority

missing-implementation

needs-design

ops

performance

priority

high

priority

medium

product

qa

QA

question

ready

ready

research

scripts

security

security-hardening

smartcontracts

tech-debt

testing

ux

UX

v2

verification

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

code/cl8y-dex-terraclassic!964

No description provided.