DX: add make test-charts-integration for one-command local charts Vitest setup #205

Closed
opened 2026-05-27 12:37:10 +00:00 by PlasticDigits · 8 comments
PlasticDigits commented 2026-05-27 12:37:10 +00:00 (Migrated from gitlab.com)

Problem statement

Frontend charts integration tests (npm run test:integration) are implemented and run in CI, but local/agent verification is easy to get wrong when using a shared QA stack (dex_indexer + indexer on :3001).

Running integration tests without the fixture seed fails with 404 on the seeded pair candle endpoint — not because charts integration is missing, but because the harness expects a known fixture row that CI seeds automatically.

Observed during GitLab #199 verification: ChartsPage.integration.test.tsx → GET /api/v1/pairs/{addr}/candles returns candles for seeded pair failed until indexer/scripts/seed-charts-integration.sql was applied manually to dex_indexer.

Requirements

Add a single Make target (proposed name: test-charts-integration; alias tests-charts-integration optional) that:

  1. Uses the same fixture contract as CI and docs (CHARTS_INTEGRATION_PAIR_ADDRESS = terra1paircontractabc).
  2. Ensures Postgres is reachable (host :5432 or compose — follow existing scripts/lib/postgres-dev.env / AGENTS_LOCAL_POSTGRES_DEV.md conventions).
  3. Ensures the target database exists and migrations are applied (sqlx migrate run in indexer/).
  4. Applies charts integration seed idempotently (safe to re-run; no duplicate-key failures on second invocation).
  5. Verifies indexer HTTP is healthy at VITE_INDEXER_URL (default http://127.0.0.1:3001) — fail fast with a clear message if down (do not silently skip).
  6. Runs npm run test:integration via scripts/with-node.sh (same as other frontend Make targets).
  7. Documents the target in docs/testing.md, Makefile help, and skills/AGENTS_TESTING_P2_EPIC.md.

Out of scope (for this issue)

  • Changing what the integration tests assert (fixture pair stays fixed).
  • Replacing the CI job (CI may call the same script for DRY, but not required for acceptance).

Relevant files

File Role
Makefile Add test-charts-integration target; update help
scripts/setup-postgres-dev-databases.sh Pattern for idempotent DB ensure
scripts/lib/postgres-dev.env Default DATABASE_URL / credentials
indexer/scripts/seed-charts-integration.sql Fixture data (may need idempotent rewrite: INSERT … ON CONFLICT or upsert script)
frontend-dapp/src/test/chartsIntegrationConstants.ts Pair address must stay in sync with seed
frontend-dapp/src/pages/ChartsPage.integration.test.tsx Integration test suite
frontend-dapp/vitest.config.integration.ts Vitest integration config
frontend-dapp/package.json test:integration script
.github/workflows/test.yml CI reference implementation (migrate → seed → indexer → vitest)
docs/testing.md Local run docs (§ Integration Tests — Charts)
skills/AGENTS_TESTING_P2_EPIC.md Agent playbook commands
skills/AGENTS_LOCAL_POSTGRES_DEV.md Host Postgres setup
  1. New script scripts/test-charts-integration.sh (or scripts/qa/test-charts-integration.sh):

    • Source scripts/lib/postgres-dev.env (+ optional repo .env).
    • Default CHARTS_INT_DATABASE_URL to DATABASE_URL (dex_indexer) so QA stacks work; allow override via env for isolated DB (e.g. cl8y_charts_int like CI).
    • ensure_db + cd indexer && sqlx migrate run.
    • Run idempotent seed (update SQL or wrap in a small shell upsert/truncate-and-seed for the fixture pair only).
    • curl -sf "${VITE_INDEXER_URL:-http://127.0.0.1:3001}/health" — exit 1 with hint: "Start indexer first (make start-qa or make indexer-dev)".
    • bash scripts/with-node.sh --cwd frontend-dapp -- env VITE_INDEXER_URL=… npm run test:integration.
  2. Makefile:

    test-charts-integration:
    	@chmod +x scripts/test-charts-integration.sh
    	./scripts/test-charts-integration.sh
    
  3. Docs: Replace the 5-step manual bullet list with make test-charts-integration as the primary local path; keep env overrides documented.

  4. Optional (nice): CI job step calls ./scripts/test-charts-integration.sh with CHARTS_INT_DATABASE_URL=… after starting its ephemeral indexer — reduces drift.

Acceptance criteria

  • make test-charts-integration from repo root passes when Postgres + indexer on :3001 are up (host or QA stack).
  • Second consecutive run succeeds (idempotent seed).
  • Clear error when Postgres or indexer is unreachable (non-zero exit, actionable message).
  • make help lists the new target.
  • docs/testing.md updated — one-command path is primary.
  • skills/AGENTS_TESTING_P2_EPIC.md updated with the Make command.
  • No change to CHARTS_INTEGRATION_PAIR_ADDRESS / seed pair address without updating both SQL and TS constant.

Verification checklist

  • Fresh shell: Postgres up, indexer up on :3001 → make test-charts-integration → all integration tests green (7/7 at time of writing).
  • Run make test-charts-integration twice back-to-back — no duplicate-key / seed errors.
  • Stop indexer → make test-charts-integration → fails fast with helpful message (not obscure Vitest 404).
  • grep test-charts-integration Makefile docs/testing.md skills/AGENTS_TESTING_P2_EPIC.md — docs reference the target.
  • Optional: run against shared QA dex_indexer (not only empty DB) — candle test passes after Make target runs.

Priority

P3 — DX / agent ergonomics; CI already covers charts integration.

## Problem statement Frontend **charts integration** tests (`npm run test:integration`) are implemented and run in CI, but local/agent verification is easy to get wrong when using a shared QA stack (`dex_indexer` + indexer on `:3001`). Running integration tests without the fixture seed fails with **404** on the seeded pair candle endpoint — not because charts integration is missing, but because the harness expects a known fixture row that CI seeds automatically. **Observed during GitLab #199 verification:** `ChartsPage.integration.test.tsx` → `GET /api/v1/pairs/{addr}/candles returns candles for seeded pair` failed until `indexer/scripts/seed-charts-integration.sql` was applied manually to `dex_indexer`. ## Requirements Add a **single Make target** (proposed name: `test-charts-integration`; alias `tests-charts-integration` optional) that: 1. Uses the same fixture contract as CI and docs (`CHARTS_INTEGRATION_PAIR_ADDRESS` = `terra1paircontractabc`). 2. Ensures Postgres is reachable (host `:5432` or compose — follow existing `scripts/lib/postgres-dev.env` / `AGENTS_LOCAL_POSTGRES_DEV.md` conventions). 3. Ensures the target database exists and migrations are applied (`sqlx migrate run` in `indexer/`). 4. Applies charts integration seed **idempotently** (safe to re-run; no duplicate-key failures on second invocation). 5. Verifies indexer HTTP is healthy at `VITE_INDEXER_URL` (default `http://127.0.0.1:3001`) — **fail fast** with a clear message if down (do not silently skip). 6. Runs `npm run test:integration` via `scripts/with-node.sh` (same as other frontend Make targets). 7. Documents the target in `docs/testing.md`, `Makefile help`, and `skills/AGENTS_TESTING_P2_EPIC.md`. ### Out of scope (for this issue) - Changing what the integration tests assert (fixture pair stays fixed). - Replacing the CI job (CI may call the same script for DRY, but not required for acceptance). ## Relevant files | File | Role | |------|------| | [`Makefile`](Makefile) | Add `test-charts-integration` target; update `help` | | [`scripts/setup-postgres-dev-databases.sh`](scripts/setup-postgres-dev-databases.sh) | Pattern for idempotent DB ensure | | [`scripts/lib/postgres-dev.env`](scripts/lib/postgres-dev.env) | Default `DATABASE_URL` / credentials | | [`indexer/scripts/seed-charts-integration.sql`](indexer/scripts/seed-charts-integration.sql) | Fixture data (may need idempotent rewrite: `INSERT … ON CONFLICT` or upsert script) | | [`frontend-dapp/src/test/chartsIntegrationConstants.ts`](frontend-dapp/src/test/chartsIntegrationConstants.ts) | Pair address must stay in sync with seed | | [`frontend-dapp/src/pages/ChartsPage.integration.test.tsx`](frontend-dapp/src/pages/ChartsPage.integration.test.tsx) | Integration test suite | | [`frontend-dapp/vitest.config.integration.ts`](frontend-dapp/vitest.config.integration.ts) | Vitest integration config | | [`frontend-dapp/package.json`](frontend-dapp/package.json) | `test:integration` script | | [`.github/workflows/test.yml`](.github/workflows/test.yml) | CI reference implementation (migrate → seed → indexer → vitest) | | [`docs/testing.md`](docs/testing.md) | Local run docs (§ Integration Tests — Charts) | | [`skills/AGENTS_TESTING_P2_EPIC.md`](skills/AGENTS_TESTING_P2_EPIC.md) | Agent playbook commands | | [`skills/AGENTS_LOCAL_POSTGRES_DEV.md`](skills/AGENTS_LOCAL_POSTGRES_DEV.md) | Host Postgres setup | ## Recommended fix 1. **New script** `scripts/test-charts-integration.sh` (or `scripts/qa/test-charts-integration.sh`): - Source `scripts/lib/postgres-dev.env` (+ optional repo `.env`). - Default `CHARTS_INT_DATABASE_URL` to `DATABASE_URL` (`dex_indexer`) so QA stacks work; allow override via env for isolated DB (e.g. `cl8y_charts_int` like CI). - `ensure_db` + `cd indexer && sqlx migrate run`. - Run idempotent seed (update SQL or wrap in a small shell upsert/truncate-and-seed for the fixture pair only). - `curl -sf "${VITE_INDEXER_URL:-http://127.0.0.1:3001}/health"` — exit 1 with hint: *"Start indexer first (make start-qa or make indexer-dev)"*. - `bash scripts/with-node.sh --cwd frontend-dapp -- env VITE_INDEXER_URL=… npm run test:integration`. 2. **Makefile**: ```makefile test-charts-integration: @chmod +x scripts/test-charts-integration.sh ./scripts/test-charts-integration.sh ``` 3. **Docs**: Replace the 5-step manual bullet list with `make test-charts-integration` as the primary local path; keep env overrides documented. 4. **Optional (nice):** CI job step calls `./scripts/test-charts-integration.sh` with `CHARTS_INT_DATABASE_URL=…` after starting its ephemeral indexer — reduces drift. ## Acceptance criteria - [ ] `make test-charts-integration` from repo root passes when Postgres + indexer on `:3001` are up (host or QA stack). - [ ] Second consecutive run succeeds (idempotent seed). - [ ] Clear error when Postgres or indexer is unreachable (non-zero exit, actionable message). - [ ] `make help` lists the new target. - [ ] [`docs/testing.md`](docs/testing.md) updated — one-command path is primary. - [ ] [`skills/AGENTS_TESTING_P2_EPIC.md`](skills/AGENTS_TESTING_P2_EPIC.md) updated with the Make command. - [ ] No change to `CHARTS_INTEGRATION_PAIR_ADDRESS` / seed pair address without updating both SQL and TS constant. ## Verification checklist - [ ] Fresh shell: Postgres up, indexer up on `:3001` → `make test-charts-integration` → all integration tests green (7/7 at time of writing). - [ ] Run `make test-charts-integration` **twice** back-to-back — no duplicate-key / seed errors. - [ ] Stop indexer → `make test-charts-integration` → fails fast with helpful message (not obscure Vitest 404). - [ ] `grep test-charts-integration Makefile docs/testing.md skills/AGENTS_TESTING_P2_EPIC.md` — docs reference the target. - [ ] Optional: run against shared QA `dex_indexer` (not only empty DB) — candle test passes after Make target runs. ## Priority **P3** — DX / agent ergonomics; CI already covers charts integration. ## Related - GitLab **#104** — charts integration suite - GitLab **#199** — P2 epic verification (manual seed papercut observed) - [`docs/testing.md` § Integration Tests (Frontend)](docs/testing.md#integration-tests-frontend)
PlasticDigits commented 2026-05-27 13:53:38 +00:00 (Migrated from gitlab.com)

mentioned in commit a09e3c02b2

mentioned in commit a09e3c02b26567e3a6e1ff42873d5c986bee1e3f
PlasticDigits commented 2026-05-27 13:53:50 +00:00 (Migrated from gitlab.com)

Implemented in a09e3c0 on main (GitLab #205).

Summary

Added make test-charts-integration (alias make tests-charts-integration) backed by scripts/test-charts-integration.sh:

  1. Sources scripts/lib/postgres-dev.env (+ optional repo .env)
  2. Ensures target DB exists (default dex_indexer; override with CHARTS_INT_DATABASE_URL)
  3. Runs sqlx migrate run in indexer/
  4. Applies idempotent charts fixture seed (indexer/scripts/seed-charts-integration.sql — ON CONFLICT DO NOTHING, fixed candle open_time)
  5. Fail-fast indexer check at VITE_INDEXER_URL/health (default :3001)
  6. Runs npm run test:integration via scripts/with-node.sh with default LocalTerra LCD/RPC env for #166 pool-ref tests

Docs / invariants

Verification checklist

  • Fresh shell with Postgres + indexer on :3001 (+ LocalTerra LCD for full 7/7): make test-charts-integration → all integration tests green
  • Run make test-charts-integration twice back-to-back → no duplicate-key / seed errors
  • Stop indexer → make test-charts-integration → non-zero exit with actionable message (not Vitest 404)
  • grep test-charts-integration Makefile docs/testing.md skills/AGENTS_TESTING_P2_EPIC.md → docs reference the target
  • Optional: shared QA dex_indexer stack → candle test passes after Make target runs

@brouie — please verify on your QA stack when convenient. Leaving issue open until confirmed.

Implemented in `a09e3c0` on `main` (GitLab #205). ## Summary Added **`make test-charts-integration`** (alias `make tests-charts-integration`) backed by [`scripts/test-charts-integration.sh`](scripts/test-charts-integration.sh): 1. Sources [`scripts/lib/postgres-dev.env`](scripts/lib/postgres-dev.env) (+ optional repo `.env`) 2. Ensures target DB exists (default `dex_indexer`; override with `CHARTS_INT_DATABASE_URL`) 3. Runs `sqlx migrate run` in `indexer/` 4. Applies **idempotent** charts fixture seed ([`indexer/scripts/seed-charts-integration.sql`](indexer/scripts/seed-charts-integration.sql) — `ON CONFLICT DO NOTHING`, fixed candle `open_time`) 5. **Fail-fast** indexer check at `VITE_INDEXER_URL/health` (default `:3001`) 6. Runs `npm run test:integration` via `scripts/with-node.sh` with default LocalTerra LCD/RPC env for #166 pool-ref tests ## Docs / invariants - [`docs/testing.md`](docs/testing.md) — one-command path is primary; fixture pair invariant cross-linked - [`skills/AGENTS_TESTING_P2_EPIC.md`](skills/AGENTS_TESTING_P2_EPIC.md) — agent command + invariant row - [`skills/AGENTS_LOCAL_POSTGRES_DEV.md`](skills/AGENTS_LOCAL_POSTGRES_DEV.md) — Make target cross-link - **Fixture invariant:** `CHARTS_INTEGRATION_PAIR_ADDRESS` (`terra1paircontractabc`) must stay in sync with seed SQL + [`frontend-dapp/src/test/chartsIntegrationConstants.ts`](frontend-dapp/src/test/chartsIntegrationConstants.ts) ## Verification checklist - [ ] Fresh shell with Postgres + indexer on `:3001` (+ LocalTerra LCD for full 7/7): `make test-charts-integration` → all integration tests green - [ ] Run **`make test-charts-integration` twice** back-to-back → no duplicate-key / seed errors - [ ] Stop indexer → `make test-charts-integration` → non-zero exit with actionable message (not Vitest 404) - [ ] `grep test-charts-integration Makefile docs/testing.md skills/AGENTS_TESTING_P2_EPIC.md` → docs reference the target - [ ] Optional: shared QA `dex_indexer` stack → candle test passes after Make target runs @brouie — please verify on your QA stack when convenient. Leaving issue **open** until confirmed.
PlasticDigits commented 2026-05-29 03:13:21 +00:00 (Migrated from gitlab.com)

mentioned in issue #211

mentioned in issue #211
PlasticDigits commented 2026-05-29 05:41:53 +00:00 (Migrated from gitlab.com)

mentioned in issue #230

mentioned in issue #230
PlasticDigits commented 2026-05-29 05:41:54 +00:00 (Migrated from gitlab.com)

marked as related to #230

marked as related to #230
PlasticDigits commented 2026-05-29 13:23:19 +00:00 (Migrated from gitlab.com)

mentioned in commit da335fa4e1

mentioned in commit da335fa4e1ac9606669c94ec4885364d29280685
PlasticDigits commented 2026-05-29 13:24:08 +00:00 (Migrated from gitlab.com)

Verification update (agent)

Verified GitLab #205 on branch verify/issue-205, merged to main as da335fa.

What was wrong

  1. Fixture candles outside API window — seed used 2024-06-01 timestamps; indexer GET .../candles defaults to a 90-day lookback (DEFAULT_CANDLE_LOOKBACK_DAYS), so the candle test returned [] even after seeding.
  2. Stale limit-order pair addresses — integration tests hardcoded a pre-deploy pair; after make deploy-local the first EMBER/CORAL pair differs.
  3. Harness edge cases — localhost for LCD (IPv6 ::1 hangs), host sqlx migrate timeouts, and LCD queries from host when port-forward is flaky.

Fixes shipped

  • indexer/scripts/seed-charts-integration.sql — delete/refresh fixture 1h candle to current UTC hour (documented vs docs/indexer-invariants.md).
  • scripts/test-charts-integration.sh — 127.0.0.1 LCD/RPC defaults, docker-network sqlx fallback, _lcd_curl via compose exec, factory → VITE_LIMIT_ORDER_INTEGRATION_*.
  • frontend-dapp/src/test/limitOrderIntegrationConstants.ts + Vitest describe.skipIf when pair env unset (CI / charts-only).
  • Docs/skills cross-links updated.

Checklist for @brouie

  • Postgres up (docker compose up -d postgres) + indexer on :3001 (make indexer-dev or QA)
  • make test-charts-integration → 5 charts tests pass; 2 limit tests run only after make deploy-local + factory resolve (or skip with message)
  • Run make test-charts-integration twice — no duplicate-key / seed errors
  • Stop indexer → make test-charts-integration → non-zero exit with actionable message (not Vitest 404)
  • grep test-charts-integration Makefile docs/testing.md skills/AGENTS_TESTING_P2_EPIC.md
  • Optional: shared QA dex_indexer — candle test passes after Make target

Note: Full 7/7 green was confirmed for charts (5/5) with indexer + seed; limit-order 2/2 need a deployed LocalTerra factory (this agent hit chain reset / host LCD port quirks). Please confirm 7/7 on your stack before closing.

## Verification update (agent) Verified GitLab **#205** on branch `verify/issue-205`, merged to `main` as **da335fa**. ### What was wrong 1. **Fixture candles outside API window** — seed used `2024-06-01` timestamps; indexer `GET .../candles` defaults to a **90-day** lookback (`DEFAULT_CANDLE_LOOKBACK_DAYS`), so the candle test returned `[]` even after seeding. 2. **Stale limit-order pair addresses** — integration tests hardcoded a pre-deploy pair; after `make deploy-local` the first EMBER/CORAL pair differs. 3. **Harness edge cases** — `localhost` for LCD (IPv6 `::1` hangs), host `sqlx migrate` timeouts, and LCD queries from host when port-forward is flaky. ### Fixes shipped - `indexer/scripts/seed-charts-integration.sql` — delete/refresh fixture `1h` candle to **current UTC hour** (documented vs `docs/indexer-invariants.md`). - `scripts/test-charts-integration.sh` — `127.0.0.1` LCD/RPC defaults, docker-network **sqlx** fallback, `_lcd_curl` via compose exec, factory → `VITE_LIMIT_ORDER_INTEGRATION_*`. - `frontend-dapp/src/test/limitOrderIntegrationConstants.ts` + Vitest `describe.skipIf` when pair env unset (CI / charts-only). - Docs/skills cross-links updated. ### Checklist for @brouie - [ ] Postgres up (`docker compose up -d postgres`) + indexer on `:3001` (`make indexer-dev` or QA) - [ ] `make test-charts-integration` → **5 charts tests pass**; **2 limit tests run** only after `make deploy-local` + factory resolve (or skip with message) - [ ] Run `make test-charts-integration` **twice** — no duplicate-key / seed errors - [ ] Stop indexer → `make test-charts-integration` → **non-zero exit** with actionable message (not Vitest 404) - [ ] `grep test-charts-integration Makefile docs/testing.md skills/AGENTS_TESTING_P2_EPIC.md` - [ ] Optional: shared QA `dex_indexer` — candle test passes after Make target **Note:** Full **7/7** green was confirmed for **charts (5/5)** with indexer + seed; limit-order **2/2** need a deployed LocalTerra factory (this agent hit chain reset / host LCD port quirks). Please confirm 7/7 on your stack before closing.
PlasticDigits commented 2026-05-29 14:36:26 +00:00 (Migrated from gitlab.com)

Verification complete (agent)

Re-ran full #205 checklist with infra healthy on main (543ca70).

Check Result
make test-charts-integration (Postgres + indexer :3001 + LocalTerra LCD) 7/7 passed
Second consecutive run (idempotent seed) 7/7 passed (INSERT 0 0, DELETE 1 + INSERT 0 1 on candle refresh only)
Indexer down (VITE_INDEXER_URL=http://127.0.0.1:39999) Fail-fast with actionable message (exit 1, no Vitest 404)
grep + make help Makefile, docs/testing.md, skills/AGENTS_TESTING_P2_EPIC.md reference target
Factory pair resolution for #166 Resolved (terra146ypndz… from factory terra1hrpna9v7…)

Harness fixes from da335fa (90-day candle seed window, 127.0.0.1 LCD defaults, factory env, sqlx docker fallback) confirmed in this environment.

Closing as verified.

## Verification complete (agent) Re-ran full **#205** checklist with infra healthy on `main` (`543ca70`). | Check | Result | |-------|--------| | `make test-charts-integration` (Postgres + indexer `:3001` + LocalTerra LCD) | **7/7 passed** | | Second consecutive run (idempotent seed) | **7/7 passed** (`INSERT 0 0`, `DELETE 1` + `INSERT 0 1` on candle refresh only) | | Indexer down (`VITE_INDEXER_URL=http://127.0.0.1:39999`) | **Fail-fast** with actionable message (exit 1, no Vitest 404) | | `grep` + `make help` | **Makefile**, `docs/testing.md`, `skills/AGENTS_TESTING_P2_EPIC.md` reference target | | Factory pair resolution for #166 | **Resolved** (`terra146ypndz…` from factory `terra1hrpna9v7…`) | Harness fixes from **da335fa** (90-day candle seed window, `127.0.0.1` LCD defaults, factory env, sqlx docker fallback) confirmed in this environment. Closing as verified.
PlasticDigits (Migrated from gitlab.com) closed this issue 2026-05-29 14:36:27 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
code/cl8y-dex-terraclassic#205
No description provided.