Labels
No labels
agent:fix_bugfix
agent:fix_conflicts
agent:fix_security
agent:gap_analysis
agent:implement
agent:implement
agent:implement
agent:open_issues
agent:ready
agent:research
agent:security_audit
agent:verify
architecture
backend
blocker:hybrid
blocker:launch
blocker:limit-orders
blocker:v2
block:log_only
block:security
bug
ci
contracts
correctness
deploy
dev
devops
docs
documentation
duplicate
e2e
enhancement
epic
feature
frontend
functional-completion
gas
good first issue
governance
help wanted
high-risk
hooks
hybrid
indexer
infra
infrastructure
integrators
invalid
launch-blocker
limit-orders
localnet
localterra
low priority
missing-implementation
needs-design
ops
performance
priority
high
priority
medium
product
qa
QA
question
ready
ready
research
scripts
security
security-hardening
smartcontracts
tech-debt
testing
ux
UX
v2
verification
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
code/cl8y-dex-terraclassic#335
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Follow-up from MR !809 (#324): enable Postgres +
indexer/.envprovisioning on Cursor Cloud Agent VMs so indexer integration tests (cargo test --tests, especiallyapi_route_solve) can run without a full LocalTerra wasm deploy. MR !809 verified #324 via lib tests only and explicitly SKIPped integration coverage because Postgres /indexer/.envwere unavailable in the Cloud Agent environment.This issue covers the infra + automation gap, not re-implementing #324 solver logic (already merged).
Current codebase
Postgres stack (Docker Compose)
docker-compose.yml—postgresservice on127.0.0.1:5432, default usercl8y_legal/ passwordcl8y_legal, DBdex_indexer, init scripts indocker/postgres-init/.scripts/lib/postgres-dev.env—DATABASE_URL,TEST_DATABASE_URL,POSTGRES_SUPERUSERbootstrap vars (#245).Database bootstrap +
indexer/.envsyncscripts/setup-postgres-dev-databases.sh— idempotent: bootstrapscl8y_legalvia superuser when missing, createsdex_indexer+dex_indexer_test, upsertsDATABASE_URL/TEST_DATABASE_URLintoindexer/.envviascripts/lib/upsert-dotenv.sh. Requires hostpsql; if absent, warns and only writes env URLs (cannot create DBs).scripts/lib/postgres-bootstrap-role.sh— superuser bootstrap helper.Makefile—make startruns compose + best-effort setup script;make wait-healthywaits for Postgres and re-runs setup;make test-setup-postgresruns static + optional live Docker bootstrap regression.Full deploy path (writes complete
indexer/.env)scripts/deploy-dex-local.shphase 6.2 — runs setup script then overwritesindexer/.envwithDATABASE_URL,TEST_DATABASE_URL,FACTORY_ADDRESS,ROUTER_ADDRESS,LCD_URLS,CORS_ORIGINS, etc.indexer/.env.example— documents required vars;indexer/.envis gitignored.Cloud Agent full-stack setup (heavyweight)
scripts/setup-cloud-agent-localterra.sh/make setup-cloud-localterra— starts dockerd,make start+wait-healthy, optionalbuild-optimized(~10–15 min),deploy-dex-local.sh, starts indexer in tmux. Does writeindexer/.env, but only after full deploy (or deploy skip when stamp matches).AGENTS.md§ LocalTerra documents this path; does not document a lightweight Postgres-only path for integration tests.Integration test expectations
indexer/tests/common/mod.rs—dotenvy::dotenv()loadsindexer/.env;setup_pool()connects toTEST_DATABASE_URL(fallback hardcodedcl8y_legal@127.0.0.1:5432/dex_indexer_test); runssqlx::migrate!(); exclusive file lock at/tmp/cl8y-dex-indexer-test.seed.lockfor shared DB.indexer/tests/including:indexer/tests/api_route_solve.rs— route solve POST/GET, hybrid merge, #283/#324 cache tier isolation (route_solve_get_cache_tier_isolation,route_solve_get_cache_same_tier_reuses_lcd), 3-hop hybrid (#192).indexer/tests/api_route_solve_db_hybrid.rs,indexer/tests/security.rs,indexer/tests/db_orderbook_mirror.rs, etc.cargo test --tests -j 1 -- --test-threads=1(docs/testing.md§ Shared Postgres).MR !809 / #324 verification gap
api_route_solveintegration → SKIP ("Requires Postgres (indexer/.env); not run in Cloud Agent VM").indexer/src/api/best_execution.rs(concurrent_solve_tests); HTTP-level cache/concurrency invariants remain in integration tests only.scripts/qa/verify-issue-245.shrunscargo test --test api_route_solvebut assumes Postgres +indexer/.envalready exist; no Cloud Agent–specific bootstrap step.CI / reference automation
.github/workflows/test.ymlreference Indexer job uses ephemeralpostgres:16service withpostgres:postgres(notcl8y_legal) and runscargo test(lib + integration in default profile) — does not mirror local compose credentials orindexer/.envdotenv path..gitlab-ci.yml— builds release binary only; no Postgres service or integration test job.Why this is needed
route_solve_get_cache_*) or route-solve response shape would not be caught by lib tests alone.cargo test --test api_route_solve. Today the only documented Cloud path is full-stack setup.indexer/.envis gitignored and incomplete without automation. Fresh Cloud Agent checkouts haveindexer/.env.examplebut noindexer/.env. Integration tests load dotenv from CWD; missing file → fallback URL may fail if Postgres/role/DB not provisioned.setup-postgres-dev-databases.shrequirespsqlon the host; Cloud Agent VMs may lackpostgresql-client, causing silent skip of DB creation while still writing URLs — tests then fail at connect/migrate with opaque panics.skills/AGENTS_LOCAL_POSTGRES_DEV.mddocuments local dev well but not a minimal Cloud Agent bootstrap (Postgres-only, no chain).Constraints and guardrails
route_solve_get_cache_tier_isolationandroute_solve_get_cache_same_tier_reuses_lcdmust remain the authority for tier isolation vs same-tier sharing; do not weaken assertions to make setup "pass."cl8y_legalas the app role — do not revert topostgres:postgresas the default app user (scripts/lib/postgres-dev.env, #245).-j 1 -- --test-threads=1; do not enable parallel integration runs against onedex_indexer_test.--fresh).indexer/.envstays gitignored; scripts write it locally only.sg docker -c '…'(AGENTS.md); new scripts must follow existing patterns inscripts/setup-cloud-agent-localterra.sh.export PATH="/usr/local/cargo/bin:$PATH"in tmux sessions).Relevant files
docker-compose.yml,docker/postgres-init/scripts/lib/postgres-dev.env,scripts/lib/postgres-bootstrap-role.sh,scripts/lib/upsert-dotenv.sh,scripts/setup-postgres-dev-databases.sh,scripts/test-setup-postgres-dev-databases.shscripts/setup-cloud-agent-localterra.sh,AGENTS.md,Makefile(setup-cloud-localterra,test-setup-postgres,wait-healthy)scripts/deploy-dex-local.shindexer/.env.example,indexer/.env(gitignored, created by scripts)indexer/tests/common/mod.rs,indexer/tests/api_route_solve.rs,indexer/tests/api_route_solve_db_hybrid.rsscripts/qa/verify-issue-245.sh(pattern for newverify-issue-324or generic target)docs/testing.md,skills/AGENTS_LOCAL_POSTGRES_DEV.md,skills/AGENTS_HYBRID_QUOTING.md,skills/AGENTS_INDEXER_HYBRID_BEST_EXECUTION.md.github/workflows/test.yml(indexer job),.gitlab-ci.ymlRecommended direction
scripts/setup-cloud-agent-indexer-postgres.shand/ormake setup-indexer-postgresthat:_ensure_dockerdpattern).docker compose up -d postgres(not LocalTerra).pg_isreadyvia host ordocker compose exec).indexer/.envsync (DATABASE_URL,TEST_DATABASE_URLminimum).setup-postgres-dev-databases.shfor Cloud VMs — when hostpsqlis missing, fall back todocker compose exec -T postgres psql …(same asmake wait-healthypartially does forpg_isready). Fail loudly if DBs cannot be created.--postgres-onlyonscripts/setup-cloud-agent-localterra.shto run step 1 without wasm build/deploy.make setup-indexer-postgres— bootstrap only.make test-indexer-integration— depends on setup, runscd indexer && cargo test --tests -j 1 -- --test-threads=1.make verify-issue-324— lib concurrent/cache tests +api_route_solvecache tier tests (subset covering MR !809 integration gap).AGENTS.md§ new subsection "Indexer integration tests (Postgres-only)" pointing to the lightweight path; cross-link inskills/AGENTS_LOCAL_POSTGRES_DEV.md.scripts/test-setup-postgres-dev-databases.sh— static assert that cloud bootstrap script exists and is referenced from AGENTS.md; optional live test withSKIP_LIVEoff on a VM with Docker..github/workflows/test.ymlindexer job withcl8y_legal+dex_indexer_testor document intentional divergence.Acceptance criteria
indexer/.envwith validDATABASE_URLandTEST_DATABASE_URLwithoutbuild-optimized/deploy-dex-local.cd indexer && cargo test --test api_route_solve -- --test-threads=1passes after the bootstrap on a fresh Cloud Agent VM (with Docker + Rust already provisioned per AGENTS.md).make test-indexer-integration(or equivalent) passes the full integration suite serialized (--tests -j 1 -- --test-threads=1).psqlis absent but Docker is available, DB creation still succeeds via compose exec fallback.AGENTS.mddocuments the Postgres-only path and distinguishes it frommake setup-cloud-localterra.make test-setup-postgresstatic checks cover the new automation.make verify-issue-324runs lib #324 tests +api_route_solvecache tier integration tests and is referenced from MR !809 verification docs /skills/AGENTS_HYBRID_QUOTING.md.Test plan (functional paths)
indexer/.envindexer/.envcreated with both DB URLs;dex_indexer+dex_indexer_testexistdocker compose up -d postgresvia scriptcargo test --test api_health -- --test-threads=1greencargo test --test api_route_solve route_solve_get_cache -- --test-threads=1greenapi_route_solveapi_route_solvebinary green serializedcargo test --tests -j 1 -- --test-threads=1greenpsqlpostgres:postgres)cl8y_legalcargo test --libstill works without bootstrapmake test-setup-postgresTest plan (attack, hack, and abuse vectors)
route_solve_get_cache_tier_isolationafter bootstraproute_solve_get_cache_same_tier_reuses_lcdcargo test --testswithout-j 1indexer/.envgitignored; not committedpostgres-bootstrap-role.shidempotent)TEST_DATABASE_URLin env127.0.0.1:5432only (docker-compose.yml)indexer/tests/common/lcd_mock.rs); no live LCD amplification in default integration pathVerification criteria
Third parties (and Cloud Agents closing this issue) should be able to run:
Done when: MR !809-style verification no longer lists
api_route_solveas SKIP on Cloud Agent VMs; AGENTS.md documents the path; acceptance criteria checkboxes satisfied.Related
cl8y_legalbootstrapskills/AGENTS_LOCAL_POSTGRES_DEV.md— local Postgres playbookmentioned in commit
caeaa9389ementioned in merge request !818
Implemented in !818.
Summary: Lightweight Cloud Agent Postgres bootstrap (
make setup-indexer-postgres) provisions compose postgres +indexer/.envwithout wasm deploy. Hostpsqloptional via compose exec fallback.Verification (this VM):
make setup-indexer-postgres— PASS (idempotent)make verify-issue-324— PASScargo test --test api_route_solve— PASS (23/23)make test-setup-postgres— PASSmake test-indexer-integration— FAIL at pre-existingindexer_ingestion_hardening(4 tests; same on origin/main)Issue stays open until MR merges. Removing agent labels.
mentioned in commit
a4127b646c#335 — verified the bootstrap + automation + docs on current main (merged !818, commits
caeaa93/a4127b6). 7 of the 8 acceptance items hold; the 8th (full integration suite green) is blocked by a pre-existing, unrelated test failure that I root-caused — details below.What passes:
make setup-indexer-postgresprovisions compose postgres + syncs indexer/.env with DATABASE_URL + TEST_DATABASE_URL, no build-optimized / deploy-dex-local. Non-destructive — it upserts the DB URLs and left the rest of indexer/.env intact.--postgres-only, distinct from setup-cloud-localterra.The one that doesn't (acceptance #3 / make test-indexer-integration full suite):
So #335's actual goal — Postgres-only integration testing without a wasm deploy — is achieved, and it surfaced exactly the kind of pre-existing gap it was built to expose (these integration tests were SKIP on the cloud agent that did the #292 work). The indexer_ingestion_hardening failure is its own test-debt fix (update the 3 stale wiremock matchers to the #292 query=/page/limit format), not in #335's scope.
@PlasticDigits — #335 itself is good on 7/8; the full-suite criterion is blocked only by the stale-mock test debt above. Want me to ship the test-only mock fix (small separate MR) so the suite goes green, or file it as its own issue? Your call.
Fix the mock as part of this issue with new mr
mentioned in commit
fec80e46edmentioned in merge request !823
Implemented the remaining test-debt fix requested in the last comment — !823.
Change:
swap_events_hybrid_columnsintegration fixture now uses_contract_address(wascontract_address), matching #285 parser scoping. Bootstrap infra from !818 was already onmain; ingestion hardening wiremocks were aligned inde3b956(#292).Acceptance checklist
make setup-indexer-postgresapi_route_solveafter bootstrapcd indexer && cargo test --test api_route_solve -- --test-threads=1make test-indexer-integrationmake setup-indexer-postgres×2psqlabsent → compose exec fallbackPOSTGRES_PSQL_MODE=composeAGENTS.mdPostgres-only pathmake test-setup-postgresmake test-setup-postgresmake verify-issue-324make verify-issue-324Issue stays open until !823 merges.
mentioned in commit
593ab1353f"## Verification \u2014 issue #335 (agent:verify)\n\nVerified on Cursor Cloud Agent VM against current
main(includes !818 bootstrap + !823 ingestion mock fix).\n\nIssue: https://gitlab.com/PlasticDigits/cl8y-dex-terraclassic/-/work_items/335\n\n### Acceptance criteria\n\n| # | Criterion | Command / check | Result |\n|---|-----------|-----------------|--------|\n| 1 | One-command Postgres bootstrap (no wasm deploy) | Removedindexer/.env, ranmake setup-indexer-postgres| PASS \u2014 composepostgresonly (LocalTerra not started);indexer/.envcreated withDATABASE_URL+TEST_DATABASE_URL(cl8y_legal@127.0.0.1:5432) |\n| 2 |api_route_solveafter bootstrap |cd indexer && cargo test --test api_route_solve -- --test-threads=1| PASS \u2014 23/23 |\n| 3 | Full integration suite serialized |make test-indexer-integration| PASS \u2014 all binaries green (cargo test --tests -j 1 -- --test-threads=1, ~163s) |\n| 4 | Bootstrap idempotent |make setup-indexer-postgres\u00d72 | PASS \u2014 second run exit 0, DBs already exist |\n| 5 | Hostpsqlabsent \u2192 compose exec fallback |command -v psql\u2192 absent; bootstrap log | PASS \u2014[setup-postgres] using docker compose exec psql (host psql not installed)|\n| 6 |AGENTS.mdPostgres-only path documented | reviewAGENTS.md\u00a7 Indexer integration tests +skills/AGENTS_LOCAL_POSTGRES_DEV.md| PASS \u2014 distinct frommake setup-cloud-localterra;--postgres-onlyon cloud script |\n| 7 |make test-setup-postgresstatic gate |make test-setup-postgres| PASS \u2014OK: setup-postgres static checks|\n| 8 |make verify-issue-324|make verify-issue-324| PASS \u2014 3/3 (libconcurrent_solve,hybrid_cache_key, integrationroute_solve_get_cache) |\n\n### Additional functional checks\n\n| Scenario | Result |\n|----------|--------|\n|api_healthsmoke after bootstrap | PASS \u2014 1/1 |\n|make verify-issue-324cache tier tests (route_solve_get_cache_*) | PASS \u2014 tier isolation + same-tier reuse (A1/A2) |\n|indexer_ingestion_hardening(previously blocking full suite) | PASS \u2014 6/6 |\n| Lib-only path without Postgres | PASS \u2014cargo test --lib131/131 withindexer/.envtemporarily removed |\n| Postgres port binding (A7) | PASS \u2014127.0.0.1:5432indocker-compose.yml|\n|indexer/.envnot committed (A4) | PASS \u2014 gitignored (.gitignore:2:.env) |\n|--postgres-onlyonsetup-cloud-agent-localterra.sh| PASS |\n\n### Notes\n\n- MR !809 / #324 integration gap is closed:api_route_solvecache-tier HTTP tests run on Cloud Agent VMs without wasm deploy.\n- No repository changes were required during this verification pass.\n\nClosing as verified.\n"Verification — issue #335 (agent:verify)
Verified on Cursor Cloud Agent VM against current
main(includes !818 bootstrap + !823 ingestion mock fix).Issue: https://gitlab.com/PlasticDigits/cl8y-dex-terraclassic/-/work_items/335
Acceptance criteria
indexer/.env, ranmake setup-indexer-postgrespostgresonly (LocalTerra not started);indexer/.envcreated withDATABASE_URL+TEST_DATABASE_URL(cl8y_legal@127.0.0.1:5432)api_route_solveafter bootstrapcd indexer && cargo test --test api_route_solve -- --test-threads=1make test-indexer-integrationcargo test --tests -j 1 -- --test-threads=1, ~163s)make setup-indexer-postgres×2psqlabsent → compose exec fallbackcommand -v psql→ absent; bootstrap log[setup-postgres] using docker compose exec psql (host psql not installed)AGENTS.mdPostgres-only path documentedAGENTS.md§ Indexer integration tests +skills/AGENTS_LOCAL_POSTGRES_DEV.mdmake setup-cloud-localterra;--postgres-onlyon cloud scriptmake test-setup-postgresstatic gatemake test-setup-postgresOK: setup-postgres static checksmake verify-issue-324make verify-issue-324concurrent_solve,hybrid_cache_key, integrationroute_solve_get_cache)Additional functional checks
api_healthsmoke after bootstrapmake verify-issue-324cache tier tests (route_solve_get_cache_*)indexer_ingestion_hardening(previously blocking full suite)cargo test --lib131/131 withindexer/.envtemporarily removed127.0.0.1:5432indocker-compose.ymlindexer/.envnot committed (A4).gitignore:2:.env)--postgres-onlyonsetup-cloud-agent-localterra.shNotes
api_route_solvecache-tier HTTP tests run on Cloud Agent VMs without wasm deploy.Closing as verified.
mentioned in commit
c0baf17f46mentioned in commit
c99c7d335ementioned in commit
c9fa5ad7damentioned in commit
f090a17315mentioned in commit
d50d39a3edmentioned in commit
59c1013758mentioned in commit
59c1013758mentioned in commit
15bfbff71amentioned in commit
4ebf10632d