E2E: Solana tests fail — placeholder program ID, bridge not initialized, airdrop fails #70

Closed
opened 2026-03-24 06:13:23 +00:00 by Brouie · 13 comments
Brouie commented 2026-03-24 06:13:23 +00:00 (Migrated from gitlab.com)

make solana-test-e2e fails 5/6 (only validator running check passes).

Three root causes:

  1. E2E tests hardcode placeholder program ID CL8YBr1dg3So1ana111111111111111111111111111 instead of actual deployed ID CNWdzCX1a2YHZ3iYn9mrAPWXFZWqC5kJn6mdoCFsfLsC. test_solana_bridge_program_exists fails with AccountNotFound.

  2. Bridge not initialized on docker Solana validator — setup-bridge.sh outputs 'Solana Program: (not set)' and doesn't run any Solana initialization. Bridge PDA Fnuht9t not found.

  3. Airdrop test expects 1 SOL but gets 0 — docker validator faucet may not be configured or test account not funded.

Environment: make start + make deploy on QA server. Anvil + LocalTerra + Solana (v1.18.26) + Postgres all healthy. EVM and Terra deployed and configured. Solana programs deployed (cl8y_bridge + cl8y_faucet).

make solana-test-e2e fails 5/6 (only validator running check passes). Three root causes: 1. E2E tests hardcode placeholder program ID CL8YBr1dg3So1ana111111111111111111111111111 instead of actual deployed ID CNWdzCX1a2YHZ3iYn9mrAPWXFZWqC5kJn6mdoCFsfLsC. test_solana_bridge_program_exists fails with AccountNotFound. 2. Bridge not initialized on docker Solana validator — setup-bridge.sh outputs 'Solana Program: (not set)' and doesn't run any Solana initialization. Bridge PDA Fnuht9t not found. 3. Airdrop test expects 1 SOL but gets 0 — docker validator faucet may not be configured or test account not funded. Environment: make start + make deploy on QA server. Anvil + LocalTerra + Solana (v1.18.26) + Postgres all healthy. EVM and Terra deployed and configured. Solana programs deployed (cl8y_bridge + cl8y_faucet).
PlasticDigits commented 2026-03-24 06:38:43 +00:00 (Migrated from gitlab.com)

Pushed follow-up on feat/solana-integration (local deploy env + QA console hints when EVM/Terra/forge/LocalTerra fails).

@brouie could you review when you have a moment? Leaving this issue open until you confirm Solana E2E on QA.

Pushed follow-up on `feat/solana-integration` (local deploy env + QA console hints when EVM/Terra/forge/LocalTerra fails). @brouie could you review when you have a moment? Leaving this issue open until you confirm Solana E2E on QA.
Brouie commented 2026-03-24 06:49:00 +00:00 (Migrated from gitlab.com)

Partial progress after latest pull: 2/6 pass (up from 1/6). Bridge PDA now exists (104 bytes). Remaining 4 fail because test accounts have no SOL on docker validator — deposit_native fails with 'Attempt to debit an account but found no record of a prior credit'. Need airdrop to test keypair on docker Solana validator.

Partial progress after latest pull: 2/6 pass (up from 1/6). Bridge PDA now exists (104 bytes). Remaining 4 fail because test accounts have no SOL on docker validator — deposit_native fails with 'Attempt to debit an account but found no record of a prior credit'. Need airdrop to test keypair on docker Solana validator.
PlasticDigits commented 2026-03-24 07:57:36 +00:00 (Migrated from gitlab.com)

@brouie Could you review when you have a moment?

Pushed fixes on feat/solana-integration (52636b8) aimed at the remaining Solana E2E failures (test accounts not funded on the Docker validator):

  • docker-compose: solana-test-validator now binds 0.0.0.0 inside the container so RPC + faucet/airdrop work through Docker port mapping (host still published on 127.0.0.1 only).
  • make solana-test-e2e / test_solana_flows.rs: SOLANA_RPC_URL support, plus airdrop_and_confirm() with retries and balance polling so airdrops are reliable against a slow validator.

Please re-run make solana-test-e2e on QA after make start + deploy/init and confirm whether all six ignored tests pass. Thanks!

@brouie Could you review when you have a moment? Pushed fixes on `feat/solana-integration` (52636b8) aimed at the remaining Solana E2E failures (test accounts not funded on the Docker validator): - **docker-compose:** `solana-test-validator` now binds `0.0.0.0` inside the container so RPC + faucet/airdrop work through Docker port mapping (host still published on 127.0.0.1 only). - **`make solana-test-e2e` / `test_solana_flows.rs`:** `SOLANA_RPC_URL` support, plus `airdrop_and_confirm()` with retries and balance polling so airdrops are reliable against a slow validator. Please re-run `make solana-test-e2e` on QA after `make start` + deploy/init and confirm whether all six ignored tests pass. Thanks!
Brouie commented 2026-03-24 08:05:32 +00:00 (Migrated from gitlab.com)

4/6 E2E pass now (up from 1/6). airdrop + bridge_exists + deposit_flow + validator all pass. 2 remaining: (1) cancel_flow needs SOLANA_OPERATOR_KEYPAIR matching bridge operator C4kB9S... — bridge operator doesn't match ANCHOR_WALLET. (2) evm_to_solana_flow still failing — need details.

4/6 E2E pass now (up from 1/6). airdrop + bridge_exists + deposit_flow + validator all pass. 2 remaining: (1) cancel_flow needs SOLANA_OPERATOR_KEYPAIR matching bridge operator C4kB9S... — bridge operator doesn't match ANCHOR_WALLET. (2) evm_to_solana_flow still failing — need details.
Brouie commented 2026-03-24 08:07:32 +00:00 (Migrated from gitlab.com)

Both remaining failures (evm_to_solana + cancel_flow) are same root cause: bridge operator is C4kB9S4idFz1aXdc2dMeeo9H52uVdFkiupCMTKoTxNa3 but ANCHOR_WALLET is ~/.config/solana/id.json (12TChu...). Need either SOLANA_OPERATOR_KEYPAIR set or bridge re-initialized with matching operator.

Both remaining failures (evm_to_solana + cancel_flow) are same root cause: bridge operator is C4kB9S4idFz1aXdc2dMeeo9H52uVdFkiupCMTKoTxNa3 but ANCHOR_WALLET is ~/.config/solana/id.json (12TChu...). Need either SOLANA_OPERATOR_KEYPAIR set or bridge re-initialized with matching operator.
PlasticDigits commented 2026-03-24 09:06:34 +00:00 (Migrated from gitlab.com)

mentioned in commit da14cbff8e

mentioned in commit da14cbff8e2fdf23f6c6773bb70037decc92b4fd
PlasticDigits commented 2026-03-24 09:06:42 +00:00 (Migrated from gitlab.com)

Fixed in commit da14cbf on feat/solana-integration. All three root causes addressed:

  1. Program ID persistence — deploy-solana now saves SOLANA_PROGRAM_ID to .deploy/local.env (new write_deploy_env_solana() in lib-local-deploy-env.sh). Both setup-bridge and solana-test-e2e load it automatically, so the placeholder ID is no longer used when a real deploy keypair exists.

  2. Bridge initialization — setup-bridge.sh now calls initialize-bridge.sh (idempotent) before chain registration, and funds the admin wallet with SOL first. This creates the bridge PDA that was missing.

  3. SOL via setup, not faucet — cl8y_faucet is for test SPL tokens only. SOL is now distributed by setup-bridge.sh via solana airdrop CLI. E2E tests use fund_from_admin() (system transfer from pre-funded admin wallet) instead of request_airdrop.

@Brouie — please review when you get a chance. You can test with make deploy && make solana-test-e2e on QA.

Fixed in commit da14cbf on `feat/solana-integration`. All three root causes addressed: 1. **Program ID persistence** — `deploy-solana` now saves `SOLANA_PROGRAM_ID` to `.deploy/local.env` (new `write_deploy_env_solana()` in `lib-local-deploy-env.sh`). Both `setup-bridge` and `solana-test-e2e` load it automatically, so the placeholder ID is no longer used when a real deploy keypair exists. 2. **Bridge initialization** — `setup-bridge.sh` now calls `initialize-bridge.sh` (idempotent) *before* chain registration, and funds the admin wallet with SOL first. This creates the bridge PDA that was missing. 3. **SOL via setup, not faucet** — `cl8y_faucet` is for test SPL tokens only. SOL is now distributed by `setup-bridge.sh` via `solana airdrop` CLI. E2E tests use `fund_from_admin()` (system transfer from pre-funded admin wallet) instead of `request_airdrop`. @Brouie — please review when you get a chance. You can test with `make deploy && make solana-test-e2e` on QA.
Brouie commented 2026-03-25 03:27:50 +00:00 (Migrated from gitlab.com)

mentioned in issue #67

mentioned in issue #67
PlasticDigits commented 2026-03-25 04:03:03 +00:00 (Migrated from gitlab.com)

@brouie — pushed fix in commit a60944e on feat/solana-integration addressing the last 2 failing E2E tests (evm_to_solana_flow + cancel_flow).

Root cause: setupTest() in bridge.test.ts generated a random operator keypair on every run. When setup-bridge.sh initialized the bridge via ts-mocha, the on-chain operator ended up as a random pubkey whose private key was lost after the process exited. The Rust E2E tests then couldn't find a matching local keypair to sign as operator.

Fix: setupTest() now reads SOLANA_OPERATOR_KEYPAIR env var (keypair file path) when set, and setup-bridge.sh / initialize-bridge.sh pass SOLANA_OPERATOR_KEYPAIR=$SOLANA_KEYPAIR so operator = admin = the known local wallet. Also replaced request_airdrop with fund_from_admin in canceler_solana_destination.rs for consistency.

Could you re-run make deploy && make solana-test-e2e on QA and confirm all 6 pass? Thanks!

@brouie — pushed fix in commit a60944e on `feat/solana-integration` addressing the last 2 failing E2E tests (evm_to_solana_flow + cancel_flow). **Root cause:** `setupTest()` in `bridge.test.ts` generated a random operator keypair on every run. When `setup-bridge.sh` initialized the bridge via `ts-mocha`, the on-chain operator ended up as a random pubkey whose private key was lost after the process exited. The Rust E2E tests then couldn't find a matching local keypair to sign as operator. **Fix:** `setupTest()` now reads `SOLANA_OPERATOR_KEYPAIR` env var (keypair file path) when set, and `setup-bridge.sh` / `initialize-bridge.sh` pass `SOLANA_OPERATOR_KEYPAIR=$SOLANA_KEYPAIR` so operator = admin = the known local wallet. Also replaced `request_airdrop` with `fund_from_admin` in `canceler_solana_destination.rs` for consistency. Could you re-run `make deploy && make solana-test-e2e` on QA and confirm all 6 pass? Thanks!
Brouie commented 2026-03-25 05:33:26 +00:00 (Migrated from gitlab.com)

Pulled a60944e + f5dec31. Attempted make deploy but LocalTerra container exits immediately (logs show help text instead of startup). EVM + Solana deploy fine but bridge setup incomplete without Terra. Can't verify 6/6 E2E until LocalTerra is fixed. Will retest once resolved.

Pulled a60944e + f5dec31. Attempted make deploy but LocalTerra container exits immediately (logs show help text instead of startup). EVM + Solana deploy fine but bridge setup incomplete without Terra. Can't verify 6/6 E2E until LocalTerra is fixed. Will retest once resolved.
Brouie commented 2026-03-25 07:47:30 +00:00 (Migrated from gitlab.com)

LocalTerra fixed (stale volume — wiped and restarted). Full stack deployed. E2E results: 4/6 pass, 2/6 fail.

Passed: test_solana_bridge_program_exists, test_solana_admin_funded, test_solana_validator_running, test_solana_cancel_flow

Failed:

  1. test_solana_to_evm_deposit_flow — InstructionError Custom(2006): 'EVM chain must be registered on Solana'. Chain registration may be incomplete in setup-bridge.sh.
  2. test_evm_to_solana_flow — InstructionError Custom(6024) NotNativeToken: 'Withdrawal token must be native SOL (Pubkey::default) for native execution' at withdraw_execute_native.rs:53. E2E test may be passing wrong token mint for native withdrawal.
LocalTerra fixed (stale volume — wiped and restarted). Full stack deployed. E2E results: 4/6 pass, 2/6 fail. Passed: test_solana_bridge_program_exists, test_solana_admin_funded, test_solana_validator_running, test_solana_cancel_flow Failed: 1. test_solana_to_evm_deposit_flow — InstructionError Custom(2006): 'EVM chain must be registered on Solana'. Chain registration may be incomplete in setup-bridge.sh. 2. test_evm_to_solana_flow — InstructionError Custom(6024) NotNativeToken: 'Withdrawal token must be native SOL (Pubkey::default) for native execution' at withdraw_execute_native.rs:53. E2E test may be passing wrong token mint for native withdrawal.
PlasticDigits commented 2026-03-25 08:23:41 +00:00 (Migrated from gitlab.com)

@brouie Please pull feat/solana-integration (commit e357940) or check out tag review/brouie-solana-e2e-70, then on QA run:

make deploy && make solana-test-e2e

This batch fixes native-token handling in test_evm_to_solana_flow, runs Solana E2E tests with --test-threads=1 to avoid deposit nonce races (Anchor 2006), and tightens setup-bridge.sh mocha grep to registers a chain only. Please confirm all six ignored tests pass. Thanks!

@brouie Please pull `feat/solana-integration` (commit e357940) or check out tag `review/brouie-solana-e2e-70`, then on QA run: ``` make deploy && make solana-test-e2e ``` This batch fixes native-token handling in `test_evm_to_solana_flow`, runs Solana E2E tests with `--test-threads=1` to avoid deposit nonce races (Anchor 2006), and tightens `setup-bridge.sh` mocha grep to `registers a chain` only. Please confirm all six ignored tests pass. Thanks!
Brouie commented 2026-03-25 08:42:53 +00:00 (Migrated from gitlab.com)

6/6 E2E ALL PASS after dev fix (dest_token all-zeros for native SOL + --test-threads=1). LocalTerra fix was stale volume — wiped and restarted. Full stack: Anvil x2 + LocalTerra + Solana + Postgres all healthy. Closing.

6/6 E2E ALL PASS after dev fix (dest_token all-zeros for native SOL + --test-threads=1). LocalTerra fix was stale volume — wiped and restarted. Full stack: Anvil x2 + LocalTerra + Solana + Postgres all healthy. Closing.
Brouie (Migrated from gitlab.com) closed this issue 2026-03-25 08:42:55 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
code/cl8y-bridge-monorepo#70
No description provided.