fix(operator): dest-approved Terra→EVM withdrawals stay unexecuted #170
Labels
No labels
agent:implement
agent:ready
backend
bug
cannot-reproduce
confirmed
desktop
docs
documentation
duplicate
enhancement
feature
frontend
good first issue
help wanted
high-risk
in-review
invalid
mobile
needs-triage
P0-critical
P1-high
P2-medium
P3-low
qa
QA
question
ready
report
responsive
security
security-escalate
smart-contract
solana
tablet
test-pass
ux
wallet-issue
wallet:keplr
wallet:metamask
wallet:station
wallet:walletconnect
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
code/cl8y-bridge-monorepo#170
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Hash Verification (
/verify?hash=) for a Terra Classic → BNB economic-token transfer shows overall Pending, Hash matches, source deposit present, and destination withdraw Approved (not Executed, not Canceled). That aggregate badge is the intended mapping for “approved, awaiting execution.” The defect is that execution never completes after the cancel window, and the verify surface gives the user no EVM-side reason.Bundle (do not split):
withdrawExecuteMint/withdrawExecuteUnlockdest-approved EVM withdrawals once the on-chain cancel window has elapsed (Terra→EVM uses the same EVM writer as EVM→EVM).CancelWindowActive, RPC) must not be treated as terminal and dropped from the in-memory execute queue.This is not a request to weaken on-chain withdraw caps. Manual unstick of one live row stays off this ticket (ops bot).
Related but not this ticket: #164 (MegaETH/LCD stalling approval /
getDeposit), #127 (Transfer Status 4/4 UX when rate-limited), #139 (Terra pending-history pagination), closed #47 (Monitor list always Pending), closed #100 (BelowMin after fee — different classifier).Current codebase
Hash-query status (correct for this screenshot shape)
packages/frontend/src/hooks/useHashVerification.tssetsstatustoverifiedonly whendest.executed,canceledwhendest.cancelled, andpendingwhendest.approved(“approved, awaiting execution”).packages/frontend/src/components/verify/DestHashCard.tsxlabels dest state Executed / Canceled / Approved / Pending from those flags.packages/frontend/src/pages/HashVerificationPage.tsxqueries all configured chains, shows source vs dest cards, and records monitor results.showRateLimitInfoisdestChain.type === 'cosmos'only. Solana dest getsSolanaRecipientExecutePanel+useApprovalCountdown. EVM dest has neither rate-limit hook nor execute CTA on this page.packages/frontend/src/hooks/useEvmExecutionRateLimitStatus.ts+packages/frontend/src/services/evmExecutionRateLimit.tsalready computetemporarily-blocked/permanently-blockedfor Transfer Status (GL-127).HashComparisonPanelalready renders those banners from aterraRateLimitStatusprop — Verify never passes EVM dest into that path.Operator execute after approve
V2 dest lifecycle: deposit (source) →
WithdrawSubmit(dest) →WithdrawApprove→ cancel window →withdrawExecuteMint/withdrawExecuteUnlock. EVM contract comment: after the window, anyone may execute.packages/operator/src/writers/evm.rs::process_pendingrunsprocess_pending_executions, thenenumerate_and_approve, then execute again (INV-OP-W11 restart recovery).approved_hashesbut still on the pending set, it re-queues execute with delay 0. IfgetPendingWithdrawsaysapprovedand not executed/cancelled, it enqueues withdelay_from_onchain_approved_at.enqueue_execution_if_absentreturns immediately whenterminal_executionscontains the hash — no later retry even if a daily window later has headroom.process_pending_executionscallssubmit_execute_withdrawwhen the in-memory delay elapsed. Success oris_terminal_execute_errorboth move the hash intoterminal_executionsand out ofpending_executions.packages/operator/src/writers/mod.rs::is_terminal_execute_erroris terminal for already-executed / cancelled / missing /BelowMinPerTransaction. It is not terminal forCancelWindowActiveor period/per-tx rate-limit selectors — those should retry. If execute never enterspending_executions(writer livelock, single-RPCsubmit_execute_withdrawusing onlyself.rpc_url, process restart without re-enumeration), dest stays Approved forever.submit_execute_withdrawdoes not use the method-level RPC fallback used foreth_getLogs(#138). A send/receipt failure on the one URL is a retry or a silent stall depending on the error string.Frontend Transfer Status vs Verify
/transfer/:xchainHashId) is the stepper (GL-127 / #127) and can show rate-limit freeze on step 4/4.Why the new implementation is needed
terminal_executionsafter a retryable revert (or never enqueueing it) makes dest-Approved rows unrecoverable until a process restart that also missesterminal_executions— and restart still skips hashes already in that map.Constraints / guardrails
maxPerTransaction/maxPerPeriod, skipTokenRateLimit, or add an operator bypass that executes below-min or over-period amounts. Retry afterperiodEndsAtfor period blocks; keepBelowMinPerTransactionterminal (#100 / INV-OP-W11).CancelWindowActive. Zero delay afterapprovedAt + windowmay still revert once; that is retryable, not terminal.approved && !executed && !cancelled.computeEvmExecutionRateLimitStatus/useEvmExecutionRateLimitStatusand existingHashComparisonPanelbanners.Relevant files
packages/operator/src/writers/evm.rsprocess_pending,enqueue_execution_if_absent,process_pending_executions,submit_execute_withdrawpackages/operator/src/writers/mod.rsremaining_cancel_window_secs,is_terminal_execute_error, testspackages/operator/src/rpc_fallback.rspackages/frontend/src/hooks/useHashVerification.tspackages/frontend/src/pages/HashVerificationPage.tsxpackages/frontend/src/hooks/useEvmExecutionRateLimitStatus.tspackages/frontend/src/services/evmExecutionRateLimit.tspackages/frontend/src/components/verify/HashComparisonPanel.tsxpackages/frontend/src/components/verify/DestHashCard.tsxevm.rs/mod.rspackages/frontend/src/hooks/useHashVerification.integration.test.tsRecommended direction
approvedAt + cancelWindow: confirmgetPendingWithdraw(approved,!executed,!cancelled). If execute is not inpending_executions, fix enumeration re-queue. If it is queued butsubmit_execute_withdrawfails, classify the revert.attempts, and keep retryable errors inpending_executionswith backoff (period-end or short poll). Onlyis_terminal_execute_errormay enterterminal_executions. Period rate-limit must not be added to the terminal list.dest.approved && !dest.executed && destChain.type === 'evm', passuseEvmExecutionRateLimitStatusintoHashComparisonPanel(same banners as Terra). Show cancel-window remaining from on-chainapprovedAt+ destcancelWindow(do not hardcode 24h — #44). Keep Hash matches / Pending.Acceptance criteria
WithdrawSubmitApproved and cancel window elapsed, operatorwithdrawExecute*succeeds and destexecutedbecomes true without a manual contract call.CancelWindowActiveand period rate-limit reverts are retried; they are not stored interminal_executions.BelowMinPerTransaction/ cancelled / already-executed stay terminal; no retry storm; no execute of cancelled hashes.TokenRateLimit/ min-per-tx unchanged. No new admin execute-bypass.useHashVerification).Given / When / Then
Given a Terra Classic → EVM dest withdraw that is Approved, not Executed, not Canceled, source deposit verified, and
block.timestamp > approvedAt + cancelWindowWhen the EVM writer
process_pendingcycle runsThen the writer sends
withdrawExecuteMintorwithdrawExecuteUnlockfor that xchain hash and destexecutedis true (unless a non-terminal on-chain limiter is active, in which case it retries after the window and Verify names that limiter)Given the same hash queried on Hash Verification with dest Approved and not executed
When dest chain is EVM
Then overall status is Pending, dest state is Approved, and the page shows cancel remaining and/or EVM execution rate-limit status (reuse GL-127 classifier)
Test plan (functional paths)
CancelWindowActiveon first executeterminal_executionstemporarily-blocked; not terminalmaxPerPeriodpermanently-blocked; no cap changeBelowMinPerTransactionTest plan (attack, hack, and abuse)
enqueue_execution_if_absentmust not reset an in-flight timerTokenRateLimitVerification criteria
remaining_cancel_window_secs,is_terminal_execute_error(retryable vs terminal), re-queue of dest-approved hashes, no delay reset.useHashVerificationtests stay green./verify?hash=for a dest-approved Terra→BNB fixture after the window → execute lands or blocker is named; after execute, refresh shows verified.Out of scope
getDeposit/ Terra writer approval resilience (#164).PENDING_WITHDRAWShistory index (#139).First-pass model recommendation
Recommendation: grok-high
Rationale: Operator execute queue + terminal vs retryable revert classification + Hash Verification EVM wiring is more than three production files and crosses
packages/operatorwriters andpackages/frontendverify. It is wallet / bridge / 2-of-3 adjacent (founder-required): a wrong terminal drop or a premature execute skips the cancel window or rate limit. Verification is mixed (writer unit tests + Verify UI), not a single helper with a deterministic local fixture. Composer criteria fail on file/subsystem count, cross-cutting state, and human-required surface — not because the title is long.cl8y-agent-control: queued
implementjobb421e8bc-ada6-482d-a41f-fff888be4ed9(not executed; no Hetzner VM).PR #171 merged to
mainat2906caa(#171). Branchissue/170deleted on origin.Acceptance criteria (code review + local tests)
AC1–AC8 pass in tree: dest-approved EVM execute after cancel window with method-level RPC fallback;
CancelWindowActive/ period-full stay retryable (notterminal_executions); BelowMin / cancelled / already-executed stay terminal; Hash Verification keeps dest-Approved as Pending (not verified) and names EVM cancel remaining + GL-127 rate-limit banners. No contract / rate-limit bypass in the diff. #164 / #127 / #139 not touched.Local sanity on
73d1852(this workstation):cargo test --bins writers::— 47 passedcargo test --bins rpc_fallback— 8 passedcargo clippy --bins -- -D warnings— cleanhashVerifyExecuteBlocker+HashComparisonPanel(19), plusDestHashCard/evmExecutionRateLimit/ hash verification tests (50)Woodpecker (problem)
Required check
ci/woodpecker/pr/woodpeckernever posted (0 commit statuses on the PR head, same as #166 / #168). This repo has no Forgejo webhook toci.cl8y.comand no.woodpecker.yaml.fj pr mergereturned 405 until a maintainerforce_mergeafter the local suite. OpenBao (10.10.0.4) was unreachable from this session (wg-quick@wg0inactive), so the Woodpecker project could not be enabled here.Still open after merge (not this ticket)
/verify?hash=live check for Pending + blocker, then verified after executeFollow-up issue to be filed next (this ticket stays closed).
Follow-up tracking: #172
cl8y-agent-control: needs_human inbox card POST failed. Job stays parked.