Bot swarm stalls on Anvil when buy cooldown + idle chain freeze block.timestamp #99

New issue

Closed

opened 2026-04-30 05:05:47 +00:00 by PlasticDigits · 7 comments

PlasticDigits commented 2026-04-30 05:05:47 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

Summary

The default local Anvil + bot swarm path (SKIP_ANVIL_RICH_STATE=1 → START_BOT_SWARM=1) can stall after an initial burst of indexed buys. The Recent buys panel then shows a fixed head tx and ages that look wrong (chain time frozen; frontend separate issue #already fixed separately).

Root cause

1. Per-wallet buy cooldown — DeployDev defaults TimeCurve.buyCooldownSec to 300s unless YIELDOMEGA_DEPLOY_NO_COOLDOWN=1 / YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC (see GitLab #88, scripts/start-local-anvil-stack.sh).
2. Bot wait logic — timecurve_bot.actions._wait_until_buy_allowed sleeps until latest block timestamp >= nextBuyAllowedAt(sender).
3. Anvil automine + idle chain — With standard automine, no new blocks (and thus no block.timestamp advance) occur while every swarm wallet is sleeping off cooldown and no other transaction mines.

Result: chain tip and timestamps freeze; swarm workers that depend on advancing chain time (e.g. defender waiting for “<900s window”, buys after cooldown) never make progress until something external mines a block.

Impact

• Local QA: “live” buy feed goes quiet; operators assume indexer/UI bugs.
• Docs say swarm is for demos; current defaults do not reliably produce continuous activity without extra env.

Recommended fixes (pick one or combine)

A — Document + script default (lowest risk)

• In bots/timecurve/README.md and scripts/start-local-anvil-stack.sh header/next-steps: when START_BOT_SWARM=1, recommend YIELDOMEGA_DEPLOY_NO_COOLDOWN=1 (or short YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC) for continuous buys on Anvil.
• Optionally: if START_BOT_SWARM=1 && SKIP_ANVIL_RICH_STATE=1, warn when cooldown env not set.

B — Anvil periodic mining

• Start Anvil with --block-time N (or documented equivalent) when swarm is enabled so block.timestamp advances during sleeps (validate exact Foundry flag / behavior for this repo’s Anvil version).

C — Dev-only RPC in bots

• On chain id 31337, when _wait_until_buy_allowed would sleep longer than X seconds, call evm_increaseTime / Anvil helper once (behind YIELDOMEGA_ANVIL_TIME_WARP=1 or similar) so QA does not depend on operator flags. Must be clearly dev-only and off by default.

Acceptance criteria

• Repro is documented in-repo (link from start-local-anvil-stack.sh or bots/timecurve/README.md): default swarm + 300s cooldown + automine → stalled chain time after burst.
• At least one shipped mitigation: documented required env for continuous swarm activity or code/script change so a fresh SKIP_ANVIL_RICH_STATE=1 + swarm run produces new Buy events over ~10 minutes without manual cast send / wallet txs (or explicitly scope “burst only” and set expectations).
• No change to non-Anvil / production bot behavior without an explicit opt-in flag.
• Optional: add a short CI or script check (e.g. smoke that fail-logs if defender prints infinite “waiting for on-chain time” without block advance — brittle; only if low maintenance).

References

• bots/timecurve/src/timecurve_bot/actions.py — _wait_until_buy_allowed
• bots/timecurve/src/timecurve_bot/strategies/defender.py — reset-band / time window waits
• scripts/start-local-anvil-stack.sh — YIELDOMEGA_DEPLOY_NO_COOLDOWN, START_BOT_SWARM
• GitLab #88 — DeployDev buy cooldown env

## Summary The default **local Anvil + bot swarm** path (`SKIP_ANVIL_RICH_STATE=1` → `START_BOT_SWARM=1`) can stall after an initial burst of indexed buys. The **Recent buys** panel then shows a fixed head tx and ages that look wrong (chain time frozen; frontend separate issue #already fixed separately). ## Root cause 1. **Per-wallet buy cooldown** — `DeployDev` defaults `TimeCurve.buyCooldownSec` to **300s** unless `YIELDOMEGA_DEPLOY_NO_COOLDOWN=1` / `YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC` (see GitLab #88, `scripts/start-local-anvil-stack.sh`). 2. **Bot wait logic** — `timecurve_bot.actions._wait_until_buy_allowed` sleeps until `latest` block `timestamp >= nextBuyAllowedAt(sender)`. 3. **Anvil automine + idle chain** — With standard automine, **no new blocks** (and thus **no `block.timestamp` advance**) occur while every swarm wallet is sleeping off cooldown and **no other transaction mines**. Result: chain tip and timestamps freeze; swarm workers that depend on advancing chain time (e.g. `defender` waiting for “&lt;900s window”, buys after cooldown) never make progress until something external mines a block. ## Impact - Local QA: “live” buy feed goes quiet; operators assume indexer/UI bugs. - Docs say swarm is for demos; current defaults do not reliably produce continuous activity without extra env. ## Recommended fixes (pick one or combine) **A — Document + script default (lowest risk)** - In `bots/timecurve/README.md` and `scripts/start-local-anvil-stack.sh` header/next-steps: when `START_BOT_SWARM=1`, recommend **`YIELDOMEGA_DEPLOY_NO_COOLDOWN=1`** (or short `YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC`) for continuous buys on Anvil. - Optionally: if `START_BOT_SWARM=1` && `SKIP_ANVIL_RICH_STATE=1`, **warn** when cooldown env not set. **B — Anvil periodic mining** - Start Anvil with **`--block-time N`** (or documented equivalent) when swarm is enabled so `block.timestamp` advances during sleeps (validate exact Foundry flag / behavior for this repo’s Anvil version). **C — Dev-only RPC in bots** - On chain id **31337**, when `_wait_until_buy_allowed` would sleep longer than X seconds, call **`evm_increaseTime`** / Anvil helper **once** (behind `YIELDOMEGA_ANVIL_TIME_WARP=1` or similar) so QA does not depend on operator flags. Must be clearly dev-only and off by default. ## Acceptance criteria - [ ] Repro is **documented** in-repo (link from `start-local-anvil-stack.sh` or `bots/timecurve/README.md`): default swarm + 300s cooldown + automine → stalled chain time after burst. - [ ] At least **one** shipped mitigation: **documented required env** for continuous swarm activity **or** code/script change so a **fresh** `SKIP_ANVIL_RICH_STATE=1` + swarm run produces **new `Buy` events** over ~10 minutes **without** manual `cast send` / wallet txs (or explicitly scope “burst only” and set expectations). - [ ] No change to **non-Anvil** / production bot behavior without an explicit opt-in flag. - [ ] Optional: add a **short CI or script check** (e.g. smoke that fail-logs if defender prints infinite “waiting for on-chain time” without block advance — brittle; only if low maintenance). ## References - `bots/timecurve/src/timecurve_bot/actions.py` — `_wait_until_buy_allowed` - `bots/timecurve/src/timecurve_bot/strategies/defender.py` — reset-band / time window waits - `scripts/start-local-anvil-stack.sh` — `YIELDOMEGA_DEPLOY_NO_COOLDOWN`, `START_BOT_SWARM` - GitLab #88 — DeployDev buy cooldown env

PlasticDigits commented 2026-04-30 05:19:21 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

mentioned in commit 39dd51b9a1

mentioned in commit 39dd51b9a123463ea72e192130bb431dce0409af

PlasticDigits commented 2026-04-30 05:20:01 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

Implemented fixes A + B for GitLab #99 on `main` (merged).

What changed

A — Document + defaults

• Header + `bots/timecurve/README.md`: repro (300s cooldown + automine stalls chain time during all-wallet sleeps), recommend `YIELDOMEGA_DEPLOY_NO_COOLDOWN=1` / `YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC` for dense swarm demos.
• When `SKIP_ANVIL_RICH_STATE=1` and `START_BOT_SWARM=1` and DeployDev would use default-long cooldown (no `YIELDOMEGA_DEPLOY_NO_COOLDOWN=1` and no explicit `YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC`), the stack prints a Note (GitLab #99) before deploy.

B — Anvil interval mining

• When this script starts Anvil and `START_BOT_SWARM=1`, it appends Foundry `anvil --block-time ` (default 12 via `YIELDOMEGA_ANVIL_BLOCK_TIME_SEC`; set 0 to disable and use pure automine).
• If the RPC port was already bound, the script cannot retrofit `--block-time` — it warns (GitLab #99).

Also: removed accidental duplicate Kumbaya `yieldomega_frontend_merge_kumbaya_vite_full` block; docs + `skills/verify-yo-bot-swarm-local-anvil/SKILL.md`; invariants § #99; guardrails #33.

Not implemented: Issue C (dev-only `evm_increaseTime` in bots) — left out per scope (non-Anvil / production bots unchanged).

---

@brouie please verify:

• `SKIP_ANVIL_RICH_STATE=1 bash scripts/start-local-anvil-stack.sh` (fresh port): log shows Anvil interval mining: --block-time 12s (unless `YIELDOMEGA_ANVIL_BLOCK_TIME_SEC=0`).
• Same run with default swarm + default cooldown: Note (GitLab #99) prints; with `YIELDOMEGA_DEPLOY_NO_COOLDOWN=1` the note does not print.
• After initial swarm burst, wait few minutes: indexer Recent buys / `GET /v1/timecurve/buys` still picks up new rows (timestamps advancing under interval mining).
• Bind something on `8545` then `START_BOT_SWARM=1`: reused-RPC warning mentions GitLab #99 / `--block-time`.

Issue left open for your sign-off.

Implemented **fixes A + B** for [GitLab #99](https://gitlab.com/PlasticDigits/yieldomega/-/issues/99) on \`main\` (merged). ## What changed **A — Document + defaults** - Header + \`bots/timecurve/README.md\`: repro (300s cooldown + automine stalls chain time during all-wallet sleeps), recommend \`YIELDOMEGA_DEPLOY_NO_COOLDOWN=1\` / \`YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC\` for dense swarm demos. - When \`SKIP_ANVIL_RICH_STATE=1\` **and** \`START_BOT_SWARM=1\` **and** DeployDev would use **default-long** cooldown (no \`YIELDOMEGA_DEPLOY_NO_COOLDOWN=1\` and no explicit \`YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC\`), the stack prints a **Note (GitLab #99)** before deploy. **B — Anvil interval mining** - When this script **starts** Anvil and \`START_BOT_SWARM=1\`, it appends Foundry **\`anvil --block-time <sec>\`** (default **12** via \`YIELDOMEGA_ANVIL_BLOCK_TIME_SEC\`; set **0** to disable and use pure automine). - If the RPC port was **already bound**, the script **cannot** retrofit \`--block-time\` — it warns (GitLab #99). Also: removed accidental **duplicate Kumbaya** \`yieldomega_frontend_merge_kumbaya_vite_full\` block; docs + [\`skills/verify-yo-bot-swarm-local-anvil/SKILL.md\`](skills/verify-yo-bot-swarm-local-anvil/SKILL.md); invariants § #99; guardrails **#33**. **Not implemented:** Issue **C** (dev-only \`evm_increaseTime\` in bots) — left out per scope (**non-Anvil / production bots unchanged**). --- @brouie please verify: - [ ] \`SKIP_ANVIL_RICH_STATE=1 bash scripts/start-local-anvil-stack.sh\` (fresh port): log shows **Anvil interval mining: --block-time 12s** (unless \`YIELDOMEGA_ANVIL_BLOCK_TIME_SEC=0\`). - [ ] Same run with **default** swarm + default cooldown: **Note (GitLab #99)** prints; with \`YIELDOMEGA_DEPLOY_NO_COOLDOWN=1\` the note **does not** print. - [ ] After initial swarm burst, wait **few minutes**: indexer **Recent buys** / \`GET /v1/timecurve/buys\` still picks up **new** rows (timestamps advancing under interval mining). - [ ] Bind something on \`8545\` then \`START_BOT_SWARM=1\`: reused-RPC **warning** mentions GitLab #99 / \`--block-time\`. Issue left **open** for your sign-off.

PlasticDigits commented 2026-04-30 05:25:09 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

mentioned in issue #100

mentioned in issue #100

Brouie commented 2026-04-30 09:02:47 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

Verified fixes A + B for #99 on dd1c59a HEAD today. Items 1-3 PASS, Item 4 deferred. Closing as primary fix verified; will retest Item 4 separately if needed.

Stack state

Boot via: SKIP_ANVIL_RICH_STATE=1 START_BOT_SWARM=1 bash scripts/start-local-anvil-stack.sh (defaults: cooldown=300s, --block-time 12s, swarm referrals on)

Item 1 — --block-time 12s log line: PASS

Stack log on first 5s of boot:
Anvil interval mining: --block-time 12s (GitLab #99; YIELDOMEGA_ANVIL_BLOCK_TIME_SEC=0 disables).
Confirmed via ps: anvil --host 127.0.0.1 --port 8545 --code-size-limit 524288 --accounts 30 --block-time 12

Item 2 — Note (GitLab #99) prints with default cooldown: PASS

Stack log:
Note (GitLab #99): SKIP_ANVIL_RICH_STATE=1 + START_BOT_SWARM=1 with default DeployDev buyCooldownSec=300.
For continuous indexer demo traffic, prefer: YIELDOMEGA_DEPLOY_NO_COOLDOWN=1 (and optionally YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC=…).
Anvil is started with --block-time when this script launches it so sleeps still advance chain time.

Item 3 — Recent buys advancing under interval mining: PASS

Indexer block progression observed:

• Boot: block 7
• T+2min: block 40
• T+5min: block 48 (with indexer paused via SIGSTOP, 21 ReferralApplied rows ingested)
• T+10min: block 78 (post-SIGCONT recovery, 40 ReferralApplied rows)

/v1/timecurve/buys?limit=10 returns 10 rows with advancing block_number values. New rows continue to land via the swarm's 18 bots (3× seed-local/defender/fun/shark/pvp/rando) under interval mining.

Item 4 — RPC port already-bound warning: DEFERRED

Did not exercise the bind 8545 then START_BOT_SWARM=1 path this session — leaving for follow-up. The reused-RPC code path is documented but not visually confirmed.

Closure

Primary mining + cooldown + advance-under-sleep behaviour verified end-to-end. 18 swarm bots produced 40+ buys + 40 ReferralApplied rows under the 12s block-time. Closing.

cc @PlasticDigits

Verified fixes A + B for #99 on `dd1c59a` HEAD today. Items 1-3 PASS, Item 4 deferred. Closing as primary fix verified; will retest Item 4 separately if needed. ## Stack state Boot via: `SKIP_ANVIL_RICH_STATE=1 START_BOT_SWARM=1 bash scripts/start-local-anvil-stack.sh` (defaults: cooldown=300s, --block-time 12s, swarm referrals on) ## Item 1 — `--block-time 12s` log line: PASS Stack log on first 5s of boot: `Anvil interval mining: --block-time 12s (GitLab #99; YIELDOMEGA_ANVIL_BLOCK_TIME_SEC=0 disables).` Confirmed via `ps`: `anvil --host 127.0.0.1 --port 8545 --code-size-limit 524288 --accounts 30 --block-time 12` ## Item 2 — Note (GitLab #99) prints with default cooldown: PASS Stack log: `Note (GitLab #99): SKIP_ANVIL_RICH_STATE=1 + START_BOT_SWARM=1 with default DeployDev buyCooldownSec=300.` `For continuous indexer demo traffic, prefer: YIELDOMEGA_DEPLOY_NO_COOLDOWN=1 (and optionally YIELDOMEGA_ANVIL_BUY_COOLDOWN_SEC=…).` `Anvil is started with --block-time when this script launches it so sleeps still advance chain time.` ## Item 3 — Recent buys advancing under interval mining: PASS Indexer block progression observed: - Boot: block 7 - T+2min: block 40 - T+5min: block 48 (with indexer paused via SIGSTOP, 21 ReferralApplied rows ingested) - T+10min: block 78 (post-SIGCONT recovery, 40 ReferralApplied rows) `/v1/timecurve/buys?limit=10` returns 10 rows with advancing block_number values. New rows continue to land via the swarm's 18 bots (3× seed-local/defender/fun/shark/pvp/rando) under interval mining. ## Item 4 — RPC port already-bound warning: DEFERRED Did not exercise the `bind 8545 then START_BOT_SWARM=1` path this session — leaving for follow-up. The reused-RPC code path is documented but not visually confirmed. ## Closure Primary mining + cooldown + advance-under-sleep behaviour verified end-to-end. 18 swarm bots produced 40+ buys + 40 ReferralApplied rows under the 12s block-time. Closing. cc @PlasticDigits

Brouie (Migrated from gitlab.com) closed this issue 2026-04-30 09:02:48 +00:00

PlasticDigits commented 2026-04-30 09:24:44 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

mentioned in issue #102

mentioned in issue #102

PlasticDigits commented 2026-05-01 06:11:33 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

mentioned in issue #104

mentioned in issue #104

PlasticDigits commented 2026-05-05 13:57:31 +00:00 (Migrated from gitlab.com)

• Copy link
• Copy Markdown

mentioned in issue #162

mentioned in issue #162

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

cac-design-issue-544

renovate/configure

No results found.

Labels

Clear labels   

agent:fix_bugfix

  

agent:fix_conflicts

  

agent:fix_security

  

agent:gap_analysis

  

agent:implement

  

agent:open_issues

  

agents

  

agent:security_audit

  

agent:verify

  

area

timecurve

  

arena

  

arena-v2

  

audit

  

block:feature_review

  

block:security

  

bug

  

contracts

  

design

  

devops

  

docs

Documentation

  

documentation

  

enhancement

  

epic

Epic / umbrella issue

  

feature

  

frontend

  

indexer

  

low priority

Low priority, investigate other issues first

  

megaeth

  

operations

  

ops

Deploy and operations

  

performance

  

QA

  

qa checklist

  

referrals

  

removal

Legacy removal

  

security

  

smart-contracts

  

tech-debt

  

testing

  

time-arena

TimeArena v2 revamp

  

timecurve

  

visual-qa

No labels

agent:fix_bugfix

agent:fix_conflicts

agent:fix_security

agent:gap_analysis

agent:implement

agent:open_issues

agents

agent:security_audit

agent:verify

area

timecurve

arena

arena-v2

audit

block:feature_review

block:security

bug

contracts

design

devops

docs

documentation

enhancement

epic

feature

frontend

indexer

low priority

megaeth

operations

ops

performance

QA

qa checklist

referrals

removal

security

smart-contracts

tech-debt

testing

time-arena

timecurve

visual-qa

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

code/yieldomega#99

No description provided.