feat(collect): competitor fee/liquidity watch into raw notes ingest #19

Merged

PlasticDigits merged 1 commit from issue/14 into main

2026-09-22 08:56:38 +00:00

PlasticDigits commented

2026-09-22 08:51:07 +00:00

Owner

Summary

Implements cl8y-research#14: scheduled competitor fee/liquidity watch → SourceKind::CompetitorWatch raw notes (inbox), separate from the week pipeline.

watch CLI (fixtures default, --live-watch for committed DeFiLlama protocol URLs)
COMPETITOR_WATCH_PAGES + check_competitor_watch_url (exact URL, SSRF fail-closed)
claims_for_week_emit numeric firewall so competitor TVL/fees do not satisfy week unsourced()
Fixtures under fixtures/competitor-watch/, docs/skills updated

Fixes #14

Acceptance criteria

AC	Status
AC1 `CompetitorWatch` + `parse_stored_kind` round-trip	Done
AC2 `watch` → `notes.json` / `gaps.md`, no MDX/plan	Done
AC3 Live GET only table URLs	Done (allowlist tests)
AC4 JSON fee/TVL claims + gaps for nulls	Done
AC5 HTML extractor (unit tests; no committed HTML URL yet)	Partial — mechanism tested, table is JSON-only
AC6 `search` hits `competitor_watch`	Done
AC7 Week emit ignores competitor claims	Done (`claims_for_week_emit`)
AC8 Per-page gaps, exit 0	Done
AC9 Docs/skills/invariants	Done
AC10 CI fixtures only	Done

Test plan

cargo fmt
cargo clippy --all-targets -- -D warnings
cargo test --all-targets
cargo run -- watch --fixtures fixtures/competitor-watch --out runs/watch/test

Gaps / follow-ups

No committed HTML fee page URL yet (venues without stable public JSON documented in docs/competitor-watch.md).
Live --live-watch not run in CI (operator verification).
Worker log verification not required for this change (no runtime incident).

## Summary Implements cl8y-research#14: scheduled competitor fee/liquidity watch → `SourceKind::CompetitorWatch` raw notes (inbox), separate from the `week` pipeline. - `watch` CLI (fixtures default, `--live-watch` for committed DeFiLlama protocol URLs) - `COMPETITOR_WATCH_PAGES` + `check_competitor_watch_url` (exact URL, SSRF fail-closed) - `claims_for_week_emit` numeric firewall so competitor TVL/fees do not satisfy week `unsourced()` - Fixtures under `fixtures/competitor-watch/`, docs/skills updated Fixes #14 ## Acceptance criteria | AC | Status | | --- | --- | | AC1 `CompetitorWatch` + `parse_stored_kind` round-trip | Done | | AC2 `watch` → `notes.json` / `gaps.md`, no MDX/plan | Done | | AC3 Live GET only table URLs | Done (allowlist tests) | | AC4 JSON fee/TVL claims + gaps for nulls | Done | | AC5 HTML extractor (unit tests; no committed HTML URL yet) | Partial — mechanism tested, table is JSON-only | | AC6 `search` hits `competitor_watch` | Done | | AC7 Week emit ignores competitor claims | Done (`claims_for_week_emit`) | | AC8 Per-page gaps, exit 0 | Done | | AC9 Docs/skills/invariants | Done | | AC10 CI fixtures only | Done | ## Test plan ```bash cargo fmt cargo clippy --all-targets -- -D warnings cargo test --all-targets cargo run -- watch --fixtures fixtures/competitor-watch --out runs/watch/test ``` ## Gaps / follow-ups - No committed HTML fee page URL yet (venues without stable public JSON documented in `docs/competitor-watch.md`). - Live `--live-watch` not run in CI (operator verification). - Worker log verification not required for this change (no runtime incident).

PlasticDigits added 1 commit

2026-09-22 08:51:07 +00:00

feat(collect): competitor fee/liquidity watch into raw notes ingest

ci/woodpecker/push/woodpecker Pipeline was successful

Details

ci/woodpecker/pr/woodpecker Pipeline was successful

Details

f70fa38ea1

Add CompetitorWatch source kind, allowlisted page table, watch CLI with
fixture cassettes, numeric firewall for week emit, and docs/skills for #14.

Fixes #14

Co-authored-by: Cursor <cursoragent@cursor.com>