code/cl8y-research
2
0
Fork 0
Code Issues 10 Pull requests Projects Releases Packages Wiki Activity

Security: DNS pin, Replicate redirects/budget, secret lint, and CI scanning #10

New issue
Open
opened 2026-08-31 05:33:46 +00:00 by PlasticDigits · 4 comments
PlasticDigits commented 2026-08-31 05:33:46 +00:00 (Migrated from gitlab.com)
Copy link

Summary

Close the High/Medium/Low findings in gap analysis §5 (Security). Live collectors, Replicate, Telegram, and the posts API depend on these controls.

Source: gaps/GAP_1788152435.md §5.

Current codebase

Strengths: HTTPS-only indexer helper, exact host match, private/link-local/metadata blocked, punycode rejected, userinfo rejected, explicit ports rejected, open-redirect query keys rejected, no redirects on indexer GET, parameterized SQL, secret-marker fail on artifacts, injection fixtures in tests/issue_plan.rs.

Sev Finding Location
H No DNS pinning. Allowlisted hostname can rebind to a private IP after resolve. allowlist.rs, fetch_allowlisted_json
H HttpPredictor follows redirects; indexer client does not. replicate.rs vs collect.rs
H Budget + create_count increment before POST; failure may have no prediction id. HttpPredictor::create_once
H poll returns on first HTTP 200 even if status is starting/processing. HttpPredictor::poll
M IPv6 blocking is string prefixes (fd, fe80:), not a real IP parser. is_blocked_ip_host
M Secret lint is substring markers only; redact unused (fail-closed). secrets.rs
M Telegram is_team is fixture-attacker-controlled; live must use member API. telegram.rs
M INDEXER_HOSTS includes indexer.bridge.cl8y.com while docs say that API does not exist. invariants.rs
M No Gitleaks / cargo-audit / cargo deny in CI. .gitlab-ci.yml
L tracing unused; errors via eprintln. main.rs
L Session 0600 documented, not implemented (Bot API preferred). README
L Posts API (future) needs cache headers, rate limit, draft vs published. n/a (coordinate with posts API issue)

SSRF fixture covers http://169.254.169.254/ via ssrf_url. No mock live HTTP SSRF after DNS.

Why this is needed

Turning on --live-sources / --live without DNS pin and redirect policy is a cloud-metadata SSRF. Budget-before-create and poll-on-starting waste money and emit empty posts. CI has no advisory audit.

Constraints / guardrails

  1. Fail closed. Do not “best-effort” fetch after a pin failure.
  2. Do not weaken host allowlists to make tests pass.
  3. Replicate create-once rule stays: never retry POST create.
  4. Gitleaks must not require secrets in the repo; use .gitleaks.toml allowlist only for fixtures that are fake.
  5. Telegram is_team fix may land in the Telegram issue; this issue should add the test contract and/or shared helper if that issue is separate — do not duplicate conflicting implementations. Prefer one helper used by both.
  6. Remove indexer.bridge.cl8y.com from allowlist until a real documented API exists (docs/bridge-indexer-gap.md).

Relevant files

src/allowlist.rs, src/collect.rs, src/replicate.rs, src/secrets.rs, src/telegram.rs, src/invariants.rs, src/main.rs, .gitlab-ci.yml, README.md, tests/issue_plan.rs

Recommended direction

  1. After HTTPS parse: resolve hostname; require all A/AAAA in public ranges; connect to that IP with TLS SNI = hostname (or use a resolver hook). Reject if any address is private/link-local/metadata/unspecified.
  2. Parse IPv4/IPv6 properly (std::net::IpAddr).
  3. Replicate: redirect(Policy::none()); charge after id; poll until terminal (overlap with Replicate issue — implement once, test here as security cases).
  4. Secret lint: keep fail-closed; add optional high-entropy token detector for sk- / bearer-shaped strings in artifacts; use redact only for logs, not to wash MDX into publish.
  5. CI: gitleaks + cargo audit (and/or cargo deny advisories).
  6. tracing on HTTP errors without headers Authorization.

Acceptance criteria

  • Mock/getaddrinfo hook or integration test: hostname allowlisted but resolves to 127.0.0.1 / 169.254.169.254 → fetch error, no request body to that IP (or documented connect fail).
  • Replicate client does not follow redirects.
  • IPv6 unique-local / loopback rejected via IpAddr, not starts_with("fd") only.
  • indexer.bridge.cl8y.com not in INDEXER_HOSTS until gap doc is updated with a real API.
  • CI job: gitleaks on the checkout; cargo audit (network allowed in that job only).
  • Logs do not print Authorization or TELEGRAM_BOT_TOKEN.
  • README session 0600 either implemented or removed in favor of Bot API.

Test plan — functional paths

  1. Existing allowlist unit tests still pass.
  2. DNS pin positive: mock resolve to a public test IP or skip live network and unit-test the IP classifier.
  3. Replicate redirect 302 → error.
  4. cargo audit job exists (may warn; fail on RUSTSEC critical as policy documented).

Test plan — attack, hack, and abuse

  1. DNS rebinding — two-address response including private; reject.
  2. Redirect chain to metadata on indexer and Replicate.
  3. IPv6 mapped IPv4 loopback (::ffff:127.0.0.1).
  4. Punycode still rejected.
  5. Open-redirect query still rejected.
  6. Secret in artifact — high-entropy line without REPLICATE_ prefix still fails publish if detector is in scope; if deferred, document leftover risk.
  7. Gitleaks — planted AKIA… style string in a test file that is not allowlisted should fail the job (use a CI-only fixture path that is expected to fail in a unit of the gitleaks config test, or a dry-run doc). Prefer: gitleaks on normal fixtures stays green (no real secrets).

Verification criteria

  1. Reviewer reads pin + redirect tests.
  2. CI shows gitleaks + audit jobs.
  3. Gap §5 table items are checked off or explicitly deferred with reason in the MR.
## Summary Close the **High/Medium/Low** findings in gap analysis §5 (Security). Live collectors, Replicate, Telegram, and the posts API depend on these controls. Source: [`gaps/GAP_1788152435.md`](https://gitlab.com/PlasticDigits/cl8y-research/-/blob/main/gaps/GAP_1788152435.md) §5. --- ## Current codebase Strengths: HTTPS-only indexer helper, exact host match, private/link-local/metadata blocked, punycode rejected, userinfo rejected, explicit ports rejected, open-redirect query keys rejected, **no redirects** on indexer GET, parameterized SQL, secret-marker fail on artifacts, injection fixtures in `tests/issue_plan.rs`. | Sev | Finding | Location | | --- | --- | --- | | **H** | No **DNS pinning**. Allowlisted hostname can rebind to a private IP after resolve. | `allowlist.rs`, `fetch_allowlisted_json` | | **H** | `HttpPredictor` **follows redirects**; indexer client does not. | `replicate.rs` vs `collect.rs` | | **H** | Budget + `create_count` increment **before** POST; failure may have no prediction id. | `HttpPredictor::create_once` | | **H** | `poll` returns on first HTTP 200 even if status is `starting`/`processing`. | `HttpPredictor::poll` | | **M** | IPv6 blocking is string prefixes (`fd`, `fe80:`), not a real IP parser. | `is_blocked_ip_host` | | **M** | Secret lint is substring markers only; `redact` unused (fail-closed). | `secrets.rs` | | **M** | Telegram `is_team` is fixture-attacker-controlled; live must use member API. | `telegram.rs` | | **M** | `INDEXER_HOSTS` includes `indexer.bridge.cl8y.com` while docs say that API does not exist. | `invariants.rs` | | **M** | No Gitleaks / `cargo-audit` / `cargo deny` in CI. | `.gitlab-ci.yml` | | **L** | `tracing` unused; errors via `eprintln`. | `main.rs` | | **L** | Session `0600` documented, not implemented (Bot API preferred). | README | | **L** | Posts API (future) needs cache headers, rate limit, draft vs published. | n/a (coordinate with posts API issue) | SSRF fixture covers `http://169.254.169.254/` via `ssrf_url`. No mock live HTTP SSRF after DNS. --- ## Why this is needed Turning on `--live-sources` / `--live` without DNS pin and redirect policy is a cloud-metadata SSRF. Budget-before-create and poll-on-starting waste money and emit empty posts. CI has no advisory audit. --- ## Constraints / guardrails 1. Fail closed. Do not “best-effort” fetch after a pin failure. 2. Do not weaken host allowlists to make tests pass. 3. Replicate create-once rule stays: never retry POST create. 4. Gitleaks must not require secrets in the repo; use `.gitleaks.toml` allowlist only for fixtures that are fake. 5. Telegram `is_team` fix may land in the Telegram issue; this issue should add the **test contract** and/or shared helper if that issue is separate — do not duplicate conflicting implementations. Prefer one helper used by both. 6. Remove `indexer.bridge.cl8y.com` from allowlist until a real documented API exists (`docs/bridge-indexer-gap.md`). --- ## Relevant files `src/allowlist.rs`, `src/collect.rs`, `src/replicate.rs`, `src/secrets.rs`, `src/telegram.rs`, `src/invariants.rs`, `src/main.rs`, `.gitlab-ci.yml`, `README.md`, `tests/issue_plan.rs` --- ## Recommended direction 1. After HTTPS parse: resolve hostname; require all A/AAAA in public ranges; connect to that IP with TLS SNI = hostname (or use a resolver hook). Reject if any address is private/link-local/metadata/unspecified. 2. Parse IPv4/IPv6 properly (`std::net::IpAddr`). 3. Replicate: `redirect(Policy::none())`; charge after id; poll until terminal (overlap with Replicate issue — implement once, test here as security cases). 4. Secret lint: keep fail-closed; add optional high-entropy token detector for `sk-` / bearer-shaped strings in artifacts; use `redact` only for logs, not to wash MDX into publish. 5. CI: `gitleaks` + `cargo audit` (and/or `cargo deny` advisories). 6. tracing on HTTP errors without headers `Authorization`. --- ## Acceptance criteria - [ ] Mock/getaddrinfo hook or integration test: hostname allowlisted but resolves to `127.0.0.1` / `169.254.169.254` → fetch error, no request body to that IP (or documented connect fail). - [ ] Replicate client does not follow redirects. - [ ] IPv6 unique-local / loopback rejected via `IpAddr`, not `starts_with("fd")` only. - [ ] `indexer.bridge.cl8y.com` not in `INDEXER_HOSTS` until gap doc is updated with a real API. - [ ] CI job: gitleaks on the checkout; `cargo audit` (network allowed in that job only). - [ ] Logs do not print `Authorization` or `TELEGRAM_BOT_TOKEN`. - [ ] README session `0600` either implemented or removed in favor of Bot API. --- ## Test plan — functional paths 1. Existing allowlist unit tests still pass. 2. DNS pin positive: mock resolve to a public test IP **or** skip live network and unit-test the IP classifier. 3. Replicate redirect 302 → error. 4. cargo audit job exists (may warn; fail on RUSTSEC critical as policy documented). --- ## Test plan — attack, hack, and abuse 1. **DNS rebinding** — two-address response including private; reject. 2. **Redirect chain** to metadata on indexer **and** Replicate. 3. **IPv6 mapped IPv4** loopback (`::ffff:127.0.0.1`). 4. **Punycode** still rejected. 5. **Open-redirect query** still rejected. 6. **Secret in artifact** — high-entropy line without `REPLICATE_` prefix still fails publish if detector is in scope; if deferred, document leftover risk. 7. **Gitleaks** — planted `AKIA…` style string in a test file that is **not** allowlisted should fail the job (use a CI-only fixture path that is expected to fail in a unit of the gitleaks config test, or a dry-run doc). Prefer: gitleaks on normal fixtures stays green (no real secrets). --- ## Verification criteria 1. Reviewer reads pin + redirect tests. 2. CI shows gitleaks + audit jobs. 3. Gap §5 table items are checked off or explicitly deferred with reason in the MR.
PlasticDigits commented 2026-08-31 05:35:02 +00:00 (Migrated from gitlab.com)
Copy link

marked as related to #4

marked as related to #4
PlasticDigits commented 2026-08-31 05:35:03 +00:00 (Migrated from gitlab.com)
Copy link

marked as related to #5

marked as related to #5
PlasticDigits commented 2026-08-31 05:35:09 +00:00 (Migrated from gitlab.com)
Copy link

marked as related to #3

marked as related to #3
PlasticDigits commented 2026-09-02 02:18:49 +00:00 (Migrated from gitlab.com)
Copy link

mentioned in issue #12

mentioned in issue #12
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
code/cl8y-research#10
No description provided.