code/cl8y-research
2
0
Fork 0
Code Issues 10 Pull requests Projects Releases Packages Wiki Activity

Posts API (Option B): host published blog and seed from CL8Y-web #2

New issue
Open
opened 2026-08-31 05:32:21 +00:00 by PlasticDigits · 5 comments
PlasticDigits commented 2026-08-31 05:32:21 +00:00 (Migrated from gitlab.com)
Copy link

Summary

Ship Option B (approved): cl8y-research is the source of truth for published blog posts. A public read API serves list/post/RSS/hero. CL8Y-web stops taking per-post MDX merge requests. Seed the store from the existing cl8y.com post in ~/repos/CL8Y-web.

Companion consumer work lives in CL8Y-web and is blocked until this API is up (GET /v1/posts returns the seeded roadmap post and /health is green).

Gap analysis: gaps/GAP_1788152435.md §3.3. Original worker: #1.

Current codebase

CL8Y-web is a static Vite SPA. Posts are MDX in src/blog/posts/*.mdx, indexed by import.meta.glob + virtual:blog-post-meta, prerendered to /blog and /blog/:slug, RSS via scripts/generate-rss.mjs.

There is one published post:

  • Slug: cl8y-roadmap-cmm-bridge-yieldomega (2026-05-10)
  • File: CL8Y-web/src/blog/posts/cl8y-roadmap-cmm-bridge-yieldomega.mdx
  • Hero: CL8Y-web/public/images/blog/cl8y-roadmap-cmm-bridge-yieldomega-hero.jpg

This worker emits runs/<week>/post.mdx + publish-mr.md copy instructions. src/publish.rs builds an in-memory DraftMr (auto_merge: false) and never talks to GitLab. There is no HTTP server, no posts table, no publish approval command.

wordCount is computed at site build today. After Option B it must be computed at publish in this repo (still never authored by the model).

Why this is needed

Every weekly draft is a marketing-site MR, full yarn CI, Puppeteer prerender, and review of 2000-word MDX in a repo whose job is positioning and token directory. That is the time sink. Hosting approved posts here keeps secrets and generation off the SPA and lets CL8Y-web stay a fetcher.

Existing visitors must not lose the roadmap post when the glob goes away. Import it (body + frontmatter + hero bytes) as the first published row.

Constraints / guardrails

  1. No auto-publish to cl8y.com. Public API serves status = published only. Drafts stay private (operator/CLI). Human publish (CLI or authenticated admin) is required.
  2. Do not push CL8Y-web main or auto-merge. Per-post MDX MRs are retired after the consumer issue ships; this issue does not edit the Vite app except via the linked CL8Y-web ticket.
  3. Do not remount RETIRED_HOMEPAGE_MODULES or rewrite CL8Y_WHITEPAPER.md.
  4. MDX contract unchanged: title, description, slug, date, author, image, tags. Never store authored wordCount; compute it on publish from body.
  5. Hero URL remains /images/blog/<slug>-hero.jpg (API may serve the bytes). Never src/blog/assets.
  6. Sanitizer + mechanical lint must pass before a row can become published.
  7. Secrets never appear in JSON, RSS, or image metadata (REPLICATE_, BOT_TOKEN, api_key, BEGIN ).
  8. CORS: allow https://cl8y.com and https://www.cl8y.com only (plus localhost for dev). Rate-limit public GETs.
  9. Cache-Control on public GETs; do not cache drafts.
  10. Invite links, Telegram raw text, and untrusted sources must not leak through post JSON.

Relevant files

Path Role
src/publish.rs Draft MR helper to replace/extend with publish store
src/mdx.rs Frontmatter emit/parse/sanitize
src/lint.rs Mechanical lint gate before publish
src/pipeline.rs Emit writes post.mdx + publish-mr.md
src/store.rs / migrations/001_init.sql Persist posts + heroes (new tables)
docs/blog-contract.md Contract to keep
CL8Y-web/src/blog/blogIndex.ts Current consumer (other repo)
CL8Y-web/src/blog/posts/cl8y-roadmap-cmm-bridge-yieldomega.mdx Seed source
CL8Y-web/public/images/blog/cl8y-roadmap-cmm-bridge-yieldomega-hero.jpg Seed hero

Recommended direction

  1. Add Axum (or equivalent) behind --features postgres (or a server feature): bind API_BIND (default 0.0.0.0:8080).
  2. Schema: posts (slug unique, status draft|published, frontmatter columns, body_mdx, word_count, published_at, hero_bytes or object storage path). Reuse Postgres from the store issue.
  3. Public routes:
    • GET /health and GET /ready (see Coolify/health issue; stub here if that issue is later)
    • GET /v1/posts — published list, newest first
    • GET /v1/posts/:slug — published body + meta
    • GET /v1/rss.xml
    • GET /v1/images/blog/:slug-hero.jpg
  4. Operator: cl8y-research import-web --from <CL8Y-web checkout> seeds the roadmap post as published. cl8y-research publish --slug <s> promotes a pipeline draft after lint.
  5. Pipeline emit writes a draft row (or runs/ + ingest) instead of instructing an MDX copy.
  6. OpenAPI or a frozen JSON schema fixture for CL8Y-web.

Do not put Replicate/Telegram tokens on public routes.

Acceptance criteria

  • Public GET /v1/posts lists the seeded cl8y-roadmap-cmm-bridge-yieldomega with required meta (no wordCount authored in MDX; numeric wordCount in JSON is computed).
  • GET /v1/posts/cl8y-roadmap-cmm-bridge-yieldomega returns body equivalent to the current MDX (modulo computed fields).
  • GET /v1/images/blog/cl8y-roadmap-cmm-bridge-yieldomega-hero.jpg returns a JPEG (image/jpeg).
  • GET /v1/rss.xml includes that item with title, description, /blog/<slug> link (canonical host configurable).
  • Draft pipeline output is not on public GET until publish.
  • Missing slug → 404. Unpublished slug → 404 (not 401).
  • CORS allowlist; unknown origin rejected.
  • README documents import + publish + local curl against compose.
  • CL8Y-web consumer issue can be unblocked when /health is 200 and the seeded post is listed.

Test plan — functional paths

  1. Import seed — Point import at a fixture copy of the roadmap MDX + hero. Assert published row, slug, tags, JPEG bytes, computed wordCount > 0.
  2. List / get / RSS / hero — HTTP tests (axum test or reqwest to bound server).
  3. Draft isolation — Insert draft slug weekly-secret; public GET list/get/RSS omit it; 404 on get.
  4. Publish gate — Dirty MDX (em dash, unofficial address) cannot publish; clean draft can.
  5. Slug uniqueness — Second import of same slug does not duplicate; collision policy documented.
  6. CORS — Origin: https://cl8y.com allowed; Origin: https://evil.test not.
  7. Health — /health 200 without DB; /ready 200 only with DB (or document if combined).

Test plan — attack, hack, and abuse

  1. Unpublished leak — Guessable draft slug must 404. No status filter query that returns drafts.
  2. Path traversal — GET /v1/images/blog/../../.env rejected.
  3. XSS in JSON — Body may contain Markdown; JSON must not include <script> that sanitize would have stripped; Content-Type application/json (not HTML) for post GET.
  4. RSS injection — Title/description XML-escaped; no javascript: links.
  5. Secret exfil — Seed/post body with REPLICATE_API_TOKEN= fails publish.
  6. Cache poisoning / host header — Canonical links in RSS use configured public origin, not Host from the client.
  7. Rate / flood — Document or implement a GET rate limit; test that the limiter does not skip /health.
  8. Overwrite production — Import must not clobber a newer published body without an explicit --force.

Verification criteria

A reviewer can:

  1. docker compose up -d + migrate + import-web from a CL8Y-web checkout (or fixture).
  2. curl list/get/RSS/hero and see the 2026-05-10 roadmap post.
  3. Confirm a new week draft is absent from public GET until publish.
  4. Hand the public base URL to the CL8Y-web issue as the unblock signal.
## Summary Ship **Option B** (approved): `cl8y-research` is the source of truth for published blog posts. A public read API serves list/post/RSS/hero. `CL8Y-web` stops taking per-post MDX merge requests. Seed the store from the existing cl8y.com post in `~/repos/CL8Y-web`. Companion consumer work lives in **CL8Y-web** and is **blocked** until this API is up (`GET /v1/posts` returns the seeded roadmap post and `/health` is green). Gap analysis: [`gaps/GAP_1788152435.md`](https://gitlab.com/PlasticDigits/cl8y-research/-/blob/main/gaps/GAP_1788152435.md) §3.3. Original worker: [#1](https://gitlab.com/PlasticDigits/cl8y-research/-/issues/1). --- ## Current codebase `CL8Y-web` is a static Vite SPA. Posts are MDX in `src/blog/posts/*.mdx`, indexed by `import.meta.glob` + `virtual:blog-post-meta`, prerendered to `/blog` and `/blog/:slug`, RSS via `scripts/generate-rss.mjs`. There is **one** published post: - Slug: `cl8y-roadmap-cmm-bridge-yieldomega` (2026-05-10) - File: `CL8Y-web/src/blog/posts/cl8y-roadmap-cmm-bridge-yieldomega.mdx` - Hero: `CL8Y-web/public/images/blog/cl8y-roadmap-cmm-bridge-yieldomega-hero.jpg` This worker emits `runs/<week>/post.mdx` + `publish-mr.md` copy instructions. `src/publish.rs` builds an in-memory `DraftMr` (`auto_merge: false`) and never talks to GitLab. There is **no HTTP server**, no `posts` table, no publish approval command. `wordCount` is computed at **site build** today. After Option B it must be computed at **publish** in this repo (still never authored by the model). --- ## Why this is needed Every weekly draft is a marketing-site MR, full `yarn` CI, Puppeteer prerender, and review of 2000-word MDX in a repo whose job is positioning and token directory. That is the time sink. Hosting approved posts here keeps secrets and generation off the SPA and lets `CL8Y-web` stay a fetcher. Existing visitors must not lose the roadmap post when the glob goes away. Import it (body + frontmatter + hero bytes) as the first `published` row. --- ## Constraints / guardrails 1. **No auto-publish to cl8y.com.** Public API serves `status = published` only. Drafts stay private (operator/CLI). Human `publish` (CLI or authenticated admin) is required. 2. Do **not** push `CL8Y-web` `main` or auto-merge. Per-post MDX MRs are retired after the consumer issue ships; this issue does not edit the Vite app except via the linked CL8Y-web ticket. 3. Do not remount `RETIRED_HOMEPAGE_MODULES` or rewrite `CL8Y_WHITEPAPER.md`. 4. MDX contract unchanged: `title`, `description`, `slug`, `date`, `author`, `image`, `tags`. Never store authored `wordCount`; compute it on publish from body. 5. Hero URL remains `/images/blog/<slug>-hero.jpg` (API may serve the bytes). Never `src/blog/assets`. 6. Sanitizer + mechanical lint must pass **before** a row can become `published`. 7. Secrets never appear in JSON, RSS, or image metadata (`REPLICATE_`, `BOT_TOKEN`, `api_key`, `BEGIN `). 8. CORS: allow `https://cl8y.com` and `https://www.cl8y.com` only (plus localhost for dev). Rate-limit public GETs. 9. Cache-Control on public GETs; do not cache drafts. 10. Invite links, Telegram raw text, and untrusted sources must not leak through post JSON. --- ## Relevant files | Path | Role | | --- | --- | | `src/publish.rs` | Draft MR helper to replace/extend with publish store | | `src/mdx.rs` | Frontmatter emit/parse/sanitize | | `src/lint.rs` | Mechanical lint gate before publish | | `src/pipeline.rs` | Emit writes `post.mdx` + `publish-mr.md` | | `src/store.rs` / `migrations/001_init.sql` | Persist posts + heroes (new tables) | | `docs/blog-contract.md` | Contract to keep | | `CL8Y-web/src/blog/blogIndex.ts` | Current consumer (other repo) | | `CL8Y-web/src/blog/posts/cl8y-roadmap-cmm-bridge-yieldomega.mdx` | Seed source | | `CL8Y-web/public/images/blog/cl8y-roadmap-cmm-bridge-yieldomega-hero.jpg` | Seed hero | --- ## Recommended direction 1. Add Axum (or equivalent) behind `--features postgres` (or a `server` feature): bind `API_BIND` (default `0.0.0.0:8080`). 2. Schema: `posts` (`slug` unique, `status` draft|published, frontmatter columns, `body_mdx`, `word_count`, `published_at`, `hero_bytes` or object storage path). Reuse Postgres from the store issue. 3. Public routes: - `GET /health` and `GET /ready` (see Coolify/health issue; stub here if that issue is later) - `GET /v1/posts` — published list, newest first - `GET /v1/posts/:slug` — published body + meta - `GET /v1/rss.xml` - `GET /v1/images/blog/:slug-hero.jpg` 4. Operator: `cl8y-research import-web --from <CL8Y-web checkout>` seeds the roadmap post as **published**. `cl8y-research publish --slug <s>` promotes a pipeline draft after lint. 5. Pipeline emit writes a **draft** row (or `runs/` + ingest) instead of instructing an MDX copy. 6. OpenAPI or a frozen JSON schema fixture for CL8Y-web. Do **not** put Replicate/Telegram tokens on public routes. --- ## Acceptance criteria - [ ] Public `GET /v1/posts` lists the seeded `cl8y-roadmap-cmm-bridge-yieldomega` with required meta (no `wordCount` authored in MDX; numeric `wordCount` in JSON is computed). - [ ] `GET /v1/posts/cl8y-roadmap-cmm-bridge-yieldomega` returns body equivalent to the current MDX (modulo computed fields). - [ ] `GET /v1/images/blog/cl8y-roadmap-cmm-bridge-yieldomega-hero.jpg` returns a JPEG (`image/jpeg`). - [ ] `GET /v1/rss.xml` includes that item with title, description, `/blog/<slug>` link (canonical host configurable). - [ ] Draft pipeline output is **not** on public GET until `publish`. - [ ] Missing slug → 404. Unpublished slug → 404 (not 401). - [ ] CORS allowlist; unknown origin rejected. - [ ] README documents import + publish + local `curl` against compose. - [ ] CL8Y-web consumer issue can be unblocked when `/health` is 200 and the seeded post is listed. --- ## Test plan — functional paths 1. **Import seed** — Point import at a fixture copy of the roadmap MDX + hero. Assert published row, slug, tags, JPEG bytes, computed wordCount > 0. 2. **List / get / RSS / hero** — HTTP tests (axum test or `reqwest` to bound server). 3. **Draft isolation** — Insert draft slug `weekly-secret`; public GET list/get/RSS omit it; 404 on get. 4. **Publish gate** — Dirty MDX (em dash, unofficial address) cannot publish; clean draft can. 5. **Slug uniqueness** — Second import of same slug does not duplicate; collision policy documented. 6. **CORS** — `Origin: https://cl8y.com` allowed; `Origin: https://evil.test` not. 7. **Health** — `/health` 200 without DB; `/ready` 200 only with DB (or document if combined). --- ## Test plan — attack, hack, and abuse 1. **Unpublished leak** — Guessable draft slug must 404. No `status` filter query that returns drafts. 2. **Path traversal** — `GET /v1/images/blog/../../.env` rejected. 3. **XSS in JSON** — Body may contain Markdown; JSON must not include `<script>` that sanitize would have stripped; Content-Type `application/json` (not HTML) for post GET. 4. **RSS injection** — Title/description XML-escaped; no `javascript:` links. 5. **Secret exfil** — Seed/post body with `REPLICATE_API_TOKEN=` fails publish. 6. **Cache poisoning / host header** — Canonical links in RSS use configured public origin, not `Host` from the client. 7. **Rate / flood** — Document or implement a GET rate limit; test that the limiter does not skip `/health`. 8. **Overwrite production** — Import must not clobber a newer published body without an explicit `--force`. --- ## Verification criteria A reviewer can: 1. `docker compose up -d` + migrate + `import-web` from a CL8Y-web checkout (or fixture). 2. `curl` list/get/RSS/hero and see the 2026-05-10 roadmap post. 3. Confirm a new `week` draft is absent from public GET until publish. 4. Hand the public base URL to the CL8Y-web issue as the unblock signal.
PlasticDigits commented 2026-08-31 05:32:57 +00:00 (Migrated from gitlab.com)
Copy link

mentioned in issue #4

mentioned in issue #4
PlasticDigits commented 2026-08-31 05:34:25 +00:00 (Migrated from gitlab.com)
Copy link

mentioned in issue CL8Y-web#7

mentioned in issue CL8Y-web#7
PlasticDigits commented 2026-08-31 05:34:57 +00:00 (Migrated from gitlab.com)
Copy link

marked as related to CL8Y-web#7

marked as related to CL8Y-web#7
PlasticDigits commented 2026-08-31 05:34:58 +00:00 (Migrated from gitlab.com)
Copy link

marked as related to #7

marked as related to #7
PlasticDigits commented 2026-08-31 05:34:59 +00:00 (Migrated from gitlab.com)
Copy link

marked as related to #9

marked as related to #9
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
code/cl8y-research#2
No description provided.