Proposal: expand `/writings` into a long-form drafting pipeline¶

Status: draft (logged at Low / phased priority per operator; spare-cycles work, post-July-15 arXiv deadline LIS-186)

GitHub issue: #258

Author: operator (issue spec) + Claude (this writeup)

Date: 2026-05-25

Problem¶

The bench's current /writings route is a draft list + create button. That covers a small fraction of what a "drafting pipeline" needs:

No stage progression (everything is draft or published; the operator's actual flow is idea → outline → draft → ready → published)
No live Markdown preview (just a textarea)
No export-for-Substack / Alignment Forum / LessWrong (operator manually copies markdown each time)
No source-link from /news (when that ships) into a draft
No structure-or-clarity AI passes

The operator's goal: one quality long-form piece per week, routed by content type: - Research reflections → AF / LW (credentialing for the research-engineer pivot) - Methods explainers → Substack - News responses → Substack - Learning logs / tutorials → personal blog or kept private (not on AF/LW)

Hard principle (non-negotiable)¶

Sonnet assists structure and clarity only (outline, plain-language pass, tighten, titles) and never writes the substance.

The pivot's credibility depends on this. AI-generated takes from a sycophancy researcher are self-undermining. The proposal preserves this with explicit guardrails on every AI-touching surface:

Sonnet outputs are always suggestions rendered in a sidebar, never inline edits
Sonnet refuses to "write a draft from this title" — only restructures, tightens, or proposes alternative phrasings of operator-written prose
Sonnet refuses to fabricate hooks, citations, or claims; if the operator hasn't written them, the model says "Add the operator's source/claim here before I can help"

Proposed design¶

(From #258. Highlights here; the GH issue body is canonical.)

Data model¶

piece (
  id              UUID PK
  user_id         UUID FK → auth.users
  type            TEXT  -- 'research_reflection' | 'methods_explainer'
                          | 'news_response' | 'learning_log' | 'tutorial'
  venue           TEXT  -- 'substack' | 'lesswrong' | 'alignment_forum'
                          | 'personal' | null (= undecided)
  stage           TEXT  -- 'idea' | 'outline' | 'draft' | 'ready' | 'published'
  title           TEXT
  body_md         TEXT  -- the writing itself; large
  source_ref      JSONB  -- {kind: 'news_item' | 'attempt' | ...,  id, title, url}
                          -- so /news items promote with their context intact
  word_count      INTEGER  -- computed on save
  created_at      TIMESTAMPTZ
  updated_at      TIMESTAMPTZ
  published_at    TIMESTAMPTZ  -- null until published
  published_url   TEXT  -- where the published piece lives, set post-publish
)

Routes (phased)¶

Phase 1 (MVP): - GET /writings — pipeline board (kanban-style: idea / outline / draft / ready / published columns) - GET /writings/{id} — split-pane editor: markdown left, live GFM preview right. Autosave. - POST /writings/{id}/stage — advance stage with audit-trail entry - GET /writings/{id}/export?venue=substack — clean Markdown copy-paste output with venue-specific formatting (e.g., LW collapsible-block syntax for "Epistemic status") - GET /writings/{id}/export?venue=lesswrong — same shape, different syntax tweaks - Deep links to open Substack/LW editor URLs pre-populated where possible (Substack supports ?title=...&body=... query params on /publish)

Phase 2: - /news integration: each news item gets a "Draft a response" CTA that creates an idea piece with source_ref populated and the news headline pre-filled as title - Sonnet structure/clarity passes (button: "Suggest tighter outline", "Pass for plain language", "Propose 3 titles"). All output rendered in a sidebar, never inline. Each suggestion has explicit "Apply" and "Discard" buttons

Phase 3 (optional, gated): - Experimental Substack cookie-auth cross-post, scoped to Substack Notes only (not full posts). Loud fragility warning in the UI: "Substack has no official publish API. This uses unofficial endpoints that can break without notice. Verify the post landed before relying on it."

Why no full Substack publish API¶

The Substack Developer API as of writing reads public profile data only — no publish endpoint. Programmatic full-post publishing requires cookie-auth against private endpoints, which is fragile and ToS-adjacent. Phase 1 ships export + deep-links; the human is always the last click before publish.

Strategy ordering (operator-stated)¶

Content type	Venue	Priority
Research reflection	AF / LW	highest — credentialing, seeds arXiv framing
Methods explainer	Substack	high — production-eval depth
News response	Substack	medium — cadence keeper
Learning log / tutorial	personal	low — keep OFF AF/LW

The pipeline board sorts by type priority within each stage so the operator naturally drafts the highest-leverage pieces first.

What this proposal adds on top of the GH issue¶

Three technical follow-ups to lock at design-review time:

1. Reuse the existing `writings` table; don't introduce a new `piece` table¶

The current schema has writings (id, user_id, title, body_md, source_refs_json, status, word_count, created_at, updated_at, published_at). Migrating to a new piece table would require a data backfill that's expensive for marginal naming wins.

Recommend: rename status to stage and broaden its value set (idea / outline / draft / ready / published / archived). Add type + venue + published_url as new nullable columns. The schema diff is a single migration with three ADD COLUMN statements + a check constraint update on stage.

The existing writings_repo keeps its name; the existing /writings and /writings/{id} route paths keep their public URLs. The visible change is the editor UI + the kanban board.

2. Sonnet guardrail = hard constraint in the prompt, not a soft instruction¶

The "never writes substance" principle is too important to rely on prompt soft-instructions. Build it in:

No "generate draft" endpoint at all. The route literally doesn't exist; there's no API surface for "give me a draft from this title."
Sonnet system prompts reject substance generation. If a clarity-pass request includes a paragraph that's empty or just [TODO], Sonnet returns "I see a [TODO] here. Add your draft text and I'll help tighten it." It does NOT fill in the TODO.
All Sonnet outputs render in a sidebar with explicit Apply/Discard. No inline edits. The operator's text is never mutated automatically.
Audit trail: every applied Sonnet suggestion records what was suggested and what was kept. The operator can review "did I let the model rewrite something it shouldn't have?" — and so could a fellowship reviewer if they ever wanted to verify.

This isn't paranoia. The operator is making a credibility argument with the writing; the safeguard infrastructure becomes part of the argument itself ("here's how I made sure the takes were mine").

3. RLS posture (same pattern as migration 010)¶

The schema changes touch a table that already has RLS enabled (migration 010 covered writings). New columns inherit the existing writings_owner_only policy automatically — no policy migration needed. Just verify post-schema-change that the policy still applies.

Open decisions¶

Phase 3 (cookie-auth cross-post): ship or skip?
Recommend skip. Substack's no-publish-API state isn't a bench problem to solve. The operator's last click before publish is fine. If Phase 3 ships, the fragility warning becomes a UI maintenance burden — "did this break since last week?"
Pipeline board: kanban or flat-list-with-stage-filter?
Recommend kanban for the first cut. Drag-and-drop between stages is concrete and the visual progression matches the operator's mental model. If it feels heavy, fall back to a flat list with stage chips.
Where does Sonnet's API spend live?
The bench has spend_cap already. Tag writing-pipeline spend with kind="writing_assist" so daily/weekly caps can target it specifically. The operator can budget "$2/week for writing assists" independently of Reflect generation cost.
News integration timing: in Phase 1 or Phase 2?
Phase 2 (as written in the issue). The pipeline board + editor + export are the core value. /news itself is on the roadmap as a separate piece and doesn't exist yet. Coupling Phase 1 to /news means Phase 1 blocks on /news shipping.

Scope estimate¶

Phase	Files	Lines	PRs
Phase 1: schema diff + kanban + editor + preview + export	~12-15	~1500-2000	1-2
Phase 2: /news integration + Sonnet structure passes	+5-7	+500-800	+1
Phase 3: Substack Notes cross-post (optional)	+3	+300-500	+1

Migration 011 (writings_stage_and_type.sql) is CODEOWNERS — three ADD COLUMN + one check constraint update. Cleanest done as a fresh operator-authored migration.

The kanban board is the largest concrete UI surface in the bench so far. Worth budgeting accordingly when picked up.

Why later, not now¶

Operator explicitly logged Low / phased with the same trigger as /connecting: capstone arXiv submission deadline (LIS-186) on 2026-07-15. Phase 1 alone is a 1500-2000 LOC build with a real UI scope — that's a 1-2 week investment that competes directly with capstone hours.

The operator can start the writing habit now in any Markdown editor — Obsidian, iA Writer, even VS Code. The bench's pipeline is an accelerant for a habit, not a prerequisite to it.

Trigger to ship: capstone arXiv submission landed (post-2026-07-15), AND the operator notices the friction of "where's my draft, what stage was it in, how do I export it" is actually slowing the writing cadence.

Acceptance criteria (when picked up)¶

Phase 1¶

Pipeline board renders all stages with drag-and-drop (or stage chips if drag-and-drop is deferred)
Editor renders Markdown + live GFM preview side-by-side
Autosave within 2s of last keystroke; "saved" indicator
Export buttons produce clean Markdown for Substack + LW + AF + personal (4 venue-tagged outputs)
Deep links open the relevant publish surface with title pre-populated where the venue supports it
Stage transitions logged in an audit table (or denormalized into piece.stage_history_json)
Sonnet guardrails in place: no "generate draft" endpoint exists; clarity-pass refuses to fill [TODO]s

Phase 2¶

/news items each have a "Draft a response" button that creates an idea piece with source_ref populated
Sonnet structure-pass / clarity-pass / titles-pass each rendered in sidebar with Apply/Discard
Audit trail records suggested → applied diffs

Phase 3 (if shipped)¶

Cookie-auth cross-post to Substack Notes only, gated by a feature flag
Loud UI warning: "Unofficial endpoint. Verify before relying."
Failure cases (cookie expired, endpoint changed) surface clearly, not silently swallowed

Proposal: expand /writings into a long-form drafting pipeline¶