AI Tips Ladder · Full Guides

The complete walkthroughs from that post

THU · AUG 06 · ED 582
BEGINNER PROMPTING

01 · Fact-Check Your Draft Before It Ships

A fact-check pass on your own draft is hard because you already believe every sentence in it; splitting the text into atomic claims breaks that fluency and forces each fact to stand alone. You get a ranked shortlist of two or three genuinely shaky items to verify instead of re-reading 1,500 words hoping something jumps out. Use it on anything with numbers, dates, quotes, or attributions before it ships — posts, newsletters, decks, client reports.
  1. 01Open a fresh chat in any chat model (ChatGPT, Claude, Gemini) — a new conversation, so nothing you discussed while writing leaks in and gets treated as established fact.
  2. 02Turn web search OFF for this first pass; you want the model's raw uncertainty, not a confident answer it just laundered through a search result. Most tools expose this as a tools/toggle menu next to the message box; if you can't find it, add 'Do not search the web' to the prompt instead.
  3. 03Paste the prompt below, then paste your draft underneath the ---DRAFT--- marker. If the piece is over ~1,000 words, do it in sections rather than all at once.
  4. 04Read the returned table and ignore everything rated HIGH. Your work list is every LOW row, plus any MEDIUM row whose type is statistic, date, or attribution.
  5. 05Verify each flagged claim against a primary source — the original study, the company's own press release or docs page, the actual transcript — not a blog post citing it. Search the exact figure or quoted phrase in quotation marks.
  6. 06Resolve every flagged claim one of three ways: correct it and save the source URL, soften it to a hedge the source actually supports ('roughly', 'in one 2024 study'), or cut the sentence. Do not leave a claim you couldn't source.
  7. 07Re-run the same prompt on just the rewritten paragraphs to confirm your fixes didn't introduce a new unsupported claim.
  8. 08Keep the source URLs in a comment block or footnote at the bottom of the working doc, so the next editor doesn't re-verify the same facts from scratch.
You are a fact-checking editor. Do not rewrite, improve, or comment on my draft. Split the text below into ATOMIC CLAIMS: each one a single statement that could be independently verified true or false. Break compound sentences apart. Ignore opinions, predictions, jokes, and pure narration — only extract statements of fact about the world. For each claim, output a row in a markdown table with these columns: 1. # 2. CLAIM — the atomic factual statement, in your own words, one sentence 3. TYPE — one of: statistic | date | attribution/quote | named-entity | causal | definition | general 4. CONFIDENCE — HIGH / MEDIUM / LOW, meaning your confidence that this is TRUE AS STATED 5. WHY — under 20 words: what specifically would need checking (the number, the year, who said it, the causal link) 6. HOW TO CHECK — the single best search query or primary source type to confirm it Rules: - Sort the table by confidence, LOW first. - Be harsh. If a number is oddly precise, a date is recent, a quote is attributed to a person, or a claim asserts causation, that alone caps confidence at MEDIUM. - Do NOT use web search. I want your honest uncertainty, not a confirmed answer. - Flag separately, under a heading "UNSUPPORTED IMPLICATIONS", anything the draft implies without stating outright. - End with one line: the 3 claims that would do the most damage if wrong. ---DRAFT--- [paste your draft here]
Watch out: Confidence ranking triages your attention — it does not clear the HIGH rows. Models are most confident about widely-repeated wrong facts, so a familiar-sounding stat can sail through. Spot-check one HIGH row per piece. · If you let it search in the same pass, it will often 'confirm' a claim by finding an SEO article that copied the same error from the same original source. Chase the primary source yourself. · Paste a 4,000-word draft and it will quietly skim the middle and return a plausible-looking short table. Work in ~1,000-word chunks and sanity-check that the claim count matches the density of the section.
↳ original source
INTERMEDIATE TOOLS

02 · Run OpenAI's Open Model On Your Laptop

A 20B-parameter model running locally costs nothing per token, works on a plane, and never sends text to a third party — so it suits bulk drafting, rewriting, classification and anything covering client data, contracts or PII. It speaks the OpenAI API, so existing code switches over by changing two lines (base_url and api_key). Use it for high-volume or sensitive work; keep a frontier API model for tasks needing top-tier reasoning or long context.
  1. 01Download LM Studio for your OS from https://lmstudio.ai/download, install it, and launch it once so it creates its model directory (~/.lmstudio on macOS/Linux).
  2. 02Open a terminal and run `lms --version`. Recent builds ship the CLI with the app; if the command is not found, run `npx lmstudio install-cli` (needs Node) or `~/.lmstudio/bin/lms bootstrap`, then open a NEW terminal and check again.
  3. 03Download the model: `lms get openai/gpt-oss-20b`. It is roughly a 12 GB download; pick the offered quantization (MXFP4 is the default and the one the 16 GB figure assumes). In the GUI the same job is the Discover tab — the magnifier icon in the left sidebar — searching for 'gpt-oss-20b'.
  4. 04Start the API server: `lms server start`. Confirm with `lms server status`, then `curl http://localhost:1234/v1/models` — you should see gpt-oss-20b listed.
  5. 05Load the model into memory with an explicit context size so RAM use is predictable: `lms load openai/gpt-oss-20b --context-length 8192 --gpu max`. Run `lms ps` to see it resident. (Skip this and the server will just-in-time load on the first request, which makes that request slow.)
  6. 06Point any OpenAI SDK at http://localhost:1234/v1 with a dummy API key and model id `openai/gpt-oss-20b` — see the snippet. Streaming, /v1/chat/completions, /v1/completions and /v1/embeddings all work the same way as against api.openai.com.
  7. 07When a call misbehaves, run `lms log stream` in a second terminal — it prints the exact prompt the server received and the tokens it returned, which is faster than guessing at SDK-side errors.
  8. 08Free the RAM when you're done: `lms unload --all` and `lms server stop`. Quitting the LM Studio app also stops the server, so leave it running (or use `lms daemon`) if a script needs the endpoint later.
# pip install openai from openai import OpenAI client = OpenAI( base_url="http://localhost:1234/v1", api_key="lm-studio", # unused locally, but must be a non-empty string ) resp = client.chat.completions.create( model="openai/gpt-oss-20b", messages=[ # gpt-oss reads a reasoning level from the system message: low | medium | high {"role": "system", "content": "You are a concise business editor.\nReasoning: low"}, {"role": "user", "content": "Draft a 120-word internal note explaining why we are moving invoicing to net-30 terms."}, ], temperature=0.7, ) print(resp.choices[0].message.content)
Watch out: The ~16 GB is FREE RAM, not installed RAM. On a 16 GB machine with a browser open the model will swap and crawl; close things first, or drop --context-length to 4096 — context is what silently balloons memory beyond the weights. · gpt-oss is a reasoning model: it emits an internal analysis channel before the answer. If raw chain-of-thought leaks into your output, set 'Reasoning: low' in the system message and read only message.content, not the reasoning field. · The server listens on localhost only and dies when you quit LM Studio — a cron job or another machine hitting it will fail silently. For unattended use start it from the CLI and check `lms server status` first. Also, port 1234 is a common default; if it's taken, start with `lms server start --port 1235` and update base_url.
↳ tool: LM Studio · official docs · original source
ADVANCED PROMPTING

03 · Make The Model Grade Its Own Output

A rubric pass catches the specific weak spots in a draft — the vague opening, the unsupported claim — without you re-reading every line, and it turns "make it better" into a targeted rewrite of only the parts that failed. Use it on anything with a repeatable quality bar you'd otherwise eyeball: outreach emails, landing copy, job descriptions, PRD sections, weekly reports. Skip it for short throwaway text, where writing the rubric costs more than reading the draft.
  1. 01Write the rubric before you generate anything: 4-6 named criteria scored 1-10, and for each one a sentence describing what a 10 actually looks like for this piece (e.g. 'Specificity: every claim names a number, product, or date'). Vague criteria produce vague scores.
  2. 02Open a new chat at chatgpt.com, pick a reasoning-capable model from the model dropdown at the top of the chat window, and generate the draft the way you normally would.
  3. 03In the SAME thread — do not start a new chat, the draft must stay in context — paste the scoring prompt from the snippet below, with your criteria filled in.
  4. 04Spot-check the scoring before you trust it: pick the two highest scores and confirm the quoted evidence actually appears in the draft. If the model scored everything 8+, reply 'You scored too generously. Re-score with at least two criteria below 8 and quote the exact weakest line for each.'
  5. 05Ask for the surgical rewrite: 'Rewrite ONLY the parts responsible for the criteria scoring under 8. Reproduce every other sentence verbatim, unchanged. Then list what you changed and which criterion each change targeted.'
  6. 06Re-run the scoring prompt once on the rewritten version to confirm the low scores moved and nothing that scored 9 dropped. Stop after this round — a third pass typically shuffles wording without raising quality.
  7. 07Save the rubric and scoring prompt for reuse: paste them into a ChatGPT Project's instructions field (create a Project in the left sidebar, then open its instructions), or into Settings → Personalization → Custom instructions if you want them applied to every chat. Keeping them in a plain text file you paste in also works and avoids affecting unrelated chats.
You are grading the draft you just produced above. Do not rewrite anything yet. Score it against this rubric, 1-10 per criterion: 1. <criterion name> — a 10 means: <what excellent looks like here> 2. <criterion name> — a 10 means: <...> 3. <criterion name> — a 10 means: <...> 4. <criterion name> — a 10 means: <...> Rules: - Output a table with columns: Criterion | Score | Exact quote from the draft that justifies the score | What a 10 would require instead. - The quote must be copied verbatim from the draft. If you cannot find a supporting quote, score it 5 or lower. - Be a harsh grader. 8 means "a demanding reader would not change this." At least one criterion must score below 8. - After the table, list the criteria scoring under 8, in priority order. - Do not rewrite the draft in this message.
Watch out: Models grade their own work generously. Without the "at least one below 8" rule and the verbatim-quote requirement, you get a wall of 8s and 9s and the whole pass is theater. · Rewrites leak. Asking for a fix often returns a fully regenerated draft where the parts that scored 9 quietly got worse — always demand the untouched sections be reproduced verbatim, and diff the result against your original. · In a long thread the model may score its own summary of the draft rather than the draft itself. If the quotes in the table don't match your text word-for-word, re-paste the draft explicitly above the scoring prompt.
↳ tool: ChatGPT · official docs
PRO AGENTS

04 · Route Cheap, Escalate Only When Needed

Classification, extraction, and routing steps rarely need a frontier model — they need a correct label from a fixed set, which a small model does at roughly a tenth the token price. Routing those steps to Haiku 4.5 and keeping Opus 5 for reasoning, planning, and code generation typically cuts agent spend substantially with no user-visible quality change, and per-step tags in AI Gateway show you exactly where the money went. Use this once an agent has more than two or three LLM calls per run and you can measure quality with an eval set.
  1. 01Inventory every model call: run `grep -rn "generateText\|streamText\|generateObject\|generateImage" src/ --include=*.ts` and write each call site into a table with columns: step name, what it outputs, cheap or hard. Label a step cheap only if its output is a label, an enum, a boolean, a field extraction, or a short rewrite; label it hard if it plans, reasons over multiple facts, writes code, or produces the user-facing final answer.
  2. 02Link the project and turn on AI Gateway: run `vercel link`, then open `https://vercel.com/{team}/{project}/settings` and search settings for "AI Gateway" to enable it, then run `vercel env pull .env.local` to provision `VERCEL_OIDC_TOKEN` (no provider API keys needed). Install with `pnpm add ai@^6`.
  3. 03Confirm the exact model slugs before hardcoding them — run a one-off `node -e "import('ai').then(async ({gateway}) => console.log((await gateway.getAvailableModels()).map(m => m.id).filter(id => id.startsWith('anthropic/'))))"` and copy the real IDs for Haiku and Opus from the output. Slugs use dots for versions (`anthropic/claude-haiku-4.5`), not hyphens.
  4. 04Create `lib/models.ts` as the single place that maps a tier to a model, tags every call with its step name, and exposes a cheap-first helper with an escalation path (see snippet). Nothing else in the codebase should contain a model string.
  5. 05Convert the call sites: replace each hardcoded `model:` with `...pick('cheap', 'step-name', userId)` or `...pick('hard', 'step-name', userId)` per your table from step 1. Start with the steps you labeled cheap that run most often — those pay back first.
  6. 06Make escalation checkable, not vibes-based: gate the retry on a hard signal you can assert — a Zod parse failure, an enum value of `other`/`unknown`, a required field coming back empty, or a downstream tool rejecting the argument. Log every escalation with the step name so you can count them.
  7. 07Re-run your eval set against both configs and compare: `pnpm eval > eval-cheap.txt` on the new branch and `git stash && pnpm eval > eval-baseline.txt` on the old one, then diff per-step accuracy. Ship only if each cheap step's accuracy is within your tolerance (commonly 1-2 points) of the Opus baseline; promote any step that regresses back to hard.
  8. 08Verify the savings and put a floor under them: open `https://vercel.com/{team}/{project}/ai` and filter the logs by your `step:` tags to see spend and token counts per step, then in AI Gateway settings set a monthly budget alert and a per-user token cap so a regression or abuse spike surfaces as an alert instead of an invoice.
// lib/models.ts — the only file in the repo that names a model import { gateway } from 'ai' // Verify these against gateway.getAvailableModels() before shipping. export const CHEAP = 'anthropic/claude-haiku-4.5' // classify, extract, route export const HARD = 'anthropic/claude-opus-5' // reason, plan, write code export function pick(tier: 'cheap' | 'hard', step: string, userId?: string) { return { model: gateway(tier === 'cheap' ? CHEAP : HARD), providerOptions: { gateway: { user: userId, // per-user usage + rate limiting tags: [`step:${step}`, `tier:${tier}`], // per-step cost attribution // Provider-outage fallback only — NOT a quality escalation: models: tier === 'cheap' ? [HARD] : [], }, }, } } // lib/route.ts — cheap first, escalate on a checkable failure import { generateObject } from 'ai' import { z } from 'zod' import { pick } from './models' const Route = z.object({ intent: z.enum(['billing', 'support', 'sales', 'unknown']), reason: z.string().min(1), }) export async function classifyIntent(text: string, userId: string) { try { const cheap = await generateObject({ ...pick('cheap', 'classify-intent', userId), schema: Route, prompt: text, }) // Escalate only on a hard signal: the model could not place it in a real bucket. if (cheap.object.intent !== 'unknown') { return { ...cheap.object, escalated: false, usage: cheap.usage } } } catch (err) { // Schema parse / validation failure is also a hard signal — fall through. console.warn('cheap classify failed, escalating', err) } const hard = await generateObject({ ...pick('hard', 'classify-intent-escalated', userId), schema: Route, prompt: text, }) return { ...hard.object, escalated: true, usage: hard.usage } }
Watch out: Never escalate on a self-reported confidence score — small models are badly calibrated and will hand you a confident 0.9 on a wrong label. Gate on something you can assert yourself: schema parse failure, an `unknown` enum value, an empty required field, or the downstream tool rejecting the argument. · Watch your escalation rate. If more than ~25-30% of a step's calls fall through to Opus you are now paying for two calls instead of one and have made that step more expensive, not cheaper — count escalations per step from the `step:` tag and either fix the cheap prompt or move the step permanently to hard. · The gateway's `models: [...]` fallback list fires on provider errors (429/503/outage), not on bad answers — it will not rescue a wrong classification. Quality escalation has to be your own retry, as in the snippet. Also note that a long shared system prompt or a big tool schema dominates input tokens on every call, so a cheap step carrying the full agent context saves far less than the sticker price suggests; trim the prompt for cheap steps and confirm the savings in the per-step token counts.
↳ tool: Vercel AI Gateway · official docs
🛠️ We build the AI systems & automations behind tips like these.
Follow @conon.ai for the daily brief — news + tips, every day.
@conon.ai · automated daily AI brief