Compare two versions of a prompt properly
Refuses to call a winner from one run each, and notes that easy inputs make both look identical.
| Category | Using AI › Writing prompts |
|---|---|
| Tags | AnalyzingReviewingTable |
I have two versions of a prompt. **Work out which is better and why.** 1. **Diff them.** ***List every difference, including ones that look cosmetic*** — **word order, emphasis, where a rule sits** 2. **Per difference, predict what it changes.** ***Say which ones should change nothing*** — *those are the ones to cut from whichever version wins* 3. **What to test them on** — ***the inputs where they would differ, not typical inputs.* **Two prompts agree on easy cases** 4. **What "better" means here** — ***state the judging criterion before looking at outputs.* **Otherwise I will pick the one that reads nicer** **Then, given the outputs I pasted:** - **Where they actually differed**, and ***which difference in the prompt caused it*** - ***Where the difference is noise*** — **one run each proves nothing. Say how many runs it would take to be sure** - **Whether a third version combining both would be better**, and what it would drop ⚠️ ***Do not declare a winner from a single run each.*** **Say what the evidence supports and what it does not.**
After pasting, fill in the fields at the bottom (Version A · Version B · What each produced)
What this prompt does
The usual mistake is running both on a typical input once. Easy inputs make two prompts agree, and one run cannot separate a real difference from variance.
More in this category
| 4 Optimized Versions of A Prompt (in Arabic) | |
| $500/Hour AI Consultant Prompt | |
| Adaptive Thinking Framework | |
| Agent Organization Expert | |
| AI Kickstart prompt |