The AI Learning Hub Journal

When the Model Is Simply Wrong

When no rephrasing will fix itnot every bad output means you prompted badly — knowing the difference is a real skill that saves real hoursOUTSIDE WHAT IT DOES RELIABLY — NO PROMPT FIXES THESElong multi-step arithmetic with no calculator or toolwhat happened recently, or what is true right nowcounting or editing individual letters and charactersfacts that were never written down anywherehere a better prompt improves the presentation, not the correctness — which makes the wrong answer more persuasiveTHE STOPPING RULE — THREE GENUINELY DIFFERENT ATTEMPTSattempt 1 — added contextattempt 2 — task split upattempt 3 — new anglestill wrong? stopthree rephrasings of the same request do not count — each attempt must change something realthen:shrink the tasksupply what was missingswitch to a proper tooldo it yourselfERRORS YOU CANNOT SEEcatching a mistake takes knowledge you may not have — in a new field, everything reads equally plausiblethere, lean on external sources — use the model to learn the shape of a subject, not as the authority on its detailsTHREE DIFFERENT ATTEMPTS, THEN CHANGE SOMETHING STRUCTURALcheap retries feel productive — that is how an hour disappears into a five-minute task
Some tasks sit outside what a model does reliably — after three genuinely different attempts, change something structural instead of the wording.

Not Every Failure Is a Prompting Failure

There is a belief, encouraged by a lot of enthusiastic content, that any bad output means you prompted badly. It is not true, and believing it wastes real time. Some tasks are genuinely outside what a language model does reliably: precise arithmetic over many steps without tools, knowing what happened recently or what is true right now, counting or manipulating individual characters, anything requiring information that was never written down anywhere. In these cases a better prompt improves the presentation of the answer and not its correctness — which is worse than useless, because it makes the wrong answer more persuasive. Knowing the difference between "I asked badly" and "this is not a job for this tool" is a real skill.

  • Multi-step precise arithmetic without a calculator or tool is unreliable
  • Current, live or very recent facts are not something a model knows by itself
  • Information that was never written down cannot be recalled, only invented
  • On these tasks a better prompt improves fluency, not truth

Three Strikes and Change Approach

A practical stopping rule: if three genuinely different attempts have not got you there, stop prompting and change something structural. Different does not mean rephrased — it means you supplied more context, or split the task, or approached it from another angle. Three of those failing is strong evidence the problem is not the wording. At that point the options are: break the task into smaller pieces the model can do, give it the information it was missing, use a tool built for this job, or do it yourself. Continuing to iterate past this point feels productive because each attempt is cheap, and it is the most common way people lose an hour to a task that needed five minutes of a different method.

  • Three genuinely different attempts, not three rephrasings, is the stopping signal
  • Then: shrink the task, supply what was missing, change tool, or do it yourself
  • Cheap retries make the sunk-cost trap easy to fall into
  • Recognising the ceiling early is worth more than one more clever prompt

Errors You Cannot See

One uncomfortable dependency underlies all of this: your ability to catch a mistake scales with how much you already know about the subject. In a field you understand, a wrong claim jumps out. In a field you are new to, everything reads as equally plausible and you cannot supervise the output at all — which is exactly the situation in which people most want to use AI. It is not an argument against using it. It is an argument for being deliberate: in unfamiliar territory, weight external sources more heavily, prefer verifiable claims over confident summaries, and treat AI as a way to learn the shape of a subject rather than as an authority on its details.

  • You cannot verify what you do not understand — beginners face the highest risk
  • In unfamiliar areas, lean on external sources rather than the model's confidence
  • Ask for claims in a checkable form so verification is possible at all
  • Use it to get oriented in a new field; do not use it as the final authority there

Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.