The AI Learning Hub Journal

Why It Makes Things Up

Why it says wrong things so convincinglythe confident tone is not evidence — it is just how the words came outYou ask it a questionIt picks a likely next piece, then another, then anotherThere is no separate step where it checks whether what it is saying is true,and no internal flag that lights up when it has started guessingFLUENT — AND IT HAPPENS TO BE RIGHT“Canberra is the capital of Australia.”The likely continuation also matchedthe way the world actually is.CONFIDENCE IN THE WORDINGRight by overlap, not by checkingFLUENT — AND IT IS SIMPLY WRONG“Her second novel opens with that line.”The likely continuation had nothingreal sitting behind it this time.CONFIDENCE IN THE WORDINGWrong, and worded exactly the sameSame process, same tone, same certainty — the difference is not visible in the textWHAT ACTUALLY TELLS YOU WHICH ONE YOU GOTCheck it elsewhereagainst something that isnot the same modelAsk for the sourcethen actually open it — alink can be invented tooNotice the topicniche, recent or very exactfacts are the risky onesTreat tone as styleconfidence and hedging arewriting choices, not proofThere is no moment where it knows the answer and decides to give you a different one
Nothing in how an answer is worded tells you whether it is true — the wording was chosen by the same process either way

Hallucination Is the System Working as Designed

When a model invents a quotation, a statistic, a court case or a book that does not exist, that is called a hallucination. It is tempting to read it as the model lying, but nothing has gone wrong internally. The model was asked to produce the most plausible continuation and it did. A fake citation looks exactly like a real one — author, year, plausible title — because it was generated from the pattern of real citations. There is no separate fact-checking stage inside the model that could have caught it. Truth is not a variable the system tracks; plausibility is.

  • Made-up references, dates and quotes are the classic failure, and they look completely normal
  • The model cannot flag them because it has no independent record of what is true
  • Risk is highest for specific, verifiable, rarely-written-about details
  • Risk is lowest for restructuring or explaining material you supplied yourself

Confidence Is a Writing Style, Not a Signal

Human writers usually hedge when they are unsure. That correlation makes us read confident prose as informed prose, and models write in confident prose almost always, because most of the text they learned from was written by people who were confident. So the usual social cue you rely on — does this person sound sure? — has been decoupled from whether the content is right. Some systems now express uncertainty more, but you should never treat tone as evidence. A fabricated answer and a correct answer arrive with identical body language.

  • Fluency and confidence tell you nothing about accuracy — they are style, not evidence
  • Asking "are you sure?" often produces a confident correction rather than a genuine confidence estimate
  • It may also cave and agree with you when you were wrong, because agreement is a common pattern too
  • Trust verification, not tone

Where the Danger Actually Sits for You

The realistic risk is not a wild, obvious lie. It is a small, confident, wrong detail sitting inside four correct paragraphs, in a subject you do not know well enough to notice. That is exactly the situation you are in when you use AI for homework in a topic you are still learning. The defence is not paranoia, it is proportion: verify anything that carries a name, a number, a date, a source or a formula, and relax about phrasing, structure and explanation. Ask yourself what happens if this specific claim is wrong. If the answer is "I lose marks" or "I mislead someone", check it.

  • Highest risk: named sources, statistics, dates, legal or medical specifics, anything niche
  • Lower risk: rewording your own work, generating practice questions, explaining a concept several ways
  • Ask the model to show its reasoning — errors are easier to spot in steps than in conclusions
  • If two independent sources cannot confirm it, do not put it in your work

Try It Yourself

Reading about hallucination is not the same as catching one. The quickest way to see it is to ask about something you happen to know better than the model does.

◆ Try it yourself

Pick a subject you know inside out — your team's last season, a game you play properly, a band, the streets near you. Ask any AI chat tool a detailed question about it, one whose answer has to contain names, dates or numbers. Then read the reply as the expert you are and mark it.

Tell me about [your specific topic]. Be detailed and specific — include names, dates and numbers.
How you'll know it worked
  • At least one detail came back wrong, and you knew it was wrong without looking anything up
  • The wrong part was written in exactly the same confident tone as the correct parts
  • The errors clustered in the most specific claims — the names, numbers and dates
◆ See it for yourself
Open the Temperature dial in the library →

Turn sampling up and down and watch the output change.

Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.