The AI Learning Hub Journal

"It Worked When I Tried It"

"It worked when I tried it" — the most dangerous sentence hereyou tried it once, under ideal conditions — that is a single observation, not evidence it worksYOUR ONE TRY — THE GREEN TICKonce, on your machine, withyour data, on your network,in the way you had in mindproof it can work — not proof it worksgenerated code handles the intended pathwell, and every other path thinlyWHAT YOU HAVE NOT TRIED YET — FIVE MINUTES OF AWKWARD CASESsubmit the form emptya name in the email boxpress the button twice, fastrefresh halfway throughuse it on a phonea paragraph in a one-word boxfresh browser, not signed ingo back and submit againTIME AND REPETITION FIND MORErun your list twice in a row — second tries fail because something is left over from the firstrun it again tomorrow, and with more data — things expire overnight, and lists slow down as they grow"I HAVE SEEN IT SUCCEED"one observation under ideal conditions— capability"IT WORKS FOR OTHER PEOPLE"awkward cases, twice, tomorrow, someone else— reliabilitymost prototype disappointment lives in the gap between these two sentencesONE GREEN RUN IS A START, NOT A VERDICTfive minutes of awkward cases, a second run, and a next-day run catch most of what a prototype hides
One success under ideal conditions shows the thing can work — trying the awkward cases, twice, and again tomorrow shows it does.

One Success Is Not Evidence

The most dangerous sentence in prototype building is "it worked when I tried it". You tried it once. On your machine, on your network, with your data, in the way you had in mind, at a moment when nothing else was happening. That is a single observation under ideal conditions. It tells you the thing is capable of working, rather than that it does work. This is not a demand for rigour you do not have time for. It is simply noticing the difference between "I have seen it succeed" and "I have reason to believe it succeeds for other people". Those are genuinely different claims, and most prototype disappointment lives in the gap between them.

  • One success under ideal conditions shows capability, not reliability
  • "I saw it work" and "it works for others" are different claims
  • Your machine, data, network and habits are all part of the test conditions
  • Most disappointment lives in the gap between those two sentences

Try the Awkward Cases

Reliability comes from deliberately trying the things you would never do naturally. Submit the form empty. Put a name in the email box. Paste a paragraph into a field expecting a word. Press the button twice quickly. Use it on a phone. Refresh in the middle. Go back and try to submit again. These take five minutes, and they are where prototypes break. Generated software handles the intended path well and the unintended paths thinly. It is also worth trying it as a brand new person: a fresh browser, not signed in, no history. Plenty of prototypes only work for the person who has been using them all week, and nobody notices until a stranger arrives.

  • Empty fields, wrong types, absurd lengths, double clicks, refreshes
  • Generated code handles the intended path well and the rest thinly
  • Try it as a new person: fresh browser, not signed in, no saved data
  • Five minutes of awkward cases finds most of what a prototype hides

The Same Thing, Twice, Tomorrow

Some failures only appear with time or repetition. Something works once and fails on the second attempt, because a value is left over from the first. Something works today and fails tomorrow, because a temporary permission expired overnight. Something works with three records and crawls with three hundred. You cannot check for all of this, and you should not try. But do the cheap version. Run your checklist twice in a row, and run it again the next day before you show anyone. Those two habits catch a surprising share of the failures that otherwise show up during a demo, which is the worst possible time to find them.

  • Run the list twice in a row — second-attempt failures are common
  • Run it again the next day; some things expire overnight
  • Try it with more data than you have been testing with
  • These two cheap habits catch most of the classic demo-day failures

Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.