The AI Learning Hub Journal

Code You Cannot Read, and When to Stop

You have built something you cannot fully check — build accordinglythat is a permanent condition of building this way, not a phase you grow out ofit works — and it may hold mistakes, odd choices and gaps you cannot seereviewing the code is not one of your options — your safety must come from elsewhereMATCH IT TO WHAT YOU CAN SUPERVISEa few users, not the publiclow-value data, nothing sensitiveonly actions you can undonothing automatic that touches a persona small blast radius is the whole strategySTOP AND BRING IN AN ENGINEER WHENmoney moves through itit stores anything you would hate to see publishedpeople outside now depend on itit acts on its own, or a law has been named at youa review is a few hours — far cheaper than what it preventsTHE DRIFT — NOBODY DECIDES TO DEPEND ON A PROTOTYPEa prototype worksmore people use it each monththe business now depends on itNOT SHIPPING IS A REAL ANSWER — OFTEN THE PROFESSIONAL ONEa spent weekend is the worst reason to expose people to risk — the prototype already answered your questionIF YOU CANNOT READ IT, KEEP IT SMALL, SLOW AND SUPERVISEDlimits, caps, checklists and other people testing — those controls work, and they are the only ones you have
You cannot review code you cannot read, so keep its blast radius small — and when money, outsiders or automatic actions arrive, stop and get help.

The Honest Version of the Risk

Here is the part that deserves a clear statement. You have built something functional that you cannot fully evaluate. That is a real and permanent condition of building this way, not a temporary gap that goes away with experience. The code may contain mistakes that only appear under conditions you never tested. It may contain choices you would not have made if you had understood them, and gaps that are invisible from the outside. You cannot review it, so your safety comes from elsewhere. It comes from limiting what the thing is allowed to do, from what you chose not to collect, from caps and checklists and other people testing. Those controls are genuinely effective. They are also the only ones you have.

  • You cannot evaluate what you built — this is a condition, not a phase
  • Problems can exist that are invisible from the outside and untriggered so far
  • Your safety comes from limits and habits, not from reviewing code
  • Those controls work, and they are the only ones you have

What This Means in Practice

It does not mean do not build. It means match what you build to what you can supervise. Small blast radius: few users, low-value data, reversible actions, nothing automatic that affects a person. Keep the thing you cannot read away from the things that would hurt if they went wrong. It also means resisting a specific and very common drift. A prototype works, gets used by more people each month, and slowly becomes load-bearing for a business — without anyone ever deciding it should. Nobody chooses that. It happens by default, one small success at a time, and it is worth naming so you can notice it happening to you.

  • Match what you build to what you can supervise
  • Keep unreadable code away from money, health, safety and automatic actions
  • The common failure is drift: a prototype quietly becoming load-bearing
  • Nobody decides to depend on a prototype; it happens by default

Clear Signals to Bring In an Engineer

Some signals are unambiguous. Write them down in advance, and you will recognise them under pressure. You are storing anything you would hate to see published. Money moves through it. People outside your organisation depend on it for something that matters to them. A regulation has been named at you. It handles data about children or health. It acts on its own, without a person approving. It has become something your business would genuinely struggle without. Any one of these means stop and get a professional to look at it. Not necessarily to rebuild it — often just to review it and tell you what needs attention. That review is cheap compared with what it prevents.

  • Sensitive data, money movement, or outsiders depending on it
  • A named regulation, children's data, or health information
  • Anything that acts automatically without a person approving
  • A review is usually a few hours, and far cheaper than what it prevents

When Not to Ship

Finish with the decision people find hardest. Do not ship it if you cannot say what it stores and who can see it. Do not ship it if the keys are still in the code. Do not ship it if you have no spending caps. Do not ship it if you have not run your checklist as a stranger. Do not ship it if you would be unable to take it down, or to tell affected people, when something goes wrong. And do not ship it just because you have spent a weekend on it. Sunk effort is the worst possible reason to expose other people to risk. Not shipping is a legitimate outcome, and often the professional one. The prototype still did its job: it answered the question. That was always what it was for.

  • Not shipping is a real decision, not a failure of nerve
  • No caps, keys in code, unknown data flows, untested as a stranger: all stop signs
  • Effort already spent is the worst reason to expose other people to risk
  • The prototype answered your question — that was always its actual job

Prefer slides, quizzes, and saved progress? Read this lesson in the library — free, no sign-up.