The demo always works. You show the client a lead coming in, the AI scoring it, the response firing, the CRM updating. Everything runs cleanly. They nod. They sign off. You deploy.
Three weeks later, something breaks. Not dramatically. Usually quietly. A webhook stops firing because the third-party API changed its response format. A prompt that worked perfectly in testing starts producing inconsistent output when the real data arrives with messiness that the test data never had. A workflow that ran fine for a hundred leads hits an edge case on lead number 101 and fails silently, writing nothing to the database, alerting no one.
This is the gap between AI automations that work in demos and AI automations that survive in production. Most people building these systems are optimising for the demo. The production failure modes are different problems entirely, and they need to be designed around from the start.
Here is what actually goes wrong, and what I do differently.
The inputs are never as clean as the test data
The first thing that kills a production automation is input assumptions. You build the workflow around a Typeform response that looks like: "I run a coaching business, I have 30 clients, I need help with scheduling." Clean, structured, intentional.
Real users send: "hi i saw ur post can u help me with stuff for my business maybe scheduling or whatever". Same intent. Completely different shape.
A rules-based classifier fails here. An LLM handles it, but only if you have designed the prompt to handle ambiguity rather than expecting clean input. Every system I build is tested against deliberately messy inputs: incomplete sentences, missing fields, conflicting information, multiple questions in one message. If the AI cannot handle those, the system is not ready.
"The automation that breaks is never the one you tested. It is the one you assumed would never happen."
Beyond the LLM layer, every other node in the workflow needs to handle unexpected input gracefully. What happens when a webhook fires with a missing field? What happens when an API returns a 429 instead of a 200? Most workflows I have seen in the wild just error out. A production-grade workflow catches those cases and routes them somewhere useful: a fallback value, a human escalation queue, a logged alert.
No monitoring means no visibility when things go wrong
The second failure mode is invisible breakage. The workflow stops doing what it should and nobody finds out for days because there is no alerting. The business owner notices when a client mentions they never received a follow-up. By then, ten other leads have had the same silent failure.
A system without monitoring is not a finished system. This is one of the first things I establish with clients before anything is built. Monitoring is not an optional add-on. It is part of the architecture.
In practice this means: every workflow that touches a critical path needs an error handler. Every error handler needs to route somewhere a human will actually see. For most clients, that is a Slack channel. A critical path failure at 2am fires a Slack alert. Someone sees it in the morning. The issue gets fixed before the client's day starts.
I also build execution logs into every production workflow. Every run writes a record: timestamp, inputs received, output produced, success or failure. Those logs sit in Airtable or a Google Sheet and can be reviewed at any time. If a client says "something felt off last Tuesday," I can pull the log for last Tuesday and see exactly what happened.
Prompts that work in testing drift in production
LLM behaviour is not deterministic. A prompt that produces excellent output in 95% of cases will produce unreliable output in the other 5%. At small volume, you never see the 5%. At production volume, you see it constantly.
The solution is not to find the perfect prompt. The solution is to build validation after the prompt. Every AI-generated output in my workflows passes through a checking node before it is acted on. That node looks for the expected structure, the required fields, the output format the next step depends on. If the validation fails, the workflow does not continue. It routes to a fallback.
What the fallback looks like depends on the use case. For a lead scoring system, a fallback might be a default score of "medium" and a human review flag. For an email generation workflow, it might be a template email instead of the AI-generated one. The point is that the system degrades gracefully rather than producing garbage downstream.
No human escalation path
The third failure mode is designing a system that assumes the AI will always be right. It will not. An AI qualification system will occasionally misread a message. A voice agent will occasionally encounter a caller with a question outside its training. A support bot will occasionally face a complaint that needs a human response.
Every system I build has a defined escalation path for the cases the AI cannot handle. The threshold is set deliberately, not generically. For a real estate lead system I built, the AI escalates any lead it scores below 6 on confidence, not just below 6 on quality. There is a difference. A lead can be clearly high quality but written in a way the AI is uncertain about. That uncertainty is the trigger for human review, not the final classification.
"The system needs to know what it does not know. That is a design decision, not an AI capability."
The escalation needs to go somewhere real. A flag in a database nobody checks is not an escalation path. A Slack message to the person who will actually act on it is.
Built to demo, not to run
The last failure mode is architectural. Systems built for demos are built to show a happy path running cleanly. Systems built for production are built assuming the happy path will be interrupted constantly, and designing every interrupt to be recoverable.
This means things like: idempotency on webhook handlers so duplicate triggers do not create duplicate records. Rate limit handling on API calls so a burst of inbound leads does not blow through a third-party API's limits and start failing silently. Retry logic on transient failures so a temporary network issue does not cause a permanent data loss.
None of this is visible in a demo. All of it is the difference between a system a business can depend on and one that runs fine for three weeks and then quietly starts missing things.
What I do instead
Every production system I deliver has the same baseline: error handlers on every critical node, Slack alerting for failures, execution logs for auditability, output validation after every AI step, a defined human escalation path with a real destination, and a 30-day post-launch support window where I watch the logs and fix anything that surfaces under real conditions.
That last part matters. The first 30 days of a production system are when most of the edge cases appear. Real data is always messier than test data. Real users always do things you did not anticipate. Building systems that degrade gracefully and alert quickly is what makes the difference between automation that earns trust and automation that gets turned off after the first failure.
The demo is the easy part. Production is the work.