Every few months, another company announces they ran an AI pilot. Lots of excitement, a decent budget, a few months of tinkering. Then quietly, nothing ships. The project gets shelved, or it limps along producing results nobody trusts.
People blame the model. They blame the vendor. They blame the team for not being “AI-ready.”
I’d argue most of them are looking in the wrong place.
The Real Problem Has Nothing to Do With AI
Here’s something I’ve seen enough times to stop being surprised by it: organizations don’t actually know how their own work gets done.
Not really. Not in a way they could write down.
They know it works. People show up, things happen, outcomes come out the other end. But ask someone to document the exact steps, the edge cases, the unspoken rules, the judgment calls made at 4pm on a Tuesday? That’s where it gets uncomfortable.
Humans pick this stuff up over months, sometimes years. A new hire watches, asks questions, makes mistakes, gets corrected. The knowledge lives in the room. In Slack threads. In the heads of the three people who’ve been around the longest. Nobody ever needed to write it all down because people are surprisingly good at learning things implicitly.
Agents are not.
What Agents Actually Need
An AI agent doesn’t have the luxury of sitting next to someone for six months. It needs the workflow spelled out. Completely. What triggers it, what the inputs look like, what good output looks like, what to do when something doesn’t fit neatly into the expected shape.
Leave any of that fuzzy and the agent will fill the gaps, usually in ways that seem reasonable on paper and fall apart in practice.
This isn’t a limitation that’s going away with the next model release. It’s structural. You’re taking something that lived in people’s heads and habits, and trying to run it in a system that requires it to be explicit. That translation has to happen. Someone has to do that work.
Most organizations skip it. Not out of laziness, but because they genuinely don’t realize how much implicit knowledge they’re sitting on.
The Pilot Starts Before You Touch the AI
Here’s what I think is underappreciated: the process of decomposing your workflows for an agent is valuable in itself, completely separate from whether the AI ever runs correctly.
When you’re forced to write down what your team actually does, step by step, you find things. Bottlenecks nobody named before. Steps that exist only because of how a tool worked five years ago. Decisions that three different people make three different ways depending on the day.
That clarity is useful no matter what. Some organizations come out of a failed AI pilot with the most accurate picture of their operations they’ve ever had. That’s not nothing.
But it’s also why bringing in an outside team sometimes works better than running the pilot internally. Not because outside teams are smarter. It’s that they ask the questions your people stopped asking a long time ago. “Why does this step happen here?” “What would happen if we skipped it?” “Who actually decides this?” Those questions feel obvious from the outside and almost rude from the inside.
Most of What You Build Won’t Ship. That’s Fine.
One more thing worth sitting with: in AI development, the real intellectual property isn’t the code you write or the agent you build. It’s what you learned trying.
Most experiments don’t make it to production. That’s true in any serious engineering context, but it’s especially true with AI, where the output is generative and the edge cases are endless. You run something, it half-works, you understand something new about your process or your data or your users, and you adjust.
The teams that treat every failed pilot as a loss are going to keep losing. The ones that are building a body of knowledge, capturing what broke and why, testing assumptions about their own operations before they test assumptions about the model — those teams are compounding on something real.
A failed pilot that taught you what your actual workflow looks like is not a failure. It’s the first step you probably should have taken anyway.
So What Should Organizations Do Differently?
Before buying a platform or spinning up a pilot, it’s worth spending time on a few basic things:
- Pick one workflow. Not “operations.” One specific, repeatable process with a clear input and a clear output.
- Write it down completely. Not at a high level. Every step, every decision point, every exception you can think of.
- Find the gaps. The places where the documentation doesn’t match what actually happens. Those gaps are exactly where an agent will struggle.
- Treat the first build as a learning exercise, not a delivery commitment.
None of this is glamorous. It doesn’t make for a great press release. But it’s the work that makes everything else actually stick.
The technology is ready. The question is whether the organization is, and that’s a question about process clarity, not compute budgets.