I’ve had some version of the same conversation more than a dozen times this year. A newly appointed Head of AI, six to twelve months into the role, describing the same arc almost word for word: a strong start, a couple of pilots that impressed the room, and then a stall nobody quite knows how to explain. The AI works. The excitement was real. And somehow nothing has actually made it to production, or the one thing that did is now the subject of a difficult conversation with risk or compliance that should have happened months earlier.
The pattern showed up often enough, across different industries and different-sized organizations, that I stopped treating it as a coincidence. Something structural is happening in the first few months of this role, consistently, regardless of how good the leader or the technology is. That’s the reason for this piece. Not a theory about how the role should work, but a pattern I kept hearing described in almost identical terms by people who had no reason to compare notes with each other.
It turns out the pattern shows up in the data too, not just in these conversations. A widely cited MIT study from July 2025, The GenAI Divide, found that 95% of enterprise generative AI pilots failed to deliver measurable financial return. McKinsey’s own research tells a similar story from a different angle: 78% of organizations now use AI in at least one business function, but only about one in three has scaled a pilot beyond that stage. Two separate studies, two different methodologies, the same finding I kept hearing described anecdotally. The gap isn’t between companies that have good AI and companies that don’t. It’s between the roughly one in twenty that get a pilot into production and everyone else stuck in what’s now bluntly called the POC graveyard.
Here’s the detail worth sitting with. MIT’s researchers were explicit that the failures weren’t about model quality. They called it a “learning gap,” a mismatch between the tools and how the organization actually integrates them. That’s not a technology problem. It’s an operating model problem, the same thing I was hearing in every one of those conversations, just without the label. Which means the first 90 days of a new Head of AI role are usually spent solving the wrong thing.
The instinct that guarantees you join the 95%
Most people stepping into a Head of AI role for the first time do the obvious thing: pick a visible use case, get it demoed in front of leadership fast, and use that win to build momentum for the next one. It’s an understandable instinct. It’s also close to a guarantee that the pilot joins the graveyard eventually, because a demo that impresses a room and a system that survives a deployment decision are different achievements, and only the operating model determines whether you can tell the difference before it’s expensive to find out.
The pattern that kills pilots isn’t usually the AI underperforming. It’s that nobody defined who tests it, what “ready” means, or what happens when the answer is no. The pilot sits in a permanent limbo between “interesting demo” and “approved for production,” because there was never a gate designed to move it through in either direction. Eventually leadership loses patience, not with the technology, with the ambiguity, and the whole initiative quietly stops.
What the first 90 days should actually build
The job isn’t to ship the most visible AI win by day 90. It’s to build the operating model that determines whether every use case after that has a real path to production, or joins the pile of things nobody ever formally killed.
Days 1 to 30: inventory and diagnose. Find every AI system currently running, including the shadow AI nobody officially approved. For each one, ask a version of the one-line test: who owns this, and who’s actually accountable for testing it. If those trace to the same person, or the same reporting line, that’s not a detail to fix later. It’s the finding that should shape everything in the next 60 days. This is also the point to honestly place the organization on a maturity ladder, from no real testing function to mature recurring assurance, rather than assuming good intent means the structure is already sound.
Days 31 to 60: design the operating model, not the next pilot. Decide, deliberately, what you as Head of AI will own, use-case intake, delivery standards, platform choices, and what must sit outside your incentives entirely: independent testing and the authority to hold a release. This is the part most new Heads of AI skip, because it doesn’t produce a demo. It produces something more valuable: a deployment gate with three real outcomes, approve on evidence, remediate and resubmit, or escalate an unresolved disagreement to someone with actual authority to decide. Without that gate, every use case defaults to the fourth outcome nobody designed: indefinite limbo.
Days 61 to 90: run one use case through the whole model, end to end. Not ten pilots. One, pushed all the way through intake, build, independent testing, and a real deployment decision, with an evidence pack that could survive being read by someone skeptical of it. The thing being piloted in your first 90 days isn’t the AI capability. It’s whether your operating model actually works under real conditions. If it does, every use case after this one inherits a working process instead of starting from zero.
The objection, and why it’s backward
The obvious pushback: leadership wants to see AI wins in 90 days, not an org chart. Fair. But a single use case that cleanly passes a real deployment gate, with evidence a risk committee would actually accept, is a more credible 90-day story than ten demos that all quietly stall six months later for the same reason MIT’s research already named: not the model, the integration. Shipping fast without an operating model doesn’t save time. It moves the delay from before launch, where it’s cheap to fix, to after launch, where it shows up as an incident, a retracted claim, or a program leadership stops trusting.
Where independence fits, and why it isn’t a phase two problem
The testing function’s independence isn’t a governance nicety to bolt on once the pilots are working. It’s the part of the operating model that determines whether “approved” means anything. A Head of AI who builds delivery capability first and plans to add independent testing later is building on the same foundation that produced the 95% failure rate, just with better tooling. The organizations in MIT’s 5% didn’t get there by moving faster. They got there with a process that could tell the difference between a system that worked in a demo and one that was actually ready, before it mattered.
If you’re 30 days into a Head of AI role and you can’t yet answer who tests what you’re building, and whether they can actually stop a launch, that’s not a later problem. It’s the first 90 days, whether it feels like it or not. Instead of creating an operating model from scratch, utilise an enterprise-ready operating model like Protum (https://protum.ai) where you can answers questions from an end to end outcome perspective.
#HeadOfAI #AIOperatingModel #AIGovernance #AITesting #EnterpriseAI #AITransformation