Why 95% of AI Projects Fail and How to be the 5% with Jon Krohn
As AI capabilities double every seven months, the world is racing to harness its power. But here’s the hard truth: 95% of AI projects never make it into production. They fail quietly: over budget, under-delivering, or never connecting to real business outcomes.
In a recent episode of the Signal to Noise Podcast, host Michael Newcomer sat down with Jon Krohn, Co-Founder and CEO of Y Carrot and host of the Super Data Science podcast, to unpack what separates the 5% of successful AI projects from the rest.
The Data Science Skill Shift
According to Jon, the role of a data scientist is transforming rapidly. Tools like Claude Code and Cursor have made coding assistance so advanced that being incredible at writing perfect Python code is no longer what sets top data scientists apart. Instead, success now relies on experimental design, curiosity, and clear thinking, the ability to design and interpret experiments, identify confounders, and extract real signals from noise.
It’s not the syntax; it’s the scientific mindset. The best data scientists today think like researchers: hypothesis-driven, methodical, and relentless about validation. You can’t rely on intuition alone. You have to understand how results can mislead you.
Why Communication Makes or Breaks AI Projects
One of the biggest surprises in Jon’s career is that communication is the number-one skill he and other leaders look for when hiring data scientists.
It’s not enough to build accurate models. You need to translate complexity into clarity: to explain what the model does, how it drives value, and what risks or assumptions exist. Without that, even the best model will gather dust in a forgotten Jupyter notebook.
Jon cites a sobering MIT study from the research lab NANDA showing that 95% of AI projects fail to create value in production. Why? Because of the communication gap between the technical teams building the models and the executives approving and deploying them. If the people funding the project don’t understand what it does or how to measure its success, it never gains traction.
How to Be in the 5% That Succeed
Jon’s answer is refreshingly simple: define success upfront.
Before writing a single line of code, set clear KPIs – cost, accuracy, speed, or any combination that matters for your business. Align everyone on what good looks like.
Without those benchmarks, teams end up optimizing for the wrong goals, often chasing technical elegance over business impact.
Another key insight: most leaders instinctively look to customer-facing applications as the obvious AI opportunities. But according to Jon, that’s usually a mistake.
There’s vastly more ROI in automating back-office operations. That’s where the data lives, and that’s where inefficiencies are buried.
AI isn’t just about the shiny front end; it’s about quietly optimizing the boring stuff that keeps your business running.
The Organizational Challenge
If the technology is moving so fast, why are companies still so slow to adopt it? Jon believes organizational inertia is the real bottleneck.
AI readiness isn’t just about data infrastructure; it’s about culture. Are your teams aligned on goals? Are decision-makers open to iteration? Do you reward experimentation? Those human factors often determine success long before model accuracy does.
How to Stay Ahead in a Field That Doubles Every Seven Months
Jon points to a chart from MTER showing that the complexity of tasks AI can perform reliably doubles every seven months. Today, that’s roughly equivalent to automating a two-hour human task with 90% accuracy. In seven months, it’ll be four hours. Then eight.
That exponential curve is why Jon believes this is the best time in history to be in AI. But it also means businesses must prepare for rapid evolution, building adaptable systems, clear governance, and a culture of continuous learning.
Why Communication and Empathy Still Matter Most
Despite all the automation talk, Jon’s philosophy comes back to something deeply human. The leaders who thrive in this new era are those who balance technical ambition with empathy.
“Great leadership is empathy. I want everyone I work with to feel heard, to know their ideas matter.”
It’s a fitting reminder that as AI takes over more of the doing, the listening matters more than ever.
Final Thoughts
AI success isn’t about luck or hype; it’s about structure, clarity, and human judgment.
To be in the 5% that succeed, you need:
- Clear KPIs that define success from day one.
- A communication bridge between builders and decision-makers.
- A focus on invisible, high-ROI operations.
- A culture that rewards curiosity and iteration.
Jon’s message is ultimately optimistic: as AI evolves, so must we. The organizations that combine disciplined thinking with human empathy won’t just survive the AI revolution; they’ll define it.
Jon’s Background
Jon Krohn is the Co-Founder and CEO of Y Carrot, a data science consultancy delivering AI solutions to Fortune 50 companies, startups, and government agencies. As the host of the globally recognized Super Data Science podcast and author of the #1 bestselling book “Deep Learning Illustrated,” he brings vast expertise in AI education and practical machine learning applications. Currently serving as a Machine Learning Practice Fellow at Lightning AI, Jon combines his academic background with hands-on industry experience to help organizations implement effective AI solutions.
Listen now: Signal to Noise Episode 8: Why 95% of AI Projects Fail and How to Be the 5% with Jon Krohn
