Rob Collie is the founder and CEO of P3 Adaptive, a Microsoft Solutions Partner for Data and AI that serves hundreds of mid-market and Fortune 1000 clients. A former Microsoft engineering leader on the Excel, Bing, and Power BI teams, Rob led the Power BI wave after leaving Microsoft and has authored three previous business-technology books (92,000+ copies sold). He also hosts the Raw Data with Rob Collie podcast. His fourth book, Fair Game: Customizing AI to Your Business Is Easier Than You Think (August 2026), turns that practitioner credibility on the AI moment.
You spent more than a decade at Microsoft helping develop business intelligence features in Excel and Power BI before founding P3 Adaptive in 2013. How did that transition from building software inside Microsoft to solving data problems for clients shape your current view of enterprise AI?
When I was leading product teams at Microsoft, we were building software that needed to work for the entire world, and that couldn’t be tailored to needs of any specific customer. We used to call it “ordering a pizza whose toppings were acceptable to 300 million people.” There’s necessarily a lowest-common-denominator vibe to that job, as well as a certain distance from individual customers.
There was a definite prestige to working on that big stage, but it wasn’t nearly as emotionally satisfying as helping specific clients realize their unique ambitions. Working closely with a client, we have the chance to become invested in their success, and to explore creative solutions which would never fit into the one-size-fits-all business model of Big Software. It’s more intellectually stimulating in many ways, and the direct connection with our clients makes the wins far more fulfilling.
But there’s greater responsibility, too. At Microsoft, a single unhappy customer was just a statistic, and I shrugged off complaints every day as part of simply doing my job. But at P3 Adaptive, a single unhappy client means we’ve failed. There are no statistics. We have a responsibility to every single relationship.
I learned many valuable things at Microsoft and wouldn’t trade that experience for the world, but I often refer to myself as a “recovering software engineer,” because success now means operating very differently.
And that’s exactly the lens I carry into enterprise AI. Off-the-shelf AI is the ultimate 300-million-person pizza – a genuine marvel, engineered to be individually valuable to everyone while tailored to no one. But organizational AI wins will come from the tailoring – from getting close to one specific company and customizing the AI to its data, its processes, its definitions. I got to spend my career on both sides of that divide, and it left me with zero doubt about the side on which enterprise AI will win.
In Fair Game, you argue that many companies have approached artificial intelligence backwards by distributing general-purpose chatbot licenses rather than building systems that understand their operations. Where do off-the-shelf AI assistants reach their limits, and what signals indicate that a business needs something customized?
Off-the-shelf AI has a PhD in everything except your business. It’s read the whole internet, but the internet is missing your company’s definition of “active customer,” your pricing logic, your operational processes, and which of your two systems to trust when they disagree. That knowledge will never be public. So the generic AI that’s a world-beater for personal use falls short for real business use, and the gap between the two experiences is both discouraging and confusing.
Today, basically everyone’s answer to “what to do about AI” has been “buy subscriptions and find out.” I think that is a natural first step, so I’m not critical of anyone who’s done that. Instead I am sympathetic – no one is really taking the time to explain that the off-the-shelf subscriptions aren’t enough, nor why. So I think that businesses are basically right where we should expect them to be – trying the thing that’s available and starting to learn that it’s insufficient.
The fix isn’t touching the AI model itself – you don’t have to become an LLM researcher. It’s everything you surround the model with: your data, your instructions written in plain English, and normal software. When you find yourself typing the same context into a chatbot for the fifth time this week, that’s the tell. Whatever you keep re-explaining is exactly what a custom system should already know – every time it wakes up.
You use the term “Crafters” to describe data-savvy business professionals who can build valuable custom AI systems without being traditional software developers. What characteristics distinguish a Crafter, and how can leaders identify these people within their existing workforce?
A Crafter is someone born with an itch to solve problems with tools. Roughly one in 16 knowledge workers has it, in my experience. They were the Excel power users, then the Power BI generation, then the people IT labeled “shadow IT.” They’re your analysts, your finance modelers, your ops leads – people who grew up in the business and discovered a knack for tooling.
Two traits make them ideal for AI work. First, systems thinking: they instinctively decompose a messy process into inputs, rules, and outputs, much like professional software developers. Second, grounding in the business: they know which numbers the CFO actually watches and what the person asking a question is really asking. You cannot teach either one in a bootcamp.
How to find yours: follow the spreadsheets. Right now at your company, there are spreadsheets, dashboards, and automations at the heart of critical workflows. None of them were built by IT, and each one has an author. Start there. And then start to gauge how they might turn their talents toward customized AI solutions.
Why do you believe Crafters, rather than developers alone, are best positioned to lead many internal AI projects, and how should responsibilities be divided among business experts, data teams, software engineers, information technology departments, and security teams?
Because the hard part of custom AI isn’t code – it’s context. The single highest-leverage activity in an AI project is deciding what the system needs to know about your business, and Crafters carry that knowledge natively. A brilliant engineer parachuting in from three org-chart hops away has to conduct months of interviews to learn what your ops lead already knows by reflex.
But this is emphatically not a developers-are-obsolete story. The division of labor I recommend has three factors, and none of them is seniority or personality: work gravitates toward professional developers as reusability, complexity, and sensitivity rise. Anything customer-facing, anything touching sensitive data, anything making autonomous decisions – that’s developer territory, and as agents multiply, those scarce engineering skills become more valuable, not less. Work gravitates toward Crafters where business-process nuance dominates.
There’s also an underrated middle ground: the Crafter builds, the developer audits. IT and security shouldn’t be gatekeepers who approve projects into existence – they should own the paved road. Provide the sanctioned platforms, the data access rules, the review checkpoints, and let the people closest to the problems do the building. Treat the whole thing as a maturity model, not a fence.
Custom AI needs access to company-specific terminology, metrics, processes, and institutional knowledge. What role do semantic models and existing business intelligence infrastructure play in giving AI an accurate understanding of a company?
They’re the decoder ring. Right now your company’s definitions – what counts as an active customer, which costs belong in gross profit – live in people’s heads and in a thousand slightly inconsistent spreadsheets. An AI agent can’t reliably reason about your data until those definitions are written down in a form a machine can trust. The industry has started calling this discipline “context engineering,” and I’d translate the term this way: it’s the work of structuring what your business knows so an AI can actually use it. The analysts made it sound new. BI practitioners have been doing a version of it for fifteen years.
That’s the good news hiding in plain sight: if you invested in the BI era (and particularly if you invested in Power BI), you might already own a head start. A well-built semantic model is exactly the machine-readable capture of business meaning that agents need. The companies that treated their semantic layer as an afterthought are discovering that the “boring” definitional work they skipped is now the toll booth on the road to AI. And critically, this work is deeply specific to your business – which is precisely why it’s the durable advantage. Every vendor can sell you the same model. Nobody can sell you your own definitions.
You built a custom AI editor, known as Eddie, to help develop Fair Game. What did the system actually do during the writing process, and what did its successes and failures teach you about designing AI around a highly personal workflow?
To be clear, I wrote every paragraph of the book from scratch, while Eddie mostly sat and waited. At times I’d spend hours hammering out an entire section of a chapter before asking “him” to read it. Other times I’d be bouncing things off of him every few minutes. But crucially, Eddie was on call 24/7. I could get feedback just as easily at three am as at one pm, and he would turn it around in a minute or less. In total, I suspect Eddie read the manuscript at least thirty times over. No human being could have done this job, because no human being would want it.
He tracked promises I made in Chapter Three and called me out when Chapter Twelve forgot them. He learned my writing style and then enforced it – holding me to the best version of my own voice instead of letting me drift into Humorless Business Author mode. He told me when I was being lazy and when I was beating a dead horse. We had real disagreements, and sometimes he won.
The biggest design lesson: Eddie’s “brain” is written in English and lives in a folder. Every time he gave feedback that missed – too generic, wrong register, forgetting a rule I’d already stated – the fix was to write the correction down and make it part of his permanent context. The failures weren’t AI failures; they were gaps in what I’d bothered to teach him. That loop – notice the miss, encode the lesson, watch it stick – is the entire craft of custom AI in miniature. And it’s why I ended up building specialized Eddies for publicity, competitive research, and website messaging. Same LLM underneath. But different specialists.
Many organizations believe they must completely clean and centralize their data before attempting custom AI. How much data readiness is genuinely required to begin, and how can companies start producing value without waiting for a perfect foundation?
Data perfection is not a prerequisite, and that’s good news because perfection never arrives. If you set out to first build a perfect data estate, as many consulting outfits would advise, you will be building what I call “plumbing for its own sake” – expensive pipes running everywhere, but when you finally get around to installing a faucet, you find that there’s no pipe where you need it.
Our company instead advocates a “faucets first” approach. Pick a specific use case and work backwards from business impact rather than forward from infrastructure. Build an MVP from that use case, and do so with minimal new infrastructure. Iterate on the MVP until it’s production ready, and then step back and evaluate how you might harden your infrastructure to support it. That delivers business impact more quickly, minimizes cost, and informs future projects – at both the faucet and the plumbing levels.
A custom AI prototype can appear impressive during a demonstration but become unreliable when exposed to real employees, changing data, and edge cases. What evaluation, monitoring, and human oversight should be established before an internal AI system becomes operational?
With a few notable exceptions, I think demos are less valuable in the AI era than they were in the software era. Software demos always overpromised and we all knew it. But AI demos will be even further removed from your reality.
AI is about workflow. And there is nothing more custom than the thousands of workflows powering the operations of a specific organization. Go back to the “new hire with a PhD in everything” metaphor. How much training – and hands-on experience working at your company – does a new hire require before they are effective at your company? How can a demo possibly account for all of that?
So we use demos to get people thinking. To show them the art of the possible. Not to sell them a product. The real demo starts with the prototype of the custom solution. The MVP. And then we iterate and improve. Rapidly.
At some point it’s ready for a soft launch or pilot program. And again, we learn – together – and rapidly improve based on that learning. This is often the phase where monitoring, evaluation, and oversight start coming into sharp focus. The things you end up needing are often very different from what you would have guessed going in.
How can companies empower Crafters to experiment without creating a new generation of shadow AI systems, duplicated workflows, security vulnerabilities, and tools that nobody is responsible for maintaining?
Remember where shadow IT came from: it wasn’t malice, it was the necessary fulfillment of unmet demand. Crafters build because problems bother them – that’s the gene. If the sanctioned path means waiting a year, shadow AI will fill the gap – and fill it below the radar, where it’s most dangerous.
So make the sanctioned lane the easy lane. Give Crafters an approved platform with the security guardrails already baked in – identity, data access, logging – so the compliant choice is also the convenient one. Keep a lightweight registry: anything that graduates from personal experiment to something a second person depends on gets written down, with a named owner. That single rule kills most of the orphaned-tool problem, because tools with names attached don’t get abandoned quietly.
Then apply the escalation model: experiments run free, but as something becomes mission-critical – more users, more sensitivity, more autonomy – it earns progressively more engineering review. The Crafter keeps ownership of the business logic; a developer hardens what needs hardening. The goal is a maturity pipeline, not a permission process. Companies already ran this exact movie with spreadsheets, and the winners weren’t the ones who banned Excel.
For a company beginning its first custom AI initiative, how should it select the initial use case, measure whether the project is delivering meaningful business value, and decide whether to expand, redesign, or abandon it?
We have two flavors of starting point we use with our clients.
Option one, look for the jobs nobody’s doing – not the jobs you’d like to eliminate. There’s a question I love asking managers: where have you thought, “if I had one person constantly watching this and thinking about it, things would get meaningfully better – but I could never justify a whole hire for it”? Those are often your best starting points. They’re safe, they build confidence, nobody feels like a target, and the counterfactual is honest: the alternative wasn’t a human doing it well, it was nobody doing it at all (like my editor friend Eddie).
Option two, look into replacing dashboards with data agents. As simple as dashboards seemed, they fell far short in practice of delivering on their promise. When someone has a business question, it’s a lot of work for them to translate that question into the dashboards landscape. Where is the dashboard that answers this question? What is it named? Does such a dashboard even exist? And if you manage to find the “right” one, is it clear and convenient to use? Do you have to repeatedly manipulate it, writing down or screenshotting multiple versions to assemble the overall picture you need?
In the AI era, you just take your business question – in your own words – and type it (or dictate it!) to a data agent who then handles all of that for you, and returns a certified, well-researched answer – visuals included – in a minute or two. When you have a follow-up question, it’s happy to quickly answer that, too – in the meeting while decisions can still be made.
The common thread behind both of these starter options? They both address pain points that employees will embrace rather than resist. You don’t want your early AI initiatives seeding distrust. You want them to instead bring employees to the table. You want employees suggesting improvements and new project ideas. Because again, your company is composed of thousands of workflows, and your employees know them better than you do.
On expand, redesign, or abandon – be kind to yourself, because the research on this is genuinely comforting: most successful AI deployments had failures before them. A first project that produces a lesson instead of a payoff is tuition, not evidence that AI doesn’t work. My rule of thumb: if people are using it, expand it. If people aren’t using it, you need to know why not, and that can be a wide range of answers, from “because it doesn’t work well” to “because I don’t understand it” to “it frightens me.” The answer informs whether you improve, redesign, or abandon. You don’t have to predict where all of this lands. You just have to start somewhere honest.
Thank you for the great interview, readers should also read Fair Game: Customizing AI to Your Business Is Easier Than You Think.
