Two items sit next to each other on a Q3 product roadmap. The first reads "AI assistant in the sidebar." The second reads "Cut the four hours a month finance teams spend matching invoices to purchase orders." Both could run on the same model. Only the second one tells anyone what to build, who it is for, and how to know whether it worked.
The first line is how a lot of AI roadmaps get written in 2026. A board member asks where the AI is, a competitor ships a copilot, sales wants something new for the demo, and "we need AI in our product" turns into a roadmap item without anyone naming the customer who asked for it. That is a reaction. It gets presented as an AI product strategy, and it is an expensive one, because the engineering quarter is real even when the customer value never shows up.
Customers do not buy technology. They buy a problem going away.
Everything below follows from that one sentence: how to spot a technology-led roadmap, why those features ship and then sit unused, and a four-question test to run on every AI idea before it gets a sprint. If you have not yet confirmed there is a market for your product at all, start with market research before you build an AI product . This piece assumes you have customers and a product, and are deciding what to put in it next.
"What can we do with AI?" is the wrong first question
There are two ways to start an AI roadmap conversation, and they produce different products.
The first question is "what can we do with AI?" It sounds ambitious and open-minded. Somebody opens a model playground, tries a few prompts against your data, and finds that it can summarize records, draft emails, answer questions about the account, and generate a report. Every one of those works in the demo. So all of them go on the list.
The second question is "what problem do our customers have that costs them the most?" That one is slower and, frankly, less fun. It sends someone to support tickets, churn interviews, usage logs and the spreadsheets customers keep next to your product because your product does not do the job. It might end with an AI feature. It might not.
The trouble with the first question is that it always has an answer. Models in 2026 can do something plausible with almost any input, so a technology-first brainstorm never returns "don't build this." It only returns more candidates. The second question can return "this problem is too rare to matter" or "this is a missing filter, not a model," and that ability to say no is exactly what makes it useful for prioritisation.
Using the invoicing example from the top: the sidebar assistant came from asking what the model could do. The invoice-matching item came from noticing that customers export data every month-end and reconcile it by hand. One of those has a buyer, a frequency and a cost attached. The other has a launch date.
Why technology-led AI features ship and then go unused
A technology-led feature rarely fails on launch day. It fails slowly over the following quarter, usually in the same four ways:
- A curiosity spike, then decay. Usage jumps in the first week as people click the new button, then flattens. Nobody returns because nothing in their work changed.
- More use in sales calls than in accounts. The feature becomes a demo asset. Prospects like it. Existing customers do not open it.
- No movement in the metric it was supposed to move. Nobody can say what that metric was, which is the actual problem.
- Support tickets asking how to turn it off or why it gave a confident wrong answer.
None of this is unique to AI. Pendo's 2019 Feature Adoption Report found that 80% of features in the average software product are rarely or never used. AI features inherit the same risk with a few extra costs: inference bills that scale with usage you hoped for , accuracy you now have to monitor , and a trust problem when the output is wrong in front of a customer. The broader numbers on AI projects point the same way. MIT's NANDA initiative reported in 2025 that around 95% of the generative AI pilots it studied had produced no measurable impact on profit and loss.
So the failure starts before engineering ever sees a ticket. The root cause sits upstream. When a feature starts from the technology, three things are missing from day one: a named user, a frequency, and a baseline cost. Without them, nobody can say whether the feature worked, so nobody can decide to improve it, reprice it or kill it.
There is a competitive cost too. Model capability is getting cheaper and more available every year, and the same APIs are open to your competitors. A feature whose only advantage is "we call a model here" is the easiest thing in your product to copy. What is hard to copy is a deep understanding of one painful workflow and the data that comes from solving it well.
The signals that your AI roadmap is technology-led
A technology-led roadmap is one where items are defined by the capability they use rather than by the customer outcome they deliver. You can check your own roadmap against these signals in about twenty minutes.
- Items are named after the technology. "AI copilot," "agent," "chat with your data," "smart summaries." If you cover the word "AI" and the item no longer says what it does for anyone, it is technology-led.
- Nobody can name the customer and the job. Ask the owner "which role, at which kind of customer, does this help with which task?" A pause is a signal.
- The success metric is the launch. Or it is "adoption of the AI feature," which measures whether people clicked it, not whether their work improved.
- It exists because a competitor announced one. Matching a competitor's press release is a positioning decision, and it should be argued as one.
- Nobody knows how to price it. If you cannot state the value, you cannot charge for it, and the feature ends up bundled for free and judged on vibes.
- Sales asked for it for the demo, not because a customer asked for the outcome.
Two or three of these on one item is normal early in planning. Five or six means you are looking at a technology in search of a problem. Kill or rewrite it before it gets a sprint. Our piece on killing weak AI ideas before you build goes deeper on how to make that call fast.
The problem-first test: four questions before anything gets built
Every AI item on your roadmap should pass four questions, in this order. The order is the point: three questions about the problem come before a single question about AI.
- Who has the problem? Name a specific role at a specific type of customer. "Accounts payable leads at mid-market distributors" is an answer. "Users" is not.
- How often do they hit it? Daily or weekly pain beats quarterly pain. Frequency is what turns a feature into a habit, and habits are what show up in retention.
- What does it cost them today? Hours, errors, missed revenue, compliance exposure, or the workaround they pay for instead: a spreadsheet, an extra hire, an outsourced step. If nobody can estimate the cost, the problem is not understood well enough to build for.
- Is AI actually the best way to solve it? Compare against the simpler fixes first: better defaults, rules, search, templates, an integration, or a change to the workflow. AI earns its place when the inputs are messy and variable, and when "mostly right" is acceptable or easy to review.
| Question | A good answer sounds like | A red flag sounds like |
|---|---|---|
| Who has the problem? | "Month-end close owners at our 50 to 500 seat accounts" | "Everyone could use this" |
| How often? | "Every week, sometimes daily" | "When they need it" |
| What does it cost today? | "About four hours a month per customer, plus the errors we see in tickets" | "It would be nice to have" |
| Is AI the best fix? | "Inputs are unstructured PDFs from hundreds of suppliers" | "Because we need an AI feature" |
The third question is where most items fall apart, and it is the one leadership should push hardest on. A problem with a cost attached gives you a business case, a success metric and a price at the same time. If you want to see how quickly an AI business case dies without one, read about why AI business cases stall in the second meeting .
When the honest answer is not AI
Run the test seriously and some items will come back with an answer nobody on the AI working group wanted to hear: build a better form.
That is the test working. A problem-first roadmap still has AI on it (plenty, usually). The difference is that AI has to compete for each slot with simpler options, and sometimes it loses.
A few common cases where it loses:
- The task needs to be exactly right every time. Tax calculations, billing totals and compliance thresholds should stay deterministic. A model that is right most of the time is worse than a rule that is right every time.
- The task is rare. A report someone builds once a quarter may need a good template, not a generator.
- The real problem is missing data or a broken workflow. No model fixes a field that nobody fills in.
Saying "not AI" also protects the AI items that do survive. Every weak AI feature you ship spends some of your customers' trust in the next one. For a fuller list of where automation does not belong, see when not to use AI .
What it looks like when you build backwards from the problem
Teams that get this right tend to share a few habits.
The feature is named after the outcome. "Auto-match invoices to purchase orders," not "AI assistant." The name is a promise the customer can check.
There is a baseline before there is a build. Someone measured how long the task takes today, how often it happens, and how many errors it produces. That number becomes the success metric, and it is written down before the first prompt.
It ships narrow. The first version goes to the segment with the most frequent pain, not to every account at once. Narrow scope makes quality measurable and failures cheap.
It is judged on the customer metric. Did the hours go down? Did the errors drop? Usage of the feature is a leading indicator. The result is the hours and errors.
Intercom's Fin is a public example of a team building backwards from a measurable problem. Customer support teams have a clear, frequent, expensive problem: conversations that need an answer. Intercom built Fin around resolving those conversations and reports its resolution rate publicly. It also prices Fin per resolution, so the customer pays when the problem actually goes away. You can like or dislike the product. The shape is right: the outcome defines the feature, the metric and the price.
That last point is why an AI product strategy turns out to be a business strategy. When the problem is quantified, pricing and packaging follow from it. Value you can name is value you can charge for. Value you cannot name gets bundled in for free and cut in the next budget review.
Run the test on your roadmap
Take every AI item on your current roadmap and put it through the four questions this week. Keep the items that survive. Rewrite the ones that have a real problem hiding behind a technology name. Cut the rest, and say so out loud, so the team stops carrying them.
If you want help doing that with evidence rather than opinion, a Discovery Sprint is built for exactly this. In one week, we work through your roadmap with your product and leadership team, put each AI item through the problem-first test against your own customer data, and leave you with a prioritised list, including the items to drop.
AI Transformation Discovery - a one-week sprint that turns "we need AI in our product" into a short list of problems worth solving.
