Your Data Is the Bottleneck, Not the Model

Ninety percent of organizations say their data is AI-ready. Eighty-seven percent say data readiness is the single biggest thing stopping them from putting AI into production. Both numbers come from the same survey, answered by the same people (DataHub, 2026 State of Context Management Report).

That gap is not a contradiction in the data. It's a contradiction in how teams diagnose their own systems. The assistant gives a confident wrong answer, and the first three moves are almost always a better model, a longer prompt, and a reranker. Nobody checks whether the right document was even reachable.

There is a name for what those three moves miss, and it is not data quality. It's the authority gap: nothing in the system says which of your four plausible sources is the true one. Fix that and most of the rest gets cheaper. Skip it and a model upgrade buys you a more fluent version of the same wrong answer.

The symptoms that get blamed on the model

Most of what teams describe as "the model isn't good enough" looks, on inspection, like this:

• Confidently wrong answers with no uncertainty signal. The system doesn't hedge, doesn't flag low confidence, and produces fluent prose that happens to be false.
• Retrieval pulling the wrong table. The number is real. It came from a deprecated table nobody archived, or from the staging copy, or from the one of four revenue columns that finance stopped using two quarters ago.
• Evaluation scores that held and production accuracy that didn't. The eval set was curated. Production data is not.
• Answers that change depending on phrasing. Two users ask the same question different ways and get different numbers, because the retrieval layer surfaces different sources each time.
• An agent acting on an incomplete record. A purchase order goes out, a ticket closes, a discount applies, all correctly executed against a record missing a field.
• The same question answered differently in two departments. Each has its own source of record and no way for a machine to tell which one is authoritative.

None of those are model capability failures. They're failures of what reached the model, or what the model was trained on.

There's a reason the misdiagnosis is so consistent, and it has nothing to do with competence. Model upgrades are legible. There's a version number, a benchmark, a release note, a procurement path. Data work has none of that, which makes it easy to defer and hard to get funded. So the instinct goes where the instinct can act.

Which means the useful question isn't "is our data good." It's narrower, and you can answer it in an afternoon.

How to tell whether it's the data or the model

Here's a sequence you can run on a single failing query, without a platform, a vendor, or a budget line. It takes about an hour.

Step 1. Pick one failure you can verify by hand. Not a category of failure, one instance. You need to know the correct answer independently, which means you or someone on the team has to go and find it in the source system. Write down the correct answer before you touch the AI system.

Step 2. Ask the system the question again and capture what it retrieved. Not just the answer, the context. Which documents, which rows, which chunks. If your stack doesn't let you see retrieved context, that's finding number one, and it's a serious one. You cannot debug what you cannot observe.

Step 3. Paste the correct source material directly into the context and ask again. Bypass retrieval entirely. Hand the model the passage or the row that contains the answer you verified in step 1.

Step 4. Read the outcome. This is where the diagnosis happens:

Step 5. If step 4 pointed at the data, ask the ambiguity question. Count how many tables, documents, or pages in your environment could plausibly answer the question you tested. If the answer is more than one, ask whether anything in the system marks which is correct. Not a person who knows. A field, a tag, a lineage record, something a retrieval layer can read.

That last question is the one that usually ends the debate. In most environments the honest answer is that a human used to make that call, and the AI system removed the human from the loop without replacing what the human knew.

Run this on three or four failures rather than one. A single instance can mislead. A pattern across four is a diagnosis. And if the pattern says data, the next thing to understand is why that verdict got so much more expensive than it used to be.

The same defect fails differently in ML, generative AI, and agentic systems

The data problem hasn't changed much in ten years. What changed is what happens downstream when a defect gets through.

In classical machine learning, a bad record produces a wrong prediction that lands in a report or a dashboard. Someone eventually notices that the number looks off, an analyst investigates, the model gets retrained. The damage is bounded and the feedback loop, while slow, exists.

Generative AI removes the feedback loop. A retrieval-backed assistant reading a stale or contradictory source produces a fluent, confident answer with no signal that anything is wrong. There is no "this looks off" moment, because the output doesn't look off. It looks like every other correct answer the system has produced. The failure stays invisible until someone acts on it.

Agentic systems remove the review step as well. When the system's output is an action rather than a sentence, a data defect becomes a committed purchase, a closed ticket, a sent email, an applied discount. As Dataiku put it in their 2026 analysis of this split, the further AI moves from prediction to action, the less tolerance there is for data quality failures and the harder those failures are to catch before they cause damage.

Blast radius grows while visibility shrinks. The same missing field that produced a slightly-off forecast in 2019 can produce a confidently wrong customer answer in 2024 and an executed transaction in 2026. That's the real reason data readiness became urgent, and it isn't because the data got worse.

So the bar moved. What most teams are still measuring against is the old one.

AI-ready is not the same as clean

Here's where a lot of data quality programs waste their first six months. "Clean" is a property of a dataset measured against a general standard. "Ready" is a judgment about a specific dataset against a specific workload, and the two frequently disagree.

DataHub makes this point better than anyone else writing on the topic, and the clearest example is outliers. Analytics pipelines routinely strip outliers so patterns become legible to a human reading a chart. Fraud detection and anomaly models need exactly those outliers, because the outliers are the signal. Run the analytics cleaning pass on the fraud model's training set and you've made the data cleaner and less ready at the same time.

The same table can pass one readiness test and fail another:

• A customer table with eighteen months of history is ready for a churn model and unfit for a seasonality forecast that needs five years.
• A product catalog with clean SKUs and no written descriptions is ready for inventory reconciliation and useless to a support agent that has to answer questions about what a product does.
• A contract repository with perfect metadata and scanned image PDFs is ready for search and unready for extraction.

Readiness also has a second half that traditional data quality never had to cover. For a decade, readiness was measured against human analysts, who bring their own context. An analyst pulling a revenue table knows which of the four revenue columns finance uses at quarter close, because they asked someone once and remembered. An AI agent has no equivalent. It gets whatever the retrieval layer hands it.

So data earns the label when four things are true at once: it's accurate and complete enough for the specific job, its origin and freshness are traceable, its meaning is documented somewhere a machine can reach, and its trustworthiness can be evaluated by the consuming system rather than vouched for by a person.

That fourth condition is the authority gap restated as a requirement, and it's the one almost nobody has. It's also the one that turns a pile of clean tables into something an agent can use.

What the evidence actually says about fixing data versus tuning models

Almost every article written on this subject asserts that data quality matters and cites a survey. Very few cite a measurement. There are measurements, and they're narrower than the marketing claims built on top of them, which is exactly why they're worth knowing.

Mohammed and colleagues ran the most careful study of the relationship. They polluted six data quality dimensions systematically and measured the effect on nineteen machine learning algorithms across classification, regression, and clustering, separating three scenarios: polluted training data, polluted test data, and both (arXiv:2207.14529, published in Information Systems, 2025). The finding that matters for a practitioner is that the effect is dimension-specific and algorithm-specific rather than uniform. Some algorithms tolerate missing values well and collapse under inconsistent representations. Some do the reverse. "Improve data quality" is not a single lever.

On the direct comparison, a 2024 study in Scientific Reports held the architecture fixed at ResNet-18 and compared a model-centric approach against a data-centric one across MNIST, Fashion-MNIST, and CIFAR-10. The data-centric approach won by a relative margin of at least three percent (Scientific Reports, 2024).

Three percent is not a revolution, and it's on image benchmarks rather than your enterprise data estate. Take it for what it is: evidence that the effect is real and measurable in a controlled setting, from the only kind of study that can establish direction of causation. The enterprise numbers are larger and softer. McKinsey's 2026 analysis found that only seven percent of companies have fully scaled AI across their organization, with data readiness named as a primary constraint.

In our work with teams debugging production AI systems , the pattern holds more strongly than the benchmark numbers suggest, for a specific reason. Benchmark datasets have one source of truth by construction. Enterprise environments have four, and no machine-readable way to pick. That ambiguity has no equivalent in an academic study, and it's usually the largest single cause of wrong answers we find. It also explains why the order you fix things in matters more than how much you fix.

Which defect to fix first

The standard advice ends at "improve your data quality," which is roughly as actionable as "be better at engineering." Defects are not equally expensive to fix and not equally damaging to leave, so sequence matters more than coverage.

A working order, roughly cheapest-and-highest-leverage first:

1. Resolve source-of-record ambiguity. For the questions your system actually answers, decide which dataset is authoritative and record that decision somewhere a machine can read. This is often a week of decisions rather than a quarter of engineering, and it invalidates everything downstream if you skip it. There's no point improving the accuracy of a table that shouldn't have been retrieved.

2. Fix freshness and lineage on the datasets that serve the workload. Not the whole estate. The specific tables and document sets your retrieval layer touches. Know when each was last updated and where it came from. This kills the stale-source symptom directly.

3. Make meaning machine-readable. Column descriptions, document metadata, definitions of what a field means and when it applies. This is the piece that replaces the analyst who knew which revenue column finance trusts. It's unglamorous, and it's the highest-value work most teams haven't started.

4. Complete the fields the workload depends on. Again, scoped. Completeness across the estate is a program. Completeness on the six fields your agent reads before acting is a sprint.

5. Then accuracy at the record level. This is where most data quality programs start, and it's the most expensive place to start. Record-level accuracy work on a dataset that isn't authoritative and isn't documented is effort spent on a foundation you haven't verified.

Map each step back to the symptom list at the top of this article and you'll see that steps 1 through 3 kill most of them. That ordering is the argument. Broad remediation programs that begin at step 5 are why so many data quality initiatives run for a year and produce no visible improvement in AI output.

The obvious objection at this point is that a tool should handle all of this. Partly true, and the part that isn't is the expensive part.

What automated data quality tooling solves, and what it doesn't

Manually authored validation rules don't scale. A data team defines expectations for a few dozen critical tables, the estate grows to thousands, and coverage becomes a rounding error. Databricks made this case in their 2026 write-up on monitoring at scale, and it's the honest reason the category moved toward learned expectations and anomaly detection rather than hand-written assertions.

The current generation of tools, from platform vendors like Databricks and Collibra to specialists like Soda, Acceldata, DQLabs, and Alation, learns what normal looks like for a dataset and flags deviation. That solves a problem hand-written rules could not: it scales detection across an estate nobody has time to hand-annotate.

What it doesn't solve is the ambiguity. Monitoring tells you a dataset changed unexpectedly. It does not tell you that your retrieval layer should have been reading a different dataset entirely. An anomaly detector watching four revenue tables will happily report that all four are healthy, because all four are. The defect isn't in any of them. It's in the absence of a statement about which one is true.

So treat observability as necessary and insufficient. It closes the detection gap. The authority gap, which is where most production AI failures we've traced actually live, closes with decisions and documentation rather than with software. No procurement cycle will do that for you, which is why it keeps getting skipped.

Where the data-centric argument came from

This isn't a new idea, and it helps to know the lineage when you're making the case internally.

Andrew Ng began arguing publicly for what he called data-centric AI around 2021, framing it as the discipline of systematically engineering the data used to build an AI system rather than iterating on the model. His argument rested on two observations from field work. The first was labeling inconsistency: reasonable, well-trained people disagree about whether a pill is chipped or scratched, and that ambiguity propagates into the system as noise. The second was that more data isn't always available or useful. A factory may have made only fifty defective parts. A hospital may have few X-rays of a rare condition. In those settings a small, consistently labeled dataset beats a large, noisy one (MIT Sloan, 2022).

The argument has aged well, and the shift from model-centric to data-centric thinking is now the default position in research (Jarrahi et al., arXiv:2211.14611, 2022). What changed between then and now is who it applies to. In 2021 this was advice for teams training their own models. In 2026, when most organizations consume a foundation model rather than train one , the model is fixed by definition. Everything you control is data, retrieval, and context. The data-centric argument stopped being one school of thought and became the only remaining lever.

What good looks like

The end state is narrower than a clean data estate, and it's achievable in a quarter rather than a multi-year program.

• One authoritative source per question the system answers, recorded somewhere a machine can read, not in the head of the person who's been here longest.
• Lineage a retrieval layer can traverse, so the system knows where a number came from and when.
• Semantics documented where retrieval reaches, so a field's meaning travels with the field.
• Freshness monitored on the datasets that serve the workload, not on the whole estate uniformly.
• A diagnostic habit, meaning the team tests the data before it tests a new model. When the assistant gives a wrong answer, the first move is to look at what it retrieved.

That last one is cultural rather than technical, and it's the one that compounds. Teams that check retrieval first find their real constraints in an afternoon. Teams that reach for the model first find them a quarter later, after the upgrade lands and the same question is still answered wrong.

Key takeaways

• The symptoms most often read as model weakness (confident wrong answers, evals that held and production that didn't, answers that vary by phrasing) are usually retrieval or source-data failures.
• A four-step test settles it: verify an answer by hand, capture what the system retrieved, paste the correct source in directly, and read the result.
• Escalating stakes explain the urgency. The same defect produced a wrong number in a dashboard, then a confident hallucination, then an executed action, each with less visibility than the last.
• Ready is not clean. Readiness is judged against a specific workload, and it now includes machine-readable meaning and machine-evaluable trust.
• Sequence beats coverage. Close the authority gap first, then freshness and lineage, then semantics. Record-level accuracy work last, not first.

Find your constraint before you fund the fix

The diagnostic in this article costs an hour and tells you which side of the model-versus-data question your system is actually on. The harder question is what to do about it once you know, and where the highest-leverage work sits in your specific environment.

AI Readiness Snapshot - a free 30-minute call that maps where AI will have the most immediate impact on your team's cost and reliability, starting from what you're already running.

Book a free 30-min readiness call