Max Laktsionau September 4, 2026 11 min read AI-assisted

What Your PRD Is Missing When the Product Is Probabilistic

Q: What failure modes should an AI feature PRD specify?

Four states, each needing a different response: wrong but confident , uncertain , refusing , and unavailable (provider down, rate-limited, or timing out). Malformed output is worth naming as a fifth where the feature parses structured responses. For each state the requirement answers two questions in this order: what does the user see, and what does the system do next. For wrong-but-confident, specify what limits the blast radius, such as a checkable source citation, a confirmation step before anything irreversible, or a review queue for high-stakes outputs. For unavailable, specify the fallback explicitly: a smaller or faster model, a cached response, a non-AI path, or an honest message. A useful rule of thumb is to degrade capability before you degrade the user's momentum, preserving context and partial progress instead of forcing a restart. If a PRD names an AI task and no fallback for it, the feature is not fully specified.

Leadership Workflow

"The assistant returns an accurate summary of the ticket." That sentence passes review, gets signed off, and cannot be verified, because there is no single run of the system that either satisfies it or fails it.

That one line is where most AI product requirements come apart. Everything upstream of it worked. The problem was framed, the users were named, the scope was agreed, the non-goals were written down. Then one clause assumed a deterministic system, and every later argument about whether the feature is finished has nothing to appeal to.

Here is the useful way to see it: in an AI PRD, the parts about intent survive and the parts about verification break. Four kinds of requirement that traditional software never needed now decide whether the product works. How output quality gets evaluated. What happens when the model is wrong. What the system is allowed to see and do. What it costs to run.

Below is what belongs in each one, and the seven questions an engineering lead should be able to answer before anyone signs.

What a traditional PRD still gets right

Start with the reassuring part, because much of the writing on this topic implies you need a new document. You do not.

The problem statement carries over unchanged. So does the user and the job they are trying to do, the scope and the non-goals, the success metrics tied to business outcomes, and the stakeholder alignment the document exists to create. A PRD for an AI feature is still a PRD. If your current one is vague about who the user is, adding an evaluation section will not save it.

The parts that survive are the parts about intent. The parts that break are the parts about verification. Once the system produces a different answer to the same input twice , every clause written in the language of "the system shall" needs a second look. Start with the one you have already written.

Where a normal requirement quietly breaks

Take the ticket-summary line above and put it under pressure. What does "accurate" mean here. Accurate against what reference. On which tickets. How many of them. Who decides.

None of those questions have answers in the document, so they get answered informally later, usually in a review where a product manager and an engineer look at four examples and disagree. That is the failure. The document left a definition out, and the model gets blamed for it.

The fix is a different shape, not more words. Here is the same requirement written two ways:

Written the old way	Written for a probabilistic system
The assistant returns an accurate summary of the ticket.	On a held-out set of 200 tickets sampled across the four highest-volume queues, at least 90 percent of summaries contain every field a support lead marks as required, and no summary states a fact absent from the ticket thread. Graded by two reviewers against a published rubric, disagreements resolved by a third.
Response time should be fast.	95th-percentile time to first token under 2 seconds, full response under 8 seconds, measured at the API boundary under expected peak load.
The assistant should handle unclear requests gracefully.	When the retrieved context does not contain an answer, the assistant states that it cannot answer and offers the human handoff. It does not produce a partial answer. This behavior is checked by 30 adversarial cases in the eval set.

The second column is longer, and that is the point. Its figures are illustrative rather than measured, chosen to show the shape a requirement takes. It is longer because it contains the decisions somebody was going to have to make anyway, made now by the people who should make them rather than later by whoever happens to be in the room.

Three of the right-hand cells lean on something the left-hand ones never mention: an eval set. That is the section most PRDs do not have.

Evaluation requirements, the section nobody writes

This is the section most missing from real AI PRDs, and it is the one that decides whether the feature can ever be called done.

An evaluation requirement specifies how output quality is measured. At minimum it names five things:

The eval set. How many examples, drawn from where, and covering which cases. Real production inputs beat invented ones. Say who builds it and who owns it, because an eval set with no owner rots within a quarter.
The grading method. Human review against a written rubric, an automated check, a model-based grader, or some combination. If a model grades the output, say what validates the grader.
The threshold. A number that separates ship from do-not-ship. Not "high quality". A percentage, on a named metric, over a named slice.
Offline before ship, online after. Offline evaluation runs against the fixed set and gates the release. Online evaluation watches real traffic, because production inputs drift away from any set you froze.
The regression gate. The check that must pass before anyone swaps the model, changes the prompt, or updates the retrieval index. Without this clause, a routine dependency bump silently changes product behavior.

Write these as fields in the document, not as a paragraph of intent. A product manager and an engineering lead reading the same five fields can disagree productively. A paragraph about quality gives them nothing to disagree about.

One caution. An eval set is a sample. It tells you the system behaved on inputs like the ones you chose, which is a smaller claim than it sounds. Teams that treat a green eval as proof of correctness are the ones surprised six weeks later.

Acceptance criteria that survive non-determinism

Acceptance criteria are where the evaluation requirement meets the user story, and they need the same treatment.

The move is from a claim about one run to a bounded claim about many. "The summary is correct" becomes "on the eval set, at least N percent of summaries meet the rubric." That is testable, repeatable, and arguable in the way a requirement should be.

Two things belong in acceptance criteria for an AI feature that are easy to forget:

Confidence and abstention. What the product does when the model is not sure is a product decision. Does it answer anyway. Does it answer with a caveat. Does it decline and hand off. Somebody will decide this, and if the PRD is silent the decision gets made by whoever writes the prompt on a Thursday afternoon.

Behavior on the tail. Most acceptance criteria describe the common case. For a probabilistic system, the interesting cases are the rare ones, and they need their own criteria: inputs in the wrong language, empty inputs, hostile inputs, inputs that look like the common case but are not.

Reforge's guide to writing a PRD for a generative AI feature, written by a working AI product manager, makes the same core point from the other direction: the fundamental skills are unchanged, but the document has to account for output that is non-deterministic. The skill is not new. The section is.

Criteria tell you when the system is right often enough. They say nothing about the rest of the time.

Failure-mode requirements

Traditional software fails in ways engineering owns. An AI feature fails in ways the user sees, which makes failure a product surface and puts it in the PRD.

Four states are worth specifying separately, because they need different responses:

Wrong but confident. The system produces a plausible answer that is incorrect. This is the expensive one. Specify what limits the blast radius: a source citation the user can check, a required confirmation step before anything irreversible happens, or a review queue for high-stakes outputs.
Uncertain. The system can tell it is on thin ice. Specify what the user sees and what the system does next, and treat that as a designed screen rather than an error state.
Refusing. The system declines, correctly or incorrectly. Specify what the user is offered instead, because a bare refusal with no path forward reads as a broken product.
Unavailable. The model provider is down, rate-limiting, or timing out. Specify the fallback: a cached response, a degraded non-AI path, or an honest message. Something will happen here, and the only question is whether you chose it.

For each state the requirement answers two questions. What does the user see, and what does the system do next. Write it in that order.

Safety and data-boundary requirements

Safety in a PRD comes down to a short list of line items that constrain what you are allowed to build.

Untrusted input. If the feature reads content the user did not author, such as an email body, a web page, a shared document, that content can carry instructions. Say what the system is allowed to do on the strength of retrieved text alone.
Data the feature may see. Name the sources it can read and the ones it cannot. Permission boundaries that hold in the application do not automatically hold inside a retrieval index.
Personal data in prompts. Whether it is allowed, what is redacted, and what leaves your infrastructure.
Tenancy. In a multi-tenant product, state explicitly that retrieval cannot cross a tenant boundary, and how that is tested.
Logging and retention. What of the prompt and the response is stored, for how long, who can read it, and whether it can be used to improve the system.

The UK government's DEFRA AI SDLC guidance is unusual among the public guidance we reviewed in treating these as first-class requirement material rather than a compliance appendix. That is the right instinct. Each one is a decision that changes the architecture , so it belongs in the document that precedes the architecture.

The same is true of the numbers nobody puts in a PRD.

Operational constraints are functional requirements

For traditional software, cost and latency are non-functional requirements, filed at the back and reviewed at the end. For an AI feature they change what you are allowed to build, which makes them functional.

A retrieval step that adds 900 milliseconds may rule out an inline interaction and push the feature into a background job. A cost per call that looks trivial at pilot volume may rule out running the feature on every record, and if you have not started building yet, those same two numbers are the sharpest filter you can put on a vendor's proposal. Those are scope decisions, and finding them after the architecture is fixed is expensive.

State the constraints as numbers:

Latency budget. Time to first token and time to complete, at the percentile that matters, under expected peak.
Cost ceiling. Per call, and per accepted result. The second number is the honest one, because a cheap call that gets rejected twice is not cheap.
Volume assumptions. Calls per day at launch and at the volume you expect in a year. Then say what happens if it is ten times that.
Model version pinning. Which model, which version, and whether the provider is allowed to update it underneath you.
Deprecation plan. What happens when that version is retired. Which evals must pass before the replacement ships.

The last two are the ones almost nobody writes (they read like operations, not product), and they are the ones that turn a working feature into an incident on a Tuesday morning six months from now.

How to run the requirements review

The document is only half of it. The review is where the missing sections become visible.

Get the product manager and the engineering lead in the same room with the draft, and work through this list. Every question should have an answer in the document, not in someone's head.

What is the eval set, who built it, and what does passing look like as a number.
Which acceptance criterion tells us this feature is finished.
What does the user see when the system is wrong, and what limits the damage.
What is the feature allowed to read, and what does it do with untrusted text.
What is the latency budget and the cost per accepted result.
What must pass before we change the model.
Who owns the definition of acceptable behavior for this feature, by name.

That last question matters more than it looks. Evaluation thresholds, refusal behavior, and the point at which a human takes over are all product decisions with real consequences, and unowned decisions default to whoever is closest to the code. Name the owner in the document.

If three or more of those questions have no answer, the PRD is describing a deterministic product. That is not the one being built.

Pressure-test your requirements process before the next build

The gap this article describes is rarely a knowledge gap. Teams know their AI features behave differently. The requirements process just has not caught up, and the cost of that shows up as rework after the architecture is set.

AI Transformation Discovery - a one-week sprint that puts your requirements process for AI features under pressure, before the next build starts, and hands back a concrete roadmap.

Book a Discovery Sprint

Frequently asked questions

The difference is verification, not intent. A traditional PRD's problem statement, users, scope, non-goals and success metrics all carry over to an AI feature unchanged. What breaks is every clause written in the language of "the system shall", because a probabilistic system gives a different answer to the same input twice, so a requirement like "returns an accurate summary" has no run that either satisfies it or does not.

Four requirement types have to be added to close that gap: evaluation requirements (how output quality is measured and what score ships), failure-mode requirements (what the user sees when the system is wrong, uncertain, refusing, or unavailable), safety and data-boundary requirements (what the feature may read, what it may act on, what is logged), and operational constraints (latency budget, cost per accepted result, model version pinning and deprecation). If a PRD for an AI feature contains none of these, it is describing a deterministic product.

Replace the claim about a single run with a bounded claim about many. "The summary is correct" becomes "on a named eval set, at least N percent of outputs meet a written rubric." That form is testable, repeatable, and something a product manager and an engineering lead can actually disagree about in review.

A workable acceptance criterion for an AI feature names five things: the eval set and its size, the grading method, the metric, the threshold that separates ship from do-not-ship, and the slice of inputs it applies to. Because scores vary between runs, it is common practice to run the evaluation several times and set an acceptable variance band rather than a single hard cutoff. Two behaviors are easy to leave out and worth writing explicitly: what the system does when it is not confident, and how the criteria apply to tail inputs such as empty, hostile, or wrong-language requests.

At minimum, five fields: the eval set (how many examples, drawn from where, covering which cases, and who owns it), the grading method (human review against a rubric, an automated check, or a model-based grader with something that validates the grader), the metric and threshold, the split between offline evaluation that gates the release and online evaluation that watches real traffic, and the regression gate that must pass before anyone swaps the model, edits the prompt, or rebuilds the retrieval index.

Published guidance commonly puts a starting ground-truth set at roughly 50 to 100 examples for an MVP and around 200 for a fuller set, and recommends breaking accuracy into dimensions such as factual correctness, completeness, relevance and format compliance rather than reporting one number. Write these as named fields rather than a paragraph about quality, and treat a passing eval as evidence about the inputs you chose, not proof of correctness.

Four states, each needing a different response: wrong but confident, uncertain, refusing, and unavailable (provider down, rate-limited, or timing out). Malformed output is worth naming as a fifth where the feature parses structured responses.

For each state the requirement answers two questions in this order: what does the user see, and what does the system do next. For wrong-but-confident, specify what limits the blast radius, such as a checkable source citation, a confirmation step before anything irreversible, or a review queue for high-stakes outputs. For unavailable, specify the fallback explicitly: a smaller or faster model, a cached response, a non-AI path, or an honest message. A useful rule of thumb is to degrade capability before you degrade the user's momentum, preserving context and partial progress instead of forcing a restart. If a PRD names an AI task and no fallback for it, the feature is not fully specified.

In the PRD. For traditional software, cost and latency are non-functional requirements reviewed near the end. For an AI feature they change what you are allowed to build, which makes them functional. A retrieval step that adds most of a second can rule out an inline interaction and push the feature into a background job. A per-call cost that looks trivial at pilot volume can rule out running the feature on every record.

State them as numbers: a latency budget expressed as time to first token and time to complete at the percentile that matters under expected peak, a cost ceiling per call and per accepted result, volume assumptions at launch and in a year, the pinned model version, and the deprecation plan naming which evaluations must pass before a replacement ships. Latency targets should be set from the user surface rather than the model, since the same call can be fine in a background workflow and unusable in an autocomplete. Version pinning buys reproducibility at the price of scheduled migration work later, which is exactly the trade-off the PRD should record.

It has to be a named person in the document, and in practice the product manager owns the definition while engineering owns the implementation that meets it. Evaluation thresholds, refusal and abstention behavior, and the point at which a human takes over are product decisions with real consequences, not implementation details.

The reason to write the name down is that unowned decisions do not disappear, they default to whoever is closest to the code, which usually means they get made while someone is writing a prompt rather than in a review. Common practice is a shared behavior layer with a tight loop rather than a handoff: the product manager frames the user problem and the bar, engineering prototypes the simplest behavior that could meet it, both watch real sessions, and the specification is updated after the prototype rather than frozen before it.

Editorial statement

This article was produced with AI assistance and has undergone human review and editorial control. Max Laktsionau holds editorial responsibility for its content within the meaning of Article 50(4) of Regulation (EU) 2024/1689.