Where the decision boundary falls · 8 min read

AI focus groups: what they can decide

A synthetic panel turns twelve options into three in twenty minutes, for roughly the cost of nothing. That is real and it is worth having. It also cannot close a decision, and the reason is not model quality — it is that a persona has no last time to tell you about.

Almost every tool in this category sells the first half of the job. This page is about the second half, and about measuring the distance between them.

Short answer

What can an AI focus group actually decide?

It can sort a wide set of options fast and surface the common objections. It cannot tell you what someone did last time the problem occurred, what they pay today, or whether anyone will buy — because those are facts about behaviour rather than about language. Treat it as phase one, and close with real respondents.

Where the boundary falls

Good at: ranking options, comparing prices for direction, reading sentiment, rehearsing the objections you will meet
Blind to: what someone actually did, what they currently pay, and anything nobody in the panel has ever experienced — because none of them has experienced anything
Panels converge on the majority view and drift towards your question's framing, so the rare objection — the expensive one — is the least likely to appear
The pattern the industry settled on in 2026 is two phases: a synthetic panel to narrow a wide set, then real respondents to confirm the survivors
User Interviews surveyed 150 researchers in May 2026 alongside five moderated interviews: 97% use AI somewhere in their workflow, only 8% trust AI-generated participants for a decision that commits money
The check that matters is the one almost nobody runs: ask real people the same questions and keep the number showing how far the panel was off

Judge a panel's output

Source: User Interviews, State of User Research 2026

What a panel settles, task by task

The split is not about how good the model is. It is about whether the answer requires an event that happened to somebody.

Task	Panel	Why
Sorting a long list of options	Yes	Ranking, comparing and sorting are exactly where synthetic panels behave enough like people to be worth the twenty minutes. Twelve positioning lines become three.
Finding the objections you have not thought of	Partly	It surfaces the common objections reliably and the rare ones almost never — the panel converges on the majority view, and the rare objection is the expensive one.
Reading how a price lands	Partly	Useful for direction and for spotting a price that is obviously wrong. Not useful for the actual number, because nobody in the panel has ever paid for anything.
What someone did last time this happened	No	There was no last time. A persona generates a plausible account rather than recalling an event, and a plausible account is indistinguishable from a real one in the transcript.
What they currently pay, and to whom	No	This is a fact about invoices. It exists in the world, not in language, and cannot be recovered from a model however the persona is described.
Whether anyone will buy	No	Demand is behaviour. The only instruments are a payment, a deposit, a signed commitment, or an alternative someone cancelled.

The two phases

Phase one

Narrow it, in minutes

Ten to twenty personas built from public writing by people in that market answer your questions as a group, and you can then question any one of them by voice — which is where the useful detail usually shows up, because a follow-up gets a specific answer where a survey gets a rating.

Honestly: They are synthetic, we say so on the page that generates them, and nothing here is evidence about demand.

Phase two

Check it with people who exist

The same hypotheses become a survey with a public link, generated from your own project rather than from a template. Nothing is scored until at least five real respondents have answered, and the result is compared against what the panel predicted.

Honestly: The comparison is the point. It is also where most of the work lands on you: the survey has to reach people, and a demand test with nobody in it measures nothing.

Why this matters more than panel quality. Every tool in this category is getting better at phase one, and they are all getting better at roughly the same rate. The difference that survives is whether anything ever checks the panel against people who exist — and whether the tool is willing to write down how far off it was.

How to run phase one without fooling yourself

Ask about the past, not the plan. A persona answering “would you use this” is generating agreement. The same persona answering “what did you do the last time this happened” produces a specific, checkable story — and the fact that the story is invented becomes visible, which is exactly what you want from a rehearsal.

Read the dissent, not the average. The majority view in a synthetic panel is the least informative thing in it, because convergence is the documented failure mode. One persona objecting for an unusual reason is worth more than nine agreeing.

Write down what would change your mind before you run it. If no panel outcome could have stopped you, the panel was entertainment. This one sentence is the cheapest quality control in research.

Then interrogate individuals. Group output is a summary; the detail lives in follow-ups. Being able to question one persona by voice matters for the same reason a follow-up beats a rating scale.

Run both phases

A fifteen-minute interview sets up the hypotheses, the panel rehearses the objections, and the same questions then go out to real respondents so the two can be compared. The verdict is computed against thresholds rather than written by a model.

Start free →

15 min · No credit card · summary free

Frequently asked questions

What is an AI focus group?+

A panel of synthetic respondents — language-model personas built from public writing by people in a market — answering your questions as a group. It runs in minutes instead of weeks and costs a rounding error next to a recruited panel. What it produces is a rehearsal of the objections your idea will meet, which is genuinely useful and is not the same as evidence about demand.

Are AI focus groups accurate?+

Accurate at what is the only useful form of the question. On structured tasks — ranking options, comparing prices, reading sentiment — synthetic panels track human samples closely enough to sort a wide list. On anything requiring recall of a real event, they do not track at all, because there is no event to recall. The industry has settled on a split that says this plainly: in a 2026 study by User Interviews (150 researchers surveyed, alongside five moderated interviews), 97% reported using AI somewhere in their workflow while only 8% trusted AI-generated participants for a decision that actually commits money.

Can an AI focus group replace real customer interviews?+

No, and the reason is structural rather than a question of model quality. A persona cannot tell you what it did last time the problem occurred, what it paid, or what it tried before, because none of that happened. It also converges towards the majority view and drifts towards the framing of your question, so the objection that would have killed your launch is the one least likely to appear. Use the panel to decide what to ask real people, then ask them.

How do you know whether the AI panel was right?+

You compare it with the same questions answered by real people, and you keep the number. That is the part most tools skip: they produce the panel and stop. Here the same hypotheses go out as a public survey, nothing is scored until at least five real respondents have answered, and the system then records how far its earlier prediction sat from what those people actually said. A tool willing to store its own error is making a different kind of claim than one that is merely fast.