Validation survey: test your prediction on real people
By the time you send a survey you already believe something: an AI verdict told you, or your own reasoning did. Most validation surveys are built to agree with that belief. This one is built to measure how wrong it was.
The difference is not the questions alone. It is one step before the link goes out, and one number you keep afterwards.
Short answer
How do you validate a startup idea with a survey?
Write your prediction down before sending anything, turn what your plan depends on into hypotheses that could be false, ask only about past behaviour, hide the screening answer among plausible options, and read the results only after at least five responses. Then record the distance between your prediction and what people said.
The five steps
- Prediction first: a result read without a prior prediction almost always feels expected, which is hindsight bias doing the validating
- Hypotheses: three to five statements your plan depends on, each question tied to exactly one of them
- Past behaviour only: the last time, the current workaround, the current spend. Never "would you use" or "would you pay"
- Hidden screening: the role you want sits among plausible alternatives, so nobody can guess their way in
- Five before reading: five answers show direction, not a percentage to quote
- Keep the gap: prediction minus result, per hypothesis. It is the number that tells you how far to trust your next guess
The five steps
Write the prediction down
Before the link goes anywhere, write what you expect: how many people had the problem last month, what they use today, what they spend. One line each, with a number where you can.
Why: Without it, the result will feel obvious whatever it is. The prediction is what turns a survey into a measurement.
Turn beliefs into hypotheses
List the three to five things your plan depends on as statements that could be false: "they lose invoices every month", "they already pay for a tool". Tie every question to exactly one of them.
Why: A question with no hypothesis behind it produces an interesting answer you cannot act on.
Ask only about the past
Every question asks what happened or what is true now: the last time, the current workaround, the current spend. Never whether they would use, like or pay for something.
Why: Future and opinion questions invite kindness. Past behaviour has no polite version.
Hide the screening answer
Ask who the respondent is with a list of plausible roles from your market, with the one you want sitting among the others. Read these answers first and set aside anyone outside your audience.
Why: If the right answer is obvious, people who want to help, or want a reward, pick it. In paid online panels, respondents misstating who they are to qualify is a well-documented problem.
Read after five, then keep the gap
Do not read results until at least five people have answered. Mark each hypothesis confirmed, rejected or unclear, and write the distance from your prediction next to it.
Why: Early answers overweight whoever replied fastest. The gap is the number you will want in three months, when you are deciding whether to trust your next guess.
A template you can copy
Two screening questions and eight about behaviour, in an order that warms people up before asking about money. Replace the brackets; keep the tense in the past or present.
| # | Part | Question |
|---|---|---|
| 1 | Screening | Which of these best describes you? (six roles from your market, yours among them) |
| 2 | Screening | How often do you [do the activity your product touches]? (daily / weekly / monthly / rarely / never) |
| 3 | Warm-up | Tell us briefly what a normal week of [that activity] looks like for you. |
| 4 | Problem | When was the last time [the problem] happened to you? What happened? |
| 5 | Problem | What did it cost you that time: time, money, a customer, something else? |
| 6 | Current solution | What do you do about it today? Which tools, people or workarounds? |
| 7 | Current solution | What have you already tried that did not work, and why did you stop? |
| 8 | Intensity | In the last three months, how many times has this come up? |
| 9 | Money | What do you currently spend on dealing with it, per month, in money or hours? |
| 10 | Open | Is there anything about [the problem] we did not ask that matters? |
- ✕ Would you use a tool that does this?
- ✕ How much would you pay for it?
- ✕ Do you like the idea?
- ✕ Is this a big problem for you?
- ✓ What do you use for this today?
- ✓ What do you spend on it now, per month?
- ✓ When did it last happen, and what did you do?
- ✓ How many times in the last three months?
Reading it honestly
Start with the screening answers. If most respondents are outside your audience, the rest of the survey describes someone else’s problem, however clear it looks.
Then go hypothesis by hypothesis. Confirmed means most people in your audience described the event you predicted, unprompted. Rejected means they described a different reality. Unclear is a legitimate result, and it is what a second, narrower round is for.
Finally, write the gap next to each prediction, including the direction. Were you optimistic or pessimistic? One survey tells you about the idea. Several surveys with their gaps kept tell you about your own judgement, and that lasts longer than any single idea.
Let the verdict be the prediction
In GoNoGo the prediction already exists before the survey does: a voice interview of about fifteen minutes produces a computed verdict and its hypotheses. Reality Check then builds the survey from them in this order, waits for five real answers, marks each hypothesis, and records the gap between the AI score and theirs in your project.
Start free →15 min · No credit card · summary free · Reality Check with a full validation