A small sample can reveal an experience, expose a failure mode or help refine a question. It usually supports limited population claims, especially when recruitment is selective. Report counts, selection and context, and distinguish what was observed from how common it might be.
Match the claim to the way evidence was collected
A small number of interviews may be deliberately chosen to explore contrasting experiences. That design can be useful without estimating prevalence. Explain who was invited, who participated and why those cases were selected. A striking account can establish that a problem occurred for that participant; it cannot establish that most users encounter it. Keep direct observations distinguishable from the analyst's interpretation of their meaning.
Small numerical samples also vary substantially when individual cases change. Report the count behind a percentage and the period covered. Three of five participants is clearer than 60% of customers when the five are volunteers. Even a carefully drawn probability sample may have substantial sampling uncertainty. A larger number does not repair a selection process that excludes relevant people, and a small unbiased sample does not promise precise estimates.
Use depth and contrast instead of manufactured precision
Inspect individual cases and deliberately seek situations that could challenge the emerging explanation. If four participants struggle with account setup, ask whether the fifth used a different device, received assistance or already knew the product. These differences can identify questions for a wider study. They should not be used to quietly remove inconvenient cases or redefine success after the interviews have finished.
Preserve context when summarising themes. Record the task, setting and assistance provided, while protecting personal information. Do not treat the number of comments as the number of people affected: one vocal participant can produce many remarks. If quantifying a qualitative coding exercise, explain whether the unit is a participant, a session or a coded passage. Review ambiguous codes rather than presenting judgement as mechanically objective.
Plan the next evidence needed for the actual decision
Decide whether the present evidence justifies a reversible fix, a further investigation or a broader measurement exercise. A reproducible broken link can be corrected without estimating the percentage of all visitors affected. A major investment based on expected demand requires a defensible population and stronger evidence. Identify what remains unknown before choosing recruitment, observation periods or sample size for the next stage.
Do not invent a confidence interval simply because a stakeholder requests one. Its meaning depends on the sampling process, model and assumptions. Explain those prerequisites and seek statistical input when required. The useful output may instead be a case summary, a list of plausible mechanisms and a targeted test plan. Record what would change the recommendation so that additional data can resolve a decision rather than merely enlarge a spreadsheet.
A fictional five-person booking study
A fictional scenario to explain the method, not a customer result or a market benchmark.
A fictional training provider observes five volunteers attempting to reschedule a booking. Three cannot find the reschedule control, one finds it after assistance and one completes the task unaided. Reporting that 60% of all customers fail would overstate what these sessions show. The recruitment link was sent through a user community, and all five participants had desktop access.
The researcher reports three unassisted failures, one assisted completion and one unaided completion among these five sessions. Session notes show that two failing participants looked under the original confirmation email and the third searched the account page. The team can investigate those pathways and test a clearer label. It cannot estimate mobile failure or overall customer prevalence from this study. Before changing its entire booking system, it plans broader recruitment across devices and recent booking types, with a defined task and success rule. The invented study illustrates evidence boundaries, not a validated product outcome. Notes preserve the assistance given in each session so a future reviewer can distinguish independent completion from a coached result.
TAKE THIS INTO YOUR NEXT CONVERSATION
A practical next-step checklist.
- Report participant counts and recruitment method.
- Separate observed failures from prevalence claims.
- Define the unit used when coding themes.
- Choose the next study around the unresolved decision.
Common questions.
Can one participant reveal a real defect?
Yes. A reproducible defect can be established for that case without estimating how often the wider population encounters it.
Does showing a percentage make five interviews quantitative evidence of prevalence?
No. Formatting a count as a percentage does not change recruitment, coverage or uncertainty.
Further reading & source context.
These primary references provide background. They are not evidence for the fictional example above and do not endorse Accuralytix.
Found an error or a missing qualification? Send an editorial correction.
Check my data readiness.
Use a practical tool to turn this idea into a clearer starting point.
Check my data readiness