What's a good sample size?
What’s a good sample size?
How many responses do we need?
How many responses do we need to make it significant?
How many responses do we need for statistically sound results?
What’s the minimum number of interviews we need to get robust results?
Just a few variations on a question we hear regularly from our clients; but what’s the answer? Well, in theory, it’s a simple question, but the answer varies by project, industry and target audience. A client operating in the general consumer environment could expect a very different response to one specialising in a niche sector where the total population (or target) is much smaller.
The textbook approach would be thus:
Before you calculate a sample size, you need to determine a few things about the target population and the sample you need:
Population Size — How many people fit your target demographic or population?
Margin of Error or the Confidence Interval — No sample will be perfect, so you must decide how much error to allow. The confidence interval determines how much higher or lower than the population mean you are willing to let your sample mean fall.
Confidence Level — How confident do you want to be that the actual mean falls within your confidence interval? The most common confidence interval is 95% confident.
Next we’d make the calculations using an online calculator or the old-fashioned way with paper and pen and we’d determine our target sample size.
So, this is ok in a perfect world, but what about in the real world? What about clients with hard to reach audiences or small target databases? A lot of our work focuses on these niche audiences, where the task of completing interviews is significantly more difficult than mainstream research studies. In these cases, its often more about absolutely maximising the response rates from our limited target audience and using the experiences we have acquired to do this.
We’ve worked with trade audiences (e.g. builders, plumbers, engineers, electricians, etc) for years and we’ve developed techniques to maximise survey responses from these time poor and hard to engage respondents. We have delivered some truly memorable results for clients where the sample sizes obtained have exceeding expectations and allowed significant deeper dives into audience sub-groups such as merchant and brand preferences, turnover and spend as well as more traditional sub-analysis like age and experience in the sector.