Reliability

The reliability of a study points to the extent to which the findings are interpretable to a larger whole. In other words, whether the results represent the entire group and not just the respondents. Did enough respondents complete the questionnaire?

The reliability rate indicates the probability that repeating the survey will produce the same results. With a confidence level of 95%, there is a 95% chance that the answers will be the same with other respondents within this target group. Depending on the type of survey, a reliability rate of 95% to 99% is typically adhered to.

Number of respondents needed

How many respondents are needed depends on how large the research population is. Is the research related to a large group? Such as Dutch youngsters between 15 and 18 years old who have participated in a particular project. Or a small group, such as public libraries within a certain cao or all inhabitants of Persingen (village with less than 100 inhabitants in Gelderland)? In the case of a large research population, you need more respondents than with a small group.

But with a small research population, you need a relatively more prominent part of your research population. I use a practical guideline: with large groups (from 5,000 onward), you need about 400 respondents; for smaller groups, this number increases. The exact calculation depends on various factors, such as margin of error* and the Homogeneity of the research population**. Here I use the sample calculator as a guideline and then stay on the safe side.

Right people?

Reliability also depends on whether you have asked the right people. Did you only invite people you know to fill out the questionnaire or of whom you know they are satisfied? Or did you randomly select people? In my blog on sampling, I describe how to select the right respondents. Note that not everyone you ask will cooperate in the survey. In addition, a number of questionnaires are always dropped because they are not correctly completed (few questions completed or internally incongruent). Therefore, your sample should be larger than the number of completed questionnaires you need.

* Error margin: The percentage by which the answer may differ from reality. This is similar, for example, to the correction of speed measurements. The speed measurement contains a possible margin of error of 5%, for which traffic control always corrects to the lower value.

** Homogeneity of the study population: The degree to which the members of the study population are similar to each other. For example, students participating in a project are very different from each other (they are about the same age but have very different opinions) and are therefore not a homogeneous research population. In that case, it is better to draw the sample larger. In contrast, groups that voluntarily visit each other and are questioned on a topic will be more homogeneous, and there will be a less extreme divergence of opinions. This, in turn, allows for more detailed questions to be asked.