What means reliability?
The reliability of a study relates to the extent to which the researcher can translate the findings into a larger picture So whether the results are representative of the entire group and not just the respondents. Did enough respondents complete the questionnaire?
The reliability rate indicates the probability that repeating the survey will produce the same results. With a confidence level of 95% there is a 95% chance that the answers will be the same with other respondents within this target group. Depending on the type of survey, a reliability percentage of 95% or 99% is used.
How many respondents are needed depends on how large the research population is. Is it about a large group, such as Dutch youngsters between 15 and 18 years old who have participated in a particular project, or a small group, such as public libraries within a certain cao or all inhabitants of Persingen (village with less than 100 inhabitants in Gelderland)? With a large research population, you need more respondents than with a small group. However, with a small research population, you need a relatively larger group of respondents. A practical guideline I use: with large groups (5,000 and up), you need about 400 respondents; for smaller groups, an increasing number. The exact calculation depends on various factors, such as margin of error*, the homogeneity of the research population**. Here I use the sample calculator as a guideline and then make sure I am on the safe side. In this blog you will also find more information about drawing a good sample.
Reliability also has to do with whether you asked the right people. Did you only ask people to fill out the questionnaire you know or know to be satisfied? Or did you select people at random? Note that not everyone you ask will cooperate in the survey. In addition, there are always questionnaires that are discarded because they are not completed correctly (few completed questions or incongruent). Hence, your survey should be larger than the number of completed questionnaires you need.
Margin of error:
The percentage that the answer may differ from reality. For example, this is similar to the correction on speed measurements. The speed measurement contains a possible margin of error of 5%, for which CJIb always corrects downwards.
Homogeneity of the research population:
The degree to which the members of the research population are similar to each other. For example, students participating in a project are very different from each other (they are about the same age but have very different opinions) and are therefore not a homogeneous research population. In that case, it is better to draw the survey more extensive. In contrast, groups that voluntarily visit each other and are questioned on that topic will be more homogeneous. There will be a less extreme divergence of opinions. This offers the possibility of asking more detailed questions.