A questionnaire is a commonly used research method to measure the effects of an activity, project, or program. The question is whether a questionnaire is always an appropriate method. Do you want to make statements about the entire target group and collect a lot of data? Then a questionnaire is a smart method. Do you want insight into the underlying motivations and opinions of your target group? Then a questionnaire is a less wise choice. Why is a questionnaire a good method, and why a less good method? The pros and cons explained:

Pros:

• A questionnaire allows you to survey a large group of people. This is necessary if you want the results of your research to be representative of the entire research group. You can then make statements like: '80% of the visitors say they learned something from the activity'.
• An interviewer or observer cannot influence answers; this ensures objectivity.
• The way of asking questions is standardized, ensuring unambiguous answers.
• In-depth statistical analyses can be made, for example, to subgroups or to establish correlations.

Cons:

• You have little influence on the response rate, often resulting in a relatively low response rate with this method.
• The respondent cannot tell his story freely; the answers are primarily pre-programmed.
• Respondents tend to answer socially appropriately, even when it is anonymous.
• Underlying motivations are difficult to ascertain with a questionnaire, making the questions and answers superficial. There is no option for follow-up questions.
• Respondents cannot complete the questionnaire. If they quit at the halfway point, the questionnaire is of little use.
• You can ask a limited number of questions, as response rates drop the more extended the questionnaire is.

Be creative when choosing your research methods. There are many ways to collect your data. And you can make all sorts of combinations. Think beyond the standard research methods.

By adapting standard methods, you can make it easy for respondents; this will increase your response rate. People will enjoy participating in your research. For example, tear-off cards are very short questionnaires. And short interviews are questionnaires with many open questions. A brief conversation in which a respondent can tell their story over a cup of coffee is a nicer experience than filling out a questionnaire.

Using panels makes good use of people who want to participate in your research, and they are often rewarded for doing so. You pay per respondent, and the panel administrator is happy to help you assess representativeness. You can often select your target group very precisely based on all kinds of background characteristics.

Combining methods gives depth to the information collected. For example, you can interpret the results of a questionnaire in group discussions. Alternatively, you can compile a questionnaire based on a literature review. Or first, analyze existing data files and then ask what is missing in a questionnaire. That way, you don't have to ask several things in your questionnaire, and you can go in-depth about the subject matter in your questionnaire.

You can make respondents keep a logbook and combine this with tracking. This way, you can see what the respondents do, and the respondent also describes it. Please keep the privacy laws in mind.

Combinations I like to use are observations and short conversations based on a questionnaire. Based on what you have observed, you ask some questions. For example, why someone did what they did or how they experienced it.

After formulating your goals, designing the research instrument, and collecting information, it is time to analyze and report the data. Some things to keep in mind when analyzing and reporting your research:

• Analysis and reporting go hand in hand in practice, while they are actually two different steps that can be done separately. By performing the steps at the same time, you keep the overview and save time.
• Before you start the analysis, it is good to check whether the fieldwork was carried out and recorded correctly. You check whether you have a sufficient response rate (especially for quantitative research), the representativeness of your data and whether you were able to collect adequate information (especially for qualitative research)
• Describe your research group. Especially in quantitative research, you may be asked questions about how you arrived at your conclusions and data.
• The core of the analysis and reporting is answering your sub-questions. Therefore, always consider whether the information also answers the sub-question and whether it adds anything.
• In qualitative research, you're going to group the answers by topic and sub-question. In quantitative methods, you will look at percentages, mean scores, and correlation with a data processing program like excel or spss.
• Determine the suitable form for your report. Consider the report's purpose (to inform, persuade, provoke action, etc.) and who the target group is.
• Make connections between the details and conclusions in your report. Which answers to sub-questions enhance each other? Where do you see links throughout the study? Pull the paragraphs and chapters together with these connections and conclusions.

Questionnaires often use scales to measure respondents' opinions but also to look at what they did. However, you can also use scales in observations, logs, tear-off cards, and interviews.

Which scales are there?

There are several different scales you can use, and they all give slightly different information.

• 2-point scale: yes/no
•  3-point scale: a lot/little/not at all or yes/neutral/no or
• 4-point scale: totally disagree/disagree/agree/ totally agree
• 5-point scale: totally disagree/disagree/no opinion/agree/fully agree or 5 stars (from very dissatisfied to very satisfied)
• 10-point scale: a report mark from 1 to 10

Which scale do you choose?

• With the 2-point scale, respondents must fill in extremes; it is either yes or no. This scale gives a very clear picture, and there is no middle ground. Sometimes respondents find this difficult, as people prefer to keep a low profile or add more nuance. Only use this scale when it's about a straightforward yes or no, like with questions about behavior: you have or have not visited a museum.
• The 3-point scale allows respondents to choose extremes but also to answer neutrally. However, when often responding neutrally, this does not yield much for the survey. Also, use this scale especially when asking about behavior: How often do you drink coffee? Not - sometimes - every day
• With the 4-point scale, there is no room for neutral answers, and respondents have to make a choice. It does allow for some nuance, which respondents like. This scale is used when you ask for an opinion, and you want them to choose.
• Another option when asking for opinion or attitude is the 5-point scale. Respondents like this scale because it allows for nuance and provides a middle ground if they cannot or do not want to choose. You can broaden a scale to 7, but that gives the illusion of more distinction, while you can't interpret that. Since this scale is very frequently used, respondents and readers of your research will understand this scale. The five-point scale allows you to present a range of statements that collectively measure something. This gives you a good idea of the opinion or attitude of respondents about a particular subject. When putting together the theories, you need to pay attention to the validity and coherence of your theories.
• The 10-point scale has a broad division between very bad and excellent and everything in between. Everyone is familiar with report cards; however, the disadvantage is that there are some differences in interpretation in the details. Everyone agrees that up to 5 is insufficient, but what is the difference between an 8 and a 9? In addition, some people never give a 10 because perfect does not exist, while others give a 10 if they think it is very good. So this scale gives respondents a lot of room to add nuance, but it is difficult to define. Only use this scale if you want to compare your score with other scores or with scores from previous measurements. In that case, ask for a general grade and express the different parts on a different scale.

The validity of a study explains the extent to which the questions asked measure what they are supposed to measure. In other words, are the questions asked unambiguous. Could the respondent have interpreted the question differently than you had intended it? And regarding the questionnaire as a whole: do the questions asked in the questionnaire answer the research question.

By challenging a questionnaire, you can find out if it is valid. You can do this by presenting the questionnaire to a test person and having them think out loud while completing it. Another possibility is to present the questionnaire to a test group and have them ask questions as soon as a question is unclear. It is essential that your test persons/group resemble your research population. So if you want to present the questionnaire to young people aged 15 to 18, your test persons/group should consist of young people aged 15 to 18.

After this test, especially with a test group, analyze the answers and ask for advice from a fellow researcher. Are the answers consistent? Is the logic within the questionnaire correct? Are the results consistent with similar studies?

It is more important to test, adapt, and retest with newly developed research instruments. Especially if you want to measure more abstract concepts such as attitude or development, extensive testing is needed to get a valid questionnaire.

When using an existing questionnaire, you should take a moment to see if it has already been tested for comprehension and consistency.

In addition to validity, the concept of reliability is often discussed when conducting research. Want to know more about this? Also read my blog about reliability.

When you conduct interviews, you can find out a lot. You collect primarily qualitative data. (There are some tricks to collect quantitative data as well.) Before you start an interview, it is crucial that you have well-defined interview questions. Because you don't ask questions just for fun: respondents have to answer the central question of your research. That is why I am giving you tips on how to design your interview questions as well as possible:

1. 1. Have clearly worded main and sub-questions ready.
   2. Ask questions that connect to your main and sub-questions. To ensure all of them can be answered, mark behind each question which main or sub-question the question connects to.
   3. Put the questions in a logical order that avoids overlap.
   4. Include introductory text in your interview questions. This text should state why participants are being interviewed and how long the interview will take.
   5. Formulate your questions so that the interviewee understands them. You can test this by practicing the protocol with your colleagues. This way, you will also notice if your questions are asked in the proper order.
   6. Make sure to ask your questions objectively. This will prevent biases that will cause the research to produce the wrong answers and will allow for continued questioning during your interview. Prejudice occurs when you ask leading questions such as "Don't you agree?", "Would you...?" or "Is it true that...?".
   7. Make sure there is room to ask more in-depth questions. In-depth questions start with 'why', 'how', 'what' and 'who'. In these questions, you will find the 'gold nuggets' that will provide special insights for your research.
   8. Write a closing text explaining what will be done with the survey results.

   Finally, I would like to inform you that remaining objective during the interview is essential. Your own opinion plays no role in this; make sure the interviewee can tell their own story.

   More tips to prepare for your interview? Then please read my blog '15 tips for a good interview'.

Many people think of interviewing as the widely used method in which the interviewer asks questions to the interviewee. But there are other ways to interview. In this blog, I'll give you some examples.

1. Mind mapping: ask your respondents to write down as much information as possible concerning a central topic. Give your respondent(s) enough time to do this, and then discuss the mind map together. The advantage of mind mapping is that you give respondents the freedom to map out their thoughts and provide them with the opportunity to come up with unexpected ideas. This way, you open doors during the interview that would otherwise remain closed.

2. Post-it: At a few points during the interview, ask your respondents to summarize the interview up to this point in keywords on post-it notes. Ask them to structure these notes. At the end of the interview, discuss the content and structure of the post-it notes. Because you let the respondents structure the interview themselves and allow them to respond immediately, you increase the validity of your research.

3. Use existing (visual) material: during the interview, you let the respondent(s) react to an image, film, sound clip, or data. After an initial reaction, you can use the material to ask various questions or start a (group) discussion. The use of various resources makes an interview livelier. The conversation becomes different by talking about something tangible in front of you or something you just watched together. By waiting for the reactions, you can get unexpected responses.

4. Fill in a schedule or timeline together: you can structure your interview with a plan or timeline that you make up together. Respondents can thus structure their experience, opinion, and knowledge. During the interview, black holes stand out that you can fill in or explain together. If the time frame is extensive, you can use a white roll of wallpaper in a group interview. That way, everyone has insight into the timeline. Respondents will make highlights while filling in a schedule or timeline. It helps respondents when they can work visually assisted.

Twitter

- #Tip Use a different interviewing method than the widely used question-answer. LINK

- Another new blog is online. Interviewing: What forms are there, and when do you use them? LINK

- Besides the well-known question and answer method, there are other ways you can interview. You can read more examples in this blog: LINK

- With an interview, you don't always have to work with a prepared questionnaire. You can also let respondents get started themselves. LINK

LinkedIn groups

Culture Education with Quality, Network Culture Education and Cultural Participation, Coaching and Training

The reliability of a study points to the extent to which the findings are interpretable to a larger whole. In other words, whether the results represent the entire group and not just the respondents. Did enough respondents complete the questionnaire?

The reliability rate indicates the probability that repeating the survey will produce the same results. With a confidence level of 95%, there is a 95% chance that the answers will be the same with other respondents within this target group. Depending on the type of survey, a reliability rate of 95% to 99% is typically adhered to.

How many respondents are needed depends on how large the research population is. Is the research related to a large group, such as Dutch youngsters between 15 and 18 years old who have participated in a particular project, or a small group, such as public libraries within a certain cao or all inhabitants of Persingen (village with less than 100 inhabitants in Gelderland)? In the case of a large research population, you need more respondents than with a small group. But with a small research population, you need a relatively more prominent part of your research population. I use a practical guideline: with large groups (from 5,000 onward), you need about 400 respondents; for smaller groups, this number increases. The exact calculation depends on various factors, such as margin of error* and the Homogeneity of the research population**. Here I use the sample calculator as a guideline and then stay on the safe side.

Reliability also depends on whether you have asked the right people. Did you only invite people you know to fill out the questionnaire or of whom you know they are satisfied? Or did you randomly select people? In my blog on sampling, I describe how to select the right respondents. Note that not everyone you ask will cooperate in the survey. In addition, a number of questionnaires are always dropped because they are not correctly completed (few questions completed or internally incongruent). Therefore, your sample should be larger than the number of completed questionnaires you need.

* Error margin: The percentage by which the answer may differ from reality. This is similar, for example, to the correction of speed measurements. The speed measurement contains a possible margin of error of 5%, for which traffic control always corrects to the lower value.

** Homogeneity of the study population: The degree to which the members of the study population are similar to each other. For example, students participating in a project are very different from each other (they are about the same age but have very different opinions) and are therefore not a homogeneous research population. In that case, it is better to draw the sample larger. In contrast, groups that voluntarily visit each other and are questioned on a topic will be more homogeneous, and there will be a less extreme divergence of opinions. This, in turn, allows for more detailed questions to be asked.

Suppose you want to research a target group of as many as 10,000 people. Do you have to interview all 10,000 people to get the right results? Certainly not; only a part of the research population needs to participate in your research to get representative results. Let me explain what representativeness means and when the results are deemed characteristic.

Representativeness means the degree to which the respondents in a sample group are a good reflection of the target group of your research. Therefore, your research is representative, which means that the conclusion of your research is true for 'everyone' in your research population.

A sample

If you have a research population of 10,000 people, you will ultimately need to interview 400 people to arrive at the opinion of the larger group. This does not mean that you only need to approach 400 people. You have to deal with a response rate. This is the percentage of people who participate in your survey. Your response rate depends on the subject you are researching, how easy and fun it is to participate in your research, and what people get in return. I often use a response rate of 30% because I usually research fun subjects, and I am experienced in making it easy to participate in a survey. I also ask the client for a nice gift for the people who participate in the survey. Tips to increase your response rate

Because not everyone will participate in your survey, you will need to have a larger sample group. If you need 400 respondents and assume a response rate of 30%, you will need to interview a sample group of 400/30%=1,333.

When you conduct a survey, you must keep in mind that the smaller the research population is, the larger the number of respondents will be to arrive at the desired representative results. Sometimes, however, respondents' input is more valuable than the number of participants. In this case, you're referring to qualitative research, and it can be more important to focus on the research results than the representativeness in some cases.

If you want to know how many respondents you need for your research population: go to a sample calculator. These will often immediately tell you how many people you need to approach in your sample.

In a previous blog, I wrote about how best to draw a sample. In the blog, I briefly described the difference between a select and a random sample. There is a substantial difference between the two.

In a random sample, everyone in the focus population has an equal chance of being in the sample. This sample is also called a probability sample or random selection. There are several methods for doing a random sample:

• Single random sampling: the lottery method, you randomly select a number of people. This can be done manually or by computer.
• Systematic random sampling: you choose a random number and then go through your list in steps, for example, 20-120-220 etc.
• The cluster or bunch sampling: the list is divided into several clusters. And then, a group is chosen at random.
• The quota sampling: In advance, it is known how many respondents are needed; when that number is reached, the data collection stops.

In a select sample, not everyone has a chance to be in the sample group. The results apply only to the group being studied. There are a number of selective sampling options:

• Snowball sampling: At first, one person is talked to; this person is asked if they know anyone who would also be interesting to talk to. And so on until the sample is large enough. When doing this, make sure you approach people with different perspectives; otherwise, your sample is flawed. Use this method when conducting interviews.
• The convenience sample: the researcher approaches people from his circle or the circle of colleagues until the sample is large enough. Of course, this sampling method is not fully representative because the respondents are chosen on insignificant factors (would know).
• Stratified sample: you divide the population into different groups. A representative number of respondents are chosen from each group.
• The two or multi-stage sampling: a sample is first drawn from several main categories. After this, a representative number of respondents are chosen from the selected main categories. For example, first a number of cities are determined, and from these cities, a neighborhood and then a street per neighborhood. This method is not a good reflection of the population.