Doing research can benefit you in many ways. You gain insights with which you can make informed decisions and take appropriate actions. Provided you do it right. But if you don't, research will not (or hardly) provide you with what you want and will only cost you unnecessary time. In this blog, I will mention a few pitfalls to watch out for to help you on your way:

• You are researching because you have to. You start a research project because you need the information. For example, to improve your project, provide accountability, feed decision-making, or create support. However, think about what you want to achieve with the research. Do not research because it fits the process or because it is the way it should be done when you don't need that information.
• Asking the wrong research question ultimately prevents you from getting the answers you are looking for. Formulating the right research question is essential to obtaining the information you need. Formulate the research question based on the information you need. The research question often cannot be changed during the research. If you are collecting information, you can not deviate too much from this during the data collection. Pay attention to this when formulating your research question.
• Choosing the wrong research method. The research method you choose depends on the type of information you need. Hence, don't choose a questionnaire if you want to know underlying motivations. Or: don't choose interviews if you want a lot of numbers and percentages from a large group of people.
• Becoming lost in the amount of information. Once you have gathered all the information, the key is not to get lost in the information and get back to your research question. The results of a research study are not a collection of facts but a coherent answer to your question. Therefore, the facts are clustered so that connections are transparent (and thus formulated), and the conclusions are a logical consequence and answer your question.
• Doing nothing with the results. Just doing a survey won't get you there. The results need to be implemented. How you do this depends on the research you have done. Learning moments or action points often emerge logically from a research study. Sit down for this (with colleagues) to formulate an action plan or implementation plan. Formulate concrete agreements with colleagues about what you will do with the information.

Want to read more about how to go through the steps of doing research properly? Then read these previously published blogs:

- How to arrive at the right research question

- How to choose a suitable research method

- How to conduct data collection?

- Tips for analyzing and reporting research

A questionnaire is a commonly used research method to measure the effects of an activity, project, or program. The question is whether a questionnaire is always an appropriate method. Do you want to make statements about the entire target group and collect a lot of data? Then it is a smart method. Do you want insight into the underlying motivations and opinions of your target group? Then a questionnaire is a less wise choice. Why is a questionnaire a good method, and why a less good method? Here you will find the pros and cons explained:

Pros:

• It allows you to survey a large group of people. This is necessary if you want the results of your research to be representative of the entire research group. You can then make statements like: '80% of the visitors say they learned something from the activity'.
• An interviewer or observer cannot influence answers; this ensures objectivity.
• The way of asking questions is standardized, ensuring unambiguous answers.
• In-depth statistical analyses can be made, for example, to subgroups or to establish correlations.

Cons:

• You have little influence on the response rate, often resulting in a relatively low response rate with this method.
• The respondent cannot tell his story freely; the answers are primarily pre-programmed.
• Respondents tend to answer socially appropriately, even when it is anonymous.
• Underlying motivations are difficult to ascertain with a questionnaire, making the questions and answers superficial. There is no option for follow-up questions.
• Respondents cannot complete the questionnaire. If they quit at the halfway point, it is of little use.
• You can ask a limited number of questions, as response rates drop the more extended the questionnaire is.

In conclusion, questionnaires can be a valuable research tool when applied correctly and when the research objectives align with the method's strengths. They are particularly useful for gathering data from a large and diverse audience, enabling researchers to make generalizations and perform in-depth statistical analyses. However, the limitations of questionnaires, such as potential response bias, inability to probe underlying motivations, and low response rates, should be carefully considered. Depending on the research goals, it may be necessary to complement questionnaire data with other research methods, such as interviews or focus groups, to gain a more comprehensive understanding of the target group's opinions and motivations

After formulating your goals, designing the research instrument, and collecting information, it is time to analyze and report the data. Some things to keep in mind when analyzing and reporting your research:

• Analysis and reporting go hand in hand in practice, while they are actually two different steps that can be done separately. By performing the steps at the same time, you keep the overview and save time.
• Before you start the analysis, it is good to check whether the fieldwork was carried out and recorded correctly. You check whether you have a sufficient response rate (especially for quantitative research), the representativeness of your data and whether you were able to collect adequate information (especially for qualitative research)
• Describe your research group. Especially in quantitative research, you may be asked questions about how you arrived at your conclusions and data.
• The core of the analysis and reporting is answering your sub-questions. Therefore, always consider whether the information also answers the sub-question and whether it adds anything.
• In qualitative research, you're going to group the answers by topic and sub-question. In quantitative methods, you will look at percentages, mean scores, and correlation with a data processing program like excel or spss.
• Determine the suitable form for your report. Consider the report's purpose (to inform, persuade, provoke action, etc.) and who the target group is.
• Make connections between the details and conclusions in your report. Which answers to sub-questions enhance each other? Where do you see links throughout the study? Pull the paragraphs and chapters together with these connections and conclusions.

In a previous blog, I wrote about how best to draw a sample. In the blog, I briefly described the difference between a select and a random sample. There is a substantial difference between the two.

In a random sample, everyone in the focus population has an equal chance of being in the sample. This sample is also called a probability sample or random selection. There are several methods for doing a random sample:

• Single random sampling: the lottery method, you randomly select a number of people. This can be done manually or by computer.
• Systematic random sampling: you choose a random number and then go through your list in steps, for example, 20-120-220 etc.
• The cluster or bunch sampling: the list is divided into several clusters. And then, a group is chosen at random.
• The quota sampling: In advance, it is known how many respondents are needed; when that number is reached, the data collection stops.

In a select sample, not everyone has a chance to be in the sample group. The results apply only to the group being studied. There are a number of selective sampling options:

• Snowball sampling: At first, one person is talked to; this person is asked if they know anyone who would also be interesting to talk to. And so on until the sample is large enough. When doing this, make sure you approach people with different perspectives; otherwise, your sample is flawed. Use this method when conducting interviews.
• The convenience sample: the researcher approaches people from his circle or the circle of colleagues until the sample is large enough. Of course, this sampling method is not fully representative because the respondents are chosen on insignificant factors (would know).
• Stratified sample: you divide the population into different groups. A representative number of respondents are chosen from each group.
• The two or multi-stage sampling: a sample is first drawn from several main categories. After this, a representative number of respondents are chosen from the selected main categories. For example, first a number of cities are determined, and from these cities, a neighborhood and then a street per neighborhood. This method is not a good reflection of the population.

From the conversations I have with various people who do research stems some confusion about the term research methods. There are theories on which you can conduct a research study, and you have multiple ways of collecting data.

In my blogs, I regularly talk about research methods, referring to the various ways of collecting data.

Once you have clarity about your research questions, you examine what information you can collect and how. There are many different methods to do this. You have qualitative research methods and quantitative research methods. Qualitative research methods are not about facts and figures but rather about how and why. Quantitative research methods are more about facts and figures that can be compared. Also, check out my blog about the difference between qualitative and quantitative research methods.

There are all sorts of research methods. My overview blog of different research methods lists different ways. Also, be creative with the ways you collect data. Look beyond the standard practices of research and make interesting combinations.

Once you have chosen a research method, thorough preparation is essential. Look at what you need per method and take the time to set up your measuring instrument properly. Discuss it with colleagues and test the measuring instrument before using it.

It is not always necessary to collect all the data yourself. You may also use existing sources. Think of databases like SCP or CBS, from which you can buy data, but also from your own administration or cash register system. Use existing databases if you want to know more about a particular subject on which national data is collected or if you require quantitative data about your own organization. A lot of data is readily available. Just make clever use of this.

TIPS

1. Formulate a clear purpose and research question and sub-questions that you seek answers to with existing research sources.
2. Provide keywords and search terms derived from your purpose and research question. This will give you a clear direction to look for relevant sources.
3. Collect current information.
4. Make sure the sources are relevant to your problem statement.
5. Keep track of which information you get from where, so you and your client can see which sources were used.
6. The information provided by existing data sources does not always fully match the problem statement or might be incomplete. In such cases, it is wise to combine existing source research with other research methods.

The range of research methods on offer is enormous, so you can sometimes not see the wood for the trees. That's why I give you a handy overview of different research methods in this blog. With some of the research methods, you will find a link to a blog with more information.

In this overview, I characterize quantitative and qualitative research methods. Would you like to know more about these? In my previous blog, 'When to choose: qualitative research or quantitative research,' you will find information about the difference between these two types of research.

Qualitative Research

• Group discussion: Conversation or discussion with several people about one or more topics.
• Individual interview: Structured or unstructured conversation with one person in which you go into detail about one or more topics.
• Literature Review: Research in which you use research, theories, and information already available (e.g., from a library or on the Internet) based on a problem statement.
• Mystery visitor: Research method in which you use experts who behave as customers or visitors and assess the quality of service or organization.

Quantitative research

• Observing: Observing actual behavior and recording responses.
• Questionnaires: Recording data and opinions of groups of people using a pre-prepared questionnaire. You can have this questionnaire completed digitally or in writing by a large group of people.
• Tear tickets: A research method in which you get a large group of people to answer one question quickly. This can, for example, be done by giving the audience a piece of paper with a statement on it before a show. After the show, the audience can indicate whether they agree or disagree with the statement by making a tear in the piece of paper.
• Existing source research: conducting research using existing datasets of quantitative data that other researchers have already collected. You then use the dataset again to answer a new question.
• Informal conversations: During an informal conversation that is already taking place, you will ask a few specific questions. You record the answers afterward and repeat them to multiple respondents. The respondents are not aware that they are participating in a survey.
• Ten-minute interviews: Short interviews to find out about respondents' experiences, opinions, and motivations. It is stated in advance how long the interview will last. You can use a timer.
• Tracking: following respondents (e.g., visitors to a museum or customers in a store) through a distinct area. This can be done through the Wi-Fi or Bluetooth of their own devices, but also with a device that you give to respondents, with which you follow them.
• Logbook: A document (digital or written) in which you have visitors or participants record events and specific data.

Observation is a method of finding out and recording actual behavior. You choose the observation method to determine how activities, programs, or projects are received and when you want to register spontaneous reactions. While observing, you make an objective report of what you see; there is no direct contact with the respondents. I give you some tips on how to observe effectively:

1. When observing, use an observation list. With a structured observation list, you consistently record objective information, and you can quickly analyze the collected data.
2. While observing, show an open and inquisitive attitude. Avoid jumping to conclusions about behaviors. Make an objective report of what you see.
3. Name or record verbatim what you actually see happening, not what you think is happening.
4. Don't just look at random gestures. A loose gesticulation doesn't say much. Its meaning becomes understandable only when combined with other gestures and signals.
5. Depending on the situation, make it known that you will observe people, for example, if you will follow them during the activity.
6. Combine observations with a (short) interview to indicate what you have seen.
7. By making many observations, you can make representative statements about your research topic or group with supporting figures.
8. The observers should be given clear instructions so that everyone records the observed in the same way and that interpretation of observation is avoided. To make different observers equal, you can do the first observations together.
9. When observing, you can use all kinds of technical aids. There are various tracking devices on the market, but you can also consider a stopwatch to help you measure how long someone is using something.
10. Always observe and consider the circumstances. If someone is standing in a cold room with their arms tightly crossed, chances are they are not defensive and closed, but simply cold.
11. Make a schedule of when to observe in advance and do so at different times under different conditions so that the picture you get is as varied and complete as possible.
12. Disrupt the situation you are observing as little as possible. Thus, do not talk to those being observed and be as inconspicuous as possible.

In my previous blogs, I explained how to determine the right research question and how to choose the proper research method. The next step in the research process is data collection. While gathering data, there are several things you need to keep in mind. I'll give you some tips:

• Determine your research population. It is not always necessary to interview everyone to get a representative picture. Especially in the case of large numbers, such as the voting behavior of all Dutch citizens, a representative sample is drawn.
• Keep a close eye on representativeness while collecting the data (in quantitative research). Does everyone in your sampling actually cooperate? If your subgroups are not equally represented, you can still question the missing respondents or decide to leave out a particular subgroup.
• Check whether you have enough information, especially in qualitative research. If this is not the case, you can conduct an additional interview.
• The data must be registered in a way that the data is clearly arranged and ready for analysis. Having a large pile of completed questionnaires is in itself insufficient for analysis. The best way to register results depends on the research method you have used.

It is not always necessary to speak to everyone to get a representative picture. Especially with large numbers, such as the voting behavior of all Dutch citizens, a representative sample is drawn. A representative sample contains a sufficient number of respondents. Be aware of drop-outs; not everyone in the selection will cooperate with your survey, and all subgroups should be represented, men-women, young-old, ...

Two methods

You can draw your sample from a database of addresses (and background characteristics), such as ticket ordering service (theater) or a purchased address file. You can also ask people directly for your visit, for instance, at the entrance/exit of a museum/theatre. In this case, you determine in an objective way who is asked, for example, by asking every 5th person.

Three steps

1. To ensure that your sample is large enough, you first need to know the minimum number of people who should participate in your study. This depends on the size of your research population, the reliability you are aiming for, and the margin of error you are willing to accept. Calculating your required sample size is done with a sample calculator.
2. Next, you need to assess the likelihood of people cooperating with your research. People on the street are less likely to cooperate with your research than if they have an affinity with the subject (e.g., the theater they frequent) or if the questionnaire is done in a classroom setting. Based on this, you will determine how many people you should ask.
3. Then you're going to see if you have any subgroups in your research population. You can take this into account when drawing the sample or check and correct it afterward. If you take this into account beforehand, you will draw a selective sample. Consider which subgroups you want to take into account and how large these groups are. This is especially easy if you have a database with addresses and characteristics. But also, if you approach respondents directly (e.g., at the entrance of the museum), you can take this into account (only ask men or mjk holders).

Afterward, when you check for proper representation of the subgroups, you draw an a-selective sample. With the completed questionnaires, you're going to check that all subgroups are equally represented.