In computer science, the phrase “garbage in, garbage out” is used to express the idea that poor quality or flawed input data will produce faulty output data. This same principle applies to market research survey design. The data collected from a survey is only as good as the writing and organization of the questions asked.
Designing surveys capable of generating quality data and actionable insights can be a complex process. In this post we will discuss the steps and strategies involved in effective survey design.
Defining the Research Objectives
The first step in survey design is to clearly establish the objectives of the survey. Researchers need to know what they want to learn before they can figure out how to learn it. The objectives should be attainable, specific (rather than general), and represent the most important takeaways the researcher hopes to receive from the study. Once the objectives are set, they will help guide what topics are included in the survey, what questions are asked, and even how questions are worded.
Choosing Survey Question Types
Before writing the survey questions, researchers first have to decide the types of questions to ask. Survey questions can be divided into two distinct categories: closed-ended questions and open-ended questions.
- Closed-ended questions: In closed-ended survey questions, respondents select from a finite set of pre-defined responses. Examples of closed-ended questions include simple yes/no questions, multiple choice questions, and Likert Scale or ratings questions.
- Open-ended questions: In open-ended survey questions, respondents are asked to reply to each question in their own words in a free-form text box, rather than selecting from a set list of options.
Although open-ended questions provide deep insight into the feelings and thought processes of respondents, they are not as objective or easily quantifiable as closed-ended questions. They also take more time and effort for respondents to answer which can lead to survey fatigue. With these factors in mind, most survey designs primarily consist of closed-ended questions followed by one or two open-ended questions near the end of the survey.
Writing Survey Questions
Taking the time to write clear, concise, well-worded survey questions helps to ensure reliable responses and improves response rates. Although writing well-worded questions is as much an art as a science, the list below offers general guidelines for effective question design:
- Keep wording simple. Use simple words and simple sentence structure. Clear, concise wording asks respondents to do less guesswork and increases the accuracy of the resulting data.
- Keep questions neutral. Avoid opinion, bias or loaded words that might lead the respondent toward a particular response.
- Avoid double-barreled questions. Questions that ask for feedback on two different things within the same question are difficult to answer and analyze.
- Do not use absolutes. Terms such as “always,” “all,” “every,” etc., force respondents to agree or disagree completely without nuance or scale.
- Use reference frames. Make sure respondents are all considering the same time and place when answering a question.
- Avoid negative and double-negative questions. Using negative words like “not” or “prevent” and double-negative questions confuse respondents about whether to answer “yes” or “no”.
- Avoid jargon, technical terms and acronyms. Make sure the entire target audience understands the language level and what the question is asking.
- Anticipate all answer choices. Make sure the answer choices are unique (do not overlap) and include all possible options. If an exhaustive list of options is not known, include an “Other-Specify” option.
- Explain sensitive or unexpected questions. Provide a short explanation to justify why a potentially sensitive question is important to the research and reassure respondents that their responses will remain confidential.
Determining Survey Length and Question Sequence
Even with clear, concise, well-worded questions, respondents will eventually lose interest in or abandon surveys that are too long or disorganized.
Optimal Survey Length
To avoid survey fatigue, keep the questionnaire as short as possible. GeoPoll recommends that from start to finish, surveys take no more than 10 minutes to complete, which usually equates to a maximum of 30 questions. Open-ended questions take longer to complete than close-ended questions, allowing for fewer questions overall. By testing the survey both internally and with a pilot group of respondents, researchers can see if the drop-off rates for their survey are higher than expected and adjust before sending the survey to the full sample.
Question Sequence Guidelines
The order in which questions are asked in a survey can influence how they are answered. It can also impact completion rates. General guidelines to consider when organizing the sequence of questions in a survey include:
- Similar to a natural conversation, start the survey with general impersonal questions that will be easy for the respondent to answer. This helps to warm-up respondents and get them interested and involved in the survey.
- Gradually increase question complexity and specificity but avoid overwhelming respondents by asking multiple difficult questions one after the other.
- Group questions by topic and in a logical sequence (again, similar to the natural flow of a conversation). Questions asked out of context can frustrate and confuse respondents.
- Save more sensitive questions until later in the survey. Research suggests respondents may be more willing to answer sensitive questions after already putting in the effort to answer earlier questions, and if they are offended by a sensitive question late in the survey, it will not influence their responses to previous questions.
- Conclude the survey with more general easy to answer questions such as demographics as a warm-down to leave respondents with a positive final impression of the experience.
Conduct Surveys Around the World with GeoPoll
GeoPoll has experience designing and administering surveys all over the world for clients ranging from global brands and international development organizations to local media stations and NGOs. Survey design can be a difficult task and due to its impact on the outcome of a study, GeoPoll’s research experts offer assistance at varying levels based on each client’s needs. We understand that every project is unique and are committed to using our expertise to guide our clients through key decisions to produce the most accurate insights possible. To learn more about GeoPoll’s survey development processes and various research modes, please contact us here.