Authors:

• Lidia Awad
• Amanda Berman

Conducting survey research involving sensitive topics poses a significant challenge. Whether seeking information from vulnerable populations, such as young women and adolescents, or a specific subset, such as HIV positive individuals or survivors of gender-based violence, reaching these populations and asking them difficult questions requires delicacy and thoughtfulness.

The post Conducting Remote Research with Sensitive Topics or Vulnerable Populations appeared first on GeoPoll.

Designing surveys capable of generating quality data and actionable insights can be a complex process. In this post we will discuss the steps and strategies involved in effective survey design.

Defining the Research Objectives

The first step in survey design is to clearly establish the objectives of the survey. Researchers need to know what they want to learn before they can figure out how to learn it. The objectives should be attainable, specific (rather than general), and represent the most important takeaways the researcher hopes to receive from the study. Once the objectives are set, they will help guide what topics are included in the survey, what questions are asked, and even how questions are worded.

Choosing Survey Question Types

Before writing the survey questions, researchers first have to decide the types of questions to ask. Survey questions can be divided into two distinct categories: closed-ended questions and open-ended questions.

• Closed-ended questions: In closed-ended survey questions, respondents select from a finite set of pre-defined responses. Examples of closed-ended questions include simple yes/no questions, multiple choice questions, and Likert Scale or ratings questions.
• Open-ended questions: In open-ended survey questions, respondents are asked to reply to each question in their own words in a free-form text box, rather than selecting from a set list of options.

Although open-ended questions provide deep insight into the feelings and thought processes of respondents, they are not as objective or easily quantifiable as closed-ended questions. They also take more time and effort for respondents to answer which can lead to survey fatigue. With these factors in mind, most survey designs primarily consist of closed-ended questions followed by one or two open-ended questions near the end of the survey.

Writing Survey Questions

Taking the time to write clear, concise, well-worded survey questions helps to ensure reliable responses and improves response rates. Although writing well-worded questions is as much an art as a science, the list below offers general guidelines for effective question design:

• Keep wording simple. Use simple words and simple sentence structure. Clear, concise wording asks respondents to do less guesswork and increases the accuracy of the resulting data.
• Keep questions neutral. Avoid opinion, bias or loaded words that might lead the respondent toward a particular response.

• Avoid double-barreled questions. Questions that ask for feedback on two different things within the same question are difficult to answer and analyze.

• Do not use absolutes. Terms such as “always,” “all,” “every,” etc., force respondents to agree or disagree completely without nuance or scale.

• Use reference frames. Make sure respondents are all considering the same time and place when answering a question.

• Avoid negative and double-negative questions. Using negative words like “not” or “prevent” and double-negative questions confuse respondents about whether to answer “yes” or “no”.

• Avoid jargon, technical terms and acronyms. Make sure the entire target audience understands the language level and what the question is asking.

• Anticipate all answer choices. Make sure the answer choices are unique (do not overlap) and include all possible options. If an exhaustive list of options is not known, include an “Other-Specify” option.
• Explain sensitive or unexpected questions. Provide a short explanation to justify why a potentially sensitive question is important to the research and reassure respondents that their responses will remain confidential.

Determining Survey Length and Question Sequence

Even with clear, concise, well-worded questions, respondents will eventually lose interest in or abandon surveys that are too long or disorganized.

Optimal Survey Length

To avoid survey fatigue, keep the questionnaire as short as possible. GeoPoll recommends that from start to finish, surveys take no more than 10 minutes to complete, which usually equates to a maximum of 30 questions. Open-ended questions take longer to complete than close-ended questions, allowing for fewer questions overall. By testing the survey both internally and with a pilot group of respondents, researchers can see if the drop-off rates for their survey are higher than expected and adjust before sending the survey to the full sample.

Question Sequence Guidelines

The order in which questions are asked in a survey can influence how they are answered. It can also impact completion rates. General guidelines to consider when organizing the sequence of questions in a survey include:

• Similar to a natural conversation, start the survey with general impersonal questions that will be easy for the respondent to answer. This helps to warm-up respondents and get them interested and involved in the survey.
• Gradually increase question complexity and specificity but avoid overwhelming respondents by asking multiple difficult questions one after the other.
• Group questions by topic and in a logical sequence (again, similar to the natural flow of a conversation). Questions asked out of context can frustrate and confuse respondents.
• Save more sensitive questions until later in the survey. Research suggests respondents may be more willing to answer sensitive questions after already putting in the effort to answer earlier questions, and if they are offended by a sensitive question late in the survey, it will not influence their responses to previous questions.
• Conclude the survey with more general easy to answer questions such as demographics as a warm-down to leave respondents with a positive final impression of the experience.

Conduct Surveys Around the World with GeoPoll

GeoPoll has experience designing and administering surveys all over the world for clients ranging from global brands and international development organizations to local media stations and NGOs. Survey design can be a difficult task and due to its impact on the outcome of a study, GeoPoll’s research experts offer assistance at varying levels based on each client’s needs. We understand that every project is unique and are committed to using our expertise to guide our clients through key decisions to produce the most accurate insights possible. To learn more about GeoPoll’s survey development processes and various research modes, please contact us here.

The post How to Write and Design Effective Surveys appeared first on GeoPoll.

A Likert scale is a type of scale used in survey research that measures respondents’ attitudes towards a certain subject. Likert scale questions are single-choice, closed-ended questions, and the primary benefit of using a Likert scale is that it provides more granular information on people’s attitudes towards a subject than a simple yes/no question type. By using a Likert scale, researchers can assess varying levels of agreement, importance, quality, and other factors.

The term Likert comes from the creator of the Likert Scale, Rensis Likert, a social psychologist who invented the scale in the 1930s, and Likert scales are now so commonly used that the term is often used interchangeably with any type of rating scale. While Likert scales are a type of rating scale, they are specific in that they provide respondents with a range of text-based answers that lie along a scale. A full Likert scale consists of a series of related statements, but the term is also used to describe a single-statement question with ‘Likert-type’ responses:

Full Likert scale:

Likert-type question:

Likert scales or Likert-type questions are often used to rank level of agreement with a statement on a scale from 1) Strongly disagree to 5) Strongly agree, however they have a wide range of uses and can also measure items including frequency, quality, importance, and satisfaction. Likert scales can be either unipolar, which measures on a one-way scale, such as from ‘not at all helpful’ to ‘very helpful’, or bipolar, which measures two opposite forces – for example ‘agree’ versus ‘disagree’ or ‘satisfied’ versus ‘dissatisfied’. In either case, the first option should be the furthest in sentiment from the last option.

Likert scales are most commonly 5-point or 7-point scales with a neutral middle-point, such as ‘neither agree nor disagree’ ‘neutral’ or ‘undecided’, but 4 or 6-point Likert scales which eliminate a neutral option can be used when a researcher wants to force a respondent to provide a clear opinion. While a 7-point Likert scale can provide an even greater level of granularity than a 5-point Likert scale, respondents may find it harder to distinguish between the options in a longer scale. Since a 7-point scale takes up more room on a screen, 5-point scales are often preferable for survey modes such as SMS or other mobile-based survey modes, which limitations around question length or screen size.

Likert Scale Examples

In addition to the granularity they bring over to survey research, Likert scales are useful as they provide a number of pre-written answer options that are applicable to a wide range of scenarios, from customer satisfaction to public opinion research. For example, the ‘disagree to agree’ Likert scale as shown below can be applied to nearly any topic by asking respondents to rate their level of agreement with a statement regarding brand affinity, political beliefs, and more. Some of the most useful Likert scale question examples include:

Agree to Disagree Likert Scale

• Strongly Disagree
• Disagree
• Neither agree nor disagree
• Agree
• Strongly Agree

Satisfaction Likert Scale

• Very dissatisfied
• Somewhat dissatisfied
• Neither dissatisfied or satisfied
• Somewhat satisfied
• Very satisfied

Likelihood Likert Scale

• Very unlikely
• Somewhat unlikely
• Neither likely nor unlikely
• Somewhat likely
• Very likely

Good to bad Likert Scale

• Very poor
• Poor
• Average
• Good
• Excellent

Frequency Likert Scale

• Never
• Rarely
• Sometimes
• Often
• Always

Usage of Likert Scale Questions

A major benefit of Likert-type questions is their flexibility, as the above questions can be used to gather information regarding sentiment towards a wide range of topics. By using Likert-type questions or a full Likert scale in conjunction with simple yes/no questions, open-ended questions, and other question types, a researcher can form a complete picture of views towards:

• A product or brand: How likely are you to buy this ice cream brand again?
• Customer satisfaction levels: How satisfied are you with the service being provided to you?
• Political beliefs: Rate your level of agreement with the new government tax policy from strongly disagree to strongly agree
• Actions of individuals or companies: Do you strongly approve, somewhat approve, somewhat disapprove, or strongly disapprove of Company X’s recent decision?

To write a good Likert scale or Likert-type questions, researchers should ensure that the statement or statements being evaluated is clear, the answer options are distinct from each other (as demonstrated in the above pre-written choices), and the scale encompasses all possible options, rather than assuming an experience was satisfactory or that a respondent will positively react to a statement.

At GeoPoll we have a range of experience in developing Likert scales and Likert-type questions for multiple methodologies, including SMS surveys, voice call surveys through Computer Assisted Telephone interviewing, and link-based surveys. To learn more about our capabilities or speak to a member of our team about an upcoming project, please contact us.

The post Likert Scale Examples and Definition appeared first on GeoPoll.

Despite mobile-based methodologies being the safest and most effective way to gather data during a crisis such as COVID-19, there are still unknown factors when using mobile to collect data. Who can be reached, what modes are best suited to each project, and how questionnaires should be designed are just a few of the questions that come up when organizations are looking to transition projects from face-to-face methodologies to mobile surveys.

While formal research on the usage of mobile surveys is sparse, below is an overview of the research available and anecdotal evidence from GeoPoll’s 8+ years conducting remote survey work in Africa, Asia, and Latin America.

What can mobile phone surveys be used for?

Mobile surveys have been used as a tool by reputable organizations including the World Bank, The United Nations World Food Programme, Unilever, GIZ, and Insight2Impact for several years. There is some debate over if mobile surveys can yet be used to fully replace face-to-face studies, however, there is agreement that mobile surveys excel at collecting rapid data during crises. USAID, FHI360, Keystone Accountability, and others have utilized remote methodologies to identify vulnerable populations and shifting trends during crises including the 2014-2015 West Africa Ebola outbreak. Organizations such as the World Food Programme have spearheaded the usage of mobile surveys in multiple regions to gather food security data, finding that mobile was able to correctly identify trends and seasonal shifts in food security.

Additionally, there is evidence that mobile is better at gathering data on sensitive questions than in-person modes. Research GeoPoll conducted with Kantar TNS found that SMS respondents were more likely to indicate that they felt unsafe in their homes and that they have gone without food than respondents from the compared face-to-face survey.

Who can be reached with remote mobile surveys?

The mobile methodologies that are feasible during COVID-19 (CATI, SMS, IVR, and mobile web or mobile application) require respondents to have access to a mobile phone through which they will answer survey questions. While some studies, such as the World Bank’s Listening To Africa project, provide mobile phones to respondents through initial face-to-face outreach, at present organizations can only reach respondents through mobile phones they already have. This reduces the sampling frame of any given study to those with access to mobile devices, but mobile penetration in Africa and other regions is increasing each year; a 2017 study by Pew research found over 80% of the population in 6 African countries had access to some sort of mobile phone.

Studies have found that the mobile population overrepresents those who are more educated, male, and younger age groups in many countries, but it’s still possible to target those who are older or less educated – it simply may require a larger sample base to draw from to find these respondents. GeoPoll has often targeted very specific groups, from farmers of certain crops to mothers of young children, and is able to do so through careful sampling methods and screening questions. We can also create samples which are nationally representative by key demographics through a stratified random sampling approach and use of quotas to reach the desired sample size within each demographic group.

Finally, GeoPoll uses mobile-friendly Living Standard Measures questions to calculate the LSM or SEC group of respondents. GeoPoll’s recent studies on coronavirus and other studies have included respondents from the lowest SEC groups, who typically do not have running water or electricity in their homes. Certain modes may be better for targeting certain groups; CATI has been found to be better at reaching older age groups, while IVR may reach those less educated.

Who you will target also depends on the sample source; GeoPoll has access to mobile subscriber databases in over 60 countries which we can draw from, or we can gather sample through an enhanced Random Digit Dialing (RDD) process, or recruitment through online, radio, or TV advertisements. GeoPoll can also send surveys to provided lists of contacts for those looking to reach a pre-recruited group.

What mobile survey mode should I use?

Current available modes from GeoPoll include:

• SMS: Surveys conducted via a 2-way SMS chat
• CATI: Voice calls placed by trained interviewers working remotely during COVID-19
• IVR: Automated voice calls with a recording
• Mobile web: A mobile web link that is sent via SMS or another mode and opens a basic web browser
• Mobile application: Mobile applications which administer surveys with those with access to smartphones.

The ideal mode for conducting research will depend on a variety of factors, including questionnaire length, budget, and target respondent groups. It has been found that CATI and IVR are generally more expensive than SMS, however, cost varies widely by mode and is also dependent on countries studied, screening criteria, number of questions, and other factors. In order to get a true estimate of cost, you should contact a research firm such as GeoPoll who can provide price quotes by mode for your specific project. 

How should I design questionnaires for mobile?

As many remote mobile methodologies are self-administered (with the exception of CATI) and have other limitations such as character limits, questionnaires for mobile must keep the mode of survey research in mind. For example, while one study found that individual question length didn’t affect response rates, studies GeoPoll and others have done have found that longer SMS studies have lower completion rates. Additional studies have demonstrated that certain question types may work better in certain modes; for example, a GeoPoll study found that SMS select-all-that-apply questions yield fewer answers than forced-choice questions.

What other factors should I consider when conducting mobile research?

Other factors to consider when embarking on a mobile research project include:

• Incentives: Findings are mixed on the use of incentives; A study in Ghana and Tanzania found small incentives increased completion rates, but that higher incentives had similar effects to lower amounts, and other studies have found lesser effects. Some surveys may also cost respondents airtime; GeoPoll and other services often use zero-rated shortcodes to send messages so that they can be received and replied to even when respondents do not have airtime.
• Language: Many countries speak multiple languages, and surveys should be offered in more than one language – for SMS, services such as GeoPoll offer multi-language surveys, and for CATI, research providers should have multiple interviewers with different language skills. GeoPoll’s call centers are staffed with interviewers who often speak up to 6 languages, depending on the country.
• Local Context: It has also been found that the local context is important. For example, dialects and wording intent can vary throughout regions, which is why GeoPoll always uses local interviewers to conduct voice calls and makes multiple checks on other survey types before sending them out. GeoPoll has also found that in countries such as Nigeria, female interviewers have higher response rates.
• Speed: The speed at which you are looking to collect data will have an impact on the mode you choose. SMS and mobile web surveys can be sent to tens of thousands of respondents at once, while the speed at which CATI interviews are conducted depends on the number of interviewers hired. GeoPoll has also found that IVR response rates can be very low, which can slow down data collection.
• Cost: Your budget for conducting research will be a factor in determining mode, sample size, and other aspects of data collection. While there is a tendency to think lower-cost equals lower-quality, this is not always the case; For many projects, SMS and mobile web surveys are a good option for gathering quality data at lower costs than CATI.
• Data Output: Data can often be delivered in multiple formats – some research organizations will provide raw data in Excel or SPSS, and others may do data analysis for you. GeoPoll has a full-service research team who do data cleanup and analysis and can provide raw data, create reports, and build custom dashboards.

Conducting Mobile Surveys during COVID-19

Although there is still much research to be done on mobile surveys, coronavirus provides researchers an opportunity to test new methodologies that will be referred back to for many years to come. While mobile may not yet reach every person on the globe, it will within the coming decades, and so we must continue to test different methodologies in order to better understand the nuances of mobile data collection.

When conducting mobile research, we recommend using a firm like GeoPoll that has years of experience in the nuances of remote mobile methodologies. To request a quote from GeoPoll for mobile-based research, please contact us here.

The post Frequently Asked Questions around Mobile Phone Surveys appeared first on GeoPoll.

SMS surveys are a useful method for conducting mobile research due to their ease, convenience, and wide reach – by administering surveys through SMS, people can answer questions without needing to download and install an application or pick up a phone call at a specific time. Using SMS, research is conducted remotely on an individual’s own mobile devices, with an initial opt-in message being sent first, followed by a series of questions that are answered by typing in answers and sending them back to the same number. SMS surveys are especially useful in emerging markets, where mobile penetration is continually growing, but smartphone and Internet penetration remains low, making online surveys less representative of a country’s population.

SMS surveys are also more cost-effective and faster than face-to-face or voice call interviews, though those modes can reach illiterate populations, which makes them beneficial for certain studies. While SMS surveys can be adapted for many research objectives, some basic guidelines should be followed when designing any SMS survey.

Tips for Designing SMS Surveys

Keep Questions and Answers Under 160 Characters: This rule is both the most important and most challenging to adapt to when designing SMS surveys, especially if you are used to writing questions for online or face-to-face interviews. With SMS, each question and corresponding answer choices must fit into 160 characters, the standard length of a text message. If the text is any longer, it will be split into multiple messages – while messaging services attempt to deliver these in the correct order, they can often come in incorrectly. An example of a question and answer text that fits into 160 characters is below. In order to achieve this, the question has been trimmed of unnecessary spaces, and the text kept as simple as possible while maintaining the integrity of the question:

• • In what types of stores do you shop for food and drink?
    1)Malls
    2)Retail shops
    3)Department Stores
    4)Supermarket
    5)Specialty Stores
    6)Discount Stores
    7)Other
    

To measure question length, you can use the =LEN formula in Excel – GeoPoll’s standard questionnaire template for SMS has this formula in a column to the right of every question. Using abbreviations or slang can also help remove unnecessary characters, though overuse of these terms is not recommended. Finally, if it proves impossible to fit all questions and answer options into 160 characters, you can break the message into two by placing the question in the first message and ending with a message such as “Press 1 to see answer options.” By breaking the messages within the survey design rather than risking them getting split on delivery, you can control how they appear to the survey respondent.

Design Short Questionnaires: In addition to making individual question length short, the overall questionnaire length should also be quite short. GeoPoll recommends that SMS surveys should take about 10-15 minutes to complete, which usually equates to about 30-40 questions, including any screener questions and demographic questions. By testing the survey both internally and with a pilot group of respondents, researchers can see if the drop-off rates for their survey are higher than expected and adjust before sending the survey to the full sample.

Use Clear Opt-in and Instructional Language: As with any survey, the first message should be an opt-in to that specific survey, even if the respondent has already opted-in to be part of a general pool of survey respondents. SMS surveys are no exception, and opt-in language should be clear around the survey length and provided incentive when applicable. Instructional language, such as text that says “Reply with a number” for a multiple-choice question, should also be clear and easy to understand.

Avoid Certain Types of Questions: A critical component of developing SMS questionnaires is understanding what question types they excel at versus those they do not support. Certain question types, such as matrix questions or those that include pictures or videos, are generally not supported by SMS. In addition, research has shown that select all that apply question types do not perform as well via SMS than via other modes such as web link, with respondents selecting fewer choices on average when answering via SMS. Single-choice, yes/no, and ranking questions are all excellent question types to use in an SMS survey. SMS also supports open-ended questions, though due to the character limit of a text message, responses may be shorter than seen with other modes.

Conduct SMS Surveys with GeoPoll

SMS surveys are useful in a wide variety of applications and should be considered for surveys that can be adapted to fit the above guidelines. By partnering with GeoPoll, you will get access to a team of experts in SMS survey design, along with unique platform features such as free-to-respondent messages sent through a short code, and the ability to immediately incentivize respondents upon survey completion. To talk to GeoPoll about conducting your own SMS survey, contact us today.

The post How to Optimize Surveys for SMS appeared first on GeoPoll.