Organizations such as the World Bank and the World Food Programme have been ahead of the curve, testing how different mobile methodologies compare to traditional F2F methods, or providing respondents recruited in-person with mobile phones for follow up studies. Others have used remote methodologies when in-person data collection was inadvisable, such as during the 2014-2015 West African Ebola outbreak, but have primarily relied on in-person methodologies where possible.

As the COVID-19 outbreak continues it is clear that for researchers, coronavirus may represent a tipping point in the adoption of remote research methods. Even with precautions in place, face-to-face interactions put both interviewers and respondents at higher risk of contracting a devastating virus, and yet there is a pressing need for data on both COVID-19 itself and the impact it will have across countries and sectors. The question has become not if researchers will need to transition to remote-based methodologies, but how we as a research community can make the most of this unanticipated shift to remote data collection.

Mobile-Based Research in Emerging Regions: Where We are Now

When GeoPoll began offering surveys via 2-way SMS in 2012, we encountered obstacles surrounding not only the buildout of our platform, but also a resistance to the idea that reliable, valid research could be conducted through remote surveys administered through mobile phones. Entering the industry as a mobile technology company, we did not recognize all the nuances and biases of gathering data through methodologies which had not yet been tested at scale.

We overcame questions surrounding the representativeness and quality of data collected through mobile by implementing strict quality control systems and running research experiments that expanded our knowledge of the strengths and weaknesses of various mobile modes. One study we conducted found that select-all-that-apply style questions have a lower number of responses in SMS than forced-choice questions, while another found that drop-off rates increase for SMS questionnaires as they get longer.

Studies by other organizations have examined response and completion rates, and the sample skew of various modes. Research on mobile samples has found that mobile phone owners tend to be more educated than general populations, and that the population reached varies depending on the mode uses. Studies have also looked at how to improve SMS opt-in rates, finding that higher incentives don’t lead to higher response rates, but that sending reminders can improve the participation of older and less educated respondents. The World Food Programme has shown that mobile surveys can accurately demonstrate food security trends at a lower cost than face-to-face interviews, and The World Bank has conducted various studies during health crisis or conflict which demonstrate the usefulness of mobile surveys especially during rapid-response situations.

The Future of Mobile-Based Research in Emerging Region

Although mobile-based research in countries in Africa, Asia, and Latin America has increased in the past few years, it has often supplemented in-person research. For fully-mobile studies such as those being conducted during COVID-19, Computer Assisted Telephone Interviewing, live voice calls conducted from a call center or remotely, has emerged as a favored mode as it allows for longer interviews than SMS, is interviewer-administered, and can reach illiterate populations.

While limited research is available that directly compares results from CATI studies to face-to-face research, studies have found CATI to have higher response rates than other mobile modes, and that CATI is better at reaching older populations than SMS or IVR, an automated voice-call method. During COVID-19, CATI is being used by multiple academic and governmental groups who are looking to balance the need for remote data collection with research requirements. GeoPoll itself has seen a huge increase in demand for CATI surveys, resulting in the completion of nearly 100,000 CATI interviews in the month of July alone.

In research, there is often a fear of the unknown, but the benefits of remote data collection – namely the relative speed, wide reach, and relatively low cost – often outweigh the methodological challenges. During a time when in-person research is impossible, researchers have the ability to test these methods at scale so they can be used not only during times of crisis, but incorporated as part of a viable research approach in all projects. Research is ever-evolving, and at a moment when many are being forced to experiment with new methods, we have the opportunity to document learnings that will benefit the industry even after the pandemic has subsided.

The post COVID-19: A Tipping Point for Remote Research appeared first on GeoPoll.

Despite mobile-based methodologies being the safest and most effective way to gather data during a crisis such as COVID-19, there are still unknown factors when using mobile to collect data. Who can be reached, what modes are best suited to each project, and how questionnaires should be designed are just a few of the questions that come up when organizations are looking to transition projects from face-to-face methodologies to mobile surveys.

While formal research on the usage of mobile surveys is sparse, below is an overview of the research available and anecdotal evidence from GeoPoll’s 8+ years conducting remote survey work in Africa, Asia, and Latin America.

What can mobile phone surveys be used for?

Mobile surveys have been used as a tool by reputable organizations including the World Bank, The United Nations World Food Programme, Unilever, GIZ, and Insight2Impact for several years. There is some debate over if mobile surveys can yet be used to fully replace face-to-face studies, however, there is agreement that mobile surveys excel at collecting rapid data during crises. USAID, FHI360, Keystone Accountability, and others have utilized remote methodologies to identify vulnerable populations and shifting trends during crises including the 2014-2015 West Africa Ebola outbreak. Organizations such as the World Food Programme have spearheaded the usage of mobile surveys in multiple regions to gather food security data, finding that mobile was able to correctly identify trends and seasonal shifts in food security.

Additionally, there is evidence that mobile is better at gathering data on sensitive questions than in-person modes. Research GeoPoll conducted with Kantar TNS found that SMS respondents were more likely to indicate that they felt unsafe in their homes and that they have gone without food than respondents from the compared face-to-face survey.

Who can be reached with remote mobile surveys?

The mobile methodologies that are feasible during COVID-19 (CATI, SMS, IVR, and mobile web or mobile application) require respondents to have access to a mobile phone through which they will answer survey questions. While some studies, such as the World Bank’s Listening To Africa project, provide mobile phones to respondents through initial face-to-face outreach, at present organizations can only reach respondents through mobile phones they already have. This reduces the sampling frame of any given study to those with access to mobile devices, but mobile penetration in Africa and other regions is increasing each year; a 2017 study by Pew research found over 80% of the population in 6 African countries had access to some sort of mobile phone.

Studies have found that the mobile population overrepresents those who are more educated, male, and younger age groups in many countries, but it’s still possible to target those who are older or less educated – it simply may require a larger sample base to draw from to find these respondents. GeoPoll has often targeted very specific groups, from farmers of certain crops to mothers of young children, and is able to do so through careful sampling methods and screening questions. We can also create samples which are nationally representative by key demographics through a stratified random sampling approach and use of quotas to reach the desired sample size within each demographic group.

Finally, GeoPoll uses mobile-friendly Living Standard Measures questions to calculate the LSM or SEC group of respondents. GeoPoll’s recent studies on coronavirus and other studies have included respondents from the lowest SEC groups, who typically do not have running water or electricity in their homes. Certain modes may be better for targeting certain groups; CATI has been found to be better at reaching older age groups, while IVR may reach those less educated.

Who you will target also depends on the sample source; GeoPoll has access to mobile subscriber databases in over 60 countries which we can draw from, or we can gather sample through an enhanced Random Digit Dialing (RDD) process, or recruitment through online, radio, or TV advertisements. GeoPoll can also send surveys to provided lists of contacts for those looking to reach a pre-recruited group.

What mobile survey mode should I use?

Current available modes from GeoPoll include:

• SMS: Surveys conducted via a 2-way SMS chat
• CATI: Voice calls placed by trained interviewers working remotely during COVID-19
• IVR: Automated voice calls with a recording
• Mobile web: A mobile web link that is sent via SMS or another mode and opens a basic web browser
• Mobile application: Mobile applications which administer surveys with those with access to smartphones.

The ideal mode for conducting research will depend on a variety of factors, including questionnaire length, budget, and target respondent groups. It has been found that CATI and IVR are generally more expensive than SMS, however, cost varies widely by mode and is also dependent on countries studied, screening criteria, number of questions, and other factors. In order to get a true estimate of cost, you should contact a research firm such as GeoPoll who can provide price quotes by mode for your specific project. 

How should I design questionnaires for mobile?

As many remote mobile methodologies are self-administered (with the exception of CATI) and have other limitations such as character limits, questionnaires for mobile must keep the mode of survey research in mind. For example, while one study found that individual question length didn’t affect response rates, studies GeoPoll and others have done have found that longer SMS studies have lower completion rates. Additional studies have demonstrated that certain question types may work better in certain modes; for example, a GeoPoll study found that SMS select-all-that-apply questions yield fewer answers than forced-choice questions.

What other factors should I consider when conducting mobile research?

Other factors to consider when embarking on a mobile research project include:

• Incentives: Findings are mixed on the use of incentives; A study in Ghana and Tanzania found small incentives increased completion rates, but that higher incentives had similar effects to lower amounts, and other studies have found lesser effects. Some surveys may also cost respondents airtime; GeoPoll and other services often use zero-rated shortcodes to send messages so that they can be received and replied to even when respondents do not have airtime.
• Language: Many countries speak multiple languages, and surveys should be offered in more than one language – for SMS, services such as GeoPoll offer multi-language surveys, and for CATI, research providers should have multiple interviewers with different language skills. GeoPoll’s call centers are staffed with interviewers who often speak up to 6 languages, depending on the country.
• Local Context: It has also been found that the local context is important. For example, dialects and wording intent can vary throughout regions, which is why GeoPoll always uses local interviewers to conduct voice calls and makes multiple checks on other survey types before sending them out. GeoPoll has also found that in countries such as Nigeria, female interviewers have higher response rates.
• Speed: The speed at which you are looking to collect data will have an impact on the mode you choose. SMS and mobile web surveys can be sent to tens of thousands of respondents at once, while the speed at which CATI interviews are conducted depends on the number of interviewers hired. GeoPoll has also found that IVR response rates can be very low, which can slow down data collection.
• Cost: Your budget for conducting research will be a factor in determining mode, sample size, and other aspects of data collection. While there is a tendency to think lower-cost equals lower-quality, this is not always the case; For many projects, SMS and mobile web surveys are a good option for gathering quality data at lower costs than CATI.
• Data Output: Data can often be delivered in multiple formats – some research organizations will provide raw data in Excel or SPSS, and others may do data analysis for you. GeoPoll has a full-service research team who do data cleanup and analysis and can provide raw data, create reports, and build custom dashboards.

Conducting Mobile Surveys during COVID-19

Although there is still much research to be done on mobile surveys, coronavirus provides researchers an opportunity to test new methodologies that will be referred back to for many years to come. While mobile may not yet reach every person on the globe, it will within the coming decades, and so we must continue to test different methodologies in order to better understand the nuances of mobile data collection.

When conducting mobile research, we recommend using a firm like GeoPoll that has years of experience in the nuances of remote mobile methodologies. To request a quote from GeoPoll for mobile-based research, please contact us here.

The post Frequently Asked Questions around Mobile Phone Surveys appeared first on GeoPoll.

In-person research is one of the oldest methods of data collection and remains popular today despite the high costs often associated with this methodology. Face-to-face methodologies are still used regularly in emerging regions such as those in Africa, Asia, and Latin America, where researchers looking to reach lower socioeconomic classes often turn to in-person interviewers.

However, the coronavirus outbreak has made in-person research impossible in many countries due to travel restrictions and lockdowns, and inadvisable in other regions due to the health risks associated with interviewers traveling door-to-door. In order to keep interviewers and their subjects safe, researchers must quickly find alternative ways to collect data, especially when that data can provide vital information on food availability, healthcare-seeking behaviors, and consumer perceptions.

Best Practices for Remote Data Collection

At GeoPoll we have been conducting research in emerging regions through remote, mobile-based methodologies since our launch in 2014. Our expertise in collecting data through modes including SMS, mobile web, and Computer-Assisted Telephone Interviewing (CATI) enables us to assist those looking to transition in-person projects to remote modes. Below are some of our tips for adapting in-person studies to remote methodologies:

Assess available research modes: Remote research modes that are commonly used in emerging regions include 2-way SMS, CATI voice calls, IVR calls, mobile web links, and mobile applications. While CATI call centers may not be able to continue to operate out of a central location in all countries, providers such as GeoPoll are already working on systems to allow interviewers to conduct calls from their own homes.

Consider the population you are reaching: While in-person modes can reach all socioeconomic classes, increased mobile penetration around the globe allows for remote research to be conducted. When looking to reach lower socioeconomic classes, voice call interviews conducted through CATI will provide the closest comparison to in-person research as they can reach illiterate populations. IVR may also be utilized, but often has low response rates and high costs.

Use a reputable sample provider: Unless you are reaching your own list of respondents, you will need to use a reputable sample provider to gather a representative sample. Firms including GeoPoll have pre-stratified databases of mobile respondents in many markets. We have also seen success using Random Digit Dialing methods in certain countries.

Adjust your survey design: Surveys designed for face-to-face administration will need to be adjusted for a remote methodology. SMS surveys must be designed with several factors in mind including character limits and overall questionnaire length, and mobile web questionnaires should also be on the short side to minimize drop-off rates. CATI surveys can be slightly longer, but researchers should still take length and survey structure into consideration.

Compensate respondents: As with in-person research, respondents should be compensated for their time. If messages are not zero-rated, researchers should also take into consideration the data costs associated with responding to a survey. GeoPoll has found that incentive amounts of $0.50 to $1.00 are sufficient in most markets, and can deposit incentives immediately upon survey completion through several methods including airtime and mobile money.

Work with a partner experienced in remote data collection: When administered with care, remote methodologies can collect data that is of equal quality to in-person research. However, to get the best results you should work with a research organization that has experience in best practices around study design, research modes, and sampling techniques for a remote study.

Conduct Remote Research with GeoPoll

GeoPoll’s team are experts in remote data collection methodologies and can help you transition in-person studies to remote, mobile-based methodologies quickly and effectively. To speak to a member of our team about your project, please contact us today.

The post How to Transition Face-to-Face to Remote Research Methodologies appeared first on GeoPoll.

The survey targeted medium and large businesses in Liberia that paid and filed relevant business taxes. The overall goal of the survey was to better understand the perceptions and experiences that Liberia’s business taxpayers had filing and paying taxes with the Liberian Revenue Authority. Before the survey was full implemented in September 2017, GeoPoll performed a series of tests via text message (SMS) to better understand the target sample.

Selecting Survey Modes

Once GeoPoll understood the complexities and sensitivities of the study as well as the limited sample size of business taxpayers, GeoPoll initially recommended to the use of Computer Assisted Telephone Interviews (CATI) as the mode for the project. However, after a number of discussions with DAI RG3 team members, the team agreed that GeoPoll would test the survey via SMS to gain insight if response rates increased after changing key factors with implementation of the survey, despite the complex and sensitive nature of the survey’s subject matter.

Beginning in July 2017, GeoPoll tested a SMS survey with 39 questions with advanced conditional logic and routing of respondents. The sample GeoPoll utilized initially consisted of 1,756 business taxpayers in Liberia, with an additional 120 eligible respondents being added after several DAI RG3 programmatic events that invited taxpayers. The majority of these sample members had been previously contacted by GeoPoll through informative one-way messages, but were not pre-notified directly regarding the survey nor had the majority of the sample previously take a SMS survey on GeoPoll’s platform. These factors were an important characteristic of the sample, since prior engagement and pre-notification of a survey are all empirically proven to increase response rates, especially if pre-notification of a survey occurs within the same mode.

For approximately a month and a half, from July until the conclusion at the end of August, GeoPoll tested:

• The frequency of reminder messages to respondents as well as sending the initial survey invitation message different times throughout the day and different days of the week.
• Increased airtime incentive amounts from five USD to 10 USD. GeoPoll offers a small airtime incentive for all of its surveys, but this amount is normally significantly less than five USD.
• Different wording of the Opt-in messages to garner more attention from potential respondents.
• Additionally, towards the end of testing, DAI RG3 staff made direct calls to a portion of the respondents in the sample to ask them to take the survey they received via SMS.

Testing Results

The SMS testing confirmed GeoPoll’s original suggestion that SMS would not be a functional mode for a study with a very limited sample and a sensitive topic. Due to the specific targeting of this survey, GeoPoll saw much lower response rates than seen with other SMS survey projects. Although nearly 60 sample members initially engaged with the opt-in message, only four completed the survey entirely. These results allowed GeoPoll and the DAI RG3 Team to make the collective decision to switch entirely to CATI for the full implementation.

The Survey

In the final stage of the project, GeoPoll utilized a sample of approximately 3,000 medium and large business tax payers in Liberia to conduct a 41 question CATI survey, achieving 506 completed surveys. CATI is a survey method by which data is collected by an interviewer through calling phone numbers to reach respondents, asking them questions from a computerized script and recording the data electronically on a computer. CATI software is customizable and allows for researchers to control the flow of a survey based on a respondent’s answers to questions, as well as information already known about the respondent. The benefits of this mode include:

• Interviewers allow for in depth probing to complex/difficult questions.
• Interviewers are able to gauge respondent understanding of questions, and probe for clarification if needed, and ensure appropriate responses are given.
• The mode reaches populations that are not as accessible using other modes.

These benefits made CATI an ideal mode for conducting a very targeted survey on a complex and sensitive topic such as filing taxes.

Impact Assessed

An initial assessment of the data displays that business taxpayers in Liberia believe that many of filing processes are burdensome, although the majority of respondents noted that it only took one or two hours to file their respective returns. However, the majority of respondents had problems filing their business taxes due to the tax s system being down or because the wait at the Liberian Revenue Authority was too long. DAI’s RG3 program and GeoPoll will continue to analyze the data and results yielded from this unique study.

The post Testing Survey Modes to Collect Taxpayer Data in Liberia appeared first on GeoPoll.

Jeremy Bowles (PhD student in Government at Harvard University) and Horacio Larreguy (Associate Professor of Government at Harvard University) are conducting a study in partnership with GeoPoll using CATI to understand how perceptions of candidates to the Liberian House of Representatives are affected by a series of debates held ahead of the October 10 general election. What is CATI and what are the best practices in CATI project implementation?

The study is conducted in partnership with United States Agency for International Development (USAID), the USAID-funded Internews Liberia Media Development (LMD) project, and NORC at the University of Chicago, an objective non-partisan research institution. Through a baseline and end-line survey across all 73 electoral districts in Liberia, GeoPoll is administering a survey through computer-assisted telephone interviewing (CATI) that tracks perceptions of district-level candidates before and after the 2017 general election.

What is CATI?
CATI is a method of mobile data collection through which live enumerators call a respondent on their mobile phone and administer the survey live with the respondent. The study leaders and GeoPoll determined CATI as the preferred mobile data collection method due to logistical and financial conditions that did not allow for an in-person survey, concerns that an SMS survey would limit the length of the survey, and the politically-sensitive nature of the questions. 

How do we decide on CATI?
During the design stage of any project, GeoPoll works with its project partners to assess which mode is most appropriate for the project’s end goal. CATI is frequently advantageous in situations where the target beneficiary group may be illiterate, or when the survey covers a complex or sensitive topic.

CATI Best Practices
Launching the CATI center in Liberia for this project highlights several best practices in CATI project implementation, especially on projects aimed at reaching hard-to-reach populations: 

1. Understand the mode most appropriate for the target beneficiary population at the design stage. In instances where the survey covers complex or sensitive topics, a live call survey method may encourage a higher response rate than another method of mobile survey data collection.
2. For a location-based study, understand how to reach respondents at their verified specific location. This project aims to reach respondents at a very specific location – their electoral district – which was not pre-identifiable in the GeoPoll database. Through a survey design that includes routing CATI enumerators to a list of possible electoral districts per county, GeoPoll was able to locate users at this specific sub-location to assure that the candidate they were referencing was that respondent’s district candidate.
3. In survey design, understand how the target population will interpret political questions. For instance, what type of voting information will respondents have on hand? Will respondents know their voting center or candidate names? This information is necessary to ensure respondents’ ability to answer questions accurately.

Project Implementation
To implement the Liberia Candidate Debates survey, GeoPoll set up a CATI calling center specifically for the project.  Steps GeoPoll took to set up the CATI center included: 

1. GeoPoll hired approximately 25 Liberian enumerators and call managers. Local enumerators are important in CATI so that calls are done in the local language and dialect;
2. GeoPoll CATI implementation specialists traveled to Monrovia to train the local enumerators and set up data collection and quality control systems;
3. Utilizing the GeoPoll database, which has more than 1 million mobile phone users in Liberia, enumerators called up to 300 phone numbers per day, which were pre-stratified by county in Liberia. The respondent’s particular electoral district within that county was then confirmed through questions in the survey;
4. The baseline survey assessed respondents’ levels of political information, knowledge of particular political topics, preferences over policy issues in the local district, and voting intentions. Respondents were asked if they had listened to the district-level candidate debates, what the most important issue was in their district, and which Representative running in their district they planned to vote for. As with all GeoPoll surveys, all answers to the survey are confidential. These perceptions will be compared to the results of a post-survey after the election.

GeoPoll completed the baseline survey of nearly 5,000 respondents prior to the election; the end-line survey is currently under way. Watch for future installments of this blog post series to see the later stages of the project.

The post Tracking Perceptions of District-Level Candidates in Liberia through Computer-Assisted Telephone Interviewing (CATI) appeared first on GeoPoll.