Research Panel vs. Sample

The terms “research panel” and “sample” may seem like they are used interchangeably, but, in fact, the two are different in both what they are and how they are used.

Market Research Panels

In the most general sense, panels refer to groups of pre-recruited respondents that have consented to taking surveys; however, the word “panel” can refer to both large, general market research panels recruited by a research company to take any study, and more specific panels who take studies on a specific topic or even the same study over time. For example, a research organization may have a panel based on a specific location or interest—like a USA panel or a panel of farmers. Each panelist on this type of panel consents to be contacted by the research organization if they are selected to participate in a future research project and typically provides the organization with their demographic information. Researchers then select respondents to participate in research projects as they arise based on the demographic needs of each particular project. This type of panel keeps recruitment costs and turnaround time low for individual projects. Market research firms typically have access to these panels, and they are often used by organizations who do not have access to their own respondents to take research studies.

The other use of the word panel refers to strategically selected groups of people who have agreed to participate in a series of surveys that occur at regular intervals over a given period of time. This type of panel is often used for gathering longitudinal data on a particular topic that all participants have some sort of connection to. Sometimes, questionnaires are even identical for each survey in the series that a research panel participates in, which allows for insights to be drawn based on changes in responses to the same questions, by the same respondents, over the course of the survey series.

As an example of this type of panel, an organization focused on providing aid to people living without adequate access to food may create a research panel with the goal of tracking their organization’s success at getting food into the mouths of low-income earners in Darfur. The respondents on the panel would be recruited based on the qualifications of: income below a pre-determined threshold and must live in Darfur. Researchers typically take this one step further when creating a research panel by ensuring that the panel has representation per the demographic distribution of the population in the area—in order to have the panel be representative of the population being studied. Depending on the panel setup and size, each survey may be distributed to the entire panel, or may be distributed to a random selection of respondents in the larger panel. Once there have been two or more surveys distributed in the series, the organization can begin to compare and contrast the resulting data and draw insights on how their food security projects are performing, which is why utilizing a research panel methodology is desirable.

Research Sample

The word sample refers to the specific group who responded to a study. Every single research project has a sample, whether the sample was sourced from a larger panel, or the sample was recruited specifically for the project at hand. Even studies that are conducted over time with the same set of respondents have a sample. The word sample simply refers to the people who participated in a single study, so in survey series’ each individual survey’s participants would be called the sample—even if the samples are the same for each survey in the series.

Pros and Cons of Research Panels

A research panel methodology provides more benefits than just tracking changes over time. For example, research panels often result in high response rates and cheaper recruitment costs overall due to the fact that the respondents opt-in to participate in surveys before beginning the survey series. These benefits, however, come with some risk. Depending on the length and/or complexity of the questionnaire, as well as how frequently the respondents are expected to participate, respondents may become fatigued with the work involved in participation in the long-term. Panels often need to be refreshed so that new respondents are brought in to replace those who no longer wish to participate.

To maintain strong panel retention and high response rates, it is best to weigh the opinions of research experts on how to best balance questionnaire length, frequency of survey participation, and incentives provided to respondents based on the mode of research. Researchers that conduct panel surveys often have learned through years of trial and error how to best engage various populations, which can ensure a project makes the most of the allotted budget.

GeoPoll’s Panels and Capabilities

GeoPoll is a research company focused on utilizing mobile telecommunications technology to reach people in areas of sub-Saharan Africa, Asia, and Latin America. Our expertise in rural and remote areas of the world lend tremendous resources to humanitarian aid organizations and expanding corporate enterprises alike. We use several sampling methodologies, and can create new panels for ongoing research or recruit respondents for one-time studies from our own database of respondents who are pre-stratified by demographics. To learn more about how GeoPoll can help your team accomplish research goals, contact us today.

The post What are Research Panels? appeared first on GeoPoll.

Weighting is a statistical technique in which datasets are manipulated through calculations in order to bring them more in line with the population being studied. The key difference between the initial sample composition and weighting is that weights are applied after data is collected, and allow researchers to correct for issues that occurred during data collection. For this reason, weighting is also known as post-stratification, as it takes place after the sample has been selected, as opposed to pre-stratification, which is used to balance a sample before data has been collected.

Researchers applying weights are most often weighting on demographic characteristics, such as age, gender, location, and education, but weighting can also account for the differences between those who partake or do not partake in research studies (known as self-selection bias). Weights can also minimize any effects the survey design or data collection mode may have on the sample makeup and resulting data.

In addition to weighting on common demographic variables, studies have found that weighting based on other variables such as internet usage and political affiliation can further reduce bias in some cases. If conducting a phone survey, for example, weights can be applied based on mobile versus landline phone users.

Survey Weighting Methods: Raking and Cell Weighting,

There are several ways in which the actual weighting is performed. Two of the most common include cell-based weighting and raking:

Cell-based Weighting

One of the simplest types of weighting, cell-based weighting can be used when you know the number of respondents your sample should have who are, for example, males age 15-24 or females age 25-34. If your desired sample included 100 males aged 15-24 and 80 females aged 25-34 but should have included 80 males aged 15-24 and 120 females aged 25-34, you can apply simple cell-based weights as illustrated to the left.

Raking or RIM Weighting 

Raking, also known as random iterative method (RIM) weighting or iterative proportional fitting, is a slightly more complex method that can be used when you are weighting to a number of variables, but may not know how the variables interlock; For example, if you need 100 females and 120 people aged 25-34, but do not know how many females aged 25-34 are required. With raking, a researcher would first balance the sample based on one variable, such as gender, and then on the next variable, such as age. If the adjustments for one variable affect another variable too much, then more adjustments are performed until a balanced sample is achieved.

Raking is one of the most common and accepted methods of weighting for public opinion surveys, as it allows for weighting based on multiple variables and aims to adjust each variable by as small an amount as possible. It can be performed quite quickly using a statistical software such as SPSS.

Other methods of weighting include matching, in which a researcher selects a set of cases that is representative of the population from another dataset and aims to match cases from the dataset being studied. Logistic regression modelling and propensity weighting are other types of weighting that are used to account for selection bias amongst a sample. For a more in-depth explanation of various weighting methods see this paper from Pew Research and this from the Journal of Official Statistics.

Pros and Cons of Weighting Data

As with any technique used to manipulate a dataset, there are both pros and cons of weighting, and several guidelines that should be kept in mind when weighting data.

Advantages of weighting data include:

• Allows for a dataset to be corrected so that results more accurately represent the population being studied.
• Diminishes the effects of challenges during data collection or inherent biases of the survey mode being used
• Ensure the views of hard-to-reach demographic groups are still considered at an equal proportion to the population in the final data.

Disadvantages of weighting data are:

• Can over-represent the views of one or several people who may not be an accurate reflection of their entire demographic group
• Can inadvertently introduce additional biases into the dataset
• Can make the findings more variable as it increases the standard deviation of answers (check)

In order to reduce the impacts of data weighting, it’s recommended to weight by as few variables as possible. As the number of weighting variables goes up, the greater the risk that the weighting of one variable will confuse or interact with the weighting of another variable. Also, when data must be weighted, try to minimize the sizes of the weights. A general rule of thumb is never to weight a respondent less than .5 (a 50% weighting) nor more than 2.0 (a 200% weighting).

Additional Information on Data Weighting

GeoPoll provides nationally representative data through a combination of a carefully selected sample frame, use of quotas to manage the demographic composition of those who respond to surveys, and application of weights where necessary. GeoPoll can use multiple weighting methods, including both cell-based and raking, based on the project specifications. To learn more about GeoPoll’s data collection and research process, please contact us here.

The post Weighting Survey Data: Methods and Advantages appeared first on GeoPoll.

In this series of blog posts, GeoPoll will outline the various aspects that make up a sample and why each one is important. First, we will examine how sample is selected and the differences between a probability sample and a non-probability sample.

Probability Sample vs Non-Probability Sample

There are two main methods of sampling: Probability sampling and non-probability sampling. In probability sampling, respondents are randomly selected to take part in a survey or other mode of research. For a sample to qualify as a probability sample, each person in a population must have an equal chance of being selected for a study, and the researcher must know the probability that an individual will be selected. Probability sampling is the most common form of sampling for public opinion studies, election polling, and other studies in which results will be applied to a wider population. This is the case whether or not the wider population is very large, such as the population of an entire country, or small, such as young females living in a specific town.

Non-probability sampling is when a sample is created through a non-random process. This could include a researcher sending a survey link to their friends or stopping people on the street. This type of sampling would also include any targeted research that intentionally samples from specific lists such as aid beneficiaries, or participants in a specific training course. Non-probability samples are often used during the exploratory stage of a research project, and in qualitative research, which is more subjective than quantitative research, but are also used for research with specific target populations in mind, such as farmers that grow maize.

Generally speaking, non-probability sampling can be a more cost-effective and faster approach than probability sampling, but this depends on a number of variables including the target population being studied. Certain types of non-probability sampling can also introduce bias into the sample and results. For general population studies intended to represent the entire population of a country or state, probability sampling is usually the preferred method.

Types of Probability Sampling

There are several sampling methods that fall under probability sampling. In each method, those who are within the sample frame have some chance of being selected to participate in a study. Four of the common types of probability sampling are:

Simple Random Sample: The most basic form of probability sampling, in a simple random sample each member of a population is assigned an identifier such as a number, and those selected to be within the sample are picked at random, often using an automated software program.

Stratified Random Sample: A stratified random sample is a step up from complexity from a simple random sample. In this method, the population is divided into sub-groups, such as male and female, and within those sub-groups a simple random sample is performed. This enables a random sample that is representative of a larger population and its specific makeup, such as a country’s population. 

Cluster Sample: In cluster sampling, a population is divided into clusters which are unique, yet represent a diverse group – for example, cities are often used as clusters. From the list of clusters, a select number are randomly selected to take part in a study.

Systematic Sample: Using a systematic sample, participants are selected to be part of a sample using a fixed interval. For example, if using an interval of 5, the sample may consist of the fifth, 10^th, 15^th, and 20^th, and so forth person on a list.

Types of Non-Probability Sample

In non-probability sampling, those who participate in a research study are selected not by random, but due to some factor that gives them the chance of participating in a study that others in the population do not have. Types of non-probability sample include:

Convenience Sample: As its name implies, this method uses people who are convenient to access to complete a study. This could include friends, people walking down a street, or those enrolled in a university course. Convenience sampling is quick and easy, but will not yield results that can be applied to a broader population.

Snowball Sample: A snowball sample works by recruiting some sample members who in turn recruit people they know to join a sample. This method works well for reaching very specific populations who are likely to know others who meet the selection criteria.

Quota Sample: In quota sampling, a population is divided into subgroups by characteristics such as age or location and targets are set for the number of respondents needed from each subgroup. The main difference between quota sampling and stratified random sampling is that a random sampling technique is not used in quota sampling; For example, a researcher could conduct a convenience sample with specific quotas to ensure an equal number of males and females are included, but this technique would still not give every member of the population a chance of being selected and thus would not be a probability sample.

Purposive or Judgmental Sample: Using a purposive or judgmental sampling technique, the sample selection is left up to the researcher and their knowledge of who will fit the study criteria. For example, a purposive sample may include only PhD candidates in a specific subject matter. When studying specific characteristics this selection method may be used, however as the researcher can influence those who are selected to take place in the study, bias may be introduced.

GeoPoll Sampling Methods

GeoPoll uses all of the sampling approaches described above based on the needs and can use probability-based methods for our sample selection, including stratified random sampling, to build nationally representative samples. To learn more, please contact us.

The post Probability and Non-Probability Samples appeared first on GeoPoll.

Reaching the lowest SECs through mobile research

Conducting research with the poorest populations in countries such as Ethiopia and South Sudan has typically been done through in-person interviews. However, GeoPoll has been able to implement high-quality research methods that can reach the lowest SEC populations via the mobile phone, and at a much more reasonable price than face-to-face methods. The secret to our success came from looking at what variables have typically been a barrier to engaging the lowest SEC populations via mobile phone and working with those variables in mind.

Mobile based research that is conducted through certain modes, like SMS and Mobile Web, use written questionnaires to engage respondents. Such modes require a respondent to have sufficient reading and writing skills in order to participate in the survey. In the lowest SEC populations, literacy rates are often low. In order to engage people in low literacy populations, GeoPoll uses voice call surveys, rather than SMS or mobile web surveys, which enables us to conduct research remotely without relying on written surveys.

GeoPoll’s most popular research mode that uses voice calls is Computer Assisted Telephone Interviewing (CATI). CATI surveys are conducted by live enumerators via a voice call with a respondent. Enumerators guide respondents through the survey and record the responses in a GeoPoll mobile application on an electronic tablet device for future analysis. GeoPoll’s CATI call centers are staffed by enumerators fluent in local languages and dialects of the target population to eliminate communication barriers that could exist between the enumerator and respondent. Although CATI is very effective at reaching the low-level SEC populations in emerging markets via mobile phone, GeoPoll also offers Computer Assisted Personal Interviewing (CAPI) as a research mode to aid the in-person interview process.

Reaching mid-level SECs in emerging markets through mobile research methods

When setting out to reach mid-level SECs, literacy rates of the target population should still be considered but there are. In fact, the World Bank published that as of 2018 55% of adults age 15+ in sub-Saharan Africa are literate.

GeoPoll often advises that clients targeting a relatively literate mid-level SEC use the SMS mode of research. SMS is a strong method for collecting research in emerging markets because surveys can be conducted completely remotely, data is available for analysis in near real-time, and costs can be lower than voice call methods. Additionally, SMS surveys are able to engage all people from low-to-mid level SECs up to the wealthiest populations because all literate mobile phone users can receive SMS messages regardless of their device type.

Reaching the wealthy through mobile based research in emerging markets

Although the number of feature phone and smartphone users in emerging markets is increasing, there is still a large portion of people without access to the Internet through their mobile phones. Because of this, survey methods that require Internet access primarily reach those in mid-to-upper tier SEC populations. The importance of this factor in a study will depend on the project at hand.

The modes of mobile research that require Internet connectivity conducted by GeoPoll are mobile web and mobile application. The advantages of these research modes are the capabilities in comparison to the SMS and CATI modes. Mobile web and mobile application surveys can support images, videos, complex question types, and they have lower associated costs than the other modes mentioned in this post. The drawbacks are rooted in the narrower frame of respondent reach.

Reaching all SECs through mobile

At GeoPoll we understand the importance of reaching survey respondents across all social classes for quality research and data. This is why we offer a variety of methodologies to our clients. GeoPoll’s range of survey modes and methods allow us to strategically reach populations that many other research organizations cannot.

With all of the information provided above, it may be hard to determine the best method to use for your survey to reach your target population. Keep in mind that each project has a unique set of needs; GeoPoll’s research experts have years of experience facilitating mobile based research projects all over the globe and can provide guidance on the survey mode, or combination of survey modes, best for your project. GeoPoll research experts are always available to answer your questions and address any concerns you may have. Contact us to speak with a research expert today.

The post Reaching Different Socioeconomic Classes through Mobile Based Research in Emerging Markets appeared first on GeoPoll.

To skip to how GeoPoll builds nationally representative samples, click here.

What is a Nationally Representative Sample

A nationally representative sample is one that has a strong enough similarity to the population of the country being studied that results will be valid. This means ensuring that the sample represents the country’s population in key demographic characteristics.

Being that each country has different population compositions, a sample in a survey will vary depending on the country being studied. For example, in Nigeria, the population skews much younger than in the United States, with estimates that half of the Nigerian population are aged 30 or younger. Given this, a study conducted in Nigeria with a sample size of 500 would include 250 respondents who are 30 or younger, whereas the same study conducted in the U.S. or Europe would have a smaller number of respondents from that age bracket, in line with the aging populations in those regions.

How to Build a Nationally Representative Sample

The first step to building nationally representative samples is to determine the most important demographic variables to consider given the project goals and local context. Age, gender, location, and a measure of socioeconomic class are all commonly used variables in building a nationally representative sample. In many countries, race and religion are also important to include to ensure the sample is as similar to the country’s population as possible.

Population data is typically taken from national censuses, but in emerging markets, where census data is often unreliable, determining the makeup of a nationally representative sample can be challenging. To mitigate this, research agencies such as GeoPoll use the most recent widely accepted estimates for population demographics. In countries where national census bureau data is not available, we often use population estimates from the U.S. Census Bureau’s International Data Base, which compiles multiple data sources to create population and demographic projections.

Sample size is also a consideration when thinking about building a nationally representative sample, as larger sample sizes will have higher confidence intervals and lower margins of error.  A sample size of around 400 will provide a margin of error of 5% at the 95% confidence level for population sizes above 10,000, and the larger the sample the lower the margin of error becomes.

Once the appropriate sample size and the variables being used to build the sample have been determined, the requirements can be broken down into actual numbers of respondents needed.

In Ghana, a sample size of 400 sample size, nationally representative by age, gender, and location, would look like the below:

• 197 male respondents and 203 female respondents
• 121 aged 16-25, 97 age 26-35, 72 age 36-45, 110 age 46+
• 78 respondents from Ashanti region
• 37 respondents from Brong-Ahafo region
• 36 respondents from Central region
• 43 respondents from Eastern region
• 65 respondents from Greater Accra region
• 40 respondents from Northern region
• 17 respondents from Upper East region
• 34 respondents from Volta region
• 39 respondents from Western region

This sampling technique is also known as quota sampling, and below we explain further how GeoPoll targets specific demographics in our database of respondents to reach the quotas we set for a nationally representative study.

Using Quotas for Nationally Representative Studies in Africa, Asia, and Latin America

Quota sampling can become quite complex depending on the number of variables included, and if they are independent or interlocking, meaning two or more variables are grouped. While GeoPoll’s sampling technique depends on the project specifications, in general, our platform sets limits for each demographic group, which enables us to meet the quotas needed for national representation.

In the example above, to achieve a nationally representative sample of 400, GeoPoll would first send an initial opt-in message to a large group of database members. Depending on the requirements, this initial group may be randomly selected, or we may use demographic information that has been collected from previous GeoPoll surveys users have opted-in to to create a stratified random sample. Once survey responses begin coming in, GeoPoll monitors which quotas are being filled, and closes quotas as the desired sample size per group is achieved. If respondents whose demographics match a quota that has already been filled opt-in to the survey, they are told they are no longer eligible in order to prevent over-representation of that group.

GeoPoll collects, regularly verifies, and securely stores the demographic profiles of our respondents, so that if we have not reached a target for one subgroup, we can recruit more respondents in the necessary subgroup until the targets are met. In cases where budgetary constraints or other factors make reaching the required quotas difficult, GeoPoll can also use weighting to bring the achieved sample more in line with population estimates.

Due to our wide reach in emerging markets, GeoPoll is able to achieve the required demographic quotas needed for nationally representative studies, including reaching respondents in many regions, and of multiple age groups, races, and religions. By using multiple survey modes, including voice calls to access illiterate populations, and in-person enumerators in areas that have little to no mobile connectivity, GeoPoll further ensures that all segments of a population are represented.

To get more detailed information on GeoPoll’s sampling process and learn how we reach nationally representative populations in countries throughout Africa, Asia, or Latin America, please contact us today.

The post How GeoPoll Conducts Nationally Representative Surveys appeared first on GeoPoll.