Weighting is a statistical technique in which datasets are manipulated through calculations in order to bring them more in line with the population being studied. The key difference between the initial sample composition and weighting is that weights are applied after data is collected, and allow researchers to correct for issues that occurred during data collection. For this reason, weighting is also known as post-stratification, as it takes place after the sample has been selected, as opposed to pre-stratification, which is used to balance a sample before data has been collected.

Researchers applying weights are most often weighting on demographic characteristics, such as age, gender, location, and education, but weighting can also account for the differences between those who partake or do not partake in research studies (known as self-selection bias). Weights can also minimize any effects the survey design or data collection mode may have on the sample makeup and resulting data.

In addition to weighting on common demographic variables, studies have found that weighting based on other variables such as internet usage and political affiliation can further reduce bias in some cases. If conducting a phone survey, for example, weights can be applied based on mobile versus landline phone users.

Survey Weighting Methods: Raking and Cell Weighting,

There are several ways in which the actual weighting is performed. Two of the most common include cell-based weighting and raking:

Cell-based Weighting

One of the simplest types of weighting, cell-based weighting can be used when you know the number of respondents your sample should have who are, for example, males age 15-24 or females age 25-34. If your desired sample included 100 males aged 15-24 and 80 females aged 25-34 but should have included 80 males aged 15-24 and 120 females aged 25-34, you can apply simple cell-based weights as illustrated to the left.

Raking or RIM Weighting 

Raking, also known as random iterative method (RIM) weighting or iterative proportional fitting, is a slightly more complex method that can be used when you are weighting to a number of variables, but may not know how the variables interlock; For example, if you need 100 females and 120 people aged 25-34, but do not know how many females aged 25-34 are required. With raking, a researcher would first balance the sample based on one variable, such as gender, and then on the next variable, such as age. If the adjustments for one variable affect another variable too much, then more adjustments are performed until a balanced sample is achieved.

Raking is one of the most common and accepted methods of weighting for public opinion surveys, as it allows for weighting based on multiple variables and aims to adjust each variable by as small an amount as possible. It can be performed quite quickly using a statistical software such as SPSS.

Other methods of weighting include matching, in which a researcher selects a set of cases that is representative of the population from another dataset and aims to match cases from the dataset being studied. Logistic regression modelling and propensity weighting are other types of weighting that are used to account for selection bias amongst a sample. For a more in-depth explanation of various weighting methods see this paper from Pew Research and this from the Journal of Official Statistics.

Pros and Cons of Weighting Data

As with any technique used to manipulate a dataset, there are both pros and cons of weighting, and several guidelines that should be kept in mind when weighting data.

Advantages of weighting data include:

• Allows for a dataset to be corrected so that results more accurately represent the population being studied.
• Diminishes the effects of challenges during data collection or inherent biases of the survey mode being used
• Ensure the views of hard-to-reach demographic groups are still considered at an equal proportion to the population in the final data.

Disadvantages of weighting data are:

• Can over-represent the views of one or several people who may not be an accurate reflection of their entire demographic group
• Can inadvertently introduce additional biases into the dataset
• Can make the findings more variable as it increases the standard deviation of answers (check)

In order to reduce the impacts of data weighting, it’s recommended to weight by as few variables as possible. As the number of weighting variables goes up, the greater the risk that the weighting of one variable will confuse or interact with the weighting of another variable. Also, when data must be weighted, try to minimize the sizes of the weights. A general rule of thumb is never to weight a respondent less than .5 (a 50% weighting) nor more than 2.0 (a 200% weighting).

Additional Information on Data Weighting

GeoPoll provides nationally representative data through a combination of a carefully selected sample frame, use of quotas to manage the demographic composition of those who respond to surveys, and application of weights where necessary. GeoPoll can use multiple weighting methods, including both cell-based and raking, based on the project specifications. To learn more about GeoPoll’s data collection and research process, please contact us here.

The post Weighting Survey Data: Methods and Advantages appeared first on GeoPoll.

What is A Sample Frame?

The sample frame is the specific source of respondents that is used to draw the sample from. This could be a map from which specific areas are outlined, a list of registered voters, a phonebook, or another source which specifically defines who will and will not be included in the sample. The sample frame should be representative of the sample universe, which is the broader definition of the sample makeup. For example, if a researcher is looking to study attitudes of students at a specific university, the definitions may look like the below:

• Sample Universe: Current students at University X
• Sample Frame: List of all 10,000 currently enrolled students provided by the admissions office
• Sample: 400 randomly selected students from the list of enrolled students who participate in the research study.

In a general population study, the sample frame may be ‘all households in Country A,’ from which a researcher can randomly select which households take part in a study.

Sampling Error or Non-Sampling Error

When speaking about a sample frame and it’s representatively of the overall population being studied, we must also consider who is not included in the sample frame. Often those who did not participate in a research study are just as important to consider as those who were represented, as without them, key items may be skewed or missed. There are a few types of sampling error, also referred to as non-sampling error:

• Coverage Error: When a sampling frame does not sufficiently cover the population required for a study there is a coverage error. For example, if a national survey is being conducted by telephone and the sample frame is taken from a phonebook, but not all households are listed in the phonebook. A telephone or internet survey will also exclude those who do not use telephones or the internet.
• Nonresponse Error: This error describes those who were contacted for a survey but were unable to or did not want to participate. This could include those who are selected for a telephone or in-person interview and do not pick up the phone or answer their door, or those who answer but refuse to participate.
• Interviewer Error: This error occurs when an interviewer incorrectly records a response for a participant of a study. This is a form of interviewer bias that can be introduced in telephone and in-person interviews. This bias could be due to voice tone or other characteristics and may influence a respondent’s likelihood to participation or their actual answers. For example, GeoPoll has found that females may be more comfortable answering questions from female interviewers.
• Processing Error: This error refers to the technical processing of a study’s data points and errors that occur as data is collected with the use of a technology platform, or during data entry as well as data coding, cleaning, and editing.
• Response Error: This error describes those who participate in a study that either intentionally or accidentally provide inaccurate responses to a study’s questions. This can occur for a variety of reasons related to the comprehension and memory of a study’s participants. Additionally, response error can occur due to social desirability bias that can be introduced into a study when a participant answers in a way they believe would be more acceptable and accurate to their conceptualization of a study’s objective or in a way that abides by social norms. Social desirability has the potential to be introduced into any study, but if often apparent in studies covering sensitive or taboo topics for a particular society.

The above errors can be mitigated through careful sample frame selection and testing of various modes to reduce non-sampling errors. For interview-administered surveys, rigorous training of interviewers is needed to help reduce the influence of biases. For self-administered surveys, understanding local context while in the design stage is important to be able to formulate questions that can be understood clearly and accepted as valid areas of inquiry by the population of interest.

GeoPoll Sample Frames

The creation of a sampling frame for GeoPoll projects depends on client needs, project specifications, and other factors including survey mode. While sampling frames are unique for each project, there are a few common sampling frames that we use which are outlined below.

• Mobile subscribers within a certain country: GeoPoll primarily conducts research through mobile-based methodologies including voice calls and SMS messages. Due to this, sample frames for our studies are often those who have access to a mobile device within each country. GeoPoll reaches mobile subscribers in two primary ways: Partnerships with mobile network operators which enable us to call or send messages to their opted-in subscribers, and Random Digit Dialing (RDD). Using an intelligent RDD process, GeoPoll is able to randomly generate valid phone numbers that match the format of those in each country.
• Census data: GeoPoll also relies on census data and census estimates both to inform nationally representative demographic breakdowns and to create sample frames when conducting in-person research. The availability of up-to-date census data varies by country and requires a researcher to understand what information from reputable sources is available. One resource that can be used to look at each country’s local bureau of statistics and at the U.S. Census Bureau’s International Data Base.
• Aid Beneficiaries: When working with international development clients, GeoPoll is able to survey aid beneficiaries if given their contact information. This requires organizations to provide GeoPoll with a list of beneficiaries’ phone numbers or other contact information.

Determining the appropriate sample frame and other sample criteria for any one project is a complex process that cannot be represented in full here, however, we hope we have given you some insight into how GeoPoll approaches sampling. To learn more about GeoPoll’s processes please contact us here.

The post Sample Frame and Sample Error appeared first on GeoPoll.

In this series of blog posts, GeoPoll will outline the various aspects that make up a sample and why each one is important. First, we will examine how sample is selected and the differences between a probability sample and a non-probability sample.

Probability Sample vs Non-Probability Sample

There are two main methods of sampling: Probability sampling and non-probability sampling. In probability sampling, respondents are randomly selected to take part in a survey or other mode of research. For a sample to qualify as a probability sample, each person in a population must have an equal chance of being selected for a study, and the researcher must know the probability that an individual will be selected. Probability sampling is the most common form of sampling for public opinion studies, election polling, and other studies in which results will be applied to a wider population. This is the case whether or not the wider population is very large, such as the population of an entire country, or small, such as young females living in a specific town.

Non-probability sampling is when a sample is created through a non-random process. This could include a researcher sending a survey link to their friends or stopping people on the street. This type of sampling would also include any targeted research that intentionally samples from specific lists such as aid beneficiaries, or participants in a specific training course. Non-probability samples are often used during the exploratory stage of a research project, and in qualitative research, which is more subjective than quantitative research, but are also used for research with specific target populations in mind, such as farmers that grow maize.

Generally speaking, non-probability sampling can be a more cost-effective and faster approach than probability sampling, but this depends on a number of variables including the target population being studied. Certain types of non-probability sampling can also introduce bias into the sample and results. For general population studies intended to represent the entire population of a country or state, probability sampling is usually the preferred method.

Types of Probability Sampling

There are several sampling methods that fall under probability sampling. In each method, those who are within the sample frame have some chance of being selected to participate in a study. Four of the common types of probability sampling are:

Simple Random Sample: The most basic form of probability sampling, in a simple random sample each member of a population is assigned an identifier such as a number, and those selected to be within the sample are picked at random, often using an automated software program.

Stratified Random Sample: A stratified random sample is a step up from complexity from a simple random sample. In this method, the population is divided into sub-groups, such as male and female, and within those sub-groups a simple random sample is performed. This enables a random sample that is representative of a larger population and its specific makeup, such as a country’s population. 

Cluster Sample: In cluster sampling, a population is divided into clusters which are unique, yet represent a diverse group – for example, cities are often used as clusters. From the list of clusters, a select number are randomly selected to take part in a study.

Systematic Sample: Using a systematic sample, participants are selected to be part of a sample using a fixed interval. For example, if using an interval of 5, the sample may consist of the fifth, 10^th, 15^th, and 20^th, and so forth person on a list.

Types of Non-Probability Sample

In non-probability sampling, those who participate in a research study are selected not by random, but due to some factor that gives them the chance of participating in a study that others in the population do not have. Types of non-probability sample include:

Convenience Sample: As its name implies, this method uses people who are convenient to access to complete a study. This could include friends, people walking down a street, or those enrolled in a university course. Convenience sampling is quick and easy, but will not yield results that can be applied to a broader population.

Snowball Sample: A snowball sample works by recruiting some sample members who in turn recruit people they know to join a sample. This method works well for reaching very specific populations who are likely to know others who meet the selection criteria.

Quota Sample: In quota sampling, a population is divided into subgroups by characteristics such as age or location and targets are set for the number of respondents needed from each subgroup. The main difference between quota sampling and stratified random sampling is that a random sampling technique is not used in quota sampling; For example, a researcher could conduct a convenience sample with specific quotas to ensure an equal number of males and females are included, but this technique would still not give every member of the population a chance of being selected and thus would not be a probability sample.

Purposive or Judgmental Sample: Using a purposive or judgmental sampling technique, the sample selection is left up to the researcher and their knowledge of who will fit the study criteria. For example, a purposive sample may include only PhD candidates in a specific subject matter. When studying specific characteristics this selection method may be used, however as the researcher can influence those who are selected to take place in the study, bias may be introduced.

GeoPoll Sampling Methods

GeoPoll uses all of the sampling approaches described above based on the needs and can use probability-based methods for our sample selection, including stratified random sampling, to build nationally representative samples. To learn more, please contact us.

The post Probability and Non-Probability Samples appeared first on GeoPoll.

To skip to how GeoPoll builds nationally representative samples, click here.

What is a Nationally Representative Sample

A nationally representative sample is one that has a strong enough similarity to the population of the country being studied that results will be valid. This means ensuring that the sample represents the country’s population in key demographic characteristics.

Being that each country has different population compositions, a sample in a survey will vary depending on the country being studied. For example, in Nigeria, the population skews much younger than in the United States, with estimates that half of the Nigerian population are aged 30 or younger. Given this, a study conducted in Nigeria with a sample size of 500 would include 250 respondents who are 30 or younger, whereas the same study conducted in the U.S. or Europe would have a smaller number of respondents from that age bracket, in line with the aging populations in those regions.

How to Build a Nationally Representative Sample

The first step to building nationally representative samples is to determine the most important demographic variables to consider given the project goals and local context. Age, gender, location, and a measure of socioeconomic class are all commonly used variables in building a nationally representative sample. In many countries, race and religion are also important to include to ensure the sample is as similar to the country’s population as possible.

Population data is typically taken from national censuses, but in emerging markets, where census data is often unreliable, determining the makeup of a nationally representative sample can be challenging. To mitigate this, research agencies such as GeoPoll use the most recent widely accepted estimates for population demographics. In countries where national census bureau data is not available, we often use population estimates from the U.S. Census Bureau’s International Data Base, which compiles multiple data sources to create population and demographic projections.

Sample size is also a consideration when thinking about building a nationally representative sample, as larger sample sizes will have higher confidence intervals and lower margins of error.  A sample size of around 400 will provide a margin of error of 5% at the 95% confidence level for population sizes above 10,000, and the larger the sample the lower the margin of error becomes.

Once the appropriate sample size and the variables being used to build the sample have been determined, the requirements can be broken down into actual numbers of respondents needed.

In Ghana, a sample size of 400 sample size, nationally representative by age, gender, and location, would look like the below:

• 197 male respondents and 203 female respondents
• 121 aged 16-25, 97 age 26-35, 72 age 36-45, 110 age 46+
• 78 respondents from Ashanti region
• 37 respondents from Brong-Ahafo region
• 36 respondents from Central region
• 43 respondents from Eastern region
• 65 respondents from Greater Accra region
• 40 respondents from Northern region
• 17 respondents from Upper East region
• 34 respondents from Volta region
• 39 respondents from Western region

This sampling technique is also known as quota sampling, and below we explain further how GeoPoll targets specific demographics in our database of respondents to reach the quotas we set for a nationally representative study.

Using Quotas for Nationally Representative Studies in Africa, Asia, and Latin America

Quota sampling can become quite complex depending on the number of variables included, and if they are independent or interlocking, meaning two or more variables are grouped. While GeoPoll’s sampling technique depends on the project specifications, in general, our platform sets limits for each demographic group, which enables us to meet the quotas needed for national representation.

In the example above, to achieve a nationally representative sample of 400, GeoPoll would first send an initial opt-in message to a large group of database members. Depending on the requirements, this initial group may be randomly selected, or we may use demographic information that has been collected from previous GeoPoll surveys users have opted-in to to create a stratified random sample. Once survey responses begin coming in, GeoPoll monitors which quotas are being filled, and closes quotas as the desired sample size per group is achieved. If respondents whose demographics match a quota that has already been filled opt-in to the survey, they are told they are no longer eligible in order to prevent over-representation of that group.

GeoPoll collects, regularly verifies, and securely stores the demographic profiles of our respondents, so that if we have not reached a target for one subgroup, we can recruit more respondents in the necessary subgroup until the targets are met. In cases where budgetary constraints or other factors make reaching the required quotas difficult, GeoPoll can also use weighting to bring the achieved sample more in line with population estimates.

Due to our wide reach in emerging markets, GeoPoll is able to achieve the required demographic quotas needed for nationally representative studies, including reaching respondents in many regions, and of multiple age groups, races, and religions. By using multiple survey modes, including voice calls to access illiterate populations, and in-person enumerators in areas that have little to no mobile connectivity, GeoPoll further ensures that all segments of a population are represented.

To get more detailed information on GeoPoll’s sampling process and learn how we reach nationally representative populations in countries throughout Africa, Asia, or Latin America, please contact us today.

The post How GeoPoll Conducts Nationally Representative Surveys appeared first on GeoPoll.

One of these research terms is ‘a panel’ which, more often than not, is used interchangeably with ‘sample’. What exactly is a research panel, why is it often confused for a sample, how does it work and are there tips on working with panels?

Understanding Research Panels

A research panel is a group of respondents recruited by research companies to that take part in a survey by answering specific research questions in several sessions over a period of time – a week, month or even years.

Many panels are constituted for quantitative research as a representative sample of a general population. Research panels are more often than not created around interests or around specific products and services such as media audience measurement or consumer insights for the FMCG sector. Through a panel, a researcher is able to track changes in behavior over a period of time – this is also referred to as longitudinal data.

Is There A Difference Between A Panel And A Sample?

A sample is a pool of survey respondents recruited by a research company who are eligible to participate in surveys which can or daily surveys meant to understand behavioral change. A panel is a subset of a sample where survey respondents within the large pool of a research company’s database are recruited based on similar traits to answer questions for the collection of longitudinal data (data from the same person over a set amount of time)

Importance Of Panel Research

The most important trait of panel research as a marketing research method is the quality and usefulness of the data (pdf) which is ultimately determined by the measurements applied by using statistical methods in data analysis that translate the data into information and eventually insights that drive decision making.

The analysis of panel research data such as the continuous consumer purchasing behavior can provide guidance in areas such as pricing, competitor analysis (share of shelf) advertising effectiveness and sales projections.

Benefits

• This method gives you a high response rate as the respondents have expressly opted in and are willing to set aside time to participate in the surveys.
• Due to the diversity and size of panel members, marketers are able to record behavioral changes across different demographics.
• There is more depth in detail in the panel research insights as data analysts are able to co-relate psychographic & demographic data to have a better understanding of the research subject.
• Recruitment of research panels on mobile and internet are cheaper and more convenient due to the growing population of mobile and internet users in the world.
• The cost of recruitment for mobile/online panels is minimal as there is no need to print out questionnaire or travel (focus groups). However, the form of incentives to respondents can drive up the cost.
• Because the panel has been created around a specific focus area of interest it is easier to engage the respondents as they are familiar with your organisation and the information you are seeking from them

Challenges

• The cost of acquiring and retaining a panel can drive up the overall project cost due to a high churn rate which is sometimes as a result of the length of questions or low incentives.
• Sometimes respondents can compromise the quality of data by failing to give honest information about themselves or their behavior. More often than not, such respondents are in it for the incentives and didn’t sign up to help you. It’s therefore important to routinely validate your panel for authenticity randomly and on a regular interval to preserve the quality of data.
• Although emerging markets such as Africa are seeing increased mobile and internet penetration rates, there is still a huge percentage of the population that remains offline. Naturally, those who will be recruited to join mobile or online panels are those with devices or are online. It is therefore advisable to combine various modes for your panel depending on topic or area of focus and if there is a need to have a nationally representative sample.

Panel Research Methods

In order to carry out research studies using a panel to evaluate the thoughts and feelings of a population, there are several different styles of collecting data that you may choose depending on your needs. These include:

1. Filling out a diary
2. Focus groups
3. Online surveys
4. Mobile surveys

The Growth Of Mobile & Online Panels

Traditionally, panel research has been used as a qualitative method. Under this method, the most popular and effective tool for collecting longitudinal data has been focus groups which are interview based sessions in which moderator interviews panelists to collect the data from the same sample over and over again.
Another tool within panel research has been the diary method where respondents take notes in a book or journal documenting what they did, watched or bought over time. This diary tool has been very popular in the past especially in collecting audience measurement data.

Due to advancements in technology coupled with the need for market research agencies to deliver fast, reliable and cost-effective solutions to a data-driven marketplace has led to the adoption of mobile and online surveys which have proved effective in efficiency, quicker turn-around times, and at times, at a fraction of the cost.

Recruiting A Research Panel

There are various channels and modes that can be used to recruit respondents who will become an active sample in a panel. GeoPoll has found that two modes, mobile & online, are the most effective for recruiting new panels of survey respondents, with mobile being the most ideal in emerging markets.

The channels within the mobile mode can vary, going from text messages (SMS), mobile web or mobile app, to voice calls such as those used in Computer-Assisted Telephone Interviewing (CATI) and Interactive Voice Response (IVR) methods. Online recruitment modes include email, social media, and digital advertising. In order to effectively recruit a panel, we send them a short survey that collects crucial demographic information such as their gender, age, education level, and social economic status. 

Our Panels & Capabilities

We currently have a database of over 240 million active users. As the leading mobile surveying platform in emerging markets, our media audience measurement, brand health tracking and customer satisfaction tracking solutions provide marketers with consumer insights drawn from our numerous mobile-based research panels.

Apart from recruiting panels for our research solutions, GeoPoll provides multiple ways for market research agencies who need to use our platform to recruit panels through either mobile SMS, mobile web or mobile app.

If you have questions on how to utilize our sample or you’d like to partner with us to recruit a panel, please get in touch with us

The post Understanding Research Panels; Mobile, Online & How They Work appeared first on GeoPoll.