At Global Surveyz, we split the survey creating process into three distinct phases: 

The Build, which is where we get to know you and work out your requirements, the type of questions your survey needs and the specifics of your survey sample;

The Data Collection, where we get down to work with collecting the data; 

The Report, where we put all the data through its paces to produce a report tailored to your specific needs. 

Build

Quantitative data

Quantitative data is information involving numbers, so that the data can be used for statistical analysis or mathematical equations. This kind of information is usually collected in the form of an answer to a question such as “how much?” or “how many?” also known as a closed-ended question.  The information gathered can then be analysed using various methods such as cross-tabulation, trend analysis etc, to provide data as to customer behaviour.

Qualitative data

Qualitative data is information which is not based on a numerical value. This will often involve asking the respondent for an opinion. This kind of information can be collected using closed-ended questions, but a more valuable response will be given if an open-ended question is used, so the respondent has a text box to write whatever they wish to. 

Grid Questions

A grid is a great way to combine several questions into one matrix. This can be useful as a space-saving device, or for the neater presentation of the survey. It’s important to know the underlying psychology of grid matrices, or you could end up with less accurate data. According to research (Click Here) if a survey includes a high proportion of grid questions, respondents can end up speeding or “satisficing”, but  this can be alleviated with some simple grid-design tricks such as shading alternate lines. 

Likert scale

The Likert scale is a matrix used in surveys which uses a scale to rate the possible answers. This scale can be in the form of numbers (usually 1-5 or 1-10, though other scales are used for particular questions) or in the form of verbal statements (for example, dissatisfied, neutral, somewhat satisfied, satisfied, extremely satisfied) or even using emojis (for example 😒😐😀😄). These types of questions are very popular with survey-builders as they typically garner the most useful data, however, as with grid questions, it is important to be circumspect in the use of scale matrices, and to use fatigue-busting tactics such as humour to keep the respondent engaged. 

Red herring questions

Red-herring questions are a very important part of survey quality control. The most common type of red-herring question is a multiple choice question in which one of the answers will be a fictitious company name. By inserting these types of questions into the survey, it is possible to weed out any responses which seem to have been written either by someone who is rushing to finish the survey and therefore not answering carefully, which results in false data, or someone who is completing the survey simply to gain the incentive, and does not actually fit the quota, or even surveys which have been filled in by bots!

ESOMAR28

The ESOMAR 28 questionnaire is designed to help find the best research company for the information you need. It includes detailed questions regarding the recruitment and administration of respondents, so that  you get the best sampling for your business needs. The questionnaire was created by the European Society for Opinion and Market Research, a not-for-profit organisation that operates as a guide to the best practises for market research.

Collect

Incentivisation

Data collection companies often use incentive schemes to recruit respondents. This is important because surveys have become so popular that it is essential to find the right respondents for your required demographic. A gift voucher at the end of a 15 minute survey is a great way to ensure a constant supply of eager respondents. There is a danger, though, of people signing up to survey websites simply to collect the incentives, and this can lead to incorrect and useless data, which is where quality control comes in: by using various tools, such as time-stamping (to ensure that a respondent isn’t taking too long to answer, or is answering much too quickly), red-herring questions etc,it is possible to ensure that only the data you need will appear on your survey report.

Sampling

When a series of questions is administered to the entirety of an intended population, this is known as a census. This can be very costly and time-consuming, and is not necessary in many cases. A survey involves addressing questions to a certain group within a population, known as a sample. There are several methods of sampling, for example “simple random sampling” and “stratified sampling”,as well as specific ways to determine the ideal sample size. 

Sample bias

If the criteria for sampling excludes a particular type of respondent, this could lead to sample bias. For example, excluding any previous customers who gave negative feedback would lead to a sample bias leaning towards positive feedback only, and would result in skewed data which is ultimately unlikely to be in the company’s best interests.

Research panel

A research panel is a type of survey sample. This will sometimes be a group created from your in-house email list, or it may be a group curated especially for the survey as they comply with specific criteria. A research panel is usually more focused than a random sample group which is useful for certain types of data collection.

Respondent fraud

Respondent fraud is a complication that can affect survey data and lead to incorrect results. The main reason for this kind of fraud is in order to cash in on multiple incentives offered by most data collection agencies. One type of respondent fraud involves submitting multiple responses from one computer (and therefore representing the same person over and over). This is often easy to detect by cross-referencing responses with IP addresses and invalidating any duplicates, or even cancelling all the instances, since it is not clear that the responses are part of the required demographic. Another type of fraud involves the use of “bots” or automatic form-fillers. These can usually be weeded out by using red-herring questions and open-ended questions.

Response Rate

The response rate is usually expressed as a percentage, and refers to the ratio of people who answered the questionnaire relative to those who received it. For example, if 342 people respond to a survey where the sample size was 1000, the response rate is 34.2%. Knowing your response rate is very important in assessing the relevance of your  target sample and clarity of your questionnaire.

Completion Rate (Respondents vs Completes)

Once you know your response rate, you can delve even deeper into the stats and calculate the completion rate. This refers to questionnaires which are answered in their entirety, and is usually expressed as a percentage of the completion rate. For example, if 300 people from a sample size of 1000 return the questionnaire, but only 150 answered it fully, the response rate would be 30% and the completion rate is 50%. The completion rate is important because it will affect the overall reliability of your data, for example, if many respondents are leaving the same questions blank, then your sample size for those questions is much lower than your overall sample size. The completion rate will also give you valuable information about the ease with which respondents answered the questions, and whether it was fun or frustrating for them, so that you can create a more user-friendly survey.

Incidence rate

Another important rate to be aware of is the incidence rate. This is the percentage of respondents who pass the screening process of the survey. For example,if  the survey includes screening questions to discover which IT managers use a specific type of software, the incidence rate will be the percentage of the total who respond in the positive. The incidence rate will give you important information as to the feasibility of your survey. 

Soft Launch vs Full Launch

A “soft launch” of a survey involves sending it out to a small sample size in order to get some preliminary information, and to refine any issues that weren’t visible during the build phase. The drawback is timing: a soft launch requires enough time to send out the first draft of the survey, wait for responses and then analyse the results before creating a second draft. When you are on a tight deadline, it may be necessary to go directly to a full launch. 

Analyze and Report

Outliers

You may be familiar with the term “outliers” from your statistics class. It is a way of pinpointing data which falls outside the normal trend. It is important for a survey provider to be able to assess the nature of any outliers in the data collected, because they can indicate various possible meanings. An outlier may raise the suspicion of respondent fraud, or it may be something different altogether. Sometimes it points to human error, perhaps a respondent pressed the wrong button to an answer in the questionnaire, or possibly the answer was transcribed incorrectly. There is always the possibility that it is an accurate piece of data, and that the respondent is remarkable in this aspect compared to the general population!

Crosstabs 

Cross-tabulation analysis, known as crosstabs, is a way of analysing data by comparing multiple tables. A normal table compares 2 categorical variables, but a crosstab can show the relationship between those variables and other data, thus providing a contingency table. Crosstabs form a whole category of data management involving statistical analysis, and can be essential when turning the information gathered by the survey into data which your business can use to determine future trends and projections.

ANOVA

ANOVA, analysis of variance, is a statistical tool used to measure the difference in the mean value of different groups. For example, you may want to know the likelihood that any particular group would use your website, and ANOVA would examine the data according to particular parameters, for example gender, age or location, and compare the different results.

Regression Analysis

There are many different types of regression analysis, but most people are only familiar with two: linear and logistic. These are used to compare different factors from survey data in order to understand the relationship between variables. For example, you may want to understand the relationship between sales and seasonal trends, and linear analysis would map this from the survey data.

Conclusion Validity

The conclusion validity of your survey is the extent to which the findings seem legitimate. If there is a discrepancy between the data and the expectations, this could point to a problem within the survey itself, or with the administration of the survey. Another possibility is that there is some kind of bias at play. And there is always the possibility that one of the factors is wrong! So you may have an error in the statistical analysis, or your hypothesis may have been off the mark. Checking for conclusion validity will enable you to decide which of these factors need to be  double-checked.

At Global Surveyz, we speak the language of surveys fluently, and we are here to walk you through the process, explaining everything you need to know, and working collaboratively as a team to ensure that your survey delivers the best data for your business. Contact us today and find out how a personalised survey could benefit your business.