Methodology Coresight Research regularly surveys consumers, and the following sections provide context for understanding the results of these surveys. What Survey Data Represent In most cases, data will represent the proportion of survey respondents that selected a survey response—for example, 10% of consumers shop on social media. This usually applies also to most breakdowns, such as by demographic groups, which means those kinds of results differ from a demographic profile. For example, 50% of consumers in the 18–24 age group shopping at Walmart does not mean 50% of Walmart’s shoppers are in the 18–24 age group. Using the same example, a demographic profile would break down Walmart shoppers into age groups so that, when totalled up, all groups sum to 100%. However, in some cases, we will analyze data to provide a profile or breakdown. Please refer to the header descriptions and notes surrounding charted survey data, which will specify what the data represent. About Margins of Error Achieving 100% confidence in a survey result is impractical. The margin of error represents the maximum difference we would expect to find between our survey results and actual, “real world” behaviors. For example, +/-5% at a 95% confidence interval means we would expect our data points to be within 5 PPTs of actual behavior. The convention in survey research is to use a 95% confidence level which allows for still useful ranges around results for achievable sample sizes. This range around the sample result is the margin of error. For example, the margin of error for a random sample of 400 at a 95% confidence level is +/-5%. If you need to be 99% confident that the survey result for a sample of 400 reflects the truth, you must increase the margin of error to +/-7%. See the next section for specific margins of error for different survey sample sizes. The margin of error applies to every survey measure, so the margin of error is applied to each result when making comparisons. Statistical significance can be inferred where differences are outside of the margins of error. For example, if 50% of 400 survey respondents report liking ice cream, you can be 95% confident that the truth about ice cream preference lies between 45% and 55%. If 60% of the same survey report liking cake, you can be 95% confident that the truth about cake preference lies between 55% and 65%. Since these ranges overlap at 55%, you cannot be 95% confident that more people like cake than ice cream based on this survey. However, you could be 90% confident that more people like cake than ice cream because the margin of error at the 90% confidence level is only ±4% so there is no overlap. Specific Margins of Error The following margins of error apply to surveys that represent consumers in general (i.e., the general population): 400 respondents: Results have a maximum margin of error of +/- 4.9%, at a 95% confidence level. 1,600 respondents: Results have a maximum margin of error of +/- 2.4%, at a 95% confidence level. 2,000 respondents: Results have a maximum margin of error of +/- 2.2%, at a 95% confidence level. 2,500 respondents: Results have a maximum margin of error of +/- 2.0%, at a 95% confidence level. At a lower confidence level, the margin of error reduces. For example, at an 80% confidence level, the margins of error fall to 3.2% for 400 respondents and 1.6% for 1,600 respondents. All margins of error are stated at their highest point, which is at the 50% proportion of respondents (e.g., “50% of respondents chose option A”). As percentages of respondents decrease and increase either side of the 50% mark, the margins of error narrow to fewer percentage points. For example, the margin of error for “10% of respondents” would be lower than one for “50% of respondents” for an identical survey with an identical base.
