About the Episode:
In this episode of Statistically Speaking, Kerry Cunningham and Sara Boostani introduce two fundamental concepts in statistics: descriptive and inferential statistics. While most B2B benchmark reports stop at descriptive statistics, Kerry and Sara explain why that’s often not enough — and what it actually takes to make reliable claims about the group being studied. The episode also introduces hypothesis testing — including why statisticians start by assuming there is no effect until the data proves otherwise.
Topics Covered
The difference between descriptive and inferential statistics
- Descriptive statistics summarizes what’s in your data — things like averages, medians, and how respondents answered a given question
- Inferential statistics uses probability and statistical testing to draw conclusions about the broader group you’re studying — not just the people who responded.
Why we sample populations instead of measuring everyone
- It is rarely possible or practical to measure an entire population, so researchers collect a sample and report on its characteristics
- Most benchmark reports are based on a sample — the key question is whether the researchers have used inferential statistics to ensure their findings can be generalized beyond the people who responded
Hypothesis testing
- The foundation of inferential statistics: assume there is no effect until there is enough evidence to prove otherwise
- Similar to the principle of “innocent until proven guilty” — the burden of proof is on the researcher to demonstrate that a real difference or effect exists
A real research example from 6sense
- In their marketer compensation study, 515 B2B marketers in North America participated, with an average salary of $172,935 — but that figure alone is purely descriptive
- To make claims about what influences all B2B marketers’ pay, inferential statistics are required
The limitations of most B2B benchmark reports
- Many widely-read reports only report descriptive statistics, meaning their findings may not reliably reflect the broader population
- A difference of one point between two scores on a sample of 150 people is likely not statistically meaningful — but without inferential stats, you can’t know for sure
Key Takeaways
- Descriptive statistics tells you what was found in a sample — nothing more
- Inferential statistics is what allows researchers to make confident claims about a broader population
- If a benchmark report isn’t showing any statistical tests, it’s likely only reporting descriptive statistics
- Always ask: did they do the math to confirm that what they found in their sample reflects the real world?
- Researchers have a responsibility to clearly communicate the limitations of their findings and the population to which results can be generalized
