In our summary of confidence intervals, we discuss their purpose and utility across statistical analyses. In this post, we discuss the most common type of confidence interval – the 95% Confidence Interval.
The 95% Confidence Interval is the most common because it is based on the most common measure of statistical significance, p < .05. The probability assigned to a confidence interval is based on the p-value for the measure in question. This confidence interval states that we expect new measures to fall inside of this confidence interval 95% of the time. That is another way of saying that we have 95% confidence that our findings will be replicated in the real world. That corresponds to a finding of p = .05.
For example, in a recent survey, we saw that buyers in IT departments found their interactions with sellers a bit more useful than non-IT buyers, giving them an average rating of 4.1 out of 5. The 95% confidence interval is the interval, or range, within which we would expect to find IT buyers ratings of vendor helpfulness 95% of the time. In this case, the 95% confidence interval tells us that if we asked another group like the one we just measured, we would expect to find an average rating for IT buyers between 3.97 and 4.16. For non-IT buyers, the average was 3.87, and their confidence interval is between 3.81 and 3.92.
Finding that confidence intervals for two measures do not overlap, we can generally, but not always, conclude that the difference is statistically significant.
The example given above is also an excellent example of how a statistically significant finding is not a meaningful one.
