7 Common Pitfalls of Data Analytics

by Mark Laughlin – Director of Enterprise Technology, Key Information Systems, Inc.

When considering which data to consider in order to make informed and accurate business decisions, there are 7 biases that can cause inaccuracies and bad decisions that must be considered:

1) Confirmation Bias
Confirmation bias occurs when there’s a desire to prove a hypothesis, assumption, or opinion and there is eagerness to find support for the data though it is not accurate upon further investigation.

2) Selection Bias
Selection bias occurs when data is selected subjectively rather than objectively or when non-random data has been selected. With a population selection that is not representative of the actual population, the results are skewed and therefore not freat quality.

3) Outliers
Outliers are extreme data values that measure significantly above or below the range of normal values or the pattern of normal distribution. They are common and particularly difficult to catch for people whose main task isn’t statistics because they don’t see abnormalities.

4) Simpson’s Paradox
When groups of data are combined, sometimes data can reverse. This is called Simpson’s Paradox and is often the reason for research and medical findings to flip flop from one major finding to another because the data can be see in different ways.

5) Overfitting And Underfitting
The data overfitting model includes noise which complicates data, while underfitting, an opposite and overly simplistic model, equally skews results.

6) Confounding Variables
Omitting a crucial variable can have profound effects because a perceived relationship between two variables may be proven false because it has been overlooked.

7) Non-Normality: The Bell Does Not Toll
Some statistical tests assume that a normal distribution exists, when in fact there may not be a “normal” so the results may be inaccurate due to lack of having an identified base.

For more information about how to avoid these biases, contact KeyInfo for assistance with your data analytics and for more information on this topic please read Lisa Morgan’s article 7 Common Biases That Skew Big Data Results.