One Needs to Be Careful When Dismissing Outliers: A Realistic Example

Authors

Carlos Fajardo, The University of Texas at El PasoFollow
Olga Kosheleva, The University of Texas at El PasoFollow
Vladik Kreinovich, The University of Texas at El PasoFollow

Publication Date

8-2016

Comments

Technical Report: UTEP-CS-16-54

Abstract

Traditional approach to eliminating outliers is that we compute the sample mean μ and the sample standard deviation σ, and then, for an appropriate value k₀ = 2, 3, 6, etc., we eliminate all data points outside the interval [μ − k₀ * σ, μ + k₀ * σ] as outliers. Then, we repeat this procedure with the remaining data, eliminate new outliers, etc., until on some iteration, no new outliers are eliminated. In many applications, this procedure works well. However, in this paper, we provide a realistic example in which this procedure, instead of eliminating all outliers and leaving adequate data points intact, eliminates all the data points. This example shows that one needs to be careful when applying the standard outlier-eliminating procedure.