Why Best-Worst Method Works Well
Technical Report: UTEP-CS-22-96a
To appear in Proceedings of the 11th IEEE International Conference on Intelligent Systems IS'22, Warsaw, Poland, October 12-14, 2022.
Abstract
In many cases, experts are much more accurate when they estimate the ratio of two quantities than when they estimate the actual values. For example, if it difficult to accurately estimate the height of a person on a photo, but if we have two people standing side by side, we can easily estimate to what extent one of them is taller than the other one. To get accurate estimates, it is therefore desirable to use such ratio estimates. Empirical analysis shows that to obtain the most accurate results, we need to compare all the objects with either the "best" object -- i.e., the object with the largest value of the corresponding quantity -- or the "worst" object -- i.e., the object with the smallest value of this quantity. In this paper, we provide a theoretical explanation for this empirical observation.