Bias and Fairness in Effectiveness Evaluation by Means of Network Analysis and Mixture Models


Information retrieval effectiveness evaluation is often carried out by means of test collections. Many works investigated possible sources of bias in such an approach. We propose a systematic approach to identify bias and its causes, and to remove it, thus enforcing fairness in effectiveness evaluation by means of test collections.

Proceedings of the 10th Italian Information Retrieval Workshop, Padova, Italy, September 16-18, 2019