Technical Reports

The problem of combining the ranked preferences of many experts is an old and surprisingly deep problem that has gained renewed importance in many machine learning, data mining, and information retrieval applications. Effective rank aggregation becomes difficult in real-world situations in which the rankings are noisy, incomplete, or even disjoint. We address these difficulties by extending several standard methods of rank aggregation to consider similarity between items in the various ranked lists, in addition to their rankings. The intuition is that similar items should receive similar rankings, given an appropriate measure of similarity for the domain of interest. In this paper, we propose several algorithms for merging ranked lists of items with defined similarity. We establish evaluation criteria for these algorithms by extending previous definitions of distance between ranked lists to include the role of similarity between items. Finally, we test these new methods on both synthetic and real-world data, including data from an application in keywords expansion for sponsored search advertisers. Our results show that incorporating similarity knowledge within rank aggregation can significantly improve the performance of several standard rank aggregation methods, especially when used with noisy, incomplete, or disjoint rankings.