Can you figure out which of two rank lists of results is from Google? That’s the simple objective of a new game that we have designed. The data collected from this game allows us to understand people’s preference for relevance and diversity. So far, hundreds of people have played this and tested their knowledge of Google. Perhaps you want to give it a go to? It takes only 2 minutes!

How much can we sacrifice relevance for diversity/fairness in search results before it starts affecting people’s satisfaction? To answer this question, we designed experiments that we have been running through Amazon’s Mechanical Turk platform. So far, we have learned that people are quite receptive of diversity in their search results, but overdoing so can result in lost satisfaction.