The First Monday peer-reviewed journal recently published “Do Web Search Engines Suppress Controversy?” by Susan Gerhart, a software engineering professor at Embry-Riddle Aeronautical University. Driving the paper is this sentiment:
“The dilemma of controversies is that the searcher beginning to explore a topic doesn’t know the search terms to investigate a controversy unless it is revealed with reasonable visibility, e.g. not item number 879 in search results, nor buried three links away from result number 30.”
In other words, if you are just starting to research a topic, and have no idea if there are any controversies surrounding said topic, how will you ever know if the search engine has a bias toward not revealing those controversies?
This paper explores the hypothesis that, as Gerhart puts it: “A given, well–known specific controversy will not be revealed in the top search results.” She then creates an experiment to test this hypothesis, by outlining both a broad topic, and a related controversial subtopic. An example is “Albert Einstein” as the broad topic, and “Did Einstein’s first wife, Mileva Maric, receive appropriate credit for scientific contributions to Einstein’s early work” as the subtopic. The question is, do search engines leave out the more controversial bits, the stuff that, taken as a whole, provide texture and context to any searcher’s understanding of a topic?
For the many examples she tested, Gerhart found proof on both sides of the ledger, and the paper left me disappointed that she could not come to a more decisive conclusion. She did note that in fact most search engines were roughly equal in their performance in the experiments. And she has some interesting thoughts on how controversies are integrated (or not) into the web at large, and some suggestions as to how various actors on the web – site authors, researchers, search engines – might better organize themselves to portray a more relevant set of SERPs to any particular query.
All in all, I liked this paper, as it forced me to think about the politics and architecture of search engine results. She introduces the idea of “sunny” vs. “dark” search results, and concludes that “sunny” results – those that do not include controversies, tend to float toward the top. Her final conclusion:
“Web search engines do not conspire to suppress controversy, but their strategies do lead to organizationally dominated search results depriving searchers of a richer experience and, sometimes, of essential decision–making information. These experiments suggest that bias exists, in one form or another, on the Web and should, in turn, force thinking about content on the Web in a more controversial light.”
The one thing Dr. Gerhart left out entirely is the effect of blogs. As most of us certainly know, when the blogosphere latches onto a controversy (or just a politically-driven meme), that aspect of a topic usually shoots to the top of the SERPs. As with most good papers, this one left me feeling like there is much work yet to be done.