Quite the kerfuffle brewing over this site, built by a Microsoft employee, given folks the chance to blind taste test Bing, Google, and Yahoo. Bing was doing well early, but that might be due to the fact a lot of Microsoft folks took the test first. In any case, SAI has a good write up of the whole affair...
My issue with this is that it’s just about ten blue links. Bing in fact is about an application, including a good UI, on top of the base of ten blue links.
I agree the 10 blue link format is not most accurate measure of a search engine’s worth. That said it is a great measure of the relevance of each engine’s algorithm.
Wonder why they took down the results, though? When the site first launched it kept a running tab of number of votes and % preferred by engine. Last I checked it was something like 40% Google, 33% Bing, and 27% Yahoo with 20k+ votes in. MSFT gag order?!?
People still want the 10 blue links — they just want them to be relevant to what they’re looking for.
You rarely see good relevant content on Google for highly commercialized queries any more. A blind search test might open some eyes. You should be encouraging this kind of research.
Eric Schmidt’s idea of “brand value” (which he feels should clean up the cesspool) seems to be large autogenerated/user-content-generated mush sites — many of which just coincidentally pack a lot of Google AdSense onto their pages.
Very amazing web site search engine vote ! i am 20 search recent result 9 Google , 4 Bing, 7 Yahoo…
I feel like we’ve had this discussion a number of times over the past half decade. And there are two issues that really need to be addressed:
(A) A proper “relevance-only” evaluation of just the quality of the results needs to be done. To do this, we need hundreds of independent users, tens of thousands of queries, and the full buy-in of all the major search engines, to consent to the rules of the evaluation. Most search engines, Google included, are unwilling to submit to such external tests. Sure, they do this sort of testing internally all the time. But no one seems to be willing to do it, externally.
(B) No matter what the “average” performance of each search engine is, what I as a user need is some sort of measurement about how well each search engine works for the queries that I do. Think of this as “personalized” search engine performance metrics. What we really need to see is a standardized set of data primitives along the lines of “how many queries did I do, that were successful (I clicked something). What were the ranks of the things I clicked? How many queries did I do that were unsuccessful (nothing clicked). How many times did I have to reformulate my query, in order to turn an unsuccessful attempt into a successful one?
Now, once these metrics are defined, they need to be open, meaning that I can download all my data from a search engine. Not just my query history data. But the full set of interactions, clicks, etc. All the statistics in and around my data.
And then all the search engines need to make this same data available. Then, and only then, will the user really be able to see how well a particular search engine works for them. Not through “gut feelings”, but through real, sustained, direct measurement.
This was a theme at the recent O’Reilly eTech conference: It’s only when the user has the power to measure, and capture those measurements, that real change and consciousness can occur.
The more interesting tests are when you get the 10 blue links and put a random branding on it. Put Google’s results in Bing’s results page and put Bing’s results in a Google results page.
I believe that Google recently stated that they have done this and that they found that objective relevance was about equal. Some demographics tended to self-report that results in Google’s branding were more relevant regardless of which search engine the results came from. (By ‘self-report’ I mean “I feel that this one was best.” versus metrics regarding search success)
Regardless, its great marketing! Taking a page from the old Pepsi challenge. I love it.
i just bought a bunch of microsof stock I hope they rock
Thank you very much.
Bing might steal some market from google at this initial stage. But in the long term it will lose all, and even more if MS does not add something really innovative.
The success so far of Microsoft’s new search engine Bing depends on which numbers you look at. Different sources claim that Bing is still growing, has lost traffic, or has leveled off. Some say Bing is already ahead of Yahoo, while others say that was temporary. I don’t know which set of figures is true
Bing certainly has some ground to cover! I found this review too, gives some features that are exclusive to bing
http://www.expertsem.com/2009/06/04/bingcom-a-local-search-review/
You guys may have heard in the media today that I said the Microsoft will commit up to 10% of it’s profits to developing our search and web resources, the key component being Bing.com. Why they think this is such a big deal is a mystery to me; I’m only talking about 2-3 billion dollars a year. What they should be asking is, “What does Steve plan to do with the other 90%?”
Well, just let me say it this way, “There’s more stuff out there than the internet!”
I’m talking: Theme Parks (VistaWorld anybody?), Circuses, computerized furniture stores, Casinos, Malls, Toys, Food, Health spas like my personal one back at the office where I am WELL attended to!
Think BIGGER people! The Internet is not enough!