Comments on: More On Yahoo, Google, Index, Size

By: lorraine

lorraine — Sat, 10 Jun 2006 13:28:15 +0000

Can’t believe you said it either but I’m still laughing. Iwas referring to Font size and I can bearly see it. Don’t laugh I’m serious. Can you give me an answer. Thanks L

By: Brien

Brien — Fri, 09 Sep 2005 02:34:01 +0000

I know this is an old post, but I've seen all kinds of increases in my search engine saturation for my sites, first from Yahoo right before their big index claim, and now Google looks like its ready to fire back. Number of pages available for my sites has tripled in the Google index in less then a few weeks. I'd bet their going to do something astronomical to their number soon. In some cases it appears that they are crawlign more and in other cases it looks like they might have loosened their de-duplication rules. Either they ordered even more hardware, or maybe every developers workstation is crawling in its idle time now!

Anyone else hearing bits in this space?

By: Jean VÃƒÂ©ronis

Jean VÃƒÂ©ronis — Fri, 19 Aug 2005 10:49:37 +0000

Jack> Thanks for citing my post in your comment above.

I have added a detailed analysis of the NCSA study, which shows that it is totally flawed:

http://aixtal.blogspot.com/2005/08/yahoo-pages-manquantes-2.html

--jv

By: Kendall Willets

Kendall Willets — Sun, 14 Aug 2005 21:59:06 +0000

Thanks for the study, Matt. One problem I noticed is that spam is biased towards random word combinations, so queries like "ensiform teleprompter" return a lot of spam and porn sites. Apparently the web spammers have adopted the same tactic used by email spammers, of loading a page with random terms to dilute the significance of the spam terms.

Have you made any attempt to separate spam from the results?

By: Burton Floyd

Burton Floyd — Sun, 14 Aug 2005 20:37:29 +0000

Size might not matter if you are "Waving the right Wand" We believe we have found that wand making zipcode searches fast obsolete.

Burton
http://roadjunctions.blogspot.com

By: zzz

zzz — Sun, 14 Aug 2005 20:26:22 +0000

"any claims that any one company can accurately estimate another's index are simply not defensible"

Implying they can say whatever they want because nobody can prove them wrong anyway.

Whatever the size of their index may be (i can't believe i just wrote that), this story just reeks of pr (as they admit as well). Ultimately however, people don't care for "index size", they just care for better search results.

By: Matt Cheney

Matt Cheney — Sun, 14 Aug 2005 20:03:16 +0000

Several researchers (myself included) at the National Center for Supercomputing Applications at the University of Illinois at Urbana-Chamapign ran a fairly extensive study about this issue (about 10,000 queries) and found that Google returns more results than Yahoo in almost every single case. We found that Google returned well over 150% more total results and gave more results in about 97% of our queries.

Our full study and test code is available online at: http://vburton.ncsa.uiuc.edu/indexsize.html

By: Jack DeNeut

Jack DeNeut — Sat, 13 Aug 2005 01:16:34 +0000

Jean Véronis has an interesting post about Google and Yahoo index size on his blog at: http://aixtal.blogspot.com/2005/08/yahoo-19-billion-pages.html

Most of what he writes is in French, but he translates some interesting articles (like this one) into English.

By: Kendall Willets

Kendall Willets — Fri, 12 Aug 2005 23:17:11 +0000

I don't know how many have sat through Peter Norvig's "size matters" presentation on machine learning algorithms, but his central thesis is that the worst algorithm outperforms the best ones, given a large enough sample size.

He also said that about 30% of their crawl is duplicates, so I can see where there's room for debate about net index sizes.

By: Aki

Aki — Fri, 12 Aug 2005 22:14:51 +0000

John, I also did a quick study and found some interesting results. Specifically, while Yahoo estimates more results on their first results page, they usually only deliver 30% of that estimator. Also, with respect to unique and even total results, Google seems to give me 50-70% more hits. You can look at my queries and results in my entry at: http://blog.akashjain.org/2005/08/12/is-yahoos-index-really-bigger-methinks-not-really-googles-index-seems-50-larger/
- definitely interested in anyone else's thoughts.