Had a nice chat earlier in the week with Yahoo’s Tim Cadogan, who in the face of the paid inclusion dust up has been given, or perhaps even has willingly taken, the role of Explain It, Then Explain It Again Yahoo. Tim knows from controversy and market overreaction, he was at Overture before paid search was cool, and there are some similarities to how the market is reacting to Yahoo’s CAP program. Namely, he believes, the whole content acquisition program, misunderstood now, will eventually be seen as a rational approach to the market, and, ultimately will lead to a much more robust search experience.
Now, I stand by what I wrote earlier, that CAP forces the issue of trust and transparency to the fore, and most small business owners (arguably those most affected by this program) are not yet ready to trust a big company like Yahoo. They feel coerced to pay up, mainly by fear of missing something if they don’t. Tim said he understood these sentiments, and said that only time will prove that CAP has no influence over ranking. He also points out that CAP is entirely optional, and that Yahoo’s goal is to crawl the entire web as throughly as possible, regardless of paid inclusion.
I pointed out to him that if Yahoo could simply state that its organic crawl and resultant index was as comprehensive and fresh as Google’s, then perhaps folks would see that CAP is simply an added value service for URL submission, reports, and the like. Tim responded that indeed, Yahoo stands by its index and results as the equal of Google, and again time will tell.
In any case, whatever your view on the paid inclusion issue, it’s clear Google will make hay on this for sometime to come. Over time, what matters is results, and this is where Tim thinks CAP will bear fruit. He makes the point, echoed by Gary Price and others, that the press pretty much overlooked what he thinks is a huge differentiator for Yahoo: CAP’s ongoing program to index the deeper, structured web, in particular CAP’s work to surface databases at the University of Michigan and Library of Congress. I agree, this is a big deal, and in the long run will matter a lot as the web gets more and more structured (to good end, in my mind).
I noted that perhaps the LoC and other deep web efforts were discounted by much of the press as so much PR to cover the paid inclusion portion of the CAP program, and Tim said he hoped the press wasn’t *that* cynical. The press, cynical?!
PS – if you want to hear Tim talk about Yahoo in the pre CAP days, a webcast of a talk he gave a year ago (thanks for the correction, Brad) is available here.
PPS – Look who has the #1 result on Yahoo search for “Yahoo CAP“. Given the minor fracas that post started about bias in algorithms…that’s pretty funny.
PPS – I forgot to ask Tim, and wish I had, about the fact that when you sign up for SiteMatch (that’s the name of the paid inclusion portion of CAP), you agree to pay .15 to .30 cents a click for anyone who clicks on the URLs you submit. That’s basically embedding Overture into the main listings, and it could get quite expensive (or lucrative, depending on your business), and it strikes me as…unusual. In other words, it adds the very distinct ka-ching of commerce into organic results (yes, I know, paid inclusion already has that ka-ching, but it’s not so transactional – it’s not *on the click*…). A post over at this new search marketing site, The Pre-Commerce Blog, got me thinking about that…
Great post. The proponents of paid inclusion have always claimed that it improves search results, if handled properly. Too often, I suspect, that has seemed like hollow corporate spin. But as Tim says, the proof is in the pudding.
It will be interesting to see if Google is able to stick to its philosophical guns when it has to worry about public shareholders. Assuming that Yahoo can pull off the PR on CAP, which I
As always, a great post.
Cleared the confusions up which were placed in my head about the Yahoo! PFI services.
Sid
John,
Thanks for the mention. Yes, on the surface, it COULD be a good thing indexing “deeper web” content.
However, just crawling and indexing it (structured or unstructured) to already large databases means little. The tools MUST be made available for the searcher to find the material.
Let’s not forget that most people only look at the first few results, use an average of about three terms, and don’t take advantage of “advanced search” functionality.
Adding structured content to an unstructured database can eliminate the ability to search on specific fields that the structured database in its native form might offer.
I think that one (of many) technologies that could help is dynamic clustering. Vivisimo and others are in this space. Vivisimo says that it’s technology helps the searcher utilize “selective ignorance.” Vivisimo has another interesting demo at Clustermed.org.
Finally, keeping content in smaller (foucused, targeted, niche databases, fee-based databases) and then federating them at the time of the search (depending on the needs of the searcher)could be another potential solution.
David Seuss of Nothern Light discussed this in the speech you linked to the other day.