Last week I wrote a post in which I opined a bit about Facebook search. In it I wrote:
Facebook is way more than its newsfeed, and its search play is key to proving that value, and extending it….No doubt building Facebook search today is akin to building Google ten years ago – bigger, most likely, in terms of data, algorithmic, and platform challenges.
If only I had waited a few days, I could have pointed to Fred’s piece in Wired, out this week. He profiles the ongoing feud between the King of Search, Google, and the upstart, Facebook. In his piece, he writes:
For the last decade or so, the Web has been defined by Google’s algorithms—rigorous and efficient equations that parse practically every byte of online activity to build a dispassionate atlas of the online world. Facebook CEO Mark Zuckerberg envisions a more personalized, humanized Web, where our network of friends, colleagues, peers, and family is our primary source of information, just as it is offline. In Zuckerberg’s vision, users will query this “social graph” to find a doctor, the best camera, or someone to hire—rather than tapping the cold mathematics of a Google search. It is a complete rethinking of how we navigate the online world, one that places Facebook right at the center. In other words, right where Google is now.
I agree that of all the contenders out there right now (including Twitter), Facebook has the most data, position, and potential to upset Google’s dominance of the web. But I disagree with one premise of the piece, which is that Facebook’s proprietary approach to the data it stores presents a blind spot to Google that gives Facebook a competitive edge. Fred writes:
Together, this data comprises a mammoth amount of activity, almost a second Internet. By Facebook’s estimates, every month users share 4 billion pieces of information—news stories, status updates, birthday wishes, and so on. They also upload 850 million photos and 8 million videos. But anyone wanting to access that stuff must go through Facebook; the social network treats it all as proprietary data, largely shielding it from Google’s crawlers. Except for the mostly cursory information that users choose to make public, what happens on Facebook’s servers stays on Facebook’s servers. That represents a massive and fast-growing blind spot for Google, whose long-stated goal is to “organize the world’s information.”
I think it’s a major strategic mistake to not offer this information to Google (and anyone else that wants to crawl it.) In fact, I’d argue that the right thing to do is to make just about everything possible available to Google to crawl, then sit back and watch while Google struggles with whether or not to “organize it and make it universally available.” A regular damned if you do, damned if you don’t scenario, that….
For an example of what I mean, look no further than Twitter. That service makes every single tweet available as a crawlable resource. And Google certainly is crawling Twitter pages, but the key thing to watch is whether the service is surfacing “superfresh” results when the query merits it. So far, the answer is a definitive NO.
Why?
Well, perhaps I’m being cynical, but I think it’s because Google doesn’t want to push massive value and traffic to Twitter without a business deal in place where it gets to monetize those real time results.
Is that “organizing the world’s information and making it universally available?” Well, no. At least, not yet.
By making all its information available to Google’s crawlers (and fixing its terrible URL structure in the process), Facebook could shine an awfully bright light on this interesting conflict in interest.
I really appreciate this take on the existing feud between Google and Facebook along with the potential “blindspot” that is Twitter. I personally can’t see how FB could ever be positioned to “replace” Google. The whole “social graph” premise from Zuckerberg is quite interesting…if these massive egos can somehow find a way to coexist:) – Cheers!
Great article. Something I’ve been pondering as well …
http://frostyland.blogspot.com/2009/06/who-killed-social-media.html
While it’s nice to advocate “open access” from an arm chair John, that is not how the Internet has been, is played currently, or will be played for quite some time.
The under dog always goes for “open” to gain market share.
Google used to open it’s search APIs to developers and for commercial use – no longer. In fact, it has shut down most services that dare to even scrape it under the auspices of “traffic abuse control.” Ironic that Google itself is the mother of all scrapers.
Don’t for a second think that Twitter gives Google it’s firehose feed. No, they make Google crawl it like Google makes folks crawl YouTube, Blogspot, and every other Google property.
Check Google’s robots.txt file and you’ll get a great story.
There is no way Facebook or Twitter should give access to others, just as Google does not give others access it’s index…
Let alone transparency to it’s AdSense pricing and payout systems…
great post. Fb won’t share all because there are privacy issues if people want things private. maybe that will change for people who share all. an interesting thing in all this is how crucial developers are to the success of this strategy. Fb could have thousands of developers working on using fb data for search, in addition to using the graph to make websites and apps more social. in Fred’s great article where he cites a Google employee who says fb doesn’t have the brain power to take on those problems, what he has to keep in mind is that it’s really the brain power of as many people fb can attract by offering a big business opportunity, distribution and a commitment to openness and transparency. Google doesn’t open search and that’s another point to add to your great one about conflict of interest: goog doesn’t want to commoditize search by opening it, even if it would mean there could be significant advances. That’s where goog is stifling innovation by not enabling it and we can all only hope fb succeeds in disrupting goog by being open and a great platform. It’s exciting.
I think it is much more than the privacy issue. Google would need the data in Facebook much more, than Facebook would need some traffic from Google.
This is just another example, where Google approach is reaching its limits. Certainly they have the resources to broaden their approach and reinventing themselves.
Don’t for a second think that Twitter gives Google it’s firehose feed. No, they make Google crawl it like Google makes folks crawl YouTube, Blogspot, and every other Google property.
I agree to This!
First, bored. Busy all replica handbags day doing nothing of, Chanel handbags put some words toget http://www.brandbagonline.com I her to form a so-calle replica d articles, want to know Replica bag t these things is no replica , bags handbag t a thing, might as well name it t Replica bags the pretext of students to see the attitudes of the people of the world were plpecu Replica chanel bags laying, chat to meet their own spiritual emptiness. Louis Vuitton , This is the "zero essay" The main re, chanel bag , ason out.
The biggest challenge for Facebook would be to get user approval to share what essentially is private information shared amongst network of friends. Social networking sites are often used as alter-egos by users who would not acknowledge comments they make on these sites in real life and here is where the challenge for facebook lies. google’s ability to ferret out factual information is a key differentiator and magnet for users who expect a factual result for queries. Facebook unless it becomes more than a alter-ego medium for old buddies to connect will be left with a whole server farm of information that maybe of interest to data marketers or spammers … but of no relevance to the serious researcher
part of the problem with Google and Twitter is that because Twitter is real time and every user has equal weight, it’s very easy to game Twitter and be the top result (ie most recent) for any topic. That’s dangerous territory for Google to play with.
Tweets are not “information”, and neither is a Facebook wall. While the social graph is valuable, the content is a modern version of GeoCities.
There you go again, John. Drinking your own Kool-aid. This is precisely why people either like you or don’t. Your prognostications are sometimes off base but at least you have the guts to make them in public.
Facebook is a collection of mostly useless information. No matter how much the spinsters try they will never convince us old timers that Facebook is a “second Internet”. It’s just pure nonsense, John.
As for massive data sets — if you really want to see something scary check out: http://www.rakedin.com. They are very quietly building a truly impressive site.
FB can’t make their users stuff open to Google – 99% of Facebook users wouldn’t want that stuff open to the public.
I just want to be able to search for stuff that I or others have posted – links, videos, notes, etc. – while I’m INSIDE facebook. Right now the facebook search is useless for anything but finding people or groups.
The other day I wanted to find a link that I remembered posting a couple of months ago, but I wasn’t about to go fishing through my old posts page by page. As far as I know, there is no way to search for that sort of information on facebook.
The biggest hurdle Facebook will have to overcome is the extra layer of interface. Google exists right in your browser and is seemless with overall web activity.
To use Facebook you have to log in, etc. etc.
What matters most to me (most of the time) is what my friends think (or know) about a given topic… not what Google’s bots and algorithms think (or know) about that topic.
So it stands to reason that Facebook could fairly easily create a MORE RELEVANT search experience by simultaneously searching what my friends know AND what Google knows about any topic, and displaying those results on the same page, either side by side or one above the other.
In many cases (especially early on) my friends won’t know anything about most of my searches… (I mean, Facebook won’t have anything in my “socialgraph index”) but that’s okay! In those cases, Google will be there as “backfill” and my experience won’t be any less relevant than it would be if I had just gone to Google first.
Over time my friends will be adding more and more information to Facebook, and “Friend Matches” will be more and more commonplace.
Once this happens, it’s game over Google.
tx
m
#jerky — while still interesting, your comment lost some charm when I realised that you didn’t mean spinsters as in maiden aunts
Google doesn’t open search and that’s another point to add to your great one about conflict of interest: goog doesn’t want to commoditize search by opening it, even if it would mean there could be significant advances. That’s where goog is stifling innovation by not enabling it and we can all only hope fb succeeds in disrupting goog by being open and a great platform. It’s exciting.
Well said. Facebook should open its index if-and-only-if Google opens its index. Would be only fair.
From what JG said, Is Facebook going to really open its index, or are we just contemplating something that is not possible because of privacy, infrastructure and other issues?
I dont believe is game over for google, since it has the structure and broad approach that at this moment is absent from FB.
Eduardo: I imagine that there could be some sort of possibility around giving Facebook users the option of opening themselves up to Google. For any single or multi-person communications on Facebook, no information would be released to Google until all participating parties explicitly gave the “opt-in” signal.
But even if it’s the users who ultimately have responsibility and control over their own “robots.txt” Facebook information, Facebook still has to make it technically possible for users to even do that. And that technical possibility won’t occur until the political/competitive ramifications get worked out.
“For an example of what I mean, look no further than Twitter. That service makes every single tweet available as a crawlable resource.”
Are you sure of that? The Twitter.com robots.txt file seems to indicate otherwise.
#Google Search Engine Robot
User-agent: Googlebot
# Crawl-delay: 10 — Googlebot ignores crawl-delay ftl
Disallow: /*?
Disallow: /*/with_friends
#Yahoo! Search Engine Robot
User-Agent: Slurp
Crawl-delay: 1
Disallow: /*?
Disallow: /*/with_friends
#Microsoft Search Engine Robot
User-Agent: msnbot
Crawl-delay: 10
Disallow: /*?
Disallow: /*/with_friends
# Every bot that might possibly read and respect this file.
User-agent: *
Disallow: /*?
Disallow: /*/with_friends
Here is another point that may run contrary
there is no comparison in google and facebook … twitter might have some effect though…http://www.seo-strategies.org/2009/03/twitter-help-engine/
I can’t see the connection between Facebook and Google. For me FB has been a fun way to communicate with family and friends and Google for serious searching, business content and my job search. Maybe I’m not taking full advantage of FB but I suspect that almost everyone on FB would prefer their content remain “closed” vs. Google and or Twitter…
It’s surprising to me that people don’t see the connection (er, competition) brewing between Facebook and Google.
What do people use Google (Search) for? To find stuff (places, products, information, answers, etc)…
Before Google came along (and used its computers to “organized the world’s information”), where’s the first place you looked when you needed to find stuff? Usually you asked your friends.
So… now Facebook knows what your friends know (not totally, completely, yet, but they are getting there) it stands to reason that instead of querying Google for “family friendly things to do in Vail, CO” people will increasingly begin querying Facebook for “family friendly things to do in Vail, CO” with the expectation that if any of their friends know anything about “family friendly things to do in Vail, CO” they will find MORE RELEVANT answers on Facebook than they will on Google.
It is important to note that for the first few years it will be necessary for Facebook to use some sort of general purpose web index (Google, Yahoo, Bing) as “back fill” to compensate for the fact that right now (and presumably for a couple years) Facebook’s index of what my friends know will be thin… so in a worst case scenario, if I search Facebook for “family friendly things to do in Vail, CO” even if my friends don’t know anything about that query, at least I get what I would have gotten if I went to Google first.
Again, (old) game over, Google.
tx
m
Dave: I’m rusty on robots.txt syntax, but I think that Twitter file is mostly about blocking robots from searching Twitter’s search results (and it must be an old file, since the “with_friends” pages don’t even exist anymore). Individual tweet pages are definitely visible in Google results.
Now on to my question for everyone who thinks so social search is the future: If your friends are so damn smart, why did you start using Google to begin with?
Seriously. It’s like the social search advocates think Google only used an algorithmic solution because human-mediated data wasn’t available. We had human-mediated data in the 1990s. It was called “Yahoo!” and it didn’t scale. That’s why Google kicked Yahoo’s ass.
I use Google because the world is too big to depend on my friends. They can’t tell me all the stuff I need to know every day. A few days ago, I needed to know the service hours of the Oklahoma City public library. Asking my friends was not a useful option, and I’m pretty sure searching their Facebook feeds would have been even more useless. Who the hell posts status updates about library hours (if they’re not a librarian)?
Frankly, I thing a lot people advocating social search are, to put it bluntly, nerds, who have a lot of nerd friends, therefore read a lot of unusually nerdy status updates. Out in the non-nerd world, most social network status updates are “my daughter’s soccer team won” and “I hate Mondays.” It’s not useful data. Building a cool tool to process useless data might be a great exercise in programming, but it’s unlikely to be the future of search.
Michael,
This isn’t about replacing google’s web index, it’s about augmenting it (or influencing it) with things that your friend’s know.
tx
m
more and more the Google search application that we all know and love today will be phased out, replaced by more advanced tools that have more inputs and more variables, capable of delivering greater relevance. Even Google’s founders and CEO have stated many times in the recent past that they are currently only scratching the surface with page rank.
Relevance is subjective.
Relevance is personal.
At the end of the day, a search engine is a recommendation engine.. you type in a query, and it recommends the pages where you are most likely to find your answer… In most cases the recommendations that you get from your friends are more valuable (more relevant) than the recommendations that you get from a stranger, even if it’s a really smart stranger… In the product that I am proposing, if your friends can’t help you, then you always have the really smart stranger to fall back on… Google is “back fill.”
A very interesting read. I would like to respond with a blog post.
Just relax and wait to see after a few years FB becomes less relevant.
If FB is that useful like Google, then by now they should have answered two main questions.
1 – What is the main purpose of FB?(Instead of every time tweaking it to make it as if it is still relevant whenever new cool thing is on the surface even if unrelated; like what they are doing when the hype is up for twitter. I say because FB lacks main theme and lacks primary vision.)
2 – How to monetize what they have?
Frankliy speaking,twitter has a fast development in all the world,from us, EU to China. Its effect is more than facebook in Asia now. However, I did not find the realy competitive of twitter. What is the real different with facebook. Maybe I did not find the core thing of twitter, but I don’t think it could do better than google.
http://paydotcom.net/?affiliate=599752
look at this it works for real i did make money in 2 days with this
“I use Google because the world is too big to depend on my friends. They can’t tell me all the stuff I need to know every day. A few days ago, I needed to know the service hours of the Oklahoma City public library. Asking my friends was not a useful option, and I’m pretty sure searching their Facebook feeds would have been even more useless. Who the hell posts status updates about library hours (if they’re not a librarian)?”
Michael – well said!
“I use Google because the world is too big to depend on my friends. They can’t tell me all the stuff I need to know every day. A few days ago, I needed to know the service hours of the Oklahoma City public library. Asking my friends was not a useful option, and I’m pretty sure searching their Facebook feeds would have been even more useless. Who the hell posts status updates about library hours (if they’re not a librarian)?”
Okay… at this point I may as well give the whole idea away, even though the product isn’t quite complete…
What if (in addition to simultaneously searching Google and your “friend index”) you had an easy, non-intrusive way to notify your friends that you were looking for information about the Oklahoma City public library?
Think about the world of possibilities that this opens…
Every time I am looking for something on Google (or eBay or Amazon or Craigslist), I can (VOLUNTARILY) notify my friends… In this way I not only search what my friends have already posted to the web on a given topic, I can search what is in their brains, right now!!!!!!!!!!!
http://www.zennybuddy.com
tx
m
@mike
“What if (in addition to simultaneously searching Google and your “friend index”) you had an easy, non-intrusive way to notify your friends that you were looking for information about the Oklahoma City public library?”
Mike, do you know what most of the so called ‘friends’ will end up doing? Guess… They will Google and send you links that they found from googling…hahahaha
So imagine also notifying to your friends and getting those googled responses from them after a week or month….
Michael:
There are two big differences between now and back in the mid-90s:
– the volume of content being created by individual users
– the ease of connecting with friends
Social search allows you to discover knowledge in your connections. Among my friends I probably know the activities of 5-7 friends in a fair amount of depth. But for the other 300 or so people who I’m connected too, I don’t know all of their activities.
If I’m thinking of going to Brazil, I would know whether the 5-7 friends had been there. Social search would help me discover if the other 300 had been.
What makes social search even more powerful is that I have the ability to connect with those people and discover information that they haven’t published. The mere fact that they had a status update that said “going to brazil” allows me to tap into the knowledge in their head.
I built a prototype of this 2 years ago. Even with 3-4 hardcoded friends and the relatively limited amount of content being posted back then, the results were very compelling.
Of course it won’t work for every query — I just did a search for the roll call on the 2006 voting rights act reauthorization.
But for things like finding doctors, mechanics, vacation destinations, restaurants, etc., social search would be my go to place.
@Rick,
FB makes their users stuff open to Google. and we cant do anything.
Interesting post. I have stumbled and twittered this for my friends. Hope others find it as interesting as I did.
Naomi from Free Classifieds posting service
Tweets are not “information”, and neither is a Facebook wall.
@sorunsuzhost Really? It’s how I found out that Michael Jackson was dead. Isn’t that information? It’s also how I found out that a friend of mine is into radio controlled airplanes… something that I was considering as a gift for my son. I ended up exchanging a ton of valuable information with this friend of mine. He even offered to give me one of his older planes, for free! Tell me, how would that have happened via Google search?
ınteresting posts.thanks.
Both have their place. Turning to my “social graph” for information seems like a step backwards in the world of knowledge.
No disrespect to my friends, but unless you are lucky enough to have a friend who is a doctor, a chemist, a vet, an engineer etc etc then they probably aren’t going to be all that knowledgeable on the things you want to learn.
I dispute that this is the way we do things outside of the online world, actually we all walk round going “how did we find that out before Google”. It is crazy to think we are going to go back to ‘Chinese whispers’ again!
Fine dude.bt u left me confused abt the google .i want to know how it is a cold strategy player…i got to know abt the twitter followers when i visited http://thetwittersecret.com/…bt i could’nt understand abt the google.
Thanks for sharing your info. I really appreciate your efforts and I will be waiting for your further write ups thanks once again.
html5 contact form| html5 development