free html hit counter Of Note in Search Biz Archives - Page 4 of 181 - John Battelle's Search Blog

Signal, Curation, Discovery

By - December 11, 2010

discovery co. logos.png

This past week I spent a fair amount of time in New York, meeting with smart folks who collectively have been responsible for funding and/or starting companies as varied as DoubleClick, Twitter, Foursquare, Tumblr, Federated Media (my team), and scores of others. I also met with some very smart execs at American Express, a company that has a history of innovation, in particular as it relates to working with startups in the Internet space.

I love talking with these folks, because while we might have business to discuss, we usually spend most of our time riffing about themes and ideas in our shared industry. By the time I reached Tumblr, a notion around “discovery” was crystallizing. It’s been rattling around my head for some time, so indulge me an effort to Think It Out Loud, if you would.

Since its inception, the web has presented us with a discovery problem. How do we find something we wish to pay attention to (or connect with)? In the beginning this problem applied to just web sites – “How do I find a site worth my time?” But as the web has evolved, the problem keeps emerging again – first with discrete pieces of content – “How do I find the answer to a question about….” – and then with people: “How do I find a particular person on the web?” And now we’ve started to combine all of these categories of discovery: “How do I find someone to follow who has smart things to say about my industry?” In short, over time, the problem has not gotten better, it’s gotten far more complicated. If all search had to do was categorize web content, I’d wager it’d be close to solved by now.

But I’m getting ahead of myself.

Our first solution to the web’s initial discovery problem was to curate websites into directories, with Yahoo being the most successful of the bunch. Yahoo became a crucial driver of the web’s first economic model: banner ads. It not only owned the largest share of banner sales, but it drove traffic to the lion’s share of second-party sites who also sold banner ads.

But directories have clumsy interfaces, and they didn’t scale to the overwhelming growth in the number of websites. There were too many sites to catalog, and it was hard to determine relative rank of one site to another, in particular in context of what any one individual might find relevant (this is notable – because where directories broke down was essentially around their inflexibility to deal with individual’s specific discovery needs. Directories failed at personalization, and because they were human-created, they failed to scale. Ironically, the first human-created discovery product failed to feel…human).

Thus, while Yahoo remains to this day a major Internet company, its failure to keep up with the Internet’s discovery problem left an opening for a new startup, one that solved discovery for the web in a new way. That company, of course, was Google. By the end of the 1990s, five years into the commercial web, discovery was a mess. One major reason was that what we wanted to discover was shifting – from sites we might check out to content that addressed our specific needs.

Google exploited the human-created link as its cat-herding signal. While one might argue around the edges, what Google did was bring the web’s content to heel. Instead of using the site as the discrete unit of discovery, it used the page – a specific unit of content. (Its core algorithm, after all, was called PageRank – yes, named after co-founder Larry Page, but the entendre stuck because it was apt).

Google search not only revolutionized discovery, it created an entire ecosystem of economic value, one that continues to be the web’s most powerful (at least for now). As with the Yahoo era, Google became not only the web’s largest seller of advertising, it also became the largest referrer of traffic to other sites that sold advertising. Google proved the thesis that if you find a strong signal (the link), and curate it at scale (the search engine), you can become the most important company in the Internet economy. With both, of course, the true currency was human attention.

But once again, what we want to pay attention to is changing. Sure, we still want to find good sites (Yahoo’s original differentiation), and we want to find just the right content (Google’s original differentiation). But now we also want to find out “What’s Happening” and “Who’s Doing What”, as well as “Who Might I Connect With” in any number of ways.*

All of these questions are essentially human in nature, and that means the web has pivoted, as many have pointed out, from a site- and content-specific axis to a people-specific axis. Google’s great question is whether it can pivot with the web – hence all the industry speculation about Google’s social strategy, its sharing of data with Facebook (or not), and its ability to integrate social signal into its essentially HTML-driven search engine.

While this drama plays out, the web once again is becoming a mess when it comes to discovery, and once again new startups have sprung up, each providing new approaches to curate signal from the ever-increasing noise. They are, in order of founding, Facebook, Twitter, and Tumblr, and oddly enough, while each initially addressed an important discovery problem, they also in turn created a new one, in the process opening up yet another opportunity – one that subsequent (or previous) companies may well take advantage of.

Let me try to explain, starting with Facebook. When Facebook started, it was a revelation for most – a new way to discover not only what mattered on the web, but a way to connect with your friends and family, as well as discover new people you might find interesting or worthy of “friending.” Much as Google helped the web pivot from sites to content, Facebook became the axis for the web’s pivot to people. The “social graph” became an important curator of our overall web experience, and once again, a company embarked on the process of dominating the web: find a strong signal (the social graph), curate it at scale (the Facebook platform), and you may become the most important company in the Internet economy (the jury is out on Facebook overtaking Google for the crown, but I’d say deliberations are certainly keeping big G up at night).

But a funny thing has started to happen to Facebook – at least for me, and a lot of other folks as well. It’s getting to be a pretty noisy place. The problem is one, again, of scale: the more friends I have, the more noise there is, and the less valuable the service becomes. Not to mention the issue of instrumentation: Facebook is a great place for me to instrument my friend graph, but what about my interests, my professional life, and my various other contextual identities? Not to mention, Facebook wasn’t a very lively place to discover what’s up, at least not until the newsfeed was forced onto the home page.

Credit Twitter for that move. Twitter’s original differentiation was its ability to deliver a signal of “what’s happening”. Facebook quickly followed suit, but Twitter remains the strongest signal, in the main because of its asymmetrical approach to following, as opposed to symmetric friending. Twitter is yet another company that has the potential to be “the next Yahoo or Google” when it comes to signal, discovery, and curation, but it’s not there yet. Far too many folks find Twitter to be mostly noise and very little signal.

In its early years, things were even worse. When I first started using Twitter, I wrote quite a bit about Twitter’s discovery problem – it was near impossible to find the right folks to follow, and once you did, it was almost as difficult to curate value from the stream of tweets those people created.

Twitter’s first answer to its discovery problem – the Suggested User List – was pretty much Yahoo 1994: A subjective, curated list of interesting tweeters. The company’s second attempt, “Who To Follow,” is a mashup of Google 2001 and Facebook 2007: an algorithm that looks at what content is consumed and who your follow, then suggests folks to follow. I find this new iteration very useful, and have begun to follow a lot more folks because of it.

But now I have a new discovery problem: There’s simply too much content for me to grok. (For more on this, see Twitter’s Great Big Problem Is Its Massive Opportunity). Add in Facebook (people) and Google search (a proxy for everything on the web), and I’m overwhelmed by choices, all of them possibly good, but none of them ranked in a way that helps me determine which I should pay attention to, when, or why.

It’s 1999 all over again, and I’m not talking about a financing bubble. The ecosystem is ripe for another new player to emerge, and that’s one of the reasons I went to see the folks at Tumblr yesterday.

As I pointed out in Social Editors and Super Nodes – An Appreciation of RSS, Tumblr is growing like, well, Google in 2002, Facebook in 2006, or Twitter in 2008. The question I’d like to know is….why?

I’m just starting to play with the service, but I’ve got a thesis: Tumblr combines the best of self expression (Facebook and blogging platforms) with the best of curation (Twitter and RSS), and seems to have stumbled into a second-order social interest graph to boot (I’m still figuring out the social mores of Tumblr, but I am quite certain they exist). People who use Tumblr a lot tell me it “makes them feel smarter” about what matters in the web, because it unpacks all sorts of valuable pieces of content into one curated stream – a stream curated by people who you find interesting. It’s sort of a rich media Twitter, but the stuff folks are curating seems far more considered, because they are in a more advanced social relationship with their audience than with folks on Twitter. In a way, it feels like the early days of blogging, crossed with the early days of Twitter. With a better CMS and a dash of social networking, and a twist. If that makes any sense at all.

Tumblr, in any case, has its drawbacks: It feels a bit like a walled garden, it doesn’t seem to play nice with the “rest of the web” yet, and – here’s the kicker – finding people to follow is utterly useless, at least in the beginning.

Just as with Twitter in the early days, it’s nearly impossible to find interesting people to follow on Tumblr, even if you know they’re there. For example, I knew that Fred Wilson, who I respect greatly, is a Tumblr user (and investor), so as soon as I joined the service, I typed his name into the search bar at the top of Tumblr’s “dashboard” home page. No results. That’s because that search bar only searches what’s on your page, not all of Tumblr itself. In short, Tumblr’s search is deeply broken, just like Twitter’s search was back in the day (and web search was before Google). I remember asking Evan Williams, in 2008, the best way to find someone on Twitter, and his response was “Google them, and add the word Twitter.” I’m pretty sure the same is true at present for Tumblr. (It’s how I found Fred, anyway).

Continuing the echoes of past approaches to the same problem, Tumblr currently provides a “suggested users” like directory on its site, highlighting folks you might find interesting. I predict this will not be around for long – because it simply doesn’t solve the problem we want it to solve. I want to find the right users for me to follow, not ones that folks at Tumblr find interesting.

If Tumblr can iron out these early kinks, well, I’d warrant it will take its place in the pantheon of companies who have found a signal, curated it at scale, and solved yet another important discovery problem. The funny thing is, all of them are still in the game – even Yahoo, who I’ve spent quite a bit of time with over the past few months. I’m looking forwarding to continuing the conversation about how they approach the opportunity of discovery, and how each might push into new territories. Twitter, for example, seems clearly headed toward a Tumblr-like approach to content curation and discovery with its right hand pane. Google continues to try to solve for people discovery, and Facebook has yet to prove it can scale as a true content-discovery engine.

The folks at Google used to always say “search is a problem that is only five-percent solved.” I think now they might really mean “discovery is a problem that will always need to be solved.” Keep trying, folks. It gets more interesting by the day.

* I’m going to leave out the signals of commerce (What I want to buy) and location (Where I am now) for later ruminations. If you want my early framing thoughts, check out Database of Intentions Chart – Version 2, Updated for Commerce, The Gap Scenario,and My Location Is A Box of Cereal for starters.

  • Content Marquee

Google's "Opinion" Sparks Interesting Dialog On Tying of Services to Search

By - December 02, 2010

the search cover.pngYesterday’s post on Google having an algorithmic “opinion” about which reviews were negative or positive sparked a thoughtful response from Matt Cutts, Google’s point person on search quality, and for me raised a larger question about Google’s past, present, and future.

In his initial comment (which is *his* opinion, not Google’s, I am sure), Cutts remarked:

“…the “opinion” in that sentence refers to the fact our web search results are protected speech in the First Amendment sense. Court cases in the U.S. (search for SearchKing or Kinderstart) have ruled that Google’s search results are opinion. This particular situation serves to demonstrate that fact: Google decided to write an algorithm to tackle the issue reported in the New York Times. We chose which signals to incorporate and how to blend them. Ultimately, although the results that emerge from that process are algorithmic, I would absolutely defend that they’re also our opinion as well, not some mathematically objective truth.”

While Matt is simply conversing on a blog post, the point he makes is not just a legal nit, it’s a core defense of Google’s entire business model. In two key court cases, Google has prevailed with a first amendment defense. Matt reviews these in his second comment:

“SearchKing sued Google and the resulting court case ruled that Google’s actions were protected under the first amendment. Later, KinderStart sued Google. You would think that the SearchKing case would cover the issue, but part of KinderStart’s argument was that Google talked about the mathematical aspects of PageRank in our website documentation. KinderStart not only lost that lawsuit, but KinderStart’s lawyer was sanctioned for making claims he couldn’t back up… After the KinderStart lawsuit, we went through our website documentation. Even though Google won the case, we tried to clarify where possible that although we employ algorithms in our rankings, ultimately we consider our search results to be our opinion.”

The key point, however, is made a bit later, and it’s worth highlighting:

“(the) courts have agreed … that there’s no universally agreed-upon way to rank search results in response to a query. Therefore, web rankings (even if generated by an algorithm) are are an expression of that search engine’s particular philosophy.”

Matt reminded us that he’s made this point before, on Searchblog four years ago:

“When savvy people think about Google, they think about algorithms, and algorithms are an important part of Google. But algorithms aren’t magic; they don’t leap fully-formed from computers like Athena bursting from the head of Zeus. Algorithms are written by people. People have to decide the starting points and inputs to algorithms. And quite often, those inputs are based on human contributions in some way.”

Back then, Matt also took pains to point out that his words were his opinion, not Google’s.

So let me pivot from Matt’s opinion to mine. All of this is fraught, to my mind, with implications of the looming European investigation. The point of the European action, it seems to me, is to find a smoking gun that proves Google is using a “natural monopoly” in search to favor its own products over those of competitors.

Danny has pointed out the absurdity of such an investigation if the point is to prove Google favors its search results over the search results of competitors like Bing or others. But I think the case will turn on different products, or perhaps, a different definition of what constititues “search results.” The question isn’t whether Google should show compeitors standard search results, it’s whether Google favors its owned and operated services, such as those in local (Google Places instead of Foursquare, Facebook etc), commerce (Checkout instead of Paypal), video (YouTube instead of Hulu etc.), content (Google Finance instead of Yahoo Finance or others, Blogger instead of WordPress, its bookstore over others, etc.), applications (Google Apps instead of MS Office), and on and on.

That is a very tricky question. After all, aren’t those “search results” also? As I wrote eons ago in my book, this most certainly is a philosophical question. Back in 2005, I compared Yahoo’s approach to search with Google’s:

Yahoo makes no pretense of objectivity – it is clearly steering searchers toward its own editorial services, which it believes can satisfy the intent of the search. … Apparent in that sentiment lies a key distinction between Google and Yahoo. Yahoo is far more willing to have overt editorial and commercial agendas, and to let humans intervene in search results so as to create media that supports those agendas. Google, on the other hand, is repelled by the idea of becoming a content- or editorially-driven company. While both companies can ostensibly lay claim to the mission of “organizing the world’s information and making it accessible” (though only Google actually claims that line as its mission), they approach the task with vastly different stances. Google sees the problem as one that can be solved mainly through technology – clever algorithms and sheer computational horsepower will prevail. Humans enter the search picture only when algorithms fail – and only then grudgingly. But Yahoo has always viewed the problem as one where human beings, with all their biases and brilliance, are integral to the solution.

I then predicted some conflict in the future:

But expect some tension over the next few years, in particular with regard to content. In late 2004, for example, Google announced they would be incorporating millions of library texts into its index, but made no announcements about the role the company might play in selling those texts. A month later, Google launched a video search service, but again, stayed mum on if and how it might participate in the sale of television shows and movies over the Internet.

Besides the obvious – I bet Google wishes it had gotten into content sales back in 2004, given the subsequent rise of iTunes – there’s still a massive tension here, between search services that the world believes to be “objective” and Google’s desire to compete given how the market it is in is evolving.

Not to belabor this, but here’s more from my book on this issue, which feels pertinent given the issues Google now faces, both in Europe and in the US with major content providers:

… for Google to put itself into the position of media middle man is a perilous gambit – in particular given that its corporate DNA eschews the almighty dollar as an arbiter of which content might rise to the top of the heap for a particular search. Playing middle man means that in the context of someone looking for a movie, Google will determine the most relevant result for terms like “slapstick comedy” or “romantic musical” or “jackie chan film.” For music, it means Google will determine what comes first for “usher,” but it also means it will have to determine what should come first when someone is looking for “hip hop.”

Who gets to be first in such a system? Who gets the traffic, the business, the profits? How do you determine, of all the possibilities, who wins and who loses? In the physical world, the answer is clear: whoever pays the most gets the positioning, whether it’s on the supermarket shelf or the bin-end of a record store. ….But Google, more likely than not, will attempt to come up with a clever technological solution that attempts to determine the most “objective” answer for any given term, be it “romantic comedy” or “hip hop.” Perhaps the ranking will be based on some mix of PageRank, download statistics, and Lord knows what else, but one thing will be certain: Google will never tell anyone how they came to the results they serve up. Which creates something of a Catch-22 when it comes to monetization. Will Hollywood really be willing to trust Google to distribute and sell their content absent the commercial world’s true ranking methodology: cold, hard cash?

…Search drives commerce, and commerce drives search. The two ends are meeting, inexolerably, in the middle, and every major Internet player, from eBay to Microsoft, wants in. Google may be tops in search for now, but in time, being tops in search will certainly not be enough.

Clearly, as a new decade unfolds, search alone is not enough anymore, and my prediction that Google will protect itself with the shield of “objectivity” has been upended. But the question of how Google ties its massive lead in search to its new businesses in local, content, applications, and other major markets remains tricky, and at this point, quite unresolved.

In Google's Opinion….

By - December 01, 2010

Google Opinon.png

Wow, I’ve never seen this before. Check out Google’s post, responding to the New York Times story about a bad actor who had figured out a way to make a living leveraging what he saw as holes in Google’s approach to ranking.

How Google ranks is the subject of increasing scrutiny, including and particularly in Europe.

From Google’s blog:

Even though our initial analysis pointed to this being an edge case and not a widespread problem in our search results, we immediately convened a team that looked carefully at the issue. That team developed an initial algorithmic solution, implemented it, and the solution is already live.

What I find fascinating is the way Google handled this. Read this carefully:

Instead, in the last few days we developed an algorithmic solution which detects the merchant from the Times article along with hundreds of other merchants that, in our opinion, provide an extremely poor user experience. The algorithm we incorporated into our search rankings represents an initial solution to this issue, and Google users are now getting a better experience as a result.

What word stands out? Yep, “opinion.”

Think on that for a second. If ever there was an argument that algorithms are subjective, there it is.

(Oh, and by the way, the last paragraph in the blog post clearly is directed at the regulators in Europe, if you think about it….)

Web 2 Conversation: Robin Li

By - November 18, 2010

I found myself really engaged with Robin (CEO, Baidu) in this conversation, and I found his answers to some difficult questions – like doing business under the Chinese government’s rule – to be refreshingly honest.


Web 2 Conversation: Eric Schmidt

By -

Given how silent this site has been over the past few weeks, it only seems right to share with you all the result of all my work. So I’ll be posting videos of some of the highlights of the Web 2 Summit this past week, starting with our kick off conversation with Eric Schmidt. Enjoy.


Mayer to Location: Big.

By - October 12, 2010

marissa-mayer.jpg

Today I was in a meeting with a number of consultants to a very large technology company. Their job: market research, essentially. They called to ask me my thoughts on the media and technology world, in particular as it might play out in the next five or so years. They were responsible for helping the Fortune 50 company navigate an increasingly complicated world.

I love these kind of free association tasks, because while it’s not easy to be right, it’s also pretty easy to not be wrong if the questions are smart. I’ve been a student of technology cycles for a couple of decades, and often times what’s directly in front of you is, in fact, the next big thing.

So when I got this question: “What’s the next big thing after social?” I didn’t lose a beat in answering: “Location.”

Now, many, many folks before me have been saying this for years. I’m in no way first. But I’m an early convert, in particular, as it relates to what I call the conversation economy. And the reason is simple: Once someone can declare where they are, they add extraordinary context to both search and social, and to their expectations of what a search or a social connection might yield. For an example, see The Gap Scenario.

In short, location is a key factor in the future of search, social, commerce, and media, among a lot of other things. And that’s why the news today that Google’s Marissa Mayer, long the VP of Search Products at Google, is taking over responsibilities for the location business, strikes me as a Big Deal.

Some have argued this is a demotion for Mayer, a Google stalwart and press favorite. But if in fact Google is “parking” Mayer in a “non job” due to her status as an early and long standing employee, I can’t imagine a more strategic area for her to park. And given Mayer’s success and wealth, I can’t imagine she’d stay at Google if she weren’t committed to a new role that she believes will be game changing. She has way too many other options, including, well, not working for as long as she’d like.

I for one don’t think that’s what is going on. Local is the most important signal to emerge in the database of intentions since the link. Once a consumer demands that businesses respond to their intent in the context of where they are, right now, well…the first to get that response right, wins.

Happy Bday, Google

By - September 27, 2010

Screen shot 2010-09-27 at 7.50.56 AM.png

Google is 12 today, as Sept. 27 marks the anniversary of the company’s actual incorporation. Of course the company has roots prior to the date, but ya gotta pick a date. Happy birthday Google, as they say in China, may you live in interesting times….

Comscore Search Results for August: Different from Nielsen

By - September 16, 2010

Comscore’s search results are out for August, and unlike Nielsen, they do not show Bing as beating Yahoo. On the contrary, Yahoo gained the most ground, Google lost share (by not growing as quickly), and Microsoft grew only slightly. To the charts:

Screen shot 2010-09-16 at 2.36.57 PM.png

Screen shot 2010-09-16 at 2.37.11 PM.png

Screen shot 2010-09-16 at 2.38.28 PM.png

Nielsen: Bing Overtakes Yahoo In Search Share

By - September 14, 2010

image002.gif

Screen shot 2010-09-14 at 7.19.37 AM.png

Way back in January, I predicted (see #8) that Microsoft’s Bing would overtake Yahoo in share to become #2 in search. Today, at least one measurement service seems to have validated that claim, which at the time was a bit far-fetched, given the nearly seven point gap in share between the two companies.

These figures do not consider the Yahoo/Bing’s search deal. From the release, which is not yet up on the site:

According to new research released by The Nielsen Company, for the first time, MSN/Windows Live/Bing Search overtakes Yahoo! as the #2 search engine in the U.S., with a 13.9% share of search volume in August 2010, a 0.25% delta increase from last month.

Although Google saw little change in its month-over-month search volume, it still dominates the search market, accounting for 65% of all U.S. searches.

Yahoo! followed Google and MSN/Windows Live/Bing Search with a 13.1% share of U.S. searches, falling from a 14.6% share in July 2010 to 13.1% (a 1.2% delta decrease or an 8% relative decrease).

In terms of a year-over-year comparison, Google has seen little change in its share of search while Yahoo! has seen a small but steady decline, going from a 16% share to 13.1% (a delta drop of 2.9% or a relative drop of 18%). MSN/Windows Live/Bing’s share has grown from 10.7% in August 2009 to 13.9% (a delta increase of 3.2% or a relative increase of 30%).

Google Instant: The Headlines and Quick Takes

By - September 08, 2010

Screen shot 2010-09-08 at 10.04.33 PM.png

Google today introduced what many are calling a major evolution in search interface today, sparking a landslide of commentary about the impact on SEO, mobile, competitors, search share, revenue, you name it.

It’s a lot to digest, and as much as I’d like to have a definitive statement on Google’s move to “instant search,” I don’t. Yet. I prefer to use it for a while, and think on it a bit more. I will admit that my initial response is more “meh” than “WOW!” – but then, I can’t really back that up. In the main, I think any major shift in search interface that is still predicated on typing inside a command line is most likely not going to change things much.

Then again, there are scores of folks who don’t share that half-formed sentiment. Here are some of the most prominent:

Live Blogging Google ‘Streaming” Search Event & How To Watch Live (SEL) Danny’s coverage of the news as it happened.

Search: now faster than the speed of type (Google Blog) The official announcement, with video.

Google Instant Makes SEO Irrelevant (Rubel) Not so fast, says Matt, below.

Thoughts on Google Instant (Matt Cutts) Matt is a key guy on search quality at Google. He says Google Instant will not kill SEO, among other things.

About Google Instant (Google.com) More from Google on why they did it.

Google Instant: A Mobile App Approach to Search (GigaOm) Interesting and cogent insight.

Google Instant Search: The Complete User’s Guide (SEL) As you would expect, second day overview on the first day from SEL.

Google Instant officially announced. Never underestimate speed. (TNW) Speed is the focus of Google’s announcement.

Google Just Killed The “I’m Feeling Lucky Button” (GOOG) (SAI) And, according to SAI, made a cool 100MM+ in the process.

More after a few days of using it…