free html hit counter Future of Search Archives | Page 2 of 8 | John Battelle's Search Blog

Facebook Is No Longer Flat: On Graph Search

By - January 15, 2013

A sample Graph Search result for the query “friends photos before 1999.”

By now the news is sweeping across the blogosophere and into the mainstream press: Facebook is doing Search!

Well, not so fast. Facebook is not doing search, at least not search Google-style. However, the world’s largest social network has radically re-engineered its native search experience, and the result is not only much, much better, it’s also changed my mind about the company’s long term future.

Yesterday, Tom Stocky, Facebook Product Management Director, and Lars Rasmussen, Engineering Director, gave me a sneak peek of today’s much anticipated announcement (it’s gonna be a phone! A new Newsfeed! A big acquisition!). So as to not bury the lead, Facebook has built what it’s calling “Graph Search,” a solidly conceived structured-search service which leverages the company’s massive trove of personal data in any number of new ways (some obvious, some nuanced, and some glaring omissions). But before I get to the details, I want to write about why this matters so much.

Prior to seeing the new search, I was not certain Facebook would ever live up to the hype it has accrued over its short life. It’s a good service, but it’s flat – over time, it struck me, people would tire of tending to it. They set up their social graph, toss a few sheep, poke some pals (or not), “like” this or that (often off-domain), waste hours on Farmville, and then…engagement drops slowly over time. I’m also not a fan of Facebook’s domain-specific approach to the world, as many of you know. Facebook’s new search doesn’t address Facebook’s walled garden mentality (yet), but it nails the first issue. Once this search product is rolled out to all of its members, Facebook will no longer be flat.

This is a big deal on many fronts. First and foremost, Facebook has an engagement problem, particularly in markets (like the US) where its use has become ubiquitous and many of its original users are two, three or more years into the “Facebook habit.” While the company doesn’t talk about this issue, I am confident it’s real (in private conversations with people at Facebook, it’s called the “set it up and forget it” problem). If people do not constantly feed Facebook with engagement, its value attenuates over time. As the service slows in overall growth, engagement with its current base becomes critical. New connections are the lifeblood of a service like Facebook. Without a steady stream of meaningful Likes, Friend Requests, declared Interests, and such, the platform would wither.

Put another way, Facebook needed a service that layered a fresh blanket of value over its core topography. Graph Search is it.

Zuckerberg’s Engagement

One sign of how important this new search is? According to the folks I spoke to yesterday, Facebook’s mercurial founder and CEO Mark Zuckerberg calls Search the “third pillar” of the company’s service, elevating it to the level of Newsfeed and Timeline, the two most important new features since Facebook’s launch (Open Graph is probably up there as well, but it’s true value remains locked up until there is mortar connecting it all, which Search could well be).

A team of engineers and product folk have been working on Graph Search for more than a year, and Zuckerberg has been engaged with them the entire time. The team has been in “lockdown” – a exclusive state of focus on one product so as to ship it as quickly as possible – for the past 34 days. Lockdown is a time honored and rather prestigious occurence inside Facebook, dating back to Zuckerberg’s original Facemash dorm room programming outburst. During the Search team lockdown, Stocky told me, he and Rasmussen got plenty of 2 AM emails and unexpected late night visits from the CEO.

In other words, this is A Really Big Deal for the company.

Why? Well, a quick tour of the product will explain.

What Is It? 

Graph Search subsumes Facebook’s previous search offering, which was extremely weak and focused mainly on the use case of navigation (finding people and pages).  The new service takes full advantage of the face that Facebook is, at its core, a massive structured database of tagged entities. The initial beta “indexes” four main types of these entities: People, Photos, Places, and Interests. Over time, I am told, Facebook will expand its index to include all Facebook posts and even the Open Graph – which means the “rest of the web.”

But for now, users can search across four main categories, using a slick set of intuitive verbs (“lives,” “like,” “work,” etc.), nouns (“San Francisco,” “Indian,” “restaurants,” “friends” etc.), prepositions (“before,” “with,” “in”) and pronouns (“who,” that,” etc.). This makes for a richly structured set of results: “Friends of friends who live in San Francisco and like Indian restaurants,” for example. Or “Friends who have been to Ireland,” or “Photos of friends before 1990.” Once you get the hang of it, the possible pivot points are endless, and the results are quite intriguing.

Stocky and Rasmussen, both ex-Googlers, walked me through a few intriguing use cases, one of which harkens back to one of Facebook’s original use cases – dating – and another which looks forward and presents a threat to LinkedIn’s current strength: Recruiting.

Let’s say you’re single, and you’re interested only in dating engineers who are also friends of your friends. With Graph Search, it’s ridiculously easy to find “friends of friends” who are also engineers. (And single, of course). You can look at their pictures, profiles, interests, and then ask for an introduction from whichever of your pals happens to be connected to one who looks like a good prospect (you could also just “poke” the guy if you wanted to…). Want only C++ programmers, or Indian C++ programmers, or  Indian C++ programmers under 35 years of age? Done.

Or, let’s say you work at, I dunno, Google. And you want to recruit product management talent from, say, Facebook. Again, the best way to get to that talent is probably a friend. So why not do a search for “friends of friends who work at Facebook and are product managers”? Why not, indeed.

One can imagine such functionality will create a lot of new engagement on the service. And not just from people “friending” prospective beaus or hires. Recall that when Google burst onto the scene, it prompted a dramatic response from owners of web pages, who immediately began rewiring their sites to be optimized for search. Similarly, Facebook’s Graph Search will incent Facebook users to “dress” themselves in better meta-data, so as to be properly represented in all those new structured results. People will start to update their profiles with more dates, photo tags, relationship statuses, and, and, and…you get the picture. No one wants to be left out of a consideration set, after all.

Facebook Gets More Weather

Last year I wrote a post titled “Facebook Is Now Making Its Own Weather.” The focus was on Facebook’s Newsfeed, and how an economy of value was now in place to game Facebook’s “edgerank” algorithm, which determines what stories show up in a person’s feed. With Graph Search, I expect a similar ecosystem will emerge. All of a sudden, two things will be true that previously were not: Facebook users will be using search, a lot, creating liquidity in Facebook “SERPS.” And secondly, there will be significant perceived value in being included in those search results, both for individuals (I want to be considered for that job at Google!) and for companies/brands (I want to message to anyone looking for a job!).

While Graph Search is in very early beta, I don’t think I’m going out on a limb by predicting that it won’t be long before Facebook integrates a product that lets marketers purchase ads in these new search results. It already has a similar product, which is by default included in suggested searches (the “auto completed” queries suggested to a user as they enter terms). At the moment, however, paid listings are not included in search results. They will be. Which means, of course, the rise of a native SEO/SEM ecosystem inside Facebook. Add in Open Graph search across the web, and presto…Google’s got some serious potential competition. (Well, not exactly presto. Incorporating Open Graph is going to take some serious chops and time. But still…).

Even without incorporation of Open Graph or Posts, Graph Search is going to change the game for brands and people on the Facebook service. As I watched Stocky and Rasmussen put their product through its paces, I couldn’t help but wonder how much new traffic the product will drive around the Facebook Platform. Will Facebook be watching “conversions” – clickthroughs from search results to profiles and pages? Of course they will! Will Facebook report those referrals to individuals and brands, much as Google Analytics does for webpage today? Not yet…but wait for it. It’ll come….

 What’s Missing: Sharing Results

I’ve already noted that Graph Search does not index content (posts) or the Open Graph, though I’m told that’s coming. But the big miss, from my point of view, is the inability to share search results.

Share search results? Who’d want to do that? Well, in web search, very few of us. That’s because with rare exception, open web search is not an inherently social action – it’s private and it’s ephemeral. But inside the walls of Facebook, it’s definitionally so. In fact, I’d argue that every single “result page” in Graph Search is a “media object” in its own right. If you search for “pictures of friends before 1990,” for example, you get the equivalent of a Pinterest board of your friends’ childhood shots. Wouldn’t you like to post that on your timeline so your pals can see it? Better yet, wouldn’t you like to export it to Pinterest or Tumblr? Of course you would (but, alas, I don’t expect Facebook will allow it, under cover of “protecting user privacy.” More on that in a second.)

Or take another example. Say you have a pal in Southern California who is despondent after being dumped by her boyfriend. You do a quick Graph Search for “single friends of friends under 30 who work in Los Angeles.” The results look pretty promising. Don’t you want to shoot them over to your pal with the subject line “Don’t despair, there are plenty of fish in the sea!” Of  course you do.

I mean, just a query like “Photos I Like” is a huge feature win for Facebook. And who wouldn’t want to post a montage of “Photos I Like” to their timeline? (Or, ahem, their personal blog?!)

For now, you can’t share the results of your searches with anyone else, and that’s a bug that should be a feature. When I brought up the issue, I was told that the privacy implications of sharing searches were extremely complicated. Because of past missteps and current scrutiny, Facebook is going to tread cautiously here (privacy was a central theme in Graph Search’s launch). I certainly understand why, but while those issues are sorted, I expect there are going to be a lot of screen shots of Graph search results being shared around the web.

Bigger privacy issues will likely arise around what might be called the Randi Zuckerberg principle – as in “Oh shit, I didn’t realize I’d show up in that circumstance!” Graph Search is going to expose all manner of privacy controls as super important, and send millions back to Facebook’s sometimes-confusing dashboards, so as to appropriately re-tool settings such that nothing untwoard shows up in this important new functionality.

And to me, this is a Very Good Thing. A couple of years ago, I wrote a post titled  The Rise of Digital Plumage in which I predicted that we’d all become habituated to “dressing” ourselves in structured data, so as to best present ourselves to the world at any given time. Graph Search is another important tool in our ever-growing digital wardrobe, one that motivates us to understand and manage the implications of our ever-expanding digital footprint.

Facebook just posted an announcement about its new search here.  The initial beta will roll out slowly, folks will have to ask to join a waitlist to get the service. I’ll be updating this post as the news is discussed and digested….

  • Content Marquee

With Google’s 2012 Zeitgeist, You Won’t Learn Much. Why?

By - December 13, 2012

Guess what? This guy was big this year. Really!

I think readers know that on balance, I’m a fan of Google. I recently switched to the Nexus 4 (more coming on that front as I settle into really using it). I believe the company has a stronger core philosophy than many of its rivals. Overall, given that it’s nearly impossible to avoid putting your data into someone’s cloud, I believe that Google is probably the best choice for any number of reasons.

But that doesn’t mean I won’t criticize the company. And every year about this time, I end up doing just that.

Because the annual Google Zeitgeist came out this week, and I’ve spent a bit of time digging into it. And once again, I’m pretty disappointed.

In the past I’ve criticized Google for failing to ask interesting questions of the massive amount of data it collects on search patterns each year. Once again, this lament applies. I honestly do not care what top ten TV Shows, Sports Stars, Songs, or even People we collectively care about, because there is *never* a surprise in those results.

But Google knows so much more….and could really tease out some insights if it cared to. Imagine if Google took its massive search query database and worked with some of the leaders in the open data movement to mine true insights? Sure, Google would have to be careful about how it released the data, but the output would be extraordinary, I’d warrant.

Instead, we find out that Gangnam Style was a big deal this year. No shit!?

But it gets worse. Not only is Zeitgeist rife with pop culture fluff, as you drill down into it by country, eager perhaps to find something interesting, it turns out Google has chosen to eliminate certain potentially sensitive categories altogether.

For the US and most other countries, for example, there is a “What is….” category, which shows the top search queries that start with “What is…” For the US, the answers are

  1. What is SOPA
  2. What is Scientology
  3. What is KONY
  4. What is Yolo
  5. What is Instagram
  6. What is Pinterest
  7. What is Lent
  8. What is Obamacare
  9. What is iCloud
  10. What is Planking

But is there a “What Is…” for Saudi Arabia? Nope. China? Uh-uh. The United Arab Emirates? No sir. Egypt? Move along.

Hmmm.

Oddly, Google did provide “What is…” was for Singapore, where people living under that “benign dictatorship” were interested in the same things as the US –  “What is SOPA”,  “What is Scientology” and, for politicians, who is “Mitt Romney.”

For the US only, you can drill down into all manners of other categories past the main page, including News, Science, Tech, Humanities, and Cities. Those are pretty interesting categories, but Google only provides them for the US, which is a shame.

Furthermore, I find it interesting that Google, with all of its translation technology, does not have a translation button on the results pages for countries where the majority of the searches are in languages other than English. This is most likely due to political sensitivities, because if you run some of the results through Google Translate (do you believe I had to do that?!), you get some stuff that I am sure does not please the regimes of countries like China, Saudi Arabia, and the UAE.

For example, here are some of the top searches for Saudi Arabia, translated (roughly I am sure) by the Google Translate service:

Student outcomes Arab Idol insurance Ramadan Series 2012 Mohamed Morsi explosion Riyadh Burma Free Syrian Army Shura Council tornado Sandy

But again, you aren’t going to get much more insight into what Saudi folks are *really* thinking about, because Google failed to ask the interesting questions, like those it has in the “News” section of the US Zeitgeist. I’d sure be interested in “Political Gaffes,” “Election Issues,” and “News Sources,” in Saudi Arabia, China, or the UAE.

In fact, for Saudi Arabia, Google has ommitted the “Top News Searches” box that is on several of the other country pages (even Egypt). Instead, the topics for Saudi Arabia (besides trending searches and people) focus on sports and entertainment stars, fashion designers, TV shows, and the like. Deep, Google. Thanks.

Now, the datasets are different for each country, and it may be that Google simply didn’t have enough trending data to surface interesting political insights for these controversial countries.

Somehow, though, I don’t buy that. This set of lists feels extremely human vetted – I’m guessing an awful lot of hand wringing went into chosing what to show and what might prove problematic to Google’s best interests were it to see the light of day.

If that is the case, I urge the company to have more courage. I bet if Google open sourced its query data sets (eliminating any chance of PII getting out, of course), I bet academics, data scientists, and just plain interested folks would let loose an explosion of insight. Pop up the rainbird of data, Google, and let the ecosystem flourish. We’d all be the richer for it.

Facebook Is Now Making Its Own Weather

By - November 09, 2012

(image) The past month or so has seen the rise and fall of an interesting Internet tempest – the kind of story that gets widely picked up, then quickly amplified into storms of anger, then eventually dies down as the folks who care enough to dig into the facts figure out that the truth is somewhere outside the lines of the original headline-grabbing story.

The topic this time around centers on Facebook’s native ad unit called “Sponsored Stories,” and allegations that the company is gaming its “Edgerank” algorithm such that folks once accustomed to free promotion of their work on Facebook must now pay for that distribution.

Edgerank determines the posts you see in your Facebook newsfeed, and many sites noticed that sometime early this Fall, their traffic from Facebook shrank dramatically. Others claimed traffic had been declining since the Spring, but it wasn’t until this Fall that the story gained significant traction.

I’ve been watching all this play out – first via an angry post on the New York Observer site in which the author posits that Facebook is “broken on purpose” so as to harvest Sponsored Story revenue. An even angrier post on the same theme came five weeks later on a site called Dangerous Minds. From it:

Spring of 2012 was when bloggers, non-profits, indie bands, George Takei, community theaters, photographers, caterers, artists, mega-churches, high schools, tee-shirt vendors, campus coffee shops, art galleries, museums, charities, food trucks, and a near infinite variety of organizations; individuals from all walks of life; and businesses, both large and small, began to detect—for it was almost imperceptible at first—that the volume was getting turned down on their Facebook reach. Each post was now being seen only by a fraction of their total “fans” who would previously have seen them.

The author goes on to argue that Facebook was breaking the implicit contract between himself – an independent blogger – and Facebook, the corporation.

…as a publisher of a medium readership blog, I used to get a great deal from using Facebook—but I understood it to be a two-way reciprocal arrangement because I was driving traffic back to Facebook as well, and reinforcing their brand awareness with prominent widgets on our blog.

Now, if you’ve read my Thneeds post, you know I’m sympathetic to this point of view. I believe large social platforms like Facebook and Twitter “harvest” content from the Indpendent Web, and leverage the traffic and engagement that this content creates on their platforms to their own benefit via scaled advertising offerings. Most of us are fine with the deal – we promote our work on social sites, social sites drive traffic back to us. We like that traffic, either just because we like more folks reading our work, or, in the case of commercial sites like this one, because we serve ads against it.

Now, as I’ve noted many times over the past six months, this bargain is breaking down, because it’s getting harder and harder to monetize traffic using standard display advertising units. That’s not Facebook’s problem, per se, it’s ours. (See here for my suggestions as to how to solve it).

Nevertheless, for many sites, the spectre of losing significant traffic from Facebook means a serious blow to revenues. And from the point of view of the Dangerous Minds blogger, Facebook first cut his traffic off, then began asking him to pay to get it back (in the form of promoting his posts via Sponsored Stories).

This makes for a very good narrative: corporate greed laid bare. It got picked up by a lot of sites, including Ars Technica and even the aforementioned George Takei, who is upset that he’s lost the ability to push his posts to all 2.9 million of his Facebook fans.

Turns out, the truth is a lot more complicated. I’ve done some reporting on this issue, but not nearly as much as TechCrunch did. In a follow up to the Dangerous Minds story, TechCrunch claimed to have debunked the entire story. Titled Killing Rumors With Facts: No, Facebook Didn’t Decrease Page Feed Reach To Sell More Promoted Posts, the story argues that Facebook didn’t change its algorithms to drive up revenue, but rather to cull “spammy posts” from folks’ newsfeeds.

Facebook has always shown just a percentage of all possible posts in a given person’s newsfeed. Anyone paying attention already knew that. The company uses its Edgerank algorithm to determine what it thinks might be interesting to an individual, and sometime in the past few months, I can confirm through sources which wish to remain anonymous that Facebook made a pretty significant change to Edgerank that penalized posts that it felt were not high quality.

Of course, that begs the question: How does Facebook determine what “quality” is? The answer, in the main, is by measuring engagement – is the post shared, liked, clicked on, etc? If so, then it is seen as quality. If not, it’s demoted in value.

Is this sounding familiar to anyone yet? In short, Facebook just executed a Panda.

I held back from writing anything till this predictable cycle played out, because I had a theory, one that I believe is now confirmed: Facebook is now making its own weather, just like Google, and in the past couple months, we’ve witnessed the first widespread instance of a Facebook weather event.

For those of you who don’t know quite what I’m talking about, a bit of history. Ten or so years ago, the ecosystem around search began to notice shifts in how Google drove traffic around the web. Google would make a change to its algorithms, and all of a sudden some sites would see their traffic plummet (other sites sometimes saw the opposite occur). It seemed to those injured that the only way to get their Google traffic back was to buy Google AdWords – corporate greed laid bare. This story played out over and over, to the point where the weather events started to get names, just like hurricanes do. (The first was called Boston).

Early last year Google made a major change to its algorithms that penalized what it believed was lower quality content. Dubbed “Panda,” the changes targeted “content farms” that cranked out SEO friendly pages as AdWords bait. This had dramatic effects on many sites that specialized in “gaming” Google. It also hit sites that weren’t necessarily playing that game – updates like Panda often create collateral damage. Over time, and as it always does, Google fine-tuned Panda until the ecosystem stabilized.

I believe that Facebook is now learning how to manage its own weather. I don’t know the Dangerous Minds website well enough to know if it deserved the drop in traffic that occurred when Facebook had its Panda moment. But one thing does strike me as interesting to note: A significant drop in traffic means a particular site is losing audience that has proactively decided to click on a link inside their newsfeed. That click means the person leaves Facebook and goes to the the Dangerous Minds site. To me, that’s a pretty serious sign of engagement.

However, one might argue that such a signal is not as important to Facebook as internal ones such as “liking” or “sharing” across the Facebook network. To that end, I am sure we’ve not heard the last round of serious grumbling that Facebook is gaming its own Edgerank algorithm to benefit Facebook’s internal goals – to the detriment of the “rest of the web.” Be they publishers or folks like George Takei, who after all wants to push his Facebook fans to any  number of external links where they might buy his books or sign up to meet him at the next Comic Con, the rest of the web depends on “social traffic” from Facebook. The question is, should they optimize for that traffic, or will their efforts be nullified in the next Edgerank update?

Facebook is learning how to tread the delicate line between its own best interests, and those of its users – and the Internet That Is Not Facebook. Google does this every day – but it has a long history as a distributor of traffic off its main site. Facebook, not so much. Over time, the company will have to decide what kind of a relationship it wants to have with the “rest of the web.” It will probably have to start engaging more openly with its own ecosystem, providing guidance on best practices and how to avoid being penalized. This is a practice that took Google years to hone, and many still think the company has a lot of work to do.

Regardless, Facebook is now making its own weather. Now comes the fun part: Trying to predict it.

Bing Tries Harder, But For Me, It’s A Draw

By - September 06, 2012

It’s not easy being number two. As a marketer, you have limited choices – you can pretend you’re not defined by the market leader, or, you can embrace your position and go directly after your nemesis.

For years, Bing executives have privately complained about how hard it is to “break the Google habit,” even as they refused to market directly against Google. They were Avis, always trying harder.

No more. Today Microsoft announced its “Bing It On” challenge, a direct descendant of the iconic Pepsi challenge more than 30 years ago (the fact that I still remember that marketing campaign, and feel good about it, is a testament to its power).

It’s always a risk to ask consumers to test products blind, side by side, but Bing is doing it: Right here at “Bingiton.com.”

I bit and took the challenge – how did it go?

My first query has been my baseline for more than ten years – my own name (“john battelle“). Yeah, it’s a vanity search, but all of us have very strong opinions about what comes up when we put our names into search.

The winner? It was close, but Bing won. Its results seemed fresher – the Google screen had stuff about me from eBay and the BusinessWeek exchange in the first page (I never use eBay, and haven’t been active on that BusinessWeek page for more than two years). The Bing side also had my LinkedIn profile, which I consider important, though it also had an old picture of me flipping off the camera from 1998 (that’s getting very old), and a picture of a former business partner who isn’t me at all.

My second search – the misspelled (on purpose) “bset hotels sydney” made me question how the results were being delivered to the test site. Given how much I know about Google’s SERPs, it was pretty easy for me to tell which side was Google (it’s the left – the giveaway is the list of hotels with integrated reviews). But the results didn’t look quite like I was used to at Google. Here’s a comparison:

The main reason? This test had stripped out Google’s Maps feature for some reason, which certainly penalized the page from a visual and utility standpoint. Doesn’t seem like a fair fight.

So I gave that one a draw and moved onto another search.

Next up I tried a search I know both engines have had a bit of trouble with. I often lose the URL of my son’s boy scout troop, and have to search around for it a bit – it used to be buried in a nested Web 1.0 service, but recently was updated with its own URL, which unfortunately has terrible SEO. My first query usually doesn’t work, but it leads me in the right direction. It’s been a year or so since I’ve tried this (my son is older now), so I thought this might be a fresh search with some history to it. The query is “troop 43 larkspur california“.

The winner was most certainly Google. It found the old website (which has been impossible to find in the past) and the new one built in the last two years.

My next query was very utilitarian. My dad had a scare last night and is staying overnight at the hospital. I need to call the main line to check how he’s doing. So I entered “marin general hospital phone.” I figure if you put the word “phone” in there, the search engine should understand I need the phone number.

The Bing results had the number in the snippet of the first result. Google had it broken out clearly, but as the fourth result. Again, I know on Google I always get a map. But there was no map in these results. Also, I know that Bing prides itself on breaking out phone numbers, but I didn’t see the familiar Bing phone breakout box. Oh well, I had to go with Bing, because the information I needed was surfaced in the first result.

So going into my last search, it was two for Bing, one draw, and one for Google.

 

My last test was “winter rentals stinson beach” – a search I’ve done recently – and with some frustration – as I am taking a place there to write over the winter. I know what good results look like here, given I’ve done a lot of poking around already. It was relatively easy for me to pick a winner. It was Google, which filtered out most of the single home entries (I don’t want to find one home, I want to find listings with lots of them) and it also highlighted services and a local realtor I happen to know has the best inventory in the area.

So for me, the test concluded as a draw – two wins for Bing, two for Google, and one disqualification. Not exactly the two-to-one ratio in favor of Bing that Microsoft claims is the average, but then again, not bad either.

Remember, this is an entirely non scientific and subjective “test.” And of course, this test by its nature must exclude any personalization, search history, or other important bells and whistles that search engines use to tailor results to ongoing clients.

In the end, Bing proved to me that it deserves to be considered equal to Google for a variety of use cases. I don’t know if that’s enough to break the Google habit, but it certainly will get folks talking. And that’s an important part of marketing, isn’t it?!

What Is Search Now? Disjoined.

By - August 31, 2012

(image shutterstock)

Today I answered a question in email for a reporter who works for Wired UK. He asked smart questions, as I would expect from a Wired writer. (Some day I’ll tell you all my personal story of Wired UK – I lived over there for the better part of a year back in 1997, trying to make that magazine work. I mostly failed – but it’s up and running strong now.)

In any case, one question in particular struck me. The writer is preparing a piece on the future of search. (I’ll link to it when it comes out). What big problems, he asked, still plague search?

That got me thinking. Here’s my answer:

The largest issue with search is that we learned about it when the web was young, when the universe was “complete” – the entire web was searchable! Now our digital lives are utterly fractured – in apps, in walled gardens like Facebook, across clunky interfaces like those in automobiles or Comcast cable boxes. Re-uniting our digital lives into one platform that is “searchable” is to me the largest problem we face today. 

It may be worth expanding on that sentiment. When it broke out in the mid 1990s, the web was society’s first at-scale digital artifact.  It spread in orders of ten, first thousands, then millions, then hundreds of millions of pages – and on it went, to the billions it now encompasses. Everybody wanted to “be” on the web – a creator class started making pages and companies and services, a consumer class started “surfing” this vast new digital object, and our collective conscience marvelled at what we had created together: millions of small pieces loosely joined. And the key and unappreciated point is this: those pieces were indeed joined.

It was that joining – through links, of course – that made search possible, that created what is unquestionably the most powerful and lasting new company of the past 20 years – Google.* But as I wrote in Why Hath Google Forsaken Us? A Meditation, Google’s core model – built on the open, linked world of the web – is under threat from the advance of the iPhone and the app, the Facebook and the Path, the automobile console, the Xbox, the cable box, and countless other “unlinked” digital artifacts.

Google knows this. Why else invest so much in Android, in Google+, in Motorola (it’s not just phones, it’s also cable boxes), in self-driving cars, for goodness sake? Google wants a foothold wherever digital information is created and shared, and man, are we creating a sh*t ton of it. Problem is, we’re not making it easy – or even possible – to link all this stuff together, should we care to.

Which takes me back to that core question the Wired reporter asked me: What’s the biggest problem plaguing search? In short, it’s that our digital world is no longer small pieces loosely joined. It’s also big chunks separate and apart. And that makes search – in its most broadest interpretation – damn near impossible.

Which leads to another question: What then, is search? Of course, the Wired reporter asked me that as well. My answer:

Search is now more than a web destination and a few words plugged into a box. Search is a mode, a method of interaction with the physical and virtual worlds. What is Siri but search? What are apps like Yelp or Foursquare, but structured search machines? Search has become embedded into everything, and has reached well beyond its web-based roots.

So we all search now, all the time, across all manner of artifacts, large and small. But our searches are not federated – we can’t search across these repositories, as we could across that wonderful, vast, loosely joined early world of the web. We’ve lost the connection.

Call me a fool, but I think the need for that connection will be so strong, that in time, we’ll sew all our digital artifacts back together again. At least, I certainly hope we will. Right now, it ain’t looking so likely – what with patent wars, wagon circling by big platforms, and the like. But I’m an optimist – and I hope you are as well.

* Sorry but Facebook isn’t there – yet. And Microsoft and Apple, well, they may make a play for that crown either 20 years ago or 20 years hence, but if you ask me for the most important company ever that launched as a native web business, the answer is indisputably Google.

 

 

 

Here We Go Again: The Gray Market in Twitter and Facebook

By - August 07, 2012

So, casually reading through this Fast Company story about sexy female Twitter bots, I come across this astounding, unsubstantiated claim:

My goal was to draw a straight line from a Twitter bot to the real, live person whose face the bot had stolen. In the daily bot wars–the one Twitter fights every day, causing constant fluctuations in follower counts even as brands’ followers remain up to 48% bot–these women are the most visible and yet least acknowledged victims…

There it was, tossed in casually, almost as if it was a simple cost of doing business – nearly half of the followers of major brands could well be “bots.”

The article focuses on finding a pretty woman whose image had been hijacked, sure, but what I found most interesting (but sadly unsurprising) was how it pointed to a site that promises to a thousand  followers to anyone who pays…wait for it…about $17. Yes, the site is real. And no, you shouldn’t be surprised, in the least, that such services exist.

It has always been so.

Back when I was reporting for The Search, I explored the gray market that had sprung up around Google (and still flourishes, despite Google’s disputed attempts to beat it back). Fact is, wherever there is money to be made, and ignorance or desperation exists in some measure, shysters will flourish. And a further fact is this: Marketers, faced with CMO-level directives to “increase my follower/friend counts,” will turn to the gray market. Just as they did back in the early 2000s, when the directive was “make me rank higher in search.”

Earlier this week I got an email from a fellow who has been using Facebook to market his products. He was utterly convinced that nearly all the clicks he’s received on his ad were fake – bots, he thought, that were programmed to make his campaigns look as if they were performing well. He was further convinced that Facebook was running a scam – running bot networks to drive performance metrics. I reminded him that Facebook was a public company run by people I believed were well intentioned, intelligent people who knew that such behavior, if discovered, would ruin both their reputation as well as that of the company.

Instead, I suggested, he might look to third parties he might be working with – or, hell, he might just be the victim of a drive-by shooting – poorly coded bots that just click on ad campaigns, regardless of whose they might be.

In short, I very much doubt Facebook (or Twitter) are actively driving fraudulent behavior on their networks. In fact, they have legions of folks devoted to foiling such efforts.Yet there is absolutely no doubt that an entire, vibrant ecosystem is very much engaged in gaming these services. And just like Google had at the dawn of search marketing, Twitter and Facebook have a very – er – complicated relationship with these fraudsters. On the one hand, the gray hats are undermining the true value of these social networks. But on the other, well, they seem to help important customers hit their Key Performance Indicators, driving very real money into company coffers, either directly or indirectly.

I distinctly recall a conversation with a top Google official in 2005, who – off the record – defended AdSense-splattered domain-squatters as “providing a service to folks who typed the wrong thing into the address bar.” Uh huh.

As long as marketers are obsessed with hollow metrics like follower counts, Likes, and unengaged “plays,” this ecosystem will thrive.

What truly matters, of course, is engagement that can be measured beyond the actions of bots. It is coming. But not before millions of dollars are siphoned off by the opportunists who have always lived on the Internet’s gray edge.

On Mayer, Yahoo!, and The (Other) Customer

By - July 18, 2012

Mayer at the Web 2 Summit, San Francisco

(image James Duncan Davidson)

I try to let big news percolate for a few days before weighing in, and it seems even more appropriate to follow that playbook when it came to the scrum around Marissa Mayer joining Yahoo.

Yes, I’ve known both Marissa Mayer (and Ross Levinsohn) professionally, for more than a decade, but so do many other folks, and it seems nearly all of them – Steven Levy and Kara Swisher intelligently among them – have weighed in, multiple times, on what this all means. If you want a rundown, just search for “Marissa Mayer” in Google News.

The coverage has taken its usual course from “Holy Shit!” to “What Will Happen to Ross?” to “Wait, Is Mayer Right for the Job” to “Here’s Our Advice/The Things That Need  to Be Fixed/What Mayer’s Focus Should Be” types of pieces.

This won’t really be any of those. Instead, I find myself thinking about the things I’ve not really seen much coverage of, at least in depth. And true to what I’ve spent a fair amount of time thinking about, they all come down to the intersection of media and technology, and the role marketing plays in that landscape.

When I spoke to Mayer after she was named CEO, I asked the question, almost as a joke – “So is Yahoo! a media or a technology company?” She was quick to respond that she just does not get the debate – of course it’s both. What matters, she pressed, is creating great products that surprise and delight Yahoo! customers.

I couldn’t agree more, yet there is an important nuance here – just who *are* Yahoo’s customers?

Let me step back here and posit something that might upset more than a few of you: Yahoo has two sets of customers, and of course the “end user” is one of them. But the other is the marketer.  And media companies – or “tech companies driven by media revenues,” or however else one might want to phrase it – sometimes ignore this fact at their peril.

I’ll let those of you who find such a statement anathema go ahead and click away – here’s a nice unicorn chaser if you’d like – or you can flame me in the comments (I do respond to most, as long as they’re in English and don’t employ more than the occasional insult).

But those of you who’ve continued to read probably know that I believe, deeply, that commercial publishing is a conversation between three key parties: The reader (or viewer), the publisher/content creator, and the marketer. And while it’s generally been true that this conversation has been all kinds of broken during much of the web’s history, the truth is, it needn’t be that way. Six years ago (!) I wrote a series of posts describing the rise of conversational media and imploring that marketers learn to join the conversation. I think it’s fair to say that this is happening, at scale.

Beyond the contributions of pioneers like Federated Media (yes, I had to plug us), the rise of “native” advertising formats is proof of this. Twitter’s promoted suite is one growing example, as is Facebook’s Sponsored Stories (and its attendant focus on getting brands to be true publishers on the Facebook platform). Pinterest, WordPress (in partnership with FM), and Tumblr are hard at work on “native” solutions for their services as well. All of these advertising solutions pale, however, in comparison to the original “native” advertising format of the Web: Google AdWords.

Many have pointed out that Mayer’s principle weakness, when compared to Levinsohn, is her lack of traditional media and marketing chops. I can say from very deep experience that the marketing business is very much a relationship business – CMOs and agency leaders live in a world driven by ideas, creative and content – and they want to know the people who they do business with, and trust them in a way that is difficult to model algorithmically. Mayer’s detractors point out that she’s not spent much time wooing Madison Avenue, or dealing with the inevitable headaches born of the complex, people-driven businesses that are agencies, marketing clients, and content partners.

While there is some truth in this criticism, I think it overlooks a few things. First and foremost, Mayer is a very fast study, and she already knows how important the traditional media business is to Yahoo. Hell, a quick overview of the company’s financials bears this out, as does a visit to any of its properties, which are dominated by advertising. Yahoo may have a lot of technology behind the covers, but its products are nearly all media products – content intended to gather an audience and provide a place for marketers to message to that audience. More than half of Yahoo’s revenues come from “display” advertising, most of the rest comes from search, which is also marketer driven.

Secondly, Mayer will be a big draw of talent, and not just engineering talent. She understands that if she can’t retain Levinsohn and/or his recent CRO Michael Barrett (I certainly hope she can), she’ll need to attract top tier media minds to the business. And I think she’ll succeed at doing just that.

But to me, the thing many are missing is that Mayer will bring her fanatical product focus to more than just Yahoo’s consumer-facing media offerings. She’ll also be staring at the company’s advertising products, and asking this simple question: How can we do better?

To answer that question, Mayer will need to do more than study the data (though of course, that will be important). She’ll need to sit down with a wide swath of Yahoo’s marketing customers and ask them what they want from their investment in her platform. She’ll hear an awful lot of conflicting advice, but it’s in the bricollage from all the feedback that the best ideas come out. Mayer can’t afford to immediately tack away from all those boxes and rectangles cluttering up the Yahoo! experience, nor should she – it turns out that display advertising does indeed work for marketers. But the larger question remains: Can we do better?

The answer lies in executing the subtle and ongoing iterative work of true digital publisher – improving the core product experience both sets of customers – consumers of the media experience, as well as marketers looking to be part of that experience in a more native fashion. And again, from a quick study of Yahoo’s products, there’s plenty of improvements to be made.

An important and related part of the work ahead for Mayer and her team will be deciding what role ad tech and search will play in Yahoo’s future. Despite purchasing Right Media back in 2007, Yahoo has never been seen as a leader in ad tech, and word on the street in the weeks prior to Mayer’s ascension was that Yahoo was about to outsource its ad technology platform to market leader Google. Of course, such a move is fraught with regulatory and business implications. And Mayer may well decide it’s in Yahoo’s best interest to invest in own its own destiny when it comes to the machine-driven world of ad serving and programmatic audience buying. But trust me, what Yahoo does here will be an extremely important directional indicator.

Which brings us to search. It’s been widely reported that Yahoo’s 2009 deal to outsource core search to Microsoft hasn’t worked out as well as either party wished it would. Given how important search is to Yahoo overall, and how deeply knowledgeable Mayer is in this particular field, I’d expect big changes in Yahoo Search. The company recently unveiled a new search product called “Axis,” which seems like a neat idea but feels a bit too complicated for most consumers to really grok. Mayer will likely take Occam’s Razor to search, and I expect the results will be quite positive.

But it’s the other side of Yahoo’s revenue equation – the branded display market – where Mayer will face her greatest challenges, and find her biggest opportunities. Yahoo isn’t a startup like Pinterest, Tumblr,  or even Twitter, where founders can leverage massive user growth to raise enough capital to “figure out how best to implement appropriate native marketing solutions.” Yahoo is nearly 20 years old, and it’s got a very deep, tangled, and somewhat tarnished brand in the minds of its best advertising customers. It’s true that creating world-beating consumer-facing products will go a long way toward fixing that brand. But those products must be informed by – and even created for – both sets of customers – the consumers of content, as well as those who pay for them to be created in the first place.

Larry Page Makes His Case

By - April 05, 2012

Given the headlines, questions, and legal actions Google has faced recently, many folks, including myself, have been wondering when Google’s CEO Larry Page would take a more public stance in outlining his vision for the company.

Well, today marks a shift of sorts, with the publication of a lenthy blog post from Larry titled, quite uninterestingly, 2012 Update from the CEO.

I’ve spent the past two days at Amazon and Microsoft, two Google competitors (and partners), and am just wrapping up a last meeting. I hope to read Page’s post closely and give you some analysis as soon as I can. Meanwhile, a few top line thoughts and points:

- Page pushes Google+ as a success, citing more than 100 million users, but still doesn’t address the question of whether the service is truly being used organically, rather than as a byproduct of interactions with other Google products. I’m not sure it matters, but it’s a question many have raised. He also doesn’t address, directly, the tempest over the integration of G+ into search.

- Page also does not directly address the issue of FTC privacy investigations into the company, not surprising, given any company’s response to these investigations is usually “no comment.” However, Google might have explained with a bit more gusto the reasons for its recent changes.

- Page tosses out another big number, this one around Android: 850K activations a day. Take that, Apple!

- Page uses the words “love” and “beauty” – which I find both refreshing and odd.

- Page also talks about making big bets, focusing on fewer products, and how it’s OK to not be exactly sure how big bets are going to make money. This is a topic where Google has a ton of experience, to be sure.

More when I get out of my last meeting….

If-Then and Antiquities of the Future

By - April 03, 2012

Over the past few months I’ve been developing a framework for the book I’ve been working on, and while I’ve been pretty quiet about the work, it’s time to lay it out and get some responses from you, the folks I most trust with keeping me on track.

I’ll admit the idea of putting all this out here makes me nervous – I’ve only discussed this with a few dozen folks, and now I’m going public with what I’ll admit is an unbaked cake. Anyone can criticize it now, (or, I suppose, steal it), but then again, I did the very same thing with the core idea in my last book (The Database of Intentions, back in 2003), and that worked out just fine.

So here we go. The original promise of my next book is pretty simple: I’m trying to paint a picture of the kind of digital world we’ll likely live in one generation from now, based on a survey of where we are presently as a digital society. In a way, it’s a continuation and expansion of The Search – the database of intentions has expanded from search to nearly every corner of our world – we now live our lives leveraged over digital platforms and data. So what might that look like thirty years hence?

As the announcement last year stated:

WHAT WE HATH WROUGHT will give us a forecast of the interconnected world in 2040, then work backwards to explain how the personal, economic, political, and technological strands of this human narrative have evolved from the pivotal moment in which we find ourselves now.

That’s a pretty tall order. At first, I spent a lot of time trying to boil any number of oceans – figuring out who to talk to in politics, energy, healthcare, technology, and, well, just about every major field. It quickly became quite evident that I’d end up with a book a thousand miles wide and one inch deep – unless I got very lucky and stumbled upon a perfect narrative actor that tied it all up into one neat story. Last time Google provided me that actor, but given I’m writing a book about how the world might look in 30 years, I’m not holding my breath waiting for another perfect protagonist to step out a time machine somewhere.

But what if those protagonists are already here? Allow me to explain…

For the past few months I’ve been stewing on how the hell to actually write this book I’ve promised everyone I would deliver. The manuscript is not actually due till early next year, but still, books take a lot of time. And every day that goes by without a clear framework is a day partially lost.

A couple of months ago, worried that I’d never figure this thing out (but knowing there had to be a way), I invited one of  my favorite authors (and new Marin resident) Steven Johnson over to my house for a brainstorming session. I outlined where I was in my thinking, and posed to him my essential problem: I was trying to do too much, and needed to focus my work on a narrative that paid off the promise, but didn’t read like a textbook, or worse yet, like a piece of futurism. As I said to Steven, “If I write a book that has a scene where an alarm clock wakes you up on a ‘typical morning in 2045,’ please shoot me.”

It’s not that I don’t appreciate futurism – it’s just that I truly believe, as William Gibson famously put it, that the future is already here, it’s just unevenly distributed. If I could just figure out a way to report on that future, to apply the tools of journalism to the story of the future we’re creating, I’d come up with a book worth reading. Of course, it was this approach we took in the early years of Wired magazine. Our job, as my colleague Kevin Kelly put it, was to send writers off in search of where the future was erupting, with instructions to report back.

To find that future, we asked our writers (and editors) to look hard at the present, and find people, places or things that augured what might come next. Hence, issue one of Wired had articles about the future of war, education, entertainment, and sex, based on reporting done in the here and now. While we didn’t call it such, over the years we developed an “If-Then” approach to many of the stories we’d assign. We’d think out loud: “If every school had access to the Internet, then what might change about education?” Or, “If the government had the ability to track everything we do both offline and on, then what might our society look like?” The conditional “If” question followed with a realistic “Then” answer provided a good way to wrap our heads around a sometimes challenging subject  (and for you programmers out there, we’d also consider the “ands” as well as the “elses.”)

Next, we’d ask a reporter to go find out all he or she could about that scenario – to go in search of artifacts from the future which told a story of where things might be going. (Wired, in fact, later created the popular “Found: Artifacts from the Future” series in the pages of the magazine.)

As an early reader and contributor to Wired, Steven knew all this, and reminded me of it as we spoke that day at my house. What if, he asked me, the book was framed as a series of stories about “future antiquities” or “future relics” (I think he first dubbed them “Magic Coins”)? Could we find examples of things currently extant, which, if widely adopted over the next generation, would presage significant changes in the world we’ll be inhabiting? Why, indeed, yes we could. Immediately I thought of five or six, and since that day, many more have come to mind.

Now, I think it bears unpacking this concept of what I mean by “widely adopted.” To me, it means clearing a pretty high hurdle – by 2045 or so, it’d mean that more than a billion people would be regularly interacting with whatever the future antiquity might be.  When you get a very large chunk of the population engaged in a particular behavior, that behavior has the ability to effect real change on our political, social, and cultural norms. Those are the kind of artifacts I’m looking to find.

As a thought experiment, imagine I had given myself this assignment back in the early 1980s, when I was just starting my love affair with this story as a technology reporter (yes, there’s a symmetry here – that’s 30 years ago – one generation past). Had I gone off in search of digital artifacts that presaged the future, ones that I believed might be adopted by a billion or more people, I certainly would have started with the personal computer, which at that point was counted in the high hundreds of thousands in the US. And I also would have picked the Internet, which was being used, at that point, by only thousands of people. I’d have described the power of these two artifacts in the present day, imagined how technological and social change might develop as more and more people used them, and spoken to the early adopters, entrepreneurs, and thinkers of the day about what would happen if a billion or more people were using them on a regular basis.

An antiquity from the 1980s, with its future descendant (image from machinelake.com)

Pushing the hypothetical a bit further, I imagine I’d find the Dan Bricklins, Vint Cerfs, Ray Ozzies, and Bill Gates of the day, and noticed that they hung out in universities, for the most part. I’d have noticed that they used their computers and online networks to communicate with each other, to share information, to search and discover things, and to create communities of interest. It was in those universities where the future was erupting 30 years ago, and had I been paying close attention, it’s plausible I might have declared email, search, and social networks – or at least “communities on the Internet” – as artifacts of our digital future. And of course, I’d have noticed the new gadget just released called the mobile phone, and probably declared that important as well. If more than a billion people had a mobile phone by 2012, I’d have wondered, then what might our world look like?

I’m pretty sure I’d have gotten a lot wrong, but the essential framework – a way to think about finding and declaring the erupting future – seems a worthy endeavor. So I’ve decided to focus my work on doing just that. It helps that it combines two of my favorite approaches to thinking – anthropology and journalism. In essence, I’m going on a dig for future antiquities.

So what might some of today’s artifacts from the future be? I don’t pretend to have an exhaustive list, but I do have a good start. And while the “If-Then” framework could work for all sorts of artifacts, I’m looking for those that “ladder up” to significant societal change. To that end, I’ve begun exploring innovations in energy, finance, health, transportation, communications, commerce – not surprisingly, all subjects to which we have devoted impressive stone buildings in our capital city. (Hence my trip to DC last week.)

Here’s one example that might bring the concept home: The Fitbit. At present, there are about half a million of them in the world, as far as I can tell (I’m meeting with the company soon). But Fitbit-like devices are on the rise – Nike launched its FuelBand earlier this year, for example. And while the first generation of these devices may only appeal to early adopters, with trends in miniaturization, processing power, and data platforms, it’s not hard to imagine a time when billions of us are quantifying our movement, caloric intake and output, sleep patterns, and more, then sharing that data across wide cohorts so as to draw upon the benefits of pattern-recognizing algorithms to help us make better choices about our behavior.

If that were to happen, what then might be the impact on our healthcare systems? Our agricultural practices and policies? Our insurance industries? Our life expectancies? I’m not entirely sure, but it’d sure be fun to try to answer such questions.

I won’t tip my hand as to my entire current list of Future Antiquities, but I certainly would welcome your ideas and input as to what they might be. I’d also like your input on the actual title of the book. “What We Hath Wrought” is a cool title, but perhaps it’s a bit….too heady. Some might even call it overwrought. What if I called the book “If-Then”? I’m thinking about doing just that. Let me know in comments, and as always, thanks for reading.

Apple Gets Into (App) Search

By - February 26, 2012

It took longer than I thought it would, but it’s finally happened. Apple’s admitted that it needs real search to bring it’s tangled app universe to heel, and purchased Chomp, a leading third-party app review and search service.

Nearly two years ago I wrote this piece:  Apple Won’t Build a (Web) Search Engine. From it:

…but it will build the equivalent of an app search engine. It’s crazy not to. In fact, it has to. It already has app discovery via the iTunes store, but it’s terrible, with no signal that gives reliable results based on accrued intent.

What Apple needs is a search engine that “crawls” apps, app content, and app usage data, then surfaces recommendations as well as content . To do this, mobile apps will need to make their content available for Apple to crawl. And why wouldn’t you if you’re Yelp, for example? Or Facebook, for that matter? An index of apps+social signal+app content would be quite compelling.

What Apple will NOT do is crawl the entire web.

I still think that last sentence will remain true. It’s my hope that Apple continues to develop Chomp and help “real search” – the kind I describe above, to happen in iOS.