free html hit counter Book Related Archives | Page 6 of 29 | John Battelle's Search Blog

Liquid Markets Win: On Boats, Planes and Automobiles

By - July 30, 2013

Marthas_1775_Blask-wbOne of the key themes in our upcoming book has to do with the interaction of information and the physical world – in particular, how all things physical become “liquid” when activated by just the right information. But when you’re writing (and thinking out loud) about this topic, it’s easy to fall into an academic cadence, because information theory is a thicket – just try reading “The Information” in one sitting, for example.

So I’ve found it’s best to just tell stories instead (and to be honest, I’d wager that nearly all information theory should be reduced to narrative, because narrative is how we as humans make sense of information, but I digress). Here’s a story that happened just this past weekend.

If you’ve been reading for more than a year, you know that I spend a good part of August working on an island off the coast of Massachusetts. It’s a special place where my great grandmother settled in the 19th century, the kind of place where you visit graveyards with your kids to remind them of their own history, then hit a carousel and ice cream shop in the afternoon.

Anyway, this year my time on the island is unfortunately brief, what with my coming back to FM and various other entanglements. So every day “on island” is a precious one. Last week my son and I went on an East Coast college tour, driving from Washington DC (Georgetown, American) through Pennsylvania (Bucknell, etc) to Boston (Northeastern, BU, etc). After touring MIT on Friday, we decided to come to the Island a day early, on Sat. That way we could open up the rental house, get the car, and prepare for the rest of our family – my wife and two daughters –  who were flying in on Sunday.

On Sunday my son and I were settled into our rental, eagerly awaiting Mom and sisters’ arrival. They landed in Boston just fine, but hit a major hitch with their connection on a tiny airline called Cape Air. Now Cape Air’s largest plane seats about 8 people, as they specialize in one thing – hopping from Boston down to the Islands. Apparently some cross winds and rain fouled up the routes, and long story short, my family’s flight was delayed for two hours, then cancelled altogether (oh, and they lost my wife’s luggage too). By then, it was almost 9pm, and too late to drive the 80 miles down from Boston to the ferry in Falmouth, which is the only other way to get to the island. (The last ferry leaves at 9.45 pm).

We were all distraught – there they were, just 80 miles away, but with no way to get to us. We were going to lose one of our precious days on the island, and it just stunk.

Then a fellow stranded passenger mentioned a possible solution to my wife: There was a service in Falmouth that aggregated private commercial boats for use as water taxis. Maybe they’d be able to help?

My wife mentioned this to me, and sure enough, a quick Google search found them. At 8.30 pm I called the service and “Captain Jim” hooked me up with another lobster boat captain ready to take my family across the sound, even late at night. Awesome!

But we still needed to get the girls that 80 miles down to Falmouth, and it was past “business hours.” I’ve used a lot of car services in my business life, and I know that they are not exactly very flexible – you have to make a reservation well in advance, and they cost a lot of money for long drives. I wasn’t expecting much, but I started calling as many as I could find, asking if they had any cars near Boston’s Logan airport *right now*.

The car services I called acted exactly as one might expect. Two put me on interminable hold while the “checked to see if they had a car near the airport.” A third flat out refused to try. A fourth asked me to put the girls into a taxi and send them back into downtown Boston, where they might have a car. And so on.

That’s when I tweeted this out:

 

And this is when the story starts to get interesting, in terms of liquid, information-driven markets interacting with the physical world. It turns out, Uber *does* have the ability for someone to drop a pin remotely, I just didn’t know it – I didn’t have *the information* I needed. Twitter solved that in an instant, as one of my followers quickly clued me in about how to do it. In two minutes I was called by an Uber driver at the airport, who was ready to whisk my family down to the waiting lobster boat. An hour and a half later, I picked them up on a private dock near the house. What the private boat service and Uber did was take inactive, physical objects, in this case a Lincoln towncar and a fishing boat, and turn them into kinetic, liquid, real-time addressable assets. And the main reason this was possible? Information cycling through a digital foundation of cell phone towers, Google, and apps like Uber.

What’s even more interesting about this story are the economics: The cost to fly three people via Cape Air to the Island was nearly 25% *more* than taking Uber and the chartered water taxi. Add to this the fact that the Uber driver was far more friendly and eager to please than your typical car service guy, and the water taxi was both fun and nearly twice as fast as the ferry.

In short, having the a liquid, information-driven market of cars and boats created a cheaper, faster, and way more enjoyable experience for my family. That’s a great thing, to my mind, and it makes me optimistic about the coming liquid economy. But if I operated Cape Air or Carey Limousine, I’d be more than nervous right about now….

 

  • Content Marquee

An Afternoon at the Media Lab: Where The Lines Between Physical and Digital Are Permeable

By - July 29, 2013

IcanHasHOloYesterday I took my son to the MIT Media Lab, hallowed ground for me, as reading Stewart Brand’s 1988 “The Media Lab” propelled me toward helping to create Wired magazine, where I edited the founding Director of the Lab, Nicholas Negroponte, for five years (he wrote the back column of the magazine).

For this visit, I met up with David Kong, one of the lab’s alumni wizards, who took us on a whirlwind tour of the place (David’s work on microfluidics is, I believe, some of the most important stuff being done today, but more on that in another post). I spent a day there last summer with Director Joi Ito, and it’s amazing to see how much progress can be made in a year.

Instead of describing everything, I think I’ll let video do the work – one of the Lab’s core values is to always be demo’ing, and my son and I saw half a dozen incredible projects, all demo’d by the people who created them.

First up was the Opera of the Future group, which evolved out of the HyperInstruments lab. Akito Van Troyer gave us a tour of the components used in the recently staged “Death and the Powers” piece, which was a finalist for the Pulitzer in music.  Here’s some video of that performance:

Akito also turned us onto a very cool beat machine he built as a side project, a hack based on actuators and tempo that turns anything into a percussion instrument. Here’s a video of that I found on YouTube:

After that we went to see Xiao Xiao, who works in the Lab’s Tangible Media group (this is the part of the Lab most directly connected to the themes of the upcoming book). She showed us the MirrorFugue, which is just amazing, in particular, to sit down at the keyboard as it’s playing. It’s magical, which is pretty much the goal of the entire Lab. Here’s a video of that:

MirrorFugue III from Xiao Xiao on Vimeo.

You can probably sense a theme by now – all this work is about blurring the lines between physical and digital, atoms and bits. An extraordinary world is soon to be settled by pioneers in this space, and we’re all of us fascinated by it – it’s why we love the idea (if not necessarily the look) of Google Glass, or 3D printing (I met the co-founder of FormLabs while at the Lab), or cool gadgets like the NFC ring.

The Media Lab is a place where folks are actively creating the future. Over and over, I heard this refrain: “I took some off the shelf parts, hacked them together, and wrote some code.” Simple, right?

One example: Makey Makey, which went viral earlier this year with the “banana piano.” The idea is bigger than turning fruit into keyboards, however. It’s about making nearly anything physical a portal into the digital world, and bringing the digital right back into the physical. I met with Eric Rosenbaum, one of the creators, in his lab, which is called “Lifelong Kindergarten” (yeah, I know.) Here’s a short video about Makey Makey:

As the border between physical and digital gets more permeable, a new kind of literacy emerges. And that literacy is built on a foundation of code – whether it’s the codes of letters and words, or the code of bits and algorithms. Rosenbaum showed me Scratch, a graphical programming language used by hundreds of thousands of kids across the world. I’m determined to learn how to code, at least enough to be dangerous (I took classes in Pascal about 30 years ago…). Maybe Scratch is where I’ll start.

Next up I met Dan Novy, from the Lab’s Object-Based Media Group. He showed us a number of great projects he’s working on, including holo-presence (with a sense of humor, see photo at top) and new forms of augmented experience. Check out this video about redefining the home entertainment experience:

Dan also took us into a small room with a voice aware projection device in the center. Using his voice, Dan told a children’s story, and the four walls of the room lit up with visual images related to the storybook. It’s early days, but we discussed what might happen when this device is miniaturized and connected to consumer “narrative catchers” like Facebook, Path, Google Glass, and the like. Also next to the projector was an object – what it is, it doesn’t matter, but for this example it was a bottle of perfume – and when you pick up that object, “memories” related to that object are projected onto the walls. So imagine what might happen when you pick up that ornament from Christmas three years ago and hang it on the tree, and images from that Christmas past flash onto your home’s walls….

The last demo we saw was perhaps the most well known of Dan’s group’s work. In essence, they turned a basketball net into a data collection device, so as to measure the force of a slam dunk. The technology is amazing, watch Dan talk about it here:

The Media Lab is truly an extraordinary place, and seeing it with my son made it even more magical. I’ve toured it a few times now, but I’ll never tire of coming back. The work happening there is helping to define the world all our kids will be living in soon.

The PRISMner’s Dilemma

By - July 16, 2013

prismSometimes when you aren’t sure what you have to say about something, you should just start talking about it. That’s how I feel about the evolving PRISM story – it’s so damn big, I don’t feel like I’ve quite gotten my head around it. Then again, I realize I’ve been thinking about this stuff for more than two decades – I assigned and edited a story about massive government data overreach in the first issue of Wired, for God’s sake, and we’re having our 20th anniversary party this Saturday. Shit howdy, back then I felt like I was pissing into the wind –  was I just a 27-year-old conspiracy theorist?

Um, no. We were just a bit ahead of ourselves at Wired back in the day.  Now, it feels like we’re in the middle of a hurricane. Just today I spoke to a senior executive at a Very Large Internet Company who complained about spending way too much time dealing with PRISM. Microsoft just posted a missive which said, in essence, “We think this sucks and we sure wish the US government would get its shit together.” I can only imagine the war rooms at Facebook, Amazon, Google, Twitter, and other major Internet companies – PRISM is putting them directly at odds with the very currency of their business: Consumer trust.

And I’m fucking thrilled about this all. Because finally, the core issue of data rights is coming to the fore of societal conversation. Here’s what I wrote about the issue back in 2005, in The Search:

The fact is, massive storehouses of personally identifiable information now exist. But our culture has yet to truly grasp the implications of all that information, much less protect itself from potential misuse….

Do you trust the companies you interact with to never read your mail, or never to examine your clickstream without your permission? More to the point, do you trust them to never turn that information over to someone else who might want it—for example, the government? If your answer is yes (and certainly, given the trade-offs of not using the service at all, it’s a reasonable answer), you owe it to yourself to at least read up on the USA PATRIOT Act, a federal law enacted in the wake of the 9/11 tragedy.

I then go into the details of PATRIOT, which has only strengthened since 2005, and conclude:

One might argue that while the PATRIOT Act is scary, in times of war citizens must always be willing to balance civil liberties with national security. Most of us might be willing to agree to such a framework in a presearch world, but the implications of such broad government authority are chilling given the world in which we now live—a world where our every digital track, once lost in the blowing dust of a presearch world, can now be tagged, recorded, and held in the amber of a perpetual index.

So here we are, having the conversation at long last. I plan to start posting about it more, in particular now that my co-author Sara M. Watson is about to graduate from Oxford and join the Berkman Center at Harvard (damn, I keep good company.).

I’ve got so many posts brewing in me about all of this. But I wanted to end this one with another longish excerpt from my last book, one I think encapsulates the issues major Internet platforms are facing now that programs like PRISM have become the focal point of a contentious global conversation.

In early 2005, I sat down with Sergey Brin and asked what he thinks of the PATRIOT Act, and whether Google has a stance on its implications. His response: “I have not read the PATRIOT Act.” I explain the various issues at hand, and Brin listens carefully. “I think some of these concerns are overstated,” he begins. “There has never been an incident that I am aware of where any search company, or Google for that matter, has somehow divulged information about a searcher.” I remind him that had there been such a case, he would be legally required to answer in just this way. That stops him for a moment, as he realizes that his very answer, which I believe was in earnest, could be taken as evasive. If Google had indeed been required to give information over to the government, certainly he would not be able to tell either the suspect or an inquiring journalist. He then continues. “At the very least, [the government] ought to give you a sense of the nature of the request,” he said. “But I don’t view this as a realistic issue, personally. If it became a problem, we could change our policy on it.”

It’s Officially Now A Problem, Sergey. But it turns out, it’s not so easy to just change policy.

I can’t wait to watch this unfold. It’s about time we leaned in, so to speak.

Hasta La Alta Vista, Baby.

By - June 29, 2013

I just saw the news that Yahoo! is “sunsetting” Alta Vista, one of the first “good” search engines. This makes me a little misty, as Alta Vista was the search engine I used BG – Before Google – and it had a real shot at *being* Google, had its various owners not utterly screwed it up over the years. Did you know, for example, that at one point Alta Vista was the largest and most widely used search tool on the web? Its driving force, Lois Monier, once told me “search should be a pencil” – he was adamant that Alta Vista not become a portal.

But Alta Vista was owned by DEC, a now dead computer company, which was bought by Compaq, another now dead computer company. And they made it a portal. And through the now defunct Overture, the assets of Alta Vista made their way to Yahoo!, a still alive portal. But now, Alta Vista is going to truly be dead.

It’s hard to watch an important player in the early Internet go away – and it makes me reflect on a couple of things. First, how much or our own culture and history we’re losing day by day, even despite the best efforts of archivists like Brewster Kahle. And secondly, on a personal note, Alta Vista was the search engine that helped me find my birth mother way back in 1995, when my wife was pregnant with our first child, and my life-long wonderings as an adoptee took on a new urgency. Alta Vista pointed me to an online forum for people like me, and there I found a person who helped me find my mother. Pretty cool.

So I’ll miss you, Alta Vista. And if any of you want to know the engine’s journey, well, there’s a book for that.

The Book Lives On

By - June 09, 2013

Faithful readers will recall that about three months ago, I announced my return to FM as CEO. I also mentioned that the projects I’d been working on – notably OpenCo and The Book, would have to be retooled given my new commitment to the company I started back in 2005 (when I last wrote a book). In the post, I wrote:

I love the book I’m working on, and I don’t plan to abandon it (I’m bringing on a co-author). And I love the conferences I do, and I’ll still be doing them (though I’ll be hiring someone to run them full time). But my first love is the company I started in 2005, whose story is not only unfinished, it’s at the height of its running narrative.

I’m very, very pleased to announce that I’ve found that co-author – her name is Sara M. Watson, and she’s simply the perfect partner for me to be working with on this book. You can read her post about it here. Sara and I met over Twitter, after she noticed the theme of the CM Summit – “Bridging Data and Humanity.” We spoke on the phone and I learned that the intersection of society and data was her passion – and that her background was an awful lot like mine. She started her career as a liberal arts major from Harvard (during the time Facebook was just a dorm room project), toiled in the narrative fields of enterprise IT, became fascinated with the story of information, and decided to head to graduate school to study it (she’ll finish her Masters from Oxford in a few months). After Oxford, Sara has some amazing plans lined up (I can’t talk about them yet) that dovetail perfectly into our shared work.

I started my career as a liberal arts major from Berkeley, wrote about enterprise IT for a few years, then followed my passion for the digital narrative into graduate school as well (also at Berkeley, the Oxford of the West, or perhaps, the Harvard – sorry Stanford!). My first project out of grad school was Wired magazine. Sara’s is going to be our book. I’m honored to be working with her. Last week in London I got to meet her for the first time and spend some quality time together.

The past 12 weeks have been a whirlwind, as I’ve gotten my arms around Federated, executed four conferences in New York, Cincinnati, and London, and lucked into finding great partners for the projects I’m passionate about. Not only have I found the perfect collaborator in Sara, I’ve also found a CEO to run OpenCo, which recently had an amazing London pilot and a successful debut in New York as well. But more about him later. For this post, I want to welcome Sara to the Searchblog community, and I expect the our partnership will result in a lot more writing coming through this channel in the near future.

Here’s a video of me talking about the themes of the book, and announcing Sara as well, at Le Web last week.

Onwards!

 

Behind the Banner, A Visualization of the Adtech Ecosystem

By - May 13, 2013

I’m very proud to announce “Behind the Banner“, a visualization I’ve been producing with Jer Thorp and his team from The Office for Creative Research, underwritten by Adobe as part of the upcoming CM Summit next week. You can read more about it in this release, but the real story of this project starts with my own quest to understand the world of programmatic trading of advertising inventory – a world that at times feels rather like a hot mess, and at other times, like the future of not only all media, but all data-driven experiences we’ll have as a society, period.

I’m a fan of Terry Kawaja and his Lumascapes – Terry was an advisory to us as we iterated this project. But I’ve always been a bit mystified by those diagrams – you have to be pretty well steeped in the world of adtech to grok how all those companies work together. My goal with Behind the Banner was to demystify the 200 or so milliseconds driving each ad impression – to break down the steps, identify the players, make it a living thing. I think this first crack goes a long way toward doing that – like every producer, I’m not entirely satisfied with it, but damn, it’s the best thing I’ve seen out there so far.

I am deeply grateful to all the folks who helped us make this happen, in particular Jared Cook at Adobe, and a legion of leaders in the industry who reviewed early versions, including Walter Knapp, Bill Demas, Ned Brody, Brian O’Kelley, Ann Lewnes, and dozens more who helped me research and imagine what this might end up looking like.

So take a look and tell me what you think. It’s far too complex to embed here, so we have it running over on the CM Summit site. If nothing else, it should get folks talking, and I hope you’ll help us make it better by leaving a comment here, or sending me mail with your thoughts.

Oh, and while you are at the site, check out the conference lineup. We are almost sold out of tickets, and it’s going to be one heckuva conversation, so please join us!

The iWatch: What I Hope Apple Actually Does (But Probably Won’t)

By - February 21, 2013

(image AppleInsider) Back in April of last year, I pondered Pebble, the then-wildly successful darling of Kickstarter fame. Pebble is a wristwatch device that connects to iPhones and displays various smart things. In the piece, Does the Pebble Cause a Ripple In Apple’s Waters?, I asked whether Apple would allow such third-party hardware to play in their backyard. It struck me Apple’s entire business was about hardware. Pebble, I figured, was in for a tough road. No wonder it went to Kickstarter, I mused. VCs would never back something so clearly in Apple’s target zone. From the post:

If you watch the video explaining Pebble, it become pretty clear that the watch is, in essence, a new form factor for the iPhone. It’s smaller, it’s more use-case defined, but that’s what it is: A smaller mirror of your iPhone, strapped to you wrist. Pebble uses bluetooth connectivity to access the iPhone’s native capabilities, and then displays data, apps, and services on its high-resolution e-paper screen. It even has its own “app store” and (upcoming) SDK/API so people can write native apps to the device.

In short, Pebble is an iPhone for your wrist. And Apple doesn’t own it.

If we’ve learned anything about Apple over the years, it’s that Apple is driven by its hardware business. It makes its profits by selling hardware – and it’s built a beautiful closed software ecosystem to insure those hardware sales. Pebble forces an interesting question: Does Apple care about new form factors for hardware? Or is it content to build out just the “core” hardware platform, and allow anyone to innovate in new hardware instances? Would Apple be cool with someone building, say, a larger form factor of the iPhone, perhaps tablet-sized, driven by your iPhone?

Fast forward to now. The month’s Apple rumors have all been about the “iWatch” – the company’s next big innovation.  Apparently reliable sources – most likely now muted thanks to Apple’s exceptional PR machine – have said that 100 people are working on the device inside Apple’s HQ. And this week came news that Apple has even filed for a patent around the concept. 

If I’m Pebble, I’m not sleeping well at night.

I have no idea if Apple will actually create such a device – though I’m certain it must be testing one.

However, if Apple really wants the device to take off, the company should incorporate more than just iPhone connectivity. Here’s my wish list:

- Open platform for connectivity. Any device can connect to the device, not just iOS. I know this is wishful thinking, but…for example, Google has opted for glasses as its next big thing in wearable computing. I certainly would like the two to work together. (And how cool would it be if it worked with Android? OK, sorry. Just had to ask.)

- Sensors and software that make the device the equivalent of the Fitbit or the Jawbone Up.

- Integration with those apps, so that users don’t lose their data if they want to move to Apple’s hardware platform.

- As with Pebble, an open app ecosystem for the device, not one locked down into iOS. (I know…)

- A warranty on breakage. It’s one thing to ignore the criminal cracking that happens with nearly every iPhone in existence, because you can blame the consumer for dropping the damn thing. But if this thing is on somebody’s wrist, it’s going to get smacked around. And if Apple takes the same approach to breakage as it has for the iPhone, the device will be a failure.

That’s my major wish list. What would you want from the device?

It’s Time To Call Out Fraud In The Adtech Ecosystem

By - January 26, 2013

A confusing landscape = ripe opportunities for fraud.

As part of research I’m doing both for the book and for my upcoming conference (the CM Summit, more on that soon), I’ve been in pretty extensive conversations lately with dozens of key players in the advertising technology industry. I find the ecosystem that has developed  to be fascinating, complex, and ripe with opportunity (and deeply important to the future of our society, not just marketing). I’ll be writing about it quite a bit in coming months. But before I do, I wanted to call out a growing issue that our industry will have to tackle sooner rather than later.

Just as in the early, wild west days of search (1999-2004), the programmatic advertising business – a multi-billion dollar marketplace growing faster than search, video, or anything else for that matter – is riddled with fraud.

That’s what many very reputable sources have told me in great length over the past few months. It’s something of an open secret, and more and more people are speaking out against it. Here’s Federated Media’s Walter Knapp on the problem, back in March of last year:

The great thing about the Internet is that it is built on the foundation of openness — from the way the domain system works to the way content and publishing are increasingly democratic. The core technologies embrace openness, sharing, linking and the ability to consume content across devices and across wired or wireless connections. Unfortunately, the openness we depend on in the digital media business is also available to people who can (and will) take advantage of this openness and exploit it for their own selfish wants.

Knapp notes two forms of fraud – ad injectors, fraudulent browser plugins that take over ad calls; and the practice of inserting an entire site into a 1×1 pixel hidden on high traffic but low quality sites featuring porn or music lyrics. Both are examples I’ve heard about over and over in my reporting. A third involves “stacking” ads one behind the other, all playing video to completion, often playing in inactive tabs. A fourth features refreshing ad calls on accelerated schedules or in inactive tabs. Yet another involves running as many ads as possible out of view, simply to gain “view through attribution” on a closed loop success metric.

More people are starting to call these practices out. AppNexus CEO Brian O’Kelly prominently featured the issue of fraud in his blog post celebrating his company’s recent $75 million funding, and what he intends to use it for:

Quality We will continue to invest in cleaning up the advertising marketplace. We’re proud of our anti-piracy stance, and our 5x volume growth this year indicates that you don’t need to serve on BitTorrent sites to be an ad platform company. We are investing heavily in fighting fraud, porn, malvertising, and malicious toolbars, and we are actively working on viewability tools.

Programmatic industry watcher AdExchanger puts it this way:

AppNexus’s pledge to invest money in ad quality issues is worth calling out. The issue is becoming more pervasive as companies emerge to exploit the vulnerabilities of real-time traded inventory to data and impression fraud, malvertising, and other nefarious practices. Fraudulent activities aside, the emergence of robust ad verification and viewability tools means display ad marketplaces and buying platforms must keep a clean nose.

It’s true that many folks are working on addressing the issue, including the IAB. But the bad actors are currently far ahead of the good guys, and worse, many in our industry are turning a blind eye, hoping the problem goes away in time, without too much publicity. Why? Well, nearly everyone gets paid from fraud – the publishers, the exchanges, the data providers, and the agencies. Even the marketers,who are footing the bill, feel like they are getting value – because the success metrics they’ve set up are being  met.

But fraud hurts the ecosystem in a massive way. It means that low quality, invisible, or purely fraudulent inventory is holding down the average value of the entire marketplace – hurting high quality, engaged publishers in the process, stunting investment in quality content.

Over and over, I hear that the reason CPMs (the amount of money a marketer is willing to pay for one thousand advertising impresssions) are so low is because “there’s infinite inventory.”

Hogwash. There’s only so much time in the day, and only so many pages where actual human beings are really paying attention, and the web (including mobile) is growing at a finite pace. There are even fewer places where marketers can be assured of quality, engagement, and appropriate context. It’s time we focus on identifying them, and ridding ourselves of the true source of “infinite inventory” – fraud.

Portrait of Twitter As A Young Media Company

By - January 21, 2013

Last year I predicted that Twitter would become a media company. However, I focused mainly on the new “Discover” functionality, and I probably should have gone a lot further. In this piece, I intend to.

So I’ll start with this: 2013 will be the year Twitter starts to create, curate, and co-create media experiences on top of its platform. I hinted at this in my brief coverage of Twitter’s Oscar Index (see Twitter’s Makin’ Media), but allow me to put a bit more flesh on the bones.

So what might one make from the fact that your platform captures hundreds of millions of individuals declaring what’s going on at any give time? Well, let’s break down some of the signals in all that supposed noise. As I’ve written over and over and over in the past several years, Twitter presents a massive search problem/opportunity. For example, Twitter’s gotten better and better at what’s called “entity extraction” – identifying a person, place, or thing, then associating behaviors and attributes around that thing. This (among other reasons) is why its Discover feature keeps getting better and better. Another important signal is location – Twitter is increasingly focused on getting us to geolocate our tweets. A third signal is the actual person tweeting – his or her influence and interest graph. Yet another signal is time – when was the entity tweeted about?

Real time entity extraction crossed with signals like those described above is the Holy Grail – and I’m guessing Twitter is almost, if not already there.

Once you get good at all these things (and more), a number of really interesting possibilities open up. Identifying “big things” that are going on at any given time is something that Twitter already does – though not particularly well (the best window in is the “Trends” box on the left of the page). Regardless, Twitter has become a go-to service for quick updates about news events (Sandy, Newtown, etc), entertainment events (SuperBowl, Oscars, Grammys, etc), and well….pretty much any kind of event.

But so far, it’s not exactly easy to get the big picture of what’s really going on for any given event on Twitter. In fact, it’s rather difficult. You can search for a hashtag, or keywords you think are associated with an event, but no matter what, it’s extremely difficult to makes sense of it all. For a big event like Sandy Hook or the Oscars, there are literally millions of tweets to sift through. And those tweets have millions of pictures, links, and videos. How can you know what’s important?

This is exactly the problem that  media experiences are designed to solve. By combining intelligent algorithms (these tweets are retweeted more than others, this video is linked to more than all the others, etc) and some smart editors, Twitter can (and most likely will) surface instant windows into events as they unfold around the globe. I imagine logging into Twitter at some point in the future and seeing a dashboard not of Trends, but of “Happenings” – Events edited to my interest graph, location, and the like. When I click on on of those events, I enter a meticulously edited media experience – a pulsing, ever changing feast of information tailored around that event.

So, put in one sentance: Twitter’s going to do events soon.

What other media experiences might Twitter create? Well, extending the logic, it only makes sense that Twitter will curate media services, just as LinkedIn and now Facebook are starting to do (I argue that Graph Search is a media play here).

“Just Landed” – from 2009.

As Google has proven, words have a lot of power on the web. They have even more power when put in context at scale. Consider what happened when a data artist asked a simple question: Where are people when then tweet that they “just landed”?

Now, imagine Twitter stands up a service that allows you to see patterns around phrases like “looking for someone to…,” or “just got a job,” or “python developer,” etc. Yep, lurking inside all that Twitter data is a pretty powerful job service. And I’m only using jobs as a straw man (and because it’s a driving force of LinkedIn’s success, of course). When you have humanity whispering into your ear at scale, you can tune in any number of valuable signals. Getting a job is one important signal. But so is getting married, buying a house or a car, graduating, and, and and….well you get the picture. Standing up “media services” around these life milestones is what media companies do. They used to be called magazines. What might Twitter call them? In 2013, we’ll most likely find out.

So far I’ve proposed two new media features of Twitter: Events and Media Services. I’ll round out this post with a prediction around a third: Video. Video is a vastly under-leveraged asset on Twitter, but people are sharing millions of links to video clips every day on the service. I imagine that Twitter will soon offer some kind of video curation feature – giving its base the ability to find the most popular videos based on pivot points of time, interest, and people. Surfacing and creating more video on the Twitter service has got to be a major priority at the company. And let’s not forget that Twitter bought Vine, after all…

After all, everybody loves video. In particular, advertisers love video. After all, Twitter is already working with Neilsen to become the official barometer of television conversations.

Which brings me to the “stick the landing” portion of this particular round up. Twitter is going to make much more media this year, because Twitter is going to make much more money this year. Each of the features I described above – Events, Media Services, and Video – bring with them inherent business models. I don’t expect they’ll look like traditional display models, of course, but I would not be surprised if they strayed a bit from Twitter’s current Promoted Suite products. With new media products come new advertising products. And new revenue.

Time will tell if I got this one right. Meanwhile, what do you think?

Facebook Is No Longer Flat: On Graph Search

By - January 15, 2013

A sample Graph Search result for the query “friends photos before 1999.”

By now the news is sweeping across the blogosophere and into the mainstream press: Facebook is doing Search!

Well, not so fast. Facebook is not doing search, at least not search Google-style. However, the world’s largest social network has radically re-engineered its native search experience, and the result is not only much, much better, it’s also changed my mind about the company’s long term future.

Yesterday, Tom Stocky, Facebook Product Management Director, and Lars Rasmussen, Engineering Director, gave me a sneak peek of today’s much anticipated announcement (it’s gonna be a phone! A new Newsfeed! A big acquisition!). So as to not bury the lead, Facebook has built what it’s calling “Graph Search,” a solidly conceived structured-search service which leverages the company’s massive trove of personal data in any number of new ways (some obvious, some nuanced, and some glaring omissions). But before I get to the details, I want to write about why this matters so much.

Prior to seeing the new search, I was not certain Facebook would ever live up to the hype it has accrued over its short life. It’s a good service, but it’s flat – over time, it struck me, people would tire of tending to it. They set up their social graph, toss a few sheep, poke some pals (or not), “like” this or that (often off-domain), waste hours on Farmville, and then…engagement drops slowly over time. I’m also not a fan of Facebook’s domain-specific approach to the world, as many of you know. Facebook’s new search doesn’t address Facebook’s walled garden mentality (yet), but it nails the first issue. Once this search product is rolled out to all of its members, Facebook will no longer be flat.

This is a big deal on many fronts. First and foremost, Facebook has an engagement problem, particularly in markets (like the US) where its use has become ubiquitous and many of its original users are two, three or more years into the “Facebook habit.” While the company doesn’t talk about this issue, I am confident it’s real (in private conversations with people at Facebook, it’s called the “set it up and forget it” problem). If people do not constantly feed Facebook with engagement, its value attenuates over time. As the service slows in overall growth, engagement with its current base becomes critical. New connections are the lifeblood of a service like Facebook. Without a steady stream of meaningful Likes, Friend Requests, declared Interests, and such, the platform would wither.

Put another way, Facebook needed a service that layered a fresh blanket of value over its core topography. Graph Search is it.

Zuckerberg’s Engagement

One sign of how important this new search is? According to the folks I spoke to yesterday, Facebook’s mercurial founder and CEO Mark Zuckerberg calls Search the “third pillar” of the company’s service, elevating it to the level of Newsfeed and Timeline, the two most important new features since Facebook’s launch (Open Graph is probably up there as well, but it’s true value remains locked up until there is mortar connecting it all, which Search could well be).

A team of engineers and product folk have been working on Graph Search for more than a year, and Zuckerberg has been engaged with them the entire time. The team has been in “lockdown” – a exclusive state of focus on one product so as to ship it as quickly as possible – for the past 34 days. Lockdown is a time honored and rather prestigious occurence inside Facebook, dating back to Zuckerberg’s original Facemash dorm room programming outburst. During the Search team lockdown, Stocky told me, he and Rasmussen got plenty of 2 AM emails and unexpected late night visits from the CEO.

In other words, this is A Really Big Deal for the company.

Why? Well, a quick tour of the product will explain.

What Is It? 

Graph Search subsumes Facebook’s previous search offering, which was extremely weak and focused mainly on the use case of navigation (finding people and pages).  The new service takes full advantage of the face that Facebook is, at its core, a massive structured database of tagged entities. The initial beta “indexes” four main types of these entities: People, Photos, Places, and Interests. Over time, I am told, Facebook will expand its index to include all Facebook posts and even the Open Graph – which means the “rest of the web.”

But for now, users can search across four main categories, using a slick set of intuitive verbs (“lives,” “like,” “work,” etc.), nouns (“San Francisco,” “Indian,” “restaurants,” “friends” etc.), prepositions (“before,” “with,” “in”) and pronouns (“who,” that,” etc.). This makes for a richly structured set of results: “Friends of friends who live in San Francisco and like Indian restaurants,” for example. Or “Friends who have been to Ireland,” or “Photos of friends before 1990.” Once you get the hang of it, the possible pivot points are endless, and the results are quite intriguing.

Stocky and Rasmussen, both ex-Googlers, walked me through a few intriguing use cases, one of which harkens back to one of Facebook’s original use cases – dating – and another which looks forward and presents a threat to LinkedIn’s current strength: Recruiting.

Let’s say you’re single, and you’re interested only in dating engineers who are also friends of your friends. With Graph Search, it’s ridiculously easy to find “friends of friends” who are also engineers. (And single, of course). You can look at their pictures, profiles, interests, and then ask for an introduction from whichever of your pals happens to be connected to one who looks like a good prospect (you could also just “poke” the guy if you wanted to…). Want only C++ programmers, or Indian C++ programmers, or  Indian C++ programmers under 35 years of age? Done.

Or, let’s say you work at, I dunno, Google. And you want to recruit product management talent from, say, Facebook. Again, the best way to get to that talent is probably a friend. So why not do a search for “friends of friends who work at Facebook and are product managers”? Why not, indeed.

One can imagine such functionality will create a lot of new engagement on the service. And not just from people “friending” prospective beaus or hires. Recall that when Google burst onto the scene, it prompted a dramatic response from owners of web pages, who immediately began rewiring their sites to be optimized for search. Similarly, Facebook’s Graph Search will incent Facebook users to “dress” themselves in better meta-data, so as to be properly represented in all those new structured results. People will start to update their profiles with more dates, photo tags, relationship statuses, and, and, and…you get the picture. No one wants to be left out of a consideration set, after all.

Facebook Gets More Weather

Last year I wrote a post titled “Facebook Is Now Making Its Own Weather.” The focus was on Facebook’s Newsfeed, and how an economy of value was now in place to game Facebook’s “edgerank” algorithm, which determines what stories show up in a person’s feed. With Graph Search, I expect a similar ecosystem will emerge. All of a sudden, two things will be true that previously were not: Facebook users will be using search, a lot, creating liquidity in Facebook “SERPS.” And secondly, there will be significant perceived value in being included in those search results, both for individuals (I want to be considered for that job at Google!) and for companies/brands (I want to message to anyone looking for a job!).

While Graph Search is in very early beta, I don’t think I’m going out on a limb by predicting that it won’t be long before Facebook integrates a product that lets marketers purchase ads in these new search results. It already has a similar product, which is by default included in suggested searches (the “auto completed” queries suggested to a user as they enter terms). At the moment, however, paid listings are not included in search results. They will be. Which means, of course, the rise of a native SEO/SEM ecosystem inside Facebook. Add in Open Graph search across the web, and presto…Google’s got some serious potential competition. (Well, not exactly presto. Incorporating Open Graph is going to take some serious chops and time. But still…).

Even without incorporation of Open Graph or Posts, Graph Search is going to change the game for brands and people on the Facebook service. As I watched Stocky and Rasmussen put their product through its paces, I couldn’t help but wonder how much new traffic the product will drive around the Facebook Platform. Will Facebook be watching “conversions” – clickthroughs from search results to profiles and pages? Of course they will! Will Facebook report those referrals to individuals and brands, much as Google Analytics does for webpage today? Not yet…but wait for it. It’ll come….

 What’s Missing: Sharing Results

I’ve already noted that Graph Search does not index content (posts) or the Open Graph, though I’m told that’s coming. But the big miss, from my point of view, is the inability to share search results.

Share search results? Who’d want to do that? Well, in web search, very few of us. That’s because with rare exception, open web search is not an inherently social action – it’s private and it’s ephemeral. But inside the walls of Facebook, it’s definitionally so. In fact, I’d argue that every single “result page” in Graph Search is a “media object” in its own right. If you search for “pictures of friends before 1990,” for example, you get the equivalent of a Pinterest board of your friends’ childhood shots. Wouldn’t you like to post that on your timeline so your pals can see it? Better yet, wouldn’t you like to export it to Pinterest or Tumblr? Of course you would (but, alas, I don’t expect Facebook will allow it, under cover of “protecting user privacy.” More on that in a second.)

Or take another example. Say you have a pal in Southern California who is despondent after being dumped by her boyfriend. You do a quick Graph Search for “single friends of friends under 30 who work in Los Angeles.” The results look pretty promising. Don’t you want to shoot them over to your pal with the subject line “Don’t despair, there are plenty of fish in the sea!” Of  course you do.

I mean, just a query like “Photos I Like” is a huge feature win for Facebook. And who wouldn’t want to post a montage of “Photos I Like” to their timeline? (Or, ahem, their personal blog?!)

For now, you can’t share the results of your searches with anyone else, and that’s a bug that should be a feature. When I brought up the issue, I was told that the privacy implications of sharing searches were extremely complicated. Because of past missteps and current scrutiny, Facebook is going to tread cautiously here (privacy was a central theme in Graph Search’s launch). I certainly understand why, but while those issues are sorted, I expect there are going to be a lot of screen shots of Graph search results being shared around the web.

Bigger privacy issues will likely arise around what might be called the Randi Zuckerberg principle – as in “Oh shit, I didn’t realize I’d show up in that circumstance!” Graph Search is going to expose all manner of privacy controls as super important, and send millions back to Facebook’s sometimes-confusing dashboards, so as to appropriately re-tool settings such that nothing untwoard shows up in this important new functionality.

And to me, this is a Very Good Thing. A couple of years ago, I wrote a post titled  The Rise of Digital Plumage in which I predicted that we’d all become habituated to “dressing” ourselves in structured data, so as to best present ourselves to the world at any given time. Graph Search is another important tool in our ever-growing digital wardrobe, one that motivates us to understand and manage the implications of our ever-expanding digital footprint.

Facebook just posted an announcement about its new search here.  The initial beta will roll out slowly, folks will have to ask to join a waitlist to get the service. I’ll be updating this post as the news is discussed and digested….