Pretty much don’t need to say any more. Link.
Pretty much don’t need to say any more. Link.
This is very interesting news, but not unexpected if you’ve been paying attention. Note in the past I’ve predicted that Apple will not do web search, but will do “app search,” because app search is essentially broken, if you can even call it search to begin with. It’s more like directory navigation at this point.
What Apple needs is a search engine that “crawls” apps, app content, and app usage data, then surfaces recommendations as well as content . To do this, mobile apps will need to make their content available for Apple to crawl. And why wouldn’t you if you’re Yelp, for example? Or Facebook, for that matter? An index of apps+social signal+app content would be quite compelling. What Apple will NOT do is crawl the entire web.
Look at the valuable information that you can extract from how any one of us interacts with a well-designed application, then create a dataset for that. Say I use the New York Transit application to navigate my way through New York for 3 or 4 days… all of the questions and back-and-forth that I use that app for, which is essentially a structured search session—right? Now, match that against a set of data which is the transit map. I say, “I need to go over here. I want to go over there. I prefer this route over that route,”—that becomes a dataset that should inform other searches that I’m making on things that seemingly are unrelated but may not be. That should be available as metadata for future searches. And figuring how to inform that is as important as parsing the line or the spoken phrase that I’m making in the moment.
Now, if I take that spoken phrase and go and search for “Chicago rental car” four months after interacting with that New York Transit map application, how can we take the metadata from that interaction with New York and inform the appropriate response in Chicago. Perhaps the best suggestions would be, “Hey, you know what? You don’t need to rent a car. You can use the Chicago Transit. Here’s an app for it. You can get from the airport to everywhere you want to go without having to rent a car. Plus, you’ll save $150 which we know is a goal of yours because you’ve been interacting with the Mint application and it said that a goal of yours is that you want to save $200 a month and here’s a way that you do that”?
Tying all that together, that’s the Holy Grail because then it starts to understand you. If you only parse just the query, even if you get the natural language right and the intent right, you’re missing the whole person.
It’s now clear to me that Apple is very serious about being the Google of the post-HTML, app-driven Internet. But so is Google, so is Microsoft, and there are certainly going to be other players to boot. (Er…Like HP, which just bought Palm and plans on “doubling down” on the Web OS.) Game on.
TC broke the news today that Tynt, a search interception and user behavior data company, got a big round of funding from Panorama Capital, which is also an investor in FM. I’ve installed the Tynt service on Searchblog and I’d like to get your response. I think what the service does is quite clever and useful both to publishers and users. However, it does create new user experience for those of us who cut and paste on sites, and I’m interested if folks find the new approach worthy.
The service works like this: when you copy a snippet of text from a site with Tynt, you’ll see that Tynt appends a unique URL into the pasted text (for example, see the graphic below where I’ve copied and pasted a snippet from a Searchblog post into an email).
This URL both redirects readers back to the location from which the snippet was pasted, as well as notifies the Tynt service of the actions taken. This gives Tynt a database of user behavior – a signal of intention – that could become quite valuable. At scale, this means Tynt can, for example, build a Digg-like view of the web – without ever having to create a Digg. It all works based on behavior most readers do all the time anyway.
This data is also surfaced to publishers, which can help them improve their editorial and user experience, among other things.
Tynt also has a pop up service (see graphic below from TC – I have not implemented this yet and until recently the company did not disclose this service publicly) that identifies when certain cut and pasted text is likely to be a signal of search intent. This is based on examining the string of words that is copied. Short phrases – of a few words, for example – usually means the reader is doing a search – they are cut and pasting the text into a search bar or another search browser window.
Think about that behavior – probably something you do a lot (I certainly do). What happens? Well, you are reading a story, and come across a term or word you don’t understand, or want to research more. You highlight it, go to the search bar (or open another tab with Google in it), copy the text, paste it into Google, and find yourself on another page (the Google search page.)
Who wins in this scenario? Well, usually Google does (or whichever search engine is used). They get the search, and the probable revenue from that search (as we know, many folks click on paid search links!).
Who loses? Well, the publisher, because some number of folks who execute this behavior will leave the publisher’s page and never return. And the publisher never sees that search revenue either, even though it was the publisher which sparked the search intention in the first place. One could argue that the user loses as well, because they often run off into a back and forth search game that distracts them from their initial focus on the article they were reading.
Tynt changes this game, in that it both keeps the reader on the page, and intercepts the search behavior (and potential revenue). This I find quite interesting (as does Google, I am sure, and Bing, which I bet would love to power those Tynt searches which otherwise might go to Google…). For its major partners, Tynt splits revenues with publishers, bypassing the search engines. The company already has deals in place with scores of major publishers representing billions of page view a month. It claims to be doing 100 million searches. That makes it a player, one the major engines will have to deal with.
One to watch.
Over at SEL Gord Hotchkiss has published an interview with me on the future of search. From it:
We’re going through a shift in how folks are understanding what search really means to them. And what it means to them is “I have a need and I need it fulfilled, and I’m going to use the online medium to fulfill it in some way.” We had a very, very basic, well-understood use case for 10 years, which was Google or “like Google”—you put in a couple keywords and you get a response back. And that framework of searching and coming back with the best document to answer a query is morphing. People are asking far more complicated questions now and they’re demanding far more nuanced answers, simply because they know they’re out there….
…Search as an application where your first search isn’t the search itself but rather the search for the right application is a very, very different use case. You have the market influence and dominance of one player splintered into tens of thousands of players. You or I sitting in our office over the weekend could come up with the absolute best structured search application for determining who should be your arborist to cut your trees. And that’s a threat to Google Local Search. If the best application to determine a plumber is the plumbing app on an iPhone—you download it and it automatically pulls all the local results from Yahoo!, Bing, and Google, then pulls all the reviews from Yelp and Angie’s List, then cross-compares that with complaints filed with the Better Business Bureau and Diamond Certified—if that’s the app you use, where’s Google in all of that, right?
The folks at Aardvark have posted an ambitious paper over on the ‘vark blog. Titled after Brin and Page’s original “Anatomy of a Large-Scale Hypertextual Web Search Engine”, the paper presents the Aardvark engine and, in its authors’ words: “describes the fundamental differences between the traditional “Library” paradigm of web search — in which answers are found in existing online content — and the new “Village” paradigm of social search — in which answers arise in conversation with the people in your network.”
I have read most of the paper, which has been accepted at WWW 2010 (it reminded me of all the search papers I read in preparation for writing The Search), and found a lot worthy of interest.
First, the paper’s authors, both of whom have worked at Google, clearly have a sense of potential history here, in that they not only crib Google’s original paper’s title, they also mirror the first line (substituting “Aardvark” for “Google”, of course). Now that’s some b*lls. Of course, when Larry and Sergey first presented Google, they couldn’t even get their paper accepted (it took three tries, if I recall correctly. Someone should write a book about that…).
Second, it’s unusual for a Valley startup to lay out its architecture and technological specs as willingly as Aardvark has. There’s a lot of math in here that I couldn’t parse even if I had the will to try.
Third, we learn some cool things about how Aardvark works. Check this quote out: “…unlike quality scores like PageRank , Aardvark’s quality score aims to measure intimacy rather than authority. And unlike the relevance scores in corpus-based search
engines, Aardvark’s relevance score aims to measure a user’s potential to answer a query, rather than a document’s existing capability to answer a query.”
Also interesting: ” this involves modeling a user as a content- generator, with probabilities indicating the likelihood she will likely respond to questions about given topics. Each topic in a user profile has an associated score, depending upon the confidence appropriate to the source of the topic. In addition, Aardvark learns over time which topics not to send a user questions about…”
There’s a lot more like this in the paper, it’s worth reading. The authors even did a test of Aardvark results against Google, with the results being something of a push (see the last page for details). Not bad for an upstart service.
Lastly, we learn a lot about the service, thanks to a number of charts, including something about Aardvark’s growth, which I had not really anticipated. It’s up and to the right, as you can see from the chart.
Forget the iPad, today Google is taking another step toward its stated goal of “making search more social.” There’s a lot of goodness in here, in terms of features and approach, but it’s just silly to pretend you can do any of this without directly addressing the 400 million-person elephant in the room called Facebook. Put simply: I can’t figure out if this new service uses my Facebook social graph. And to my mind, that’s a problem.
From the blog post announcing the public beta of social search (first announced at Web 2 late last year):
We think there’s tremendous potential for social information to improve search, and we’re just beginning to scratch the surface. We’re leaving a “beta” label on social results because we know there’s a lot more we can do. If you want to get the most out of Social Search right away, get started by creating a Google profile, where you can add links to your other public online social services.
Indeed – a lot more, like make it really easy to use your Facebook social graph, the way tons of other sites and apps do. Why not just use Facebook Connect? Hang on a tick, the video giving us an overview of the service says once you create that Google Profile, you can add connections via Blogger, Twitter, and “any other online networks you might be a part of” (45 seconds in). Might that include Facebook?
OK dear readers, I’m going to do it. I’m gonna make a Google Profile, just to find out…. Well, I’m still a bit perplexed. You can add any URL as a “Link” in your profile, so I added my Facebook pages. However, once I got through the initial form (which was not simple – I had to fill out all the info I already did with Facebook and LinkedIn, and my own name is not available as a profile URL, not /johnbattelle, not jbattelle. Darn! I picked /johnlinwoodbattelle, so now you all know my middle name…) Er, anyway, there *was* a prompt to “Share It On Facebook” after all that…
Aha! Maybe this will get my Facebook social graph goodness into Google Social Search?
Not that I could tell. Just a simply “share on Facebook” implementation, declaring my profile to my FB pals. But no deep integration. As far as I can tell, my Facebook social graph will not inform my social searchin’ on Google. As I understand it from reading previous coverage of the product, Google social search *will* leverage FriendFeed, recently purchased by Facebook. But as far as I can tell, it does not leverage Facebook proper.
And that, to my mind, is just silly. Silly in the main, because as a consumer, clear, direct, and transparent integration with Facebook would be a huge *win* for my understanding of Google’s social searching. Wouldn’t it? Or am I missing something? (Besides the competitive issues, of course…)
I’ve pinged Google and other sources to find out if I’m just deeply in the dark….
Update: Google has provided me an answer to my initial question:
“If someone links to his Facebook account from his Google profile, Social Search may surface that user’s public profile page. These are the same public profile pages already available on a search of Google.com and other search engines today. While we’re interested to continue expanding the comprehensiveness of Social Search, we do not currently use your Facebook connections as part of Google Social Search.”
What I’d like to know then is this: Why not?
I’ve said before that search interfaces, stuck in the command line interface of DOS, will at some point evolve into applications on top of a commodity search index. I further opined that Bing, in particular Bing’s limited but compelling visual search, was just such an example: search as an interactive, rich application, as opposed to search as a list of results.
The commodity of search results is critical, but as we shift our usage to the mobile web, the use case for a list of results weakens. Instead, as this Bizweek article points out, we’re using apps. On their face, these apps don’t seem like search at all. Except they are.
Take the popular iPhone app Exit Strategy, for example (at left). The app helps folks navigate the NY transit system. In essence, it consolidates a subset of search queries and answers them with a combination of domain-specific structured results and an elegant user interface. The structured dataset is the NY transit map and schedule, the UI is based on the iPhone’s unique ecosystem of interface. The result: No one with this app is Googling “best route Bronx Midtown“. Instead, there’s an app for that.
Google can’t help but see this as a threat. For nearly every structured set of results, there’ll be an app for that, if there isn’t already. To my mind, the question becomes one of using search to find the best apps. I wonder how Google is surfacing iPhone apps as answers to questions pertinent to destroying its own query volume? For it seems to me that a very good result for the query above, if done on Google over an iPhone, would be “Exit Strategy.”
Huh. Yet another reason to lean into Android, no doubt.
In my predictions this week I seemed unusually glum about the state of search, writing: Traditional search results will deteriorate to the point that folks begin to question search’s validity as a service.
This statement did not go unnoticed by folks in the industry, and I received quite a few emails, Tweets, and comments asking what on earth I meant. Well, in the post I tried to explain:
This does not mean people will stop using search – habits do not die that quickly and search will continue to have significant utility. But we are in the midst of a significant transition in search – as I’ve recently written, we are asking far more complicated questions of search, ones that search is simply not set up to answer. This incongruence is not really fair to blame on search, but so it goes. Add to this the problem of an entire ecosystem set up to game AdWords, and the table is set.
Let me use this final BingTweets entry to expand on what I meant.
My statement about how we’re asking “far more complicated questions of search” is a riff on the writings I’ve done here on the BingTweets blog, specifically, my three part series on “Decisions Are Never Easy” (1, 2, 3). In short, I find that all of us are expecting search, a technology built to answer one-dimensional questions like “capital of Yemen”, to answer questions that have more than one semantic meaning (“Yemen al qaeda leadership diplomacy”). As a reader (and search entrepreneur) put it in an email to me: “When people move to complex queries (defined as two or more semantically disjunct terms), search breaks down. All it is really fit to do is deliver all the permutations. Imagine a 5-term query, all semantically disjunct. …. such as … “green tea, life quality, life expectancy, cancer, tumor”. Did you ever try and read 40,000 documents?”
Well no, none of us ever try to read all the documents search brings back – all the “permutations” that search faithfully (and rather unintelligently) renders to us. We all know by now that when we ask a complicated question of search, search will pretty much throw everything and the kitchen sink at us. And we don’t want all that information. We want our answer!
I have no doubt that such an answer is coming, but before it does, we have to go through a period of disappointment. ……. (continued …)
A new decade. I like the sound of that. I’m a bit late on these, but for some reason these predictions refused to be rushed. I haven’t had the contemplative time I usually get over the holidays, and I need a fair amount of that before I can really get my head around attempting something as presumptive as forecasting a year.
So I’ll just start writing and see what comes.
While past predictions have focused on specific companies and industry segments (like Internet marketing), I think I’ll try to stay meta this time. Except for Google, of course, which is still the only company in the Internet economy that can be seen from space. For now. But we’ll get to that.
1. 2010 will mark the beginning of the end of US dominance of the web. I am not predicting the decline of the US Internet market, but rather its eclipse in size and overall influence by other centers of web economies. In essence, this is not an Internet prediction, but an economic one, as the web is simply a reflection of the world, and the world is clearly moving away from a US-dominated model.
2. Google will make a corporate decision to become seen as a software brand rather than as “just a search engine.” I see this as a massive cultural shift that will cause significant rifts inside the company, but I also see it as inevitable. Google, once the “pencil” of the Internet, has become a newer, more open version of Microsoft, and it has to admit as much both to itself as well as to its public, or it will start to lose credibility with all its constituents. While the company flirted with the title of “media company” I think “software company” fits it better, and allows it to focus and to lean into its most significant projects, all of which are software-driven: Chrome OS, Android, Search, and Docs (Office/Cloud Apps).
This shift means Google will, by years end and with fits and starts, begin to minimize its efforts in media, including social media, seeking to embrace and partner rather than compete directly. This is a significant prediction, as Facebook is clearly Google’s most direct competitor in many areas, but Google will realize, if it has not already, that it cannot out Facebook Facebook, but it sure can be a better software company.
3. 2010 will see a major privacy brouhaha, not unlike the AOL search debacle but around social and/or advertising related data. Despite the rise of personalized privacy dashboards for most major sites, there is still no industry standard for how marketing data is leveraged, and there is a brewing war for that data between marketers, their agencies, and third parties like ad networks and measurement companies. Add in a querulous legislative environment, and it’s hard to imagine there not being some kind of major flap in the coming year.
4. By year’s end the web will have seen a significant new development in user interface design, one that will have gained rapid adoption amongst many “tier one” sites, in particularly those which cover the industry.
Despite nearly ten years of blogging, most publishing sites are still stuck in the mode of “post and push down,” which is, frankly, a terrible UI for anyone other than news hounds. Thanks to the three-headed force of social, gaming, and mobile, I think the PC web is due for a UI overhaul, and we’ll see new approaches to navigation and presentation evolve into a recognizable new standard.
5. (image) Apple’s “iTablet” will disappoint. Sorry Apple fanboys, but the use case is missing, even if the thing is gorgeous and kicks ass for so many other reasons. Until the computing UI includes culturally integrated voice recognition and a new approach to browsing (see #4), the “iTablet” is just Newton 2.0. Of course, the Newton was just the iPhone, ten years early and without the phone bit….and the Mac was just Windows, ten years before Windows really took hold, and Next was just ….oh never mind.
6. 2010 will see the rise of an open gaming platform, much as 2009 was the year of an open phone platform (Android). Imagine what might happen when the hegemony of current game development is questioned – I want open development for Halo and Guitar Hero, damnit!
7. Traditional search results will deteriorate to the point that folks begin to question search’s validity as a service. This does not mean people will stop using search – habits do not die that quickly and search will continue to have significant utility. But we are in the midst of a significant transition in search – as I’ve recently written, we are asking far more complicated questions of search, ones that search is simply not set up to answer. This incongruence is not really fair to blame on search, but so it goes. Add to this the problem of an entire ecosystem set up to game AdWords, and the table is set. Google will take most of the brand blame, but also do the most to address the issue in 2010.
8. Bing will move to a strong but distant second in search, eclipsing Yahoo in share. Of course, with the Yahoo deal, it’s rather hard to understand search share, but I measure it by “where search queries originate.” This is a pretty bold prediction, given the nearly 7-point spread between Bing and Yahoo now, but I think Microsoft will pick up significant share using cash to buy distribution.
9. Internet advertising will see a sharp increase, and not just from increased search and social media platform (PPC/PPA) spending. Brands will spend a lot more online in 2010, and most predictive models are not accounting for this rise.
10. (Image) This is probably a layup, but one never knows, layups are sometimes the ones you miss: The tech/Internet industry will see a surge in quality IPOs. However, at least one, if not more will be withdrawn as public scrutiny proves too costly and/or controversial. A corollary: There will also be a surge in M&A and “weak” IPO filings.
11. I’m out of my depth on this one, but it feels right so I’m going to go with it: We’ll see a major step forward in breaking the man/machine barrier. By this I mean the integration of technology and biology – yes, the same fantasy that fuels the blockbuster movies (Avatar, Matrix, Terminator). I’m not predicting a market product, but rather a paper or lab result that shows extraordinary promise.
12. I’ll figure out what I want to do with my book. SOGOTP, so to speak. Three years of predicting that I’ll start it is getting a bit old, eh? I feel good about branching back out into more contemplative fields, with FM in a strong position and our economy coming out from its defensive crouch.
As always, thanks for reading and responding. I look forward to 2010, it’d be hard to predict anything other than it’ll be a better year, overall, than 2009.
(This piece was written for the BingTweets blog and is part of an ongoing exploration of search underwritten by Microsoft. See my series on the interplay of search and decisions here, here, and here. I wrote the piece below before today’s web-wide conversation about content farms, but I think it’s related. We need new frameworks for search, and real time points us toward one potential path.)
The rise of real time search (just this past week, Google rolled Twitter, Facebook and Myspace data into its results) has everyone buzzing. Of course, BingTweets was the first real time mashup from a major player in search (and Microsoft has already announced its intentions to go further), but we’re just at the start of where real time search might go. What might things look like a few years from now?
In my last BingTweets post (Decisions Are Never Easy) I posited the idea of a real time service that connects us to each other based on expertise. So if I wanted to talk with someone who was an expert in buying classic cars, the service would find that expert and connect me to him or her.
I think real time search is a step toward building an ecosystem that makes such a service possible. But we have to get out of our current modes of understanding search interfaces to really grok how this might work. At present, we still see search as a modal dialog box, where we type in a request, then wait for an answer. As different search interfaces develop, new opportunities arise. We’ve seen a fair amount of innovation in search interfaces lately (here’s more on Pivot, for example), but real time data presents a significant challenge.
We can see the challenge in the companies most directly responsible for feeding data into the real time search index. Twitter recently changed its opening question from “What are you doing?” to “What’s happening?” That subtle shift invited a much more robust set of potential responses to be poured into the service (and subsequently parsed by search services). And Facebook just this week announced it will make all of its members’ status updates part of its universally public feed. Its question? “What’s on your mind?”
I recently heard from a reliable source inside Facebook that there are 40 times more status updates daily on Facebook’s network than on Twitter. That’s a lot of data to parse, whether you are a search service, or a consumer of that service’s product. What might it look like?
Well, start with the use case. Why might we want to query a real time search index? My first answer is simply this: To find out “what’s up.” Now, there are nearly endless refinements of that general concept: What’s up with the smoke I can see in the mountains behind my house? What do people who bought the Palm Pre recently think of their new phone? What bands are playing in Chicago this weekend that I might like? What’s up with Jahvid Best, will he play in Cal’s bowl game? All of these questions are variations on the theme of “What’s up?”
Given the right approach to interface, algorithms and filters, all of these queries can be answered by real time search.…
(more at BingTweets….)