Today marks the first of what I hope will become a regular series of Q&A interviews on Searchblog. The format is simple – I send an opening question to the luminary in question via email, they respond, and we go from there. First up is Gary Flake, a veteran of Overture, Yahoo and now Microsoft’s vaunted research labs (he’s founder and director of the new “Live Labs.”) Gary and I have known each other since I first began work on the book, and he’s always had a refreshingly frank outlook. I expected that to be tempered by a year at the world’s largest (and oft-criticized) software company, but I was wrong. If anything, Gary has become more outspoken. I’ve bolded the really juicy bits, but see for yourself….
You’ve been at Microsoft for nearly a year now. Are you satisfied with the pace of development on the core search product?
Broadly speaking, yes, but there is always room for improvement. We’ve been laying a foundation for the past year, trying to solve the hardest problems first. Frankly, we’ve left some low-lying fruit hanging on the vine, so to speak. On the good side, we have a rock-solid 64-bit backend architecture (probably the only one in the industry) that can scale almost indefinitely on multiple dimensions. Our relevance framework, built with machine learning technology from Microsoft Research, is now picking up steam and is getting close to parity with the competition. We even have some real product differentiation that we’ve just launched with virtual earth maps, search macros, and the image search experience. But to be honest, we pretty much blew it with the GUI for the past year. Why? There’s no good reason, really. The truth is we’ve had so much going on over the past year that it was simply more fun to focus on the core issues first, which is a mistake typical of engineers. I think the whole industry has been in a bit of a rut with respect to the user experience, and we are more guilty than most, but the good news is that there is a lot of room for improvement and we are now in pretty good shape to experiment more.
Say more about 64 bit architecture. Why is it different? What does it allow you to do with search you can’t with 32 bits? How easy or hard might it be for Google and others to migrate to a similar architecture? And can you tell us a bit more about Microsoft Research? Lastly, if you could – can you give me a specific example about what you did not like about your old UI?
“64 bits” refers to the native address space of the CPUs that we use in our architecture. Having the entire architecture be 64 bit carries several implications that are quite subtle and mostly of interest only to engineers, but I’ll take a shot at highlighting some of the important differences.
First, all search architectures tier their data stores so that the data can reside on relatively slow disks or fast RAM. For performance purposes, you want the most frequently accessed data to be on the fastest store (RAM). A 64 bit system can have vastly more RAM than a 32 bit system, which means that we can have a higher ratio of RAM to disk space (per CPU), versus others in the industry. Second, we’ve only scratched the surface on the possibilities for using advanced hyperlink analysis for web relevance. There are a class of algorithms (like Pagerank) that work well in either 32 or 64 bit systems (technically, because the algorithms can be realized via a linear sweep of hyperlinks). However, there is an even richer class of algorithms that can only be efficiently built on a 64 bit system because you essentially have to have a significant part of the web stored close to a single CPU. So, 64 bit systems pave the way for entirely new forms of relevance that look at how pages relate to one another. Finally, with the bigger address space, we may also see 64 bit systems critical to realizing more sophisticated relevance algorithms that analyze a result set in the aggregate as a final step. In this scenario, you may have a huge amount of data in memory, which 64 bit systems can do just fine. So, in short, it’s a real big deal and it will be more important as the entire industry matures.
Google and others can certainly start to migrate towards 64 bit. In fact, everyone will have to at some point. However, there is a huge cost to make the jump, in terms of hardware, porting existing code, but especially in revamping algorithms to make the most use of the new resources. We took our medicine early because we could (we didn’t have a legacy system as baggage). This is an example of us taking a longer-term view, in the sense that we increased our startup costs, but are now have a more strategic technology platform.
Microsoft Research is possibly the greatest computer science research institute in the world. MSR was also one of the key reasons why I came to MS. MS basically offered me the role of figuring out how to make MSN and MSR even more tightly connected, and the idea of being able to fully leverage MSR towards web search gave me goose bumps. Internally, we talk about what we are working on as being historic, in the sense that we are defining a computer science research agenda that could possibly influence an entire generation of computer scientists. Without MSR as part of our story, we wouldn’t stand a chance at such lofty goals.
What didn’t I like about our old UI? Where do I begin? There is so much to choose from … The fonts inconsistantly rendored and were too big. The damn search box was too small. It made no sense to have redundant buttons, links, and menus — yet we had some of each. Even the size of the page (in terms of raw HTML and CSS) was entirely too large. Our margins weren’t even consistant from browser to browser. So, to be blunt, it sucked.
But I need to also be clear on a couple of related points: none of us had any illusion that this was a pretty UI. We, including the folks who coded it up, hated it. Our team is small and growing and has always had more engineering talent than anything else. And like I said, there were just so many other problems that were more interesting to solve first. So, in the end, we are guilty of indulging our engineering passions first, which is a very human mistake to have made. This situation is also evidence that the executive team has a lot of trust in us. Of course they disliked the UI as much as us, but in the end they let the team decide the priorities – which was the right call for a variety of reasons.
Now, thankfully, we are in a much better place with the UI. In particular, we just launched a new search experience on live.com. And while it is not perfect, I think it shows more UI experimentation than just about anywhere. In the end that is the important thing: not that we nail the UI or any piece on our first try — but that we have a platform, framework, and philosophy that are all in support of trying new things so that we can progressively improve and learn.
Your take is quite refreshing, to be honest, it doesn’t sound like Microsoft. Do you sense the culture there is shifting? Or are you ever frustrated?
“Yes” on both counts, but as an answer to your second question, this says more about me than it does about Microsoft (I have a grumpy old man in me yearning to be free). I don’t think MS is particularly clear in how it communicates things to the outside world. I also think that MS often makes mistakes by trying — wait for it — too hard. Specifically, instead of doing one thing and nailing it, we’ll sometimes do six slightly duplicative things in parallel — each being slightly below critical mass — which can create confusion inside and outside of MS until we’ve had time to sort things out better. So, yes, I get frustrated at times. So it goes. For me, frustration is a biological signal that means I should try harder, which is a good and helpful thing.
I think your first question, about our company culture, is very important, so please forgive the length of this next part.
Let’s talk about people as a preface to culture, because I want to get the “evil” thing out of the way. I know the reputation that MS has with some communities. I was hacking Linux in the early nineties, I have a four digit Slashdot userid, and I’ve personally written over 100,000 lines of open source code. I say all of this to qualify my opinion. Being new to MS, the biggest surprise for me was the people. I knew they were smart. I knew they were driven. I knew they were competitive. But I had no idea they would also collectively tend towards kindness, openness, self-criticism, passion, righteousness, and even uncertainty. These are great people — from the executive team down to the rank and file — these are simply wonderful people in every way.
As for culture, I think a change has been underway within MS, but it’s not the one that you may be thinking of. This isn’t a change about core values, which have been pretty steady as far as I can tell. I think the change is actually a reaffirmation — an increase in confidence so to speak — of old core values. Let me explain.
People from Redmond often speak of a “platform” while in the valley they speak of an “ecosystem”. Here’s the surprise: both groups are talking about the same thing. To MS, Windows is a platform because it fosters a virtuous cycle in two parts: developers come to the platform because it has the most users; users come to the platform because it has the most software.
In the valley, everyone talks about Web 2.0ish things that all come down to creating something of an ecosystem where people and companies of all sizes can be both producers and consumers. In both cases, we see something of an indirect network effect, where the aggregate value — the whole — is greater than the sum of the parts.
Here’s what’s new: many folks within MS are now realizing that a lot of the stuff that’s real about Web 2.0 — not the fluffy buzzwords but the fundamental and important parts — have actually been part of MS’s core vision and mission for *decades*. Do you believe in the importance of ecosystems? Windows is the biggest one ever, that generates vastly more wealth for third parties than for MS. Do you value the long tail? MS has done more for bringing computing to the masses (both people and small businesses) than any other company. Is the democratization of tools important to you? Desktop publishing and simplified development flourish because of tools like Office and Visual Basic. What about network effects? We’ve created several that have reaped incalculable value to society.
Okay, now I know I am sounding like a corporate drone and I am well aware that for every example above, there are plenty of people in the valley that will bitch about my characterization. When you assume evil, then an ecosystem looks less than well-intentioned. When you assume goodness, then an ecosystem looks like benevolence. The truth is more complex and vastly more interesting.
So look at the facts: MS was pushing Web 2.0 values over a decade ago. It’s the realization that this is the case that’s new for MS. Our vision for where we’ve been, what we know how to do, and where we want to go is now much more crisp and consistent than it has ever been before. We know that helping people to be more successful — seeking win-win dynamics — is a major part of what we do. It’s not new, but clearly connecting the dots between our past and our future is very new and, frankly, downright exciting.
I’m sure all of you have questions for Gary as well. Post them in comments, and maybe we can lure him into the comments thread!
Thanks Gary, really interesting stuff. How soon do you think MS will be at Google’s *current* relevance level? I don’t understand why more effort has not gone into matching Google relevance as Yahoo has now done.
great interviews.
When will MS launch the “better than Google” search engine?
any hints?
1- Microsoft may have launched Web 2.0 with the advent of Outlook Web (XMLHTTP) but it is hard to imagine the real Web 2.0 being offered for Decades …
2- In terms of MSN search – The Algos have to have the appropriate BI (Business Intelligence) – no amount of Architecture (64 Bit) will compensate for inferior Algorithms…
…as strong as the weakest link…
Perhaps, 64 Bit may allow for CONCEPT SEARCH or LSI technology to be realized (with more relevant SERPs than EXCITE’s attempt in the late 90s)
But the bottom line is the FOUNDATION of the Ranking Struture.
Why is Google so relatively relevant – and have been for the past several years????
The interviewee appears to over-emphasize Hi-Tech as the answer – but ONLY a Deep, Harmonious Understanding and Collaboration between the hi-TECH and the “SERPs Intelligence” divisions can produce a more relevant Search Engine.
I wonder what Gary was referring to when he said “Frankly, we’ve left some low-lying fruit hanging on the vine, so to speak.
Do you have any idea?
When Google says “Don’t Be Evil,” this is often in the context of providing unbiased information that doesn’t favor Google or Google’s advertisers and partners.
For example, when you search for a stock symbol like [MSFT], Google provides links to Yahoo, MSN, CNN and others right along with Google Finance.
But when I do the same search on search.msn.com, I only have one choice — MSN.
Gary, do you think Microsoft will ever adopt a fully open approach that provides all results and lets the user decide, rather than limiting choice to Microsoft properties?
John & Gary, thank you for one of the most open and thoughtful interviews I’ve read recently.
Great of you and Gary to do this interview, John.
It is frank. Gary does admit that Microsoft has made several mistakes and says that the MSN Search UI “sucked”.
But, I was going over it trying to extract the key points. I had a hard time doing that.
Other than saying that MSN has failed to deliver a killing blow to Google already, which is true but obvious, it isn’t clear what Gary is saying here.
Is Gary spending time arguing that Microsoft has smart, dedicated, not evil people too? Is that really a point that is in dispute by any serious people?
Is he saying that a 64-bit architecture is a major competitive advantage (questionable) that Google cannot reproduce quickly (even more questionable)?
Is Gary claiming that the major thrust of innovation and new development is at Live.com? Or that the future of MSN Search is Live.com? Given that Live.com is run by a team of just a few people, how can that possibly be true?
In the end, this interview left me crying for more.
I’d very much like to see a second round on this interview where Gary responds in much more detail to questions about what opportunities exist in search, to what extent information from the desktop can be useful to web search, where current attempts at targeting advertising fall down, and whether Microsoft’s search strategy tends toward social search (like Yahoo), personalized search and question answering (like Google), or more powerful tools (like A9 and Clusty).
Please, dive in, feed us, give us something to think about. Gary has a long and impressive career. I desperately want to hear more about what he sees for the future of search.
I am a bit of a Google shareholder, but even I realize that part of that openness theory of theirs is because they realize that they can only expand in so many directions at once.
They have to rely on third parties to fill verticals until they can create a more efficient way of doing it. Plus what better way is there to test user acceptance of their new offering than by leaving links to old established services right next to it?
I mean, didn’t anyone notice that the Clearstation link disappeared with this upgrade?
When you do a stocks: search you can see that Clearstation was one of the offerings of the old service.
http://www.google.com/search?q=stocks%3Agoog
Why is that service suddenly no longer good enough?
As far as Google not allowing their ad systems to promote biased content that is just flat out wrong.
What kind of content is created with the most passion? Typically biased content.
What kind of content causes the most controversy and linkage (per unit effort for creating it)? Typically biased content, which gets heavily linked, and thus ranks well.
What kind of content is easiest to profit from? Biased content that makes it easy to push an idea / product / service as being exceptionally useful.
Google tries to offset this by biasing their search results to be biased toward informational listings and older websites (many of which were created before there were too many flourishing business models on the web), but clearly that is a temporary solution to a problem they helped create by teaching people to blend ads into content, which by default makes the web seem quite a bit more commercial.
Microsoft does not support ecosystems – they support biospheres. When they want you to die, they cut off your oxygen supply.
The aptly named Dr. Flake needs to get real.
So look at the facts: MS was pushing Web 2.0 values over a decade ago.
“Over a decade ago” MS didn’t even have a browser; the cutting-edge MS OS (Win95) didn’t even have a TCP/IP stack; and so on.
Why not address the fact that Microsoft’s browser doesn’t support the W3C standards? Sure, Firefox/Mozilla may not either; but they’re not the ones sitting on a $40Billion warchest, and running “the greatest computer science research institute in the world.“. With all of this research power, you’d think MS would be able to put out a standards-compliant, efficient browser? LOL.
A meaningless puff piece of utter drivel.
The bit about MS’s culture and decades’ old vision was really insightful – sure it’s easy to accuse Gary of drinking the coolaid, but I think there is merit in what he’s saying.
However, in terms of providing insight into what kinds of search problems MS is tackling, I felt the interview fell short. For instance: “In this scenario, you may have a huge amount of data in memory, which 64 bit systems can do just fine. So, in short, it’s a real big deal and it will be more important as the entire industry matures.” Why? How will this benefit me as a consumer? “Relevance” sure, but in what way? It would be great to have more dots connected here.
I’m amused that he thinks their web search UI is in a much better place, because I find the search results UI at live.com to be one of the worst I’ve experienced.
Assuming Google is unbiassed is completely foolish. They give yahoo finance link because they just started google finance. They are cranking down the ranks of advertisers so that if advertisers wants to be visible on the search engine they are forced to buy ads.
Search for “news” on msn-search. You would be surprised to see the first link. On the other hand yahoo and msn have to participate in adwords for the return favor.
This piece gives a lot more insight into the future of MSN search than the Neil Holloway story a few weeks ago. Unfortunately, that doesn’t say much.
Gary says “64 bit systems pave the way for entirely new forms of relevance that look at how pages relate to one another”.
I’ve seen strong evidence that Google is already implementing an algo that looks at the relevancy of pages when determining the weight of a link. In MSN and Yahoo, a link is a link is a link. In Google, a link from a site about gardening won’t significantly help a site about payday loans.
Even if Microsoft has hardware that is superior to Google’s, that’s only part of the search equation (Not to mention, Google is notoriously secretive about their hardware, so it is questionable to assume superior tech in this area).
Until MSN produces a search algorithim that even comes close to Google’s relevancy and lack of spam, I don’t see the point of pumping up expectations. I have no doubt that Microsoft can become a top SE in time, but it seems kind of silly to start making predictions and claiming superiority in certain areas.
I remember watching Gary Flake give a lively and fascinating talk this year at Microsoft Search Champs on the very morning of the announcement of Live Labs. I happened to make a podcast of the talk that morning (the first ever about Live Labs and its vision). Afterwards, when asked Gary if I could post it to the world, he thought maybe 2-3 seconds and said “Go for it!”. I was pleasantly surprised at the openness and rapid thinking/action that it represented.
Anyway, Gary definitely seems to get what’s happening out here on the Web and in the world. It’ll be interesting to see if a small system like Live Labs can truly change such a large system as Microsoft, without becoming one with the mothership in the process. Their incubator approach will be the one to watch I think. If they can make that work, I think we’ll see a lot more interesting things come out of Redmond.
Dion
64 bit eh??? Pretty revolutionary.
Bet google never thought of that.
I don’t know how Gary can claim that Microsoft is the first to support 64-bit, when Linux has had 64-bit support for years (and Solaris for nearly a decade). Maybe Gary would like to ask the HotMail team how simple it was to port everything to Windows… 😉
In the end, none of it matters: all that matters is the relevance of the search results. And in this core metric, MSN is far, far behind Google or Yahoo.
“there is an even richer class of algorithms that can only be efficiently built on a 64 bit system”
Can he give us some examples? Perhaps point us towards some Msn research papers?
I see Gary is hyped on the 64 bit thing. This is just promotion. I think over the next 6 months we take the wait and see approach.
The only two things that 64 bit machines give you that 32 bit don’t is extra ram and you no longer have to synchronize on longs or doubles. The synchronization is rarely a factor, so basically it’s all about RAM. There are a ton of algorithms that require random access to vast amounts of data, and it is about 1000 times faster to access random data in RAM versus on disk. The purpose of an inverted file system is to make it so random access to data is unnecessary, but it means you need to be very efficient with what data you are working with. If you had an algorithm that determined a matched document, and then actually looked up some meta data for the document (like the date it was created, it’s title, really anything), and it required a disk lookup each time then searching would take a very long time. For instance, if you had a single server that produced 10,000 matches on a single search, and you needed to do random disk lookups each time (which take roughly 10ms) it would take a min of 100 seconds to do the search. Obviously that sucks. If you load all of that metadata into ram, those random accesses to data offer little overhead on the search, maybe a few millis total. These are rough numbers, but should give you the idea. What I don’t understand is why they are all psyched about it now. 64 bit has been around for a while. We at Dumbfind have been using 64 bit machines for 2 1/2 years now.
I’m amused that he thinks their web search UI is in a much better place, because I find the search results UI at live.com to be one of the worst I’ve experienced.
I’m amused that he thinks their web search UI is in a much better place, because I find the search results UI at live.com to be one of the worst I’ve experienced.
I think they can make that work, I think we’ll see a lot more interesting things come out of Redmond.
I think they can make that work, I think we’ll see a lot more interesting things come out of Redmond.