Have had a very productive couple of days recently on the book, talking at length with various folks who in one way or another have very unique views on the search world. Before I get to Tim Koogle, who I spoke to this morning, or Shana Fisher and Geoff Yang (yesterday afternoon), I wanted to talk about my visit with Danny Hillis.
On Tuesday I flew down to LA to visit with Danny, who founded Thinking Machines. After that he became an imagineer at Disney for five or so years (“The best ‘real job’ you can have,” he quipped). Danny has a million great ideas and is something of a polymath. He recently founded Applied Minds as a way to put that skill to work (he partnered with Bran Ferren, himself a scary smart polymath).
Danny has a lot of things to say about search, it’s an area he finds rich in implications, in particular as it relates to some of the long-term projects he’s involved in, such as the Clock of the Long Now. We spent some time riffing on the future of search, and its current limitations, but … I get ahead of myself. What I really thought was incredible was the playground Danny and Bran have created for themselves at Applied Minds.
You pull up to Applied Minds unimpressed. It’s in an industrial area of Glendale (who knew there even were industrial areas of Glendale?) – windowless one-story warehouses with nameplates like “Airfoil Distribution, Ltd” or “Light Plumbing Fixture Manufacturing, Inc.” Once inside the non-descript edifice, you’re greeted by a low-ceilinged version of an internet start up – the requisite espresso maker, late-modern furniture, flat-screen displays, etc. But really, nothing worth writing home about. In fact, the place felt a bit cramped and claustrophobic.
That all changed once Danny came out to meet me. After chit chatting for a few minutes, he took me to a small room – no wider than my outstretched arms – at the far end of which stood one of those classic red English phone booths. We stepped inside – a bit cramped – and Danny lifted the receiver and dictated a passphrase of some sort. Presto – the rear wall of the booth opened, and we stepped into – nerdvana.
From a cramped phone booth into massive pure-white-lit space two-stories high, adorned with all manner of things strange and beautiful. Over to one side stood the Terminator-like skeleton of a forty-foot dinosaur, it’s 15-foot pneumatic legs gleaming and exposed. Nearly blending into the walls, itself painted movie-set white, was a tricked out Hummer-like RV refitted as a communications/command center – complete with built-in kitchen and bedroom. The space was a great big project lab, with happy geeks combing over various assemblages of wiring, motors, processors and plans like ants on a summer picnic. It’s Willy Wonka’s chocolate factory for geeks.
Applied Minds works this way: Bram and Danny and any number of partners contract with Very Large Companies or Organizations to think outside the box and come up with solutions to problems they might have. The dinosaur, for example, was a solution to Disney’s problem of overlong lines for its rides (solution: make the non-ride portions of the park more interesting by having dinosaurs roaming the streets…). Danny and Bram have, in essence, created a lab where they get paid to think orthogonal to a problem, and invent/design/prototype just about any kind of solution they can dream up. I toured at least four massive warehouses full of projects (and they have more buildings up in SF), many of which I am bound to not report upon, but all followed this basic ethic: let’s imagine a new way to approach what otherwise is an intractable/frustrating/unglamorous business problem. Clients include GM, Herman Miller, and many others, including some defense contractors. The company employs a studio model, with only 50 full time staffers, but hundreds involved at any given time on dozens of projects.
So one can imagine when Danny and I did sit down to talk about search, we’d have an interesting conversation. Besides the fact that his designs for Thinking Machines are now de facto standards for platforms like Google, we ranged from his idea of Aristotle, a Primer like AI tutor, to creating an economy of ideas through a new kind of search infrastructure. It’s fun to live in the future for a while, after so much reporting in the past and present.
For the details of our talk, well, the book is coming along slowly but surely…
That’s fascinating. Does anyone remember the title sequence of the old TV show, “Get Smart”, in which Don Adams enters the secret headquarters of CONTROL through the phone booth at the end of a corridor?
I actually bought and read Hillis’s book on the Connection Machine, but I can’t really see any resemblance to Google. Can you elaborate?
Hmmm. I wonder how this compares to, say, Interval Research? (http://www.wired.com/wired/archive/7.12/interval.html)
Google runs what I suspect is a data-parallel architecture – thousands of nodes crawling and searching largely independently of one another. Thinking Machines used a data-parallel architecture in its CM-2 design.
Thinking Machines also made one of the first distributed search engines, WAIS. The index was distributed across multiple processors.
Full disclosure: I’m an Applied Minds employee and was affiliated with Thinking Machines in the mid-90s.
–Pat / zippy@cs.brandeis.edu
I was thinking of the original CM, which was very fine-grained parallelism, with a hypercube routing topology between nodes. Much more systolic than the usual today. The model at that time was Navier-Stokes, or finite-element analysis, with a fair amount of cross-traffic between pieces.
Google is more of a shared-nothing, massively parallel processor – hack the data up into big chunks, put them on separate nodes, and ship the query to every node. Ethernet is the bus that holds it all together, and the limiting factor – bandwidth per MIP is fairly low, etc.
IMHO this is about the bottom of the scale for parallelism – more distributed processing.
Yes, the CM-2 did have a hypercube network. One could program the CM-2 using a data-parallel model, which is what the *Lisp (starlisp) programming language used. In this model, the CM-2 was treated as a SIMD (single instruction, multiple data) computer. Data was spread across the processors or, more accurately, the memory associated with each processor, and then a single instruction was run by all the processors on its local data.
Now, not all computation can be done this way. FFTs require lots of comparisons between data, which is best done by processor to processor communication, so there were operators to do local data comparisons and moves, along with system wide computations like “find the maximum value of this variable among all the processors.”
Search can be cast as a data parallel process with a scatter operation at the beginning (send the query to all processors) and a gather operation at the end (collect the results from all the processors and sort from best to worst).
–Pat / zippy@cs.brandeis.edu
Actually the results can be sorted from best to worst on each node and the top few merged into the top 10 list by the requester, to save a lot of data shipping.
That’s assuming that no holistic inter-result calculations like HITS are being done. Localrank is Google’s version of that idea; I think they do the calculation with only highly-ranked pages to save overhead.
(I’ve been trying to infer this process recently, so bear with my detail-stickling.)
Kendall – What I meant to infer was that the idea of massively parallel processing – ie having tens of thousands of CPUs working at once – is now defacto.
There’s another interesting thread along these lines. Brewster Kahle was one of the designer/architects of the CM-2, and at the end of that project started spending most of his time on search capabilities. The output of that work was WAIS (early net-heads may remember it), and led to his work at Alexa (sold to Amazon) and now the Internet Archive.
Not Bram – it’s Bran