Comments on: A Morning With Danny Hillis http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php?utm_source=rss&utm_medium=rss&utm_campaign=a_morning_with_danny_hillis Thoughts on the intersection of search, media, technology, and more. Thu, 23 May 2013 12:55:00 +0000 hourly 1 http://wordpress.org/?v=3.4.1 By: oopser http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-26063 oopser Tue, 03 Jan 2012 02:01:00 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-26063 Not Bram - it's Bran Not Bram – it’s Bran

]]> By: Dave Douglas http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23872 Dave Douglas Mon, 03 Jan 2005 03:32:57 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23872 <p>There's another interesting thread along these lines. Brewster Kahle was one of the designer/architects of the CM-2, and at the end of that project started spending most of his time on search capabilities. The output of that work was WAIS (early net-heads may remember it), and led to his work at Alexa (sold to Amazon) and now the Internet Archive. </p> There’s another interesting thread along these lines. Brewster Kahle was one of the designer/architects of the CM-2, and at the end of that project started spending most of his time on search capabilities. The output of that work was WAIS (early net-heads may remember it), and led to his work at Alexa (sold to Amazon) and now the Internet Archive.

]]> By: John Battelle http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23871 John Battelle Sun, 20 Jun 2004 03:10:25 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23871 <p>Kendall - What I meant to infer was that the idea of massively parallel processing - ie having tens of thousands of CPUs working at once - is now defacto. </p> Kendall – What I meant to infer was that the idea of massively parallel processing – ie having tens of thousands of CPUs working at once – is now defacto.

]]> By: Kendall Willets http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23870 Kendall Willets Sat, 19 Jun 2004 21:59:15 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23870 <p>Actually the results can be sorted from best to worst on each node and the top few merged into the top 10 list by the requester, to save a lot of data shipping. </p> <p>That's assuming that no holistic inter-result calculations like HITS are being done. Localrank is Google's version of that idea; I think they do the calculation with only highly-ranked pages to save overhead.</p> <p>(I've been trying to infer this process recently, so bear with my detail-stickling.)</p> Actually the results can be sorted from best to worst on each node and the top few merged into the top 10 list by the requester, to save a lot of data shipping.

That’s assuming that no holistic inter-result calculations like HITS are being done. Localrank is Google’s version of that idea; I think they do the calculation with only highly-ranked pages to save overhead.

(I’ve been trying to infer this process recently, so bear with my detail-stickling.)

]]> By: Patrick Tufts http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23869 Patrick Tufts Sat, 19 Jun 2004 20:10:39 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23869 <p>Yes, the CM-2 did have a hypercube network. One could program the CM-2 using a data-parallel model, which is what the *Lisp (starlisp) programming language used. In this model, the CM-2 was treated as a SIMD (single instruction, multiple data) computer. Data was spread across the processors or, more accurately, the memory associated with each processor, and then a single instruction was run by all the processors on its local data.</p> <p>Now, not all computation can be done this way. FFTs require lots of comparisons between data, which is best done by processor to processor communication, so there were operators to do local data comparisons and moves, along with system wide computations like "find the maximum value of this variable among all the processors."</p> <p>Search can be cast as a data parallel process with a scatter operation at the beginning (send the query to all processors) and a gather operation at the end (collect the results from all the processors and sort from best to worst).</p> <p>--Pat / zippy@cs.brandeis.edu</p> Yes, the CM-2 did have a hypercube network. One could program the CM-2 using a data-parallel model, which is what the *Lisp (starlisp) programming language used. In this model, the CM-2 was treated as a SIMD (single instruction, multiple data) computer. Data was spread across the processors or, more accurately, the memory associated with each processor, and then a single instruction was run by all the processors on its local data.

Now, not all computation can be done this way. FFTs require lots of comparisons between data, which is best done by processor to processor communication, so there were operators to do local data comparisons and moves, along with system wide computations like “find the maximum value of this variable among all the processors.”

Search can be cast as a data parallel process with a scatter operation at the beginning (send the query to all processors) and a gather operation at the end (collect the results from all the processors and sort from best to worst).

–Pat / zippy@cs.brandeis.edu

]]> By: Kendall Willets http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23868 Kendall Willets Sat, 19 Jun 2004 19:33:05 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23868 <p>I was thinking of the original CM, which was very fine-grained parallelism, with a hypercube routing topology between nodes. Much more systolic than the usual today. The model at that time was Navier-Stokes, or finite-element analysis, with a fair amount of cross-traffic between pieces. </p> <p>Google is more of a shared-nothing, massively parallel processor - hack the data up into big chunks, put them on separate nodes, and ship the query to every node. Ethernet is the bus that holds it all together, and the limiting factor - bandwidth per MIP is fairly low, etc.</p> <p>IMHO this is about the bottom of the scale for parallelism - more distributed processing.</p> I was thinking of the original CM, which was very fine-grained parallelism, with a hypercube routing topology between nodes. Much more systolic than the usual today. The model at that time was Navier-Stokes, or finite-element analysis, with a fair amount of cross-traffic between pieces.

Google is more of a shared-nothing, massively parallel processor – hack the data up into big chunks, put them on separate nodes, and ship the query to every node. Ethernet is the bus that holds it all together, and the limiting factor – bandwidth per MIP is fairly low, etc.

IMHO this is about the bottom of the scale for parallelism – more distributed processing.

]]> By: Patrick Tufts http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23867 Patrick Tufts Sat, 19 Jun 2004 17:07:02 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23867 <p>Google runs what I suspect is a data-parallel architecture - thousands of nodes crawling and searching largely independently of one another. Thinking Machines used a data-parallel architecture in its CM-2 design.</p> <p>Thinking Machines also made one of the first distributed search engines, WAIS. The index was distributed across multiple processors.</p> <p>Full disclosure: I'm an Applied Minds employee and was affiliated with Thinking Machines in the mid-90s.</p> <p>--Pat / zippy@cs.brandeis.edu</p> Google runs what I suspect is a data-parallel architecture – thousands of nodes crawling and searching largely independently of one another. Thinking Machines used a data-parallel architecture in its CM-2 design.

Thinking Machines also made one of the first distributed search engines, WAIS. The index was distributed across multiple processors.

Full disclosure: I’m an Applied Minds employee and was affiliated with Thinking Machines in the mid-90s.

–Pat / zippy@cs.brandeis.edu

]]> By: phil jones http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23866 phil jones Sat, 19 Jun 2004 15:40:12 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23866 <p>Hmmm. I wonder how this compares to, say, Interval Research? (http://www.wired.com/wired/archive/7.12/interval.html)</p> Hmmm. I wonder how this compares to, say, Interval Research? (http://www.wired.com/wired/archive/7.12/interval.html)

]]> By: Kendall Willets http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23865 Kendall Willets Fri, 18 Jun 2004 23:51:23 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23865 <p> I actually bought and read Hillis's book on the Connection Machine, but I can't really see any resemblance to Google. Can you elaborate?<br /> </p> I actually bought and read Hillis’s book on the Connection Machine, but I can’t really see any resemblance to Google. Can you elaborate?

]]> By: Mike http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23864 Mike Fri, 18 Jun 2004 21:29:19 +0000 http://battellemedia.com/archives/2004/06/a_morning_with_danny_hillis.php#comment-23864 <p>That's fascinating. Does anyone remember the title sequence of the old TV show, "Get Smart", in which Don Adams enters the secret headquarters of CONTROL through the phone booth at the end of a corridor? </p> That’s fascinating. Does anyone remember the title sequence of the old TV show, “Get Smart”, in which Don Adams enters the secret headquarters of CONTROL through the phone booth at the end of a corridor?

]]>