The Memex, The Story, and Searchstreams

"There will always be plenty of things to compute in the detailed affairs of millions of people doing complicated things." – V. Bush We love stories. It's how we understand the world. Were I to tell a friend what happened in tonight's Giants game, I wouldn't send him a box…

vannevar“There will always be plenty of things to compute in the detailed affairs of millions of people doing complicated things.”
– V. Bush

We love stories. It’s how we understand the world. Were I to tell a friend what happened in tonight’s Giants game, I wouldn’t send him a box score (though I might refer to one as I was talking to him). I’d say something like “Man, we looked terrible in the first two innings, our rookie pitcher was tight and we had back-to-back errors resulting in a three-run deficit by the second. But then AJ nailed a three-run homer that put us back in the game, and in the 5th we rang up three more (including Barry’s 689th!). It was all Giants from then on, and JT Snow was on fire …!” and so on. A story is our way of taking a journey and making it portable – we can give it to others, and we’re wired to enjoy both hearing a good story, as well as telling it.

I was thinking about this as I was researching the phrase “tempting fate” this afternoon. I was sure there was some Greek mythology behind it, some base case proof that human beings have always struggled with the question of determinism, the Gods, free will, destiny. At the very least, there had to be a good story behind it, and hell, we love good stories. So what did I do? I fired up Google and started poking around. First it was a simple “tempting fate,” but that was far too broad, not what I was looking for (though it was interesting to see a Google News story about the Athens Olympics). I called my mom, she of all knowledge mythological, and she reminded me that Shakespeare often used the Fates in his work. Armed with this new high order bit, I went back and Googled “The Fates” mythology. I was on to something. I started reading up on the three dieties of Fate, taking turns and twists through references both within Google and from links on the pages I found.

But then I realized I was treading familiar ground. I had searched before for the meaning of a phrase, during my Gilgamesh riff a couple of weeks ago. In fact, I recalled finding a great resource for quotations and literary references. But alas, I did not have the searchstream from that last Googleriff, and I had forgotten to Furl that one site I recalled seeing before. I had to start all over again. Damn, I muttered. When was A9 going to get a Mac version of its toolbar! That has the built in ability to capture and record your search history as well as the things you looked at, all in one package.

My previous searchstream was lost, alas, so I started over. Once again I found myself on a great journey, from early 20th century texts on philosophy and religion to scholarly interpretations of The Fates and their role in early Greek tragedy, Homeric epics, Shakespeare…it was great fun. And in the end I came to a much fuller understanding of my original question.

“Battelle!” those of you still reading this far down may well ask. “What the f*ck was your question, anyway?” Sorry about that. It was this: Why on earth launch the bidding process for Google shares on Friday the 13th, on the eve of a major news event shrouded in terrorist threat (the Athens Olympics), on a day the SEC has opened an investigation into you, in a week where dot com plays have abandoned ship and pulled their IPOs? I mean, what’s the harm in waiting till Monday, at least?

summer2004_opening

What’s the rush, I wondered. Why tempt fate?

Well, I think I found my answer, but for that you’ll have to read the book, assuming I’m not tempting fate by noodling around here instead of actually writing it (really, is there a difference?). Suffice to say, I didn’t find my answer in the first ten results of my initial Google search.

But what I really wish for, both to tell the story of my search, and to annotate my book, is the ability to take that searchstream and turn it into an object – a narrative thread of sorts, something I can hold and keep and refer to, a prop to aid in the telling and retelling of how I came to my answer. Tracks in the dust, so to speak, so others can follow and make their own, or follow mine and see (and question!) how I came to my conclusions. Imagine, I thought to myself, if instead of footnotes and citations, I could append searchstreams…

That’s when I remembered As We May Think, Vannevar Bush’s famous essay in The Atlantic. I had read it earlier in my research, and was struck not by the idea of the Memex, which is well understood, but by Bush’s explication of the problem – that knowledge and learning has become so complicated, so layered, so inefficient, that it is near impossible for anyone to be a generalist, in the sense Aristotle was. Bush’s answer to this problem was the Memex, of course, but what I find interesting is the mechanism by which the Memex is made potent – the mechanism for capturing the traces of a researcher’s discovery through the Memex’s corpus, and storing those traces as intelligence so the next researcher can learn from them and build upon them.

Searchstreams, I realized, are the DNA which will build the Memex from the flat soil of search as it’s currently understood. Engines that leverage searchstreams will make link analysis-based search (ie, nearly all of commercial search today) look like something out of the pre-Cambrian era. The first fish with feet are all around us – A9, Furl, del.icio.us. We have yet to build the critical mass of searchstreams by which this next generation engine might be built (nor will it necessarily be built with our tacit consent). But I can sense it coming.

Here’s to evolution….

23 thoughts on “The Memex, The Story, and Searchstreams”

  1. There are people working on the idea of capturing searchstreams. Seruku and Microsoft’s Stuff I’ve Seen project are two examples. Unlike Furl and del.icio.us, Seruku captures all your web browsing automatically and easily without requiring you to manually save a bookmark. A9 does keep your search history, but only for searches you do on A9, and only allows you to reexecute the search, not easily search or browse the pages you found with your searches.

    You’ve written about Seruku and Stuff I’ve Seen before, as have I.

  2. Do Serku and Stuff I’ve seen track sequence? It seems they’re focused on retrieving individual web pages from your history, rather than building a narrative/searchstream.

  3. fate? thats been a big area of my latest research..
    speaking of the fates, do you know they were credited as being the inventors of the alphabet ( greek alphabet at least)
    good start to think about fate is heimarmene, or tyranny of the stars.
    think about choice or chance,
    snakes and ladders
    stella maris
    “time is what death needs to grow people in” w.b.

  4. I will have to try this Seruku mentioned in the comments.

    I’ve started a mylifebits thing of my own and so far it is going well. I .pdf most non visual sort of documents and scan to .jpg most visual docs. i.e. the water bill gets .pdf, the blue ribbon from my 5th grade speech contest gets the .jpg. Still have an awful lot of scanning to do. I use a high speed scanner at work for all the .pdf stuff.

    I’d be interested in what others are doing with their own scanning and mylifebits type of stuff and I’d be really interested if anyone knows what Gordon Bell and Jim Gemmell have been up to and if they are going to talk again anytime soon in the Bay Area.

    So far I’ve got 38,960 documents on just the .pdf and .jpg part. This does not include my audio mp3s or video. Of the 38,960 documents the vast majority (34,642) are .jpgs. These are mostly digital pictures I have taken or scanned but also interesting images I’ve found at a variety of places (Google image searches, art, etc.).

  5. The generalism-no-longer-possible aspect of Bush’s thought is a big chunk, FWIW, of Heinlein’s “Beyond this Horizon” (spring 1942) — and I think it was a major part of what heinlein wanted for himself, before he turned sour. I think Alexei Panshin wrote a rather good essay on this some years ago, but god knows when and where — maybe NYRSF.

  6. I’ve started a mylifebits thing of my own and so far it is going well. I .pdf most non visual sort of documents and scan to .jpg most visual docs. i.e. the water bill gets .pdf, the blue ribbon from my 5th grade speech contest gets the .jpg. Still have an awful lot of scanning to do. I use a high speed scanner at work for all the .pdf stuff.

  7. I’d be interested in what others are doing with their own scanning and mylifebits type of stuff and I’d be really interested if anyone knows what Gordon Bell and Jim Gemmell have been up to and if they are going to talk again anytime soon in the Bay Area.

  8. I’d be interested in what others are doing with their own scanning and mylifebits type of stuff and I’d be really interested if anyone knows what Gordon Bell and Jim Gemmell have been up to and if they are going to talk again anytime soon in the Bay Area.

  9. Any relation to Battelle, the company? Speaking of Vannevar Bush and the Memex. One outcome of his original concepts is the recording now of “brain waves” or thoughts of people in real time. I noticed that he pointed out that the most important thing was the inventors’ time to reflect and think. Not so in the near future mind and thought control world, (also a world of haptics, remote torture potential, and “brain marketing”). As Bill Joy has pointed out and more recently Microsoft’s END to END security blog, increasingly, security will be the issue front and center. Along with that, major revolutionary changes in freedom to choose our thoughts, time, and IP will either be the beginning of forethought and time encoded IP rights or the end of the “so-called” free market economy and capitalism. If we are not the owners of our minds, and the ones choosing the access, who will be? The Memex idea was an augmented one. The current trends are toward mind control. Robin Ore Femtobeam

Leave a Reply

Your email address will not be published. Required fields are marked *