A Morning with Brewster

I spent much of yesterday morning talking with Brewster Kahle, of WAIS, Alexa, and Internet Archive fame. Brewster is a very fun mind, and he’s working on about ten Really Interesting Things at once. First, it’s easy to forget how important the Internet Archive’s work truly is. The public sphere is diminishing as more and more data (in particular log data) becomes owned by corporations, and the archive is one of the few institutions, outside of our often scleortic library system, dedicated to preserving our digital record on a massive scale. (Good quote: “The original purpose of libraries was preservation and access” to society’s information, but they’ve somehow become about “selection and categorization.”) He’s archiving 20 global television channels as well (see the Television Archive for more.) He’s setting up a broadband distributed wireless LAN across San Francisco, and is still cranking out books via the Internet Bookmobile.
Brewster showed me the Archive’s new “recall” search features, which have been worked up by Anna Patterson of Stanford. Now this is some cool stuff. It searches over 11 billion documents – nearly 4 times that of Google (and they’ve indexed about 1/3 of what they have). Check it out, and play with the various knobs and graphs. It points to some interesting new concepts in search.