Sub-required overview of the issues inherent to the deep web, including access to government files and the implications for mediators. Tim Bray is quoted (always a good sign). I missed the author’s call, though I’m not sure I’d have added much…Excerpts:
As new search spiders penetrate the thickets of corporate databases, government documents and scholarly research databanks, they will not only help users retrieve better search results but also siphon transactions away from the organizations that traditionally mediate access to that data. As organizations commingle more of their data with the deep Web search engines, they are entering into a complex bargain, one they may not fully understand.
…The CIA and Dick Cheney notwithstanding, there is no secret government conspiracy to hide public documents from view; it’s largely a matter of bureaucratic inertia. Federal information technology organizations may not solve that problem anytime soon. The deep Web search engines may just solve it for them….
Deriving search results from structured data sets will open up new possibilities for search engines. In all likelihood, search engines will gradually abandon the flat listings-style result pattern you see on a typical 12-page Google result. (And who ever gets to the 12th page, anyway?) Not only could deep Web search engines present more useful and manipulable views into structured data but, given some basic lingua franca of structural vocabularies, they could also aggregate those results in endlessly permutable combinations…
…Every search query is a unit of desire. Search companies, like all businesses, exist by transforming desire into hard currency. As deep Web search engines insinuate themselves into deeper and deeper levels of organizations, they will not only offload search traffic, they will trigger a series of massive disruptions in the information economy.
If you buy the Cluetrain maxim that “hyperlinks subvert hierarchy,” then surely deep Web search engines will amplify that subversion. As search engines extend their reach deeper into and across organizations, the boundaries between those organizations will feel more fluid — both to consumers and to the organizations themselves. The first thing most of us notice may be better search results.
Somewhere inside that complex apparatus of desire and fulfillment, a transformation is taking place, one whose effects we can barely foresee.