The U.S. government is developing a massive computer system that can collect huge amounts of data and, by linking far-flung information from blogs and e-mail to government records and intelligence reports, search for patterns of terrorist activity…
…The core of this effort is a little-known system called Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement, or ADVISE. Only a few public documents mention it. ADVISE is a research and development program within the Department of Homeland Security, part of its three-year-old Threat and Vulnerability, Testing and Assessment, or TVTA, portfolio. The TVTA received nearly $50 million in federal funding this year….
…What sets ADVISE apart is its scope. It would collect a vast array of corporate and public online information — from financial records to CNN news stories — and cross-reference it against U.S. intelligence and law-enforcement records. The system would then store it as “entities” — linked data about people, places, things, organizations, and events, according to a report summarizing a 2004 DHS conference in Alexandria, Va.
The storage requirements alone are huge — enough to retain information about 1 quadrillion entities, the report estimated. …
..For example: Is a burst of Internet traffic between a few people the plotting of terrorists, or just bloggers arguing? ADVISE algorithms would try to determine that before flagging the data pattern for a human analyst’s review.
…ADVISE “looks very much like TIA,” Tien of the Electronic Frontier Foundation writes in an e-mail. “There’s the same emphasis on broad collection and pattern analysis.”
My take on TIA and all this is both in the book, and here.