GlobalSpec: Domain Specific Search and the Semantic Web

I better watch out, or I’ll draw fire from Clay Shirky soon. But much of the debate over the semantic web clears my head by quite a distance – I’m more interested in what works and why. I just got off the phone with the GlobalSpec team -Jeffrey Killeen, Chairman & CEO, and John Schneiter, President. It seems to me that GlobalSpec is one of those innovations in search that works – at least for its intended audience – by adding context, organization, and tagging to a limited dataset. Sounds semantic to me.

GlobalSpec is a domain specific (or vertical) search engine. It got its start eight years ago as a classic IT play – take all the catalog-based information about engineering parts – sensors, transducers, etc. – and roll it into a huge, cross-referenced database, which you then distribute over the web. Make money by connecting customers to parts suppliers. Simple.

Over the years GlobalSpec has evolved into a robust community of a million or so engineering types who use it to find and spec parts. That alone is pretty cool (I mean, a million engineers!). But the coolest stuff was just launched: They call it “The Engineering Web” and it’s a domain-specific crawl of the web for engineering information. And not only have they crawled the web (about 100K engineering related sites, so far), they’ve also surfaced invisible web databases not found in mainstream search engines – patent and standards sites, for example, which are walled off by registration and business considerations. Anyone can use the service – it’s not limited to registered users. In essence, GlobalSpec has built a portal that drives traffic and intent through their original database business, in the process building an intelligent island of engineering information that lives in the public sphere. Of course this means they can add AdWord-like functionality, which of course they are working on.

My thought: If only cars.com was this cool.

OK, you don’t usually spend a lot of time comparing accelerometer specifications, so why should you care? Well, GlobalSpec points the way toward the creation of hundreds of powerful vertical search engines, engines which, because they are limited in domain and exclusive by nature, can in fact offer extremely cool tools to find exactly what you want. (This idea is of course not new, nor mine, but still…) They also will create important data mines of user behavior – GlobalSpec has the parametric details of every search ever made against every product in its database – which is a goldmine for companies who are trying to fathom what the market wants. Think about that for a minute…let it sink it. Yup, Battelle’s on his Database of Intentions horse again.

I wish for the day when there’s a GlobalSpec for every imaginable domain, and a meta search engine which intelligently crawls those verticals…but I’m getting ahead of myself.

Here’s why GlobalSpec points to some exciting developments in search. Because of its limited domain, GlobalSpec can use relatively simple keyword-based algorithms to surface lists of ideas or terms related to your search. This allows you to refine your search in ways that simply don’t scale in the Googleverse. These related ideas are inferred from the results of your initial query. For example, if you search on “aerodynamics“, you will get “aircraft , Flight Mechanics, Helicopter Aerodynamics, computational fluid dynamics and Theoretical Aerodynamics” as related searches.

It’s clustering without the crappy results. This stuff really only works when you are living in a gated community of sorts – out on the big bad web, there are simply too many false positives. (I’d also point out that domain specific vertical search engines in more consumer/commercial domains – such as cars.com or Expedia – are further polluted by the commercial interests of the industry they serve. They could learn a lot from the GlobalSpec approach.)

The GlobalSpec guys outlined a useful trio of attributes shared by domain-specific search engines like GlobalSpec. First is organization. This is the basic premise of domain specific engines – through organization comes efficient search. Schneiter calls his engine “parametric” – everything in the index is organized against the standards and parameters of the engineering field, making “parametric search” a reality. Second is context. Domains engines are by definition contextual, but GlobalSpec has a drop down menu next to its search box that allows you to contextualize the search even more, across a bunch of subdomains like Products & Manufacturing, Company Name, Application Notes (engineers care about this), Suppliers, and Standards. And third is access. Vertical search products, by their exclusive nature, can provide domain-specific access to the invisible web, in fact, they can enable commercial transactions that otherwise would be impossible (as GlobalSpec does, see here – at the bottom is the option to purchase an engineering standard from a company with a deep database of standards, a database which cannot be accessed via mainstream search engines). To that end, I’d argue that a fourth attribute of vertical search engines is commerce – these engines enable serious, highly efficient closed loop markets.

Come to think of it, Topix feels a lot like GlobalSpec, but for local news/advertising instead of engineering. And so on…

FWIW, GlobalSpec is backed by Warburg Pincus, and will be profitable this year. The company will not reveal revenues but say they are in a “scale position.” Worth watching….