Paul points it out as a failed dishwasher search. Mike complains about automated content as does RWW. And we all have experienced it: The Google ecosystem is failing more – failing to get us what we think we want. Failing to not frustrate us. Failing at the more complicated queries we are throwing at it. Failing to be the Google that we came to love back when the web was small and Facebook was a way for Harvard geeks to try to get laid.
Now, Google’s ecosystem is ripe for a quick buck – “content farms” that build article pages cheaply to make a quick buck off AdWords. But these articles, at least for a portion of us, don’t really provide the answers we are looking for. (thanks @thejames for the pointers.)
As Paul puts it in bemoaning his fruitless attempt to use Google for a researching a dishwasher purchase:
This is, of course, merely a personal example of the drive-by damage done by keyword-driven content — material created to be consumed like info-krill by Google’s algorithms. Find some popular keywords that lead to traffic and transactions, wrap some anodyne and regularly-changing content around the keywords so Google doesn’t kick you out of search results, and watch the dollars roll in as Google steers you life-support systems connected to wallets, i.e, idiot humans.
Google has become a snake that too readily consumes its own keyword tail. Identify some words that show up in profitable searches — from appliances, to mesothelioma suits, to kayak lessons — churn out content cheaply and regularly, and you’re done. On the web, no-one knows you’re a content-grinder.
The result, however, is awful.
Yes, it often is. But I’m not worried about this. Audiences always route around that which they don’t want, and when something better comes along as a navigational interface, we’ll pick it up, and quick. If Google doesn’t figure this out, someone else will, and the cycle will repeat.
The truth is, we’re asking far more complicated questions of search than we used to, and we’re expecting the same magic we used to get back when the web had magnitudes of order less content. Back in 2002, when we put “dishwashers” into Google, we’d probably find someone’s blog who was talking about his favorite models. Now, we have five hundred or more attempts at gaming the keyword itself, each promising a potential answer, but rarely delivering it – at least not if we have a complicated question in mind. For simple answers, content farms most likely do a fine job. But the truth is, we are not asking many simple questions of search. We’re expecting a lot more.
And in the end, this is a good thing. Our expectations drive innovation, and I can sense a major breakthrough is coming. To my mind, the essential element required for that breakthrough is human in nature. We need a new framework for search, one that allows us to leverage our inherent ability to converse. And from what I can tell, it’s closer than we might think.
2010 is going to be a very interesting year.
Well at least Google operates in a space that rewards innovation. Other “monopolies” and duopolies in our economy gain their leverage through legislation and “regulation” I’m thinking of insurance, telecom in particular. Artificial barriers to competition as opposed to honest competition. We could only be so lucky to have such problems in the other areas of our economy, as we do in the web services sector.
One minor quibble: I think you mean AdSense when you say AdWords? These companies are SEO plays, not SEM plays.
Secondly, no one has actually tried to quantify the problem and put it in perspective. Google’s Policy team is well aware of Demand and WikiAnswers but the reality is that 30 million uniques in a month is a drop in the bucket when stacked against overall web traffic processed via Google.
So there *is* indeed a place for this kind of cookie cutter content just like there’s a place in the world for fast food. Just like fine dining and McDonalds can thrive together in the world, so can The New York Times and Demand Media.
Lastly, I am NOT a Google apologist and think that they should have better, more transparent rules of the road. They should also be able to give time-of-day to publishers who aren’t dependent upon text-based content but instead use Flash, video, whatever. But those discussions are orthogonal.
“Orders of magnitude,” not “magnitudes of order,” I think…
I come back from Korea where the search engine are human powered. I used to use Google a lot, but I must say that I trust services like delicious more and more, because they are also human powered.
Are we going to see human powered services taking the lead on algorithms. We will see.
John, you make some really interesting points, but I’ve got an interesting datapoint to add to the pile. I made and saved some old search result data from 2000. It turns out that the search results 9-10 years ago weren’t perfect–even though many of us remember them as being that way.
thanks for sharing..
The end conclusion sounds nice, too nice. I’d like it to be true but it just sound too utopian. Judging by current trends there will be series of mild improvements from this point on. Unless a completely new player comes and blows everyone away.
There are thousands of such attempts made every year and all of them make a dim glow in the noise of mediocre solutions.
John, you do know that Demand Media is a strategic partner for Google. Google specifically asked them to produce content for Youtube. At the end of the day, the losers are content producers like you and me, not Google.
I find it somewhat ironic that you’re using a Google owned technology (recaptcha) to let you know that we’re human.
It’s true that there’s been a lot of new search engines that never took off but that’s largely down to the fact that people were happy with the Google experience. Now though, more and more people are getting frustrated with traditional search so we agree, expect to see big changes over the coming year.
I thought / was hoping this was going to be a post about overly cluttered SERPs that cram Local results and maps, video results, real time results, Adwords ads …. along with ‘organic results’ all onto one page.
And also, John, where is the complaint that it’s difficult to actually find and compare commercial products when looking in Google. Too often Google thinks we want a Wikipedia entry or 250 words of content when we really aren’t looking for information. These ‘content farms’ are merely gaming Google’s current inadequacies when it comes to crawling dynamic content, Flash, javascript, etc.
“Back in 2002, when we put “dishwashers” into Google, we’d probably find someone’s blog who was talking about his favorite models.”
Really? How many lone bloggers have enough dishwashers that they blog about? Not many. Most people write one rant about their failed dishwasher and, then, maybe, the new one that replaces it.
That is why appliances — unlike fashion or recipes — are so spam prone. It’s not possible for one blogger to set up a dishwasher review site as opposed to a bra or cookie review site.
My question would be is Google missing the good non-subscriber dishwasher review site or does it simply not exist?
P.S. Isn’t this precisely why consumer Reports was invented back in the pre-internet days?
John, where is the complaint that it’s difficult to actually find and compare commercial products when looking in Google. Too often Google thinks we want a Wikipedia entry or 250 words of content when we really aren’t looking for information.
The problem is that people are looking for deep analysis to complex questions not easily answered by machine analysis. The dishwasher is a great example. What Paul Kedrosky is looking for is not a listing of prices and other data, but insight. What dishwasher should I buy, and, more importantly, why?
DemandMedia, AOL, and Mahalo all attempt to capture those intentions by quickly answering the question, but clearly they are failing at answering it well.
Aardvark is the answer here, I think, and Google knows it: http://www.businessinsider.com/aardvark-mulls-over-a-30-million-offer-from-google-2009-12
Aardvark now is a one-to-one answer service, but its next step has to be to aggregate those answers and present them for search. If you search on dishwashers, you get a series of anonymized answers, and a failover to hit the network of Aardvark members.
These fast-food agents will be quickly swept under by the quality and direct content presented by real-person driven answers.
John,
I find it really difficult to fathom that what I have been writing here on your Search Blog for YEARS is now rehashed by these people you link to and now everyone pretends that that this is some kind of newfangled discovery.
I find such an approach is SEVERELY lacking in transparency and so-called journalistic ethics.
That said, I am at least happy that people are beginning to understand what I have been saying all along.
Now that people are waking up to smell the coffee, perhaps we can finally move on towards actual technological progress.
🙂 nmw
I don’t get it. When I type “dishwasher” into google, I see “Shopping results for dishwasher” right near the top. I click on it and I see a lot of choices. How is this hard?
Long time lurker who loves your work, Mr. Batelle.
This article has de-lurked me. It is an important one, one that Google inexplicably does little about.
You’ve only touched upon the tip of a huge iceberg, one that Google does not seem to hvae the DNA to bring down, because it could if it wanted to.
Google knows about the “make money online” scum that teach people to spam Google with no-value, copied content based upon high-value PPC words. The content is taken from pages with REAL content that rank highly and then re-worded TO AVOID DETECTION BY GOOGLE (not to add any value).
Then they build fake inbound links networks with yet more near-plagiarized content or automated cotnent-creators. These links help push the “money page” up in the rankings. Then work the Law Of Large Numbers:
Create hundreds of these “money pages” with inbound link networks. A few might rank and make you a few bucks.
If you have ever read these “MMO” gurus like Victor Franqui or Dennis (Griz) Brears, they head up a vicious cult whose followers believe that “dishonest” is “honesty.” The distorted reality in there is scary.
Google needs to strike these people who pollute, abuse, steal, and add zero value (at best) now. They cannot afford to wait to try to figure it out algorithmically.
The MMO gang and their link-bombing tricks are too sophisticated for an algorithm. They form organized groups (including “Brears” private “I’m in” group who do his bidding) to push each others’ pages up in the rankings.
Few people understand how vulnerable Google is to link-bombing. Googe claims to have defeated link-bombing. They lie. It is the easiest of all black hat exploits (which is not to say it is easy because it takes a great deal of brute work).
One of the MMO spammers’ favorite, and most nefarious, tactics is to attack trademarks (ex., “ABC reviews” and work their way to the top of the SERPs. People who search specific brand names are close to a purchase. That is why the PPC is so high. It is easier to link-bomb your way into the SERPs.
You can use the link-bomb to attack competitors or earn affiliate income or Google income. These are COMMERCIAL link-bombs, far more dangerous than the frivolouse POLITICAL ones because MONEY is involved.
And when there is money to be made, a trend grows. So yes, it is geting worse by the day.
How does Google fix this mess? By understanding its own shortcomings.
Yahoo! was a directory. They used humans. It took them too long to understand the power of the algorithm. Their DNA killed them.
Google is the opposite. Its DNA insists on “the algorithmic solution” (except for Michelle Obama becaues enough pressure overcomes anything, so they go to the manual fix after many lame explanations). That algorithm delivers tremndous scale, and profit margins.
It is time for Google to spend a little of those profits and defend their guidelines and squash the unethical who laugh at their guidelines and “don’t be evil” motto.
Spend a few million on humans, not the standard “quality” program they have now. Develop a program where competitors submit proven cases of link spam. Charge a substantial fee if you are worried about abuse. Insist upon detailed documentation.
Now, Google, retain HUMANS to verify those reports. A human is far superior to an algorithm (today, at least, and for the next 1-2 years at least) when it comes to the type of link-boming that is pushing all those bogus pages to the top of SERPs for searches for “dollar words.”
What do you do when you find these link-bombs?
Ban the bad guys. Make it public.
Google is spending legal dollars to protect its trademarks, but what about its guidelines and principles?
Use humans to eliminate this spamfest, to clean up the pollution. Deeps investigations will find the complicated scams on Google.
Ban them. And put their heads on stakes as lessons for others (metaphor only).
Some would say Google has no moral obligation to do any of this.
They do. They know their Guidelines are being violated, and not just slightly. Either drop the guidelines or drop the bad guys.
It is their index. There is no reason why they have to let the Web polluters into it and ruin the search experience. They add Twitter, but they don’t eliminate spam for an ever-increasing number of searches?
Where’s the logic? Google, come clean on this. You have the resources to fight this. Why don’t you?
“We need a new framework for search, one that allows us to leverage our inherent ability to converse. And from what I can tell, it’s closer than we might think.”
I think you’re right. And at the moment, I think the best shot is Aardvark.
Your perception of Google is soooo 2003. Yes they are evil…yes they are going in the wrong direction, but content spam has been almost done away with.
Your reference to making a buck off of AdWords is referring to what was known as Google arbitrage. This is where people generated tons of made for AdSense (MFA) sites and Google has smashed them as well.
You are so wrong, Jason. Check out “Grizzly Brears” sites and his rabid following. Read his stuff. People actually follow his writing and take it seriously.
Read about his “I’m in” group. Sorry, you don’t know the down and dirty on this subject.
Google actually knows about this guy and does nothing about him.
Bob
I totally agree.
We need really something new, not related to the infrastructure or based on details like “taking into account the size of the font used to represent the keyword inside the document”.
We need a paradigm shit as PageRank caused in 1998.
The fact of the matter is that we need the be able to take into account much more information from the user.
That implies two things:
1- a new search interface
2- an algorithm that can handle that enriched query.
Ascot project aims at creating the search bar 2.0.
We have developed a way to process concepts by using similarity computation between tokens.
For more information and assuming that you have 13 seconds, I think you can have a look at our blog:
http://ascotproject.blogspot.com/
Awaiting your bitter criticism ^^
Pascal Hideki Hamonic, from Tokyo
I totally agree.
We need really something new, not related to the infrastructure or based on details like “taking into account the size of the font used to represent the keyword inside the document”.
We need a paradigm shit as PageRank caused in 1998.
The fact of the matter is that we need the be able to take into account much more information from the user.
That implies two things:
1- a new search interface
2- an algorithm that can handle that enriched query.
Ascot project aims at creating the search bar 2.0.
We have developed a way to process concepts by using similarity computation between tokens.
For more information and assuming that you have 13 seconds, I think you can have a look at our blog:
http://ascotproject.blogspot.com/
Awaiting your bitter criticism ^^
Pascal Hideki Hamonic
Google does have the ability to minimize the impact of content mills by including domain name expiration data in their algorithm.
Simply put, folks generating content for unproven “money-making” pages will not generally risk the capital to renew a domain name well into the future, whereas legitimate companies will do so.
I believe Google’s algorithm is already taking this into account.
Jeffrey
I agree Jeffrey.Thanks.
I sort of agree. But until Google decides to rebuild the castle from the ground up, the foundation has already been laid. If the foundation was so good years ago, and it was praised for returning relevant results, then what has gone so wrong? The fact the the content providers, you and me, the user, have figured out how to provide relative content? Good or bad content, eventually the public will be the judge. Just like a bad meal at a bad restaraunt, eventually, the paying consumer will decide to go elsewhere.
LOL @Pascal “We need a paradigm shit” ;D
@Life: See http://sn.im/wisdom-link
🙂 nmw
I find it really difficult to fathom that what I have been writing here on your Search Blog for YEARS is now rehashed by these people you link to and now everyone pretends that that this is some kind of newfangled discovery….Now that people are waking up to smell the coffee, perhaps we can finally move on towards actual technological progress.
Hey, nmw, I had the same thought 🙂
I agree Jeffrey.Thanks.
As much as Google is trying to clean up the internet and be relevant to searchers queries, sometimes I wonder if they will ever be able to accomplish that.
Just yesterday I was trying to search for Life Insurance Canada, to see if i could get some decent life insurance quotes, and I couldn’t believe the number of irrelevant results that google was showing. I think we still need to remember that it is just a robot, and will never be able to interpret humans.
This area is very convenient designed for us and you and stab to the lawyer put, This Company is poised of expert Lawyers from all above the humanity. Attorneys who finger hand baggage in such areas of expertise as our top attorneys finger hand baggage the whole time United States. With the combination of our unique company edifice and the most up-to-date expertise, we provide clients with the greatest officially permitted representation. So please visit our put and know many more.
medical
malpractice attorney
A kidfocussel will expedientcrepresentingdren to exhibesidestbe offir most creativsituateeas by painting laself-possessedures but without having toconglomeratehapeuman racely acrossmost modernxport-howr kneel on the floor.Thus, inmost excellentilawful a good qexport bags during smalsituatet important decision.Melissa and Doug Easel