Google News RSS Scraper

Via ResearchBuzz, a tool that scrapes Google News (and Google News searches) and turns them into an RSS feed. It's called ScrappyGoo, and it's against Google's TOS. My view: change your TOS, Google, or start supporting RSS, er, Atom, er, feeds in a more robust way. This is a…

Via ResearchBuzz, a tool that scrapes Google News (and Google News searches) and turns them into an RSS feed. It’s called ScrappyGoo, and it’s against Google’s TOS. My view: change your TOS, Google, or start supporting RSS, er, Atom, er, feeds in a more robust way. This is a cool idea.

(I sense this has been done before, but I can’t recall where…)

Author: John Battelle

A founder of NewCo (current CEO), sovrn (Chair), Federated Media, Web 2 Summit, The Industry Standard, Wired. Author, investor, board member (Acxiom, Sovrn, NewCo), bike rider, yoga practitioner.

4 thoughts on “Google News RSS Scraper”

  1. It says on the Scrappygoo website where it was done before… in the UK, by Julian Bond, who at the time was working for ecademy.com [he might still be 🙂 ]. Google sent him a cease and desist letter, which Julian said he was expecting, and he said he didn’t particularly mind stopping since he knew it was against the TOS.

  2. Bah, why should any one be worried that they break Google’s TOS? They scrape our sites. Pot, Kettle Black anyone?

    Sorry, just a trifle testy this morning 🙂

  3. I gotta strongly disagree with you on this one, John.

    “Hmm… you know, this shareware author isn’t pricing his stuff right. I think I’ll just hack his software and not pay.”

    “Geez, I can’t believe that pianist is charging $20 to see his performance! What a rip. I’ll just sneak in the back.”

    Or, closer to home,

    “What a pain in the ass all these site registrations are! Sure, these companies are spending millions making content available to me for free, but screw ’em… I’ll use bugmenot.”

    I’ve said it before and I’ll say it again: Just because you may not like the terms for a particular product or service (or the terms are, frankly, indeed fubar’d) doesn’t give you the right (legally or, IMHO, morally) to use that product or service and ignore the terms. Don’t like the terms? Don’t use the product or service. Simple as that.

    I completely agree that Google *should* be embracing RSS and Atom more fully, and in fact, I spoke personally with one of the Google News folks last week and urged them to do so.

    But the bottom line is that it’s Google’s bandwidth, it’s Google’s engineering resources, and so on. Violating their terms is rude and counterproductive. Want them to embrace RSS and Atom more? Start a petition, fly a plane-with-banner over the Googleplex, etc. etc. etc. But quit the ‘civil disobedience’ crap. This ain’t a matter of civil rights or life and death 😉

Leave a Reply