An Apology To My RSS Readers – But I Had To Do It. (Updated)

Some random site running my last post without asking.

If you’re a fan of this site, you’re also probably a fan of RSS – a once-ascendant technology that has been on most everyone’s deathwatch for five or so years. According to Google’s (almost totally outdated) Feedburner service, nearly 450,000 people subscribe to this blog via RSS – although the number of you who actually read my posts is far smaller (according to Feedburner statistics, which I’ve never fully understood).

In any case, from time to time I’ve poked at you poor RSS readers, just to find out if you’re alive. Remember this piece – Is RSS Really Dead? Or this one – Once Again, RSS Is Dead. But ONLY YOU Can Save It!?

In those posts, I asked if my beloved RSS readers were really out there. Turns out, I got tons of comments back – a very high number given the work involved in declaring fealty to the creaky old standard. (It kind of felt like a reshoot of that wonderful final scene in Horton Hears a Who – “Everybody yell real loud, and maybe Google will hear, and not deprecate Feedburner…”  But I digress.)

I’ve always kept my RSS feed “full text” – which means the entire post, pictures, words and all, goes out over RSS, and can be picked up by any RSS reader anywhere on the planet. I always have held the belief that it’s more important that my work get distributed than monetized. But not everyone can afford such high minded principles. Many publishers cut their feed short, teasing folks with headlines and a snippet of the story in the hopes that people will click through to the site, where their visit can be properly “monetized” via advertising.

My new feed (sadface).

After much thought, I’m going to do the same. But not for the extra clicks and ads. It’s due to the fraud that’s taken over the content space in the Indpendent Web. Untold legions of bad actors use RSS to scrape “real” sites like this one, then wrap them with ads from exchanges to make a quick buck. The rise of programmatic fraud has made this even worse (see It’s Time To Call Out Fraud In The Adtech Ecosystem for more on this). And no, I’m not going to link to examples – but you can Google “Content Scraping” if you want to learn more.

So, consider this an apology. I am very sorry that you have to click a link to get to the content I make here every day. But also consider this a plea – as in, please do click that link at the top. I very much want you to be part of this conversation.

(And if enough of you complain, you know I’ll listen, and figure out some way around this).


UPDATE: I turned full feed back on. Thanks for all the input.

102 thoughts on “An Apology To My RSS Readers – But I Had To Do It. (Updated)”

  1. I’ve been an RSS reader for a long time. I’ll keep the truncated feed, but I apologize in advance if I don’t click through much of the time. Thanks for being cognizant of RSS readers. Whenever I talk about RSS to other people, I have to start with a quick primer on what it is… and I’ve been with Google Reader for years and years now.

  2. Truncating RSS does not solve the issue you have. Fraudsters can still scrape your site and republish them. Screen scraping is simple script to write. If that was your only concern I don’t see any reason to not publish full text in the feed.

    1. It’s a tiny bit more work to run a screen scaper, and I have found fraudsters are lazy as hell. Almost all the stuff I’ve found has been directly off my RSS feed.

      1. Sorry John, I agree with Subramanian, this sucks for your loyal readers (like me) and is at best a temporary treatment of the current symptom, but certainly not a meaningful deterrent in the larger scheme of things (as long as there’s value to be had by copying public content, it’ll get copied). I think you’re dong more to hurt the web by turning off your RSS feed than by helping these guys make some extra money by scraping your content. I do hope you’ll reconsider.

        You’re one of the few tech pundits who genuinely (and compellingly) argues for the Open Web and data portability, so if supporting long-established open standards for your own personal public blog is too much trouble to be worth it, what chance do the rest of us have, and what weight will your words carry?

      2. Your Open rationale makes sense to me. The open nature of the internet is its greatest strength and it’s biggest curse.

  3. Apology accepted. Enough websites use truncated feeds that I’m used to it.

    (Besides, the day will come when Google+ and/or Facebook will finally do the Embrace, Extend, Extinguish act on RSS… why it hasn’t happened yet I have no idea.)

    1. Sigh, probably not. But then again, not sure that many folks would use that service. Again, if I hear that you all hate this, I’ll do something else.

  4. I understand the reasoning, but why does it matter? How does this help you? Unless Google was letting those scrapers outrank your original posts, then don’t worry about it. If it’s a problem, add RSS-only footer links using WP SEO or similar tools to help Google understand that you’re the authority (most scrapers will leave them in place).

    Assuming that’s you’re outranking the scarper sites, which you clearly are, then:

    – Doing this doesn’t help you.
    – Doing this hurts your readers

    I understand that you’re not happy about it, but I’m not clear why it really matters. One of my blogs gets reposted around 100 times for each post, but I don’t care. It sucks, but my energy goes into producing additional solid content. Google knows I’m the authority, so I get the lion’s share of the traffic, and I can still offer full RSS feeds to my readers.

    1. You make fine points (I added an RSS only header a while ago, but found many scrapers deprecate the links), but there’s one additional point that bears considering: I don’t want to feed the beast that is driving fraudulent content into the programmatic advertising ecosystem. It’s less about SEO to my site, and more about those 100 posts of yours – those are probably all being monetized and that just sucks – it destroys the value of the independent web in other ways.

      1. I certainly agree that it sucks, but I still don’t quite get it.

        We both know that while those sites are being “monetized”, that:

        A – They’re making pennies per post, at best.
        B – You’re not losing any money as a result of them.

        Heck, most will probably continue to republish your new-truncated RSS feed and continue to “destroy the value of the independent web” to roughly the same degree. I agree that it’s awful and should be stopped, but I’m not fully convinced that this is the way to do it.

        If you’re simply fighting for the bigger cause, you need to rally bloggers everywhere to truncate their RSS feeds and make a difference. In the meantime, it seems more like a political statement at our expense, as it really won’t make a blip in the global RSS-republishing-garbage-sites volume.

      2. It’s exactly these economics that make it so insidious. Processing is dirt cheap, so why not create millions of spam blogs, populate them with scraped content, and make pennies per post? The money just piles up. It sucks.

  5. I’ll register my vote to keep the full feed. I agree, as long as you get most of the traffic, does it really matter? I’ll certainly stay subscribed, just probably read less content. Please make an effort to give us great headlines and descriptions.

  6. I would really miss this ‘voice in my head’ so please count me as a voice for the full feed. (FWIW, I provide a full feed on my blog, not that anyone is likely to want to steal it.)

  7. I’d just like to add my vote for the full feed. I read your blog (among others) offline on my phone on the tube. I’d only be able to click and get the full content if I had reception. Yep there’s plenty of time during the day when I have reception, or a proper computer, but I like being able to keep up with blogs on the daily commute.

      1. Instapaper or Pocket? I tend to use them quite often and they blend pretty well with Twitter and other apps. What’s the added benefits of an RSS reader over these two services?

      2. but you have to go find the stuff you want to read and save it to Instapaper or Pocket – where as with the RSS reader, it’s automatically already there waiting for you…

  8. I’ll click through. It’s not ideal, but the reasons make sense. Take this as a big compliment to the quality of your writing; I’ve dropped a lot of feeds once they required clicking-through to read.

  9. This rocks, JBat. Amazing to me that customizing content for one’s tastes is such a clunky process fraught with so much peril in 2013. This is why I think Yahoo has a shot.

  10. I was truly disheartened when i read the article title. First I thought you are no longer going to support rss feed for your blog. Anyways, this change does not affect me as i always visit the original article page to read comments and sometimes post a comment as well.

  11. Hi John. Thanks for this. As someone who reads you via RSS from time to time, this bums me out a lot. Getting your voice and message out should trump your desire to not feed the machine. Don’t make it harder for people to get at your content. I’m not sure I believe that truncating your RSS is going to have the desired effect (and it doesn’t sound like you do either). Consider this a vote for the full feed, though a truncated feed just means I will likley read less of you, but won’t abandon you altogether.

  12. Very far from ideal but you could publish full text rss under a secret URL that you share with commenters (or requesters who somehow prove they’re human/not scammers). Again, far from perfect but it might work ….

  13. Atleast to read the comments of the article from the rip-off, one gets re-directed to your original article JB. I for one enjoy the discussions/comments quite a bit.

    Goodluck with the RSS feed, I use it for sites where I am worried about their overbearing design or flashy ads which will catch everyone’s attention at the work place (remember the belly reducing ads?). Your site however I access directly via your links on Twitter, I can trust it – so far.

    1. Thinking of options, perhaps a captcha prior to sharing the RSS feed link might reduce some of that – if this practice is rampant based on your RSS feed alone. In any case I am not sure what is to prevent someone from scraping your site directly and updating content (assuming there is a diff), any small script could do that.

  14. I’m surprised that you held out for so long. I’m so used to clicking on other feeds in my reader that it doesn’t really matter.

    That said, maybe a couple more paragraphs before truncating? Just so I can decide whether to click through now or later depending on when /where I’m reading.

  15. I do understand the problem, and IF John says that he is not doing this for the extra page views, i’ll believe him. One more click for me is not such a big problem. However, the “if” of the first sentence, is still a big, capital, IF..

    1. I swear it’s not for the Ads. Compared to real ad drive sites, my traffic is very low and even if it doubled due to this it’d not matter in the larger scheme of things.

  16. This is another vote for the full feed. I read exclusively via rss, and on my phone (ios/byline/google reader) whilst commuting. I have a restricted data plan and limited reception. Full text feeds allow me to download everything via wifi before I leave in the morning.

    I enjoy the site and will probably switch to one of the multiple services that convert a truncated feed to full text. I suspect that, whilst a minor inconvenience to spammers, the only other thing this will do is make your stats more unreliable if others do the same.

    It’s a real shame that you are doing this, although it is your content and your decision.

  17. I always prefer full feed but, having been the victim of RSS scraping myself a number of times (and it seems to be increasing), I can certainly understand the frustration.

    I’ve been lucky that I’ve managed to get a number of sites to stop publishing my stuff but the truly automated sites aren’t going to care about takedown notices etc. – they’ll just throw up another site if the current one gets removed.

  18. I’m sorry I’ve no solution to propose for your problem, but I’ll complain anyway (though, you know, you don’t owe me anything, so at the end, you do what you want to).

    I’m suscribed to hundreds of RSS feeds ; I don’t read every single post that is written : I read one paragraph, maybe two, then I decide if the post is worth reading or not. With such an amoung of posts to “read”, I’m definitely not happy with having to click, wait for a full website to load, and so on. Then, unless the title itself suffices for me to know I really want to read the post (and this doesn’t happen very often), I’ll probably no try to read the full article.

    Best regards

  19. Very few of my feeds are truncated, and I tend to skip cilcking through on them quite often (it sets a higher quality bar). This is even true on mobile, where the excellent “Reader” app caches and formats the text excellently for full feeds compared to often bad mobile experiences that requires click-through.

    In summary: I tend to lean to truncated feeds only when there’s advertising revenue that I find more important that actually getting people to read my content.

  20. Hi John, I’m very much a vote for keeping it as full text. I feel your pain regarding scrapers (we all have the same problem), but changing your publishing practice to the detriment of your readers is not the way to beat them – in my humble opinion.

  21. Full feed or nothing. I rarely have the time to click into blogs that truncate. I hear your concern about ‘feeding the beast’, but you are trying to cure cancer with a toothpick. Your knowledge is more important to solving the problem than your proposed solution 🙂

  22. Most of my feeds don’t have full content in it anyway. I don’t care, if the first paragraph or the title is interesting, i will click on it

  23. Reading blogs on my tablet is just so much better with RSS. The text is always the same size! Your feed had the paragraph breaks zapped but I read it anyway. RSS really is a treasure that should be nurtured. People don’t know what they are missing.

  24. John,

    You might consider ensuring your site is “mobile friendly” first, then truncate away. 🙂

    (I don’t read your stuff in mobile personally, but if someone clicks to read in mobile, they need your site to be mobile friendly)

    Your free content, which I appreciate btw, is good enough to click on the site and read it. From the user’s side, we have Instapaper and Readability to convert the articles and read them cleanly if necessary.

    I can’t see how it’s going to solve the problem you’re having though, if “scrapers” want your content, they’ll get it, it’s a very difficult problem to solve.

  25. Other commenters have made all the points I was going to make. Thanks for sticking with full text RSS for so long. Sorry to see it go. Maybe there’s a way I can get your full posts to read on the train, sure wish I didn’t have to find it though.

  26. I’ll throw a vote for “bring it back to full text” as well. If you can measure the amount of fraud before and after the truncation it would be interesting to see if it improves the issue.

  27. So always read via the feed, and I will admit now that we’ve got to click through to read the story we will be wanting to see something that tells us why. Headlines work..

    However would you consider writing a summary paragraph on your article via RSS so we can see if want to click through ??

  28. I’m a huge RSS reader. (at least 75% of my web reading is through RSS.) I used to hate truncated feeds. But, like others, I’ve gotten used to them. In fact, when done right, it helps me get through my feeds faster. I have no problem clicking through for a story I’m interested in.

    Here’s my advice, though. Title becomes much, much more important. Also, I think it’s important to have a strong first paragraph that does show up in the feed. Enough for me to get a gist of what the article is about and care enough to read the rest.

  29. I read almost all posts via RSS, but with +1000 different posts a day, I haven’t the time or energy to click on a link to a full article. Except when the article/post seems to very interesting… So it is even more important that the first sentence and headline is extremely telling and unique. The bots will have the time to follow any link from any good blog anyway, so I don’t think you’ll beat them. I very much hope you’ll start publishing the full posts again!

  30. Have you considered embedding advertising or similar in the RSS feed? If they are going to syndicate it, they can have your adwords on their blog too.

    Alternatively, use google’s verification features to assert ownership over your content. If it’s syndicated, and they publish your verification token… well, google will see you control said content.

    For example, with google’s webmaster tools, you can set up URL removal requests.
    Removing / from the index of google’s site… would likely be a good way to discourage content theft 🙂
    I believe with that you need to set up a meta tag or upload a HTML file; but surely there must be other similar ways you can use your control over content…

    1. Yes I have considered it. And the problem is, we have no idea who you guys are, so we can’t market who you are to advertisers.

  31. I vote for full RSS text. I tend to read your work through Google reader and Newsify on iOS. I basically agree with the points raised by the people who are for full RSS text so I don’t feel the need to reiterate them. I hope you reconsider.

  32. That is a good thing to do. Even I have my blog’s RSS cut short. One thing you can do is tynt to enable backlinking from any content scraped websites. There is however a chance to accumulate some bad neighborhood links which you can disavow.

  33. I’m usually subscribed to between 100 and 200 feeds via Google Reader. I read them offline on long bus rides, in a cool little newsreader on my phone that gives me plain black on white fullscreen text in the font of my choice. There’s always more to read than I can finish, so incomplete feeds tend to get skipped.

  34. This is a real shame because I like reading your stuff but I know myself well enough to know that I won’t be clicking through regularly. There have been entire blogs that I’ve stopped reading because I couldn’t read them natively in my Google Reader.

    Sorry it’s come to this, John.

  35. John – I realize you’ve gotten a lot of comments already, but here’s another vote for full feed. Not only is this the only way that I keep up with my favorite content from all of my top sources, but I also judge the type of content by full feed as well. Sometimes I don’t have the time to read through some of your long form stuff, so I can “keep unread” for as long as I need until I get through it. And I DO get through it. I’ve probably never quoted a single author more than you. Short form stuff I can get through right away. Truncated means I need to click through on every item to judge whether or not I have the time to read through the article. While the extra step seems trivial, it isn’t. RSS is about speedy browsing, and this breaks that cycle.

  36. ANNNNNNND Unsubscribe.

    I like your writing, but honestly, it’s not good enough to go through the trouble of clicking through. There are other blogs out there that have comparable content. Sorry!

Leave a Reply

Your email address will not be published. Required fields are marked *