News Aggregation

Personal news aggregator service which scrapes for content from news sites, blogs, social networking websites and others The crawler obeys robots.txt files and robot META tags in HTML.

Tiscali Italian media news aggregator bot

Subscription based curated blogger platform.

Media monitoring, analytics and market intelligence bot which crawls your website looking for content. LexisNexis acquired Moreover media monitoring and content services, and the content is widely used and re-sold to providers of e.g. media monitoring, and other news oriented services.

Easy to use RSS reader with Android app that aggregates scaped content and allows the users to read from on location

Award winning curated social content aggregator designed for the iPad. FlipBoard says the scanner is activated 'generally in response to a user request that our service scan a social media feed (Twitter, for example) and constructs a processed feed of items to deliver in real time' - but they also do scrape content to use in the service from other sources.

feeder is a Consumer app solution allowing user to curate and manage their own personalised compliation of rss feeds. This crawler will be visiting you because users of the feeder service are interested in your news/RSS feeds. Allow this crawler if you are happy for this to happen.

RSS reader and curator that offers a personalised content portal from social media - twitter, YouTube and also mainstream news publications. Feedly doesn't obey robots.txt but says " Fetcher shouldn't retrieve feeds from most sites more than once every hour on average. Some frequently updated sites may be refreshed more often. Note, however, that due to network delays, it's possible that Fetcher may briefly appear to retrieve your feeds more frequently"

Faveeo is a smart AI-driven tool highlighting only reliable, meaningful content on any topic, by finding and following the most trusted and valuable sources

News aggegrator - sells subscriptions to customised content scrapped from RSS, social media and content web sites

Real time alerts services for PR, financial based around the Twitter firehose

CuWho is a Spanish technology blog news aggregator

The crawler for Apple News. This bot crawls your site for news and blog content. Allow this bot if you publish news or blog articles you want available to such an aggregation service.