Ingesting RSS into Elasticsearch


One of the most frequently asked questions we get is how customers can monitor particular topics of interest using open source tools such as Elasticsearch or Hadoop. In this write up we will talk about some methods in which you can setup alerting and collection of information using open source tools.

Getting the News Content

For this exercise to work properly we have to get some news articles or data that is of interest to us. A good source of news may be news.google.com or possibly RSS feeds that contain information that may be of interest. In this example below we will use RSS as the source and we will feed Elasticsearch. In reality you can use this method to really pull in pretty much any source of data.

Using Logstash to read RSS locations

Since were interested in the ingestion of RSS data into Elasticsearch we can use Logstash from the ELK stack to read RSS configuration. Below is an example of reading an RSS feed with logstash.

Logstash Input

rss { url => "https://www.yahoo.com/news/rss/mostviewed" interval => 7200 tags => ["en", "rss", "yahoonews"]

In this example the Interval will tell the running logstash process how often to read this source for new data. It really is just that simple.


74 views

Contact: (800)447-2150 Ext. 1        To contact Jigsaw simply send a message in our chat window!

  • Facebook - Black Circle
  • Twitter - Black Circle

© 2017-2018 Jigsaw Security Enterprise Inc.

Jigsaw Security Enterprise Inc is a SDVOSB - Service Connected Disabled Veteran Owned Small Business Jigsaw Security is an operator of WIMAX networks and is operating under license WQVC235 as a common carrier, non-common carrier and private communications operator. Jigsaw Security operates cable and satellite services. Courses may be provided by a third party authorized training partner in some cases. Some training is only available for cleared and US Citizens. Courses approved by the North Carolina Department of Public Safety Private Protective Services Board for licensing and CE credits. JPM program insurance is provided by an authorized Jigsaw Security Insurance Partner and is not underwritten by Jigsaw Security. For insurance information please contact our JPM program manager. Jigsaw Security operates a network through our NCBroadband brand.