Ingesting RSS into Elasticsearch


One of the most frequently asked questions we get is how customers can monitor particular topics of interest using open source tools such as Elasticsearch or Hadoop. In this write up we will talk about some methods in which you can setup alerting and collection of information using open source tools.

Getting the News Content

For this exercise to work properly we have to get some news articles or data that is of interest to us. A good source of news may be news.google.com or possibly RSS feeds that contain information that may be of interest. In this example below we will use RSS as the source and we will feed Elasticsearch. In reality you can use this method to really pull in pretty much any source of data.

Using Logstash to read RSS locations

Since were interested in the ingestion of RSS data into Elasticsearch we can use Logstash from the ELK stack to read RSS configuration. Below is an example of reading an RSS feed with logstash.

Logstash Input

rss { url => "https://www.yahoo.com/news/rss/mostviewed" interval => 7200 tags => ["en", "rss", "yahoonews"]

In this example the Interval will tell the running logstash process how often to read this source for new data. It really is just that simple.


390 views

Contact: (800)447-2150 Ext. 1        To contact Jigsaw simply send a message in our chat window!

  • Facebook - Black Circle
  • Twitter - Black Circle

© 2017-2020 Jigsaw Security Enterprise Inc.

Jigsaw Security Enterprise Inc is a SDVOSB - Service Connected Disabled Veteran Owned Small Business