Ingest is hard so we made it easy

At Jigsaw we deal with big data projects nearly daily. Our customers utilized our platform to ingest billions of items per day that provide direct insight into data by parsing out items of interest through keyword searches and items of interest.

Ask any intelligence analyst and they will tell you that having a hit list of important terms and items of interest which they can update while doing research is a top feature request for keeping up with what is happening in the world and what cyber related threats may affect their businesses or the businesses they protect. Being alerted to email compromises, hackers talking on IRC planning an attack on your organization or identifying activity on the dark web indicating that your organization may have security issues are just some of the things that are typically done with the Jigsaw platform. Search relevant items of interest and you may be able to stop the next cyber security threats or terrorist attack.

Ingest Made Easy

Jigsaw Security has been listening to our customer request. One of our latest features is the ability to directly ingest PDF, Word Documents, Images, Video and nearly any media type that may be of interest. Currently our ingest processes 190 file formats natively by pulling out metadata and by OCR recognition on PDF and image files. This allows you to quickly and easily find items of interest that are streaming into your environment. Previously ingest took the form of customized scripts which constantly needed tweaking to find things that are of interest and then create reference data to point to the files stored in cloud environments. With Jigsaw's solution, all you do is sftp or scp your files of interest to a directory, our platform then looks through the documents stored on the ingest directory and does 2 things. The first thing our platform does is extract all of the text of the documents and store that text in your platform index to make it easy to find what your looking for, the second thing that occurs is that the document is then made available to all users in your platform with a link to the original content so it can be downloaded by your users. This automated ingest makes things extremely easy whereas if you can get the data into the ingest directory using an automated or manual process, the data is instantly available to your cloud users.

Let's have a look

So let's have a look at an example of our ingest process. First let's find a document of interest to ingest. Let's go through a typical Jigsaw analyst process and see how we go from content to delivery of content. We start by using our OSINT-X platform to find items of interest.

OSINT-X: A Jigsaw Security framework for bringing content into our platform from RSS and o