ElasticSearch was integrated by Mindflow to enable users to automate their incident management and better protect their information system.

ElasticSearch Overview

Elasticsearch is a fast and scalable enterprise-wide search engine. The platform is part of an ecosystem of complementary tools (Kibana, LogStash, and Beats that create the ELK Stack) that can be used for many use cases at any level of your enterprise, from search, analytics, security, and storage.

Thanks to its components and architecture, Elasticsearch allows you to store, search, and analyze huge volumes of data in near real-time.

Instead of searching the text, it searches an index. It uses a structure based on documents instead of tables and schemas and comes with REST APIs for storing and searching the data.

To better understand how Elasticsearch works, let’s cover some basic concepts of how it organizes data and its backend components.

Documents

Documents are the basic unit of information indexed in Elasticsearch, expressed in JSON, and thus can store any structured data).

Indices

An index is a collection of documents showing similar characteristics. It’s the highest level entity that you can query against. Any documents in an index are typically logically related. The index is identified by a name used to refer to it while performing actions such as indexing, searching, updating, and deleting documents in it.

Inverted Index

Like most search engines, ElasticSeach splits your documents into individual search terms, then maps these terms to the documents those search terms occur within. By using distributed inverted indices, Elasticsearch quickly locates the best matches for full-text searches from even extensive data sets.

Also, the solution can subdivide indexes into multiple pieces called shards. These shards are fully-functional and independent “indexes” that can be hosted on any node within a cluster. You can make “replicas” or copies of your index’s shards.

Benefits

From what we’ve described, ElasticSearch is powerful and can thus be used in numerous use cases. Allowing you to connect to your ElasticSeach environment on Mindflow enables you to leverage its scalability and rapidity of analysis to your playbooks, especially in security:

Security analytics of access logs and similar logs concerning system security

Application for the access, retrieval, and reporting of data

Website storing content for effective and accurate searches

Enterprise-wide search, including document search, E-commerce product search, blog search, people search, and so on

Logging ingestion and log analytics in near-real-time and a scalable manner

Infrastructure metrics and container monitoring to analyze various metrics