EFK Archives - DEVOPS DONE RIGHT

Protected EFK Stack Setup for Kubernetes

In this blog, we will see how we can deploy the Elasticsearch, Fluent-bit, and Kibana (EFK) stack on Kubernetes. EFK stack’s prime objective is to reliably and securely retrieve data from the K8s cluster in any format, as well as to facilitate anytime searching, analyzing, and visualizing of the data.

What is EFK Stack?

EFK stands for Elasticsearch, Fluent bit, and Kibana.

Elasticsearch is a scalable and distributed search engine that is commonly used to store large amounts of log data. It is a NoSQL database. Its primary function is to store and retrieve logs from fluent bit.

Fluent Bit is a logging and metrics processor and forwarder that is extremely fast, lightweight, and highly scalable. Because of its performance-oriented design, it is simple to collect events from various sources and ship them to various destinations without complexity.

Kafka within EFK Monitoring

Today’s world is entirely internet-driven, be it in any field, we can get any product of our choice with one click.

Talking about e-commerce more in DevOps terms, the entire application/website is based on microservice architecture i.e. distributing a bulk application into smaller services to increase scalability, manageability & more process driven.

Hence, to maintain smaller services one of the important aspects is to enable their Monitoring.

One such commonly known stack is, EFK stack i.e. (Elasticsearch, Fluentd, Kibana) along with Kafka.

Kafka is basically an open-source event streaming platform and is currently used by many companies.

Question: Why use Kafka within EFK monitoring?

Answer: Well this is the first question that strikes many minds hence, in this blog we’ll focus on why to use Kafka, what are its benefits and how to integrate it with the EFK stack.

Interesting right? 🙂 let’s begin -:

Opstree’s Logging (EFK) Operator

Logging is a critical part of monitoring and there are a lot of tools for logs monitoring like Splunk, Sumologic, and Elasticsearch, etc. Since Kubernetes is becoming so much popular now, and running multiple applications and services on a Kubernetes cluster requires a centralized, cluster-level stack to analyze the logs created by pods.
One of the well-liked centralized logging solutions is the combination of multiple opensource tools i.e. Elasticsearch, Fluentd, and Kibana. In this blog, we will talk about setting up the logging stack on the Kubernetes cluster with our newly developed operator named “Logging Operator”.

Continue reading “Opstree’s Logging (EFK) Operator”