Basic Logging Setup of Loki Grafana

Let’s say you are the DevOps lead for a large e-commerce platform that runs on a microservices architecture with hundreds of services. You need to monitor the logs of all these services to quickly identify issues, troubleshoot problems, and optimize the system’s performance. You also want to be able to search and analyze logs across all services in real time and be alerted when any critical issues arise.

To address this scenario, you could use Grafana Loki as your centralized logging system. Loki is a lightweight and cost-effective solution that can handle high volumes of logs and store them in a distributed manner. You can configure each service to send logs to Loki, which will automatically index them and make them available for search and analysis.

Introduction

Loki and Grafana are two open-source projects that are commonly used together for log aggregation, analysis, and visualization.

Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be a cost-effective solution for storing and querying logs and uses a unique indexing approach to provide fast and efficient searching of log data. Loki is also highly extensible, allowing users to easily add custom logging drivers and integrate with other systems.

Grafana, on the other hand, is a popular open-source platform for visualizing and analyzing time-series data, including logs. It provides a powerful and flexible dashboarding system that allows users to create customized visualizations and alerts based on their log data. Grafana also integrates with many different data sources, including Loki, which makes it a great choice for log analysis and visualization.

Continue reading “Basic Logging Setup of Loki Grafana”

Opstree’s Logging (EFK) Operator

Logging is a critical part of monitoring and there are a lot of tools for logs monitoring like Splunk, Sumologic, and Elasticsearch, etc. Since Kubernetes is becoming so much popular now, and running multiple applications and services on a Kubernetes cluster requires a centralized, cluster-level stack to analyze the logs created by pods.
One of the well-liked centralized logging solutions is the combination of multiple opensource tools i.e. Elasticsearch, Fluentd, and Kibana. In this blog, we will talk about setting up the logging stack on the Kubernetes cluster with our newly developed operator named “Logging Operator”.

Continue reading “Opstree’s Logging (EFK) Operator”