{"id":4341,"date":"2020-10-06T18:46:08","date_gmt":"2020-10-06T13:16:08","guid":{"rendered":"https:\/\/opstree.com\/blog\/\/?p=4341"},"modified":"2020-10-08T10:56:29","modified_gmt":"2020-10-08T05:26:29","slug":"elasticsearch-cluster-monitoring","status":"publish","type":"post","link":"https:\/\/opstree.com\/blog\/2020\/10\/06\/elasticsearch-cluster-monitoring\/","title":{"rendered":"Elasticsearch Cluster Monitoring"},"content":{"rendered":"\r\n

Elasticsearch is a scalable, full-text search and analytics engine based on Apache Lucene. It is Java-based and allows you to store, search, and analyze big volumes of data quickly in near real-time. It can search and index the document files in diverse formats. Lucene is the underlying technology that Elasticsearch uses for extremely fast data retrieval.<\/p>\r\n\r\n\r\n\r\n

As flexible, scalable, and useful as Elasticsearch is, monitoring your cluster can help you ensure that the cluster is appropriately sized and handles all operations efficiently.<\/p>\r\n

<\/p>\r\n\r\n\r\n\r\n

The Elastic Stack monitoring features provide a way to keep a pulse on the health and performance of your Elasticsearch cluster.<\/p>\r\n\r\n\r\n\r\n

Elasticsearch consists of many components that had better be monitored.<\/p>\r\n\r\n\r\n\r\n

ThreadPool, Operation Times, Operation stats, Documents, Segments, Shards, CPU, Index, Memory, Disk, Network, JVM Garbage Collector, and many more …<\/p>\r\n\r\n\r\n\r\n

Prerequisites<\/h2>\r\n\r\n\r\n\r\n

Prometheus Operator
Elasticsearch Cluster
Grafana
Helm v3.2.4<\/strong><\/p>\r\n\r\n\r\n\r\n

Setup ElasticSearch Exporter<\/h2>\r\n\r\n\r\n\r\n

The purpose of\u00a0exporters<\/strong> is to fetch metrics from Elasticsearch Stack and make those metrics available on the HTTP endpoint so that Prometheus serviceMonitor can pull metrics from that endpoint.<\/p>\r\n\r\n\r\n\r\n

Step 1: <\/strong>Set the default namespace to your-elastic-search-namespace:<\/p>\r\n\r\n\r\n\r\n

kubectl config set-context <your-kubernetes-cluster-name> --namespace=<your-elasticsearch-cluster-namespace><\/code><\/pre>\r\n\r\n\r\n\r\n
Step 2:<\/strong> Create Elasticsearch Exporter deployment<\/a> using:<\/p>\r\n\r\n\r\n\r\n
helm install elasticsearch-exporter --set es.uri=http:\/\/elasticsearch-node:9200 stable\/elasticsearch-exporter<\/code><\/pre>\r\n\r\n\r\n\r\n
We can check the exporter pod and service created by helm:<\/p>\r\n\r\n\r\n\r\n
kubectl describe pods $(kubectl get pods -lapp=elasticsearch-exporter -o=jsonpath='{.items[0].metadata.name}')<\/code><\/pre>\r\n\r\n\r\n\r\n
kubectl describe svc elasticsearch-exporter<\/code><\/pre>\r\n\r\n\r\n\r\n
Check exporter exposing metrics or not<\/h2>\r\n\r\n\r\n\r\n
Login into any pod and made a curl request to exporter http endpoint:<\/p>\r\n\r\n\r\n\r\n
kubectl exec -it elasticsearch-data-0 -c elasticsearch \/bin\/bash <\/code><\/pre>\r\n\r\n\r\n\r\n
curl http:\/\/<exporter-pod-ip>:9108\/metrics<\/code><\/pre>\r\n\r\n\r\n\r\n
\"\"<\/figure>\r\n\r\n\r\n\r\n
Or you can check metrics in Prometheus as shown below:<\/p>\r\n\r\n\r\n\r\n
Setup Prometheus Operator’s ServiceMonitor<\/h2>\r\n\r\n\r\n\r\n
We create Prometheus operator’s serviceMonitor which point to elasticsearch exporter’s service:<\/p>\r\n\r\n\r\n\r\n
apiVersion: monitoring.coreos.com\/v1\r\nkind: ServiceMonitor\r\nmetadata:\r\n  labels:\r\n    prometheus: k8s\r\n  name: elastic-cluster\r\n  namespace: monitoring\r\nspec:\r\n  selector:\r\n    matchLabels:\r\n      app: elasticsearch-exporter\r\n      chart: elasticsearch-exporter-3.0.0\r\n  namespaceSelector:\r\n    matchNames:\r\n    - elastc-cluster\r\n  endpoints:\r\n  - interval: 15s\r\n    port: http\r\n    scheme: http<\/code><\/pre>\r\n\r\n\r\n\r\n
After creating serviceMonitor, we can check metrics in Prometheus:<\/p>\r\n\r\n\r\n\r\n
\"\"<\/figure>\r\n\r\n\r\n\r\n
So we have successfully export elasticsearch cluster metrics to Prometheus. Now we use these metrics to create grafana dashboards<\/p>\r\n\r\n\r\n\r\n
Generate grafana dashboard<\/h2>\r\n\r\n\r\n\r\n
Go to Create->Import<\/p>\r\n\r\n\r\n\r\n
\"\"<\/figure>\r\n\r\n\r\n\r\n
Use Elasticsearch’s Grafana Dashboard<\/a><\/p>\r\n\r\n\r\n\r\n
\"\"<\/figure>\r\n
 <\/p>\r\n\r\n\r\n\r\n
\"\"<\/figure>\r\n
 <\/p>\r\n\r\n\r\n\r\n
\"\"<\/figure>\r\n
 <\/p>\r\n\r\n\r\n\r\n
\"\"<\/figure>\r\n\r\n\r\n\r\n
 <\/p>\r\n\r\n\r\n\r\n
We can further set up alerting based on the query used to plot data on Grafana and integrate it with Alert Manager and further to the supported notification channels.<\/p>\r\n\r\n\r\n\r\n
References:<\/h2>\r\n\r\n\r\n\r\n
https:\/\/github.com\/helm\/charts\/tree\/master\/stable\/elasticsearch-exporter<\/p>\r\n\r\n\r\n\r\n
https:\/\/grafana.com\/grafana\/dashboards\/2322<\/p>\r\n
 <\/p>\r\n
Opstree is an End to End DevOps solution provider<\/p>\r\n
CONTACT US<\/a><\/p>\r\n","protected":false},"excerpt":{"rendered":"
Elasticsearch is a scalable, full-text search and analytics engine based on Apache Lucene. It is Java-based and allows you to store, search, and analyze big volumes of data quickly in near real-time. It can search and index the document files in diverse formats. Lucene is the underlying technology that Elasticsearch uses for extremely fast data … Continue reading “Elasticsearch Cluster Monitoring”<\/span><\/a><\/p>\n","protected":false},"author":159459904,"featured_media":29900,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[28070474],"tags":[768739310,705927702,223796770,768739293,705927701],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/opstree.com\/blog\/wp-content\/uploads\/2025\/11\/DevSecOps-1.jpg","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pfDBOm-181","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/4341"}],"collection":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/users\/159459904"}],"replies":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/comments?post=4341"}],"version-history":[{"count":21,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/4341\/revisions"}],"predecessor-version":[{"id":4726,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/4341\/revisions\/4726"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media\/29900"}],"wp:attachment":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media?parent=4341"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/categories?post=4341"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/tags?post=4341"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}