{"id":11797,"date":"2022-10-04T12:35:17","date_gmt":"2022-10-04T07:05:17","guid":{"rendered":"https:\/\/opstree.com\/blog\/\/?p=11797"},"modified":"2022-10-04T12:35:17","modified_gmt":"2022-10-04T07:05:17","slug":"prometheus-and-grafana-on-kubernetes","status":"publish","type":"post","link":"https:\/\/opstree.com\/blog\/2022\/10\/04\/prometheus-and-grafana-on-kubernetes\/","title":{"rendered":"Deploying Prometheus and Grafana on Kubernetes"},"content":{"rendered":"\n<p>Monitoring a Kubernetes Cluster is the need of the hour for any application following a microservices architecture. There are a bunch of solutions that one can implement to monitor their Kubernetes workload and one of them is Prometheus and Grafana. This article will help you to deploy Prometheus and Grafana in your kubernetes cluster with the help of <strong>prometheus-operator<\/strong>.<\/p>\n\n\n\n<p>But before setting up these components let&#8217;s understand a bit about each of them.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Prometheus<\/h2>\n\n\n\n<p>Prometheus is a pull-based open-source monitoring and alerting tool originally built by <strong>SoundCloud<\/strong>. It works on a time-series database and scrapes metrics at a given interval from HTTP endpoints. After Kubernetes, Prometheus joined the Cloud Native Computing Foundation in 2016 as the second hosted project.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Alertmanager<\/h2>\n\n\n\n<p>The Alertmanager takes care of alerts sent by alerting tools such as the Prometheus server. It handles grouping, silencing, and routing them to the correct receiver integration such as email, PagerDuty, Slack, etc. It also supports the inhibition of alerts.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Grafana<\/h2>\n\n\n\n<p>Grafana is the visual representation of metrics collected by a data source which in our case happens to be Prometheus. We can create or import dashboards for grafana which will make use of promQL to visually represent metrics collected by Prometheus.<\/p>\n\n\n\n<!--more-->\n\n\n\n<h2 class=\"wp-block-heading\">Deploy Prometheus-Grafana on Kubernetes<\/h2>\n\n\n\n<p>To deploy prometheus-grafana stack on kubernetes we will make use of the <a href=\"https:\/\/github.com\/prometheus-operator\/kube-prometheus\" target=\"_blank\" rel=\"noopener\">kube-prometheus<\/a> project. The&nbsp;kube-prometheu<strong>s<\/strong>&nbsp;project is designed for kubernetes cluster monitoring and is customized to collect metrics from all Kubernetes objects, with a bunch of dashboards for Grafana and alerting rules for Alertmanager. <\/p>\n\n\n\n<p>Also, in this tutorial, we will be using minikube to deploy all the resources but you can follow along with any Kubernetes cluster. Now, let&#8217;s get started. <\/p>\n\n\n\n<p><strong>Step 1<\/strong> &#8211; Clone the kube-prometheus project. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/10\/screenshot-2022-10-04-120641.png?w=1024\" alt=\"\" class=\"wp-image-12153\" width=\"800\" \/><\/figure>\n\n\n\n<p><strong>Step 2 <\/strong>&#8211; Apply the server-side manifests.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/10\/balpreet-screenshot.png?w=1024\" alt=\"\" class=\"wp-image-12156\" width=\"800\" \/><\/figure>\n\n\n\n<p><strong>Step 3<\/strong> &#8211; Apply the monitoring stack manifests.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/10\/balpreet-screenshot-2.png?w=1024\" alt=\"\" class=\"wp-image-12157\" width=\"800\" \/><\/figure>\n\n\n\n<p>To see all the resources deployed by the above commands use:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl get all -n monitoring<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/09\/screenshot-from-2022-09-18-04-25-17.png?w=1024\" alt=\"\" class=\"wp-image-11894\" width=\"800\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/10\/balpreet-screenshot-3.png?w=1024\" alt=\"\" class=\"wp-image-12159\" width=\"800\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Accessing Prometheus, Alertmanager, and Grafana through UI.<\/h2>\n\n\n\n<p>To access these resources through UI we would have to change the service type from ClusterIP to NodePort.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kubectl edit svc grafana -n monitoring<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/10\/balpreet-screenshot-4.png?w=1024\" alt=\"\" class=\"wp-image-12160\" width=\"800\" \/><\/figure>\n\n\n\n<p>If you are using minikube run the following command to access Grafana Dashboards in your browser.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>minikube service grafana --url -n monitoring<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/09\/screenshot-from-2022-09-18-04-35-00.png?w=1024\" alt=\"\" class=\"wp-image-11901\" width=\"800\" \/><\/figure>\n\n\n\n<p>Paste this url in your browser window to open Grafana.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/10\/balpreet-screenshot-5.png?w=1024\" alt=\"\" class=\"wp-image-12161\" width=\"800\" \/><\/figure>\n\n\n\n<p>By default, the username and password are set to admin, and on your first login, you will be asked to update that. After that, you will be able to access the Dashboards.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/09\/screenshot-from-2022-09-18-04-37-04.png?w=1024\" alt=\"\" class=\"wp-image-11906\" width=\"800\" \/><figcaption class=\"wp-element-caption\"><strong>Grafana<\/strong> <\/figcaption><\/figure>\n\n\n\n<p>Now repeat the same steps for Prometheus and Alertmanager. Edit the service prometheus-k8s for prometheus  and alertmanager-main for alertmanager.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/09\/screenshot-from-2022-09-18-15-12-15.png?w=1024\" alt=\"\" class=\"wp-image-11909\" width=\"800\" \/><figcaption class=\"wp-element-caption\"><strong>Prometheus<\/strong><\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2022\/10\/balpreet-screenshot-6.png?w=1024\" alt=\"\" class=\"wp-image-12162\" width=\"800\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p>Kube-prometheus stack makes use of stateful sets for prometheus and alertmanager. Since both of them are stateful applications, deployment for grafana and kube-state-metrics as both are stateless applications and daemonset for node exporter as it has to be deployed on every node. In this blog, we learned how to deploy prometheus-grafana stack on kubernetes with kube-prometheus stack telling about the components, and in the next part of the blog, we will dive into detail about the setup and rules.<\/p>\n\n\n\n<p><strong>Blog Pundits:<\/strong> <a href=\"https:\/\/opstree.com\/blog\/\/author\/deepakgupta97\/\"><strong>Deepak Gupta<\/strong><\/a> and <a href=\"https:\/\/opstree.com\/blog\/\/author\/sandeep7c51ad81ba\/\"><strong>Sandeep Rawat<\/strong><\/a><\/p>\n\n\n\n<p><strong><a href=\"https:\/\/www.opstree.com\/contact-us?utm_source=Wordpress&amp;utm_medium=Blog&amp;utm_campaign=Deploying_Prometheus_and_Grafana_on_Kubernetes\" target=\"_blank\" rel=\"noreferrer noopener\">OpsTree<\/a><\/strong> is an End-to-End DevOps Solution Provider.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.opstree.com\/contact-us?utm_source=Wordpress&amp;utm_medium=Blog&amp;utm_campaign=Deploying_Prometheus_and_Grafana_on_Kubernetes\" target=\"_blank\" rel=\"noreferrer noopener\">Contact Us<\/a><\/div>\n<\/div>\n\n\n\n<p class=\"has-text-align-center\"><strong>Connect with Us<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-social-links aligncenter is-content-justification-center is-layout-flex wp-container-core-social-links-is-layout-1 wp-block-social-links-is-layout-flex\"><li class=\"wp-social-link wp-social-link-linkedin  wp-block-social-link\"><a href=\"https:\/\/www.linkedin.com\/company\/opstree-solutions\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M19.7,3H4.3C3.582,3,3,3.582,3,4.3v15.4C3,20.418,3.582,21,4.3,21h15.4c0.718,0,1.3-0.582,1.3-1.3V4.3 C21,3.582,20.418,3,19.7,3z M8.339,18.338H5.667v-8.59h2.672V18.338z M7.004,8.574c-0.857,0-1.549-0.694-1.549-1.548 c0-0.855,0.691-1.548,1.549-1.548c0.854,0,1.547,0.694,1.547,1.548C8.551,7.881,7.858,8.574,7.004,8.574z M18.339,18.338h-2.669 v-4.177c0-0.996-0.017-2.278-1.387-2.278c-1.389,0-1.601,1.086-1.601,2.206v4.249h-2.667v-8.59h2.559v1.174h0.037 c0.356-0.675,1.227-1.387,2.526-1.387c2.703,0,3.203,1.779,3.203,4.092V18.338z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">LinkedIn<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-youtube  wp-block-social-link\"><a href=\"https:\/\/www.youtube.com\/channel\/UCeLma6SpNYH7jjYKSBNSexw\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M21.8,8.001c0,0-0.195-1.378-0.795-1.985c-0.76-0.797-1.613-0.801-2.004-0.847c-2.799-0.202-6.997-0.202-6.997-0.202 h-0.009c0,0-4.198,0-6.997,0.202C4.608,5.216,3.756,5.22,2.995,6.016C2.395,6.623,2.2,8.001,2.2,8.001S2,9.62,2,11.238v1.517 c0,1.618,0.2,3.237,0.2,3.237s0.195,1.378,0.795,1.985c0.761,0.797,1.76,0.771,2.205,0.855c1.6,0.153,6.8,0.201,6.8,0.201 s4.203-0.006,7.001-0.209c0.391-0.047,1.243-0.051,2.004-0.847c0.6-0.607,0.795-1.985,0.795-1.985s0.2-1.618,0.2-3.237v-1.517 C22,9.62,21.8,8.001,21.8,8.001z M9.935,14.594l-0.001-5.62l5.404,2.82L9.935,14.594z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">YouTube<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-github  wp-block-social-link\"><a href=\"https:\/\/github.com\/OpsTree\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M12,2C6.477,2,2,6.477,2,12c0,4.419,2.865,8.166,6.839,9.489c0.5,0.09,0.682-0.218,0.682-0.484 c0-0.236-0.009-0.866-0.014-1.699c-2.782,0.602-3.369-1.34-3.369-1.34c-0.455-1.157-1.11-1.465-1.11-1.465 c-0.909-0.62,0.069-0.608,0.069-0.608c1.004,0.071,1.532,1.03,1.532,1.03c0.891,1.529,2.341,1.089,2.91,0.833 c0.091-0.647,0.349-1.086,0.635-1.337c-2.22-0.251-4.555-1.111-4.555-4.943c0-1.091,0.39-1.984,1.03-2.682 C6.546,8.54,6.202,7.524,6.746,6.148c0,0,0.84-0.269,2.75,1.025C10.295,6.95,11.15,6.84,12,6.836 c0.85,0.004,1.705,0.114,2.504,0.336c1.909-1.294,2.748-1.025,2.748-1.025c0.546,1.376,0.202,2.394,0.1,2.646 c0.64,0.699,1.026,1.591,1.026,2.682c0,3.841-2.337,4.687-4.565,4.935c0.359,0.307,0.679,0.917,0.679,1.852 c0,1.335-0.012,2.415-0.012,2.741c0,0.269,0.18,0.579,0.688,0.481C19.138,20.161,22,16.416,22,12C22,6.477,17.523,2,12,2z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">GitHub<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-facebook  wp-block-social-link\"><a href=\"https:\/\/www.facebook.com\/opstree\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M12 2C6.5 2 2 6.5 2 12c0 5 3.7 9.1 8.4 9.9v-7H7.9V12h2.5V9.8c0-2.5 1.5-3.9 3.8-3.9 1.1 0 2.2.2 2.2.2v2.5h-1.3c-1.2 0-1.6.8-1.6 1.6V12h2.8l-.4 2.9h-2.3v7C18.3 21.1 22 17 22 12c0-5.5-4.5-10-10-10z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Facebook<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-medium  wp-block-social-link\"><a href=\"https:\/\/medium.com\/buildpiper\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M20.962,7.257l-5.457,8.867l-3.923-6.375l3.126-5.08c0.112-0.182,0.319-0.286,0.527-0.286c0.05,0,0.1,0.008,0.149,0.02 c0.039,0.01,0.078,0.023,0.114,0.041l5.43,2.715l0.006,0.003c0.004,0.002,0.007,0.006,0.011,0.008 C20.971,7.191,20.98,7.227,20.962,7.257z M9.86,8.592v5.783l5.14,2.57L9.86,8.592z M15.772,17.331l4.231,2.115 C20.554,19.721,21,19.529,21,19.016V8.835L15.772,17.331z M8.968,7.178L3.665,4.527C3.569,4.479,3.478,4.456,3.395,4.456 C3.163,4.456,3,4.636,3,4.938v11.45c0,0.306,0.224,0.669,0.498,0.806l4.671,2.335c0.12,0.06,0.234,0.088,0.337,0.088 c0.29,0,0.494-0.225,0.494-0.602V7.231C9,7.208,8.988,7.188,8.968,7.178z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Medium<\/span><\/a><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Monitoring a Kubernetes Cluster is the need of the hour for any application following a microservices architecture. There are a bunch of solutions that one can implement to monitor their Kubernetes workload and one of them is Prometheus and Grafana. This article will help you to deploy Prometheus and Grafana in your kubernetes cluster with &hellip; <a href=\"https:\/\/opstree.com\/blog\/2022\/10\/04\/prometheus-and-grafana-on-kubernetes\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Deploying Prometheus and Grafana on Kubernetes&#8221;<\/span><\/a><\/p>\n","protected":false},"author":218078523,"featured_media":29900,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[28070474],"tags":[44070,768739308,676319247,223796770,768739309,768739293,16279507,768739287,4996032],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/opstree.com\/blog\/wp-content\/uploads\/2025\/11\/DevSecOps-1.jpg","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pfDBOm-34h","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11797"}],"collection":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/users\/218078523"}],"replies":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/comments?post=11797"}],"version-history":[{"count":24,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11797\/revisions"}],"predecessor-version":[{"id":12166,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11797\/revisions\/12166"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media\/29900"}],"wp:attachment":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media?parent=11797"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/categories?post=11797"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/tags?post=11797"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}