{"id":11922,"date":"2023-07-18T11:48:58","date_gmt":"2023-07-18T06:18:58","guid":{"rendered":"https:\/\/opstree.com\/blog\/\/?p=11922"},"modified":"2023-07-18T12:30:52","modified_gmt":"2023-07-18T07:00:52","slug":"continuation-of-redis-throughput-and-management","status":"publish","type":"post","link":"https:\/\/opstree.com\/blog\/2023\/07\/18\/continuation-of-redis-throughput-and-management\/","title":{"rendered":"Continuation Of Redis Throughput and Management"},"content":{"rendered":"\n<p class=\"has-text-align-justify\">As promised in our previous blog on <a href=\"https:\/\/opstree.com\/blog\/\/2019\/04\/16\/redis-best-practices-and-performance-tuning\/\">Redis Performance tunning and Best practices<\/a>, we have explored more best practices and optimizations in Redis as a cache and database management system. This blog will share some new findings and optimizations we learned in our previous blog&#8217;s delta period.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">We know that Redis is a high-speed and flexible data storage that can fulfill different cache and database requirements. But if a system is not configured and tested correctly, even a fast and reliable one can quickly become limited. Here we will talk about the different needs of Redis as a system and how we can optimize it further to fully use it.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">So while consulting and collaborating with different Redis architects from <a href=\"https:\/\/redis.com\/\" target=\"_blank\" rel=\"noopener\">Redis Labs<\/a>, I learned different ways of designing a performance-grade, highly available, and secure Redis architecture. Based on my learning, I would like to categorize it into these dimensions:-<\/p>\n\n\n\n<ul>\n<li>Right-sizing and deployment of Redis setup.<\/li>\n\n\n\n<li>Proxy and connection pooling.<\/li>\n\n\n\n<li>Use the correct data type for storing keys.<\/li>\n\n\n\n<li>Sharding and replication strategy.<\/li>\n<\/ul>\n\n\n\n<!--more-->\n\n\n\n<h2 class=\"wp-block-heading\">Right-sizing and deployment of Redis setup<\/h2>\n\n\n\n<p class=\"has-text-align-justify\">To define the correct sizing and deployment model for Redis, we must first analyze the application&#8217;s workload and data access pattern. This insight will help determine the required memory capacity, CPU resources, and network bandwidth. Redis provides different types of deployment models:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2023\/07\/image-1.png?w=1024\" alt=\"\" class=\"wp-image-14706\" style=\"width:800px\" width=\"800\" \/><\/figure><\/div>\n\n\n<p class=\"has-text-align-justify\">In the above flow diagram, we have explained that each deployment model has a specific purpose and, accordingly, should be chosen. For example, suppose we run our workloads in production that require&lt; 32GB of cache in memory. In that case, we can opt for Redis Leader\/Follower architecture instead of Redis Cluster because the Redis Cluster cluster will increase your infrastructure and operations costs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Proxy and Connection Pooling in Redis<\/h2>\n\n\n\n<p class=\"has-text-align-justify\">Redis has a feature called &#8220;pipeline.&#8221; This feature allows a redis client to send different redis commands in an asynchronous method and read their response as a single outcome. Most of the proxy solutions provide connection pooling so that we don&#8217;t have to make a connection every time with the system.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2023\/07\/image-2.png?w=1024\" alt=\"\" class=\"wp-image-14722\" style=\"width:800px\" width=\"800\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-justify\">Also, one more significant benefit of using the proxy with Redis is that some SDKs, like .Net core, etc., don&#8217;t allow you to provide multiple endpoints in case of a Redis cluster. But a single endpoint can become a single point of failure. To overcome such issues proxy can be introduced because it will only send the requests to healthy Redis nodes like a Load Balancer.<\/p>\n\n\n\n<p>Some other benefits of using a proxy with Redis are:-<\/p>\n\n\n\n<ul>\n<li>High Availability<\/li>\n\n\n\n<li>Security<\/li>\n\n\n\n<li>Load Balancing<\/li>\n\n\n\n<li>Monitoring and metrics<\/li>\n\n\n\n<li>Rate limiting<\/li>\n<\/ul>\n\n\n\n<p>A few famous examples of Redis-supported proxies are the <a href=\"https:\/\/www.envoyproxy.io\/docs\/envoy\/latest\/intro\/arch_overview\/other_protocols\/redis\" target=\"_blank\" rel=\"noopener\">Envoy proxy<\/a> and <a href=\"https:\/\/github.com\/twitter\/twemproxy\" target=\"_blank\" rel=\"noopener\">Twemproxy<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The correct Data type for storing keys<\/h2>\n\n\n\n<p class=\"has-text-align-justify\">Again we have to comprehensively analyze our workload before defining the data type of keys. Redis supports different data types for storing the keys, such as:<\/p>\n\n\n\n<ul>\n<li>String<\/li>\n\n\n\n<li>Hash<\/li>\n\n\n\n<li>List<\/li>\n\n\n\n<li>Set<\/li>\n\n\n\n<li>Streams etc.<\/li>\n<\/ul>\n\n\n\n<p class=\"has-text-align-justify\">Each data type has different benefits and drawbacks. For example, choosing hash over string gives different advantages. Access\/updating\/deleting individual JSON fields on hashes is more accessible than the strings. We don&#8217;t have to get the whole string, decode, make changes, and set it again.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">If the size of your string object increases, we will suffer from network and bandwidth while transferring(get\/set) the whole object, whereas it&#8217;s far more performant in the case of hashes.  Hashes are more memory friendly than strings if you make an excellent benchmark to design your data size. As it is stated in the <a href=\"https:\/\/redis.io\/topics\/memory-optimization\" target=\"_blank\" rel=\"noopener\">documentation<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Sharding and Replication strategy<\/h2>\n\n\n\n<p class=\"has-text-align-justify\">In the Redis setup, we can use deployment models like replication, sharding, and replication + sharding. By default, these configurations are set to manage the low-scale data size. If we want to use Redis as a database system, we may need to further optimize these configurations. For example, the number of replicas should not exceed the number of followers. Otherwise, it will impact the performance of Redis.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">Also, Redis, by default, store the hash slot with 64 bytes of compressed data, which is too low. An ideal number of redis slot sizes can go up to 25GB, which can improve the performance of Redis by not creating too many hashes for few amount of data.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2023\/07\/image-3.png?w=1024\" alt=\"\" class=\"wp-image-14728\" style=\"width:800px\" width=\"800\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"has-text-align-justify\">So in this blog, we talked about optimizing Redis at different levels, like the operations side, which includes setup and management, and the development side, where the data insertion and structure should be managed. And that&#8217;s how an ideal optimization strategy works: optimization at every layer, including development, operations, security, etc.<\/p>\n\n\n\n<p class=\"has-text-align-justify\">I hope you guys enjoyed reading the blog. I will explore other features and optimization on Redis that can become part of our future blog series. If you have any feedback or suggestions, please comment on the blog.<\/p>\n\n\n\n<p>Till next time, Cheers!<\/p>\n\n\n\n<p><strong>Blog Pundits: <a rel=\"noreferrer noopener\" href=\"https:\/\/opstree.com\/blog\/\/author\/sandeep7c51ad81ba\/\" target=\"_blank\">Sandeep Rawat<\/a><\/strong><\/p>\n\n\n\n<p><strong>OpsTree is an End-to-End <a href=\"https:\/\/opstree.com\/services\/\" target=\"_blank\" rel=\"noreferrer noopener\">DevOps Solution<\/a> Provider.<\/strong><\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/opstree.com\/contact-us\/?utm_source=WordPress&amp;utm_medium=Blog&amp;utm_campaign=Continuation+Of+Redis+Throughput+and+Management\">Contact Us<\/a><\/div>\n<\/div>\n\n\n\n<p class=\"has-text-align-center\"><strong>Connect with Us<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-social-links aligncenter is-content-justification-center is-layout-flex wp-container-core-social-links-is-layout-1 wp-block-social-links-is-layout-flex\"><li class=\"wp-social-link wp-social-link-linkedin  wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/company\/opstree-solutions\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M19.7,3H4.3C3.582,3,3,3.582,3,4.3v15.4C3,20.418,3.582,21,4.3,21h15.4c0.718,0,1.3-0.582,1.3-1.3V4.3 C21,3.582,20.418,3,19.7,3z M8.339,18.338H5.667v-8.59h2.672V18.338z M7.004,8.574c-0.857,0-1.549-0.694-1.549-1.548 c0-0.855,0.691-1.548,1.549-1.548c0.854,0,1.547,0.694,1.547,1.548C8.551,7.881,7.858,8.574,7.004,8.574z M18.339,18.338h-2.669 v-4.177c0-0.996-0.017-2.278-1.387-2.278c-1.389,0-1.601,1.086-1.601,2.206v4.249h-2.667v-8.59h2.559v1.174h0.037 c0.356-0.675,1.227-1.387,2.526-1.387c2.703,0,3.203,1.779,3.203,4.092V18.338z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">LinkedIn<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-youtube  wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/www.youtube.com\/channel\/UCeLma6SpNYH7jjYKSBNSexw\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M21.8,8.001c0,0-0.195-1.378-0.795-1.985c-0.76-0.797-1.613-0.801-2.004-0.847c-2.799-0.202-6.997-0.202-6.997-0.202 h-0.009c0,0-4.198,0-6.997,0.202C4.608,5.216,3.756,5.22,2.995,6.016C2.395,6.623,2.2,8.001,2.2,8.001S2,9.62,2,11.238v1.517 c0,1.618,0.2,3.237,0.2,3.237s0.195,1.378,0.795,1.985c0.761,0.797,1.76,0.771,2.205,0.855c1.6,0.153,6.8,0.201,6.8,0.201 s4.203-0.006,7.001-0.209c0.391-0.047,1.243-0.051,2.004-0.847c0.6-0.607,0.795-1.985,0.795-1.985s0.2-1.618,0.2-3.237v-1.517 C22,9.62,21.8,8.001,21.8,8.001z M9.935,14.594l-0.001-5.62l5.404,2.82L9.935,14.594z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">YouTube<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-github  wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/github.com\/OpsTree\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M12,2C6.477,2,2,6.477,2,12c0,4.419,2.865,8.166,6.839,9.489c0.5,0.09,0.682-0.218,0.682-0.484 c0-0.236-0.009-0.866-0.014-1.699c-2.782,0.602-3.369-1.34-3.369-1.34c-0.455-1.157-1.11-1.465-1.11-1.465 c-0.909-0.62,0.069-0.608,0.069-0.608c1.004,0.071,1.532,1.03,1.532,1.03c0.891,1.529,2.341,1.089,2.91,0.833 c0.091-0.647,0.349-1.086,0.635-1.337c-2.22-0.251-4.555-1.111-4.555-4.943c0-1.091,0.39-1.984,1.03-2.682 C6.546,8.54,6.202,7.524,6.746,6.148c0,0,0.84-0.269,2.75,1.025C10.295,6.95,11.15,6.84,12,6.836 c0.85,0.004,1.705,0.114,2.504,0.336c1.909-1.294,2.748-1.025,2.748-1.025c0.546,1.376,0.202,2.394,0.1,2.646 c0.64,0.699,1.026,1.591,1.026,2.682c0,3.841-2.337,4.687-4.565,4.935c0.359,0.307,0.679,0.917,0.679,1.852 c0,1.335-0.012,2.415-0.012,2.741c0,0.269,0.18,0.579,0.688,0.481C19.138,20.161,22,16.416,22,12C22,6.477,17.523,2,12,2z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">GitHub<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-facebook  wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/www.facebook.com\/opstree\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M12 2C6.5 2 2 6.5 2 12c0 5 3.7 9.1 8.4 9.9v-7H7.9V12h2.5V9.8c0-2.5 1.5-3.9 3.8-3.9 1.1 0 2.2.2 2.2.2v2.5h-1.3c-1.2 0-1.6.8-1.6 1.6V12h2.8l-.4 2.9h-2.3v7C18.3 21.1 22 17 22 12c0-5.5-4.5-10-10-10z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Facebook<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-medium  wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/medium.com\/buildpiper\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M20.962,7.257l-5.457,8.867l-3.923-6.375l3.126-5.08c0.112-0.182,0.319-0.286,0.527-0.286c0.05,0,0.1,0.008,0.149,0.02 c0.039,0.01,0.078,0.023,0.114,0.041l5.43,2.715l0.006,0.003c0.004,0.002,0.007,0.006,0.011,0.008 C20.971,7.191,20.98,7.227,20.962,7.257z M9.86,8.592v5.783l5.14,2.57L9.86,8.592z M15.772,17.331l4.231,2.115 C20.554,19.721,21,19.529,21,19.016V8.835L15.772,17.331z M8.968,7.178L3.665,4.527C3.569,4.479,3.478,4.456,3.395,4.456 C3.163,4.456,3,4.636,3,4.938v11.45c0,0.306,0.224,0.669,0.498,0.806l4.671,2.335c0.12,0.06,0.234,0.088,0.337,0.088 c0.29,0,0.494-0.225,0.494-0.602V7.231C9,7.208,8.988,7.188,8.968,7.178z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Medium<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-twitter  wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/twitter.com\/opstreedevops\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M22.23,5.924c-0.736,0.326-1.527,0.547-2.357,0.646c0.847-0.508,1.498-1.312,1.804-2.27 c-0.793,0.47-1.671,0.812-2.606,0.996C18.324,4.498,17.257,4,16.077,4c-2.266,0-4.103,1.837-4.103,4.103 c0,0.322,0.036,0.635,0.106,0.935C8.67,8.867,5.647,7.234,3.623,4.751C3.27,5.357,3.067,6.062,3.067,6.814 c0,1.424,0.724,2.679,1.825,3.415c-0.673-0.021-1.305-0.206-1.859-0.513c0,0.017,0,0.034,0,0.052c0,1.988,1.414,3.647,3.292,4.023 c-0.344,0.094-0.707,0.144-1.081,0.144c-0.264,0-0.521-0.026-0.772-0.074c0.522,1.63,2.038,2.816,3.833,2.85 c-1.404,1.1-3.174,1.756-5.096,1.756c-0.331,0-0.658-0.019-0.979-0.057c1.816,1.164,3.973,1.843,6.29,1.843 c7.547,0,11.675-6.252,11.675-11.675c0-0.178-0.004-0.355-0.012-0.531C20.985,7.47,21.68,6.747,22.23,5.924z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Twitter<\/span><\/a><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>As promised in our previous blog on Redis Performance tunning and Best practices, we have explored more best practices and optimizations in Redis as a cache and database management system. This blog will share some new findings and optimizations we learned in our previous blog&#8217;s delta period. We know that Redis is a high-speed and &hellip; <a href=\"https:\/\/opstree.com\/blog\/2023\/07\/18\/continuation-of-redis-throughput-and-management\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Continuation Of Redis Throughput and Management&#8221;<\/span><\/a><\/p>\n","protected":false},"author":89038429,"featured_media":29900,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[28070474],"tags":[475,768739308,676319247,768739285,29632,1930,768739306,688976814,688729979],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/opstree.com\/blog\/wp-content\/uploads\/2025\/11\/DevSecOps-1.jpg","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pfDBOm-36i","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11922"}],"collection":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/users\/89038429"}],"replies":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/comments?post=11922"}],"version-history":[{"count":23,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11922\/revisions"}],"predecessor-version":[{"id":14764,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11922\/revisions\/14764"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media\/29900"}],"wp:attachment":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media?parent=11922"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/categories?post=11922"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/tags?post=11922"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}