{"id":11922,"date":"2023-07-18T11:48:58","date_gmt":"2023-07-18T06:18:58","guid":{"rendered":"https:\/\/opstree.com\/blog\/\/?p=11922"},"modified":"2023-07-18T12:30:52","modified_gmt":"2023-07-18T07:00:52","slug":"continuation-of-redis-throughput-and-management","status":"publish","type":"post","link":"https:\/\/opstree.com\/blog\/continuation-of-redis-throughput-and-management\/","title":{"rendered":"Continuation Of Redis Throughput and Management"},"content":{"rendered":"\n<p class=\"has-text-align-justify wp-block-paragraph\">As promised in our previous blog on <a href=\"https:\/\/opstree.com\/blog\/\/2019\/04\/16\/redis-best-practices-and-performance-tuning\/\">Redis Performance tunning and Best practices<\/a>, we have explored more best practices and optimizations in Redis as a cache and database management system. This blog will share some new findings and optimizations we learned in our previous blog&#8217;s delta period.<\/p>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">We know that Redis is a high-speed and flexible data storage that can fulfill different cache and database requirements. But if a system is not configured and tested correctly, even a fast and reliable one can quickly become limited. Here we will talk about the different needs of Redis as a system and how we can optimize it further to fully use it.<\/p>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">So while consulting and collaborating with different Redis architects from <a href=\"https:\/\/redis.com\/\" target=\"_blank\" rel=\"noopener\">Redis Labs<\/a>, I learned different ways of designing a performance-grade, highly available, and secure Redis architecture. Based on my learning, I would like to categorize it into these dimensions:-<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Right-sizing and deployment of Redis setup.<\/li>\n\n\n\n<li>Proxy and connection pooling.<\/li>\n\n\n\n<li>Use the correct data type for storing keys.<\/li>\n\n\n\n<li>Sharding and replication strategy.<\/li>\n<\/ul>\n\n\n\n<!--more-->\n\n\n\n<h2 class=\"wp-block-heading\">Right-sizing and deployment of Redis setup<\/h2>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">To define the correct sizing and deployment model for Redis, we must first analyze the application&#8217;s workload and data access pattern. This insight will help determine the required memory capacity, CPU resources, and network bandwidth. Redis provides different types of deployment models:<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2023\/07\/image-1.png?w=1024\" alt=\"\" class=\"wp-image-14706\" style=\"width:800px\" width=\"800\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">In the above flow diagram, we have explained that each deployment model has a specific purpose and, accordingly, should be chosen. For example, suppose we run our workloads in production that require&lt; 32GB of cache in memory. In that case, we can opt for Redis Leader\/Follower architecture instead of Redis Cluster because the Redis Cluster cluster will increase your infrastructure and operations costs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Proxy and Connection Pooling in Redis<\/h2>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">Redis has a feature called &#8220;pipeline.&#8221; This feature allows a redis client to send different redis commands in an asynchronous method and read their response as a single outcome. Most of the proxy solutions provide connection pooling so that we don&#8217;t have to make a connection every time with the system.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2023\/07\/image-2.png?w=1024\" alt=\"\" class=\"wp-image-14722\" style=\"width:800px\" width=\"800\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">Also, one more significant benefit of using the proxy with Redis is that some SDKs, like .Net core, etc., don&#8217;t allow you to provide multiple endpoints in case of a Redis cluster. But a single endpoint can become a single point of failure. To overcome such issues proxy can be introduced because it will only send the requests to healthy Redis nodes like a Load Balancer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Some other benefits of using a proxy with Redis are:-<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>High Availability<\/li>\n\n\n\n<li>Security<\/li>\n\n\n\n<li>Load Balancing<\/li>\n\n\n\n<li>Monitoring and metrics<\/li>\n\n\n\n<li>Rate limiting<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A few famous examples of Redis-supported proxies are the <a href=\"https:\/\/www.envoyproxy.io\/docs\/envoy\/latest\/intro\/arch_overview\/other_protocols\/redis\" target=\"_blank\" rel=\"noopener\">Envoy proxy<\/a> and <a href=\"https:\/\/github.com\/twitter\/twemproxy\" target=\"_blank\" rel=\"noopener\">Twemproxy<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The correct Data type for storing keys<\/h2>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">Again we have to comprehensively analyze our workload before defining the data type of keys. Redis supports different data types for storing the keys, such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>String<\/li>\n\n\n\n<li>Hash<\/li>\n\n\n\n<li>List<\/li>\n\n\n\n<li>Set<\/li>\n\n\n\n<li>Streams etc.<\/li>\n<\/ul>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">Each data type has different benefits and drawbacks. For example, choosing hash over string gives different advantages. Access\/updating\/deleting individual JSON fields on hashes is more accessible than the strings. We don&#8217;t have to get the whole string, decode, make changes, and set it again.<\/p>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">If the size of your string object increases, we will suffer from network and bandwidth while transferring(get\/set) the whole object, whereas it&#8217;s far more performant in the case of hashes.  Hashes are more memory friendly than strings if you make an excellent benchmark to design your data size. As it is stated in the <a href=\"https:\/\/redis.io\/topics\/memory-optimization\" target=\"_blank\" rel=\"noopener\">documentation<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Sharding and Replication strategy<\/h2>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">In the Redis setup, we can use deployment models like replication, sharding, and replication + sharding. By default, these configurations are set to manage the low-scale data size. If we want to use Redis as a database system, we may need to further optimize these configurations. For example, the number of replicas should not exceed the number of followers. Otherwise, it will impact the performance of Redis.<\/p>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">Also, Redis, by default, store the hash slot with 64 bytes of compressed data, which is too low. An ideal number of redis slot sizes can go up to 25GB, which can improve the performance of Redis by not creating too many hashes for few amount of data.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img decoding=\"async\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2023\/07\/image-3.png?w=1024\" alt=\"\" class=\"wp-image-14728\" style=\"width:800px\" width=\"800\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">So in this blog, we talked about optimizing Redis at different levels, like the operations side, which includes setup and management, and the development side, where the data insertion and structure should be managed. And that&#8217;s how an ideal optimization strategy works: optimization at every layer, including development, operations, security, etc.<\/p>\n\n\n\n<p class=\"has-text-align-justify wp-block-paragraph\">I hope you guys enjoyed reading the blog. I will explore other features and optimization on Redis that can become part of our future blog series. If you have any feedback or suggestions, please comment on the blog.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Till next time, Cheers!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Blog Pundits: <a rel=\"noreferrer noopener\" href=\"https:\/\/opstree.com\/blog\/\/author\/sandeep7c51ad81ba\/\" target=\"_blank\">Sandeep Rawat<\/a><\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>OpsTree is an End-to-End <a href=\"https:\/\/opstree.com\/services\/\" target=\"_blank\" rel=\"noreferrer noopener\">DevOps Solution<\/a> Provider.<\/strong><\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/opstree.com\/contact-us\/?utm_source=WordPress&amp;utm_medium=Blog&amp;utm_campaign=Continuation+Of+Redis+Throughput+and+Management\">Contact Us<\/a><\/div>\n<\/div>\n\n\n\n<p class=\"has-text-align-center wp-block-paragraph\"><strong>Connect with Us<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-social-links aligncenter is-content-justification-center is-layout-flex wp-container-core-social-links-is-layout-fe0a7de2 wp-block-social-links-is-layout-flex\"><li class=\"wp-social-link wp-social-link-linkedin wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/www.linkedin.com\/company\/opstree-solutions\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M19.7,3H4.3C3.582,3,3,3.582,3,4.3v15.4C3,20.418,3.582,21,4.3,21h15.4c0.718,0,1.3-0.582,1.3-1.3V4.3 C21,3.582,20.418,3,19.7,3z M8.339,18.338H5.667v-8.59h2.672V18.338z M7.004,8.574c-0.857,0-1.549-0.694-1.549-1.548 c0-0.855,0.691-1.548,1.549-1.548c0.854,0,1.547,0.694,1.547,1.548C8.551,7.881,7.858,8.574,7.004,8.574z M18.339,18.338h-2.669 v-4.177c0-0.996-0.017-2.278-1.387-2.278c-1.389,0-1.601,1.086-1.601,2.206v4.249h-2.667v-8.59h2.559v1.174h0.037 c0.356-0.675,1.227-1.387,2.526-1.387c2.703,0,3.203,1.779,3.203,4.092V18.338z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">LinkedIn<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-youtube wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/www.youtube.com\/channel\/UCeLma6SpNYH7jjYKSBNSexw\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M21.8,8.001c0,0-0.195-1.378-0.795-1.985c-0.76-0.797-1.613-0.801-2.004-0.847c-2.799-0.202-6.997-0.202-6.997-0.202 h-0.009c0,0-4.198,0-6.997,0.202C4.608,5.216,3.756,5.22,2.995,6.016C2.395,6.623,2.2,8.001,2.2,8.001S2,9.62,2,11.238v1.517 c0,1.618,0.2,3.237,0.2,3.237s0.195,1.378,0.795,1.985c0.761,0.797,1.76,0.771,2.205,0.855c1.6,0.153,6.8,0.201,6.8,0.201 s4.203-0.006,7.001-0.209c0.391-0.047,1.243-0.051,2.004-0.847c0.6-0.607,0.795-1.985,0.795-1.985s0.2-1.618,0.2-3.237v-1.517 C22,9.62,21.8,8.001,21.8,8.001z M9.935,14.594l-0.001-5.62l5.404,2.82L9.935,14.594z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">YouTube<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-github wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/github.com\/OpsTree\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M12,2C6.477,2,2,6.477,2,12c0,4.419,2.865,8.166,6.839,9.489c0.5,0.09,0.682-0.218,0.682-0.484 c0-0.236-0.009-0.866-0.014-1.699c-2.782,0.602-3.369-1.34-3.369-1.34c-0.455-1.157-1.11-1.465-1.11-1.465 c-0.909-0.62,0.069-0.608,0.069-0.608c1.004,0.071,1.532,1.03,1.532,1.03c0.891,1.529,2.341,1.089,2.91,0.833 c0.091-0.647,0.349-1.086,0.635-1.337c-2.22-0.251-4.555-1.111-4.555-4.943c0-1.091,0.39-1.984,1.03-2.682 C6.546,8.54,6.202,7.524,6.746,6.148c0,0,0.84-0.269,2.75,1.025C10.295,6.95,11.15,6.84,12,6.836 c0.85,0.004,1.705,0.114,2.504,0.336c1.909-1.294,2.748-1.025,2.748-1.025c0.546,1.376,0.202,2.394,0.1,2.646 c0.64,0.699,1.026,1.591,1.026,2.682c0,3.841-2.337,4.687-4.565,4.935c0.359,0.307,0.679,0.917,0.679,1.852 c0,1.335-0.012,2.415-0.012,2.741c0,0.269,0.18,0.579,0.688,0.481C19.138,20.161,22,16.416,22,12C22,6.477,17.523,2,12,2z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">GitHub<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-facebook wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/www.facebook.com\/opstree\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M12 2C6.5 2 2 6.5 2 12c0 5 3.7 9.1 8.4 9.9v-7H7.9V12h2.5V9.8c0-2.5 1.5-3.9 3.8-3.9 1.1 0 2.2.2 2.2.2v2.5h-1.3c-1.2 0-1.6.8-1.6 1.6V12h2.8l-.4 2.9h-2.3v7C18.3 21.1 22 17 22 12c0-5.5-4.5-10-10-10z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Facebook<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-medium wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/medium.com\/buildpiper\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M13.2,12c0,3-2.4,5.4-5.3,5.4S2.6,15,2.6,12s2.4-5.4,5.3-5.4S13.2,9,13.2,12 M19.1,12c0,2.8-1.2,5-2.7,5s-2.7-2.3-2.7-5s1.2-5,2.7-5C17.9,7,19.1,9.2,19.1,12 M21.4,12c0,2.5-0.4,4.5-0.9,4.5c-0.5,0-0.9-2-0.9-4.5s0.4-4.5,0.9-4.5C21,7.5,21.4,9.5,21.4,12\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Medium<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-twitter wp-block-social-link\"><a rel=\"noopener nofollow\" target=\"_blank\" href=\"https:\/\/twitter.com\/opstreedevops\" class=\"wp-block-social-link-anchor\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M22.23,5.924c-0.736,0.326-1.527,0.547-2.357,0.646c0.847-0.508,1.498-1.312,1.804-2.27 c-0.793,0.47-1.671,0.812-2.606,0.996C18.324,4.498,17.257,4,16.077,4c-2.266,0-4.103,1.837-4.103,4.103 c0,0.322,0.036,0.635,0.106,0.935C8.67,8.867,5.647,7.234,3.623,4.751C3.27,5.357,3.067,6.062,3.067,6.814 c0,1.424,0.724,2.679,1.825,3.415c-0.673-0.021-1.305-0.206-1.859-0.513c0,0.017,0,0.034,0,0.052c0,1.988,1.414,3.647,3.292,4.023 c-0.344,0.094-0.707,0.144-1.081,0.144c-0.264,0-0.521-0.026-0.772-0.074c0.522,1.63,2.038,2.816,3.833,2.85 c-1.404,1.1-3.174,1.756-5.096,1.756c-0.331,0-0.658-0.019-0.979-0.057c1.816,1.164,3.973,1.843,6.29,1.843 c7.547,0,11.675-6.252,11.675-11.675c0-0.178-0.004-0.355-0.012-0.531C20.985,7.47,21.68,6.747,22.23,5.924z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Twitter<\/span><\/a><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>As promised in our previous blog on Redis Performance tunning and Best practices, we have explored more best practices and optimizations in Redis as a cache and database management system. This blog will share some new findings and optimizations we learned in our previous blog&#8217;s delta period. We know that Redis is a high-speed and [&hellip;]<\/p>\n","protected":false},"author":89038429,"featured_media":29900,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[28070474],"tags":[475,768739308,676319247,768739285,29632,1930,768739306,688976814,688729979],"class_list":["post-11922","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops","tag-database","tag-devops","tag-devops-solutioning","tag-linux","tag-middleware","tag-performance","tag-redis","tag-redisarchitecture","tag-rediscluster"],"blocksy_meta":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/opstree.com\/blog\/wp-content\/uploads\/2025\/11\/DevSecOps-1.jpg","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pfDBOm-36i","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11922","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/users\/89038429"}],"replies":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/comments?post=11922"}],"version-history":[{"count":23,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11922\/revisions"}],"predecessor-version":[{"id":14764,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/11922\/revisions\/14764"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media\/29900"}],"wp:attachment":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media?parent=11922"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/categories?post=11922"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/tags?post=11922"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}