![\"\"](\"https:\/\/blog.opstree.com\/wp-content\/uploads\/2024\/04\/Screenshot-2024-04-07-at-9.44.13\u202fAM-1024x489.png\")
<\/figure><\/div>\r\n\r\n\r\nIn the above diagram, we can observe several microservices and their dependencies on each other. Service A depends on service B, and service B depends on services F and E. Service F relies on a third-party application over which we have no control.<\/p>\r\n\r\n\r\n\r\n
<\/p>\r\n\r\n\r\n\r\n
Let’s consider that service A provides the following features:<\/p>\r\n\r\n\r\n\r\n
- \r\n
- Current account balance<\/li>\r\n\r\n\r\n\r\n
- Credit card details\u00a0<\/li>\r\n\r\n\r\n\r\n
- Investment details\u00a0<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n
Normally, the third-party application has an average response time of 1 second. However, for example purposes, let’s assume that its response time increases to 5 seconds due to issues on their end. Consequently, the requests from service F to the third-party application increase from 1 second to 5 seconds. This slowdown affects service B, causing it to hold requests in wait for a response. As a result, service A also experiences delays as it waits for service B to respond. Ultimately, this cascading impact affects other features of service A, when ideally, only the credit card feature should have been impacted because service B handles the feature for credit card details.<\/p>\r\n\r\n\r\n\r\n
Let\u2019s Implement the Circuit breaker<\/h2>\r\n\r\n\r\n\r\n
To address such scenarios, we can implement circuit breakers. Essentially, we can adhere to the following two principles:<\/p>\r\n\r\n\r\n\r\n
- \r\n
- Remote communication should fail quickly in the event of an issue, rather than consuming resources while waiting for a response that may never arrive.<\/li>\r\n\r\n\r\n\r\n
- If a dependency consistently fails, it is better to halt further requests until the dependency has recovered.<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n
![\"\"](\"https:\/\/blog.opstree.com\/wp-content\/uploads\/2024\/04\/Screenshot-2024-04-07-at-9.47.30\u202fAM-1024x486.png\")
<\/figure>\r\n\r\n\r\n\r\nNow, suppose we have a deadline timeout. In the event of any issue at the end of the third-party application, such as an increase in response time from 1 second to 5 seconds, our deadline timeout will trigger the connection to fail quickly. Consequently, only the feature of service A that is dependent on the third-party application will be affected. This ensures that there is no resource utilisation while waiting for a response from the third-party app, thereby preventing any adverse effects on other features of service A.<\/p>\r\n\r\n\r\n\r\n
How to Achieve Circuit Breaker With Istio?<\/h2>\r\n\r\n\r\n\r\n
Istio is a service mesh that integrates essential network services into the infrastructure, providing features such as service discovery and policy enforcement to regulate communication among services within the mesh.<\/p>\r\n\r\n\r\n\r\n
One of the advantages of Istio is its ability to handle failures by implementing circuit breakers. Networks are inherently unreliable, and Istio’s circuit breaker feature helps mitigate the cascading effects of failures.<\/p>\r\n\r\n\r\n\r\n
Let’s consider an example within my organization where we’ve implemented Istio’s circuit breaker. Our microservices interact with several third-party applications external to our organization. Since we lack control over these third-party applications, any outages they experience directly impact our microservices. When these external applications encounter issues, our dependent microservices face delays or fail to receive responses, resulting in a buildup of requests on our pods. This situation leads to a cascading impact on our entire application.<\/p>\r\n\r\n\r\n\r\n
To address this issue, we sought to implement Istio’s circuit breaker feature. However, upon reviewing Istio’s documentation and related blogs, we discovered that they typically recommend tripping the circuit breaker for the entire domain, which isn’t suitable for our scenario. In this blog, we’ll explain how we implemented API-based circuit breakers to address our specific needs.<\/p>\r\n\r\n\r\n\r\n
Istio Circuit Breaker Components<\/h2>\r\n\r\n\r\n\r\n
- \r\n
- ServiceEntry\u00a0<\/li>\r\n\r\n\r\n\r\n
- Virtual Service\u00a0<\/li>\r\n\r\n\r\n\r\n
- Destination Rule<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n
Walkthrough With a Sample Application<\/h2>\r\n\r\n\r\n\r\n
Let’s suppose we have two source applications (our microservices) and a destination application (a third-party application) with the following APIs.<\/p>\r\n\r\n\r\n\r\n
- \r\n
- \/dest\/200<\/li>\r\n\r\n\r\n\r\n
- \/dest\/500<\/li>\r\n\r\n\r\n\r\n
- \/dest\/504<\/li>\r\n\r\n\r\n\r\n
- \/dest\/502<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n
Similarly, the source application has corresponding APIs to interact with the destination application APIs.<\/p>\r\n\r\n\r\n\r\n
For our demonstration, we will continuously call the \/src\/200 API. Meanwhile, we will begin calling the \/src\/500 APIs to trigger the circuit breaker specifically for the \/src\/500 endpoint.<\/p>\r\n\r\n\r\n\r\n
![\"\"](\"https:\/\/blog.opstree.com\/wp-content\/uploads\/2024\/04\/Screenshot-2024-04-07-at-9.50.30\u202fAM.png\")
<\/figure>\r\n\r\n\r\n\r\nAs we can see we are continuously hitting the source API 200 and getting response 200 back.<\/p>\r\n\r\n\r\n\r\n
![\"\"](\"https:\/\/blog.opstree.com\/wp-content\/uploads\/2024\/04\/Screenshot-2024-04-07-at-9.52.55\u202fAM.png\")
<\/figure>\r\n\r\n\r\n\r\nHere, we observe that we are making requests to the \/src\/500 endpoint, and we are receiving 500 responses in return. Once the number of consecutive failures surpasses the maximum threshold set by Istio’s circuit breaker, it activates, injecting 503 status codes and halting further calls to the destination application for a specified duration.<\/p>\r\n\r\n\r\n\r\n
![\"\"](\"https:\/\/blog.opstree.com\/wp-content\/uploads\/2024\/04\/Screenshot-2024-04-07-at-9.52.23\u202fAM.png\")
<\/figure>\r\n\r\n\r\n\r\n
Here, we can observe in the Istio proxy log that Istio has injected the “UH – No Healthy Upstream<\/a>“<\/em><\/strong> flag, in addition to the 503 error, because the Istio circuit breaker has been triggered due to breaching the threshold of consecutive errors.<\/p>\r\n\r\n\r\n\r\n[2022-04-18T01:48:28.608Z] \"GET \/dest\/500 HTTP\/1.1\" 503 UH \"-\" \"-\" 0 19 0 - \"-\" \"Go-http-client\/1.1\" \"4b6f0a59-e4bc-9200-a1d7-f875c2f063bc\" \"dest-app.dsdev.\" \"-\" - - 54.182.0.45:80 10.0.31.122:33336 - dest-500\r\n<\/code><\/pre>\r\n\r\n\r\n\r\n![\"\"](\"https:\/\/blog.opstree.com\/wp-content\/uploads\/2024\/04\/image-1024x674.png\")
<\/figure>\r\n\r\n\r\n\r\nHere, we are making parallel calls to the source APIs, both the \/src\/200 and \/src\/500 endpoints. In the second part of the screenshot, we observe responses with the 500 status code. Once the circuit breaker is triggered, Istio injects 503 response codes, indicating service unavailable, and ceases further requests to the destination application.<\/p>\r\n\r\n\r\n\r\n
apiVersion: networking.istio.io\/v1alpha3\r\nkind: ServiceEntry\r\nmetadata:\r\n name: dest-app\r\n namespace: default\r\nspec:\r\n hosts:\r\n - dest-app.dsdev.xyx.com\r\n ports:\r\n - number: 80\r\n name: http-port\r\n protocol: HTTP\r\n - number: 443\r\n name: https-port\r\n protocol: HTTPS\r\n resolution: DNS\r\n location: MESH_EXTERNAL<\/code><\/pre>\r\n\r\n\r\n\r\nIn the snippet of code above, we are declaring the ServiceEntry for the domain of the destination application. For additional details about ServiceEntry, you can refer to the documentation. here<\/a>.<\/p>\r\n\r\n\r\n\r\n
apiVersion: networking.istio.io\/v1alpha3\r\nkind: VirtualService\r\nmetadata:\r\n name: dest-app\r\n namespace: default\r\nspec:\r\n hosts:\r\n - dest-app.dsdev.xyx.com\r\n http:\r\n - name: \"dest-200\"\r\n match:\r\n - uri:\r\n prefix: \/dest\/200\r\n port: 80\r\n route:\r\n - destination:\r\n host: dest-app.dsdev.xyx.com\r\n port:\r\n number: 443\r\n subset: health200\r\n - name: \"dest-500\"\r\n match:\r\n - uri:\r\n prefix: \/dest\/500\r\n port: 80\r\n route:\r\n - destination:\r\n host: dest-app.dsdev.xyx.com\r\n port:\r\n number: 443\r\n subset: error500<\/code><\/pre>\r\n\r\n\r\n\r\napiVersion: networking.istio.io\/v1alpha3\r\nkind: DestinationRule\r\nmetadata:\r\n name: dest-app\r\n namespace: default\r\nspec:\r\n host: dest-app.dsdev.xyx.com\r\n subsets:\r\n - name: health200\r\n trafficPolicy:\r\n tls:\r\n mode: SIMPLE\r\n sni: dest-app.dsdev.xyx.com\r\n outlierDetection:\r\n baseEjectionTime: 20s\r\n consecutiveErrors: 200\r\n interval: 20s\r\n maxEjectionPercent: 100\r\n - name: error500\r\n trafficPolicy:\r\n loadBalancer:\r\n simple: ROUND_ROBIN\r\n connectionPool:\r\n tcp:\r\n maxConnections: 1\r\n connectTimeout: 1000ms\r\n tls:\r\n mode: SIMPLE\r\n sni: dest-app.dsdev.xyx.com\r\n outlierDetection:\r\n baseEjectionTime: 1m\r\n consecutiveGatewayErrors: 3\r\n interval: 20s\r\n maxEjectionPercent: 100<\/code><\/pre>\r\n\r\n\r\n\r\nConclusion<\/h2>\r\n\r\n\r\n\r\n
When summarizing this blog, we observe that sometimes failure can serve as a viable option to manage during disasters, ensuring the continuous operation of your application even if certain features are not functioning as expected. With the assistance of Istio, it offers a convenient way to implement this without necessitating any changes to the code base.<\/p>\r\n\r\n\r\n\r\n
Let us know in the comment section if you have any questions or feedback.<\/p>\r\n\r\n\r\n\r\n
Before you\u00a0go:<\/h3>\r\n\r\n\r\n\r\n
Clap if you liked it \ud83d\udc4f, comment and share this article to reach more community \ud83e\uddde.<\/p>\r\n\r\n\r\n\r\n\r\n\r\n
OpsTree is an End-to-End DevOps Solution<\/a> Provider.<\/strong><\/p>\r\n\r\n\r\n\r\n