{"id":8533,"date":"2021-12-28T16:54:18","date_gmt":"2021-12-28T11:24:18","guid":{"rendered":"https:\/\/opstree.com\/blog\/\/?p=8533"},"modified":"2021-12-29T12:45:44","modified_gmt":"2021-12-29T07:15:44","slug":"kafkas-solution-event-driven-architecture-otkafkadiaries","status":"publish","type":"post","link":"https:\/\/opstree.com\/blog\/2021\/12\/28\/kafkas-solution-event-driven-architecture-otkafkadiaries\/","title":{"rendered":"Kafka\u2019s Solution : Event Driven Architecture: OTKafkaDiaries"},"content":{"rendered":"\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>Heroism often results as a response to extreme events.<\/p><cite>James Geary<\/cite><\/blockquote>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"event-driven-architecture\">Event Driven Architecture:<\/h3>\n\n\n\n<p>Modern digital businesses work in real-time based events. Event-driven architecture is based on the design principle which follows loose-coupling and message-driven architecture. This Architecture helps to publish events\/messages that applications and services can consume, and then perform an action based upon those events.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"where-are-we-today\">Where are we Today?<\/h3>\n\n\n\n<p>Back in the days when we started implementing microservices, were focused more on service decoupling, communication, and security which we were going to handle in such a system.&nbsp;<\/p>\n\n\n\n<!--more-->\n\n\n\n<p>Although we were able to achieve most of the things, the challenge today has shifted to delivering responses in real-time on a mass scale. So, even after decoupling the service into microservices, we can get service timeout errors when our depending service fails to return the response when it\u2019s needed.<\/p>\n\n\n\n<p>It sounds very interesting from an architectural perspective to solve the problem of real-time as it is such an extreme challenge.  Like, today, we are seeing rising new use cases with microservices, IoT, event hubs, cloud, machine learning, and many more software solutions.<\/p>\n\n\n\n<p>An increase in demand for such apps has led today\u2019s era to EVENT Driven instead of data-driven.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"why-use-events\">Why Use events?<\/h3>\n\n\n\n<p>It took the era of big data for us to realize that while having lots of data is useful, things get slower as you have more of it. And yet, we are collecting more of it. Over the last few years, there has been a change in organizations that they must become <em>real-time<\/em>; to become real-time, they must be <em>event-driven.<\/em>&nbsp;<\/p>\n\n\n\n<p>For Instance,<\/p>\n\n\n\n<p>Any Application that has to send\/Notify its user; should not wait for the response all the time. They can just do it in real-time by publishing it on some topics and at the same time consumer applications in the next end can get the request\/message from the producing application. (<a href=\"https:\/\/opstree.com\/blog\/\/2021\/02\/02\/introduction-to-kafka-otkafkadiaries\/\">read pub-sub intro<\/a>)<\/p>\n\n\n\n<p>Social platform LinkedIn found this issue and developed a way of solving it. They created an in-house application named \u201c<strong>Kafka<\/strong>\u201d<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter is-resized\"><img decoding=\"async\" src=\"https:\/\/lh3.googleusercontent.com\/KsRhonLWg_KRC_Mk4YrrQMef5aXY-dRHpxxduaHNQ0XttdspYIhKtiC_XpljFK074y2q1uqRMXxaFfwNvS1W8tc15OyZbllgUVEW4iYMO5IaBp0ao_BuK_-P2AOXDgViMyyu3flZ=s0\" alt=\"\" width=\"-17\" height=\"-9\" \/><\/figure><\/div>\n\n\n\n<h6 class=\"wp-block-heading\" id=\"one-example-because-that-s-what-the-reader-demands\">One Example because that\u2019s what the reader demands.<\/h6>\n\n\n\n<p>Brush Vocab<strong>:&nbsp;<\/strong><br><\/p>\n\n\n\n<p><strong><em>Events<\/em><\/strong><em>&nbsp; &#8211; <\/em><em>something which is going to happen<\/em><\/p>\n\n\n\n<p><strong><em>Trigger<\/em><\/strong><em> &#8211; an event that<\/em> is responsible to provoke a behavior<em>&nbsp;<\/em><\/p>\n\n\n\n<p><strong><em>Behavior<\/em><\/strong><em> &#8211; <\/em><em>a reaction to an event.<\/em><\/p>\n\n\n\n<p>The <strong>Event<\/strong> example is simple: Check your calendar \ud83e\udd13.&nbsp; No? Okay then let&#8217;s see a real one where we are using event-driven architecture.<\/p>\n\n\n\n<p>In most of the gaming applications, there are events set for most of the user action,<\/p>\n\n\n\n<p>while playing GTA(a recent game I played) I checked that whenever I was fired at by some goons\/police my health gets decreased (obviously), but as soon as I pick a vehicle my health gets increased, or another time when police are chasing me, and I was running away from them, there was siren walkie-talkie voice all over the game sound, (people will be gone from the game I cannot see any normal people, NPC, in the game other than me and the police ) but once I hide or get far from police radar(Voiceover: All party stop, we lost him) everything works normal (NPC&#8217;s will reappear).&nbsp;<\/p>\n\n\n\n<p>All these are events that get triggered on some action and change the behavior causing that action.<\/p>\n\n\n\n<p><em><strong>In AWS we set Autoscaling<\/strong><\/em> <strong><em>group and we tune it to some CPU condition which triggers SNS topic sending an email, or maybe<\/em><\/strong><em><strong> scaling up down your instances <\/strong><\/em><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"conclusion-where-does-it-fit-today\">Conclusion: Where does it fit today?<\/h3>\n\n\n\n<p>Event-driven applications are common for use cases including IoT, fraud detection, payment processing, website monitoring, and real-time marketing.<\/p>\n\n\n\n<p>Event-driven applications often treat data as immutable, or unchangeable, making it easy to look up the values of data at previous points in time.&nbsp;<\/p>\n\n\n\n<p>So whenever information \u201c<em>changes<\/em>,\u201d what actually happens is that a new data point is created with a new time period, which means you have old and new values.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"software-solutions\">Software Solutions :<\/h4>\n\n\n\n<ul><li>Kafka&nbsp;<\/li><li>Pulsar<\/li><\/ul>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>Note: This is Series Blog A complete Kafka setup will follow this blog.  <\/p><\/blockquote>\n\n\n\n<div class=\"wp-block-columns are-vertically-aligned-center is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:100%;\">\n<p><strong style=\"font-weight:bold;\">Blog Pundit: <\/strong><a href=\"https:\/\/opstree.com\/blog\/\/author\/naveenverma023\/\"><strong>Naveen Verma <\/strong><\/a>and <a href=\"https:\/\/opstree.com\/blog\/\/author\/adeel109\/\" target=\"_blank\" rel=\"noreferrer noopener\"><strong>Adeel Ahmad<\/strong><\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/www.opstree.com\/contact-us?utm_source=blog&amp;utm_medium=wordpress+&amp;utm_campaign=kafkas-solution--event-driven-architecture-otkafkadiaries\">Opstree<\/a><a href=\"https:\/\/www.opstree.com\/contact-us?utm_source=Blog&amp;utm_medium=Wordpress&amp;utm_campaign=AWS_SECRET_MANAGER\"> <\/a>is an End to End DevOps solution provider<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button is-style-fill\"><a class=\"wp-block-button__link\" href=\"https:\/\/www.opstree.com\/contact-us\" target=\"_blank\" rel=\"noreferrer noopener\">CONTACT US<\/a><\/div>\n<\/div>\n\n\n\n<p class=\"has-text-align-center\"><strong>Connect Us <\/strong><\/p>\n\n\n\n<ul class=\"wp-block-social-links aligncenter is-content-justification-right is-layout-flex wp-container-core-social-links-is-layout-1 wp-block-social-links-is-layout-flex\"><li class=\"wp-social-link wp-social-link-linkedin  wp-block-social-link\"><a href=\"https:\/\/www.linkedin.com\/company\/opstree-solutions\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M19.7,3H4.3C3.582,3,3,3.582,3,4.3v15.4C3,20.418,3.582,21,4.3,21h15.4c0.718,0,1.3-0.582,1.3-1.3V4.3 C21,3.582,20.418,3,19.7,3z M8.339,18.338H5.667v-8.59h2.672V18.338z M7.004,8.574c-0.857,0-1.549-0.694-1.549-1.548 c0-0.855,0.691-1.548,1.549-1.548c0.854,0,1.547,0.694,1.547,1.548C8.551,7.881,7.858,8.574,7.004,8.574z M18.339,18.338h-2.669 v-4.177c0-0.996-0.017-2.278-1.387-2.278c-1.389,0-1.601,1.086-1.601,2.206v4.249h-2.667v-8.59h2.559v1.174h0.037 c0.356-0.675,1.227-1.387,2.526-1.387c2.703,0,3.203,1.779,3.203,4.092V18.338z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">LinkedIn<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-youtube  wp-block-social-link\"><a href=\"https:\/\/www.youtube.com\/channel\/UCeLma6SpNYH7jjYKSBNSexw\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M21.8,8.001c0,0-0.195-1.378-0.795-1.985c-0.76-0.797-1.613-0.801-2.004-0.847c-2.799-0.202-6.997-0.202-6.997-0.202 h-0.009c0,0-4.198,0-6.997,0.202C4.608,5.216,3.756,5.22,2.995,6.016C2.395,6.623,2.2,8.001,2.2,8.001S2,9.62,2,11.238v1.517 c0,1.618,0.2,3.237,0.2,3.237s0.195,1.378,0.795,1.985c0.761,0.797,1.76,0.771,2.205,0.855c1.6,0.153,6.8,0.201,6.8,0.201 s4.203-0.006,7.001-0.209c0.391-0.047,1.243-0.051,2.004-0.847c0.6-0.607,0.795-1.985,0.795-1.985s0.2-1.618,0.2-3.237v-1.517 C22,9.62,21.8,8.001,21.8,8.001z M9.935,14.594l-0.001-5.62l5.404,2.82L9.935,14.594z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">YouTube<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-github  wp-block-social-link\"><a href=\"https:\/\/github.com\/OpsTree\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M12,2C6.477,2,2,6.477,2,12c0,4.419,2.865,8.166,6.839,9.489c0.5,0.09,0.682-0.218,0.682-0.484 c0-0.236-0.009-0.866-0.014-1.699c-2.782,0.602-3.369-1.34-3.369-1.34c-0.455-1.157-1.11-1.465-1.11-1.465 c-0.909-0.62,0.069-0.608,0.069-0.608c1.004,0.071,1.532,1.03,1.532,1.03c0.891,1.529,2.341,1.089,2.91,0.833 c0.091-0.647,0.349-1.086,0.635-1.337c-2.22-0.251-4.555-1.111-4.555-4.943c0-1.091,0.39-1.984,1.03-2.682 C6.546,8.54,6.202,7.524,6.746,6.148c0,0,0.84-0.269,2.75,1.025C10.295,6.95,11.15,6.84,12,6.836 c0.85,0.004,1.705,0.114,2.504,0.336c1.909-1.294,2.748-1.025,2.748-1.025c0.546,1.376,0.202,2.394,0.1,2.646 c0.64,0.699,1.026,1.591,1.026,2.682c0,3.841-2.337,4.687-4.565,4.935c0.359,0.307,0.679,0.917,0.679,1.852 c0,1.335-0.012,2.415-0.012,2.741c0,0.269,0.18,0.579,0.688,0.481C19.138,20.161,22,16.416,22,12C22,6.477,17.523,2,12,2z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">GitHub<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-facebook  wp-block-social-link\"><a href=\"https:\/\/www.facebook.com\/opstree\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M12 2C6.5 2 2 6.5 2 12c0 5 3.7 9.1 8.4 9.9v-7H7.9V12h2.5V9.8c0-2.5 1.5-3.9 3.8-3.9 1.1 0 2.2.2 2.2.2v2.5h-1.3c-1.2 0-1.6.8-1.6 1.6V12h2.8l-.4 2.9h-2.3v7C18.3 21.1 22 17 22 12c0-5.5-4.5-10-10-10z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Facebook<\/span><\/a><\/li>\n\n<li class=\"wp-social-link wp-social-link-medium  wp-block-social-link\"><a href=\"https:\/\/medium.com\/buildpiper\" class=\"wp-block-social-link-anchor\" target=\"_blank\" rel=\"noopener\"><svg width=\"24\" height=\"24\" viewBox=\"0 0 24 24\" version=\"1.1\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" aria-hidden=\"true\" focusable=\"false\"><path d=\"M20.962,7.257l-5.457,8.867l-3.923-6.375l3.126-5.08c0.112-0.182,0.319-0.286,0.527-0.286c0.05,0,0.1,0.008,0.149,0.02 c0.039,0.01,0.078,0.023,0.114,0.041l5.43,2.715l0.006,0.003c0.004,0.002,0.007,0.006,0.011,0.008 C20.971,7.191,20.98,7.227,20.962,7.257z M9.86,8.592v5.783l5.14,2.57L9.86,8.592z M15.772,17.331l4.231,2.115 C20.554,19.721,21,19.529,21,19.016V8.835L15.772,17.331z M8.968,7.178L3.665,4.527C3.569,4.479,3.478,4.456,3.395,4.456 C3.163,4.456,3,4.636,3,4.938v11.45c0,0.306,0.224,0.669,0.498,0.806l4.671,2.335c0.12,0.06,0.234,0.088,0.337,0.088 c0.29,0,0.494-0.225,0.494-0.602V7.231C9,7.208,8.988,7.188,8.968,7.178z\"><\/path><\/svg><span class=\"wp-block-social-link-label screen-reader-text\">Medium<\/span><\/a><\/li><\/ul>\n\n\n\n<p><\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Heroism often results as a response to extreme events. James Geary Event Driven Architecture: Modern digital businesses work in real-time based events. Event-driven architecture is based on the design principle which follows loose-coupling and message-driven architecture. This Architecture helps to publish events\/messages that applications and services can consume, and then perform an action based upon &hellip; <a href=\"https:\/\/opstree.com\/blog\/2021\/12\/28\/kafkas-solution-event-driven-architecture-otkafkadiaries\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Kafka\u2019s Solution : Event Driven Architecture: OTKafkaDiaries&#8221;<\/span><\/a><\/p>\n","protected":false},"author":200362113,"featured_media":29900,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[28070474],"tags":[33381383,207392,728157038,4996032],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/opstree.com\/blog\/wp-content\/uploads\/2025\/11\/DevSecOps-1.jpg","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pfDBOm-2dD","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/8533"}],"collection":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/users\/200362113"}],"replies":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/comments?post=8533"}],"version-history":[{"count":16,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/8533\/revisions"}],"predecessor-version":[{"id":9520,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/8533\/revisions\/9520"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media\/29900"}],"wp:attachment":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media?parent=8533"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/categories?post=8533"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/tags?post=8533"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}