AWS Archives - Page 3 of 7 - DEVOPS DONE RIGHT.

How to Reduce AWS Data Transfer Costs: A CFO’s Guide to Cloud Savings

If you’re using AWS, you may have noticed data transfer fees being added to your expenses. These costs are often included in your cost and usage reports, but don’t be fooled, if they’re not monitored they can quickly add up and become a significant contributor to your AWS bills.

Many organizations may face unexpectedly high data transfer charges, which reach up to lakhs of rupees per year. To get a handle on these costs and potentially reduce them, it’s essential to get a clear picture of your data transfer costs and identify which resources are driving them.

This blog explores a practical scenario that sheds light on AWS data transfer pricing, highlighting the typical challenges teams encounter and offering actionable strategies to help you optimize your cloud expenditures and effectively manage AWS costs.

Continue reading “How to Reduce AWS Data Transfer Costs: A CFO’s Guide to Cloud Savings”

EKS Cluster AutoScaler

The Amazon EKS AutoScaler automatically adjusts the number of worker nodes in an Amazon EKS cluster based on resource utilization and scaling demands.

Before we started using EKS Cluster Autoscaler, managing node scaling in our Kubernetes cluster was a constant struggle. Here’s what we were dealing with:

High Costs: We had to keep extra EC2 nodes running all the time to avoid pod scheduling issues — even when they weren’t being used.

Pod Failures: If we tried to reduce node count to save money, pods would often stay Pending due to lack of resources.

Manual Work: Scaling nodes up or down was a manual task—time–consuming and error–prone.

Slow Deployments: During deployments or traffic spikes, our workloads would get stuck because there weren’t enough resources.

Unpredictable Demand: Our workloads change frequently, but our cluster capacity doesn’t— leading to overuse or underuse of nodes.

These challenges pushed us to explore Cluster Autoscaler, and it turned out to be a game–changer

Continue reading “EKS Cluster AutoScaler”

Load Testing using AWS Distributed System – Part 1

In today’s fast-paced world, ensuring your application and business can handle growing users is very important. Whether you are building a Mobile application, Web application, or REST API it is important to understand how our system performs under pressure. Load testing is one of the ways we identify the bottlenecks, performance issues, and potential feature failures before the end user identifies it.

While consulting for an Edtech platform they highlighted an issue of having a dynamic and distributed load testing environment that can scale itself if we want to generate more load to the system and once the goal is achieved it should incur minimal cost to the organization since we are not leveraging a static provisioned environment.

AWS Distributed load testing system is a powerful automated setup provided by AWS that can be installed in our AWS cloud environment. This setup helps developers and testers generate real-world traffic at scale and test how well the system handles it. In this blog, we will talk about how the AWS Distributed Load Testing system works and how it can be leveraged to improve the reliability and performance of the system.

Continue reading “Load Testing using AWS Distributed System – Part 1”

End-to-End Data Pipeline for Real-Time Stock Market Data!

Transform your data landscape with powerful, flexible, and flexible data pipelines. Learn the data engineering strategies needed to effectively manage, process, and derive insights from comprehensive datasets.. Creating robust, scalable, and fault-tolerant data pipelines is a complex task that requires multiple tools and techniques.

Unlock the skills of building real-time stock market data pipelines using Apache Kafka. Follow a detailed step-by-step guide from setting up Kafka on AWS EC2 and learn how to connect it to AWS Glue and Athena for intuitive data processing and insightful analytics.
Continue reading “End-to-End Data Pipeline for Real-Time Stock Market Data!”