A blog site on our Real life experiences with various phases of DevOps starting from VCS, Build & Release, CI/CD, Cloud, Monitoring, Containerization.
Hi Guys !! Let’s have some discussion on the helm in Kubernetes. I know, you all be thinking what’s new in that there are lots of blogs which exists right. But let’s try to understand this not in boring terms but through some of funny scenarios and examples.
LET’s GET STARTED !!!
First we would understand some basics about Kubernetes and twist some boring definitions in fun way.
Production Ready Apache Airflow
Apache Airflow is an open-source platform designed to run any sort of workflow using Python. Its flexibility lets customers define pipelines through Python scripts, utilizing loops, bash instructions, and external modules such as pandas, sklearn, and cloud carrier libraries (GCP, AWS).
Many corporations agree with Airflow for its reliability:
Pinterest: Overcame overall performance and scalability issues, lowering maintenance costs.
GoDaddy: Supports batch analytics and records teams with an orchestration device and pre-built operators for ETL pipelines.
DXC Technology: Implemented Airflow to manage an undertaking with massive facts storage desires, presenting a stable orchestration engine.
These examples spotlight Airflow’s ability to cope with complicated facts processing demanding situations through the right deployment.
Pre-signed URLs are unique web links designed to provide temporary access to a private resource, such as an Amazon S3 object, without requiring the user to provide their own security credentials.
This approach allows you to generate a URL that includes authentication details and permissions, so users or applications can upload files directly to cloud storage without needing to have credentials for the storage service.
Here’s a step-by-step guide on how to implement file uploads using pre-signed URLs to a specific storage class, specifically with AWS S3. I’ll cover how to generate a pre-signed URL in Python and how to use it in Postman.
Continue reading “Uploading Files Using Pre-Signed URLs to a Specific Storage Class”
In this blog post, we’ll explore a real-world scenario where we need to migrate and replicate data between an EC2-hosted MySQL database and Amazon RDS (Relational Database Service) MySQL instance. This setup is critical for maintaining data integrity, ensuring high availability, and facilitating seamless scalability in a production environment. Continue reading “Use Case: Backup and Replication Setup Between EC2 MySQL and RDS MySQL”
Backing up databases is crucial for ensuring data integrity, disaster recovery preparedness, and business continuity. In MySQL environments, several tools are available, each with its strengths and optimal use cases. Understanding the differences between these tools helps you choose the right one based on your specific needs.
Disaster Recovery: In the event of data loss due to hardware failure, human error, or malicious attacks, having a backup allows you to restore your database to a previous state. Database Migration: When moving data between servers or upgrading MySQL versions, backups ensure that data can be safely transferred or rolled back if necessary. Testing and Development: Backups are essential for creating realistic testing environments or restoring development databases to a known state. Compliance and Auditing: Many industries require regular backups as part of compliance regulations to ensure data retention and integrity. Continue reading “Comparison between Mydumper, mysqldump, xtrabackup”
