technical blogs Archives - Page 3 of 22

Nifi Cluster Setup with External Zookeeper

Apache NiFi is an open-source data integration and automation tool that enables the automation of data flow between different systems. NiFi provides a user-friendly interface to design, control, and manage the flow of data between various sources and destinations. The tool is particularly useful in handling data from different sources, applying transformations, and routing it to different systems in real-time.

Why use the Nifi cluster over the standalone?

Performance: Clusters can handle higher throughput and provide better performance than standalone instances due to load distribution.
Fault Tolerance: Clusters provide high availability and fault tolerance such as if one node fails, the other nodes take over the processing.
Scalability: Clusters allow for scalability by adding more nodes, whereas standalone instances have limitations in scaling.

Securing Nifi Cluster with TLS Toolkit

Apache NiFi is an easy-to-use, powerful, and reliable system to process and distribute data. As data flows between different systems it’s a good practice to ensure the integrity of the data being processed. Securing a NiFi cluster is essential for various reasons, primarily to protect sensitive data, ensure system integrity, and prevent unauthorized access. Apache NiFi provides a TLS toolkit using which we can self-sign Certificate Authority (CA) and easily issue and sign certificates in the format expected by NiFi. Continue reading “Securing Nifi Cluster with TLS Toolkit”

Enabling CORS on Azure APIM

Before we start on How to enable CORS in Azure API management service, let’s discuss the Azure APIM briefly –

What is Azure API Management?

Azure API Management is a comprehensive platform provided by Microsoft Azure that allows organizations to create, publish, secure, and analyze APIs. Whether you’re exposing APIs to external developers or managing internal APIs within your organization, Azure APIM simplifies the process and offers a range of features to ensure your APIs are both accessible and secure.

Key Features of Azure APIM:

1. API Gateway: Azure APIM acts as a gateway between your APIs and consumers. It handles requests, routing, and caching, improving API performance and reliability.

2. Security and Authentication: Secure your APIs with authentication, authorization, and OAuth support. Set up rate limiting and IP filtering to protect against abuse.

3. Developer Portal: Provide a developer-friendly portal where external developers can discover and consume your APIs. Publish documentation, code samples, and interactive API testing tools.

Unpacking Our Findings From Assessing Numerous Infrastructures – Part 2

When superior performance comes at a higher price tag, innovation makes it accessible. Quite evident from the way AWS has been evolving its services –

gp3, the successor of gp2 volumes – Offers the same durability, supported volume size, max IOPS per volume, and max IOPS per instance. The main difference between gp2 and gp3 is gp3’s decoupling of IOPS, throughput, and volume size. This flexibility to configure each piece independently – is where the savings come in.
AWS Graviton3 processors – Offers 25% better computing, double the floating-point, and improved cryptographic performance compared to its predecessors. It’s 3x faster than Graviton 2 and supports DDR5 memory, providing 50% more bandwidth than DDR4 (Graviton 2).

To be better at assessing your core infrastructure needs, knowing the AWS services is just half the battle. In my previous blog, I’ve discussed numerous areas where engineering teams often falter. Do give it a read! >>> Unpacking Our Findings From Assessing Numerous Infrastructures – Part 1

What we’ll be discussing here are –

Are your systems truly reliable?
How do you respond to a security incident?
How do you reduce defects, ease remediation, and improve flow into production? (Operational Excellence)

Continue reading “Unpacking Our Findings From Assessing Numerous Infrastructures – Part 2”

My learning in Migration of MySQL from 5.7 to 8.0

Introduction

In this blog, join me on a voyage through my personal experience my journey of Migrating MySQL from version 5.7 to 8.0. This voyage was motivated by the desire to harness the latest features, bolster security, and unlock the performance enhancements that MySQL 8.0 offers. As we navigate through the intricate migration process, I’ll share the valuable lessons learned, the hurdles encountered, and the strategies employed to overcome them.