Subhabrata Sarkar, Author at DEVOPS DONE RIGHT

We all have used IAM credentials to access our S3 buckets. But it’s not a very safe or recommended practice to keep our Access keys and Secrets stored in a server or hard code them in our codebase.
Even if we have to use keys, we must have some mechanism in place to rotate the keys very frequently (eg: using Hashicorp Vault). Another widely adopted method is to use IAM roles attached on the EC2 instance or the AWS service accessing the bucket.

But, what if we need access to the bucket from an on-premise Data Center where we can not attach an IAM role?

Yes, we can obviously use IAM credentials and secret tokens with the rotating mechanism. But setting up the key rotation mechanism itself could be another overhead if we do not have one already in place. What if we do not require keys or roles without making the bucket public?

In this blog, I will make an attempt to cater to this problem with another alternate and easy solution.

Nowadays, it is very common to run applications on Spot instances. In this scenario, where a spot instance could be terminated at any point of time because of AWS pulling back their resource or ASG Scale-In incident, we need to have something in place to handle the termination smoothly so that we can complete our final tasks before the system shutdown. It could be executing some scripts, unmounting some storage device, shipping final log files to S3, or uploading cache data in a centralized server like Redis.
Today, I will attempt to cater to this problem.

First of all, let’s think of trying to run a custom script prior to shutdown in our local system. If everything works fine, the same would be applicable for ec2 spot instances too.