The problem that we faced:
There was a cron scheduled in our environment at a particular time which used to run at 8AM. This cron read data from the s3 bucket and used to succeed but failed when there were no objects present.
Problem this Integration solves:
- Eliminates the need of keeping a scheduler in the system. Lambda will work as a Scheduler.
- No manual intervention will be required in case the file from one service is delayed by a minute or an hour due to any reason.
- Data exchange and processing between teams become more efficient and less error-prone.
- Eliminates the need of checking the presence of files on S3. It all gets automated.
Workflow:
- The object is uploaded to S3
- S3 event triggers a Lambda function
- Lambda starts Jenkins job using Jenkins API
