How to fix error “[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed” (_ssl.c:727)

While working with one of our banking sector clients (hybrid cloud ), we encountered the error:

fatal error: SSL validation failed for https://bucket_name.s3.ap-south-1.amazonaws.com/file_name  “[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed” (_ssl.c:727)

Scenario:

I was copying one file from s3 bucket to one of the newly launched servers x.x.x.x via AWS cli with below mentioned command.

command:
AWS_ACCESS_KEY_ID=XXXXXXX AWS_SECRET_ACCESS_KEY=XXXXXXX aws s3 cp s3://bucket_name/file_name

output:
fatal error: SSL validation failed for https://bucket_name.s3.ap-south-1.amazonaws.com/file_name  "[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed" (_ssl.c:727)

Above output is not giving enough information to troubleshoot this further. so i have used the openssl command.

command:
openssl s_client -connect bucket_name.s3.ap-south-1.amazonaws.com:443 -servername bucket_name.s3.ap-south-1.amazonaws.com

Output: Refer Fig1a and Fig1b

With the help of openssl command, I got to know that our network is blocking internet requests due to some proxy configured for all the internet requests.

But the question arises why it is giving certificate verification failed error ? Well, we need to understand first how TLS/SSL communication/handshake works. While performing GET operation on s3 via s3 cp command, it is making a https request on endpoint “https://bucket_name.s3.ap-south-1.amazonaws.com/file_name” which is breaking at point number 3 in Fig2 because proxy lies between client and s3 endpoint whose certificate verification is failing and also whitelisting is not done. (This type of warning also comes in browser while accessing some sites but you click on advance > proceed further to access the site)

• 

Note: Go through the blog, to know more about session keys and master secret.

Workaround:

So, workaround to this is either you create a vpc-endpoint to communicate with s3 within the private network or get your endpoint whitelisted at proxy level because all the internet requests are going via proxy in this network configured by the network proxy team.

Conclusion

So in this blog, we have seen that AWS hits a regional endpoint over SSL when you access any AWS service ( s3 in our case ) and that endpoint resolves to public IP. But any hop like proxy ( configured for all the internet requests ) in between can restrict that request and we can bypass that proxy with VPC endpoint or whitelisting can solve this problem. Thanks for reading, I’d really appreciate your suggestions and feedback.

GIF reference:

Blog Pundit: Kapendra Singh and Sanjeev Pandey

Opstree is an End to End DevOps solution provider

Connect Us

• LinkedIn
• YouTube
• GitHub
• Facebook
• Medium