How to fix error “[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed” (_ssl.c:727)

Understanding SSLCertVerificationError

While collaborating on a hybrid cloud project with a client in the banking sector, we encountered a problem:

Fatal error: SSL validation failed for https://bucket_name.s3.ap-south-1.amazonaws.com/file_name  “[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed” (_ssl.c:727)

Scenario:

I was copying one file from s3 bucket to one of the newly launched servers x.x.x.x via AWS cli with below mentioned command.

command:
AWS_ACCESS_KEY_ID=XXXXXXX AWS_SECRET_ACCESS_KEY=XXXXXXX aws s3 cp s3://bucket_name/file_name

output:
fatal error: SSL validation failed for https://bucket_name.s3.ap-south-1.amazonaws.com/file_name  "[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed" (_ssl.c:727)

Above output is not giving enough information to troubleshoot this further. so i have used the openssl command.

command:
openssl s_client -connect bucket_name.s3.ap-south-1.amazonaws.com:443 -servername bucket_name.s3.ap-south-1.amazonaws.com

Output: Refer Fig1a and Fig1b

Using the openssl command, I discovered that our network is blocking internet requests due to a proxy configuration affecting all traffic.

This raises the question: why are we encountering a “certificate verification failed” error? To address this, it’s important to understand how TLS/SSL communication and the handshake process work. When executing a GET operation on S3 using the s3 cp command, it initiates an HTTPS request to the endpoint “https://bucket_name.s3.ap-south-1.amazonaws.com/file_name” This request fails at point number 3 in Fig2 because the proxy, which resides between the client and the S3 endpoint, is causing the certificate verification to fail, and whitelisting has not been implemented. (This type of warning also comes in browser while accessing some sites but you click on advance > proceed further to access the site)

• 

Note: Go through the blog, to know more about session keys and master secret.

[ Also Read: Stunnel a Proxy to ship the log on SSL ]

Workaround:

So, workaround to this is either you create a vpc-endpoint to communicate with s3 within the private network or get your endpoint whitelisted at proxy level because all the internet requests are going via proxy in this network configured by the network proxy team.

 

Conclusion

So in this blog, we have seen that AWS hits a regional endpoint over SSL when you access any AWS Service ( s3 in our case ) and that endpoint resolves to public IP. However, if there’s a proxy in place for all internet requests, it may restrict access. To navigate this issue, we can utilize a VPC endpoint or implement whitelisting as solutions. Thank you for taking the time to read this, and I welcome your suggestions and feedback!

GIF reference: