{"id":6742,"date":"2021-06-22T13:39:44","date_gmt":"2021-06-22T08:09:44","guid":{"rendered":"https:\/\/opstree.com\/blog\/\/?p=6742"},"modified":"2021-06-22T14:07:11","modified_gmt":"2021-06-22T08:37:11","slug":"enable-support-to-provision-gp3-volumes-in-storage-class","status":"publish","type":"post","link":"https:\/\/opstree.com\/blog\/2021\/06\/22\/enable-support-to-provision-gp3-volumes-in-storage-class\/","title":{"rendered":"Enable Support to Provision GP3 Volumes in Storage Class"},"content":{"rendered":"\n

<\/p>\n\n\n\n

No matter where organizations are in their cloud journey, cloud cost optimization remains an essential and top-priority concern. As organizations align increasing workloads to the cloud in terms of size there is a high possibility to lose sight of the overall cloud environment and the costs associated with it. Therefore, it’s essential to optimize cloud costs so as to maximise return over investment (ROI). With the same concern we were working towards reducing the cost of provisioned storage for one our clients. This post covers a particularly interesting issue around the same.<\/p>\n\n\n\n\n\n\n\n

When we found that gp3 volumes are cheaper than gp2 by looking at price comparison given by aws<\/a>, we decided to migrate. Once our migration was complete w.r.t. volumes under cost optimization and mail was circulated as well stating that we would be launching only gp3 ebs volumes from now onwards, we did not expect one particular issue to creep up on us. After 2-3 days, we saw that gp2 volumes were still launching.<\/p>\n\n\n\n

We checked the volumes that were launched recently and found the culprit \u201cstorage class\u201d. We were using a storage class for dynamic volume provisioning of aws ebs volumes. Turns out, we forgot to change storage class in our eks cluster.<\/p>\n\n\n\n

\n
Warning ProvisioningFailed 9s (x5 over 59s) persistentvolume-controller Failed to provision volume with StorageClass \"gp3\": invalid AWS VolumeType \"gp3\".<\/pre>\n<\/div><\/div>\n\n\n\n
With the above error we came to know that we need to change the underlying provisioner of the storage class because provisioner: kubernetes.io\/aws-ebs <\/em>only supported gp2 volume type as per official documentation<\/a>. On further Googling to get out of this sticky situation,  we found a CSI provisioner could fix this problem. But implementing it wasn’t exactly a straight shot.<\/p>\n\n\n\n
\"\"
Ask me ?<\/figcaption><\/figure><\/div>\n\n\n\n
Problem Statement<\/h2>\n\n\n\n
Unable to launch gp3 volumes with in-tree kubernetes plugin. Followed a blog<\/a>, but ended up with another error, “not able to provision persistent volume using csi provisioner”<\/p>\n\n\n\n
Steps to reproduce<\/h2>\n\n\n\n

Delete storage class with default provisioner i.e. Fig. 1 below
kubectl delete sc buildpiper-storage<\/code><\/p>\n\n\n\n
Create storage class with csi provisioner i.e. Fig. 2 below
kubectl apply -f storage-class-csi.yaml<\/code>

Get eks worker node role name i.e. Fig. 3 below
kubectl get configmap aws-auth -n kube-system<\/code>

copy the ROLE-NAME
create IAM policy with name Amazon_EBS_CSI_Driver<\/strong><\/p>\n\n\n\n
{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"ec2:AttachVolume\",\n        \"ec2:CreateSnapshot\",\n        \"ec2:CreateTags\",\n        \"ec2:CreateVolume\",\n        \"ec2:DeleteSnapshot\",\n        \"ec2:DeleteTags\",\n        \"ec2:DeleteVolume\",\n        \"ec2:DescribeInstances\",\n        \"ec2:DescribeSnapshots\",\n        \"ec2:DescribeTags\",\n        \"ec2:DescribeVolumes\",\n        \"ec2:DetachVolume\"\n      ],\n      \"Resource\": \"*\"\n    }\n  ]\n}<\/pre>\n\n\n\n
Associate IAM Policy to Worker Node IAM Role:
1. Open AWS console -> Go to Services -> IAM -> Roles<\/p>\n\n\n\n
2. Search for role with name  ROLE-NAME <\/strong> and open it<\/p>\n\n\n\n
3. Click on Permissions<\/strong> tab – Click on Attach Policies<\/strong><\/p>\n\n\n\n
4. Search for Amazon_EBS_CSI_Driver<\/strong> and click on Attach Policy<\/strong>

Install EBS CSI Driver
kubectl apply -k \"github.com\/kubernetes-sigs\/aws-ebs-csi-driver\/deploy\/kubernetes\/overlays\/stable\/?ref=release-0.1<\/code><\/p>\n\n\n\n
\"\"
Fig. 1: Storage class with default provisioner and type <\/figcaption><\/figure><\/div>\n\n\n\n
<\/p>\n\n\n\n
\"\"
Fig. 2: storage class i.e. storage-class-csi.yaml with csi provisoner and type gp3<\/figcaption><\/figure><\/div>\n\n\n\n
<\/p>\n\n\n\n
\"\"
Fig. 3: aws-auth configmap<\/figcaption><\/figure><\/div>\n\n\n\n
<\/p>\n\n\n\n
Resolution<\/h2>\n\n\n\n
Continuing to struggle with this issue, we looked up compatible version at Kubernetes Version Compatibility Matrix<\/a>. Following this, upgraded k8s cluster from version 1.16 to 1.17 and it started working with version \u201crelease-0.10\u201d for me.<\/p>\n\n\n\n
Conclusion<\/h2>\n\n\n\n
We sought cost optimization and during that journey, came across a completely different issue. Got to know about CSI plugin that helped to launch gp3 ebs volumes via storage class. We had also faced version compatibility issue that leads to failure in achieving intended result. Overall it was a satisfying experience for me.
I have tried to keep it simple and frame the content based on my experience. If you feel it needs any correction or feedback, feel free to comment.
Keep learning and stay safe !<\/p>\n\n\n\n
Image Reference<\/p>\n\n\n\n
Image 1<\/a> and Image 2<\/a>,<\/p>\n\n\n\n
<\/p>\n\n\n\n
 Blog Pundit:<\/strong>   Adeel Ahmad<\/strong><\/a> & Naveen Verma<\/strong><\/a><\/strong><\/a><\/p>\n\n\n\n
Opstree is an End to End DevOps solution provider<\/p>\n\n\n\n
\n
CONTACT US<\/a><\/div>\n<\/div>\n\n\n\n
Connect Us <\/strong><\/p>\n\n\n\n