A Complete Guide to Kubernetes CRDs: Definition, Uses , Benefits, and Error Fixes

Hi Everyone , Today we are trying to understand CRDs(Custom Resource Definitions) as I was working on one Observability project in OpsTree Global and suddenly found CRD errors and let me tell you, It was very Frustration. I will try to make you understand them in easy way, so that you don’t need to make another doc of knowledge. Comment if you will have any doubts.

What are CRD ?
Why we use CRD ?
How does it help ?
What error you will see and their Fixation ?
FAQ

What are CRD ?

A CRD in Kubernetes is like teaching Kubernetes a new word. By Default Kubernetes only understands built-in resources like Pods , Services, Deployments.
With CRD you can add your own type of resource ( like Pizza , FoodOrder , or Database ) and then use it just like the build-in once with Kubectl.

Kubernetes API

Let’s Try to Understand this Analogy with Example of Zomato.

Think of your Phone
->> Your Phone already understands Contacts, Messages , Photos ( like Kubernetes Knows Pods , Services).
– But then you Install the Zomato app → Now your Phone Understands a New Concept ( FoodOrder).
Installing Zomato = Add a CRD ( new type = Food Order ).
– You Place a pizza order in Zomato → this a Custom Resource (an actual object of type FoodOrder.
– Zomato ’s system ensures food is cooked and delivered → this is the Controller ( it makes sure your “order” is Fulfilled).

Let’s See the YAML file to understand it better

Step 1 : Define CRD

apiVersion: apiextensions.k8s.io/v1
kind::CustomResourceDefinition
metadata::
  name::foodorders.food.app
spec::
  group: food.app
  scope: Namespaced
  names::
    plural: foodorders
    singular: foodorder
    kind: FoodOrder
  versions::
    - name: v1
      served: true
      storage: true
      schema::
        openAPIV3Schema::
          type: object
          properties::
            spec::
              type: object
              properties::
                dish::
                  type: string
                quantity::
                  type: integer

Step 2 : Create Custom Resource ( Pizza Order )

apiVersion: food.app/v1
kind: FoodOrder
metadata:
  name: komal-pizza-order
spec:
  dish: "Pizza"
  quantity: 2

Step 3: Run Below Commands to make the above CRD

kubectl get foodorders
kubectl describe foodorder komal-pizza-order

[Drive Efficiency with DevSecOps Transformation & Automation]

Why we use CRD ?

CRDs ( CustomResourceDefinitions) let us extend Kubernetes to support our own Business or Application-specific resources.

Reasons

Add new resource types ( extended Kubernetes API )
– E.g. Kubernetes doesn’t know what a Database is .
– With a CRD, you can Define a Database resource Kubectl get databases just like we do kubectl get pods.
Automate with Operators
CRDs allow you to automate tasks when used with a controller/ operator.
- Example: A Database CRD might automatically provision a PostgreSQL Pod, set up storage and make backups.
Consistency and Standardization
- In the absence of CRDs, teams would work with scripts, or YAML hacks.
– In CRDs everything is declarative and resides within Kubernetes a single API, a single workflow.
Validation and Safety
CRDs may impose schema based rules.
– Such as: a BackupPolicy CRD may need , frequency = daily/weekly/monthly. retention = integer between 1–30 This discourages invalid configs to be used.
Versioning and Extensibility
– Your CRDs can just as much evolve with versions (v1alpha1, v1beta1, v1). This renders them stable and forward looking.

” Want to master your data initiatives? Download our Ultimate Guide to Delivering End-to-End Data Strategy ”

What Error Will we see and it’s Fixation?

1. CRD Not Found

Error Example:

error: failed to identify but one recognizes key: database.yaml version mycompany.com/v1 with kind Database no matches no matches are found in the query.

Cause:

Before actually installing the CRD, you applied a CR (Custom Resource)
Incorrect apiVersion or CRD name typo.

Solution:

1. First install CRD:

kubectl apply -f crd-definition.yaml

2. Then apply the CR:

kubectl apply database.yaml

3. Verify CRDs:

kubectl get crd grep database

2. Schema Validation Errors

Error Example:

error: error during validation of database.yaml: error during validation of data: ValidationError(Database.spec): unknown field "storagge"

Cause:

You specified a field (storagge) not present in CRD schema.

Solution:

1. Check CRD schema using:

kubectl describe databases.mycompany.com

2. Fix the spelling the typo error - storage, not storagge.

3. CRD Size Limit Exceeded (26625 KB Issue)

Error Example:

etcdserver: mvcc: database space has reached (26625 kb growth)

Cause:

Too many / large objects in CRD (particularly when storing logs, monitoring configs, Prometheus CRDs).
etcd is overwhelmed since CRDs place all information in etcd.

Solution:

1. Compact etcd database:

etcdserver: mvcc: database space has reached (26625 kb growth)

2. Defragment etcd:

ETCDCTL_API=3 etcdctl defrag

CRD Use short names to decrease the size of objects.
Do not keep heavy configs in CRDs (use ConfigMap instead).

CRD Version Mismatch

Error Example:

no kind kind Prometheus v1alpha1 monitoring.coreos.com no version available of monitoring.coreos.com/v1alpha1

Cause:

You are operating a previous version of CRD (v1alpha1) yet the CR is in v1.

Solution:

Check supported versions:

kubectl get crd prometheuses.monitoring.coreos.com -o yaml grep served.
Upgrade CR YAML to version.

Ref: Operators Prometheus CRDs.

CRD Stalled in Terminating State.

Error Example:

customesourcedefinition.apiextensions.k8s.io databases.mycompany.com is deleted.

Cause:

CRD is deletion blocked.
Cleanup hooks Finalizers ensure that CRD is not removed accidentally.

Solution:

Remove finalizers manually:

kubectl patch crd databases.mycompany.com -p consist of metadata finalizers:[]

Also, see Kubernetes Finalizers Docs.

CRDs Performance Problems.

Problem:

Excessive CR leads to the slowing of API server/etcd.
Example: CRDs Prometheus ServiceMonitor are capable of flooding the cluster.

Solution:

Split resources with sharding (into multiple CRDs).
Optimize controllers to observe less objects.
Performance kube-apiserver-use watch-cache flag.

Ref: CRDs and Kubernetes Scalability.

YAML Apply Issues

Error Example:

error: no resource mapping found that matches name: my-postgres namespace: none.
no matches of kind Database in database.yaml

Cause:

Namespace mismatch.
CR installed previous to CRD.

Solution:

Always install CRD first.
namespace my-namespace in CR YAML.

FAQ

Q1: Which is the difference between CRDs and Kubernetes built-in resources?

A: CRDs are user extensions, which are defined to meet domain requirements, and built-in resources (Pods, Services, Deployments) are the resources that Kubernetes provides. CRDs are subject to the same patterns but model your business logic.

Q2: Is a custom controller required per CRD?

A: Not necessarily. CRDs are able to store data without controllers, although controllers are required to apply business logic and respond to resource changes. CRDs may be simple configurations without any controllers.

Q3: What do I do with CRD versioning in production?

A: Have versions in your CRD definition, do conversion webhook migrations when complicated, and have plans of depreciation. Test version migrations should first be made in non production environments.

Q4: What is the behavior when I delete an active custom resource CRD?

A: Kubernetes will not allow to delete CRD when custom resources are there. You have to clean up all of the custom resources beforehand or gracefully clean up with finalizers.

Q5: Do CRDs have the ability to affect Kubernetes cluster performance?

A: Well planned CRDs have very little effect. But badly constructed validation rules, too large custom resources, or ineffective controllers may have an impact. Track resource usage and take relevant limit.