Records Creation in Azure DNS from AKS ExternalDNS

Have you ever thought to create the DNS records automatically (like: A record, AAAA record and TXT record etc.) in the DNS Zones of Azure cloud from the kubernetes cluster’s yaml manifests (to be specific the External DNS) ?

NO ?

So don’t worry guys because i have done it and i thought i should share this experience with you guys. This is not limited to Azure cloud DNS Zones, It will work with AWS Route 53, Google cloud DNS, AWS cloud map and many more to mention here.

Now let me introduce you guys to the particular objects of this setup then next we will move forward to know How to setup these objects.

So let’s go………….

Context:

So basically the story behind writing this blog is that there were more than one pods resulting in more than one IP addresses and here the requirement takes place because manual mapping of domain names with the pod’s IP addresses is not okay.

Particular Objects Section:

DNS Zones: A DNS zone is used to host the DNS records for a particular domain. To start hosting your domain in Azure DNS, you need to create a DNS zone for that domain name. Each DNS record for your domain is then created inside this DNS zone.

For example, the domain ‘devopstool.ml’ may contain several DNS records, such as ‘mail.devopstool.ml’ (for a mail server) and ‘www.devopstool.ml’ (for a web site).

ExternalDNS: ExternalDNS synchronizes exposed Kubernetes Services and Ingresses with DNS providers. ExternalDNS makes Kubernetes resources discoverable via public DNS servers. Like KubeDNS, it retrieves a list of resources (Services, Ingresses, etc.) from the Kubernetes API to determine a desired list of DNS records. In a broader sense, ExternalDNS allows you to control DNS records dynamically via Kubernetes resources in a DNS provider-agnostic way.

Managed Identity: Managed Identity adds a role for the role of contributor to the DNS-RG then we add this Managed Identity with the VMSS(virtual machine scale set) to access the DNS Zone.

Prerequisites

• Azure Account
• Kubernetes Cluster
• Domain Name

Setup Steps

Step 1: DNS Zone Creation

• First of all we should have the domain name for which we want to create the DNS Zone.
• Then we can create the DNS Zone with the name of the domain name.

Step 2: Managed Identity Creation

• Create Managed service Identity.

• After creating MSI, add azure role assignment in MSI
  • Open MSI
  • Click on Azure Role Assignment -> Add role assignment1
  • Scope: Resource group
  • Subscription: use your own subscription
  • Resource group: use concerned resource group
  • Role: contributor
• Make  a note of client id and update in azure.json
  • Go to Overview -> Make a note of “Client ID”
  • Update it in azure.json value for userAssignedIdentityID

Step 3. Create azure.json file 

For template we can use this

{
  "tenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "subscriptionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "resourceGroup": "dns-zones", 
  "useManagedIdentityExtension": true,
  "userAssignedIdentityID": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"  
}

# To get Azure Tenant ID

az account show --query "tenantId"

# To get Azure Subscription ID

az account show --query "id"

# Use your own resource group

# Update userAssignedIdentityID

Step 4. Associate MSI in AKS cluster VMSS

Step 5. Create kubernetes secret

# Create Secret

kubectl create secret generic azure-config-file --from-file=azure.json

# List Secrets

kubectl get secrets

Step 6. Create external-dns.yaml manifest and deploy it.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: external-dns
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: external-dns
rules:
- apiGroups: [""]
  resources: ["services","endpoints","pods"]
  verbs: ["get","watch","list"]
- apiGroups: ["extensions","networking.k8s.io"]
  resources: ["ingresses"] 
  verbs: ["get","watch","list"]
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: external-dns-viewer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: external-dns
subjects:
- kind: ServiceAccount
  name: external-dns
  namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: external-dns
  template:
    metadata:
      labels:
        app: external-dns
    spec:
      serviceAccountName: external-dns
      containers:
      - name: external-dns
        image: registry.opensource.zalan.do/teapot/external-dns:latest
        args:
        - --source=service
        - --source=ingress
        #- --domain-filter=example.com # (optional) limit to only example.com domains; change to match the zone created above.
        - --provider=azure
        #- --azure-resource-group=externaldns # (optional) use the DNS zones from the specific resource group
        volumeMounts:
        - name: azure-config-file
          mountPath: /etc/kubernetes
          readOnly: true
      volumes:
      - name: azure-config-file
        secret:
          secretName: azure-config-file

• In –domain-filter we used the domain name
• In –azure-resource-group we used the resource group in which DNS Zone is created

# Deploy ExternalDNS 

kubectl apply -f external-dns.yml

# Verify ExternalDNS Logs

kubectl logs -f $(kubectl get po | egrep -o 'external-dns[A-Za-z0-9-]+')

# Error Type: 400

time="2020-08-24T11:25:04Z" level=error msg="azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/82808767-144c-4c66-a320-b30791668b0a/resourceGroups/dns-zones/providers/Microsoft.Network/dnsZones?api-version=2018-05-01: StatusCode=400 -- Original Error: adal: Refresh request failed. Status Code = '400'. Response body: {\"error\":\"invalid_request\",\"error_description\":\"Identity not found\"}"

# Error Type: 403

Notes: Error 403 will come when our Managed Service Identity dont have access to respective destination resource 

# When all good, we should get log as below

time="2020-08-24T11:27:59Z" level=info msg="Resolving to user assigned identity, client id is 404b0cc1-ba04-4933-bcea-7d002d184436."

Step 7: Deploy a demo application and test it

deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app1-nginx-deployment
  labels:
    app: app1-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app1-nginx
  template:
    metadata:
      labels:
        app: app1-nginx
    spec:
      containers:
        - name: app1-nginx
          image: stacksimplify/kube-nginxapp1:1.0.0
          ports:
            - containerPort: 80

clusterip-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: app1-nginx-clusterip-service
  labels:
    app: app1-nginx
spec:
  type: ClusterIP
  selector:
    app: app1-nginx
  ports:
    - port: 80
      targetPort: 80

ingress-with-externaldns.yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: nginxapp1-ingress-service
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: eapp1.devcsp.ml
    http:
      paths:
      - path: /
        backend:
          serviceName: app1-nginx-clusterip-service
          servicePort: 80

• Deploy these kubernetes objects.
• Wait for 3 to 5 minutes for record set update in DNS Zone.
• Verify external dns pod’s logs, should look like this.

time="2020-08-24T11:30:54Z" level=info msg="Updating A record named 'eapp1' to '20.37.141.33' for Azure DNS zone 'devcsp.ml'."

time="2020-08-24T11:30:55Z" level=info msg="Updating TXT record named 'eapp1' to '\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/default/nginxapp1-ingress-service\"' for Azure DNS zone 'devcsp.ml'."

Important Note: If external-dns pod logs are not the same as expected then check for the ingress controller, it must be available and if it is not then deploy it…….

helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
helm install ingress-nginx nginx-stable/nginx-ingress
helm install ingress-nginx nginx-stable/nginx-ingress --set rbac.create=true

• Now verify the external-dns pod logs it will be the same as expected

Hurraaaah 🙂

Conclusion:

Time to conclude the blog:>> I wrote this blog because There was a requirement of this kind of setup and i couldn’t find the expected results on web so it took more than enough time to make it happen so i thought there should be a proper documentation/blog for this setup.

References – GIF , Image 1 , Image 2

Blog Pundit: Bhupender rawat and Sanjeev Pandey

Opstree is an End to End DevOps solution provider

Connect Us

• LinkedIn
• YouTube
• GitHub
• Facebook
• Medium