GitHub: Self-Hosted Runner on Kubernetes

GitHub Actions is a powerful tool for automating software workflows, and it can be used to build, test, and deploy code right from GitHub. It provides a way to automate repetitive tasks and can be integrated with many popular tools and platforms.

GitHub Actions can use two types of runners: hosted and self-hosted.

Hosted runners are provided by GitHub and run on virtual machines in the cloud.
Self-hosted runners are machines that you set up and manage yourself. They run on your infrastructure, and you can customize them to meet your needs.

In this tutorial, we will show you how to set up GitHub’s self-hosted runner on Kubernetes.

Prerequisites

Before you begin, make sure you have the following:

A Kubernetes cluster
Helm Installed
Access to a GitHub repository for creating PAT and adding runners.

Installing Cert Manager on K8s cluster.

Well, actions-runner-controller(ACR) uses cert-manager for certificate management of admission webhook, so we have to ensure cert-manager is installed on Kubernetes before installing actions-runner-controller.
Refer to this link for Cert Manager Installation via Helm
Verify the installation by running the below command.

Authentication for Self-Hosted Runners

There are two ways for the actions-runner-controller to authenticate with the GitHub API (only 1 can be configured at a time, however):

Using a GitHub App (not supported for enterprise-level runners due to lack of support from GitHub)
Using a PAT(Personal Access Token)

We are going to use PAT for our use case.

To register the self-hosted runner with GitHub, generate a PAT with the repo scope. Here’s how to generate the PAT:

Go to your GitHub account settings and select “Developer settings”.
Select “Personal access tokens”.
Click the “Generate new token” button.
Give your token a name and select the repo scope.

Click “Generate token” and save the token somewhere safe.

Use the below command to create Kubernetes secret which will be used by ACR(Actions Runner Controller)

# Creating Namespace for ARC
kubectl create ns actions-runner-system

# Creating secret for ARC, so that it can register runners on github
kubectl create secret generic controller-manager -n actions-runner-system 
--from-literal=github_token=<GitHub Token>

Installing ARC on Kubernetes Cluster

Execute below mentioned Helm commands,

# Adding Helm Repo of ARC
helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller

# Updating Helm Repo
helm repo update

# Installing ARC
helm upgrade --install --namespace actions-runner-system 
--create-namespace --wait actions-runner-controller 
actions-runner-controller/actions-runner-controller --set 
syncPeriod=1m

Verify that the action-runner-controller is installed properly using the below command

Creating Runners for Repository

Create a RunnerDeployment Kubernetes object, which will create a self-hosted runner for the GitHub repository.
Create file runner.yaml and paste the below content

	apiVersion: actions.summerwind.dev/v1alpha1
	kind: RunnerDeployment
	metadata:
	name: k8s-action-runner
	namespace: actions-runner-system
	spec:
	replicas: 1
	template:
	spec:
	repository: <Repository Name>
	labels:
	– "actions_runner_dev"

view raw runner.yaml hosted with ❤ by GitHub

Here, change the Repository Name to your GitHub Repo name & labels for runners, so that you can customize your workflow in such a way that a particular job would run on a particular runner only. The below image shows custom labels on a runner.

Execute the below command to create a RunnerDeployment object.

kubectl create -f runner.yaml

Check that the pod is running using the below command.

kubectl get pod -n actions-runner-system | grep -i "k8s-action-runner"

Now, you should see actions runner pods on Kubernetes, which is also registered on GitHub. Check under Repository Settings > Actions > Runner.

Creating HRA(HorizontalRunnerAutoscaler) for Runners

Setting up a HorizontalRunnerAutoscaler for your self-hosted runner can help to ensure that you have the appropriate amount of resources available to handle your continuous integration and deployment needs. By automating the scaling process based on predefined rules and metrics, you can reduce the risk of resource constraints and ensure that your builds and deployments remain fast and reliable.

Create a file “horizontal_runner_autoscaler.yaml” & paste the below content.

	apiVersion: actions.summerwind.dev/v1alpha1
	kind: HorizontalRunnerAutoscaler
	metadata:
	name: horizontal-runner-autoscaler-dev # change name accordingly
	namespace: actions-runner-system
	spec:
	scaleTargetRef:
	kind: RunnerDeployment
	name: k8s-action-runner # Name of RunnerDeployment
	minReplicas: 1
	maxReplicas: 5
	metrics:
	– type: PercentageRunnersBusy
	scaleUpThreshold: '0.75' # The percentage of busy runners at which the number of desired runners are re-evaluated to scale up
	scaleDownThreshold: '0.3' # The percentage of busy runners at which the number of desired runners are re-evaluated to scale down
	scaleUpFactor: '1.4' # The scale up multiplier factor applied to desired count
	scaleDownFactor: '0.7' # The scale down multiplier factor applied to desired count

view raw horizontal_runner_autoscaler.yaml hosted with ❤ by GitHub

Execute the below command.

# Applying the HRA
kubectl apply -f horizontal_runner_autoscaler.yaml

# Run below to command to verify the HRA object
kubectl get HorizontalRunnerAutoscaler -n actions-runner-system

Testing HRA

Now to test HRA, I have created a “test.yaml” workflow. Remember that we have created RunnerDeployment above in which we defined labels, here they come in handy by mentioning them on the runs-on field.

	name: Share data between jobs

	on: [push]

	jobs:
	job_1:
	name: Add 3 and 7
	runs-on: actions_runner_dev
	steps:
	– shell: bash
	run: \|
	expr 3 + 7 > math-homework.txt
	– name: Upload math result for job 1
	uses: actions/upload-artifact@v3
	with:
	name: homework
	path: math-homework.txt

	job_2:
	name: Multiply by 9
	needs: job_1
	runs-on: actions_runner_dev
	steps:
	– name: Download math result for job 1
	uses: actions/download-artifact@v3
	with:
	name: homework
	– shell: bash
	run: \|
	value=`cat math-homework.txt`
	expr $value \* 9 > math-homework.txt
	– name: Upload math result for job 2
	uses: actions/upload-artifact@v3
	with:
	name: homework
	path: math-homework.txt

	job_3:
	name: Display results
	needs: job_2
	runs-on: actions_runner_dev
	steps:
	– name: Download math result for job 2
	uses: actions/download-artifact@v3
	with:
	name: homework
	– name: Print the final result
	shell: bash
	run: \|
	value=`cat math-homework.txt`
	echo The result is $value

view raw test.yaml hosted with ❤ by GitHub

The above workflow works on the push-based events, you can customize your workflow accordingly.
Now to invoke HRA, I have pushed changes multiple times into the Repo so that multiple actions would be triggered and thus enabling the HRA.
After push found the below result, HRA’s DESIRED state has been changed from 1 to 4.

kubectl get HorizontalRunnerAutoscaler -n actions-runner-system

Checking the number of runner pods,

kubectl get pod -n actions-runner-system | grep -i "k8s-action-runner"

Checking the controller logs as well,

k logs -f actions-runner-controller-7c8756c587-np4tt -c manager -n actions-runner-system

Benefits of using Self-Hosted Runner

Improved Performance: By hosting your runners, you can ensure that the build and deployment processes are faster and more reliable, as you have complete control over the hardware and networking resources.
Increased Security: GitHub self-hosted runners can be configured to run on your own servers, which provides an extra layer of security compared to using shared runners. This helps to protect sensitive information and data in your workflows.
Customizable Environments: With self-hosted runners, you can create custom environments that match your exact needs, including specific software versions and configurations.
Cost-Effective: If you have a large number of workflows or use cases that require a lot of resources, self-hosted runners can be more cost-effective than using GitHub’s shared runners or cloud-based solutions.
High Availability: With self-hosted runners, you can set up Horizontal Runner Autoscaler, which provides redundancy and high availability for your workflows.
Greater Control: Self-hosted runners give you complete control over the resources and environment used for your workflows, which can help you optimize performance and ensure that your builds and deployments run smoothly.

Overall, GitHub self-hosted runners offer greater flexibility, control, and customization options than using shared runners or cloud-based solutions. However, setting up and managing self-hosted runners requires additional effort and resources, so it’s essential to weigh the benefits against the costs and resources needed to maintain them.

Conclusion

In conclusion, the Actions Runner Controller with Self-Hosted Runner is a powerful tool that can help you improve your GitHub Actions workflows. It simplifies the management of runners by automating tasks such as the creation, scaling, and deletion of runners. It also provides an easy way to create and manage runners in a Kubernetes cluster. By using Actions Runner Controller with Self-Hosted Runner, you can take full advantage of the power of GitHub Actions while having full control over your infrastructure.