What is Kernel Patching?
The operating system kernel is the central component that controls system resources and enables hardware and software connection. Kernel patching is the act of changing or maintaining the kernel’s source code.
When to choose this type of patching?
Organizations that are thinking about kernel patching usually assess how well the available solutions work with the particular kernel version they are using, test extensively in a non-production setting, and set up reliable backup and rollback procedures to reduce risks in the production environment.
It’s critical to thoroughly evaluate the organization’s particular requirements and risks, as well as the advantages and disadvantages of kernel patching. Other considerations that might impact the choice include the organization’s overall risk tolerance, regulatory requirements, and how critical the systems are.
The primary reasons for kernel patching are:
Bug Fixes: Kernel patches are frequently developed to fix errors in software, security flaws, or other kernel-related problems. When a defect is found, developers make the required code modifications in a patch to address the issue.
Security Updates: Kernel patching is crucial to apply security updates and close these vulnerabilities to protect the system from attacks.
Hardware Support: Support for new hardware components and devices must be introduced to the kernel as they are created. To make sure that the kernel recognizes and functions with new hardware, kernel patches may be developed.
Performance Enhancements: Kernel patches can be applied to add new features or enhance the kernel’s performance. These patches might improve the functionality of the kernel or optimize already-written code.
The process of kernel patching typically involves the following steps:
Identification: Kernel developers and maintainers find problems, defects, or improvements that need to be patched.
Patch Creation: A patch file containing the precise modifications needed to fix the found problem or apply the intended improvement is made by a developer. Usually, this patch takes the form of a unified diff file that lists the lines of code that need to be added, changed, or eliminated.
Testing: To make sure the patch fixes the problem or adds the desired functionality, it is tested. It shouldn’t cause any regressions or new issues.
Distribution: Following testing and verification, the patch is either released to the general public or added to the official kernel source code.
Application: The updated kernel source code must be compiled and installed by end users or system administrators in order to apply the patch to their system’s kernel. As an alternative, users can make use of the pre-built kernel packages that their distribution offers.
Prerequisites required to accomplish the task
- Login to the master host from where you can access all the hosts (means public key of this host should be present on all the hosts).
- Or you can use public-private key pair which can access all the hosts (passing private key while running commands).
Example:
loop "-i <private-key-path> <command to run>" patch-hosts
Steps to perform the patching process and obtain the latest kernel version
First create a loop script on the master host through which we can ssh all the required hosts
Loop Script:
#!/bin/sh for i in cat $2; do echo echo $i ssh -oStrictHostKeyChecking=no -oKexAlgorithms=+diffie-hellman-group1-sha1 -q root@$i $1 echo done
1. Check the current kernel version
First you can check the current kernel version of all the hosts list
loop "rpm -qa | grep kernel" patch-hosts | tee current_kernel.out
loop: script to access all the hosts via ssh
patch-hosts: list of all the required patching hosts
current_kernel.out: output file to store the loop script results
2. Update the kernel version
Now you need to update the hosts with the latest kernel version available
loop "yum clean all ; yum -y update" patch-hosts | tee update_kernel.out
3. Check the kernel versions
Now with below command check the kernel versions available on the hosts
loop "rpm -qa | grep kernel" patch-hosts | tee list_of_kernel.out
4. Check filesystem
Now reboot the hosts so that hosts will update to new kernel version
loop "reboot" patch-hosts | tee reboot.out
5. Reboot the hosts
Now reboot the hosts so that hosts will update to the new kernel version
loop "reboot" patch-hosts | tee reboot.out
6. Check the current kernel release
When all machines come up after reboot, check the current kernel release on the required hosts
loop "uname -r" patch-hosts | tee latest_kernel.out
7. Remove old kernels
To not be stuck in the boot disk out-of-space issue on the device, means /boot is full, probably too many old kernels. The command below will remove all except the latest two kernels
loop "package-cleanup -y --oldkernels --count=2" patch-hosts
8. Check core drops
On some gen10 HP hardware, there is a problem with the hp-health service, it will crash and dump cores under / and fill the disk.
loop "ls / | grep -i core" patch-hosts | tee corelist.out
9. Fix the core drops
If there are core drops, you can fix them with the below command
loop "systemctl disable hp-health.service ; systemctl stop hp-health.service ; rm -f /core.*" coredrophosts
Using parallel script in steps 2,4,5,7 and 9 will be useful
Note: If you have a long list of patching hosts you can use parallel script instead of loop script which may save you time.
Parallel script:
#!/bin/sh for i in `cat $2`; do echo echo $i ssh -oStrictHostKeyChecking=no -oKexAlgorithms=+diffie-hellman-group1-sha1 -q & root@$i $1 & echo done
Example: Using parallel script in step 2parallel "yum clean all ; yum -y update" patchhosts | tee update_kernel.out
Caution:
It is essential to use caution when applying kernel patches and to fully understand the goals and implications of each patch. It is also recommended to thoroughly test the patched kernel in a non-production environment before deploying it in a production environment.
Conclusion:
It’s crucial to remember that patching the kernel can be a tricky and intricate procedure that calls for a solid grasp of both the internal workings of the operating system and the kernel’s source code. To maintain system stability and security, users should often rely on official kernel updates and patches from their operating system distribution rather than writing their own.
Blog Pundits: Sanjeev Pandey and Sandeep Rawat
OpsTree is an End-to-End DevOps Solution Provider.
Connect with Us
