Have you ever faced an issue in which while building the docker image, even though you have mentioned “apt-get -y update” the package you are trying to install is not found in the repository?
Here, in this blog, we are going to discuss two common scenarios that I faced along with the root cause of this problem. Let’s take a look.
First, it’s important to remember that valid intermediate images are not built again. They are loaded from cache. Dockerfile cache is considered valid if the build command gives a 0 return code.
1. Consider the following Dockerfile.
From ubuntu
RUN apt-get -y update RUN apt-get install -y python-pip
Now when we build it:
docker build -t test:image1 .
Suppose while building the above Dockerfile we created an intermediary image with the directive “RUN apt-get update”.
This intermediate image will be used for all the subsequent docker builds.
Let’s say a couple of months down the line, there is a new package-manager release of python-pip which is required by the application. Now, even though in Dockerfile we have mentioned “RUN apt-get update”, the apt repo will not get updated because that image will be taken from the cache. Hence the latest version of python-pip won’t be installed.
2. Our story is not over yet, consider the following.
Before building the below Dockerfile, turn your internet connection off.
From ubuntu
RUN apt-get -y update ; echo "status is: $?"
RUN apt-get install -y python-pip
Now when we build it:
Docker build -t test:image2 .
Output:
Err http://security.debian.org stable/updates Release.gpg
Could not resolve 'security.debian.org'
Hit http://192.168.1.100 stable Release.gpg
Hit http://192.168.1.100 stable Release
Hit http://192.168.1.100 stable/main i386 Packages
Hit http://192.168.1.100 stable/contrib i386 Packages
Hit http://192.168.1.100 stable/non-free i386 Packages
Ign http://192.168.1.100 stable/contrib Translation-en
Ign http://192.168.1.100 stable/main Translation-en
Ign http://192.168.1.100 stable/non-free Translation-en
Reading package lists... Done
W: Failed to fetch http://security.debian.org/dists/stable/updates/Release.gpg
W: Some index files failed to download. They have been ignored, or old ones used instead.
status is: 0
Aside from the ADD and COPY commands, cache checking does not look at any other file in the container to determine a cache match. For example, when processing a RUN apt-get -y update command, the files updated in the container are not examined to determine if a cache hit exists. In such a case, just the command string itself is used to find a match.
As we can see above, “apt-get update” throws a “WARN” and not an “ERROR” and the return code is 0, which is a successful response. Since the response code is zero hence this image will get cached.
The problem comes when we re-build the Dockerfile with our internet connection on, Docker will still take the “apt-get -y update” image from the cache and subsequent “RUN apt-get install -y python-pip” will fail because “apt-get -y update” didn’t trigger successfully in the last run.
Solution:
Try building the image again with the “no-cache” option. This will force the rebuilding of layers already available.
Docker build --no-cache -t test:image3 .
Conclusion :
Once an image is successfully built, all the intermediate images will be kept and reused. A valid docker intermediate image will not be built again unless we pass the “no-cache” option or delete the existing image.
Opstree is an End to End DevOps solution provider
One thought on “Hidden Secrets of Dockerfile”