Faster Builds with Docker Caching

Recently was working around speeding up CI Build, which took time around 50 min. This post is about speeding up builds with Docker caching and Buildkit. It is a little concept, but it is significant to know as this helped me to reduce timings from 50 min to 15 min.

Docker Caching

Docker caches each layer as an image is built, and each layer will only be re-built if it or the layer above it has changed since the last build. So, you can significantly speed up builds with Docker cache.

A better way would be to have multiple COPY instructions that install the packages and dependencies first and then copies your code.

To avoid invalidating the cache:

1. Start your Dockerfile with commands that are less likely to change
2. Place commands that are more likely to change (like COPY . .) as late as possible
3. Add only the necessary files (use a .dockerignore file)

Example :

The following Dockerfile is for a simple node js project which works great but here cache is not used :

Copy 
1FROM mhart/alpine-node
2
3WORKDIR /src
4# Copy your code in the docker image
5COPY . /src
6# Install your project dependencies
7RUN npm install
8# Expose the port 3000
9EXPOSE 3000
10# Set the default command to run when a container starts
11CMD ["npm", "start"]

Optimized Dockerfile :

Here step 7 is the only step where an image is rebuilt based on previous steps when we make a change to our source code.

Copy 
1FROM mhart/alpine-node:5.6.0
2WORKDIR /src
3# Expose the port 3000
4EXPOSE 3000
5# Set the default command to run when a container starts
6CMD ["npm", "start"]
7# Install app dependencies
8COPY package.json /src
9RUN npm install
10# Copy your code in the docker image
11COPY . /src

BuildKit

If you’re using a Docker version >= 19.03 you can use BuildKit, a container image builder. Without BuildKit, if an image doesn’t exist on your local image registry, you would need to pull the remote images before building in order to take advantage of Docker layer caching.

Example:

Copy 
1$ docker pull mjhea0/docker-ci-cache:latest
2
3$ docker build --tag mjhea0/docker-ci-cache:latest .

With BuildKit, you don’t need to pull the remote images before building since it caches each build layer in your image registry. Then, when you build the image, each layer is downloaded as needed during the build.

To enable BuildKit, set the DOCKER_BUILDKIT environment variable to 1. Then, to turn on the inline layer caching, use the BUILDKIT_INLINE_CACHE build argument.

Example:

Copy 
1export DOCKER_BUILDKIT=1
2
3# Build and cache image
4$ docker build --tag mjhea0/docker-ci-cache:latest --build-arg BUILDKIT_INLINE_CACHE=1 .
5
6# Build an image from remote cache
7$ docker build --cache-from mjhea0/docker-ci-cache:latest .

Originally Posted here : https://todayilearnt.xyz/posts/nancy/faster_builds_with_docker_caching/