Navigate back to the homepage

Faster Builds with Docker Caching

Nancy Chauhan
May 20th, 2020 · 1 min read

Recently was working around speeding up CI Build, which took time around 50 min. This post is about speeding up builds with Docker caching and Buildkit. It is a little concept, but it is significant to know as this helped me to reduce timings from 50 min to 15 min.

Docker Caching

Docker caches each layer as an image is built, and each layer will only be re-built if it or the layer above it has changed since the last build. So, you can significantly speed up builds with Docker cache.

A better way would be to have multiple COPY instructions that install the packages and dependencies first and then copies your code.

To avoid invalidating the cache:

  1. Start your Dockerfile with commands that are less likely to change
  2. Place commands that are more likely to change (like COPY . .) as late as possible
  3. Add only the necessary files (use a .dockerignore file)

Example :

The following Dockerfile is for a simple node js project which works great but here cache is not used :

1FROM mhart/alpine-node
2
3WORKDIR /src
4# Copy your code in the docker image
5COPY . /src
6# Install your project dependencies
7RUN npm install
8# Expose the port 3000
9EXPOSE 3000
10# Set the default command to run when a container starts
11CMD ["npm", "start"]

Optimized Dockerfile :

Here step 7 is the only step where an image is rebuilt based on previous steps when we make a change to our source code.

1FROM mhart/alpine-node:5.6.0
2WORKDIR /src
3# Expose the port 3000
4EXPOSE 3000
5# Set the default command to run when a container starts
6CMD ["npm", "start"]
7# Install app dependencies
8COPY package.json /src
9RUN npm install
10# Copy your code in the docker image
11COPY . /src

BuildKit

If you’re using a Docker version >= 19.03 you can use BuildKit, a container image builder. Without BuildKit, if an image doesn’t exist on your local image registry, you would need to pull the remote images before building in order to take advantage of Docker layer caching.

Example:

1$ docker pull mjhea0/docker-ci-cache:latest
2
3$ docker build --tag mjhea0/docker-ci-cache:latest .

With BuildKit, you don’t need to pull the remote images before building since it caches each build layer in your image registry. Then, when you build the image, each layer is downloaded as needed during the build.

To enable BuildKit, set the DOCKER_BUILDKIT environment variable to 1. Then, to turn on the inline layer caching, use the BUILDKIT_INLINE_CACHE build argument.

Example:

1export DOCKER_BUILDKIT=1
2
3# Build and cache image
4$ docker build --tag mjhea0/docker-ci-cache:latest --build-arg BUILDKIT_INLINE_CACHE=1 .
5
6# Build an image from remote cache
7$ docker build --cache-from mjhea0/docker-ci-cache:latest .

Originally Posted here : https://todayilearnt.xyz/posts/nancy/faster_builds_with_docker_caching/

More articles from Nancy Chauhan

DNS Resolution

DNS acts as the phonebook of the internet 🌐. It converts a web address such as "example.com" to an IP address, which computers use to connect. As a result, we don't have to remember complicated IP addresses 🤩.

May 6th, 2020 · 2 min read

RootConf Delhi 2020

This year I had an opportunity to attend, volunteer, and deliver a Flash talk at Rootconf Delhi 2020. In this blog, I want to share my experience as a volunteer.

May 3rd, 2020 · 3 min read
© 2018–2020 Nancy Chauhan
Link to $https://twitter.com/_nancychauhanLink to $https://github.com/Nancy-ChauhanLink to $https://www.linkedin.com/in/nancy-chauhan/Link to $https://www.instagram.com/_nancychauhan/Link to $https://medium.com/@_nancychauhan