How to Use Pulled Image as a Cache with `docker buildx`

Question

We're building Docker images in our CI, but we can't get docker buildx build to utilise a pulled image as a cache.

Here are the docker commands that are executed:

docker pull "ghcr.io/foo/bar/baz"
docker buildx build . --tag "ghcr.io/foo/bar/baz"
docker push "ghcr.io/foo/bar/baz"

How can we modify docker buildx build to use the pulled image as a cache, ensuring that not every RUN is executed if the command remains unchanged?

Here's our Dockerfile:

FROM amd64/alpine:3.16 as build0

RUN apk update && apk add autoconf bash [...]
FROM build0 as build1

RUN mkdir /tmp/build-deps && cd /tmp/build-deps && [...]

FROM build1 as build2
RUN cd /tmp/build-deps && wget ${patch_uri} && [...]

FROM build2 as build3
RUN mkdir /root/build-deps && [...]

FROM build3 as build4
RUN cd /tmp/build-deps/ && mkdir php-ext && cd php-ext && [...]

Answer 1

Use the --cache-from option when building with docker buildx

docker pull "ghcr.io/foo/bar/baz"
docker buildx build . --tag "ghcr.io/foo/bar/baz" --cache-from="ghcr.io/foo/bar/baz"
docker push "ghcr.io/foo/bar/baz"

By specifying this option, Docker will re-use layers from the pulled image where the initial steps have not changed in the Dockerfile and the layers are the same.

Answer 2

To utilize a pulled image as a cache in Docker Buildx, you can use the --build-arg BUILDKIT_INLINE_CACHE=1 argument with the docker buildx build command. This will enable BuildKit inline caching and allow the pulled image to be used as a cache for subsequent builds.

Here's how you can modify your docker buildx build command to use the pulled image as a cache:

docker buildx build . \
  --tag "ghcr.io/foo/bar/baz" \
  --build-arg BUILDKIT_INLINE_CACHE=1 \
  --cache-from "type=registry,ref=ghcr.io/foo/bar/baz" \
  --cache-to "type=registry,ref=ghcr.io/foo/bar/baz" \  # optional
  --push

Here's an explanation of the additional options:

• --build-arg BUILDKIT_INLINE_CACHE=1: This enables inline caching with BuildKit.
• --cache-from "type=registry,ref=ghcr.io/foo/bar/baz": This specifies the image to use as a cache. In this case, it uses the image you pulled from ghcr.io/foo/bar/baz.
• --cache-to "type=registry,ref=ghcr.io/foo/bar/baz": This specifies where to store the cache. It's the same registry and reference as the image you are building. This allows you to reuse the cache in subsequent builds.

With these modifications, Docker Buildx will try to use the pulled image as a cache during the build process, ensuring that not every RUN step is executed if the command remains unchanged. If the cache is available and the commands in your Dockerfile have not changed, Docker will reuse the cached layers, which can significantly speed up your CI builds.