graham/prop
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(docker): use BuildKit cache mounts for apt so .debs skip the layer

Browse source 
The last attempt still blew past the CI runner's /var cap because
apt's pre-download free-space check runs against the archives dir,
and the image overlay is where that sits. Moving /var/cache/apt and
/var/lib/apt onto BuildKit cache mounts puts the .deb storage on the
builder's cache volume (outside the image overlay), so the free-space
check runs against whatever disk backs the cache — which on the CI
runner has room.

Requires the `syntax=docker/dockerfile:1.6` directive at the top for
the --mount flag. Also removes /etc/apt/apt.conf.d/docker-clean so
the Debian base's post-invoke hook doesn't wipe the cache mount on
every apt run.
This commit is contained in:
Graham McIntire 2026-04-18 11:07:45 -05:00
parent 6e72c6811d
commit 379b8ee357
No known key found for this signature in database
GPG key ID: F4ABF488E6029E59
1 changed files with 37 additions and 26 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

63
Dockerfile
View file

@ -1,3 +1,4 @@
# syntax=docker/dockerfile:1.6
# Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian # Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian
# instead of Alpine to avoid DNS resolution issues in production. # instead of Alpine to avoid DNS resolution issues in production.
# #
@ -29,11 +30,19 @@ ARG WGRIB2_VERSION=3.6.0
# experimental until 2.0.0. # experimental until 2.0.0.
ARG G2C_VERSION=2.1.0 ARG G2C_VERSION=2.1.0
RUN apt-get update \ # BuildKit cache mounts keep the .deb archive + apt-lists out of the
&& apt-get install -y --no-install-recommends \ # image overlay and on the builder's cache storage, so image layers
build-essential gfortran cmake wget ca-certificates pkg-config \ # don't pay the disk cost and CI runners with tight /var budgets can
zlib1g-dev libaec-dev libpng-dev libopenjp2-7-dev \ # still fetch ~100 MB of packages. The `rm -rf lists/*` below targets
&& rm -rf /var/lib/apt/lists/* # the cache-mount contents; BuildKit persists whatever remains for
# the next build.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
rm -f /etc/apt/apt.conf.d/docker-clean \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
build-essential gfortran cmake wget ca-certificates pkg-config \
zlib1g-dev libaec-dev libpng-dev libopenjp2-7-dev
# NCEPLIBS-g2c: wgrib2 >= 3.x delegates GRIB2 decompression to this # NCEPLIBS-g2c: wgrib2 >= 3.x delegates GRIB2 decompression to this
# library via find_package whenever any of JASPER / OPENJPEG / PNG # library via find_package whenever any of JASPER / OPENJPEG / PNG
@ -81,9 +90,11 @@ RUN wget -q --content-disposition "https://github.com/NOAA-EMC/wgrib2/archive/re
FROM ${BUILDER_IMAGE} AS builder FROM ${BUILDER_IMAGE} AS builder
# install build dependencies # install build dependencies
RUN apt-get update \ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
&& apt-get install -y --no-install-recommends build-essential git \ --mount=type=cache,target=/var/lib/apt,sharing=locked \
&& rm -rf /var/lib/apt/lists/* rm -f /etc/apt/apt.conf.d/docker-clean \
&& apt-get update \
&& apt-get install -y --no-install-recommends build-essential git
# prepare build dir # prepare build dir
WORKDIR /app WORKDIR /app
@ -142,25 +153,25 @@ FROM ${RUNNER_IMAGE} AS final
# compile cache. Read at runtime by Microwaveprop.Application.build_timestamp/0. # compile cache. Read at runtime by Microwaveprop.Application.build_timestamp/0.
ARG BUILD_TIMESTAMP=unknown ARG BUILD_TIMESTAMP=unknown
# Base runtime deps, without cdo. `APT::Keep-Downloaded-Packages=false` # Runtime deps. BuildKit cache mounts keep the .deb archives and apt
# deletes each .deb immediately after install so apt's archive cache # lists out of the image layer, so the CI builder's tight /var disk
# doesn't pile up and OOM the /var/cache/apt disk partition the CI # cap doesn't trip during install. cdo is split into its own RUN
# builder runs on. # layer because its dep tree (netCDF/HDF5/ecCodes/Fortran) is ~1 GB
RUN apt-get update \ # and we'd rather not have a single failed cdo install invalidate the
&& apt-get -o APT::Keep-Downloaded-Packages=false install -y --no-install-recommends \ # base-runtime layer.
libstdc++6 openssl libncurses6 locales ca-certificates snmp \ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
libgfortran5 libaec0 zlib1g libpng16-16t64 libopenjp2-7 \ --mount=type=cache,target=/var/lib/apt,sharing=locked \
&& apt-get clean \ rm -f /etc/apt/apt.conf.d/docker-clean \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*.deb && apt-get update \
&& apt-get install -y --no-install-recommends \
libstdc++6 openssl libncurses6 locales ca-certificates snmp \
libgfortran5 libaec0 zlib1g libpng16-16t64 libopenjp2-7
# cdo is ~1 GB of transitive deps (netCDF, HDF5, ecCodes, Fortran RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
# runtime). Splitting into its own RUN layer means the base deps above --mount=type=cache,target=/var/lib/apt,sharing=locked \
# are already extracted + pruned before the cdo cache starts filling, rm -f /etc/apt/apt.conf.d/docker-clean \
# which is what kept the image under the 2 GB CI disk cap. && apt-get update \
RUN apt-get update \ && apt-get install -y --no-install-recommends cdo
&& apt-get -o APT::Keep-Downloaded-Packages=false install -y --no-install-recommends cdo \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*.deb
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen