fix(docker): use BuildKit cache mounts for apt so .debs skip the layer
The last attempt still blew past the CI runner's /var cap because apt's pre-download free-space check runs against the archives dir, and the image overlay is where that sits. Moving /var/cache/apt and /var/lib/apt onto BuildKit cache mounts puts the .deb storage on the builder's cache volume (outside the image overlay), so the free-space check runs against whatever disk backs the cache — which on the CI runner has room. Requires the `syntax=docker/dockerfile:1.6` directive at the top for the --mount flag. Also removes /etc/apt/apt.conf.d/docker-clean so the Debian base's post-invoke hook doesn't wipe the cache mount on every apt run.
This commit is contained in:
parent
6e72c6811d
commit
379b8ee357
1 changed files with 37 additions and 26 deletions
57
Dockerfile
57
Dockerfile
|
|
@ -1,3 +1,4 @@
|
|||
# syntax=docker/dockerfile:1.6
|
||||
# Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian
|
||||
# instead of Alpine to avoid DNS resolution issues in production.
|
||||
#
|
||||
|
|
@ -29,11 +30,19 @@ ARG WGRIB2_VERSION=3.6.0
|
|||
# experimental until 2.0.0.
|
||||
ARG G2C_VERSION=2.1.0
|
||||
|
||||
RUN apt-get update \
|
||||
# BuildKit cache mounts keep the .deb archive + apt-lists out of the
|
||||
# image overlay and on the builder's cache storage, so image layers
|
||||
# don't pay the disk cost and CI runners with tight /var budgets can
|
||||
# still fetch ~100 MB of packages. The `rm -rf lists/*` below targets
|
||||
# the cache-mount contents; BuildKit persists whatever remains for
|
||||
# the next build.
|
||||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||||
rm -f /etc/apt/apt.conf.d/docker-clean \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
build-essential gfortran cmake wget ca-certificates pkg-config \
|
||||
zlib1g-dev libaec-dev libpng-dev libopenjp2-7-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
zlib1g-dev libaec-dev libpng-dev libopenjp2-7-dev
|
||||
|
||||
# NCEPLIBS-g2c: wgrib2 >= 3.x delegates GRIB2 decompression to this
|
||||
# library via find_package whenever any of JASPER / OPENJPEG / PNG
|
||||
|
|
@ -81,9 +90,11 @@ RUN wget -q --content-disposition "https://github.com/NOAA-EMC/wgrib2/archive/re
|
|||
FROM ${BUILDER_IMAGE} AS builder
|
||||
|
||||
# install build dependencies
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends build-essential git \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||||
rm -f /etc/apt/apt.conf.d/docker-clean \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y --no-install-recommends build-essential git
|
||||
|
||||
# prepare build dir
|
||||
WORKDIR /app
|
||||
|
|
@ -142,25 +153,25 @@ FROM ${RUNNER_IMAGE} AS final
|
|||
# compile cache. Read at runtime by Microwaveprop.Application.build_timestamp/0.
|
||||
ARG BUILD_TIMESTAMP=unknown
|
||||
|
||||
# Base runtime deps, without cdo. `APT::Keep-Downloaded-Packages=false`
|
||||
# deletes each .deb immediately after install so apt's archive cache
|
||||
# doesn't pile up and OOM the /var/cache/apt disk partition the CI
|
||||
# builder runs on.
|
||||
RUN apt-get update \
|
||||
&& apt-get -o APT::Keep-Downloaded-Packages=false install -y --no-install-recommends \
|
||||
# Runtime deps. BuildKit cache mounts keep the .deb archives and apt
|
||||
# lists out of the image layer, so the CI builder's tight /var disk
|
||||
# cap doesn't trip during install. cdo is split into its own RUN
|
||||
# layer because its dep tree (netCDF/HDF5/ecCodes/Fortran) is ~1 GB
|
||||
# and we'd rather not have a single failed cdo install invalidate the
|
||||
# base-runtime layer.
|
||||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||||
rm -f /etc/apt/apt.conf.d/docker-clean \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
libstdc++6 openssl libncurses6 locales ca-certificates snmp \
|
||||
libgfortran5 libaec0 zlib1g libpng16-16t64 libopenjp2-7 \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*.deb
|
||||
libgfortran5 libaec0 zlib1g libpng16-16t64 libopenjp2-7
|
||||
|
||||
# cdo is ~1 GB of transitive deps (netCDF, HDF5, ecCodes, Fortran
|
||||
# runtime). Splitting into its own RUN layer means the base deps above
|
||||
# are already extracted + pruned before the cdo cache starts filling,
|
||||
# which is what kept the image under the 2 GB CI disk cap.
|
||||
RUN apt-get update \
|
||||
&& apt-get -o APT::Keep-Downloaded-Packages=false install -y --no-install-recommends cdo \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*.deb
|
||||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||||
rm -f /etc/apt/apt.conf.d/docker-clean \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y --no-install-recommends cdo
|
||||
|
||||
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue