prop/rust/prop_grid_rs/Dockerfile
Graham McIntire bfef11dbdf
Some checks failed
Build base image / Build and push base image (push) Successful in 14s
Build and Push / Build CI test image (push) Successful in 14s
Build and Push / Build and Push Docker Image (push) Failing after 18m31s
Build prop-grid-rs / Test, build, push (push) Successful in 23m55s
fix(ci): stage band_weights.json into the prop-grid-rs build context
json_weights_golden reads priv/algo/band_weights.json via a bare
relative path. That resolves inside `cargo test` at the crate root but
not inside the image build, whose context is rust/prop_grid_rs and
whose WORKDIR is /src — so the test aborted with "JSON file must
exist" and the build never produced an image.

grid-rs CI has been red since 2fd88a94, the commit that added the
test. Production still runs main-1785606663-95976b6 (95976b6b, the
last green build), so every prop-grid-rs change since then — including
the HRDPS rotated-pole decode fix — has silently failed to ship.

Three parts:
  * resolve the fixture from CARGO_MANIFEST_DIR so the path no longer
    depends on the cwd
  * stage the file into the build context in the workflow
  * COPY it to /priv in the builder stage, where
    CARGO_MANIFEST_DIR/../../priv resolves from /src
2026-08-05 18:01:24 -05:00

154 lines
6.7 KiB
Docker

# syntax=docker/dockerfile:1.7
#
# Multi-arch build for `prop-grid-rs`. Self-contained: builds wgrib2 +
# NCEPLIBS-g2c from source so the image has no external private-registry
# dependency. An earlier revision did `FROM prop:latest AS wgrib2-src` to
# reuse the Elixir image's pre-built wgrib2, but docker buildx in CI
# couldn't always authenticate to the private registry during that
# implicit FROM pull — the resulting failures were opaque and broke
# every grid-rs image publish after Stream C.
#
# This image ships two binaries: `prop-grid-rs` (default ENTRYPOINT,
# drains grid_tasks) and `hrrr-point-worker` (k8s `command:` override,
# drains hrrr_fetch_tasks). Runtime uid 65534 matches the fsGroup on
# the NFS scores mount.
ARG DEBIAN_VERSION=trixie-slim
ARG RUNNER_IMAGE="docker.io/debian:${DEBIAN_VERSION}"
# ---- wgrib2 builder (mirrors the same stage in the Elixir Dockerfile) ----
FROM ${RUNNER_IMAGE} AS wgrib2-builder
ARG WGRIB2_VERSION=3.8.0
# wgrib2 3.8.0's CMakeLists.txt requires g2c >= 2.3.0 via find_package.
ARG G2C_VERSION=2.3.0
# wgrib2 3.6.0 has a memory-corruption bug exposed by HRDPS rotated
# lat/lon GRIB2 files plus -lon point extraction: it emits denormal
# garbage values then aborts with `free(): invalid size`. Reproduced
# directly in the production pod against an HRDPS sample on
# 2026-04-30. 3.8.0 (verified locally against the same file) doesn't
# have the bug. Keep both Dockerfiles aligned — bumping only one
# means the Elixir-side per-QSO HRDPS lookups would still hit the
# bad version.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
rm -f /etc/apt/apt.conf.d/docker-clean \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
build-essential gfortran cmake wget ca-certificates pkg-config \
zlib1g-dev libaec-dev libpng-dev libopenjp2-7-dev
WORKDIR /tmp/g2c
RUN wget -q --content-disposition "https://github.com/NOAA-EMC/NCEPLIBS-g2c/archive/refs/tags/v${G2C_VERSION}.tar.gz" \
-O g2c.tar.gz \
&& tar xzf g2c.tar.gz \
&& cd NCEPLIBS-g2c-${G2C_VERSION} \
&& mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local \
-DCMAKE_BUILD_TYPE=Release \
-DUSE_PNG=ON -DUSE_AEC=ON -DUSE_OpenJPEG=ON -DUSE_Jasper=OFF \
&& make -j$(nproc) \
&& make install
WORKDIR /tmp/wgrib2
# JPEG2000 decode (GRIB2 packing type 40 — used by HRDPS) comes from the
# external g2c lib via USE_G2CLIB_LOW. Without that flag wgrib2's stock
# build links neither g2c nor any JPEG2000 backend (verified 2026-04-30
# by `ldd /usr/local/bin/wgrib2` showing only libc/libm and a runtime
# `*** FATAL ERROR: packing type 40 not supported ***` from HRDPS files).
#
# wgrib2 3.8.0 does NOT have USE_OPENJPEG or USE_JASPER cmake options —
# the only knob is USE_G2CLIB_LOW for png/jpeg2000 decode. The g2c lib
# we built above carries the actual OpenJPEG support.
RUN wget -q --content-disposition "https://github.com/NOAA-EMC/wgrib2/archive/refs/tags/v${WGRIB2_VERSION}.tar.gz" \
-O wgrib2.tar.gz \
&& tar xzf wgrib2.tar.gz \
&& cd wgrib2-${WGRIB2_VERSION} \
&& mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release \
-DUSE_AEC=ON -DUSE_G2CLIB_LOW=ON \
&& make -j$(nproc) \
&& make install \
&& strip /usr/local/bin/wgrib2
# ---- Rust build ----
#
# Three-stage cargo-chef layout. The `cook` layer compiles every
# dependency in release profile and is cached on the content of
# recipe.json (derived from Cargo.toml/Cargo.lock only), so source-only
# pushes skip the slow ~3 min dep compile entirely. Without chef each
# CI build re-compiled all 546 transitive deps from scratch since
# `target/` is intentionally not cache-mounted (Stream C disk-pressure
# failures); chef sidesteps that by keeping the cooked artifacts as a
# normal Docker layer that BuildKit reuses across pushes.
#
# All three cargo invocations run in the release profile so they share
# one set of artifacts. Previously clippy ran in the dev profile, which
# forced a full second compile of every dep before `cargo test --release`
# could start.
FROM rust:1.94-trixie AS chef
WORKDIR /src
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
cargo install --locked --version 0.1.77 cargo-chef \
&& rustup component add clippy
FROM chef AS planner
COPY Cargo.toml Cargo.lock* ./
COPY src ./src
COPY tests ./tests
RUN cargo chef prepare --recipe-path recipe.json
FROM chef AS builder
COPY --from=planner /src/recipe.json recipe.json
# Cook stage: layer is cached unless recipe.json changes, i.e. unless
# Cargo.toml or Cargo.lock change. Source edits leave it intact.
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
cargo chef cook --release --recipe-path recipe.json
# Source comes in only after deps are cooked, so a source-only edit
# only invalidates this final RUN — not the cook layer above.
COPY Cargo.toml Cargo.lock* ./
COPY src ./src
COPY tests ./tests
# `json_weights_golden` compares the compiled-in band weights against
# priv/algo/band_weights.json, which lives outside this build context.
# The workflow stages it into the context first; WORKDIR is /src, so
# CARGO_MANIFEST_DIR/../../priv lands at /priv.
COPY priv /priv
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
cargo clippy --release --all-targets -- -D warnings \
&& cargo test --release \
&& cargo build --release --bin worker --bin hrrr_point_worker \
&& cp target/release/worker /usr/local/bin/worker \
&& cp target/release/hrrr_point_worker /usr/local/bin/hrrr_point_worker
# ---- Runtime ----
FROM ${RUNNER_IMAGE} AS runtime
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
rm -f /etc/apt/apt.conf.d/docker-clean \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates tini \
libgfortran5 libaec0 zlib1g libpng16-16t64 libopenjp2-7
COPY --from=wgrib2-builder /usr/local/bin/wgrib2 /usr/local/bin/wgrib2
COPY --from=wgrib2-builder /usr/local/lib/libg2c.so /usr/local/lib/
COPY --from=wgrib2-builder /usr/local/lib/libg2c.so.0 /usr/local/lib/
RUN ldconfig
COPY --from=builder /usr/local/bin/worker /usr/local/bin/prop-grid-rs
COPY --from=builder /usr/local/bin/hrrr_point_worker /usr/local/bin/hrrr-point-worker
USER 65534:65534
# Default entrypoint is the grid chain worker. Override with `command:`
# in the k8s manifest for the per-QSO point worker.
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["/usr/local/bin/prop-grid-rs"]