Graham McIntire
ea27e84613
Production wgrib2 3.8.0 still hit `*** FATAL ERROR: packing type 40 not supported ***` on HRDPS files. ldd confirmed wgrib2 was linking only libc/libm — no OpenJPEG, no Jasper, no g2c. wgrib2 3.8.0 cmake doesn't have USE_OPENJPEG or USE_JASPER options. The legacy flag names we passed were silently ignored. The actual knob is USE_G2CLIB_LOW, which links wgrib2 to the external g2c lib (already built in the previous stage with OpenJPEG support). Dropped the bogus USE_PNG / USE_OPENJPEG flags and added USE_G2CLIB_LOW=ON in both Dockerfiles. Verified by reading wgrib2 v3.8.0's CMakeLists.txt: the documented JPEG2000 path is "USE_G2CLIB_LOW: Use g2c low-level decoders (png,jpeg2000)?".
116 lines
4.9 KiB
Docker
116 lines
4.9 KiB
Docker
# syntax=docker/dockerfile:1.7
|
|
#
|
|
# Multi-arch build for `prop-grid-rs`. Self-contained: builds wgrib2 +
|
|
# NCEPLIBS-g2c from source so the image has no external private-registry
|
|
# dependency. An earlier revision did `FROM prop:latest AS wgrib2-src` to
|
|
# reuse the Elixir image's pre-built wgrib2, but docker buildx in CI
|
|
# couldn't always authenticate to the private registry during that
|
|
# implicit FROM pull — the resulting failures were opaque and broke
|
|
# every grid-rs image publish after Stream C.
|
|
#
|
|
# This image ships two binaries: `prop-grid-rs` (default ENTRYPOINT,
|
|
# drains grid_tasks) and `hrrr-point-worker` (k8s `command:` override,
|
|
# drains hrrr_fetch_tasks). Runtime uid 65534 matches the fsGroup on
|
|
# the NFS scores mount.
|
|
|
|
ARG DEBIAN_VERSION=trixie-slim
|
|
ARG RUNNER_IMAGE="docker.io/debian:${DEBIAN_VERSION}"
|
|
|
|
# ---- wgrib2 builder (mirrors the same stage in the Elixir Dockerfile) ----
|
|
FROM ${RUNNER_IMAGE} AS wgrib2-builder
|
|
|
|
ARG WGRIB2_VERSION=3.8.0
|
|
ARG G2C_VERSION=2.1.0
|
|
# wgrib2 3.6.0 has a memory-corruption bug exposed by HRDPS rotated
|
|
# lat/lon GRIB2 files plus -lon point extraction: it emits denormal
|
|
# garbage values then aborts with `free(): invalid size`. Reproduced
|
|
# directly in the production pod against an HRDPS sample on
|
|
# 2026-04-30. 3.8.0 (verified locally against the same file) doesn't
|
|
# have the bug. Keep both Dockerfiles aligned — bumping only one
|
|
# means the Elixir-side per-QSO HRDPS lookups would still hit the
|
|
# bad version.
|
|
|
|
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
|
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
|
rm -f /etc/apt/apt.conf.d/docker-clean \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
build-essential gfortran cmake wget ca-certificates pkg-config \
|
|
zlib1g-dev libaec-dev libpng-dev libopenjp2-7-dev
|
|
|
|
WORKDIR /tmp/g2c
|
|
RUN wget -q --content-disposition "https://github.com/NOAA-EMC/NCEPLIBS-g2c/archive/refs/tags/v${G2C_VERSION}.tar.gz" \
|
|
-O g2c.tar.gz \
|
|
&& tar xzf g2c.tar.gz \
|
|
&& cd NCEPLIBS-g2c-${G2C_VERSION} \
|
|
&& mkdir build && cd build \
|
|
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local \
|
|
-DCMAKE_BUILD_TYPE=Release \
|
|
-DUSE_PNG=ON -DUSE_AEC=ON -DUSE_OpenJPEG=ON -DUSE_Jasper=OFF \
|
|
&& make -j$(nproc) \
|
|
&& make install
|
|
|
|
WORKDIR /tmp/wgrib2
|
|
# JPEG2000 decode (GRIB2 packing type 40 — used by HRDPS) comes from the
|
|
# external g2c lib via USE_G2CLIB_LOW. Without that flag wgrib2's stock
|
|
# build links neither g2c nor any JPEG2000 backend (verified 2026-04-30
|
|
# by `ldd /usr/local/bin/wgrib2` showing only libc/libm and a runtime
|
|
# `*** FATAL ERROR: packing type 40 not supported ***` from HRDPS files).
|
|
#
|
|
# wgrib2 3.8.0 does NOT have USE_OPENJPEG or USE_JASPER cmake options —
|
|
# the only knob is USE_G2CLIB_LOW for png/jpeg2000 decode. The g2c lib
|
|
# we built above carries the actual OpenJPEG support.
|
|
RUN wget -q --content-disposition "https://github.com/NOAA-EMC/wgrib2/archive/refs/tags/v${WGRIB2_VERSION}.tar.gz" \
|
|
-O wgrib2.tar.gz \
|
|
&& tar xzf wgrib2.tar.gz \
|
|
&& cd wgrib2-${WGRIB2_VERSION} \
|
|
&& mkdir build && cd build \
|
|
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release \
|
|
-DUSE_AEC=ON -DUSE_G2CLIB_LOW=ON \
|
|
&& make -j$(nproc) \
|
|
&& make install \
|
|
&& strip /usr/local/bin/wgrib2
|
|
|
|
# ---- Rust build ----
|
|
FROM rust:1.94-trixie AS builder
|
|
WORKDIR /src
|
|
|
|
COPY Cargo.toml Cargo.lock* ./
|
|
COPY src ./src
|
|
COPY tests ./tests
|
|
|
|
# Registry + git caches are persistent; target/ is not (kept ephemeral
|
|
# to avoid the runner disk-pressure failures observed during Stream C
|
|
# rollout). Full rebuild adds ~3 min; it's still under 6 min total.
|
|
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
|
|
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
|
|
rustup component add clippy \
|
|
&& cargo clippy --all-targets -- -D warnings \
|
|
&& cargo test --release \
|
|
&& cargo build --release --bin worker --bin hrrr_point_worker \
|
|
&& cp target/release/worker /usr/local/bin/worker \
|
|
&& cp target/release/hrrr_point_worker /usr/local/bin/hrrr_point_worker
|
|
|
|
# ---- Runtime ----
|
|
FROM ${RUNNER_IMAGE} AS runtime
|
|
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
|
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
|
rm -f /etc/apt/apt.conf.d/docker-clean \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
ca-certificates tini \
|
|
libgfortran5 libaec0 zlib1g libpng16-16t64 libopenjp2-7
|
|
|
|
COPY --from=wgrib2-builder /usr/local/bin/wgrib2 /usr/local/bin/wgrib2
|
|
COPY --from=wgrib2-builder /usr/local/lib/libg2c.so /usr/local/lib/
|
|
COPY --from=wgrib2-builder /usr/local/lib/libg2c.so.0 /usr/local/lib/
|
|
RUN ldconfig
|
|
|
|
COPY --from=builder /usr/local/bin/worker /usr/local/bin/prop-grid-rs
|
|
COPY --from=builder /usr/local/bin/hrrr_point_worker /usr/local/bin/hrrr-point-worker
|
|
|
|
USER 65534:65534
|
|
# Default entrypoint is the grid chain worker. Override with `command:`
|
|
# in the k8s manifest for the per-QSO point worker.
|
|
ENTRYPOINT ["/usr/bin/tini", "--"]
|
|
CMD ["/usr/local/bin/prop-grid-rs"]
|