Graham McIntire
4fefb81c24
Phase 3 Stream C Rust side. Completes the HrrrFetchWorker port. Pipeline: - db::claim_next_hrrr_task — FOR UPDATE SKIP LOCKED on hrrr_fetch_tasks, newest valid_time first. Accepts the points JSONB directly. - hrrr_points::process_batch — fetch surface + pressure GRIB2 once per task (tokio::try_join), decode via the existing wgrib2 plumbing, then for each requested point pull the cell and UPSERT INTO hrrr_profiles (conflict on lat/lon/valid_time). - db::complete_hrrr_task / fail_hrrr_task — status transitions; Elixir backfill re-enqueues failed rows on next /30-min scan. Shipping pieces: - new bin src/bin/hrrr_point_worker.rs - new module src/hrrr_points.rs (process_batch, upsert_profile) - new Cargo [[bin]] entry; Dockerfile builds both binaries in one stage and ships them in the runtime image so a single CI pipeline covers the whole cluster - k8s/deployment-hrrr-point-rs.yaml (1 replica, 1 Gi limit, anti-affinity against prop-grid-rs so chain + point work don't fight for wgrib2 slots). Uses the same image; command: override picks the right binary. - kustomization.yaml: include the new deployment so flux applies it - deployment-grid-rs.yaml: bump readiness initialDelaySeconds 3→15 + failureThreshold 3→6 so a slow DB connect during startup can't race the first probe 119 Rust tests green.
65 lines
2.5 KiB
Docker
65 lines
2.5 KiB
Docker
# syntax=docker/dockerfile:1.7
|
|
#
|
|
# Multi-arch build for `prop-grid-rs`. wgrib2 isn't in Debian apt — the
|
|
# Elixir image builds it from source in its own stage. Rather than
|
|
# rebuild it here (~5 min per CI run), copy the binary + libg2c out of
|
|
# the already-published Elixir image.
|
|
#
|
|
# Runtime uid 65534 matches the fsGroup on the NFS scores mount.
|
|
ARG WGRIB2_IMAGE=git.mcintire.me/graham/prop:latest
|
|
|
|
FROM ${WGRIB2_IMAGE} AS wgrib2-src
|
|
|
|
FROM rust:1.94-trixie AS builder
|
|
WORKDIR /src
|
|
|
|
# Cache deps: copy manifests first so dependency changes alone don't
|
|
# invalidate the source-layer cache.
|
|
COPY Cargo.toml Cargo.lock* ./
|
|
COPY src ./src
|
|
COPY tests ./tests
|
|
|
|
# BuildKit cache mounts persist cargo's registry + git + target
|
|
# directories across CI runs, so dependency changes are the only
|
|
# thing that triggers a full rebuild. Lint + test gate the image
|
|
# build — a regressed scorer or a clippy warning never produces a
|
|
# pushable image. `target/` is cached but the final binary is
|
|
# copied out into a non-cached path so the runtime stage can reach
|
|
# it (COPY --from= can't read from a cache mount).
|
|
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
|
|
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
|
|
--mount=type=cache,target=/src/target,sharing=locked \
|
|
rustup component add clippy \
|
|
&& cargo clippy --all-targets -- -D warnings \
|
|
&& cargo test --release \
|
|
&& cargo build --release --bin worker --bin hrrr_point_worker \
|
|
&& cp target/release/worker /usr/local/bin/worker \
|
|
&& cp target/release/hrrr_point_worker /usr/local/bin/hrrr_point_worker
|
|
|
|
# Runtime image: slim debian + the shared libs wgrib2 links against.
|
|
# libgfortran5/libaec0/zlib1g/libpng16-16t64/libopenjp2-7 mirror the
|
|
# Elixir runtime stage.
|
|
FROM debian:trixie-slim AS runtime
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
ca-certificates \
|
|
tini \
|
|
libgfortran5 \
|
|
libaec0 \
|
|
zlib1g \
|
|
libpng16-16t64 \
|
|
libopenjp2-7 \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
COPY --from=wgrib2-src /usr/local/bin/wgrib2 /usr/local/bin/wgrib2
|
|
COPY --from=wgrib2-src /usr/local/lib/libg2c.so* /usr/local/lib/
|
|
RUN ldconfig
|
|
|
|
COPY --from=builder /usr/local/bin/worker /usr/local/bin/prop-grid-rs
|
|
COPY --from=builder /usr/local/bin/hrrr_point_worker /usr/local/bin/hrrr-point-worker
|
|
|
|
USER 65534:65534
|
|
# Default entrypoint is the grid chain worker. Override with `command:`
|
|
# in the k8s manifest (or `docker run ... hrrr-point-worker`) for the
|
|
# per-QSO point worker.
|
|
ENTRYPOINT ["/usr/bin/tini", "--"]
|
|
CMD ["/usr/local/bin/prop-grid-rs"]
|