perf(grid-rs): cargo-chef + unified --release profile in CI build

The prior Docker build re-compiled all 546 transitive Rust deps from
scratch on every push because target/ is intentionally not cache-mounted
(Stream C disk-pressure failures). Two changes drop the cold-build cost
substantially:

* cargo-chef splits dep compilation into a separate Docker layer keyed
  on recipe.json (derived from Cargo.toml/Cargo.lock). Source-only
  pushes — the common case — now reuse the cooked deps as a normal
  layer instead of re-running cargo for every crate.
* clippy now runs in --release profile so it shares artifacts with
  `cargo test --release` and `cargo build --release`. Previously clippy
  ran in dev profile, forcing a full second compile of every dep before
  the release build could start.

Verified `cargo clippy --release --all-targets -- -D warnings` is clean
locally — no new lints surface under the release profile.
This commit is contained in:
Graham McIntire 2026-04-30 09:54:25 -05:00
parent 117c8e7b73
commit 3b9e8863bc
No known key found for this signature in database
GPG key ID: F4ABF488E6029E59

View file

@ -73,20 +73,51 @@ RUN wget -q --content-disposition "https://github.com/NOAA-EMC/wgrib2/archive/re
&& strip /usr/local/bin/wgrib2
# ---- Rust build ----
FROM rust:1.94-trixie AS builder
#
# Three-stage cargo-chef layout. The `cook` layer compiles every
# dependency in release profile and is cached on the content of
# recipe.json (derived from Cargo.toml/Cargo.lock only), so source-only
# pushes skip the slow ~3 min dep compile entirely. Without chef each
# CI build re-compiled all 546 transitive deps from scratch since
# `target/` is intentionally not cache-mounted (Stream C disk-pressure
# failures); chef sidesteps that by keeping the cooked artifacts as a
# normal Docker layer that BuildKit reuses across pushes.
#
# All three cargo invocations run in the release profile so they share
# one set of artifacts. Previously clippy ran in the dev profile, which
# forced a full second compile of every dep before `cargo test --release`
# could start.
FROM rust:1.94-trixie AS chef
WORKDIR /src
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
cargo install --locked --version 0.1.77 cargo-chef \
&& rustup component add clippy
FROM chef AS planner
COPY Cargo.toml Cargo.lock* ./
COPY src ./src
COPY tests ./tests
RUN cargo chef prepare --recipe-path recipe.json
FROM chef AS builder
COPY --from=planner /src/recipe.json recipe.json
# Cook stage: layer is cached unless recipe.json changes, i.e. unless
# Cargo.toml or Cargo.lock change. Source edits leave it intact.
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
cargo chef cook --release --recipe-path recipe.json
# Source comes in only after deps are cooked, so a source-only edit
# only invalidates this final RUN — not the cook layer above.
COPY Cargo.toml Cargo.lock* ./
COPY src ./src
COPY tests ./tests
# Registry + git caches are persistent; target/ is not (kept ephemeral
# to avoid the runner disk-pressure failures observed during Stream C
# rollout). Full rebuild adds ~3 min; it's still under 6 min total.
RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \
--mount=type=cache,target=/usr/local/cargo/git,sharing=locked \
rustup component add clippy \
&& cargo clippy --all-targets -- -D warnings \
cargo clippy --release --all-targets -- -D warnings \
&& cargo test --release \
&& cargo build --release --bin worker --bin hrrr_point_worker \
&& cp target/release/worker /usr/local/bin/worker \