Graham McIntire
641789449a
Pods were OOMKilled at :05 every hour at the 3 GiB cgroup limit. Idle RSS climbed 750 MiB → 1.5 GiB across pod lifetime as the hourly f00 analysis cycled surface + pressure + native-duct blobs (50 MB / 400 MB / 530 MB) simultaneously during decode. glibc malloc retains dirty pages after freeing those large transient allocations, so RSS drifted upward with each cycle and the next hourly peak eventually crossed the limit. jemalloc's background decay returns pages to the kernel on a short schedule, so RSS tracks the actual working set. Wired in at lib.rs so both `worker` and `hrrr_point_worker` inherit it. Skipped on MSVC targets (Windows CI if we ever add one); dev builds on macOS still pick it up via the generic target_env gate.
47 lines
1.8 KiB
Rust
47 lines
1.8 KiB
Rust
//! Rust extraction of the Microwaveprop HRRR propagation pipeline.
|
|
//!
|
|
//! Each module mirrors the Elixir module it ports from so golden-fixture
|
|
//! tests can compare behavior 1:1. Module boundaries chosen so failures
|
|
//! during TDD fall into the smallest possible blast radius.
|
|
//!
|
|
//! Post-Phase-3: f00 analysis and f01..f18 forecasts both live here.
|
|
//! Port targets:
|
|
//! band_config ← lib/microwaveprop/propagation/band_config.ex
|
|
//! region ← lib/microwaveprop/propagation/region.ex
|
|
//! scorer ← lib/microwaveprop/propagation/scorer.ex
|
|
//! grid ← lib/microwaveprop/propagation/grid.ex
|
|
//! decoder ← lib/microwaveprop/weather/grib2/wgrib2.ex
|
|
//! fetcher ← lib/microwaveprop/weather/hrrr_client.ex
|
|
//! scores_file ← lib/microwaveprop/propagation/scores_file.ex
|
|
//! db — new; grid_tasks claim/complete + NOTIFY propagation_ready
|
|
//!
|
|
//! Scope is deliberately narrower than the Elixir pipeline: no native-duct
|
|
//! merge, no NEXRAD, no commercial-link boost. Those are f00-only in
|
|
//! production and Elixir keeps handling f00.
|
|
|
|
// jemalloc for all non-MSVC targets (Linux release builds land here).
|
|
// Set as the global allocator at the library root so every binary in
|
|
// the crate (worker, hrrr_point_worker) inherits it. See Cargo.toml
|
|
// for the rationale behind switching off glibc malloc.
|
|
#[cfg(not(target_env = "msvc"))]
|
|
#[global_allocator]
|
|
static GLOBAL: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;
|
|
|
|
pub mod band_config;
|
|
pub mod commercial;
|
|
pub mod db;
|
|
pub mod decoder;
|
|
pub mod duct;
|
|
pub mod fetcher;
|
|
pub mod grid;
|
|
pub mod hrrr_points;
|
|
pub mod metrics;
|
|
pub mod native_duct;
|
|
pub mod nexrad;
|
|
pub mod pipeline;
|
|
pub mod profiles_file;
|
|
pub mod region;
|
|
pub mod scorer;
|
|
pub mod scores_file;
|
|
pub mod sounding_params;
|
|
pub mod telemetry;
|