Graham McIntire
02b354f5f7
The grid worker stopped writing HRRR profiles to the DB on April 14 when the forecast grid moved to /data/scores binary files, but the 24/72h-windowed prune left every is_grid_point=true row older than 72 hours sitting in the table (billions of rows across every partition back to 2016, ~90% of the 146 GB table size). Nothing reads them anymore. Replace prune_old_grid_profiles/0 with purge_grid_point_profiles/0, which walks every hrrr_profiles partition directly and deletes is_grid_point=true rows regardless of age. Per-partition DELETEs avoid a parent-table full scan. QSO-linked rows (is_grid_point=false) are untouched — those still back contact enrichment and /path. Call the new purge from the grid worker's end-of-chain hook so any grid-point row that somehow slips back in gets cleaned up within the next hour. Add `mix hrrr.purge_grid_points` for a one-shot historical cleanup.
28 lines
894 B
Elixir
28 lines
894 B
Elixir
defmodule Mix.Tasks.Hrrr.PurgeGridPoints do
|
|
@shortdoc "Delete every `is_grid_point = true` row from hrrr_profiles"
|
|
|
|
@moduledoc """
|
|
Removes every `is_grid_point = true` row from `hrrr_profiles` across every
|
|
partition. The propagation grid now lives in `/data/scores` binary files,
|
|
nothing reads grid-point rows anymore, and the grid worker no longer writes
|
|
them — this task reclaims the disk space from the legacy data that
|
|
accumulated before the April 14 cutover.
|
|
|
|
Leaves `is_grid_point = false` (QSO-linked) rows untouched.
|
|
|
|
mix hrrr.purge_grid_points
|
|
"""
|
|
|
|
use Mix.Task
|
|
|
|
alias Microwaveprop.Weather
|
|
|
|
@impl Mix.Task
|
|
def run(_argv) do
|
|
Mix.Task.run("app.start")
|
|
|
|
Mix.shell().info("Purging grid-point rows from hrrr_profiles...")
|
|
deleted = Weather.purge_grid_point_profiles()
|
|
Mix.shell().info("Purged #{deleted} grid-point rows total.")
|
|
end
|
|
end
|