Graham McIntire
1086f52c85
Both RTMA and METAR5 schemas + clients + workers were defined but never had a consumer. `find_nearest_rtma/3` and `recent_surface_obs/3` existed in `Microwaveprop.Weather` with zero callers cluster-wide; RtmaFetchWorker had test coverage but was never enqueued anywhere; no IEM 5-min ASOS fetcher was ever written. The tables sat empty in prod and the recalibrate audit was permanently flagging them BROKEN even though the empty state was the steady state. Drop the lot — schemas, clients, workers, queue config, Req.Test stubs, audit checks, and the per-table unique tests in the observation-changesets suite. New migration drops the now-unreferenced `rtma_observations` and `metar_5min_observations` tables (both 0 rows in prod). Trim the `:rtma` slot out of `backfill_only_queues` in runtime.exs so Oban Pro's Smart engine stops trying to manage a queue with no producer. Audit thresholds reset on the surviving NARR check: drop the "WARN < 5000 absolute rows" rule that didn't account for NarrFetchWorker's 0.13° spatial dedup, and point remediation at the existing BackfillEnqueueWorker cron instead of the dev-only `mix narr.backfill` task. scripts/recalibrate_algo.py: ROW_COUNT_SOURCES + DATA_GAP_SQL + DATA_GAP_RULES all stop referencing the dropped tables.
58 lines
2 KiB
Elixir
58 lines
2 KiB
Elixir
defmodule Microwaveprop.Repo.Migrations.DropUnusedRtmaMetar5Tables do
|
|
use Ecto.Migration
|
|
|
|
# Both tables landed via earlier "planned multi-source atmospheric
|
|
# data" work that never wired into scoring or enrichment. RtmaClient
|
|
# / RtmaFetchWorker / Metar5minObservation existed but had zero
|
|
# callers cluster-wide; recalibrate's audit kept flagging them as
|
|
# BROKEN even though the empty state was the steady state. Drop the
|
|
# tables along with the unused code so a future contributor can't
|
|
# read them as "real but stalled" data sources.
|
|
|
|
def up do
|
|
drop_if_exists table(:rtma_observations)
|
|
drop_if_exists table(:metar_5min_observations)
|
|
end
|
|
|
|
def down do
|
|
create table(:rtma_observations, primary_key: false) do
|
|
add :id, :binary_id, primary_key: true
|
|
add :valid_time, :utc_datetime, null: false
|
|
add :lat, :float, null: false
|
|
add :lon, :float, null: false
|
|
add :temp_c, :float
|
|
add :dewpoint_c, :float
|
|
add :pressure_mb, :float
|
|
add :wind_u_ms, :float
|
|
add :wind_v_ms, :float
|
|
add :visibility_m, :float
|
|
add :precip_mm, :float
|
|
|
|
timestamps(type: :utc_datetime)
|
|
end
|
|
|
|
create unique_index(:rtma_observations, [:lat, :lon, :valid_time])
|
|
create index(:rtma_observations, [:valid_time])
|
|
|
|
create table(:metar_5min_observations, primary_key: false) do
|
|
add :id, :binary_id, primary_key: true, null: false
|
|
add :station_id, references(:weather_stations, type: :binary_id, on_delete: :delete_all)
|
|
add :observed_at, :utc_datetime, null: false
|
|
add :temp_f, :float
|
|
add :dewpoint_f, :float
|
|
add :relative_humidity, :float
|
|
add :wind_speed_kts, :float
|
|
add :wind_direction_deg, :integer
|
|
add :sea_level_pressure_mb, :float
|
|
add :altimeter_setting, :float
|
|
add :sky_condition, :string
|
|
add :precip_1h_in, :float
|
|
add :wx_codes, :string
|
|
|
|
timestamps(type: :utc_datetime)
|
|
end
|
|
|
|
create unique_index(:metar_5min_observations, [:station_id, :observed_at])
|
|
create index(:metar_5min_observations, [:observed_at])
|
|
end
|
|
end
|