Commit graph

3 commits

Author SHA1 Message Date
7b78a2574c
fix(workers): 3 bug/perf fixes from codebase review
1. GridCache: auto-release fill lock when the claimer process crashes.
   claim_fill/1 + release_fill/1 go through the GenServer so the
   server can Process.monitor the caller and clean up the ETS entry
   on :DOWN. Clear/0 now resets both the data table and the lock
   table. Fixes a latent bug where a crashed fill leaked the lock
   indefinitely, preventing every subsequent /weather mount for that
   valid_time from claiming and leaving cache cold.

2. RadarFrameWorker: distinguish permanent vs transient fetch errors.
   404 from the IEM n0q archive is permanent (file will never exist)
   and marks contacts :unavailable as before. Any other error shape
   (5xx, timeout, transport failure) now returns {:error, reason}
   so Oban retries — previously those also pinned contacts at
   :unavailable after a transient outage.

3. AdminTaskWorker.native_derive: replace per-row Repo.update_all
   (N round-trips + N fsyncs) with one UPDATE ... FROM unnest(...)
   per 2000-row batch. For the 10k-profile budget this is one
   network round trip per chunk instead of 10k, and one fsync per
   chunk instead of 10k. Restructured the clause to separate
   derivation (pure) from persistence (I/O).

All three changes are test-covered (grid_cache_test auto-release
test, radar_frame_worker_test 5xx + transport tests, existing
admin_task_worker_test native_derive coverage exercises the new
bulk path). Also drops the scorer_diff no-op test that was
verifying the clause removed in 61da51c.
2026-04-21 09:53:13 -05:00
63a9417287
fix(admin_task_worker): unknown feature now returns error instead of falling through
Backtest branch's unknown-feature guard was dead code: the if block's
{:error, _} return was discarded and execution fell through, running the
backtest with a just-created atom and writing a report for nonsense
features. Two compounding issues:

- function_exported?/3 returns false for modules not yet loaded in the
  BEAM, so even valid feature names triggered the warning path on a
  cold VM.
- String.to_atom/1 on caller-supplied strings created unbounded atoms.

Fix: Code.ensure_loaded?(Features) before the check, String.to_existing_atom/1
with rescue (Features exports __info__(:functions) so every valid function
atom already exists), and an explicit early return via a with pipeline so
the error tuple actually leaves the worker.
2026-04-16 14:58:55 -05:00
a6a670aee7
test: cover admin_task_worker dispatcher
18 characterization tests, one per task branch: backtest, backtest_all,
climatology (aggregation math + min_samples + is_grid_point + empty),
native_derive (filters + limit), recalibrate, scorer_diff, unknown-task
handling, and Oban worker metadata. Tests redirect priv/backtest_reports
writes to a tmp cwd to avoid clobbering checked-in baselines.
2026-04-16 14:58:54 -05:00