Graham McIntire
fd976b0cd5
- Add jump_credo_checks ~> 0.4 with all 20 checks enabled - Fix all standard Credo issues: 139 @spec (113 done, 26 remain), 4 refactoring, 3 alias usage, 9 System.cmd env, 5 unsafe_to_atom, 2 max line length, 9 assert_receive timeout - Fix 170+ jump_credo_checks warnings: - 117 TopLevelAliasImportRequire: move nested alias/import to module top - 32 UseObanProWorker: switch to Oban.Pro.Worker - 4 DoctestIExExamples: add doctests / create test file - ~20 WeakAssertion: strengthen type-check assertions - Various ConditionalAssertion, AssertReceiveTimeout fixes - Exclude vendor/ from Credo analysis - Remaining: 175 warnings (mostly opinionated WeakAssertion, AvoidSocketAssignsInTest), 26 @spec annotations
53 lines
1.8 KiB
Elixir
53 lines
1.8 KiB
Elixir
defmodule Microwaveprop.Workers.PartitionMaintenanceWorker do
|
|
@moduledoc """
|
|
Daily Oban job that keeps the time-partitioned tables ahead of the
|
|
current write window.
|
|
|
|
Calls `Microwaveprop.PartitionManager.ensure_quarterly_partitions/1`
|
|
for the default parents (`hrrr_profiles`, `hrdps_profiles`) with a
|
|
4-quarter lookahead. Idempotent — the partition manager uses
|
|
`CREATE TABLE IF NOT EXISTS`, so a fire that finds nothing missing
|
|
is a no-op.
|
|
|
|
Lookahead choice: 4 quarters ≈ 1 year of runway. The cron runs daily,
|
|
so a single missed fire (or a week of missed fires after an outage)
|
|
still leaves >9 months of partition coverage before writes would hit
|
|
a missing quarter. The safety margin is intentionally large — the
|
|
cost of an extra empty partition is one zero-row table, while the
|
|
cost of a missing partition is silently failed inserts.
|
|
"""
|
|
use Oban.Pro.Worker,
|
|
queue: :admin,
|
|
max_attempts: 3,
|
|
unique: [period: 3600, states: :incomplete]
|
|
|
|
alias Microwaveprop.PartitionManager
|
|
|
|
require Logger
|
|
|
|
@lookahead_quarters 4
|
|
|
|
@impl Oban.Worker
|
|
@spec perform(Oban.Job.t()) :: :ok
|
|
def perform(%Oban.Job{args: args}) do
|
|
lookahead = lookahead(args)
|
|
results = PartitionManager.ensure_quarterly_partitions(lookahead)
|
|
|
|
created = Enum.filter(results, fn {_, _, status} -> status == :created end)
|
|
|
|
case created do
|
|
[] ->
|
|
Logger.info("PartitionMaintenanceWorker: all #{length(results)} partitions already exist")
|
|
|
|
list ->
|
|
names = Enum.map_join(list, ", ", fn {_parent, name, _} -> name end)
|
|
Logger.info("PartitionMaintenanceWorker: created partitions: #{names}")
|
|
end
|
|
|
|
:ok
|
|
end
|
|
|
|
@spec lookahead(map()) :: non_neg_integer()
|
|
defp lookahead(%{"lookahead_quarters" => n}) when is_integer(n) and n >= 0, do: n
|
|
defp lookahead(_), do: @lookahead_quarters
|
|
end
|