Move NFS share from skippy to node3 at 10.0.15.103:/data

node3's NVMe SSD is the new shared-storage home. Mount the whole
/data export from 10.0.15.103 at /data inside the pod (instead of
the old skippy export of /data/srtm at /srtm) so there's room for
future shared stuff — Parquet score files, maybe a DuckDB database,
or whatever the next thing is — without re-exporting a new path.

SRTM tiles now live at /data/srtm rather than /srtm. Update the
srtm_tiles_dir runtime config to match, and fix the two doc
pointers (CLAUDE.md, duckdb plan) that referenced the old path.

Already applied to the cluster and rolled out; pods confirmed
running with tiles visible at /data/srtm.
This commit is contained in:
Graham McIntire 2026-04-14 14:25:42 -05:00
parent 79b849e617
commit 33a8ba0956
No known key found for this signature in database
GPG key ID: F4ABF488E6029E59
4 changed files with 12 additions and 12 deletions

View file

@ -213,4 +213,4 @@ User submits QSO → enqueue_for_qso() → weather/hrrr/terrain/iemre workers
- Deployment name `prop` (3 replicas). Inspect with `kubectl -n prop get pods`, `kubectl -n prop logs deploy/prop`, `kubectl -n prop exec deploy/prop -- ...`
- Push to `github` remote for code. The `dokku` remote is obsolete — do not push there.
- Production Oban config in `config/runtime.exs` — no backfill cron, enrichment triggered by QSO submission
- SRTM tiles at `/srtm` in production container
- Shared NFS mount `/data` in production container (server: node3 at `10.0.15.103:/data`). SRTM tiles live at `/data/srtm`.

View file

@ -233,7 +233,7 @@ if config_env() == :prod do
config :microwaveprop, :email_from, {"NTMS Propagation", "graham@w5isp.com"}
config :microwaveprop, hrrr_base_url: System.get_env("HRRR_BASE_URL", "http://skippy.w5isp.com:8080")
config :microwaveprop, srtm_tiles_dir: "/srtm"
config :microwaveprop, srtm_tiles_dir: "/data/srtm"
# config :microwaveprop, start_freshness_monitor: true
config :microwaveprop, start_freshness_monitor: false
end

View file

@ -24,8 +24,8 @@ The map render path also doesn't want individual rows. It wants a dense `(lat, l
One Parquet file per `(band_mhz, valid_time)`:
```
/srtm/scores/10000/2026-04-14T18-00-00Z.parquet
/srtm/scores/24000/2026-04-14T18-00-00Z.parquet
/data/scores/10000/2026-04-14T18-00-00Z.parquet
/data/scores/24000/2026-04-14T18-00-00Z.parquet
...
```
@ -71,7 +71,7 @@ Cons:
### Option C — DuckDB on persistent disk
Single `.duckdb` file under `/srtm/scores.duckdb`:
Single `.duckdb` file under `/data/scores.duckdb`:
- One table with the same schema as `propagation_scores`
- DuckDB is columnar, compressed, and serializes writes via a single database file
@ -107,7 +107,7 @@ The `factors` storage question is the only thing that forces picking between Opt
Before this is worth doing:
1. **Separate worker pod**. Multi-writer coordination on a single file is painful; a single "propagation" pod that owns all writes is much simpler. The isolation idea from earlier in this session dovetails here.
2. **Shared volume or single-node**. Files need to live on a PVC shared between the worker and web pods, or the web pods need a cached-read path that pulls the file from the worker pod via HTTP. The existing `/srtm` NFS mount is already shared, which makes `/srtm/scores` free.
2. **Shared volume or single-node**. Files need to live on a PVC shared between the worker and web pods, or the web pods need a cached-read path that pulls the file from the worker pod via HTTP. The existing `/data` NFS mount is already shared across all pods, which makes `/data/scores` free.
3. **Baseline perf measurements** from the already-shipped Postgres changes. Don't do this if the map already feels fast.
## Read-path changes
@ -176,7 +176,7 @@ Expected timings: ~10-50 ms per band per fh vs 4-5 seconds per band per fh today
Current: 15-minute cron running `DELETE WHERE valid_time < now - 2h`.
New: 15-minute cron running `find /srtm/scores -mmin +120 -delete` (or Elixir equivalent).
New: 15-minute cron running `find /data/scores -mmin +120 -delete` (or Elixir equivalent).
Zero vacuum pressure, zero WAL, zero index maintenance.

View file

@ -103,10 +103,10 @@ spec:
seccompProfile:
type: RuntimeDefault
volumeMounts:
- name: srtm-data
mountPath: /srtm
- name: data
mountPath: /data
volumes:
- name: srtm-data
- name: data
nfs:
server: 204.110.191.8
path: /data/srtm
server: 10.0.15.103
path: /data