add blog post

content/_index.md

@@ -2,4 +2,4 @@
 title = "W5ISP"
 +++
 
-I'm Graham McIntire (W5ISP), a site reliability and security engineer based in Blue Ridge, Texas. I like building things and figuring out how they work. Licensed amateur radio operator since 1996 and plain text enthusiast.
+I'm Graham McIntire (W5ISP), a site reliability and security engineer based in Blue Ridge, Texas. I like building things and figuring out how they work. Licensed amateur radio operator since 1996 and plain text enthusiast. [I'm looking for work, please hire me!](https://linkedin.com/in/grahammcintire)

content/blog/postgres1.md

@@ -1,9 +1,11 @@
 +++
 title = "When Postgres isn't the right choice"
-date = 2026-04-14
+date = 2026-04-18
 draft = true
 +++
 
+I've been working on For the [Microwave Propagation](https://prop.w5isp.com) site I've been working on
+
   ┌────────────────────────────────────────┬───────────┬───────────┐
   │                 Phase                  │ Wall time │ % of fh=0 │
   ├────────────────────────────────────────┼───────────┼───────────┤
@@ -21,3 +23,80 @@ draft = true
   ├────────────────────────────────────────┼───────────┼───────────┤
   │ Total fh=0                             │ 562s      │ 100%      │
   └────────────────────────────────────────┴───────────┴───────────┘
+
+  The propagation map at prop.w5isp.com ingests HRRR weather every hour and re-scores the entire CONUS grid across five
+   amateur-radio microwave bands. That is 95,000 grid points × 5 bands × 19 forecast hours = ~9 million score values per chain. For
+   a long time those lived in a Postgres table called propagation_scores. They don't anymore, and the map went from sluggish to
+   instantaneous. This is the story of why.
+ 
+   The shape mismatch
+ 
+   Each hourly run did this:
+ 
+   DELETE FROM propagation_scores WHERE valid_time = $1;
+   INSERT INTO propagation_scores (lat, lon, band_mhz, valid_time, score, factors) ...
+ 
+   475,000 rows per forecast hour, funneled through four indexes (PK UUID, uniqueness tuple, valid_time, (band_mhz, valid_time)),
+   plus a JSONB factors blob for the analysis hour. Scoring + upsert took about 4m 40s per forecast hour. Across 19 hours that's
+   more than an entire hourly cron cycle, so we had to back off to every-three-hours just to keep up.
+ 
+   The problem wasn't Postgres — Postgres was doing exactly what we asked. The problem was that the shape we were asking for was
+   wrong for how the data gets read.
+ 
+   The map is a canvas heatmap. It wants one dense (lat, lon) → score array for a single (band, valid_time). We were storing it as
+   half a million row tuples with MVCC bookkeeping, toast pages, and a B-tree index per column we'd never query by. Every write
+   rematerialized metadata the reader throws away.
+ 
+   Three stopgaps that weren't enough
+ 
+   Before rewriting the store, we tuned what was there:
+ 
+   1. DELETE + INSERT …  with no ON CONFLICT. The chain worker rewrites a full (valid_time, all bands) slice every hour, so conflict
+    detection was pure waste. Skipping it shaved a chunk off.
+   2. Skip factors on forecast hours. factors is a ~200-byte JSONB per row describing the ten scoring components (rain, humidity,
+   refractivity, etc.). We only show it on the analysis hour (f00) when the user clicks a cell. Passing factors: nil for f01–f18
+   skips a JSONB encode plus a toast write for roughly half the volume.
+   3. UNLOGGED table. Writes bypass the WAL entirely. On unclean shutdown the table truncates — fine, because PropagationGridWorker
+   rebuilds it from HRRR every hour. Durability was never the point; this was a cache with extra steps.
+ 
+   These helped, but the per-row overhead of a row-oriented store against a dense numeric grid is a ceiling you can't tune past. The
+    phase was still the wall-clock dominator.
+  
+
+  The binary file
+
+  Each (band_mhz, valid_time) grid now lands in one file at /data/scores/{band}/{iso}.ntms:
+
+  magic      : 4 bytes   "NTMS"
+  version    : 1 byte    0x01
+  band_mhz   : 4 bytes   uint32 LE
+  valid_time : 8 bytes   int64 unix seconds LE
+  lat_min    : 4 bytes   float32 LE
+  lon_min    : 4 bytes   float32 LE
+  step_deg   : 4 bytes   float32 LE
+  n_rows     : 2 bytes   uint16 LE
+  n_cols     : 2 bytes   uint16 LE
+  scores     : n_rows × n_cols bytes (uint8, 0–100; 255 = no-data)
+
+  33-byte header, dense uint8 array. A full CONUS grid serializes to ~93 KB per band per hour. A cell at (lat, lon) is at byte
+  offset row × n_cols + col, so a point lookup is arithmetic plus a single File.read/1. No index, no query planner, no lock
+  manager.
+
+  Writes go through the temp-then-rename pattern:
+
+  tmp = path <> ".tmp." <> unique_suffix()
+  File.write!(tmp, binary, [:binary])
+  File.rename!(tmp, path)
+
+ With Postgres off the hot path, the full f00–f18 chain dropped from ~170 min to ~45–60 min, so we ran the cron up to hourly. The
+  15-minute prune cron reaped expired files with rm — instant.
+
+  The f00 factor breakdown
+
+  One thing binary files don't handle well is the analysis-hour factor breakdown (the 10-component panel you see when you click a
+  cell). Those factors are heterogeneous Elixir maps, not a numeric grid.
+
+  The solution was a second file format, ProfilesFile: one compressed ETF file per valid_time at
+  /data/scores/profiles/{iso}.etf.gz, storing the enriched HRRR grid keyed by {lat, lon}. On a click, we reload the relevant cell's
+   profile and rescore on demand. Same atomic-rename write pattern, different payload — ETF where heterogeneity matters, raw binary
+   where density does.

content/blog/prop1.md

@@ -0,0 +1,71 @@
++++
+title = "How can microwave contacts be made non-LOS?"
+date = 2026-04-18
+draft = false
++++
+
+How can microwave contacts be made non-LOS is a question I've had for a while now.
+From running a WISP I'm very familiar with point-to-point microwave links. However,
+amateur radio operators have been making NLOS contacts for years when they technically
+shouldn't work at all.
+
+As an example, this is the evelation path between my QTH and an [NTMS beacon on top of Texas
+Women's University's dorm building](https://prop.w5isp.com/beacons/dddca785-c3e7-45f8-8f9f-edc748f881d6):
+
+![Path profile](/images/path_profile.png)
+
+This lead to a deep rabbit hole to investigate why contacts work some times and HOW they work.
+There are already known propagation methods such as EME (moonbounce) or rain / meteor scatter, 
+however it's also possible to form ducts within the lower levels of the atmosphere with favorable
+conditions. An easy to understand demonstration is why sometimes ships appear like they're floating:
+
+![Atmospheric ducted ship](/images/atmospheric_ducting.jpg)
+
+This is because there's a duct with favorable conditions for refracting visible light -- the same 
+thing happen for microwave.
+
+At one of the [NTMS](https://ntms.org) meetings, KM5PO presented about this same thing and had used
+claude to write out some client-side pages to help predict and analyze the propagation. It was based
+on limited data, but did an okay job guessing at why the propagation worked.
+
+Fast forward a few weeks and I've now build an elixir/phoenix app at <https://prop.w5isp.com> that
+collects and processes everything. For a quick import of some known contacts made, I imported all of the ARRL 10 GHz and up contests as well as the 222 MHz and up contests.
+
+With known contacts in place, it was time to backfill them with data, including:
+
+### Atmospheric
+* HRRR (NOAA 3 km) — hourly f00–f18 propagation scoring hot path; native hybrid-sigma levels for fine-grained duct detection
+* GEFS (NOAA 0.5° ensemble) — Day 2–7 extended-horizon outlook
+* HRRR native (hybrid σ) — best-duct-band per cell, refractivity gradient at 10–50 m resolution
+* IEMRE (Iowa State 0.125° reanalysis) — gridded hourly weather for per-QSO enrichment
+* NARR (NCEI 32 km reanalysis) — pre-2014 historical backfill for old QSOs (HRRR archive doesn't cover them)
+
+### Surface observations
+* ASOS (via IEM) — airport station surface weather, used by per-QSO enrichment and the 10-minute score nudge
+* MRMS (NOAA PrecipRate) — 2-minute radar-derived rain rate that overlays HRRR's hourly precip for fast-moving cells
+* NEXRAD (Level II composite reflectivity) — current-hour precip cell detection for f00 scoring
+
+### Upper air
+* RAOB (US radiosondes via IEM) — twice-daily soundings for duct/inversion analysis
+* UWYO (Canadian radiosondes) — 00/12 Z Canadian profiles
+
+### Terrain & geography
+* SRTM (90 m local tile cache + fallback API) — ITU-R P.526-16 path diffraction + viewshed
+
+It's taken a mountain of work and time to get all of this implemented, but most of the contacts 
+have now been backfilled with this information. Feeding all of this in to Claude, we end up with an 
+[algorithm](https://prop.w5isp.com/algo) that understands and can predict future conditions based 
+on weather data.
+
+This algorithm gets turned in to code, and then we can fetch HRRR data hourly, ammend with ASOS
+when avaliable and feed it through the algorithm to determine if any given area's atmosphere is
+favorable to propagation. Our predictions are still estimated and may not have any bearing on 
+reality.
+
+Now our mission is to continue collecting as many verified contacts on 50 MHz and up and enrich
+to refine our prediction model gradually over time. Another sub-project is automated beacon 
+monitoring. Any NLOS beacon that can be automatically monitored continuously provides an enormous
+amount of data to train our model with since both points are known.
+
+If you'd like to help out, we need as many contacts during non-contest times as possible and future
+automated beacon monitoring. You can reach me at my first name at my last name dot me (it's on the front page.)

content/projects/ntmsprop.md

@@ -0,0 +1,15 @@
++++
+title = "NTMS Microwave Propagation"
+description = "Microwave propagation prediction based on current and future weather"
+
+[extra]
+url = "https://prop.w5isp.com"
++++
+
+This site started as a curiosity I've had for a while: why do some amateur radio contacts work non-line of sight on microwave frequencies when they shouldn't.
+
+It's evolved in to a community effort along with the [North Texas Microwave Society](https://ntms.org) including from a senior meteorologist in Lubbock, Tx.
+
+Hourly we collect [HRRR](https://rapidrefresh.noaa.gov/hrrr/) data for the entire continental USA, process it, then store the estimated propagation in small binary files for display.
+
+Tech stack is elixir, phoenix, postgres, k8s.

sass/overrides.scss

@@ -0,0 +1,8 @@
+h1:before,
+h2:before,
+h3:before,
+h4:before,
+h5:before,
+h6:before {
+  content: none;
+}

templates/base.html

@@ -24,6 +24,7 @@
     {% else %}<link rel="canonical" href="{{ config.base_url }}" />
     {% endif %}
     <link rel="stylesheet" href="{{ get_url(path="style.css") }}">
+    <link rel="stylesheet" href="{{ get_url(path="overrides.css") }}">
     {% if config.extra.zolanight.google_analytics_id %}
     <!-- Delayed Google Analytics loading -->
     <script>
@@ -68,7 +69,17 @@
       {% block content %}{% endblock content %}
     </main>
     <hr>
-    <footer>
+    <footer style="margin-top: 3rem;">
+      <p>&copy; Copyright Graham McIntire {{ now() | date(format="%Y") }}</p>
     </footer>
+    <script src='https://storage.ko-fi.com/cdn/scripts/overlay-widget.js'></script>
+    <script>
+      kofiWidgetOverlay.draw('w5isp', {
+        'type': 'floating-chat',
+        'floating-chat.donateButton.text': 'Support me',
+        'floating-chat.donateButton.background-color': '#323842',
+        'floating-chat.donateButton.text-color': '#fff'
+      });
+    </script>
   </body>
 </html>