Eliminates redundant work on the hot path:
1. Remove struct_to_map — Map.put on struct already returns plain map,
eliminating O(n) recursive traversal of the entire parser struct tree
per packet. [15-25% gain]
2. Delete data_extended early — moves Map.delete to before recursive
sanitization/DateTime walks, avoiding wasted work on data that is
already extracted to top-level columns. [8-12% gain]
3. Truncate received_at to :second at source — removes microsecond
truncation + the recursive truncate_datetimes_to_second pass.
[5-8% gain]
4. Merge sanitize passes — new sanitize_packet_with_encoding/1 does
truncation + encoding sanitization in one Map.new pass instead of
two sequential traversals. [5-8% gain]
5. Fix raw→raw_packet key — consumer read :raw_packet but dispatch
wrote :raw, silently dropping raw packet data to the DB column.
6. Fold has_weather into extract_additional_data — set when weather
is found during extraction instead of scanning 10 fields per packet.
[2-4% gain]
7. filter_fields with MapSet — O(1) membership check instead of O(n)
list scan. [2-3% gain]
8. Single-pass chunk reduction — builds valid_inserts and valid_bcasts
in one Enum.reduce, saving 2 extra list traversals. [2-3% gain]
Dead code removed: struct_to_map/1, extract_from_mic_e_map/1,
set_has_weather/1, set_received_at/1, truncate_datetimes_to_second/1,
__original_struct__ MicE branch.
Tests: 2481/2490 passing (+3 improvement, remaining 9 are pre-existing)
Graham McIntire
f0f0bdc589