- accounts.ex: convert get_user_by_email_and_password if→with-chain
with explicit User struct match and else→nil. Flatten delete_account
nested if (password check + sole-owner check) into single with-chain
with named else clauses, eliminating do_delete_account indirection.
- organizations.ex: flatten update_member_role nested if/case into
with-chain — parse_role guard → membership fetch → owner check,
each with explicit else branch. Prevents variable shadowing by using
current_role for the membership pattern match.
- capacity.ex: extract calculate_throughput_from_stats inner if into
multi-clause compute_throughput_from_pairs/1 — empty list clause
returns zero_throughput(), non-empty clause handles the computation.
- Replace gray->cool-steel, blue/indigo->cerulean, red->sweet-salmon, yellow/amber->wheat
- Dark sidebar: sidebar/footer use cool-steel-800 bg, text lightened for contrast
- ~11,600 color replacements across ~99 files
- Update tests for new color class names
- Remove daisyUI plugins and theme from app.css
- Define 5 custom color palettes: sweet-salmon, desert-sand, wheat, cool-steel, cerulean
- Add @utility card, badge, btn classes to replace daisyUI components
- Replace all daisyUI classes across 16+ template files
- Update tests for new class return values
- Set light mode as default theme
- Fix doctests for Accounts, Agents.Stats, Snmp to match actual behavior
- Fix dynamic_extra_test vendor post-processing tests to seed sensor data
- Fix activity_controller_test to seed devices for feed data
- Fix session_manager_test to create browser session for test
- Fix topology_test link creation for connection test
- Fix device_monitor/driver_worker tests for unique job constraints
- Fix accounts_test expired_tokens assertion (magic link token is expired)
- Fix happy_path_test and show_events_test to seed monitor data
- Fix admin user_live_test user.name -> user.email (no name field)
- Fix schema_test to seed activity data
- Fix mobile_qr_live_test to match actual template text
- Fix SnmpKit.MIB doctests and tests for enriched return values
- Fix onboarding_live, mobile_controller, mib_test weak assertions
- Remove dead code and fix credo warnings
All checks set to exit_status: 0 initially to avoid blocking on
pre-existing violations. Remove exit_status overrides as each
check category is cleaned up incrementally.
Dependency added to dev/test only.
- Narrow @spec for NetworkInsightPrompt.build/1 and NetworkSnapshot.build/1
- Match/discard unmatched returns in monitoring.ex, alerts.ex, and
cloud_latency_probe_worker.ex
- Remove unreachable :rate_limited pattern in mikrotik_webhook_worker.ex
(resolve_error returns {:rate_limited, pos_integer()}, never bare atom)
- Fix gettext pluralisation in insights_live/index.ex: dngettext was
receiving a string where it expected a non_neg_integer count
Three changes targeting the root cause of production up/down alerting:
1. Skip BruteForceProtection for /health and /health/live paths
(removes unnecessary Repo.get_by(IpBlock) DB query from every
k8s probe, matching the existing /socket/agent exemption).
2. Relax k8s readiness probe: periodSeconds 5→10, timeoutSeconds
2→5, failureThreshold 2→3, successThreshold 3→2 so transient
DB/Redis blips don't cascade into full service unavailability.
3. Use the persistent Towerops.Redix connection for health checks
instead of opening a brand-new TCP connection (handshake→AUTH→
PING→close) on every probe. Towerops.Redix.Fake gains a
set_ping_response/2 toggle for testing failure paths.
Production code (lib/): all Process.sleep calls replaced with receive
after timeout - retry backoff, exponential backoff, batch delays,
poll intervals, and settle waits.
Test code: poll_until helpers across 4 agent channel test files now
use receive after instead of Process.sleep. Various standalone
Process.sleep(N) calls replaced with :sys.get_state sync barriers
or receive after. deferred_discovery test simulation sleeps reduced
500ms -> 30ms.
Backhaul radios (ePMP, AirOS, PMP, Cambium, MikroTik, RouterOS) now
monitor their connected client count with a low threshold of 1. When
the count drops to 0 the SnmpSensorExecutor marks it CRITICAL, which
triggers an alert through the existing check → alert pipeline.
- Adds field to sensor_def type in Vendor behaviour
- Sets on all clients-type wireless sensors in
backhaul-capable vendor profiles
- Adds default executor handling: any clients sensor at 0 → status 2
- 3 new tests for clients sensor behavior
Converts the 120KB PNG logo to 17KB WebP (86% smaller). Both logo instances in the marketing layout use <picture> with WebP source and PNG fallback for maximum browser compatibility.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Adds <meta name='robots'>, twitter:card/title/description meta tags, and WebP favicon reference to the root layout. Fixes title duplication (TowerOps | TowerOps -> Home | TowerOps) by passing page_title from the page controller.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
L2: Remove 2-second Process.sleep from post_startup — the try/catch
already handles transient noproc errors from the TaskSupervisor.
L3: Remove Process.sleep from delete_device — in-flight jobs already
handle the race condition via verify_polling_assignment_unchanged.
The blocking sleep was unnecessary and delayed web requests by 500ms.
Prod paired the Oban Pro Smart engine + unique workers with core
Oban.Plugins.Lifeline, whose rescue has no unique_violation handling. An
orphaned executing CheckExecutorWorker job rescued back to available collides
with its already-scheduled successor on oban_jobs_unique_index (23505),
crashing the plugin every 60s and leaving ~150 orphans stuck. Switch prod to
Oban.Pro.Plugins.DynamicLifeline, which repairs the conflict via
Smart.clear_uniq_violation. Dev keeps core Lifeline (Basic engine).
Add a shallow /health/live endpoint that does not touch db/redis. k8s liveness
and startup probes now target it so a transient dependency outage can't kill or
block boot of a healthy pod; readiness keeps the deep /health to gate the load
balancer. (deployment.yaml probe/replica change pushed separately, after the
image carrying /health/live is live.)
Also drop unused POSTGRES_* keys from the secrets example.
CheckExecutorWorker had no Oban `unique` constraint, so every insert path
created a fresh job row: self-scheduling, JobHealthCheckWorker, the check
form/API/GraphQL, and — the main amplifier — Monitoring.schedule_check/1
called unconditionally by snmp.ex discovery after create_check/1, which is
an upsert that returns {:ok, check} even when the check already exists.
Every SNMP discovery pass re-enqueued a job for every existing check, so
the check_executors queue grew to ~70k jobs for ~5.7k checks and stopped
draining.
Add `unique: [keys: [:check_id], states: [:available, :scheduled,
:retryable], period: :infinity]`. `:executing` is excluded so perform/1's
self-reschedule does not conflict with its own running job.
Also add the `checks` and `check_executors` queues to config/dev.exs —
they were missing from dev entirely (prod's runtime.exs already had them).
SyncErrors.transient?/1 only had explicit clauses for the {:http_error, ...}
shape, but vendor API clients return {:unexpected_status, status, body}. A 502
was retried only by accident via the catch-all, and a 4xx in that shape would
have been retried 20 times. Add explicit {:unexpected_status, ...} clauses: 5xx
is transient (Oban backs off and retries), 4xx is permanent.
The @session_options module attribute used Application.compile_env, which
baked compile-time placeholders into the endpoint while runtime.exs set
the real values from SESSION_SIGNING_SALT / SESSION_ENCRYPTION_SALT env
vars. Phoenix detected the mismatch and refused to start (failing migrate
Job in k8s).
- Remove hardcoded salts from config/config.exs (no compile-time binding)
- Add stable per-env salts in dev.exs / test.exs so local + CI don't need
the env vars
- Split static cookie opts (@static_session_options) from runtime-resolved
opts in endpoint.ex; expose session_options/0 as an MFA tuple in socket
connect_info so LiveView decodes sessions with the same runtime salts
- New ToweropsWeb.Plugs.RuntimeSession wraps Plug.Session, fetches salts
from app env on first request, and caches the initialized opts in
:persistent_term (zero per-request overhead after warm-up)
- M19: Add validate_mikrotik_device, validate_mikrotik_command, validate_check,
validate_snmp_query, validate_sensor, and validate_interface functions with
string length and list size validation, wired into all call sites
- L16: Document that Cloak Vault ETS table name collision risk is mitigated by
ETS tables being node-local (only a concern on same-BEAM-node, not a supported
production configuration)
- M20: Add check_no_hardlinks and check_no_special_files to MIB upload
archive extraction to prevent hard link, device node, and FIFO attacks
- M21: Change RateLimit ETS table from :public to :protected; route hit/get/reset
through GenServer.call instead of direct ETS access
- M22: Add 8-hour impersonation timeout — store impersonated_at in session and
auto-revoke impersonation when expired
- L14: Add default limit (500) and optional offset to list_checks for pagination
SESSION_SIGNING_SALT, SESSION_ENCRYPTION_SALT, and LIVE_VIEW_SIGNING_SALT
are now loaded from environment variables in production. Dev/test keep the
previous defaults via config.exs. k8s/secrets.yaml has placeholder entries;
the user fills in real values before applying.
Use Repo.get_by(schema, id: id, organization_id: organization_id) instead of
Repo.get + pattern match so that resources from wrong orgs return :not_found
instead of :forbidden, preventing org membership discovery.
- L3: Normalize IPv4-mapped IPv6 addresses (::ffff:x.x.x.x) to IPv4 tuples
when matching against IP whitelist entries
- L7: Verify interface belongs to current org before set/clear capacity
- L12: Guard window.liveSocket behind NODE_ENV check for production safety
- L13: Replace Alert.changeset with Ecto.Changeset.change for simple field
updates (resolved_at, acknowledged_at, gaiia_impact) to prevent
accidental overwrites from the 17-field cast
- M18: Add validate_snmp_device/1 with 255-byte limits on community,
v3_auth_password, and v3_priv_password to prevent memory exhaustion
- M13: Add client-side mime_type allowlist to phx:download handler
- M14: Replace Process.sleep loop in WeatherSyncWorker with Task.async_stream
to avoid blocking the Oban queue slot
- M15: Chunk device list in CloudLatencyProbeWorker PubSub broadcasts
(100 devices per message) to prevent megabyte-sized messages
- M3: Replace blacklist-based valid_return_path? with whitelist of known app
route prefixes, plus URI decoding to prevent %2f encoding bypasses
- M6: Replace full-org device/link load in get_node_detail with targeted
join query filtering by discovered node fields
- M7: Add partial unique index on (organization_id, email) for pending
invitations (WHERE accepted_at IS NULL)
- M12: Add on_mount superuser verification to all 7 admin LiveViews for
defense-in-depth
Sonar, Splynx, VISP, NetBox, and Gaiia sync workers all followed the same
"cron runs, then Enum.map syncs every integration in the same job" shape.
A single slow tenant — anything from an upstream rate limit to a stuck
GraphQL call — would hold the maintenance queue slot for ten minutes
while every other integration waited in line.
Each worker now uses the dispatcher/per-integration pattern that
UispSyncWorker already had:
- the cron `perform(%Oban.Job{args: %{}})` fans out one job per eligible
integration via `PollingOffset.calculate_offset/2`, staggered across a
5- to 10-minute window with `unique: [period: window_seconds]`
- `perform(%Oban.Job{args: %{"integration_id" => id}})` does the actual
sync, classifying failures via `Towerops.Workers.SyncErrors.transient?/1`
so permanent errors (auth, 4xx) don't get retried forever
Worker-specific defaults preserved:
- gaiia / netbox: 30/600s window, longer interval
- sonar / splynx / visp: 10/300s window
Tests updated to match the two-stage dispatcher pattern — `perform_job/2`
with empty args now asserts the per-integration jobs got enqueued and
the actual sync is invoked via a separate `perform_job/2` call with the
integration_id arg.
- M17: default SNMPv3 auth protocol on the four SNMP executors
(interface/storage/processor/sensor) is now SHA-256 instead of MD5.
Devices that explicitly set MD5 still use it; only the fallback for
unconfigured devices changes.
- M23: permissions check now logs a warning when an org's memberships
aren't preloaded so a silent denial is visible at the call site
instead of looking identical to a real authz failure.
- L9: extracted the duplicated should_sync?/1 logic from gaiia, sonar,
splynx, visp, and netbox sync workers into
Towerops.Integrations.due_for_sync?/2. Each worker now passes its
own default interval (10 or 30 minutes) to a single shared helper.
Tests updated to cover the helper directly.
- M2: get_user_by_email/1 uses lower(email) = lower(?) so User@Example.com
and user@example.com resolve to the same account; closes a lookalike-
registration / password-reset confusion risk.
- M8: webhook auth plug no longer echoes "Webhook authentication not
configured" — the misconfiguration is logged server-side and the caller
gets a generic Internal server error so endpoints can't be probed.
- M9: coverages controller logs the underlying KMZ build error and
returns a generic "Failed to build KMZ" string — filesystem paths and
zip internals no longer leak.
- M10: account-data controller no longer crashes with MatchError when an
org has zero or multiple memberships; defaults to "member" and treats
any owner membership as owner.
- M11: ActivityController switches to a hard whitelist
(Atom.to_string/1 comparison) instead of String.to_existing_atom/1;
unknown filter types are dropped silently and the atom table can't be
grown from API input.
- M16: title-tracking MutationObserver is stored on `this` so destroyed()
actually disconnects it — fixes a memory leak per LV navigation.
- H19: /api/v1/mobile/auth/qr/verify no longer returns user_email. Knowing
a QR token now only tells the caller the token is valid; the email is
only revealed by /complete which consumes the token.
- H20: CoverageWorker max_attempts dropped from 3 to 1. The fail/2 path
already returns :ok and writes the failure onto the coverage record,
so the 3-attempt retry policy was never reachable and was misleading.
- H25: ChecksController.create rejects device_ids that don't belong to
the caller's organization (via ScopedResource.fetch). Without this an
API token could attach service checks to devices in another tenant.
Also strip bug ID references from in-code comments (per feedback that
H/M/L numbers don't survive past their bugs.md removal); commit history
keeps the audit trail.
