Built-in PagerDuty-equivalent system for on-call scheduling and alert
escalation. Users can now manage schedules, rotation layers, overrides,
and escalation policies directly in the app alongside PagerDuty.
- On-call schedules with rotation layers (daily/weekly/custom), member
management, and temporary overrides
- Escalation policies with ordered rules, timeout-based escalation,
and user/schedule targets
- Automatic escalation via Oban worker with configurable repeat count
- Email notifications via Swoosh for on-call alerts
- Resolver computes who's on-call from layer stacking and overrides
- AlertNotificationWorker integration: starts escalation alongside
PagerDuty, acknowledges/resolves incidents on alert state changes
- Device and organization schemas support escalation_policy_id
- Escalation policy picker on device form
- Schedules nav item with tabbed index (schedules + escalation policies)
- Full CRUD UI for schedules, layers, members, overrides, rules, targets
- 62 LiveView tests, 56 context/schema/resolver/escalation tests
- 26 E2E Playwright tests for smoke and critical path coverage
When a billing provider is active, all other billing providers are
hidden from the main list and collected into a compact <details>
disclosure element showing provider names and descriptions.
When a billing provider is active (enabled), the other billing providers
are dimmed and collapsed to a single row showing a lock icon and which
provider is active. This reinforces the mutually exclusive constraint.
Also fixes status emoji accumulating in page title (JS regex needed
unicode flag) and adds missing page_title to 8 LiveViews.
Fix JS regex in StatusTitle hook to use unicode flag so emoji are
properly matched and replaced instead of accumulating. Add page_title
to 8 LiveViews that were missing it, which caused "TowerOps | TowerOps"
duplication from the live_title suffix.
Add setup_help info boxes to each integration provider showing where
to create API keys, step-by-step instructions, required permissions,
and example URL patterns.
Proactively add catch-all handlers to device index/show pages to
prevent FunctionClauseError when StatusTitleComponent broadcasts
alert changes. These are the most frequently open pages where users
would encounter the crash.
The StatusTitleComponent (rendered in authenticated layout) subscribes
to organization:*:alerts PubSub topic. Since LiveComponents run in the
parent process, both the component and parent LiveView receive these
messages.
Added catch-all handler to prevent FunctionClauseError when alert
changes broadcast. The component already handles the message properly.
Removed:
- Compact view toggle button from the header
- toggle_compact_mode event handler
- @compact_mode assign
The button was not connected to any actual functionality.
The discovered tab badge was showing length(@discovered_devices) which is
the paginated list (20 items per page), not the total count. Now shows
@pagination.total_count to display the actual total (e.g., 876 devices).
- Fixed debounce_ms/0 function still using Mix.env()
- Changed to Application.get_env(:towerops, :env)
- This was causing crashes in staging/production
Add CapacityInsightWorker (every 15 min) that generates critical/warning
insights when backhaul utilization exceeds 90%/75% and auto-resolves
when it drops below 70%.
Add capacity and utilization columns to the device ports tab with
set/clear capacity controls. Add organization-level /capacity page
with summary cards and per-site capacity table. Add capacity summary
card to site show page. Add Capacity link to nav menu.
- Add handle_info for {:alert_changed, org_id} in AlertLive.Index and DashboardLive
to properly handle organization-level alert change broadcasts
- Fix get_primary_owner query to prioritize is_default memberships first,
ensuring the organization creator is always returned when multiple owners exist
Resolves 5 test failures related to PubSub message handling and
deterministic primary owner selection.
- Show sender's full name in invitation emails
- Use relative time for "Sent" column (more readable)
- Use date format for "Expires" column (shorter)
- Make invitation table horizontally scrollable
Fixes display cropping issue where long timestamps caused
the pending invitations table to overflow its container.
Convert compile-time module attributes to runtime functions to avoid
dialyzer warnings about compile-time constant comparisons:
- tick_interval_ms: function instead of @tick_interval_ms
- debounce_ms: function instead of @debounce_ms
This fixes warnings where dialyzer knew the comparisons could never
be false/true based on compile-time values.
Dynamically update page title emoji when device/alert status changes:
- Add PubSub broadcasts when alerts are created/resolved
- Create StatusTitleComponent LiveComponent to subscribe to alert changes
- Add StatusTitle JavaScript hook to update document.title in real-time
- Subscribe to organization:alerts channel on LiveView mount
- Push emoji update events when alert status changes
This provides immediate visual feedback in browser tabs when
critical/warning/healthy status changes without requiring page refresh.
Extract validation logic into separate functions to reduce:
- Cyclomatic complexity from 12 to acceptable levels
- Nesting depth from 3 to 2
Improves code maintainability and passes Credo checks.
Add colored circle emoji (🔴/🟡/🟢) to page titles based on organization
health status:
- 🔴 Red: Critical alerts (device_down, agent_offline, severity 1)
- 🟡 Yellow: Warning alerts (severity 2 or any active alerts)
- 🟢 Green: No active alerts
Provides at-a-glance status visibility in browser tabs.
The template accessed if_in_octets/if_out_octets directly on the
Interface struct, but these fields live on InterfaceStat. Access
through interface.latest_stat with nil guard to prevent KeyError
when the association is not loaded.
Allow superadmins to override free device limits and per-device pricing
on a per-organization basis. Overrides cascade through subscription
limits, billing calculations, and user-facing settings display.
- Add custom_free_device_limit and custom_price_per_device to organizations
- Add billing_override_changeset with validation
- Update SubscriptionLimits.effective_device_limit to respect overrides
- Update Billing to use effective free count and price per device
- Add Admin.update_billing_overrides with audit logging
- Add override editing UI to /admin/organizations
- Update org settings page to show effective limits/pricing
Add WATCHTOWER_SCOPE=towerops environment variable to generated
docker-compose templates. This prevents Watchtower from monitoring
unrelated containers on the same Docker host (e.g., Dokku apps).
Without scope filtering, Watchtower would attempt to monitor all
containers with watchtower.enable=true label, causing warnings
for missing images and unnecessary monitoring overhead.
Implements Phase 6 of Stripe billing integration:
- Add Billing tab to organization settings page
- Display subscription status, device usage, and estimated costs
- "Upgrade to Paid Plan" button for free tier orgs
- "Manage Billing" button to access Stripe Customer Portal
- Billing information panel showing subscription details
- Support for dark mode and responsive design
Also fixes organization name truncation in top navigation:
- Long organization names now show with ellipsis instead of being cut off
- Added max-w-xs constraint and truncate class to org switcher button
Favicon changes to green/yellow/red based on device and alert status.
LiveView computes the status server-side and passes it to a minimal
JS hook that generates PNG favicons via canvas.
Fixes fragmented translation that displayed raw '%{link}' placeholder text
instead of properly interpolated link in development mail adapter notice.
Changes:
- Consolidate separate translation calls into single interpolated string
- Use raw() helper to safely inject HTML link into translated text
- Update Spanish translation to include %{link} placeholder
- Extract updated translations with mix gettext.extract
Before: "To see sent emails, visit %{link}. the mailbox page."
After: "To see sent emails, visit the mailbox page." (with proper link)
The select_org LiveView event was updating default_organization_id in
the database but not the session's current_organization_id. Since
load_default_organization prioritizes the session value, users would
always get redirected back to the stale org.
Replaced with a controller POST action that updates both the session
and DB default before redirecting.
Router devices were inflating per-site subscriber counts because:
1. CGNAT IPs (100.64.0.0/10) in router ARP tables caused false matches
2. Routers see all traffic and shouldn't have subscribers attributed
3. Site totals were computed from raw links without filtering
Changes:
- Filter CGNAT (RFC 6598) IPs from device subscriber links
- Deduplicate accounts across devices by match method priority
- Detect routers via SNMP sys_descr and zero out per-device counts
- Derive site totals from filtered per-device impacts
- Add per-device Subs/MRR columns to site device tables
- Add device link fallback for sites without IP blocks
- Fix Gaiia sync case mismatch and execution order
- Improve device list toolbar button icons and tooltips
When the Oban queue backs up, the 60-second uniqueness window expires
and duplicate jobs stack up per device. Switch to period: :infinity so
only one poll/monitor job exists per device at any time. Add replace
option to supersede stale scheduled jobs with updated scheduled_at.
Remove :executing from unique states so self-scheduling works while
the current job runs. Set max_attempts: 1 since retrying stale polls
is pointless.
Also fix all credo --strict issues across the codebase.
The form template iterated over changeset error tuples directly
(e.g. changeset.errors[:name] returns a tuple, not a list), causing
Protocol.UndefinedError for Enumerable on Tuple.
Rewrote form to use to_form/2 and <.input> components per LiveView
conventions. Errors are now handled properly by the input component.
