graham/aprs.me
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
aprs.me/test/time_independence_fixes.md
Graham McIntire 4e00c3cf5d
test: Make tests time-independent and increase timeouts for reliability 
- Fix time-dependent tests to use relative dates instead of hardcoded values
- Increase timeout values to prevent flaky failures on slower systems/CI
- Use flexible string matching for time-related assertions
- Update Process.sleep and assert_receive timeouts for better reliability

Specific changes:
- TimeHelpers: Use pattern matching for variable time strings
- PacketsOldest: Replace hardcoded dates with relative calculations
- BroadcastTaskSupervisor: Increase timing assertion to 1s
- Movement tests: Increase refute_push_event timeout to 2s
- Integration tests: Increase all wait times by 50%
- StreamingPacketsPubSub: Increase receive timeouts

These changes ensure tests pass regardless of when they're executed
and are more resilient to performance variations in CI environments.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-02 10:07:47 -05:00

2.2 KiB
Raw Blame History

Time Independence Test Fixes

This document summarizes the changes made to ensure tests pass regardless of when they are executed.

Fixed Issues

1. TimeHelpers Tests (test/aprsme_web/time_helpers_test.exs)

  • Changed exact string matches to flexible pattern matching
  • Example: "1 minute ago" → ["1 minute ago", "about 1 minute ago"]
  • Added regex patterns for variable outputs

2. Packets Oldest Tests (test/aprsme/packets_oldest_test.exs)

  • Replaced hardcoded dates like ~U[2023-01-01 00:00:00Z]
  • Now uses relative dates: DateTime.add(now, -365 * 24 * 60 * 60, :second)
  • All date calculations are now relative to current time

3. Broadcast Task Supervisor Tests (test/aprsme/broadcast_task_supervisor_test.exs)

  • Increased timing assertion from 100ms to 1000ms
  • Makes tests less sensitive to system load and CI environment performance

4. Movement Tests (test/aprsme_web/live/map_live/movement_test.exs)

  • Increased refute_push_event timeout from 500ms to 2000ms
  • Prevents false failures on slower systems

5. Packet Consumer Tests (test/aprsme/packet_consumer_test.exs)

  • Increased Process.sleep from 20ms to 100ms for async operations
  • Increased assert_receive timeout from 1000ms to 5000ms

6. Integration Tests (test/integration/historical_loading_integration_test.exs)

  • Increased all Process.sleep calls: 500ms → 1000ms, 2000ms → 3000ms
  • Gives Wallaby more time for browser operations

7. Streaming Packets PubSub Tests (test/aprsme/streaming_packets_pubsub_test.exs)

  • Increased assert_receive timeout from 100ms to 1000ms
  • Increased refute_receive timeout from 100ms to 500ms

General Patterns Applied

  1. Relative Time Calculations: All date/time calculations now use offsets from DateTime.utc_now()
  2. Flexible Assertions: String comparisons allow for slight variations in output
  3. Generous Timeouts: Increased timeouts to handle slower CI environments
  4. No Hardcoded Dates: Removed all specific date literals

Testing Strategy

These changes ensure that:

  • Tests pass regardless of the current date/time
  • Tests are resilient to performance variations
  • Tests work reliably in CI/CD environments
  • Tests don't fail due to timezone differences