towerops/docs/nix.md

Nix Flakes Guide for Towerops Web

This document provides comprehensive guidance for using Nix flakes with the towerops-web project.

Table of Contents

• Overview
• Quick Start
• Installation
• Development Workflow
• Building
• CI/CD Integration
• Cachix Setup
• Troubleshooting
• Updating Dependencies

Overview

The towerops-web project uses Nix flakes for:

• Reproducible builds: Identical builds across development, CI, and production
• Development environment: One command (nix develop) provides fully configured environment
• Docker images: Pure Nix OCI images (~150-200 MB vs ~500 MB Debian-based)
• Binary caching: Cachix eliminates redundant compilation across machines
• Dependency management: All dependencies pinned in flake.lock

Architecture

flake.nix              # Main flake definition
├── nix/
│   ├── c-nif.nix      # C NIF shared library (cached separately)
│   ├── build.nix      # Mix release derivation
│   ├── docker.nix     # OCI image using dockerTools.buildLayeredImage
│   └── shell.nix      # Development shell with PostgreSQL, Redis, LSPs
├── flake.lock         # Locked dependency versions
├── .envrc.example     # direnv configuration example
└── shell.nix          # Compatibility shim for nix-shell

Quick Start

With Nix Installed

# Enter development environment (auto-starts PostgreSQL and Redis)
nix develop

# Or with direnv (automatic on cd)
cp .envrc.example .envrc
direnv allow

# Start Phoenix server
mix phx.server

First-Time Setup

1. Install Nix (if not already installed):

   # macOS or Linux
   curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
   

2. Clone the repository:

   git clone <repository-url>
   cd towerops-web
   

3. Enter development environment:

   nix develop
   

   On first run, this will:

   • Download and cache all dependencies
   • Initialize PostgreSQL in .nix-postgres/
   • Start Redis in .nix-redis/
   • Create towerops_dev and towerops_test databases
   • Run database migrations
   • Install pre-commit hooks

4. Start developing:

   mix phx.server
   # Navigate to http://localhost:4000
   

Installation

Nix Installation

macOS

# Using Determinate Systems installer (recommended)
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

# Or using official installer
sh <(curl -L https://nixos.org/nix/install)

Linux

# Using Determinate Systems installer (recommended)
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

# Or using official installer
sh <(curl -L https://nixos.org/nix/install) --daemon

Verify Installation

nix --version
# Should output: nix (Nix) 2.x.x

direnv Installation (Optional but Recommended)

direnv automatically loads the Nix environment when you cd into the project directory.

# macOS
brew install direnv

# Linux (Ubuntu/Debian)
sudo apt-get install direnv

# Add to your shell rc file (~/.bashrc, ~/.zshrc, etc.)
eval "$(direnv hook bash)"  # or zsh, fish, etc.

Then in the project directory:

cp .envrc.example .envrc
direnv allow

Development Workflow

Entering the Development Environment

With direnv (automatic):

cd /path/to/towerops-web
# Environment loads automatically

Without direnv (manual):

nix develop

Available Services

The development shell automatically starts:

• PostgreSQL 16 + TimescaleDB: Runs on localhost:5432

  • Databases: towerops_dev, towerops_test
  • Data directory: .nix-postgres/
  • TimescaleDB extension enabled for both dev and test databases
  • PostgreSQL built with withPackages to include TimescaleDB
  • shared_preload_libraries = 'timescaledb' set in config

• Redis: Runs on localhost:6379

  • Data directory: .nix-redis/
  • No persistence (dev mode)

Service Management

# Start services (if not auto-started)
start-services

# Stop services
stop-services

# Check if services are running
pg_isready -h localhost -p 5432
redis-cli ping

Environment Variables

Automatically set in the dev shell:

• DATABASE_URL: PostgreSQL connection string
• REDIS_URL: Redis connection string
• SECRET_KEY_BASE: Development secret (DO NOT use in production)
• CLOAK_KEY: Development encryption key (DO NOT use in production)
• MIX_ENV: Set to dev
• PHX_HOST: Set to localhost
• PORT: Set to 4000

Pre-commit Hooks

Automatically installed in the dev shell:

• mix format --check-formatted - Elixir code formatting
• mix credo --strict - Elixir linting
• nixfmt - Nix file formatting
• shellcheck - Shell script linting

Run manually:

pre-commit run --all-files

Building

Build the Elixir Release

nix build .#towerops
# Output in ./result/

# Run the release
./result/bin/towerops start

# Or use RPC
./result/bin/towerops rpc "1 + 1"

Build the Docker Image

nix build .#dockerImage
# Output in ./result

# Load into Docker
docker load < result

# Run the container
docker run --rm -p 4000:4000 \
  -e DATABASE_URL="..." \
  -e SECRET_KEY_BASE="..." \
  registry.gitlab.com/towerops/towerops:latest

Build the C NIF Separately

nix build .#towerops-nif
# Output in ./result/lib/towerops_nif.so

# Check dependencies
ldd result/lib/towerops_nif.so

Check Build Closure Size

# Show all runtime dependencies and sizes
nix path-info -rS .#dockerImage

# Show only the image size
nix path-info -S .#dockerImage

CI/CD Integration

GitLab CI Configuration

The project includes a Nix-based GitLab CI configuration in .gitlab-ci.yml.nix.

Requirements:

• GitLab Runner with Nix installed
• Docker socket mounted (for docker load)
• Cachix authentication token in CACHIX_AUTH_TOKEN variable

Setting Up NixOS GitLab Runner

1. Install GitLab Runner on NixOS:

   # /etc/nixos/configuration.nix
   services.gitlab-runner = {
     enable = true;
     services = {
       nix = {
         registrationConfigFile = "/etc/gitlab-runner/registration";
         dockerImage = "nixos/nix:latest";
         dockerVolumes = [
           "/nix/store:/nix/store:ro"
           "/nix/var/nix/db:/nix/var/nix/db:ro"
           "/nix/var/nix/daemon-socket:/nix/var/nix/daemon-socket:ro"
         ];
         tagList = [ "nix" ];
       };
     };
   };
   

2. Enable experimental features:

   nix.settings.experimental-features = [ "nix-command" "flakes" ];
   

Activating Nix CI/CD

Once the NixOS runner is configured:

# Backup current CI config
mv .gitlab-ci.yml .gitlab-ci.yml.docker

# Activate Nix CI config
mv .gitlab-ci.yml.nix .gitlab-ci.yml

# Commit and push
git add .gitlab-ci.yml
git commit -m "ci: migrate to Nix builds"
git push

Cachix Setup

Cachix provides binary caching to speed up builds across machines.

1. Create Cachix Account

Visit https://cachix.org and sign up.

2. Create Cache

# Install cachix
nix-env -iA cachix -f https://cachix.org/api/v1/install

# Create cache
cachix create towerops

# Generate keypair
cachix generate-keypair towerops

# Get public key
cachix get towerops
# Save the public key output

3. Update flake.nix

Replace the placeholder public key in flake.nix:

nixConfig = {
  extra-substituters = [
    "https://cache.nixos.org"
    "https://towerops.cachix.org"
  ];
  extra-trusted-public-keys = [
    "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
    "towerops.cachix.org-1:YOUR_ACTUAL_PUBLIC_KEY_HERE"
  ];
};

4. Configure Local Machine

cachix use towerops

This updates ~/.config/nix/nix.conf to use the cache.

5. Push Builds to Cache

# Push a specific build
nix build .#dockerImage
nix path-info --json .#dockerImage | jq -r '.[].path' | cachix push towerops

# Push all outputs
nix flake show --json | jq -r '.packages."x86_64-linux" | keys[]' | \
  xargs -I {} nix build .#{} --print-build-logs
nix flake show --json | jq -r '.packages."x86_64-linux" | keys[]' | \
  xargs -I {} nix path-info --json .#{} | jq -r '.[].path' | cachix push towerops

6. Configure CI/CD

Add CACHIX_AUTH_TOKEN to GitLab CI/CD variables:

1. Go to GitLab project → Settings → CI/CD → Variables
2. Add variable:
   • Key: CACHIX_AUTH_TOKEN
   • Value: (get from cachix authtoken)
   • Masked: Yes
   • Protected: Yes (optional)

Troubleshooting

Services Won't Start

PostgreSQL fails to initialize:

# Remove corrupted data directory
rm -rf .nix-postgres
# Re-enter shell to reinitialize
exit
nix develop

Port already in use:

# Check what's using the port
lsof -i :5432  # PostgreSQL
lsof -i :6379  # Redis

# Kill the process or change port in nix/shell.nix

Nix Build Failures

Error: experimental feature 'flakes' is not enabled:

# Add to ~/.config/nix/nix.conf
experimental-features = nix-command flakes

# Or use --extra-experimental-features
nix --extra-experimental-features "nix-command flakes" develop

Error: out of disk space:

# Clean old generations
nix-collect-garbage -d

# Clean build artifacts
nix-store --gc

C NIF compilation fails:

# Check net-snmp is available
nix-shell -p net-snmp --run "pkg-config --libs netsnmp"

# Rebuild NIF separately
nix build .#towerops-nif --rebuild

TimescaleDB Not Loading

If TimescaleDB was added after initial setup, you need to reinitialize PostgreSQL:

# Stop services and remove old data directory
stop-services
rm -rf .nix-postgres .nix-services-started

# Re-enter shell to reinitialize with TimescaleDB
exit
nix develop

This is required because shared_preload_libraries is set during initdb and the TimescaleDB shared library must be available in the PostgreSQL package.

Mix Dependencies

Error: Mix dependencies not found:

# Ensure flake.lock is up to date
nix flake update

# Rebuild with fresh dependencies
nix build .#towerops --rebuild

Vendored Oban packages: The vendor/ directory is included in the Nix build. If you update vendored packages:

# Rebuild to pick up changes
nix build .#towerops --rebuild

Docker Image Issues

Image too large:

# Check closure size
nix path-info -rS .#dockerImage

# Identify large dependencies
nix path-info -rS .#dockerImage | sort -k2 -h | tail -20

Missing files in image:

# List image contents
docker run --rm registry.gitlab.com/towerops/towerops:latest find / -name "*.mib" | head

direnv Issues

direnv: error .envrc is blocked:

direnv allow

Environment not loading:

# Check direnv status
direnv status

# Reload manually
direnv reload

Updating Dependencies

Update Nix Flake Inputs

# Update all inputs
nix flake update

# Update specific input
nix flake update nixpkgs

# Update and rebuild
nix flake update && nix build .#towerops

Update Elixir/Erlang Version

Edit nix/shell.nix and nix/build.nix to use different versions:

# Before
elixir
erlang_28

# After (if nixpkgs has newer versions)
elixir_1_20
erlang_29

Then rebuild:

nix flake update nixpkgs
nix build .#towerops --rebuild

Pin Specific Nixpkgs Version

Edit flake.nix to pin to a specific commit:

inputs = {
  # Pin to specific nixpkgs commit
  nixpkgs.url = "github:NixOS/nixpkgs/abc123def456";
  # Or pin to a release branch
  nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
};

Update Mix Dependencies

Mix dependencies are managed in mix.exs and mix.lock as usual. Nix will use these files:

# Update Mix dependencies
mix deps.update --all

# Commit updated mix.lock
git add mix.lock
git commit -m "deps: update Elixir dependencies"

# Rebuild with new dependencies
nix build .#towerops --rebuild

Advanced Topics

Cross-Platform Builds

Build for multiple architectures:

# Build for Linux ARM64 (Apple Silicon servers)
nix build .#packages.aarch64-linux.dockerImage

# Build for macOS on Linux (if configured)
nix build .#packages.x86_64-darwin.towerops

Custom Nix Overlays

Create nix/overlays.nix to override nixpkgs packages:

final: prev: {
  # Example: Use specific Elixir version
  elixir = prev.elixir.override {
    version = "1.20.0";
  };
}

Reference in flake.nix:

_module.args.pkgs = import nixpkgs {
  inherit system;
  overlays = [ (import ./nix/overlays.nix) ];
};

Development Without Nix

If Nix is not available, use the traditional development setup:

# Install Elixir via asdf
asdf install erlang 28.3
asdf install elixir 1.19.5-otp-28

# Install PostgreSQL and Redis
brew install postgresql@16 redis  # macOS
# Or: sudo apt-get install postgresql-16 redis  # Linux

# Start services manually
brew services start postgresql@16
brew services start redis

# Run development server
mix deps.get
mix ecto.setup
mix phx.server

Resources

• Nix Flakes Documentation
• NixOS Packages Search
• Cachix Documentation
• direnv Documentation
• Elixir on Nix

Getting Help

• Nix Discord: https://discord.gg/RbvHtGa
• NixOS Discourse: https://discourse.nixos.org/
• Project Issues: File issues on GitLab for project-specific problems