towerops/docs/nix.md

# Nix Flakes Guide for Towerops Web

This document provides comprehensive guidance for using Nix flakes with the towerops-web project.

## Table of Contents

- [Overview](#overview)
- [Quick Start](#quick-start)
- [Installation](#installation)
- [Development Workflow](#development-workflow)
- [Building](#building)
- [CI/CD Integration](#cicd-integration)
- [Cachix Setup](#cachix-setup)
- [Troubleshooting](#troubleshooting)
- [Updating Dependencies](#updating-dependencies)

## Overview

The towerops-web project uses Nix flakes for:

- **Reproducible builds**: Identical builds across development, CI, and production
- **Development environment**: One command (`nix develop`) provides fully configured environment
- **Docker images**: Pure Nix OCI images (~150-200 MB vs ~500 MB Debian-based)
- **Binary caching**: Cachix eliminates redundant compilation across machines
- **Dependency management**: All dependencies pinned in `flake.lock`

### Architecture

```
flake.nix              # Main flake definition
├── nix/
│   ├── c-nif.nix      # C NIF shared library (cached separately)
│   ├── build.nix      # Mix release derivation
│   ├── docker.nix     # OCI image using dockerTools.buildLayeredImage
│   └── shell.nix      # Development shell with PostgreSQL, Redis, LSPs
├── flake.lock         # Locked dependency versions
├── .envrc.example     # direnv configuration example
└── shell.nix          # Compatibility shim for nix-shell
```

## Quick Start

### With Nix Installed

```bash
# Enter development environment (auto-starts PostgreSQL and Redis)
nix develop

# Or with direnv (automatic on cd)
cp .envrc.example .envrc
direnv allow

# Start Phoenix server
mix phx.server
```

### First-Time Setup

1. **Install Nix** (if not already installed):
   ```bash
   # macOS or Linux
   curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
   ```

2. **Clone the repository**:
   ```bash
   git clone <repository-url>
   cd towerops-web
   ```

3. **Enter development environment**:
   ```bash
   nix develop
   ```

   On first run, this will:
   - Download and cache all dependencies
   - Initialize PostgreSQL in `.nix-postgres/`
   - Start Redis in `.nix-redis/`
   - Create `towerops_dev` and `towerops_test` databases
   - Run database migrations
   - Install pre-commit hooks

4. **Start developing**:
   ```bash
   mix phx.server
   # Navigate to http://localhost:4000
   ```

## Installation

### Nix Installation

#### macOS

```bash
# Using Determinate Systems installer (recommended)
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

# Or using official installer
sh <(curl -L https://nixos.org/nix/install)
```

#### Linux

```bash
# Using Determinate Systems installer (recommended)
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

# Or using official installer
sh <(curl -L https://nixos.org/nix/install) --daemon
```

#### Verify Installation

```bash
nix --version
# Should output: nix (Nix) 2.x.x
```

### direnv Installation (Optional but Recommended)

direnv automatically loads the Nix environment when you `cd` into the project directory.

```bash
# macOS
brew install direnv

# Linux (Ubuntu/Debian)
sudo apt-get install direnv

# Add to your shell rc file (~/.bashrc, ~/.zshrc, etc.)
eval "$(direnv hook bash)"  # or zsh, fish, etc.
```

Then in the project directory:

```bash
cp .envrc.example .envrc
direnv allow
```

## Development Workflow

### Entering the Development Environment

**With direnv** (automatic):
```bash
cd /path/to/towerops-web
# Environment loads automatically
```

**Without direnv** (manual):
```bash
nix develop
```

### Available Services

The development shell automatically starts:

- **PostgreSQL 16 + TimescaleDB**: Runs on `localhost:5432`
  - Databases: `towerops_dev`, `towerops_test`
  - Data directory: `.nix-postgres/`
  - TimescaleDB extension enabled for both dev and test databases
  - PostgreSQL built with `withPackages` to include TimescaleDB
  - `shared_preload_libraries = 'timescaledb'` set in config

- **Redis**: Runs on `localhost:6379`
  - Data directory: `.nix-redis/`
  - No persistence (dev mode)

### Service Management

```bash
# Start services (if not auto-started)
start-services

# Stop services
stop-services

# Check if services are running
pg_isready -h localhost -p 5432
redis-cli ping
```

### Environment Variables

Automatically set in the dev shell:

- `DATABASE_URL`: PostgreSQL connection string
- `REDIS_URL`: Redis connection string
- `SECRET_KEY_BASE`: Development secret (DO NOT use in production)
- `CLOAK_KEY`: Development encryption key (DO NOT use in production)
- `MIX_ENV`: Set to `dev`
- `PHX_HOST`: Set to `localhost`
- `PORT`: Set to `4000`

### Pre-commit Hooks

Automatically installed in the dev shell:

- `mix format --check-formatted` - Elixir code formatting
- `mix credo --strict` - Elixir linting
- `nixfmt` - Nix file formatting
- `shellcheck` - Shell script linting

Run manually:
```bash
pre-commit run --all-files
```

## Building

### Build the Elixir Release

```bash
nix build .#towerops
# Output in ./result/

# Run the release
./result/bin/towerops start

# Or use RPC
./result/bin/towerops rpc "1 + 1"
```

### Build the Docker Image

```bash
nix build .#dockerImage
# Output in ./result

# Load into Docker
docker load < result

# Run the container
docker run --rm -p 4000:4000 \
  -e DATABASE_URL="..." \
  -e SECRET_KEY_BASE="..." \
  registry.gitlab.com/towerops/towerops:latest
```

### Build the C NIF Separately

```bash
nix build .#towerops-nif
# Output in ./result/lib/towerops_nif.so

# Check dependencies
ldd result/lib/towerops_nif.so
```

### Check Build Closure Size

```bash
# Show all runtime dependencies and sizes
nix path-info -rS .#dockerImage

# Show only the image size
nix path-info -S .#dockerImage
```

## CI/CD Integration

### GitLab CI Configuration

The project includes a Nix-based GitLab CI configuration in `.gitlab-ci.yml.nix`.

**Requirements**:
- GitLab Runner with Nix installed
- Docker socket mounted (for `docker load`)
- Cachix authentication token in `CACHIX_AUTH_TOKEN` variable

### Setting Up NixOS GitLab Runner

1. **Install GitLab Runner on NixOS**:
   ```nix
   # /etc/nixos/configuration.nix
   services.gitlab-runner = {
     enable = true;
     services = {
       nix = {
         registrationConfigFile = "/etc/gitlab-runner/registration";
         dockerImage = "nixos/nix:latest";
         dockerVolumes = [
           "/nix/store:/nix/store:ro"
           "/nix/var/nix/db:/nix/var/nix/db:ro"
           "/nix/var/nix/daemon-socket:/nix/var/nix/daemon-socket:ro"
         ];
         tagList = [ "nix" ];
       };
     };
   };
   ```

2. **Enable experimental features**:
   ```nix
   nix.settings.experimental-features = [ "nix-command" "flakes" ];
   ```

### Activating Nix CI/CD

Once the NixOS runner is configured:

```bash
# Backup current CI config
mv .gitlab-ci.yml .gitlab-ci.yml.docker

# Activate Nix CI config
mv .gitlab-ci.yml.nix .gitlab-ci.yml

# Commit and push
git add .gitlab-ci.yml
git commit -m "ci: migrate to Nix builds"
git push
```

## Cachix Setup

Cachix provides binary caching to speed up builds across machines.

### 1. Create Cachix Account

Visit https://cachix.org and sign up.

### 2. Create Cache

```bash
# Install cachix
nix-env -iA cachix -f https://cachix.org/api/v1/install

# Create cache
cachix create towerops

# Generate keypair
cachix generate-keypair towerops

# Get public key
cachix get towerops
# Save the public key output
```

### 3. Update flake.nix

Replace the placeholder public key in `flake.nix`:

```nix
nixConfig = {
  extra-substituters = [
    "https://cache.nixos.org"
    "https://towerops.cachix.org"
  ];
  extra-trusted-public-keys = [
    "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
    "towerops.cachix.org-1:YOUR_ACTUAL_PUBLIC_KEY_HERE"
  ];
};
```

### 4. Configure Local Machine

```bash
cachix use towerops
```

This updates `~/.config/nix/nix.conf` to use the cache.

### 5. Push Builds to Cache

```bash
# Push a specific build
nix build .#dockerImage
nix path-info --json .#dockerImage | jq -r '.[].path' | cachix push towerops

# Push all outputs
nix flake show --json | jq -r '.packages."x86_64-linux" | keys[]' | \
  xargs -I {} nix build .#{} --print-build-logs
nix flake show --json | jq -r '.packages."x86_64-linux" | keys[]' | \
  xargs -I {} nix path-info --json .#{} | jq -r '.[].path' | cachix push towerops
```

### 6. Configure CI/CD

Add `CACHIX_AUTH_TOKEN` to GitLab CI/CD variables:

1. Go to GitLab project → Settings → CI/CD → Variables
2. Add variable:
   - Key: `CACHIX_AUTH_TOKEN`
   - Value: (get from `cachix authtoken`)
   - Masked: Yes
   - Protected: Yes (optional)

## Troubleshooting

### Services Won't Start

**PostgreSQL fails to initialize**:
```bash
# Remove corrupted data directory
rm -rf .nix-postgres
# Re-enter shell to reinitialize
exit
nix develop
```

**Port already in use**:
```bash
# Check what's using the port
lsof -i :5432  # PostgreSQL
lsof -i :6379  # Redis

# Kill the process or change port in nix/shell.nix
```

### Nix Build Failures

**Error: experimental feature 'flakes' is not enabled**:
```bash
# Add to ~/.config/nix/nix.conf
experimental-features = nix-command flakes

# Or use --extra-experimental-features
nix --extra-experimental-features "nix-command flakes" develop
```

**Error: out of disk space**:
```bash
# Clean old generations
nix-collect-garbage -d

# Clean build artifacts
nix-store --gc
```

**C NIF compilation fails**:
```bash
# Check net-snmp is available
nix-shell -p net-snmp --run "pkg-config --libs netsnmp"

# Rebuild NIF separately
nix build .#towerops-nif --rebuild
```

### TimescaleDB Not Loading

If TimescaleDB was added after initial setup, you need to reinitialize PostgreSQL:

```bash
# Stop services and remove old data directory
stop-services
rm -rf .nix-postgres .nix-services-started

# Re-enter shell to reinitialize with TimescaleDB
exit
nix develop
```

This is required because `shared_preload_libraries` is set during `initdb` and the TimescaleDB shared library must be available in the PostgreSQL package.

### Mix Dependencies

**Error: Mix dependencies not found**:
```bash
# Ensure flake.lock is up to date
nix flake update

# Rebuild with fresh dependencies
nix build .#towerops --rebuild
```

**Vendored Oban packages**:
The `vendor/` directory is included in the Nix build. If you update vendored packages:

```bash
# Rebuild to pick up changes
nix build .#towerops --rebuild
```

### Docker Image Issues

**Image too large**:
```bash
# Check closure size
nix path-info -rS .#dockerImage

# Identify large dependencies
nix path-info -rS .#dockerImage | sort -k2 -h | tail -20
```

**Missing files in image**:
```bash
# List image contents
docker run --rm registry.gitlab.com/towerops/towerops:latest find / -name "*.mib" | head
```

### direnv Issues

**direnv: error .envrc is blocked**:
```bash
direnv allow
```

**Environment not loading**:
```bash
# Check direnv status
direnv status

# Reload manually
direnv reload
```

## Updating Dependencies

### Update Nix Flake Inputs

```bash
# Update all inputs
nix flake update

# Update specific input
nix flake update nixpkgs

# Update and rebuild
nix flake update && nix build .#towerops
```

### Update Elixir/Erlang Version

Edit `nix/shell.nix` and `nix/build.nix` to use different versions:

```nix
# Before
elixir
erlang_28

# After (if nixpkgs has newer versions)
elixir_1_20
erlang_29
```

Then rebuild:
```bash
nix flake update nixpkgs
nix build .#towerops --rebuild
```

### Pin Specific Nixpkgs Version

Edit `flake.nix` to pin to a specific commit:

```nix
inputs = {
  # Pin to specific nixpkgs commit
  nixpkgs.url = "github:NixOS/nixpkgs/abc123def456";
  # Or pin to a release branch
  nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
};
```

### Update Mix Dependencies

Mix dependencies are managed in `mix.exs` and `mix.lock` as usual. Nix will use these files:

```bash
# Update Mix dependencies
mix deps.update --all

# Commit updated mix.lock
git add mix.lock
git commit -m "deps: update Elixir dependencies"

# Rebuild with new dependencies
nix build .#towerops --rebuild
```

## Advanced Topics

### Cross-Platform Builds

Build for multiple architectures:

```bash
# Build for Linux ARM64 (Apple Silicon servers)
nix build .#packages.aarch64-linux.dockerImage

# Build for macOS on Linux (if configured)
nix build .#packages.x86_64-darwin.towerops
```

### Custom Nix Overlays

Create `nix/overlays.nix` to override nixpkgs packages:

```nix
final: prev: {
  # Example: Use specific Elixir version
  elixir = prev.elixir.override {
    version = "1.20.0";
  };
}
```

Reference in `flake.nix`:

```nix
_module.args.pkgs = import nixpkgs {
  inherit system;
  overlays = [ (import ./nix/overlays.nix) ];
};
```

### Development Without Nix

If Nix is not available, use the traditional development setup:

```bash
# Install Elixir via asdf
asdf install erlang 28.3
asdf install elixir 1.19.5-otp-28

# Install PostgreSQL and Redis
brew install postgresql@16 redis  # macOS
# Or: sudo apt-get install postgresql-16 redis  # Linux

# Start services manually
brew services start postgresql@16
brew services start redis

# Run development server
mix deps.get
mix ecto.setup
mix phx.server
```

## Resources

- [Nix Flakes Documentation](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html)
- [NixOS Packages Search](https://search.nixos.org/packages)
- [Cachix Documentation](https://docs.cachix.org/)
- [direnv Documentation](https://direnv.net/)
- [Elixir on Nix](https://nixos.wiki/wiki/Elixir)

## Getting Help

- **Nix Discord**: https://discord.gg/RbvHtGa
- **NixOS Discourse**: https://discourse.nixos.org/
- **Project Issues**: File issues on GitLab for project-specific problems