towerops/docs/references/golang-best-practices.md
Graham McIntie 1590f78bdc fix: input validation, SSRF, API hardening, and cookie security
- LIKE wildcard injection: sanitize %, _ in search queries (devices, sites, gaiia)
- Jason.decode! → Jason.decode with error handling for untrusted input
- inspect() leak: replace with generic error messages, log details server-side
- SSRF protection: URL validator blocks private IPs, localhost, non-HTTP schemes
- SSRF validation added to HTTP monitoring executor and integration credentials
- GraphQL complexity limits: always applied, not just in prod
- GraphQL introspection: also check GET query params, not just body
- Stripe webhook: explicit nil/empty checks for signature and body
- Cookie security: secure flag for session (prod), http_only+secure for remember_me
- Honeybadger API key: read from env var with fallback
- String.to_integer → Integer.parse with fallback for URL params
- String.to_atom → whitelist map for HTTP methods
- Gaiia webhook: remove secret_len and expected signature from log
- Admin API: add rate limiting pipeline
- to_atom_keys: per-key fallback instead of all-or-nothing rescue
2026-03-14 14:48:59 -05:00

1883 lines
43 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Go (Golang) Best Practices Reference
> Comprehensive reference for idiomatic, production-grade Go. Concrete examples throughout.
---
## Table of Contents
1. [Idiomatic Go](#1-idiomatic-go)
2. [Concurrency](#2-concurrency)
3. [Project Structure](#3-project-structure)
4. [Networking](#4-networking)
5. [Testing](#5-testing)
6. [Performance](#6-performance)
7. [Observability](#7-observability)
8. [Build & Deploy](#8-build--deploy)
---
## 1. Idiomatic Go
### Error Handling
#### Wrapping errors with `%w`
Always wrap errors with context using `fmt.Errorf` and `%w` so callers can unwrap:
```go
func readConfig(path string) (*Config, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("readConfig %s: %w", path, err)
}
var cfg Config
if err := json.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("readConfig unmarshal: %w", err)
}
return &cfg, nil
}
// Caller can inspect:
if errors.Is(err, os.ErrNotExist) {
// handle missing file
}
```
#### Sentinel errors
Define package-level sentinel errors for conditions callers need to check:
```go
package repo
import "errors"
var (
ErrNotFound = errors.New("not found")
ErrConflict = errors.New("conflict")
ErrUnauthorized = errors.New("unauthorized")
)
func (r *Repo) GetDevice(id string) (*Device, error) {
d, ok := r.devices[id]
if !ok {
return nil, fmt.Errorf("device %s: %w", id, ErrNotFound)
}
return d, nil
}
```
#### Custom error types
Use when you need structured error data beyond a message:
```go
type ValidationError struct {
Field string
Message string
}
func (e *ValidationError) Error() string {
return fmt.Sprintf("validation: %s — %s", e.Field, e.Message)
}
// Check with errors.As:
var ve *ValidationError
if errors.As(err, &ve) {
log.Printf("bad field: %s", ve.Field)
}
```
#### Don't use `errors.Is` / `errors.As` for every error
Only sentinel/typed errors need programmatic inspection. Most errors just bubble up with wrapping — that's fine. Don't over-engineer.
### Interface Design
#### Small interfaces
The best Go interfaces have 13 methods. The stdlib sets the standard:
```go
type Reader interface {
Read(p []byte) (n int, err error)
}
type Writer interface {
Write(p []byte) (n int, err error)
}
```
Define your own small interfaces close to where they're consumed:
```go
// In the handler package, not the repo package
type DeviceFinder interface {
FindDevice(ctx context.Context, id string) (*model.Device, error)
}
type handler struct {
finder DeviceFinder
}
```
#### Accept interfaces, return structs
```go
// Good — accepts interface, returns concrete type
func NewService(repo DeviceRepo, logger *slog.Logger) *Service {
return &Service{repo: repo, logger: logger}
}
// Bad — returning an interface hides the concrete type unnecessarily
func NewService(repo DeviceRepo) ServiceInterface { ... }
```
This lets callers benefit from the full concrete type while keeping your function testable via the interface parameter.
#### Don't define interfaces prematurely
> "The bigger the interface, the weaker the abstraction." — Rob Pike
Define interfaces when you have ≥2 implementations OR need to mock in tests. Not before.
### Embedding
Use struct embedding for composition, not inheritance:
```go
type BaseModel struct {
ID string `json:"id"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
type Device struct {
BaseModel
Name string `json:"name"`
IP string `json:"ip"`
SNMPPort int `json:"snmp_port"`
}
// Device now has .ID, .CreatedAt, etc. promoted
d := Device{Name: "switch-01"}
d.ID = "dev_123"
```
Embed interfaces to signal partial implementation or to compose behaviors:
```go
type ReadCloser struct {
io.Reader
io.Closer
}
```
**Watch out**: Embedding promotes all methods, including ones you don't want. If the embedded type has a `String()` method, your type now satisfies `fmt.Stringer` — maybe unintentionally.
### Functional Options Pattern
For constructors with many optional parameters:
```go
type Server struct {
addr string
readTimeout time.Duration
writeTimeout time.Duration
maxConns int
logger *slog.Logger
}
type Option func(*Server)
func WithReadTimeout(d time.Duration) Option {
return func(s *Server) { s.readTimeout = d }
}
func WithWriteTimeout(d time.Duration) Option {
return func(s *Server) { s.writeTimeout = d }
}
func WithMaxConns(n int) Option {
return func(s *Server) { s.maxConns = n }
}
func WithLogger(l *slog.Logger) Option {
return func(s *Server) { s.logger = l }
}
func NewServer(addr string, opts ...Option) *Server {
s := &Server{
addr: addr,
readTimeout: 5 * time.Second, // sensible defaults
writeTimeout: 10 * time.Second,
maxConns: 100,
logger: slog.Default(),
}
for _, opt := range opts {
opt(s)
}
return s
}
// Usage:
srv := NewServer(":8080",
WithReadTimeout(10*time.Second),
WithLogger(myLogger),
)
```
This is clean, extensible, and self-documenting. Prefer it over config structs when options are genuinely optional.
---
## 2. Concurrency
### Goroutines + Channels
#### Basic patterns
```go
// Fire-and-forget (careful — no error handling)
go processEvent(event)
// Channel for result
ch := make(chan Result, 1)
go func() {
ch <- expensiveComputation()
}()
result := <-ch
```
#### Always know how a goroutine ends
Every goroutine you start should have a clear termination path. Leaked goroutines are memory leaks.
```go
// Bad — goroutine leaks if ctx is never cancelled and ch is never read
go func() {
ch <- doWork(ctx)
}()
// Good — select with context
go func() {
select {
case ch <- doWork(ctx):
case <-ctx.Done():
}
}()
```
### sync.WaitGroup
For waiting on a known set of goroutines:
```go
func processAll(ctx context.Context, items []Item) error {
var wg sync.WaitGroup
errs := make(chan error, len(items))
for _, item := range items {
wg.Add(1)
go func() {
defer wg.Done()
if err := process(ctx, item); err != nil {
errs <- err
}
}()
}
wg.Wait()
close(errs)
for err := range errs {
return err // return first error
}
return nil
}
```
### Mutex vs Channels
**Rule of thumb**:
- **Mutex**: Protecting shared state (maps, counters, caches)
- **Channels**: Communicating between goroutines, coordinating work
```go
// Mutex — protecting a shared map
type SafeMap struct {
mu sync.RWMutex
m map[string]int
}
func (s *SafeMap) Get(key string) (int, bool) {
s.mu.RLock()
defer s.mu.RUnlock()
v, ok := s.m[key]
return v, ok
}
func (s *SafeMap) Set(key string, val int) {
s.mu.Lock()
defer s.mu.Unlock()
s.m[key] = val
}
// Channel — pipeline stage
func square(in <-chan int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for n := range in {
out <- n * n
}
}()
return out
}
```
Use `sync.RWMutex` when reads vastly outnumber writes.
### context.Context
#### Cancellation
```go
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go worker(ctx)
// Cancel after signal
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
<-sigCh
cancel() // all workers using ctx will see Done()
}
func worker(ctx context.Context) {
for {
select {
case <-ctx.Done():
log.Println("worker shutting down:", ctx.Err())
return
default:
doWork()
}
}
}
```
#### Timeouts
```go
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel() // always defer cancel to release resources
result, err := fetchFromAPI(ctx, url)
if errors.Is(err, context.DeadlineExceeded) {
log.Println("API call timed out")
}
```
#### Context values — use sparingly
Only for request-scoped data that crosses API boundaries (trace IDs, auth tokens). Never for optional parameters.
```go
type ctxKey string
const requestIDKey ctxKey = "request_id"
func WithRequestID(ctx context.Context, id string) context.Context {
return context.WithValue(ctx, requestIDKey, id)
}
func RequestID(ctx context.Context) string {
id, _ := ctx.Value(requestIDKey).(string)
return id
}
```
### errgroup
`golang.org/x/sync/errgroup` — the standard tool for concurrent work with error propagation:
```go
import "golang.org/x/sync/errgroup"
func fetchAll(ctx context.Context, urls []string) ([]Response, error) {
g, ctx := errgroup.WithContext(ctx)
responses := make([]Response, len(urls))
for i, url := range urls {
g.Go(func() error {
resp, err := fetch(ctx, url)
if err != nil {
return fmt.Errorf("fetch %s: %w", url, err)
}
responses[i] = resp // safe — each goroutine writes to unique index
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
return responses, nil
}
```
With concurrency limit:
```go
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(10) // max 10 concurrent goroutines
```
### Worker Pool
```go
func workerPool(ctx context.Context, jobs <-chan Job, numWorkers int) <-chan Result {
results := make(chan Result, numWorkers)
var wg sync.WaitGroup
for range numWorkers {
wg.Add(1)
go func() {
defer wg.Done()
for job := range jobs {
select {
case <-ctx.Done():
return
case results <- process(job):
}
}
}()
}
go func() {
wg.Wait()
close(results)
}()
return results
}
```
### Fan-Out / Fan-In
```go
// Fan-out: distribute work to multiple goroutines
func fanOut(ctx context.Context, input <-chan Task, workers int) []<-chan Result {
channels := make([]<-chan Result, workers)
for i := range workers {
channels[i] = worker(ctx, input)
}
return channels
}
// Fan-in: merge multiple channels into one
func fanIn(ctx context.Context, channels ...<-chan Result) <-chan Result {
merged := make(chan Result)
var wg sync.WaitGroup
for _, ch := range channels {
wg.Add(1)
go func() {
defer wg.Done()
for result := range ch {
select {
case merged <- result:
case <-ctx.Done():
return
}
}
}()
}
go func() {
wg.Wait()
close(merged)
}()
return merged
}
```
---
## 3. Project Structure
### Standard Layout
```
myproject/
├── cmd/
│ ├── server/ # main application entry point
│ │ └── main.go
│ └── cli/ # CLI tool
│ └── main.go
├── internal/ # private packages — can't be imported by other modules
│ ├── server/ # HTTP/gRPC server setup
│ ├── handler/ # request handlers
│ ├── service/ # business logic
│ ├── repo/ # data access
│ ├── model/ # domain types
│ └── config/ # configuration
├── proto/ # protobuf definitions
│ └── v1/
├── migrations/ # database migrations
├── docs/
├── scripts/
├── Makefile
├── Dockerfile
├── go.mod
└── go.sum
```
#### The `/pkg` debate
**Skip `/pkg` for most projects.** It was popularized by early Go projects but adds no real value — `internal/` enforces visibility; `pkg/` is just convention. If you need to export packages for other modules, put them at the root or in a clearly named directory.
#### `internal/` is your friend
Anything in `internal/` can't be imported by other Go modules. Use it aggressively — you can always promote later.
### Dependency Injection with Wire
[Wire](https://github.com/google/wire) generates DI code at compile time — no runtime reflection.
```go
// internal/server/wire.go
//go:build wireinject
package server
import "github.com/google/wire"
func InitializeApp(cfg *config.Config) (*App, error) {
wire.Build(
NewApp,
handler.NewDeviceHandler,
service.NewDeviceService,
repo.NewPostgresRepo,
db.NewPool,
)
return nil, nil
}
```
Run `wire ./internal/server/` to generate the actual initialization code.
**Alternative**: For simpler projects, manual DI in `main()` is fine. Don't use Wire until you have enough components to justify it.
### Config Management
#### `envconfig` — simple, env-var-based
```go
import "github.com/kelseyhightower/envconfig"
type Config struct {
Port int `envconfig:"PORT" default:"8080"`
DatabaseURL string `envconfig:"DATABASE_URL" required:"true"`
LogLevel string `envconfig:"LOG_LEVEL" default:"info"`
Timeout time.Duration `envconfig:"TIMEOUT" default:"30s"`
}
func LoadConfig() (*Config, error) {
var cfg Config
if err := envconfig.Process("", &cfg); err != nil {
return nil, fmt.Errorf("load config: %w", err)
}
return &cfg, nil
}
```
#### Alternatives
| Library | Best for |
|---------|----------|
| `envconfig` | Simple env-var config, 12-factor apps |
| `koanf` | Multi-source config (env, files, flags), lighter than Viper |
| `viper` | Kitchen-sink config (heavy, pulls in many deps) |
| `ff` | Flags-first config with env var fallback |
**Recommendation**: Start with `envconfig`. Graduate to `koanf` if you need config files or multiple sources. Avoid Viper unless you're already using it.
### Makefile Patterns
```makefile
.PHONY: build test lint run clean proto generate
# Variables
BINARY := myapp
GO := go
GOFLAGS := -trimpath
LDFLAGS := -s -w -X main.version=$(shell git describe --tags --always --dirty)
CGO_ENABLED := 0
## Build
build:
$(GO) build $(GOFLAGS) -ldflags "$(LDFLAGS)" -o bin/$(BINARY) ./cmd/server
## Run locally
run:
$(GO) run ./cmd/server
## Test
test:
$(GO) test -race -count=1 ./...
test-integration:
$(GO) test -race -count=1 -tags=integration -timeout=5m ./...
test-coverage:
$(GO) test -race -coverprofile=coverage.out ./...
$(GO) tool cover -html=coverage.out -o coverage.html
## Lint
lint:
golangci-lint run ./...
## Proto generation
proto:
buf generate
## Clean
clean:
rm -rf bin/ coverage.out coverage.html
## Generate (wire, mocks, etc.)
generate:
$(GO) generate ./...
## All — CI pipeline
all: lint test build
```
---
## 4. Networking
### net/http Server Best Practices
#### Always set timeouts
```go
srv := &http.Server{
Addr: ":8080",
Handler: router,
ReadTimeout: 5 * time.Second,
WriteTimeout: 10 * time.Second,
IdleTimeout: 120 * time.Second,
// For request body size limits:
MaxHeaderBytes: 1 << 20, // 1 MB
}
```
Never use `http.ListenAndServe(":8080", nil)` in production — no timeouts = DDoS target.
#### Graceful shutdown
```go
func run(ctx context.Context) error {
srv := &http.Server{
Addr: ":8080",
Handler: newRouter(),
ReadTimeout: 5 * time.Second,
WriteTimeout: 10 * time.Second,
}
// Start server in goroutine
errCh := make(chan error, 1)
go func() {
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
errCh <- err
}
}()
// Wait for interrupt
select {
case err := <-errCh:
return fmt.Errorf("server error: %w", err)
case <-ctx.Done():
}
// Graceful shutdown with timeout
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := srv.Shutdown(shutdownCtx); err != nil {
return fmt.Errorf("shutdown: %w", err)
}
return nil
}
```
#### Middleware chains
```go
// Middleware type
type Middleware func(http.Handler) http.Handler
// Chain composes middleware
func Chain(h http.Handler, mw ...Middleware) http.Handler {
for i := len(mw) - 1; i >= 0; i-- {
h = mw[i](h)
}
return h
}
// Logging middleware
func LoggingMiddleware(logger *slog.Logger) Middleware {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
wrapped := &responseWriter{ResponseWriter: w, statusCode: 200}
next.ServeHTTP(wrapped, r)
logger.Info("request",
"method", r.Method,
"path", r.URL.Path,
"status", wrapped.statusCode,
"duration", time.Since(start),
)
})
}
}
// Recovery middleware
func RecoveryMiddleware(logger *slog.Logger) Middleware {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
if err := recover(); err != nil {
logger.Error("panic recovered",
"error", err,
"stack", string(debug.Stack()),
)
http.Error(w, "internal server error", http.StatusInternalServerError)
}
}()
next.ServeHTTP(w, r)
})
}
}
// Usage
handler := Chain(router,
RecoveryMiddleware(logger),
LoggingMiddleware(logger),
CORSMiddleware(),
)
```
#### responseWriter wrapper for capturing status
```go
type responseWriter struct {
http.ResponseWriter
statusCode int
}
func (rw *responseWriter) WriteHeader(code int) {
rw.statusCode = code
rw.ResponseWriter.WriteHeader(code)
}
```
### gRPC + Protobuf
#### Proto file conventions
```protobuf
syntax = "proto3";
package towerops.device.v1;
option go_package = "github.com/towerops/towerops/gen/device/v1;devicev1";
service DeviceService {
rpc GetDevice(GetDeviceRequest) returns (GetDeviceResponse);
rpc ListDevices(ListDevicesRequest) returns (ListDevicesResponse);
rpc WatchDevices(WatchDevicesRequest) returns (stream DeviceEvent);
}
message GetDeviceRequest {
string id = 1;
}
message GetDeviceResponse {
Device device = 1;
}
```
#### Code generation with buf
```yaml
# buf.gen.yaml
version: v2
plugins:
- remote: buf.build/protocolbuffers/go
out: gen
opt: paths=source_relative
- remote: buf.build/grpc/go
out: gen
opt: paths=source_relative
```
#### gRPC interceptors (middleware equivalent)
```go
import (
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
// Unary interceptor for logging
func loggingInterceptor(logger *slog.Logger) grpc.UnaryServerInterceptor {
return func(
ctx context.Context,
req any,
info *grpc.UnaryServerInfo,
handler grpc.UnaryHandler,
) (any, error) {
start := time.Now()
resp, err := handler(ctx, req)
logger.Info("grpc",
"method", info.FullMethod,
"duration", time.Since(start),
"error", err,
)
return resp, err
}
}
// Recovery interceptor
func recoveryInterceptor(logger *slog.Logger) grpc.UnaryServerInterceptor {
return func(
ctx context.Context,
req any,
info *grpc.UnaryServerInfo,
handler grpc.UnaryHandler,
) (resp any, err error) {
defer func() {
if r := recover(); r != nil {
logger.Error("grpc panic", "error", r, "stack", string(debug.Stack()))
err = status.Errorf(codes.Internal, "internal error")
}
}()
return handler(ctx, req)
}
}
// Server setup
srv := grpc.NewServer(
grpc.ChainUnaryInterceptor(
recoveryInterceptor(logger),
loggingInterceptor(logger),
),
grpc.ChainStreamInterceptor(
// stream interceptors...
),
)
```
#### Server-side streaming
```go
func (s *server) WatchDevices(
req *pb.WatchDevicesRequest,
stream pb.DeviceService_WatchDevicesServer,
) error {
ctx := stream.Context()
ch := s.deviceEvents.Subscribe()
defer s.deviceEvents.Unsubscribe(ch)
for {
select {
case <-ctx.Done():
return ctx.Err()
case event := <-ch:
if err := stream.Send(event); err != nil {
return fmt.Errorf("send event: %w", err)
}
}
}
}
```
### HTTP Client Patterns
#### Reuse clients — never create per-request
```go
// Package-level or injected client with connection pooling
var httpClient = &http.Client{
Timeout: 30 * time.Second,
Transport: &http.Transport{
MaxIdleConns: 100,
MaxIdleConnsPerHost: 10,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
},
}
```
#### Retry with exponential backoff
```go
func doWithRetry(ctx context.Context, req *http.Request, maxRetries int) (*http.Response, error) {
var resp *http.Response
var err error
backoff := 100 * time.Millisecond
for attempt := range maxRetries {
resp, err = httpClient.Do(req.Clone(ctx))
if err != nil {
// Network error — retry
} else if resp.StatusCode < 500 && resp.StatusCode != 429 {
return resp, nil
} else {
resp.Body.Close() // must close before retry
}
if attempt == maxRetries-1 {
break
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(backoff):
}
backoff *= 2 // exponential
}
if err != nil {
return nil, fmt.Errorf("after %d retries: %w", maxRetries, err)
}
return resp, nil
}
```
#### Always close response bodies
```go
resp, err := httpClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
// Drain body even if you don't need it — allows connection reuse
io.Copy(io.Discard, resp.Body)
```
---
## 5. Testing
### Table-Driven Tests
The Go standard for test organization:
```go
func TestParsePort(t *testing.T) {
tests := []struct {
name string
input string
want int
wantErr bool
}{
{name: "valid port", input: "8080", want: 8080},
{name: "zero", input: "0", want: 0},
{name: "max", input: "65535", want: 65535},
{name: "negative", input: "-1", wantErr: true},
{name: "too high", input: "65536", wantErr: true},
{name: "not a number", input: "abc", wantErr: true},
{name: "empty", input: "", wantErr: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ParsePort(tt.input)
if tt.wantErr {
if err == nil {
t.Fatal("expected error, got nil")
}
return
}
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got != tt.want {
t.Errorf("got %d, want %d", got, tt.want)
}
})
}
}
```
### testify vs stdlib
**stdlib** is fine for most cases. `t.Errorf` / `t.Fatalf` with clear messages work great.
**testify** helps when assertions get complex:
```go
import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestDeviceService(t *testing.T) {
svc := NewDeviceService(mockRepo)
device, err := svc.Get(ctx, "dev_123")
require.NoError(t, err) // stops test on failure
assert.Equal(t, "switch-01", device.Name)
assert.NotEmpty(t, device.ID)
assert.WithinDuration(t, time.Now(), device.CreatedAt, time.Minute)
}
```
`require` = fatal on failure (test stops). `assert` = log and continue. Use `require` for setup/preconditions, `assert` for actual checks.
### httptest
Test HTTP handlers without a real server:
```go
func TestGetDevice(t *testing.T) {
// Setup
repo := &mockRepo{
devices: map[string]*Device{
"dev_1": {ID: "dev_1", Name: "switch-01"},
},
}
handler := NewDeviceHandler(repo)
// Create request
req := httptest.NewRequest(http.MethodGet, "/devices/dev_1", nil)
rec := httptest.NewRecorder()
// Serve
handler.ServeHTTP(rec, req)
// Assert
assert.Equal(t, http.StatusOK, rec.Code)
var got Device
err := json.NewDecoder(rec.Body).Decode(&got)
require.NoError(t, err)
assert.Equal(t, "switch-01", got.Name)
}
```
For full integration tests with routing:
```go
func TestAPI(t *testing.T) {
srv := httptest.NewServer(newRouter())
defer srv.Close()
resp, err := http.Get(srv.URL + "/health")
require.NoError(t, err)
defer resp.Body.Close()
assert.Equal(t, 200, resp.StatusCode)
}
```
### Mocking with Interfaces (not frameworks)
Define the interface where it's consumed, implement a mock manually:
```go
// In handler package
type DeviceRepo interface {
Get(ctx context.Context, id string) (*model.Device, error)
List(ctx context.Context) ([]*model.Device, error)
}
// In handler test file
type mockRepo struct {
devices map[string]*model.Device
err error
}
func (m *mockRepo) Get(ctx context.Context, id string) (*model.Device, error) {
if m.err != nil {
return nil, m.err
}
d, ok := m.devices[id]
if !ok {
return nil, repo.ErrNotFound
}
return d, nil
}
func (m *mockRepo) List(ctx context.Context) ([]*model.Device, error) {
if m.err != nil {
return nil, m.err
}
var result []*model.Device
for _, d := range m.devices {
result = append(result, d)
}
return result, nil
}
```
This is simpler, more readable, and more maintainable than mock frameworks like gomock. For large interfaces, consider `moq` or `mockery` for code generation — but keep mocks simple.
### Integration Tests with testcontainers-go
```go
//go:build integration
package repo_test
import (
"context"
"testing"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/modules/postgres"
"github.com/testcontainers/testcontainers-go/wait"
)
func TestPostgresRepo(t *testing.T) {
ctx := context.Background()
pgContainer, err := postgres.Run(ctx,
"postgres:16-alpine",
postgres.WithDatabase("testdb"),
postgres.WithUsername("test"),
postgres.WithPassword("test"),
testcontainers.WithWaitStrategy(
wait.ForLog("database system is ready to accept connections").
WithOccurrence(2),
),
)
require.NoError(t, err)
t.Cleanup(func() { pgContainer.Terminate(ctx) })
connStr, err := pgContainer.ConnectionString(ctx, "sslmode=disable")
require.NoError(t, err)
// Run migrations
runMigrations(t, connStr)
// Create repo with real DB
r, err := repo.NewPostgresRepo(ctx, connStr)
require.NoError(t, err)
// Test actual queries
t.Run("create and get", func(t *testing.T) {
device := &model.Device{Name: "test-switch"}
err := r.Create(ctx, device)
require.NoError(t, err)
assert.NotEmpty(t, device.ID)
got, err := r.Get(ctx, device.ID)
require.NoError(t, err)
assert.Equal(t, "test-switch", got.Name)
})
}
```
### go test Flags
```bash
# Race detector — always use in CI
go test -race ./...
# Disable test caching (important for flaky test detection)
go test -count=1 ./...
# Run tests in parallel (default is GOMAXPROCS)
go test -parallel=4 ./...
# Only run matching tests
go test -run TestGetDevice ./internal/handler/
# Verbose output
go test -v ./...
# Short mode (skip slow tests)
go test -short ./...
# Timeout
go test -timeout=5m ./...
# Integration tests only (with build tag)
go test -tags=integration ./...
# Combined for CI:
go test -race -count=1 -timeout=10m ./...
```
In test code, respect `-short`:
```go
func TestSlowIntegration(t *testing.T) {
if testing.Short() {
t.Skip("skipping in short mode")
}
// slow test...
}
```
---
## 6. Performance
### pprof Profiling
#### Enable in HTTP server
```go
import _ "net/http/pprof"
// If using default mux, pprof is auto-registered.
// For custom mux, register explicitly:
mux.HandleFunc("/debug/pprof/", pprof.Index)
mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
mux.HandleFunc("/debug/pprof/heap", pprof.Handler("heap").ServeHTTP)
mux.HandleFunc("/debug/pprof/goroutine", pprof.Handler("goroutine").ServeHTTP)
```
#### Capture and analyze
```bash
# CPU profile (30 seconds)
go tool pprof http://localhost:8080/debug/pprof/profile?seconds=30
# Heap profile
go tool pprof http://localhost:8080/debug/pprof/heap
# Goroutine dump
curl http://localhost:8080/debug/pprof/goroutine?debug=2
# Interactive commands in pprof:
# top20 — top 20 functions by CPU/memory
# list funcName — show annotated source
# web — open flamegraph in browser
```
### Benchmarking
```go
func BenchmarkProcessEvent(b *testing.B) {
event := createTestEvent()
b.ResetTimer()
for b.Loop() {
processEvent(event)
}
}
// With sub-benchmarks for different sizes
func BenchmarkParse(b *testing.B) {
sizes := []int{10, 100, 1000, 10000}
for _, size := range sizes {
b.Run(fmt.Sprintf("size=%d", size), func(b *testing.B) {
data := generateData(size)
b.ResetTimer()
for b.Loop() {
parse(data)
}
})
}
}
// Report allocations
func BenchmarkBuildString(b *testing.B) {
b.ReportAllocs()
for b.Loop() {
buildString(1000)
}
}
```
```bash
# Run benchmarks
go test -bench=. -benchmem ./...
# Compare benchmarks (install benchstat)
go test -bench=. -count=10 > old.txt
# ... make changes ...
go test -bench=. -count=10 > new.txt
benchstat old.txt new.txt
```
### Memory Allocation Patterns
#### sync.Pool — reuse temporary objects
```go
var bufPool = sync.Pool{
New: func() any {
return new(bytes.Buffer)
},
}
func processRequest(data []byte) string {
buf := bufPool.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
bufPool.Put(buf)
}()
buf.Write(data)
// ... process ...
return buf.String()
}
```
#### Pre-allocation
```go
// Bad — grows the slice repeatedly
var result []string
for _, item := range items {
result = append(result, item.Name)
}
// Good — allocate once
result := make([]string, 0, len(items))
for _, item := range items {
result = append(result, item.Name)
}
// Same for maps
m := make(map[string]int, len(items))
```
### Escape Analysis
Check what escapes to the heap:
```bash
go build -gcflags='-m -m' ./... 2>&1 | grep 'escapes to heap'
```
Tips to reduce heap allocations:
- Return values instead of pointers when the struct is small
- Use value receivers for small types
- Avoid closures that capture variables (they often escape)
- Pre-size slices and maps
```go
// This might stay on stack (small struct, returned by value)
func newPoint(x, y int) Point {
return Point{X: x, Y: y}
}
// This always escapes to heap (pointer returned)
func newPoint(x, y int) *Point {
return &Point{X: x, Y: y}
}
```
### String Building
```go
// Bad — O(n²) concatenation
s := ""
for _, item := range items {
s += item.Name + ","
}
// Good — strings.Builder
var b strings.Builder
b.Grow(len(items) * 20) // estimate capacity
for i, item := range items {
if i > 0 {
b.WriteByte(',')
}
b.WriteString(item.Name)
}
result := b.String()
// For simple joins
result := strings.Join(names, ",")
```
---
## 7. Observability
### Structured Logging with slog (Go 1.21+)
```go
import "log/slog"
// Setup
func setupLogger(level string) *slog.Logger {
var lvl slog.Level
switch level {
case "debug":
lvl = slog.LevelDebug
case "warn":
lvl = slog.LevelWarn
case "error":
lvl = slog.LevelError
default:
lvl = slog.LevelInfo
}
return slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
Level: lvl,
AddSource: true, // adds file:line to log entries
}))
}
// Usage
logger := setupLogger("info")
logger.Info("server started", "addr", ":8080", "version", version)
logger.Error("query failed",
"error", err,
"query", query,
"duration", time.Since(start),
)
// With context (carries request ID, trace ID, etc.)
logger.InfoContext(ctx, "device updated",
"device_id", device.ID,
"changes", changes,
)
// Child logger with common fields
reqLogger := logger.With(
"request_id", requestID,
"method", r.Method,
"path", r.URL.Path,
)
```
#### slog groups for structured fields
```go
logger.Info("request",
slog.Group("http",
slog.String("method", r.Method),
slog.String("path", r.URL.Path),
slog.Int("status", status),
),
slog.Group("timing",
slog.Duration("total", totalDuration),
slog.Duration("db", dbDuration),
),
)
// Output: {"http":{"method":"GET","path":"/devices","status":200},"timing":{"total":"12ms","db":"3ms"}}
```
### OpenTelemetry
#### Tracing setup
```go
import (
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
"go.opentelemetry.io/otel/sdk/resource"
sdktrace "go.opentelemetry.io/otel/sdk/trace"
semconv "go.opentelemetry.io/otel/semconv/v1.24.0"
)
func initTracer(ctx context.Context, serviceName, version string) (func(), error) {
exporter, err := otlptracegrpc.New(ctx)
if err != nil {
return nil, fmt.Errorf("create exporter: %w", err)
}
res, err := resource.New(ctx,
resource.WithAttributes(
semconv.ServiceNameKey.String(serviceName),
semconv.ServiceVersionKey.String(version),
),
)
if err != nil {
return nil, fmt.Errorf("create resource: %w", err)
}
tp := sdktrace.NewTracerProvider(
sdktrace.WithBatcher(exporter),
sdktrace.WithResource(res),
sdktrace.WithSampler(sdktrace.AlwaysSample()), // tune in prod
)
otel.SetTracerProvider(tp)
cleanup := func() {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
tp.Shutdown(ctx)
}
return cleanup, nil
}
```
#### Using traces
```go
var tracer = otel.Tracer("myapp/service")
func (s *DeviceService) GetDevice(ctx context.Context, id string) (*Device, error) {
ctx, span := tracer.Start(ctx, "DeviceService.GetDevice",
trace.WithAttributes(attribute.String("device.id", id)),
)
defer span.End()
device, err := s.repo.Get(ctx, id)
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
return nil, err
}
return device, nil
}
```
### Prometheus Metrics
```go
import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
var (
httpRequestsTotal = promauto.NewCounterVec(
prometheus.CounterOpts{
Name: "http_requests_total",
Help: "Total HTTP requests",
},
[]string{"method", "path", "status"},
)
httpRequestDuration = promauto.NewHistogramVec(
prometheus.HistogramOpts{
Name: "http_request_duration_seconds",
Help: "HTTP request duration in seconds",
Buckets: prometheus.DefBuckets,
},
[]string{"method", "path"},
)
activeConnections = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "active_connections",
Help: "Number of active connections",
},
)
)
// Metrics middleware
func MetricsMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
wrapped := &responseWriter{ResponseWriter: w, statusCode: 200}
next.ServeHTTP(wrapped, r)
duration := time.Since(start).Seconds()
httpRequestsTotal.WithLabelValues(r.Method, r.URL.Path, strconv.Itoa(wrapped.statusCode)).Inc()
httpRequestDuration.WithLabelValues(r.Method, r.URL.Path).Observe(duration)
})
}
// Expose metrics endpoint
mux.Handle("/metrics", promhttp.Handler())
```
### Health Checks
```go
type HealthChecker struct {
db *sql.DB
redis *redis.Client
}
type HealthStatus struct {
Status string `json:"status"`
Checks map[string]string `json:"checks"`
}
// Liveness — is the process alive? Keep it simple.
func (h *HealthChecker) Liveness(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write([]byte(`{"status":"ok"}`))
}
// Readiness — can the process handle traffic?
func (h *HealthChecker) Readiness(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
status := HealthStatus{
Status: "ok",
Checks: make(map[string]string),
}
if err := h.db.PingContext(ctx); err != nil {
status.Status = "degraded"
status.Checks["database"] = err.Error()
} else {
status.Checks["database"] = "ok"
}
if err := h.redis.Ping(ctx).Err(); err != nil {
status.Status = "degraded"
status.Checks["redis"] = err.Error()
} else {
status.Checks["redis"] = "ok"
}
code := http.StatusOK
if status.Status != "ok" {
code = http.StatusServiceUnavailable
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
json.NewEncoder(w).Encode(status)
}
// Register
mux.HandleFunc("GET /healthz", health.Liveness)
mux.HandleFunc("GET /readyz", health.Readiness)
```
---
## 8. Build & Deploy
### Multi-Stage Docker Builds
```dockerfile
# Stage 1: Build
FROM golang:1.24-alpine AS builder
WORKDIR /app
# Cache dependencies
COPY go.mod go.sum ./
RUN go mod download
# Copy source and build
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build \
-trimpath \
-ldflags="-s -w -X main.version=$(cat VERSION)" \
-o /app/server \
./cmd/server
# Stage 2: Runtime
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /app/server /server
EXPOSE 8080
USER nonroot:nonroot
ENTRYPOINT ["/server"]
```
**Key points**:
- `CGO_ENABLED=0` for static binary (no libc dependency)
- `-trimpath` removes local paths from binary
- `-ldflags="-s -w"` strips debug info (smaller binary)
- `distroless` or `scratch` for minimal attack surface
- `nonroot` user for security
### CGO Considerations
**Default: Avoid CGO.** `CGO_ENABLED=0` gives you a static binary that runs anywhere.
When you need CGO (SQLite, certain crypto libs):
```dockerfile
FROM golang:1.24-alpine AS builder
RUN apk add --no-cache gcc musl-dev
# CGO_ENABLED=1 is default when gcc is available
RUN go build -o /app/server ./cmd/server
FROM alpine:3.20
RUN apk add --no-cache ca-certificates
COPY --from=builder /app/server /server
ENTRYPOINT ["/server"]
```
### Cross-Compilation
```bash
# Build for Linux AMD64
GOOS=linux GOARCH=amd64 go build -o myapp-linux-amd64 ./cmd/server
# Build for Linux ARM64
GOOS=linux GOARCH=arm64 go build -o myapp-linux-arm64 ./cmd/server
# Build for macOS ARM64 (Apple Silicon)
GOOS=darwin GOARCH=arm64 go build -o myapp-darwin-arm64 ./cmd/server
# Build for Windows
GOOS=windows GOARCH=amd64 go build -o myapp-windows-amd64.exe ./cmd/server
```
Cross-compilation only works cleanly with `CGO_ENABLED=0`. If you need CGO + cross-compile, use Docker or [zig as CC](https://dev.to/kristoff/zig-makes-go-cross-compilation-just-work-29ho).
### GoReleaser
```yaml
# .goreleaser.yaml
version: 2
builds:
- id: server
main: ./cmd/server
binary: myapp
env:
- CGO_ENABLED=0
goos:
- linux
- darwin
goarch:
- amd64
- arm64
ldflags:
- -s -w
- -X main.version={{.Version}}
- -X main.commit={{.Commit}}
- -X main.date={{.Date}}
archives:
- format: tar.gz
name_template: "{{ .ProjectName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}"
dockers:
- image_templates:
- "ghcr.io/myorg/myapp:{{ .Version }}-amd64"
dockerfile: Dockerfile
use: buildx
build_flag_templates:
- "--platform=linux/amd64"
changelog:
sort: asc
filters:
exclude:
- "^docs:"
- "^test:"
- "^ci:"
```
```bash
# Test locally
goreleaser release --snapshot --clean
# Release (triggered by git tag)
git tag v1.0.0
git push origin v1.0.0
# CI runs: goreleaser release
```
### Linting with golangci-lint
```yaml
# .golangci.yml
run:
timeout: 5m
linters:
enable:
- errcheck # unchecked errors
- govet # vet checks
- staticcheck # comprehensive static analysis
- unused # unused code
- gosimple # simplifications
- ineffassign # useless assignments
- typecheck # type checking
- gocritic # opinionated checks
- revive # flexible linter (replaces golint)
- misspell # spelling
- prealloc # pre-allocation suggestions
- unconvert # unnecessary conversions
- noctx # HTTP requests without context
- bodyclose # unclosed HTTP response bodies
- errname # error naming conventions
- errorlint # error wrapping checks
- exhaustive # exhaustive enum switches
- copyloopvar # loop variable capture bugs (Go <1.22)
linters-settings:
gocritic:
enabled-tags:
- diagnostic
- performance
disabled-checks:
- ifElseChain
revive:
rules:
- name: exported
arguments: [checkPrivateReceivers]
- name: blank-imports
- name: context-as-argument
- name: error-return
- name: error-naming
- name: increment-decrement
- name: var-naming
govet:
enable-all: true
issues:
exclude-rules:
- path: _test\.go
linters:
- errcheck
- gocritic
max-issues-per-linter: 0
max-same-issues: 0
```
```bash
# Run
golangci-lint run ./...
# Fix auto-fixable issues
golangci-lint run --fix ./...
# Only new issues (great for CI on PRs)
golangci-lint run --new-from-rev=main ./...
```
---
## Quick Reference: Go Proverbs
- Don't communicate by sharing memory; share memory by communicating.
- Concurrency is not parallelism.
- The bigger the interface, the weaker the abstraction.
- Make the zero value useful.
- `interface{}` says nothing (use generics or concrete types).
- A little copying is better than a little dependency.
- Clear is better than clever.
- Errors are values — program with them.
- Don't just check errors, handle them gracefully.
## Further Reading
- [Effective Go](https://go.dev/doc/effective_go)
- [Go Code Review Comments](https://go.dev/wiki/CodeReviewComments)
- [Go Proverbs](https://go-proverbs.github.io/)
- [Uber Go Style Guide](https://github.com/uber-go/guide/blob/master/style.md)
- [100 Go Mistakes](https://100go.co/)
- [Go Blog](https://go.dev/blog/)