# Deploy to Kubernetes

Run the generated server and worker on Kubernetes with probes, graceful shutdown, config and secrets, a PodDisruptionBudget, and an HorizontalPodAutoscaler.

This guide turns the [deployment checklist](https://zenta-dev.github.io/zever/guides/operate/deployment/) into Kubernetes manifests for a project made with `zever new`: one image per entrypoint, config in a ConfigMap, secrets in a Secret, health probes against `/healthz` and `/readyz`, and a shutdown sequence that drains traffic before the process exits. The manifests are a starting point to copy and adapt; they are not generated by the CLI.

Replace `registry.example.com/myapp`, the namespace, and the resource numbers with your own. Everything below assumes the generated server's defaults: HTTP on `8080`, gRPC on `9090`, a 10 second shutdown grace, and configuration from environment variables (`.env`).

## Images

`zever new` writes a multi-stage `Dockerfile` that builds `./cmd/server` into a distroless, non-root image. Build the worker the same way by pointing the builder at `./cmd/worker` (the Dockerfile ships a commented example for it), and tag the two images separately so web and worker scale independently.

```sh
docker build -t registry.example.com/myapp-server:1.0.0 \
  --build-arg VERSION=1.0.0 --build-arg REVISION="$(git rev-parse HEAD)" .
```

## Config and secrets

Keep structure in a ConfigMap and anything sensitive in a Secret. The layering is `defaults < .env < .env.local < process env`, so the Secret only needs the values that differ per environment, and secrets never go in the ConfigMap or the image. The `.env` file is committed (example) or injected via CI.

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: myapp-config
data:
            periodSeconds: 2
            failureThreshold: 30
          livenessProbe:
            httpGet: {path: /healthz, port: http}
            periodSeconds: 10
          readinessProbe:
            httpGet: {path: /readyz, port: http}
            periodSeconds: 5
            timeoutSeconds: 4
          lifecycle:
            preStop:
              sleep: {seconds: 5}
          resources:
            requests: {cpu: 100m, memory: 128Mi}
            limits: {memory: 256Mi}
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities: {drop: ["ALL"]}
      volumes:
        - name: config
          configMap: {name: myapp-config}
```

Why these settings:

- **Probes.** `/healthz` always returns 200 with no dependency checks, so it is the liveness and startup probe: a down database must not restart healthy pods. `/readyz` pings the database (3 second timeout) and the gRPC health state and returns 503 on failure, so it gates traffic. The `readinessProbe` timeout is above that 3 seconds.
- **Graceful shutdown.** On SIGTERM the generated server flips gRPC health to `NOT_SERVING`, calls `GracefulStop()`, shuts the HTTP server down, and closes the container, all within a 10 second grace. Keep `terminationGracePeriodSeconds` above that plus any `preStop` delay, so the kubelet does not SIGKILL a server that is still draining.
- **Endpoint removal races.** Kubernetes removes a pod from Service endpoints at the same time it starts termination, and some proxies keep sending for a moment. A short `preStop` delay lets the endpoint removal propagate before SIGTERM. The reference image is distroless and has no shell, so a `preStop` `exec` of `sleep` is not available; the example uses the native `sleep` lifecycle handler (Kubernetes 1.30 and later). On older clusters, add a tiny static sleep binary to the image. Count the delay inside the 30 seconds.
- **Read-only root filesystem.** The server needs no writable path unless you use the `local` storage adapter or a sqlite file; mount an `emptyDir` or a volume there.
- **Config reload.** Log level and the rate limiter are swapped on SIGHUP; everything else needs a rollout. A rolling restart (`kubectl rollout restart deployment/myapp-server`) is the supported way to change anything else.

Do not use a `memory` cache, queue, session, lock, or rate-limit adapter with more than one replica: each pod holds its own state. Point them at Redis or your database. See [Choosing batteries](https://zenta-dev.github.io/zever/guides/extend/choosing-batteries/).

## Service, PodDisruptionBudget, and autoscaling

```yaml
apiVersion: v1
kind: Service
metadata:
  name: myapp-server
spec:
  selector: {app: myapp, component: server}
  ports:
    - {name: http, port: 80, targetPort: http}
    - {name: grpc, port: 9090, targetPort: grpc}
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: myapp-server
spec:
  minAvailable: 2
  selector:
    matchLabels: {app: myapp, component: server}
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-server
spec:
  scaleTargetRef: {apiVersion: apps/v1, kind: Deployment, name: myapp-server}
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: {type: Utilization, averageUtilization: 70}
```

The PodDisruptionBudget keeps two pods serving through node drains; size `minAvailable` below `minReplicas`. CPU is a reasonable first HPA signal for the server.

## The worker Deployment

The worker runs the job worker and the scheduler in one process over one shared queue, and also drains the outbox relay. Give it its own Deployment and image, and no Service or HTTP probes.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-worker
spec:
  replicas: 2
  selector:
    matchLabels: {app: myapp, component: worker}
  template:
    metadata:
      labels: {app: myapp, component: worker}
    spec:
      terminationGracePeriodSeconds: 30
      securityContext:
        runAsNonRoot: true
      containers:
        - name: worker
          image: registry.example.com/myapp-worker:1.0.0
          envFrom:
            - secretRef: {name: myapp-secrets}
          volumeMounts:
          resources:
            requests: {cpu: 100m, memory: 128Mi}
            limits: {memory: 256Mi}
      volumes:
        - name: config
          configMap: {name: myapp-config}
```

Run more than one worker only with a shared queue adapter (`redis` or `db`). With more than one scheduler replica, wire a locker so a cron slot fires once; otherwise run the scheduler in a single replica. Scale workers on queue depth (`queue.Length`) with an autoscaler such as KEDA rather than on CPU. See [Scheduler and jobs](https://zenta-dev.github.io/zever/guides/background/scheduler-jobs/) and the [background jobs overview](https://zenta-dev.github.io/zever/guides/background/overview/).

## Migrations and one-off tasks

Run `zever db migrate` as a Kubernetes Job (or an init step in your pipeline) before rolling out a schema change, not inside every pod. Use a distinct Job per release so a failed migration blocks the rollout instead of racing replicas.

## Profiling and debugging

The generated server can expose pprof on a loopback-only listener (`--pprof-addr`). In a cluster, set `PPROF_ADDR=127.0.0.1:6060` on the pod and reach it with a port-forward rather than a Service (the forward connects inside the pod's network namespace, so the loopback-only bind is reachable):

```sh
kubectl port-forward deploy/myapp-server 6060:6060
go tool pprof http://127.0.0.1:6060/debug/pprof/heap
```

See [Profiling with pprof](https://zenta-dev.github.io/zever/guides/operate/observability/#profiling-with-pprof). Never run `zever tinker` against production.

- [Deployment](https://zenta-dev.github.io/zever/guides/operate/deployment/): Build, config, secrets, shutdown, and health.
- [Observability](https://zenta-dev.github.io/zever/guides/operate/observability/): OTLP export and pprof.
- [FAQ and troubleshooting](https://zenta-dev.github.io/zever/guides/operate/faq-troubleshooting/): Common problems.