Building a Cloud Native AI Platform Under $15 a Month

Why the API is small

This is a platform-engineering lab, not a chatbot launch. I wanted Terraform, Kubernetes, GitOps, CI, and observability I could operate myself. The AI stays small on purpose so the infra stays the focus.

Cap came first: $15 USD. I destroy the box when I am not using it. Managed Kubernetes and a GPU node are fine if someone else is paying. I wanted a lab I would still have in three months.

What shipped is a FastAPI gateway with the probe and metrics hooks Kubernetes needs, plus POST /v1/summarize. One function, summarize(), sits behind that POST. Tests and CI use a stub that keeps the first sentence. Real calls go to DeepSeek over an OpenAI-compatible HTTP client when I flip the mode. Routes stay stable if the backend changes. A worker, a scheduler, and Qdrant can wait. They are not in the repo yet.

Kubernetes does not care how clever the prompt is. It cares whether the process is up, and whether Postgres and Redis are reachable. Those are different questions. /health stays cheap and returns 200 while the process is alive. /ready dials both stores and returns 503 if either is missing. Liveness restarts a wedged process. Readiness keeps traffic off a pod that is running but useless.

Local first

The laptop version cost $0. Compose first: API on 8000, Postgres, Redis, a Makefile for up / test / down. Env-based config, no secrets in the image. Pytest stubs the LLM with FastAPI dependency overrides, so CI never needs a DeepSeek key. The image is multi-stage: builder venv, slim runtime, non-root uid 10001. Smaller pull, less to attack, same Python 3.12 as CI.

Then I mapped Compose onto Kubernetes by hand under the ai-platform namespace. Services become Deployments plus Services. Env files become a ConfigMap and a Secret. depends_on becomes readiness probes. I wanted to see those objects before wrapping them in a chart I did not understand yet.

In-cluster DNS is not localhost. The API pod's localhost is itself. Database and Redis URLs have to use the Service name. I knew that and still typed localhost once. Ready failed while the pods looked fine.

Image pull is the other trap. Pull policy Never plus Docker Desktop's kind cluster means the image exists on your machine and not on the node. The standalone kind CLI was empty. I switched the local cluster to Docker Desktop kubeadm so a local build tagged cloud-native-ai-api:local is visible without an import step. The tag has to match the manifest exactly. Compose uses a different image name, so that build does not help the Deployment.

Do not run Compose and the local cluster at the same time. I did. Port 8000 will lie to you. After code changes: rebuild the image, rollout restart the API deploy, then port-forward the Service.

Helm, then stop owning Postgres

Raw manifests taught the objects. Helm is what I operate. I own the API chart and consume Bitnami for Postgres and Redis. Rewriting a Postgres chart teaches little and burns a week.

Local values are the laptop knob file: never-pull, stub LLM, tiny resources, Bitnami Service DNS. Cloud overrides live in a separate hobby values file. Service names change when you switch charts. Bitnami release postgres becomes postgres-postgresql. Redis standalone becomes redis-master. Wrong host, same symptom as localhost: pods look healthy, ready is 503.

I proved upgrade and rollback. Bumping the image tag moved the pod. Rollback restored the previous tag and wrote a new history revision. Rollback is a new revision, not a rewind of the list. Rendering the chart before install caught a path typo (help instead of helm). That is a better failure than a CrashLoop.

Do not run the raw Kustomize manifests next to Helm in the same namespace. Two Deployments fight for the same mental model. Pick one path. Locally that path is Bitnami plus the API chart. On the VPS it is Argo plus hobby values. Postgres and Redis on the hobby cluster are still installed with Helm from the laptop. Only the API is GitOps'd. That gap is on the list.

CI publishes images, GitOps deploys

The workflow splits on purpose. Tests fail cheap on compile and pytest before Buildx spends time. The image job waits on green tests, then builds with layer cache in GitHub Actions. Helm lint and template run in parallel as an offline chart check, no cluster. Pull requests still build the Dockerfile without pushing, so a broken image fails before merge. Unmerged code never becomes latest. The registry is a merge gate, not a PR artifact dump.

Every build gets an immutable sha tag. latest exists only on default-branch pushes. Cloud deploys pin the SHA. Same-account GHCR push is the default GitHub token plus packages:write on the build job. No PAT for publish. A PAT with read:packages is a different secret, used later so the cluster can pull.

CI stops at the registry. Actions never run kubectl against the VPS. The hobby values file points at GHCR, IfNotPresent, and a docker-registry pull secret. Argo applies that.

image:
  repository: ghcr.io/notsubash/cloud-native-ai-api
  tag: sha-701186a
  pullPolicy: IfNotPresent

imagePullSecrets:
  - name: ghcr-pull

One Hetzner box

Terraform lives under infrastructure/terraform with reusable server, firewall, and DNS modules and a thin hobby environment root. I wrote the modules first and delayed apply until I needed the box. Scaffolding is free. Billing starts when the CX23 exists.

Child modules need their own provider source. Root required_providers is not enough. Without it, Terraform looks for hashicorp/hcloud and init fails even after the right plugins are installed. HCL will also tell you "missing argument" when you typed a hyphen instead of an equals. The error is not a syntax tip.

Cloud-init installs k3s. The firewall allows SSH and kubectl only from my home IP. Cloudflare stayed out of the apply path. An empty Cloudflare provider still wants a token and can pull a breaking provider major. No public DNS yet, so no Cloudflare in the graph.

Kubeconfig was a small comedy. Copy it from the laptop with scp, then replace 127.0.0.1 with the public IP. Running scp while already SSH'd into the VPS targets the server itself and fails. I wrote a fetch script so I would stop repeating that. Café Wi-Fi looks like an SSH hang. It is the firewall. Update the admin CIDRs, apply, then debug k3s.

Argo CD watches Git

Desired state for the API is an Argo Application pointing at helm/api with hobby values, destination namespace ai-platform. I do not helm upgrade the cloud API from the laptop. k3s already ships Traefik. I am not using it yet. Everything still comes in through kubectl port-forward from the hobby kubeconfig.

Install Argo with server-side apply. Client-side apply blows the last-applied annotation size limit on large CRDs. GitOps only sees GitHub. A local branch and an uncommitted values file produce "unable to resolve revision" or a missing file. Commit, push, then Sync.

I kept sync manual. Applying the Application CR registers desired state. Pods appear after Sync. Automate prune and selfHeal later, once the loop is boring.

The pull secret's password has to be a GitHub PAT with read:packages. I once used an image tag as the password. Argo said Healthy. The pod sat in ImagePullBackOff. Those two statuses can both be true.

Observability that fits on 4 GB

This is the ops feedback loop, and I spent the most time here. The stack is kube-prometheus-stack, Loki, Promtail. Single replicas, 3-day retention, 30-second scrape. Tempo can wait. If the node OOMs, monitoring loses, not the API.

Install order matters. I pointed Argo at a ServiceMonitor before the Prometheus Operator CRDs existed. SyncFailed: could not find monitoring.coreos.com/ServiceMonitor. Consumer before provider. Helm install the monitoring stack, then Sync the API.

Middleware records RED for every request except /metrics itself: count, latency histogram, status. Every log line carries a request_id, taken from the inbound header or generated. That id is the join key between Grafana and Loki. The useful walk is: force a 500, watch the error rate, grep the id in Loki.

The drill: force a 500 and follow it

/debug/boom returns 500 on purpose. I would not ship that. For a lab it is the fastest way to practice the path you walk when something is broken.

kubectl -n ai-platform port-forward svc/api 8000:8000

curl -si -H "X-Request-ID: boom-demo" \
  http://127.0.0.1:8000/debug/boom

Then Grafana for the error rate, Loki for that request id, and after a couple of minutes of sustained 5xx, ApiHighErrorRate goes pending, then fires. The PromQL window is a 5-minute rate. The alert waits until that ratio stays above 5% for two minutes. The annotation tells you to check Grafana and Loki for the id.

Git having the middleware is not enough. The running GHCR tag has to include that commit. Symptoms of an old image: boom is 404, and /metrics only shows summarize_requests_total with no http_requests_total. Pin a newer SHA, Sync, restart if needed.

Loki with no Promtail stores nothing useful. A wrong Promtail client URL looks like "Grafana Explore is empty" until you check the monitoring Services and match the push endpoint.

Power off is not the off switch

The VPS is the only recurring bill. GHCR, GitHub Actions free tier, Argo, and k3s are $0. Powering off overnight is convenient and still bills the reserved server. If I am pausing more than a few days, I destroy. Git and GHCR are source of truth. Nothing unique should live only on the box.

After destroy, the Hetzner console has to show no cnai-hobby. I have left a server I thought I deleted. The console is the check, not terraform exiting 0 while you are in the wrong directory. Secrets stay out of git: the Hetzner token, tfvars, state, the GHCR PAT.

What I am finishing next

The thin API, Terraform, local Kubernetes, Helm, GHCR, the hobby VPS, Argo, and monitoring are in place. What is left is the unglamorous half: a public URL, encrypted secrets, a restore you have actually run.

Wrapping up

The summarizer is the excuse. What I wanted was a repo I can destroy and rebuild, with a bill I actually keep. That loop is real now.

If you are trying to learn this stack, cap the bill first. Otherwise you will still be paying for a node you forgot you left running.