> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prbe.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Self-hosting

> Install Probe on your own Kubernetes cluster — the data-residency topology, including air-gapped.

The experiment-tracking core — API plus Postgres, with metrics and artifacts on your infrastructure — from the same Helm chart the managed service runs. **No dashboard, no vendor callback.**

The software runs entirely on your infra and never contacts us. Ordinary external egress — image pulls, an approved LLM provider — remains your policy choice, not ours.

## Requirements

| Thing | Value |
| - | - |
| Kubernetes | 1.28 or newer |
| CNPG operator | **1.26 or newer** — older silently drops the kb extensions |
| Namespace | `research`, pre-created |
| API image | `ghcr.io/prbe-ai/research-os-api` (private; multi-arch) |
| Postgres image | `ghcr.io/prbe-ai/prbe-postgres` (public GHCR) |
| Object storage | Any S3-compatible endpoint (R2, MinIO, AWS S3) plus one bucket |
| Health | `GET /healthz` (readiness, includes DB and schema) · `GET /livez` (process) |
| `environment` | `local` / `dev` / `staging` / `main` — use `main`, **never `local`** |

The default install runs with the **knowledge engine off**, which is the simplest complete topology. Turning it on is a `helm upgrade` with one extra values file.

> [!NOTE]
>
> Every step below is exercised nightly in CI against the same flow. This is the human transcription of a tested path, not a description of one.

## What engine-off means

* `/v1/search` returns `state: "partial"` with SQL-exact results only. The exact channel matches projects, experiments and artifacts — not run names.
* Project and experiment [summaries](/tracking/summaries.md) are written by whoever records the work, never by the server: a fresh install is set up that way. A coding agent that records itself keeps them with `probe summary push`. The Probe daemon needs the LLM gateway (`litellm`), which this topology turns off.
* No dashboard ships. Clients are machines using service-minted bearer tokens, so **no SSO or OIDC integration is required** — those configuration keys can be placeholders.

## Install

**1. Namespace and configuration, first**

Create the namespace, then the chart's configuration Secret holding your object-store endpoint, bucket, and the two random values the app signs with.

Keep `createNamespace: false` in your values. The ordering is deliberate: it removes the install race, because the API pod mounts that Secret at boot — and it means `helm uninstall` can never delete it, since the namespace is not chart-owned.

**2. Values**

```yaml theme={null}
environment: main
createNamespace: false

engine:
  enabled: false            # REQUIRED — the chart default is true
litellm:
  enabled: false
analytics:
  enabled: false            # no product-analytics egress

image:
  repository: ghcr.io/prbe-ai/research-os-api
  tag: "<pinned app version>"     # pin; never float

postgres:
  storageClass: <your-retain-class>   # pgdata is the system of record
  storageSize: 20Gi

ingress:
  enabled: false
```

Start from the continuously-tested overlay in `tests/selfhost/` and adapt the stand-ins.

**3. Install, without --wait**

```sh theme={null}
helm install research-os charts/research-os -n research -f your-values.yaml
kubectl -n research rollout status deployment/research-os --timeout=300s
```

> [!WARNING]
>
> **Do not pass `--wait` on first install.** The schema-migrate Job is a *post-install* hook while the API's readiness probe requires the schema — `--wait` waits for readiness before running hooks and deadlocks until timeout.
>
> Plain `helm install` blocks only on the migrate hook; the `rollout status` poll covers API readiness. Upgrades are immune, because migrate is a *pre-upgrade* hook there.

Healthy looks like: migrate Job `Completed` (control and experiment applied, kb intentionally skipped), the `research-pg` CNPG cluster healthy, the API pod Ready, the upload-reaper CronJob registered, and `/healthz` returning 200 with all three databases green.

**4. Seed the first team and credential**

A fresh install has no users. Seed one team, one owner and one API credential by SQL in the CNPG primary; the app auto-creates personal workspaces on demand, so seed nothing else.

1. **experiment** DB: the `customers` mirror row — first, matching the app's own team-create ordering.
2. **control** DB: `customers`, `users`, `team_members` (role `owner`), and an `api_tokens` row holding **only the SHA-256 digest** of the credential string.

Generate the random credential yourself and hand the plaintext to the client. It is never stored. `tests/selfhost/seed.py` is the executable reference.

> [!WARNING]
>
> Run SQL with `kubectl exec -i`. **Without `-i`, `psql` silently no-ops** — it exits 0 having done nothing.

Verify against `/v1/me`, which is the machine endpoint. `/auth/me` is the dashboard's session-only one.

**5. Verify**

The smoke suite's gates, by hand:

* `/healthz` returns 200 with three databases green
* `/v1/me` returns the seeded identity
* create project, then experiment (a question is required), then run
* post metrics and read them back
* upload an artifact: presign, PUT, confirm
* `POST /v1/search` returns `state: "partial"` with an exact hit — the expected engine-off shape

## TLS

| Posture | Configuration |
| - | - |
| No ingress | `ingress.enabled: false` — reach the API by port-forward or your own LB or mesh. What the smoke suite runs |
| Your own certificate | `ingress.enabled: true`, `clusterIssuer: ""`, and create the TLS secret yourself |
| cert-manager | Keep a `clusterIssuer` pointing at **your** issuer |

Let's Encrypt HTTP-01 works only if the API host is publicly resolvable, which is usually not true for a residency install.

## Turning the engine on

Semantic search is a `helm upgrade` with one extra values file layered on top of your engine-off values. The API rolls and the engine's `kb-migrate` runs as a pre-upgrade hook.

Afterwards `/v1/search` is no longer `"partial"`: a semantic channel is populated alongside the SQL exact one.

## Backups and recovery

Continuous WAL archiving to object storage via Barman gives point-in-time recovery with an RPO of five minutes or better. It is a values flip, with its credentials Secret named after the cluster.

> [!WARNING]
>
> Use a **Retain** reclaim storage class. `pgdata` is the system of record, and a Delete-class volume removed with the release takes it.

## Upgrades

Pin the image tag; never float it. Migrations run as a pre-upgrade hook, so `--wait` is safe on upgrade.

> [!WARNING]
>
> A pinned engine image pins the **kb schema** with it — `kb-migrate` runs alembic from that image. Bumping the app without the engine leaves kb migrations unapplied.

## Air-gapped

* Record artifacts as **references** so bytes stay in your own bucket.
* Sign in with `probe wizard --action login --base-url <url> --token <token>` (or `--endpoint-only`), or set `PROBE_TOKEN` and `PROBE_BASE_URL`.
* `probe wizard --action manual` prints the exact commands for setting agents up without a network-assisted install.

## Compliance

There is no vendor callback. Analytics egress is a values flag and off in this topology, so a compliance review can verify the claim by reading the chart rather than trusting a statement about it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.