Deployment model
A FACE instance has three workloads. The operators in CORE build most of them. The image is the same for the control plane and the runners. Only the role differs.
| Workload | Role | Built by | Container port |
|---|---|---|---|
| Control plane | SERVICE_ROLE=server. It serves the cockpit’s gRPC-web and gRPC API, holds sessions and metadata, runs schedules and self-healing, and forwards every fetch to a runner. It never runs a fetch itself. | runink operator, from a ControlPlane resource | 7100 |
| Fetch runner | SERVICE_ROLE=runner. It runs the fetch pipeline (connectors, derivations, agents) on dedicated compute. | runink operator, from a FaceInstance resource, or a static manifest (see below) | 7102 |
| Frontend sleeve | It serves the Flutter web bundle with a single-page-app fallback and reverse-proxies backend calls to the control plane. | Static manifest in CORE’s on-host bundle | 8080 in the image |
TLS for browsers ends at the platform edge in front of these workloads. FACE itself is not configured with a certificate for the browser.
The custom resources
The runink operator (in CORE, grpc/operators/runink) owns two kinds in the API group
runink.org/v1alpha1.
ControlPlane (short name cp) describes the control plane.
| Field | Effect |
|---|---|
image | The FACE backend image. |
replicas | The pod count. The Deployment uses the Recreate strategy (see Upgrades). |
raftNodeId | This control plane’s consensus node id, by convention 1. |
tenant | Becomes FACE_TENANT. When it is empty, the operator uses the one tenant that all of this control plane’s FaceInstances name. If they name more than one, FACE_TENANT is left unset and the operator logs that. |
objectStoreEndpoint | The object-store Service, as host:port with no scheme. It is used for OBJECTSTORE_ENDPOINT and for Litestream. The default is face-objectd:9000. |
embeddingUrl | Becomes EMBEDDING_URL. |
managedRunnerEndpoint | Becomes MANAGED_RUNNER_ENDPOINT: the host:port the control plane forwards fetches to. |
scheduleExecuteFetch | Sets SCHEDULE_EXECUTE_FETCH=true. |
ambassador | Adds the optional mTLS ambassador sidecar. |
probes | Tunes the kubelet probes (see Health and troubleshooting). |
env | Extra environment variables. They are appended after the operator’s own list, so a literal here overrides the operator’s value of the same name. FACE_TENANT, RUNNER_ID and the default FACE_APPFS_DIR are appended after env, so you cannot override those three with env. |
nodeSelector | A hard pin to labelled nodes, on top of the operator’s default placement. |
FaceInstance (short name fi) describes one tenant’s fetch runner. The operator
names the runner Deployment face-runner-<instance>.
| Field | Effect |
|---|---|
tenant | The tenant this runner serves. It becomes FACE_TENANT. It must equal the control plane’s tenant, because the control plane reads the fetch results this runner writes under that tenant. |
image | The FACE backend image. |
controlPlaneRef | The ControlPlane whose mesh this runner joins. |
computeUnits | The tenant’s compute budget. The operator turns it into CPU and memory requests and limits and reports them in status.allocatedCpu and status.allocatedMemory. |
tier | managed (the default) or local. |
gpu | Requests GPU scheduling. |
embeddingUrl | Becomes EMBEDDING_URL on the runner. |
inference.model | The model hint for the runner. The other inference.* fields (gpuLayers, contextSize, parallel, threads, batch) are deprecated. They are still accepted so that existing resources validate, but the operator no longer writes them anywhere. |
suspend | Scales the runner to zero without deleting the resource. |
env, nodeSelector, probes, ambassador | As on ControlPlane. |
status.phase moves through Pending, Provisioning, Ready, Degraded and
Suspended.
A third kind, ClientInstance (core.runink.org/v1alpha1), is owned by CORE’s
core operator. It records an instance’s app, tenant, image, users and subscription.
FACE reads it in two ways:
- The cockpit’s instance picker lists
ClientInstances whosespec.appmatchesRUNINK_APP(defaultface). The pod’s ServiceAccount needslistonclientinstances.core.runink.orgfor this. - The operator mirrors a
ClientInstance’s users and subscription onto the pod asAUTH_SEED_USERS,SUBSCRIPTION_TIER,SUBSCRIPTION_STATUS,SUBSCRIPTION_SEATSandTENANT_ID.
ControlPlane or FaceInstance
resource instead, usually through its env list.What the operator puts in the control-plane pod
Containers
face-backendruns/app/server serve --port 7100.litestreamrunslitestream replicate -config /app/litestream.yml. It continuously replicates the SQLite metadata database.- The ambassador sidecar is added when
spec.ambassadorenables it.
Init containers, in this order:
- Two ownership init containers prepare the connections volume and the raft volume.
The connections directory is made owner-only (mode
0700). ensure-bucketruns/app/server ensure-bucket.litestream-restorerunslitestream restore -if-db-not-exists -if-replica-exists -config /app/litestream.yml /data/face.db.
Volumes
| Mount | Kind | Holds |
|---|---|---|
/data | emptyDir | face.db (SQLITE_PATH=/data/face.db). It is rebuilt from the Litestream replica on every pod start. |
/raft | node-local hostPath, one per member | Consensus state (RAFT_DATA_DIR). |
/connections | node-local hostPath, one per member | The legacy connection layout (FACE_CONNECTIONS_DIR). When no object store is configured, it also holds the encrypted appfs root at /connections/appfs. |
Environment. The operator sets the role, port, raft settings, inference and embedding
URLs, object-store and Litestream settings, and admission defaults
(FACE_ADMIT_MAX=1, FACE_ADMIT_QUEUE=32). It also references these Secrets:
| Secret | Keys | Required on the pod | Created by the operator |
|---|---|---|---|
face-session | AUTH_JWT_SECRET | yes | yes, if it is absent. It is never rotated on reconcile. |
core-envelope-kek | CORE_ENVELOPE_KEK | yes | yes, if it is absent. It is never overwritten. |
core-mesh-ca | MESH_CA_CERT, MESH_CA_KEY | yes | no. You distribute it with core mesh ca --namespace <ns>. |
<controlplane>-admin | email, password | yes | yes. It holds the break-glass admin as ADMIN_EMAIL and ADMIN_PASSWORD. |
objectd-key-face-app, objectd-key-face-litestream, objectd-key-face-vacuum | access-key, secret-key | yes, once they exist | by CORE’s provisioner. Until they exist, the pod keeps the shared objectstore-creds pair. |
face-oauth | google-client-id, allowed-emails | optional | no |
license | LICENSE, LICENSE_PUBLIC_KEY | optional | no. You mint it with CORE’s license tool. |
A required reference to a missing Secret stops the pod before FACE runs. Kubernetes
reports it as CreateContainerConfigError and names the Secret. That is deliberate: the
cause shows up in kubectl describe pod and does not have to be found in an
application log.
The runner pod is built the same way. The differences: it runs as
SERVICE_ROLE=runner on port 7102, and RUNNER_ID is the runner Deployment’s name,
so the runner’s own state survives restarts. It does not set RAFT_NODE_ID, because
runners are not consensus voters.
The static on-host manifests
CORE’s on-host bundle (grpc/infrastructure/apps/onhost/40-face.yaml, applied with
kubectl apply -k infrastructure/apps/onhost from CORE’s grpc/ directory) declares
two FACE workloads that no operator reconciles:
face-frontend, the sleeve.BACKEND_ADDRpoints it at the control-plane Service.face-runner, a static fetch runner (SERVICE_ROLE=runner, port7102). The backend’s defaultMANAGED_RUNNER_ENDPOINTisface-runner:7102, and it points at this runner.
The control plane is not in that file. The runink operator owns it, so edit the
ControlPlane resource and not the manifest.
The frontend sleeve
The sleeve (flutter/sleeve/) is a small static Go server in a distroless image. It
serves /web and forwards a request to BACKEND_ADDR when the request is one of these:
- a gRPC-web request;
- a request under
/auth/,/api/or/webrtc/; - a dotted gRPC service path.
The release web build bakes in no backend host. The cockpit takes its origin from the page it was loaded from, so one bundle works behind any hostname.
| Variable | Default | Purpose |
|---|---|---|
PORT | 8080 | The listen port. |
WEB_DIR | /web | The Flutter bundle directory. |
BACKEND_ADDR | the control-plane Service | Where backend calls are forwarded. |
The edge may split gRPC-web off to the control plane before a request reaches the sleeve. In that case the sleeve only serves the static bundle.