Skip to content

Runners

A runner is the process that executes a fetch. It dials your sources, runs the queries and reads the telemetry. The FACE control plane orchestrates, stores connections and serves the cockpit, but it does not run fetches itself. Each fetch names a runner (runner_id), and a fetch with no runner is refused with no runner is selected.

Kinds of runner

KindWhere it runsHow the control plane reaches it
Managed runner (“Runink Managed Runner”)A dedicated FACE process (SERVICE_ROLE=runner) on separate compute next to the control planeThe control plane dials it over mutually authenticated TLS with the platform’s rotating mesh identity. There is no plaintext fallback.
Registered runnerAny runner you add with an endpoint in the cockpitThe control plane dials that endpoint. https:// endpoints use public TLS on port 443. All others use the mesh identity.
Self-hosted (enrolled) runnerYour machine, inside your network, behind NAT if neededThe runner dials out to FACE and holds one long-lived stream. Nothing dials in to it.

On the managed path, several runners can register themselves with the platform’s replicated registry by heartbeat. A tenant’s fetches stick to one live runner so its cache stays warm, and a newly added runner starts receiving work with no configuration change.

Why data can stay in your network

A self-hosted runner runs the whole fetch pipeline in-process on your machine. It dials your warehouses, APIs, readers and cameras from inside your network, and only the fetch result travels back over its outbound stream. Your sources never need an inbound firewall rule, a public address, or reachability from the FACE platform. Because the runner makes the connection, the two ends do not need to share an address family either.

Enrolling a self-hosted runner

Enrolment uses RunnerEnrollmentService.Enroll, one bidirectional stream:

  1. Get a credential. An organisation admin mints a service-account key for their own tenant (IdentityService.GenerateServiceAccountKey). Only a service-account credential can enrol a runner. A person’s session, even an admin’s, is refused with runner enrolment requires a service-account credential; a user session token cannot enrol a runner.

  2. Configure the runner. It needs these environment variables:

    VariableMeaning
    RUNNER_ENROLL_ADDRThe FACE gRPC address as the runner can reach it. Setting it turns enrolment on.
    RUNNER_ENROLL_TOKENThe service-account token. It is required whenever the address is set, and the runner refuses to start without it.
    RUNNER_IDA stable id kept across restarts, so a reconnect reuses the same slot. Up to 64 characters from A–Z a–z 0–9 - _ .. It is generated when empty.
    RUNNER_NAME, RUNNER_VERSIONDisplay only
    RUNNER_ENROLL_TLSTLS with system roots is on by default. false turns it off.
  3. Say hello. The first message must be RunnerHello with runner_id, name, version, capabilities (for example fetch) and load_score. Any other first message is refused.

  4. Get accepted. FACE answers with EnrollmentAccepted:

    • runner_id: the tenant-namespaced id.
    • tenant_id: taken from the verified token, never from the hello, so a runner cannot enrol into a tenant it holds no credential for.
    • heartbeat_interval_seconds (15).
  5. Do the work. FACE pushes RunnerWorkAssignment messages, each with a dispatch_id and a RunFetchRequest. The runner answers with a RunnerWorkResult carrying the same dispatch_id and either a RunFetchResponse or an error. A result for an unknown dispatch is dropped.

  6. Heartbeat. RunnerHeartbeat messages update load_score.

  7. Disconnect. Closing the stream de-registers the runner. A runner that re-enrols with the same id replaces its previous stream. If the stream ends, the runner reconnects after 5 seconds.

A fetch for a tenant that has an enrolled runner is dispatched down that runner’s stream. The control plane waits up to 10 minutes for the result. If none arrives, it reports runner "…" accepted the work and has not answered within 10m0s — the run may still be in progress there. It does not retry the work on control-plane compute.

Enrolled runners are listed only to their own tenant. A runner’s name, version and capabilities are its own statements, and FACE does not verify them.

Credentials on a runner

For the managed runner, the control plane reads a connection’s credential bundle once for each dispatch (an audited read). It then seals the bundle to the runner’s in-memory key under that dispatch’s context. The runner:

  • opens the bundle only for a verified mesh peer
  • uses it for that fetch
  • never writes it to disk or logs it

A restarted runner has a new key, so earlier sealed blobs cannot be opened. See /docs/security/.

Health

  • List. ListRunners always lists the managed runner. Its state is RUNNING only when a live heartbeat is in the replicated registry. Otherwise it is UNKNOWN, with a description that says liveness was not observed. compute_units is 0 when capacity is not measured.
  • Process health. GetRunnerHealth reports the answering process’s CPU, memory and goroutine count. Any metric it does not measure is named in unmeasured instead of being reported as zero.

Keeping fetches off the control plane

Set REQUIRE_RUNNER_ISOLATION=true to make a SQL fetch fail rather than fall back to the control plane when the runner cannot be reached. The failure reads …: runner isolation enforced but runner unreachable — refusing control-plane execution for …. The refusal appears on the fetch response as a notice as well as an error.

Warehouse runner settings

Snowflake and Databricks connections can carry runner-deploy settings, such as a compute pool, image, network rule, cluster policy, node type or artifact checksum (see SQL warehouses). FACE stores and format-checks them when the connection is saved:

  • A value that looks like a credential is refused.
  • runner_artifact must not contain a .. segment.
  • runner_artifact and runner_artifact_sha256 must be given together.

FACE itself does not yet deploy runners into Snowflake or Databricks. The Runners page lists those targets as not yet available.