Skip to content
Sovereign inference

Sovereign inference

Every AI feature in PULSE runs on the Runink platform’s own inference server: mistral.rs’s mistralrs-server. PULSE sends no prompt, no customer data, and no generated text to any third-party model API.

One server for everything

All AI services share one inference server and one model server object inside PULSE, so they share one set of agent prompts and parameters (agents/pulse.ini and the templates in agents/templates/).

The backend finds the server one of two ways:

  • Remote. When INFERENCE_REMOTE_URL is set, PULSE uses the shared inference plane at that address and starts nothing itself.
  • Local. Otherwise it starts mistralrs-server as a child process.

Speech-to-text works the same way with its own variable, VOXTRAL_REMOTE_URL, and its own mistralrs-server serving Voxtral. Text-to-speech is Runink TIDE’s shared ttsd service, reached through TTSD_URL (unset: the in-cluster Service; empty: voice output off). PULSE embeds no speech engine.

Both servers start in the background. If a binary or model is missing, PULSE still starts, and AI calls fail with inference server is not running instead of blocking the whole app.

Which model

INFERENCE_MODEL names the model PULSE asks the server for. Set it to the general model your inference server loads — on Runink Server, a Qwen-family model under the Apache-2.0 licence. Unset, PULSE requests the same default as the shared inference library, so a locally managed child server and PULSE agree without configuration.

Every agent uses that one model. There is no second, larger tier and no per-agent model: an agent’s section in agents/pulse.ini sets its sampling parameters and its instructions, not its weights. See Model cards.

In a cluster the name is only a label — the weights actually served are pinned by the platform’s inference deployment, and PULSE is given no model name to override it.

Against a remote plane that takes the name at face value, INFERENCE_MODEL must match what that server loaded. PULSE does not check.

Is the model ready

Two signals report readiness, and both use the same test: the engine process is running, and a readiness check against the engine has confirmed it can serve a request.

  • GET /health returns model_ready.
  • ModelService/GetModelStatus returns ready. The app polls it on startup.

A process that has started but is still loading weights counts as not ready.

Time budgets

The model runs on CPU and one generation can take many minutes. PULSE gives each generation two clocks:

ClockDefaultOverride
Wait for the first token (queue plus prompt processing)5 minutesPULSE_FIRST_TOKEN_SECONDS
DecodeSized from the agent’s token budget and the platform decode rate, capped at 45 minutesPULSE_GENERATION_TIMEOUT_SECONDS

The decode rate comes from INFERENCE_DECODE_TOK_S, which TIDE injects. If a budget is too short to produce an answer that parses, PULSE refuses up front and sends nothing to the model. The error says so and names PULSE_GENERATION_TIMEOUT_SECONDS.

Sharing the slot fairly

Optional admission control queues or refuses work when the model is busy. It is off unless PULSE_ADMIT_MAX is above zero.

VariableMeaning
PULSE_ADMIT_MAXMaximum concurrent inference calls. 0 turns admission off.
PULSE_ADMIT_QUEUEMaximum waiters before backpressure. 0 is unbounded.
PULSE_ADMIT_MODEreject (default) or degrade.
PULSE_ADMIT_WEIGHTSScheduling weights keyed by the x-subscription-tier request metadata, for example on_premises:4,cloud:4,trial:1. A request without that metadata is scheduled as default.

When the pool is full in reject mode, the caller gets ResourceExhausted: the inference plane is at capacity and declined this request; nothing was generated — retry in a few minutes.

Repeat requests

An exact repeat of a content generation is served from a cache instead of running the model again. PULSE_DEDUP_TTL sets how long an entry lives, as a Go duration. The default is 1h.

Grounding

Generators ground their output in real sources before they write:

  • Web research. A headless Chrome scraper and extractor reads live pages and search results. The diagnosis reads your website this way.
  • Product facts. Plain markdown files shipped in the image (grpc/data/corpus), read by Direct Corpus Interaction. The topic’s terms are grepped across the corpus, the files that matched are read whole, and the passages carrying the most distinct terms go into the prompt labelled with the file they came from. There is no vector index and no similarity search anywhere in PULSE. Files whose name begins with _ are rules: they are injected verbatim into every long-form generation, matched or not, because a lookup by relevance would never return them.

When nothing matches, the generator is told so in as many words and writes from the brief alone, rather than filling the gap.

Output guards

Everything PULSE can publish is screened before it reaches you, and nothing is published without a person’s approval. Operators can widen screening in Configuration; screening buffers the whole response, so a screened answer arrives at once rather than token by token.