Streaming: Kafka
FACE connects to Kafka (and to Redpanda, which speaks the same protocol) to
enumerate its topic catalogue and read declared structure from a
Confluent-compatible Schema Registry. FACE implements the handshake,
ApiVersions, Metadata and the SASL exchange natively. No Kafka client
library is used.
kafka extraction is not implemented: … rather than an empty result.Type spellings
kafka, redpanda, apachekafka (the “Apache Kafka” card).
Azure Event Hubs and Amazon MSK are not aliased, because they require SASL mechanisms this connector does not speak.
Settings (KafkaConnectionConfig)
| Setting | Meaning |
|---|---|
bootstrap_servers | One or more host:port values (default port 9092). Required, and there is no default broker. List several: Test succeeds as soon as any one of them answers. |
topics | Optional prefix filters over the catalogue. An empty list is normal and enumerates every topic the principal may see. Reading the catalogue moves no data. |
client_id | Sent in every request header. The default is runink-face-<connection id>. |
security_protocol | plaintext (default), ssl, sasl_plaintext or sasl_ssl |
sasl_mechanism | plain, the only mechanism spoken. scram-sha-256, scram-sha-512, oauthbearer and gssapi are refused by name and never downgraded to PLAIN. |
username | The SASL username (a setting) |
schema_registry_url | An http(s):// base URL. Optional: without it, topics are listed with no columns. The URL must not contain credentials. |
schema_registry_username | The registry’s HTTP-basic user |
insecure_skip_verify | Skips certificate verification for both the broker and the registry. This is an explicit opt-in. |
Credentials: password for SASL, and schema_registry_password for the registry
(a separate credential, because the registry authenticates separately).
Internal addresses are allowed by default. Set allow_internal_network=false to
refuse them.
Test
Test performs a real handshake against the bootstrap servers in order: dial, TLS
where configured, ApiVersions, and the SASL exchange where configured. It
returns success on the first broker that completes it. If none does, it returns
no bootstrap server answered: … with each broker’s error.
A successful Test does not establish:
- that the principal may list topics, because topic authorisation is checked per topic
- that the Schema Registry exists or answers, because Test does not contact it
- that any topic holds a record
Explore
Explore reads the topic catalogue (up to 5,000 topics) and, where a registry is
configured, each topic’s value subject (<topic>-value). No record body is
read.
| Topic | Columns |
|---|---|
| With a registered value schema | The schema’s fields, with the Avro type verbatim, nullability from the union, enum symbols as allowed values, and each doc as a description. This is declared structure. |
| Without one | None. FACE never infers fields from sampled messages. |
| Registry not asked, or unreadable | None. The schemas aspect shows asked=false or readable=false, so you can tell this case apart from “no schema”. |
Topic names ending in .by-<key>.v<n> (for example acme.orders.events.by-order_id.v1)
yield a partition key, marked with a name-match basis. It is never reported as
declared. Key subjects and offsets are not requested, and the aspects say so.
Sample and Profile are not available for Kafka.