Skip to content

Databricks

The test-estate safety rule: stand the test estate up only beside a test instance, never a production deployment. Its systems are throwaway test services with published credentials: change-me is every test credential. They are still encrypted and still ask for a login, because FACE refuses a connection that is not (see Connector security). Use a scratch machine or a scratch cluster.

Grade B · dataset: SQL (DDL and rows in the engine’s dialect) · type databricks

What it is

A Databricks workspace with Unity Catalog. In the test estate it holds reefer telemetry: a bronze raw table and a silver readings table in quillmere.telemetry. The connector reference is SQL warehouses and databases.

Grade

B. The download holds the model. The platform itself needs a vendor account or trial. You need a Databricks workspace.

See the grade table for all 32 types.

Settings and credentials

From the connection catalogue. Settings are ordinary fields on the connection record. Credentials are kept in a sealed bundle and never returned.

Names
Typedatabricks
Settingshost, http_path, cluster_policy_id, node_type_id, spark_version, runner_artifact, runner_artifact_sha256, runner_scope, service_principal
Credentialspersonal_access_token

The test connection

Quillmere lakehouse (Databricks)

SettingTest value
hostquillmere-dbc.lakehouse.invalid
http_path/sql/1.0/warehouses/demo

Credentials: personal_access_token. In the test estate, every credential is change-me. Credentials are write-only: they go into the sealed credential bundle and are never returned.

Stand up the test system

The safety rule at the top of this page comes first: a scratch machine or a scratch cluster, never a production or platform one.

  1. Download databricks.zip and unzip it. It holds databricks/01_schema.sql, databricks/02_data.sql and databricks/expected.json, in the engine’s own dialect.
  2. Run 01_schema.sql, then 02_data.sql, in the SQL editor of a Unity Catalog workspace. CREATE CATALOG needs the metastore privilege to create catalogs.
  3. Add the connection with the settings above, pointed at your own workspace.

What FACE finds

Against a real workspace, FACE reads Unity Catalog’s information_schema. There is no local engine, so the expected model is what that read returns for this dataset.

DatasetKindDomainPrimary keyRowsFields
telemetry.bronze_reefer_rawtablecold_chainnone963
telemetry.silver_reefer_readingstablecold_chainreading_id968

expected.json in the download has the full model: every field with its logical type, the source’s native type and nullability.

How it joins the Quillmere universe

This source carries 3 of the universe’s shared keys. The same ID means the same thing in every source: a shipment here is the same shipment in the database, the event bus, the GPS track and the claim PDF.

Universe entityFields hereAlso in
devicebronze_reefer_raw.device_id, silver_reefer_readings.device_idbigquery, cctv, files, gps, iot, kafka, mqtt, postgres, rfid, servicenow
shipmentsilver_reefer_readings.shipment_idbigquery, d365, document, excel, files, github, guidewire, iot, kafka, logisticsapi, mqtt, mysql, objectstore, postgres, redshift, salesforce, sap, servicenow, sharepoint, tms, whs, yms
trailersilver_reefer_readings.trailer_idcctv, document, excel, iot, kafka, mqtt, postgres, servicenow, tms, yms

Evidence it holds for the planted anomalies:

  • ANOM-3, A reefer temperature excursion. databricks/02_data.sql in databricks.zip, look for 'SHP-50004', TIMESTAMP (temp_c up to 11.40).

See all connectors at once for the walkthrough across every source.

Download

This is one of 25 zips: the 7 streaming types ship as replay recordings instead.

What is in the zip
  • databricks/01_schema.sql
  • databricks/02_data.sql
  • databricks/expected.json

The whole universe is universe.json. manifest.json lists every download with its size and SHA-256, and SHA256SUMS is the same list for sha256sum -c.