CCTV, surveillance & documents

Cameras, visual inspection and documents. CCTVService checks and describes camera sources and samples frames from them. SurveillanceService grades frames on the sovereign vision tier, detects objects locally at camera rate, keeps annotated evidence frames and streams events. MaterialAnalyticsService aggregates everything surveillance has seen. OCRService reads text, spreadsheet logic and source code out of documents.

The rule throughout is that an absence is reported as an absence. A frame nobody graded is not PRISTINE, a detector that did not score an object does not send 0.0, and every stage that might not run says what it did in a *_note field. Show those notes. Do not treat an empty list as “nothing there”.

Summary

Service	RPC	Kind	Purpose
CCTVService	TestConnection	Unary	Check that a camera or file source answers
CCTVService	ExtractKnowledge	Unary	Sample frames and run local detection
CCTVService	DescribeSource	Unary	Ask an RTSP camera to describe its streams
SurveillanceService	IngestSurveillanceFeed	Unary	Grade a frame or feed with the vision model
SurveillanceService	DetectLiveFrame	Unary	Local object detection on one frame, no model
SurveillanceService	GetEvidenceFrame	Unary	Fetch a retained annotated frame
SurveillanceService	TriggerSensorCueing	Unary	Broadcast a cross-sensor cue event
SurveillanceService	SubscribeTelemetry	Server streaming	Stream surveillance events
MaterialAnalyticsService	AnalyzeMaterialQuality	Unary	Aggregate sightings by material, class and sensor
OCRService	ExtractText	Unary	OCR text from documents and images
OCRService	ExtractExcelCode	Unary	Formulas, macros and VBA from a workbook
OCRService	ExtractCode	Unary	Structure of a source-code file

Camera credentials sent to these RPCs (access_key_id, secret_access_key) are used for the call only. The password is never echoed in a response.

CCTVService

Full name semantics.v1.CCTVService.

TestConnection

rpc TestConnection(CCTVServiceTestConnectionRequest) returns (CCTVServiceTestConnectionResponse);

Kind: Unary.
Auth: Bearer session.
Errors: UNAVAILABLE if endpoint_url is empty, or a file source does not exist or cannot be read. INVALID_ARGUMENT if an image file source does not decode.

For rtsp://, http:// and https:// endpoints, the server probes the camera when a reachability probe is available in the deployment. Otherwise it accepts the endpoint without contacting it. Read success and live_check_performed together:

`success`	`live_check_performed`	Meaning
`true`	`true`	The endpoint answered, or a file source exists and decodes. No video frame was decoded, so the stream itself is unverified.
`false`	`true`	The probe ran and the endpoint did not answer. `message` says how.
`true`	`false`	Nothing contacted the camera. This says nothing about reachability.

Request: CCTVServiceTestConnectionRequest

Field	Type	Description
`endpoint_url`	`string`	Required. Camera URL (`rtsp://`, `http(s)://`) or a file source.
`access_key_id`	`string`	Camera username.
`secret_access_key`	`string`	Camera password. Write-only.

Response: CCTVServiceTestConnectionResponse

Field	Type	Description
`success`	`bool`	See the table above.
`message`	`string`	What happened, safe to show verbatim.
`live_check_performed`	`bool`	Whether anything actually contacted the source. `false` does not mean failure. It means this response makes no claim about reachability.

ExtractKnowledge

rpc ExtractKnowledge(CCTVServiceExtractKnowledgeRequest) returns (CCTVServiceExtractKnowledgeResponse);

Kind: Unary.
Auth: Bearer session.
Errors: UNAVAILABLE for an empty endpoint or a local webcam (0, webcam), which the server cannot capture. Frame acquisition errors are returned as they occur.

Samples frames from the source and runs the local object detector on them. No vision model and no OCR is involved. The server tries the camera’s snapshot capability first when available, and falls back to decoding the stream. The frames come back as JPEG bytes, and text_summary reports the frame count, how the frames were obtained, and what the detector did and found.

Request: CCTVServiceExtractKnowledgeRequest

Field	Type	Description
`endpoint_url`	`string`	Required. Camera URL or file source.
`access_key_id`	`string`	Camera username.
`secret_access_key`	`string`	Camera password. Write-only.
`max_frames`	`int32`	Frames to sample. 0 or negative means 1. Capped by the deployment’s sweep limit, which is 2 by default and never more than 8.

Response: CCTVServiceExtractKnowledgeResponse

Field	Type	Description
`vector`	`repeated float`	Not populated by the server.
`text_summary`	`string`	Plain-text report: frames extracted, the frame source, the detector report and one line per detection.
`frames`	`repeated bytes`	The sampled frames, as JPEG. No OCR or vision model has read them.

Reserved: fields 1 and 4.

DescribeSource

rpc DescribeSource(CCTVServiceDescribeSourceRequest) returns (CCTVServiceDescribeSourceResponse);

Kind: Unary.
Auth: Bearer session.
Errors: INVALID_ARGUMENT if endpoint_url is empty. A device-capture camera pushes frames from the client, so the server has nothing to ask. UNAVAILABLE if the camera could not be asked.

Asks the camera what it is instead of reading back what an operator typed. For rtsp:// and rtsps:// URLs the server speaks RTSP (OPTIONS, then DESCRIBE) and returns the session description the camera stated: its streams, codecs and so on. http(s):// URLs and file paths get a supported: false description that names what would answer for them. This is an answer, not an error.

A full description does not show that the camera is publishing usable video, that a frame would decode, or that playback would be permitted. A recorder will describe a channel whose camera is unplugged. Only sampling a frame answers that. Show the description’s reason verbatim.

Request: CCTVServiceDescribeSourceRequest

Field	Type	Description
`endpoint_url`	`string`	Required. An `rtsp://` or `rtsps://` URL. Other forms are answered with `supported: false`.
`access_key_id`	`string`	Camera username. A setting, not a secret.
`secret_access_key`	`string`	Camera password, used to answer a Digest challenge. Never echoed, logged or written to lineage.

Response: CCTVServiceDescribeSourceResponse

Field	Type	Description
`description`	`SourceDescription`	What the camera said. Never unset on success. `supported: false` with a `reason` means the source publishes no catalogue, or nothing could ask it.

SurveillanceService

Full name semantics.v1.SurveillanceService.

Two paths answer different questions at very different costs:

IngestSurveillanceFeed asks the sovereign vision model what is in the frame, what it is made of and how damaged it is. It can take minutes on CPU inference. It writes a lineage record and a sighting, may retain an evidence frame, and emits a telemetry event.
DetectLiveFrame runs a local detector in the serving process and returns boxes in milliseconds. It grades nothing and writes nothing.

IngestSurveillanceFeed

rpc IngestSurveillanceFeed(IngestSurveillanceFeedRequest) returns (IngestSurveillanceFeedResponse);

Kind: Unary.
Auth: Bearer session.
Errors: INVALID_ARGUMENT if the request carries no usable input: no valid image_frame, no raw_ocr_payload, and no frames from feed_url. An image_frame that fails validation is reported as invalid image_frame.

Inputs can be combined. The server uses:

image_frame: one JPEG or PNG. It is validated from its header before decoding (at most 12 MiB and 40 megapixels) and downscaled for the vision tier.
raw_ocr_payload: text already read at the edge.
feed_url: frames sampled from a feed. max_frames has the same default and cap as ExtractKnowledge.

The vision model then grades the scene and names each inspectable object. People are never subjects. A frame holding only people, or nothing inspectable, is not assessed: the grade, classification, severity, remediation and damage percentage are all empty or zero, and that means “no finding”, not a low score. Per-object boxes come from a detection turn, region proposals or the local detector. objects[].box_origin says which.

Each ingest also:

records one sighting per object for MaterialAnalyticsService;
emits a DAMAGE_ASSESSMENT telemetry event, or DAMAGE_ASSESSMENT_NOT_PERFORMED when the frame was not assessed;
retains one annotated evidence frame, for the frame holding the worst object, only when the operator has configured a retention window. See evidence_note.

Request: IngestSurveillanceFeedRequest

Field	Type	Description
`sensor_id`	`string`	Which camera or sensor. Free text. Keys sightings and events.
`domain`	`string`	`LAND`, `AIR`, `MARITIME` or `CYBER`. With `AIR`, an aerial intruder is classified `UNAUTHORIZED_DRONE`.
`gps_coordinates`	`string`	Location, passed through to events.
`confidence_score`	`double`	The client’s own confidence. Recorded in event metadata. It does not affect the grade.
`raw_ocr_payload`	`string`	OCR text from the edge, added to the model’s input.
`image_frame`	`bytes`	One encoded JPEG or PNG frame.
`feed_url`	`string`	RTSP or HTTP video feed URL, or a file source.
`max_frames`	`int32`	Frames to sample from `feed_url`. 0 means 1. Capped at the deployment’s sweep limit (default 2, at most 8).

Response: IngestSurveillanceFeedResponse

Field	Type	Description
`threat_detected`	`bool`	Whether the grading found a threat.
`classification`	`string`	For example `NO_THREAT`, `INFRASTRUCTURE_DAMAGE` or `UNAUTHORIZED_DRONE`. Empty when not assessed.
`threat_severity`	`string`	`SEVERE`, `HIGH`, `MEDIUM` or `LOW`. Empty when not assessed.
`suggested_remediation_action`	`string`	Suggested action. Empty when not assessed.
`damage_grade`	`string`	Worst object’s grade: `PRISTINE`, `DAMAGED`, `DEFECTIVE` or `DESTROYED`. Empty when not assessed. Empty is not `PRISTINE`.
`ocr_text`	`string`	Readable text from the frames and the edge OCR payload. Never image data.
`frames_processed`	`int32`	Frames that reached the grading turn, which sees them together as one scene.
`damage_percentage`	`int32`	The vision model’s estimate for the worst object, 0 to 100. 0 when not assessed.
`subject_object`	`string`	The inspectable object assessed, such as a pallet or container door. Empty means none: every grade field is then a no-finding.
`subject_material`	`string`	What the subject is made of.
`damage_box_x`	`float`	Worst object’s box, as a 0 to 1 fraction of frame width. All four box fields zero means no localisable region.
`damage_box_y`	`float`	Box top, as a fraction of frame height.
`damage_box_w`	`float`	Box width fraction.
`damage_box_h`	`float`	Box height fraction.
`objects`	`repeated` `DetectedObject`	Every inspectable object, each with its own box. People are never listed.
`localisation_note`	`string`	Why there are no per-object boxes, when there are none. Empty when boxes are present or localisation did not apply.
`class_vocabulary`	`string`	The closed class set that `label_in_vocabulary` was graded against, for example `runink.coldchain.v1`. Empty only when there are no objects.
`frames_detected`	`int32`	Frames that each got their own detection turn. `DetectedObject.frame_index` indexes these. It can differ from `frames_processed`.
`evidence_frame`	`EvidenceFrameRef`	The retained annotated frame. Absent is the normal case, because retention is off unless the operator configures it.
`evidence_note`	`string`	Why no frame was retained, when none was.
`detection_note`	`string`	What the local detector did. Never empty. If no detector is set up in this deployment, the note says so; that is not a statement that the camera saw nothing.

DetectLiveFrame

rpc DetectLiveFrame(DetectLiveFrameRequest) returns (DetectLiveFrameResponse);

Kind: Unary.
Auth: Bearer session.
Errors: INVALID_ARGUMENT if image_frame is empty or fails the same validation as IngestSurveillanceFeed. An undecodable frame is never reported as “nothing detected”.

One frame in, the local detector’s boxes out, with no model and no network hop. Nothing is recorded: no lineage, no sighting, no evidence frame and no telemetry event. The frame is not downscaled. Every returned object has box_origin: BOX_ORIGIN_LOCAL_DETECTOR, a detector_label and a detector_label_set. Its label, material, damage_grade and object_class are empty, damage_percentage is 0 and label_in_vocabulary is absent. None of those is a finding, because nothing on this path grades.

Request: DetectLiveFrameRequest

Field	Type	Description
`sensor_id`	`string`	Which camera. Used for logging only.
`image_frame`	`bytes`	Required. One encoded JPEG or PNG frame, as captured.

Response: DetectLiveFrameResponse

Field	Type	Description
`objects`	`repeated` `DetectedObject`	One per detection, as described above.
`detection_note`	`string`	What the detection stage did. Never empty.
`detection_latency_micros`	`int64`	Wall-clock time of the detector call in the server, in microseconds. Excludes network and queueing. Use it to pace a capture loop, not as a frame rate.

GetEvidenceFrame

rpc GetEvidenceFrame(GetEvidenceFrameRequest) returns (GetEvidenceFrameResponse);

Kind: Unary.
Auth: Bearer session. The frame must belong to the caller’s tenant.
Errors: FAILED_PRECONDITION if the deployment has no evidence store, or the frame is past its retained_until. INVALID_ARGUMENT for a malformed handle. PERMISSION_DENIED if the handle belongs to another tenant. NOT_FOUND if the frame is not available. INTERNAL if the stored frame cannot be read.

Returns the annotated JPEG an ingest retained, with FACE’s boxes and labels drawn in. The raw frame is never kept. A retained frame shows a reviewer what was assessed. It is not forensically provenanced: nothing signs it or timestamps it against a trusted clock, and a drawn box is FACE’s own model’s answer, not proof.

retained_until is when FACE stops serving the frame. It is not a deletion. The stored object stays until the storage lifecycle policy that the operator configures removes it.

Request: GetEvidenceFrameRequest

Field	Type	Description
`handle`	`string`	`evidence_frame.handle` from an ingest response. Opaque: do not build one.

Response: GetEvidenceFrameResponse

Field	Type	Description
`annotated_frame`	`bytes`	The annotated JPEG.
`content_type`	`string`	Always `image/jpeg`.
`captured_at`	`google.protobuf.Timestamp`	When the source frames were ingested.
`retained_until`	`google.protobuf.Timestamp`	When FACE stops serving it.
`sensor_id`	`string`	The sensor it came from.
`boxes_drawn`	`int32`	Rectangles drawn on the image. 0 is a real answer.

TriggerSensorCueing

rpc TriggerSensorCueing(TriggerSensorCueingRequest) returns (TriggerSensorCueingResponse);

Kind: Unary.
Auth: Bearer session.
Errors: INVALID_ARGUMENT if source_sensor_id or target_sensor_type is empty, or the target type is not one of INFRARED_CAMERA, YARD_CCTV or PLC_CRANE_LOCK.

Broadcasts a SENSOR_CUE telemetry event with severity HIGH. No actuator is contacted: this service has no PTZ camera or PLC client. The status says a cue was requested.

Request: TriggerSensorCueingRequest

Field	Type	Description
`source_sensor_id`	`string`	Required. The sensor raising the cue.
`target_sensor_type`	`string`	Required. `INFRARED_CAMERA`, `YARD_CCTV` or `PLC_CRANE_LOCK`, case-insensitive.
`target_coordinates`	`string`	Where to cue. Passed to the event as `gps_coordinates`.
`alert_payload`	`string`	Free text, carried in the event’s metadata.

Response: TriggerSensorCueingResponse

Field	Type	Description
`cue_acknowledged`	`bool`	`true` once the event is broadcast.
`target_status`	`string`	`CAMERA_CUE_REQUESTED` for camera targets, or `CRANE_LOCK_REQUESTED` for `PLC_CRANE_LOCK`. The proto comment lists `CAMERA_CUE_ACTIVE` and `CRANE_LOCKED`, which the server does not return.

SubscribeTelemetry

rpc SubscribeTelemetry(SubscribeTelemetryRequest) returns (stream SubscribeTelemetryResponse);

Kind: Server streaming. Available over native gRPC and gRPC-web.
Auth: Bearer session.
Errors: None of its own. The stream stays open until the client cancels.

Streams events as IngestSurveillanceFeed and TriggerSensorCueing produce them. There is no replay, so only events after subscribing arrive. A second subscription with the same subscriber_id replaces the first. Use a unique id per client session.

Request: SubscribeTelemetryRequest

Field	Type	Description
`subscriber_id`	`string`	Unique client or session id. Empty gets a generated id.

Response: SubscribeTelemetryResponse

Field	Type	Description
`event`	`TelemetryEvent`	One event.

MaterialAnalyticsService

Full name semantics.v1.MaterialAnalyticsService.

AnalyzeMaterialQuality

rpc AnalyzeMaterialQuality(AnalyzeMaterialQualityRequest) returns (AnalyzeMaterialQualityResponse);

Kind: Unary.
Auth: Bearer session.
Errors: INVALID_ARGUMENT if exactly one of baseline_since and baseline_until is set. UNAVAILABLE if the sighting corpus or the baseline window cannot be read.

Exploratory analysis over the sightings recorded by IngestSurveillanceFeed: which materials, object classes and sensors were seen, and how their damage grades are distributed. A comparison is returned only when you supply a complete baseline window.

The unit is a sighting, not a thing. A sighting is one object in one frame of one ingest. Nothing tracks an object between frames, so a six-frame sweep of two pallets yields twelve sightings. totals.multi_frame_sweep_sightings tells you how much of a result is exposed to that.

Absent is not zero. A sighting the vision tier did not grade is counted in ungraded_sightings and nowhere else. Means and shares are optional, each ships with its denominator, and they are absent when there is nothing to average.

Read corpus_scanned first. false means no scan happened, and every count is meaningless rather than zero.

Request: AnalyzeMaterialQualityRequest

Field	Type	Description
`since`	`google.protobuf.Timestamp`	Window start. Unset means the corpus’s retention floor.
`until`	`google.protobuf.Timestamp`	Window end. Unset means now.
`sensor_id`	`string`	Exact-match filter. Empty means no filter.
`connection_id`	`string`	Exact-match filter.
`object_class`	`string`	Exact-match filter on the closed-vocabulary class, never a free label.
`material`	`string`	Exact-match filter.
`baseline_since`	`google.protobuf.Timestamp`	Baseline window start. Set it together with `baseline_until` to get a comparison.
`baseline_until`	`google.protobuf.Timestamp`	Baseline window end.

Response: AnalyzeMaterialQualityResponse

Field	Type	Description
`corpus_scanned`	`bool`	Whether a corpus was actually read.
`corpus_durable`	`bool`	Whether what was scanned survives a restart. `false` means only this process’s recent memory was read.
`corpus_note`	`string`	What was read and what it does not cover. Always set.
`window_since`	`google.protobuf.Timestamp`	Window actually folded, after clamping to retention.
`window_until`	`google.protobuf.Timestamp`	Window end actually used.
`totals`	`SightingTotals`	Totals over the window.
`by_material`	`repeated` `SightingGroup`	By material, `sighting_count` descending. No zero rows.
`by_object_class`	`repeated` `SightingGroup`	By closed-vocabulary class. Sightings never classified appear in no class group.
`by_sensor`	`repeated` `SightingGroup`	By sensor.
`comparison`	`WindowComparison`	Present only when a complete baseline window was sent.
`class_vocabularies`	`repeated string`	Every class vocabulary the sightings were graded against. More than one means class counts are not comparable across the window.

OCRService

Full name semantics.v1.OCRService.

All three RPCs take an OCRDataSource and read the file or files it resolves to. OCR runs locally in the deployment.

ExtractText

rpc ExtractText(ExtractTextRequest) returns (ExtractTextResponse);

Kind: Unary.
Auth: Bearer session.
Errors: INVALID_ARGUMENT if source is missing. UNAVAILABLE if the OCR engine is not enabled in this deployment. INTERNAL if the source cannot be resolved to files.

OCRs every resolved file, PDFs and images, and joins the text with --- Start File: <name> --- and --- End File: <name> --- markers. A file that fails is marked --- Error Processing <name> --- and skipped.

Request: ExtractTextRequest

Field	Type	Description
`source`	`OCRDataSource`	Required. What to read.

Response: ExtractTextResponse

Field	Type	Description
`text`	`string`	The combined text.
`confidence`	`optional float`	Mean word confidence reported by the OCR engine, weighted by words, over the files that stated one. Absent when no engine reported a confidence. Absent is not 0.
`metadata`	`map<string, string>`	`source_path`, `file_count`, `files_read`, `engine`, `method`, and `confidence_basis` when `confidence` is set.

ExtractExcelCode

rpc ExtractExcelCode(ExtractExcelCodeRequest) returns (ExtractExcelCodeResponse);

Kind: Unary.
Auth: Bearer session.
Errors: None as gRPC status. Failures return success: false with error_message.

Extracts formulas, macros, VBA modules and named ranges from a workbook.

Request: ExtractExcelCodeRequest

Field	Type	Description
`source`	`OCRDataSource`	The workbook to read.

Response: ExtractExcelCodeResponse

Field	Type	Description
`result`	`ExcelCodeResult`	What was extracted.
`success`	`bool`	Whether extraction succeeded.
`error_message`	`string`	Why it failed.

ExtractCode

rpc ExtractCode(ExtractCodeRequest) returns (ExtractCodeResponse);

Kind: Unary.
Auth: Bearer session.
Errors: None as gRPC status. Failures return success: false with error_message, for example Source is required, No files found at specified source, or a visual document that should go to ExtractText instead.

Reads the first resolved file. With a language_hint other than empty or auto, it is parsed as source code. Otherwise the type comes from the extension: images and documents are refused, workbooks are treated as spreadsheet code, and anything else is parsed as source code.

Request: ExtractCodeRequest

Field	Type	Description
`source`	`OCRDataSource`	Required. What to read.
`language_hint`	`string`	Language to parse as. Empty or `auto` detects from the file.

Response: ExtractCodeResponse

Field	Type	Description
`language`	`string`	Detected or hinted language.
`functions`	`repeated` `CodeFunction`	Functions found.
`classes`	`repeated` `CodeClass`	Classes found.
`imports`	`repeated string`	Imports found.
`raw_code`	`string`	The source text.
`metadata`	`map<string, string>`	Extraction metadata.
`success`	`bool`	Whether extraction succeeded.
`error_message`	`string`	Why it failed.

Messages

DetectedObject

One inspectable object located in a frame.

Field	Type	Description
`label`	`string`	What the vision model says it is, for example `pallet`. Empty on the live path.
`material`	`string`	What it is made of. Empty on the live path.
`damage_grade`	`string`	This object’s grade: `PRISTINE`, `DAMAGED` or `DEFECTIVE`. Empty means not graded.
`damage_percentage`	`int32`	This object’s estimated damage, 0 to 100.
`box_x`	`float`	Box left, 0 to 1 of frame width.
`box_y`	`float`	Box top, 0 to 1 of frame height.
`box_w`	`float`	Box width fraction.
`box_h`	`float`	Box height fraction.
`item_match`	`ItemMatch`	What this object is in your data. Unset when resolution was not requested, which is opt-in per deployment.
`label_in_vocabulary`	`optional bool`	Whether `label` is in `class_vocabulary`. Absent: the classifier was not asked, so show no mark. `true`: `object_class` carries the member. `false`: out of vocabulary, and `object_class` is empty.
`object_class`	`string`	The vocabulary member `label` resolved to. Countable. Set only when `label_in_vocabulary` is `true`.
`frame_index`	`int32`	Which sampled frame, 0-based. Meaningful only when `frames_detected` > 1. Objects are not tracked between frames.
`box_origin`	`BoxOrigin`	Which stage produced the box.
`detector_label`	`string`	The local detector’s class string, verbatim. Never copied into `label` and never mapped onto the class vocabulary. Empty when no local detector produced the object.
`detector_label_set`	`string`	The label set `detector_label` comes from, for example `coco.2017.80`.
`detector_confidence`	`optional float`	The detector’s score, 0 to 1, when it reported one. Absent means no score. A present 0.0 is a measured zero. It is not a probability that the label is right, and not a damage figure.

EvidenceFrameRef

Field	Type	Description
`handle`	`string`	Opaque handle for `GetEvidenceFrame`. Not a URL.
`captured_at`	`google.protobuf.Timestamp`	When the frames were ingested.
`retained_until`	`google.protobuf.Timestamp`	When FACE stops serving it. Not a deletion time.
`boxes_drawn`	`int32`	Rectangles drawn. 0 is a real answer.
`frame_index`	`int32`	Which sampled frame was drawn on, 0-based. One frame is retained per ingest.
`width`	`int32`	Image width in pixels.
`height`	`int32`	Image height in pixels.

TelemetryEvent

Field	Type	Description
`event_id`	`string`	Event id.
`timestamp`	`google.protobuf.Timestamp`	When it happened.
`event_type`	`string`	`DAMAGE_ASSESSMENT`, `DAMAGE_ASSESSMENT_NOT_PERFORMED` or `SENSOR_CUE`.
`severity`	`string`	`SEVERE`, `HIGH`, `MEDIUM` or `LOW`. Empty for a frame that was not assessed.
`domain`	`string`	`LAND`, `AIR`, `MARITIME` or `CYBER`, from the ingest.
`description`	`string`	Human-readable summary.
`sensor_id`	`string`	Originating sensor.
`damage_grade`	`string`	Grade, when applicable.
`gps_coordinates`	`string`	Location.
`metadata`	`map<string, string>`	For assessments: `frames_processed`, `confidence_score`, `analysis_source`. For cues: `target_sensor_type`, `target_status`, `alert_payload`.
`damage_percentage`	`int32`	Estimated damage, 0 to 100.

ItemMatch

The answer to “what is this object in our data?”. Defined in catalog.proto.

Field	Type	Description
`item`	`ItemRef`	The catalogued item. The platform has no item-master writer yet, so this is unset on every response.
`basis`	`MatchBasis`	How the match was reached. Reachable today: `MATCH_BASIS_CORPUS` and `MATCH_BASIS_UNRESOLVED`.
`confidence`	`optional float`	Unset on every response. Rank matches by `basis`.
`evidence`	`string`	What justified the match: the alias that hit or the document that mentioned it. Empty when unresolved.
`queried_label`	`string`	The label that was looked up.

ItemRef

Field	Type	Description
`sku`	`string`	Canonical identifier within the tenant.
`description`	`string`	Description.
`material`	`string`	Material, in the vision model’s vocabulary.
`uom`	`string`	Unit of measure, for example `EA`, `PAL` or `CS`.
`aliases`	`repeated string`	Labels the item is known by. Matching uses these.
`length_mm`	`double`	Length. 0 means unknown.
`width_mm`	`double`	Width. 0 means unknown.
`height_mm`	`double`	Height. 0 means unknown.
`weight_kg`	`double`	Weight. 0 means unknown.
`load_capacity_kg`	`double`	Load capacity. 0 means unknown.
`source`	`string`	Provenance of the entry.
`updated_at`	`string`	Last update.

SightingTotals

Field	Type	Description
`sighting_count`	`int32`	Sightings in the window.
`ingest_count`	`int32`	Distinct ingests.
`sensor_count`	`int32`	Distinct sensors.
`ingests_with_no_sightings`	`int32`	Ingests that were read and held no inspectable object.
`grades`	`GradeCounts`	Grade distribution.
`vocabulary`	`VocabularyCounts`	Vocabulary tally.
`damage_percent_stated_sightings`	`int32`	Denominator of `mean_damage_percent`.
`mean_damage_percent`	`optional double`	Mean over stated sightings. Absent when none stated one.
`damaged_share`	`optional double`	Share of graded sightings that are not `PRISTINE`, 0 to 1. Absent when none were graded.
`multi_frame_sweep_sightings`	`int32`	Sightings from multi-frame sweeps, where one object may be counted once per frame.

SightingGroup

Field	Type	Description
`key`	`string`	The material, object class or sensor id.
`sighting_count`	`int32`	Sightings, not things.
`ingest_count`	`int32`	Distinct ingests that contributed.
`grades`	`GradeCounts`	Grade distribution.
`vocabulary`	`VocabularyCounts`	Vocabulary tally.
`damage_percent_stated_sightings`	`int32`	Denominator of `mean_damage_percent`.
`mean_damage_percent`	`optional double`	Mean over stated sightings. Absent when the denominator is 0.
`damaged_share`	`optional double`	Share of graded sightings that are not `PRISTINE`. Absent when none were graded.
`first_seen`	`google.protobuf.Timestamp`	First sighting.
`last_seen`	`google.protobuf.Timestamp`	Last sighting.

GradeCounts

The five graded buckets plus ungraded_sightings add up to the set’s sighting_count.

Field	Type	Description
`pristine_sightings`	`int32`	Graded `PRISTINE`.
`damaged_sightings`	`int32`	Graded `DAMAGED`.
`defective_sightings`	`int32`	Graded `DEFECTIVE`.
`destroyed_sightings`	`int32`	Graded `DESTROYED`.
`other_grade_sightings`	`int32`	A grade string that is none of the four, reported rather than folded into a neighbour.
`ungraded_sightings`	`int32`	No grade stated. Excluded from every damage average.

VocabularyCounts

The three counts add up to the set’s sighting_count.

Field	Type	Description
`in_vocabulary_sightings`	`int32`	Label in the class vocabulary.
`out_of_vocabulary_sightings`	`int32`	Label graded and not in the vocabulary.
`not_asked_sightings`	`int32`	The classifier was not asked. Neither in nor out.

WindowComparison

Field	Type	Description
`baseline_since`	`google.protobuf.Timestamp`	Baseline window actually used.
`baseline_until`	`google.protobuf.Timestamp`	Baseline window end.
`current_since`	`google.protobuf.Timestamp`	Current window start.
`current_until`	`google.protobuf.Timestamp`	Current window end.
`baseline_sighting_count`	`int32`	Sightings in the baseline.
`current_sighting_count`	`int32`	Sightings in the current window.
`material_deltas`	`repeated` `GroupDelta`	One row per material present in either window.
`object_class_deltas`	`repeated` `GroupDelta`	One row per class present in either window.

GroupDelta

A key absent from one window has a real zero sighting_count there, because the window was scanned. Its mean and share stay absent.

Field	Type	Description
`key`	`string`	Material or class.
`baseline_sighting_count`	`int32`	Baseline sightings.
`current_sighting_count`	`int32`	Current sightings.
`baseline_damage_percent_stated_sightings`	`int32`	Baseline denominator.
`current_damage_percent_stated_sightings`	`int32`	Current denominator.
`baseline_mean_damage_percent`	`optional double`	Baseline mean.
`current_mean_damage_percent`	`optional double`	Current mean.
`mean_damage_percent_delta`	`optional double`	Present only when both means are present.
`baseline_damaged_share`	`optional double`	Baseline share.
`current_damaged_share`	`optional double`	Current share.
`damaged_share_delta`	`optional double`	Present only when both shares are present.

OCRDataSource

Field	Type	Description
`type`	`string`	Where the file comes from, case-insensitive: `LOCAL` (a file already staged on the server, named by base name only, or a staged directory), `NETWORK` (fetched over HTTP from `host`, `port` and `path`), `SFTP`, `GCP_BUCKET` (bucket in `host`, object prefix in `path`) or `GDRIVE` (file id in `path`). Any other value fails. The proto comment also lists `SNOWFLAKE_METADATA`, which the server does not accept.
`path`	`string`	File path, object prefix or file id, depending on `type`. For `LOCAL`, only the base name is used.
`credentials`	`string`	Write-only. For `SFTP`, JSON with `user` or `username`, `password` and `host_key`; a host key is required when the server has no known-hosts file. For `GCP_BUCKET` and `GDRIVE`, service-account JSON.
`format`	`string`	`PDF`, `DOCX`, `PPTX` or `AUTO`.
`host`	`string`	Host for `NETWORK` and `SFTP`, or bucket for `GCP_BUCKET`.
`port`	`int32`	Port for `NETWORK` and `SFTP`.

ExcelCodeResult

Field	Type	Description
`formulas`	`repeated` `Formula`	Cell formulas.
`macros`	`repeated` `Macro`	Macros.
`vba_modules`	`repeated` `VBAModule`	VBA modules.
`named_ranges`	`map<string, string>`	Named ranges and their references.

Formula

Field	Type	Description
`sheet`	`string`	Sheet name.
`cell`	`string`	Cell reference.
`formula`	`string`	The formula.

Macro

Field	Type	Description
`name`	`string`	Macro name.
`code`	`string`	Macro code.

VBAModule

Field	Type	Description
`name`	`string`	Module name.
`code`	`string`	Module code.

CodeFunction

Field	Type	Description
`name`	`string`	Function name.
`parameters`	`repeated string`	Parameters.
`return_type`	`string`	Return type.
`body`	`string`	Function body.
`line_number`	`int32`	Line where it starts.

CodeClass

Field	Type	Description
`name`	`string`	Class name.
`methods`	`repeated string`	Method names.
`properties`	`repeated string`	Property names.
`line_number`	`int32`	Line where it starts.

Enums

BoxOrigin

Value	Meaning
`BOX_ORIGIN_UNSPECIFIED`	No known stage assigned the box. This is not “no box”: check `box_w` and `box_h`.
`BOX_ORIGIN_VISION_GRADING`	The vision model’s scene-level grading turn. The weakest of the four.
`BOX_ORIGIN_VISION_DETECTION`	The vision model’s per-frame detection turn.
`BOX_ORIGIN_REGION_CLUSTERING`	Deterministic region proposal: “something is here”, with no label or grade.
`BOX_ORIGIN_LOCAL_DETECTOR`	A local detector in the serving process. Has `detector_label`. Never has a grade.

MatchBasis

Value	Meaning
`MATCH_BASIS_UNSPECIFIED`	Not set.
`MATCH_BASIS_SKU`	The detection carried a SKU outright. Cannot occur until an item master exists.
`MATCH_BASIS_ALIAS`	A catalogued alias matched. Cannot occur until an item master exists.
`MATCH_BASIS_CORPUS`	No catalogue entry, but a tenant document mentions it. `evidence` names the document.
`MATCH_BASIS_UNRESOLVED`	Nothing matched. A first-class outcome, never replaced by a nearest guess.

Lineage & activity Calling & support