kafka-ui/documentation/guides/Protobuf.md
Joseph DeChicchis 1796fd519b
Implement deserializing binary protobuf encoded message keys (#1729)
* Add protobufMessageNameForKeyByTopic option to config. Message keys are
deserialized using a protobuf schema if the config is set. Otherwise
message keys are treated as strings.

Closes #1699

* Add documentation around kafkaui's protobuf support

* Add protobufMessageNameForKey config option

* Update README with info about default types

* Imeplement support for protobufMessageNameForKeyByTopic

* fallback to FALLBACK_FORMATTER

* Add ability to publish message with protobuf key

* Change log levels to debug and add @Nullable annotations

* Attempt at fixing documentation workflow

Co-authored-by: Ilya Kuramshin <ilia-2k@rambler.ru>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: Roman Zabaluev <github@haarolean.dev>
2022-04-05 16:26:57 +04:00

1.7 KiB

Kafkaui Protobuf Support

Kafkaui supports deserializing protobuf messages in two ways:

  1. Using Confluent Schema Registry's protobuf support.
  2. Supplying a protobuf file as well as a configuration that maps topic names to protobuf types.

Configuring Kafkaui with a Protobuf File

To configure Kafkaui to deserialize protobuf messages using a supplied protobuf schema add the following to the config:

kafka:
  clusters:
    - # Cluster configuration omitted.
      # protobufFile is the path to the protobuf schema.
      protobufFile: path/to/my.proto
      # protobufMessageName is the default protobuf type that is used to deserilize
      # the message's value if the topic is not found in protobufMessageNameByTopic.
      protobufMessageName: my.Type1
      # protobufMessageNameByTopic is a mapping of topic names to protobuf types.
      # This mapping is required and is used to deserialize the Kafka message's value.
      protobufMessageNameByTopic:
        topic1: my.Type1
        topic2: my.Type2
      # protobufMessageNameForKey is the default protobuf type that is used to deserilize
      # the message's key if the topic is not found in protobufMessageNameForKeyByTopic.
      protobufMessageNameForKey: my.Type1
      # protobufMessageNameForKeyByTopic is a mapping of topic names to protobuf types.
      # This mapping is optional and is used to deserialize the Kafka message's key.
      # If a protobuf type is not found for a topic's key, the key is deserialized as a string,
      # unless protobufMessageNameForKey is specified.
      protobufMessageNameForKeyByTopic:
        topic1: my.KeyType1