Commit graph

10 commits

Author SHA1 Message Date
Ilya Kuramshin
4d20cb6958
[BE] ProtobufFilesSerde imports implementation & refactoring (#3357)
* Feature: Support more complex Protobuf files

The changes in https://github.com/provectus/kafka-ui/pull/2874 added
initial support for using more than 1 Protobuf file in Kafka UI in
absence of a proper schema registry.

This change is building upon that functionality to support more complex
scenarios in which there are multiple Protobuf files being used and not
all of them are explicitly listed (for example imports).

It's using the already present Wire library from Square to do the heavy
lifting and create a comprehensive schema from all Protobuf files and
directories listed in the Kafka UI configuration.

* Refactor schema loading logic and reuse in tests

* Add support for reading Protobufs from ZIP archives

* Remove unused ProtobufFileSerde#toLocation(Path)

* wip

* wip

* wip

* wip

* wip

* wip

* wip

---------

Co-authored-by: Jochen Schalanda <jochen.schalanda@personio.de>
Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2023-03-07 16:24:19 +04:00
mkuts12
e584b15d97
Update serialization guide (#3375)
[line](f2ec4d76de/kafka-ui-api/src/main/java/com/provectus/kafka/ui/config/ClustersProperties.java (L41))
but I'm not familiar enough with Go to be sure.
2023-02-21 17:40:05 +04:00
Ilya Kuramshin
216b4acd3d
Serdes improvements and fixes (#2989)
Serdes improvements and fixes:
1. Built-in serdel's auto-configuration separated from explicit configuration (when properties set in serde configuration)
2. Serdes initialization logic extracted into SerdesInitializer.java
3. docker-compose example with serdes configuration added
4. Serialization.md document fixed
2022-11-30 18:43:48 +04:00
Ilya Kuramshin
20ecc74dd9
Data masking (#2850)
Data masking:
1. properties & mapping added to ClustersProperties
2. DataMasking provides function that doing masking for specified topic & target
3. Masking policies implemented: MASK, REMOVE, REPLACE
2022-11-28 12:54:07 +04:00
Ilya Kuramshin
e77b913164
Documentation for pluggable serdes (#2702)
* 1. Minor serde's code improvements
2. Javadoc added to serde-related code

* Serdes configuration docs WIP

* wip

* DeserializeResult.toString impl added

* SchemeRegistry.canDeserialize checks subject existence

* Update Serialization.md

* Update README.md

Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-11-25 14:35:06 +04:00
Jochen Schalanda
3277186fc0
Add support for multiple Protobuf files (#2874)
* Add support for multiple Protobuf files

* Ensure `protobufFiles` list is not empty

* Address review comments
2022-11-07 15:15:40 +04:00
Winnie Chiu
a8ccbc96c8
Upgrade kafka in compose files (#2601) (#2777)
* Upgrade kafka in compose files (#2601)

* Fix compose files (#2601)

* Fix arm64 compose file (#2601)
2022-10-31 12:49:36 +04:00
Lucas McCain
aab1f5cea4
Fix a link for setting up SSL for kafka (#1758)
* Fix #1752

Add guide for secure broker.

* Reference docker-compose file instead of copypaste

Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-04-22 02:25:47 +03:00
Joseph DeChicchis
1796fd519b
Implement deserializing binary protobuf encoded message keys (#1729)
* Add protobufMessageNameForKeyByTopic option to config. Message keys are
deserialized using a protobuf schema if the config is set. Otherwise
message keys are treated as strings.

Closes #1699

* Add documentation around kafkaui's protobuf support

* Add protobufMessageNameForKey config option

* Update README with info about default types

* Imeplement support for protobufMessageNameForKeyByTopic

* fallback to FALLBACK_FORMATTER

* Add ability to publish message with protobuf key

* Change log levels to debug and add @Nullable annotations

* Attempt at fixing documentation workflow

Co-authored-by: Ilya Kuramshin <ilia-2k@rambler.ru>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: Roman Zabaluev <github@haarolean.dev>
2022-04-05 16:26:57 +04:00
Roman Zabaluev
f4f8b4fa67
Documentation update (#1381)
* Update issue templates

* Update PULL_REQUEST_TEMPLATE.md

* Update PULL_REQUEST_TEMPLATE.md

* Update stale labels

* Move files, minor updates

* Moved files, minor updates

* Update bug_report.md

* Update feature_request.md

* Add roadmap document

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Update SECURITY.md

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Update stale periods

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Add a stub for guidelines for QA

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Extract building/running instructions from the readme

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* move testing guidelines

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Table of contents for contributing

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Add license and release badges

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Add info about actuator endpoints

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Update logotype and slogan

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Redone contributing guide

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Minor updates

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-02 21:03:48 +03:00