Ilya Kuramshin
727f38401b
Expose cluster ACL list ( #2818 )
2023-05-02 16:34:57 +04:00
Ilya Kuramshin
47c8f8eeb5
BE: Fix protobuf map support ( #3683 )
...
* ISSUE-3674: Setting untyped object schema for "map" protobuf fields
* com.provectus.kafka.ui.util.jsonschema classes visibility refactoring
* kafka-ui-serdes.yaml changes rolled back
---------
Co-authored-by: iliax <ikuramshin@provectus.com>
2023-04-19 12:57:26 +04:00
Vlad Senyuta
87a8f08ae1
[e2e] Setup selenoid for multithreading ( #3609 )
...
* add browserSetup
* add browserSetup
* upd browserSetup
* upd browserSetup
* upd browserSetup
* upd browserSetup
* upd browserSetup
* upd read me
2023-04-04 07:56:02 +04:00
Roman Zabaluev
ff759fa455
Documentation refactoring
2023-03-30 08:37:48 +04:00
Shubham Kumar
20cc1e489b
[Docs] Update prerequisites docs ( #3392 )
...
* add prerequisites for ubuntu
* Update prerequisites.md
---------
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2023-03-27 23:26:12 +04:00
David
e72f6d6d5d
Feature: Cluster web configuration wizard ( #3241 )
...
* created wizard
* Create wizard form schema
* Wizard kafka cluster form (#3245 )
* created wizard Kafka Cluster form
* created error message
Co-authored-by: davitbejanyan <dbejanyan@provectus.com>
* Update schema.ts
* Wizard authentication (#3268 )
* created authentication form
* changed SaslType.tsx switch case
* remove console.log
* commented unused variables
* auth validation
* auth Security Protocol
* changed schema.ts username, password
* added Delegation tokens validation schema
* changed auth form
---------
Co-authored-by: davitbejanyan <dbejanyan@provectus.com>
* New Dashboard flow. Add an ability to configure clusters
* wizard kafka cluster validate (#3294 )
* kafka cluster validate
* fixed bootstrap server uncontrolled input warning error
---------
Co-authored-by: davitbejanyan <dbejanyan@provectus.com>
* Wizard schema registry (#3286 )
* created schema registry
* unused variables
* Prevent Default on click
---------
Co-authored-by: davitbejanyan <dbejanyan@provectus.com>
* feat: cleanup
* Application config API (#3242 )
* wip
* wip
* wip
* wip
* OAuthProperties added to dynamic config api
* wip
* files upload endpoint added
* rbac conf api added
* rbac conf api improvements
* App configuration validation endpoint (#3264 )
Co-authored-by: iliax <ikuramshin@provectus.com>
---------
Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Oleg Shur <workshur@gmail.com>
* add app config api client
* refactor cluster section
* refactor cluster section
* linting
* refactor Registry Form (#3311 )
* refactor Registry Form
* refactor Registry
---------
Co-authored-by: davitbejanyan <dbejanyan@provectus.com>
* auth form improvements
* refactoring
* linting
* file upload API changes
* Auth
* Start connecting to schema & payload
* Auth
* fileupload
* Wizard JMX Metrics form (#3303 )
* created JMX Metrics form
* refactor JMXMetrics.tsx styles
* added cursor on checkbox, changed styles submit button
* refactor Metrics
* refactoring
* uncomment schema connect validation
---------
Co-authored-by: davitbejanyan <dbejanyan@provectus.com>
* validate api
* refactor
* Wizard Kafka Connect form (#3300 )
* created Kafka Connect form
* renaming functions and variables
* refactor
* changed button name
* refactoring kafka connect
* made handler function, reset replace with set value,
* refactoring
* uncomment schema metrics validation
---------
Co-authored-by: davitbejanyan <dbejanyan@provectus.com>
* fixing AdminClient validation
* fixing AdminClient validation
* refactor kafka connect
* refactor metrics
* Per-cluster SSL verification settings (#3336 )
* ssl configuration moved to app & cluster level
* documentations changes
* trust all removed, global app settings removed
* extracting ssl properties settings to SslPropertiesUtil
* wip
* documentation fix
---------
Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
* SSL properties NPE fixes
* api integration
* custom fields for existing auth config
* OffsetsResetServiceTest fix
* cluster.properties structure flattening added
* kafka-ssl.yml: ssl properties moved to separate section, producer ssl properties copy added
* custom auth
* error messaging
* form submit
* feedback
* 1. defaulting metrics type to JMX
2. AdminClient id generation made uniq
* checkstyle fix
* checkstyle fix
* refactoring
* feedback
* feedback
* feedback
* feedback
* feedback
* feedback
* Wizard: Application info API (#3391 )
* Application info API added, cluster features enum renamed to `ClusterFeature`
* show config for specific envs only
* refactor widget
* Cluster connection validation err msg improved
* KSQL DB section
* Refactor + deps upgrade
* experiment: get rid of babel
* BE validations refactoring
* Update kafka-ui.yaml
fixed to string type param
* fixes #3397
* linting
* #3399 adjust size of port input
* disable selects for disabled form
* Wizard: Keystore separation (#3425 )
* wip
* wip
* compose fix
* dto structure fix
---------
Co-authored-by: iliax <ikuramshin@provectus.com>
* dynamic ops enablement properties improvements
* own keystore for each section
* linting
* fix keystore submit
* fix keystore submit
* feedback
* feedback
* refactoring
* Connect config userName field renamed
* metrics configs mapping fix
* feedback
* Wizard: Jmx ssl (#3448 )
JMX SSL implementation. Added ability to set specific ssl keystore for each cluster when connection to jmx endpoint.
* Review fixes
* upd compareVersionsOperation qase id
* add toBeAutomated into manual suite
* DYNAMIC_CONFIG_ENABLED property description added
* Resolve conflicts
* Fix issue with 400 error
* fix SR edit form
---------
Co-authored-by: davitbejanyan <dbejanyan@provectus.com>
Co-authored-by: Alexander Krivonosov <31561808+GneyHabub@users.noreply.github.com>
Co-authored-by: Oleg Shur <workshur@gmail.com>
Co-authored-by: Ilya Kuramshin <iliax@proton.me>
Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: bkhakimov <bkhakimov@provectus.com>
Co-authored-by: Mgrdich <mgotm13@gmail.com>
Co-authored-by: VladSenyuta <vlad.senyuta@gmail.com>
2023-03-09 19:36:49 +04:00
Ilya Kuramshin
4d20cb6958
[BE] ProtobufFilesSerde imports implementation & refactoring ( #3357 )
...
* Feature: Support more complex Protobuf files
The changes in https://github.com/provectus/kafka-ui/pull/2874 added
initial support for using more than 1 Protobuf file in Kafka UI in
absence of a proper schema registry.
This change is building upon that functionality to support more complex
scenarios in which there are multiple Protobuf files being used and not
all of them are explicitly listed (for example imports).
It's using the already present Wire library from Square to do the heavy
lifting and create a comprehensive schema from all Protobuf files and
directories listed in the Kafka UI configuration.
* Refactor schema loading logic and reuse in tests
* Add support for reading Protobufs from ZIP archives
* Remove unused ProtobufFileSerde#toLocation(Path)
* wip
* wip
* wip
* wip
* wip
* wip
* wip
---------
Co-authored-by: Jochen Schalanda <jochen.schalanda@personio.de>
Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2023-03-07 16:24:19 +04:00
mkuts12
e584b15d97
Update serialization guide ( #3375 )
...
[line](f2ec4d76de/kafka-ui-api/src/main/java/com/provectus/kafka/ui/config/ClustersProperties.java (L41)
2023-02-21 17:40:05 +04:00
Narekmat
c559b909db
Implement healthcheck for e2e ( #3069 )
...
* add health check
* add health checks
2023-01-04 08:50:22 +04:00
Ilya Kuramshin
0f908ce55f
[Docs] Update JMX in kafka-ui.yaml ( #3173 )
...
Co-authored-by: iliax <ikuramshin@provectus.com>
2023-01-03 14:51:04 +04:00
Boris Djurdjevic
b87665a12c
[KSQL] Implement mTLS configuration ( #2984 )
...
* feat(ksqldb): Add mTLS support
* doc(ksqldb): Add SSL configuration
* chore(compose): Add ksqldb0 to key material
* feat(ksqldb): Add SSL-enabled demo ksqlDB instance
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-12-07 07:38:13 +04:00
Ilya Kuramshin
216b4acd3d
Serdes improvements and fixes ( #2989 )
...
Serdes improvements and fixes:
1. Built-in serdel's auto-configuration separated from explicit configuration (when properties set in serde configuration)
2. Serdes initialization logic extracted into SerdesInitializer.java
3. docker-compose example with serdes configuration added
4. Serialization.md document fixed
2022-11-30 18:43:48 +04:00
Ilya Kuramshin
20ecc74dd9
Data masking ( #2850 )
...
Data masking:
1. properties & mapping added to ClustersProperties
2. DataMasking provides function that doing masking for specified topic & target
3. Masking policies implemented: MASK, REMOVE, REPLACE
2022-11-28 12:54:07 +04:00
Ilya Kuramshin
e77b913164
Documentation for pluggable serdes ( #2702 )
...
* 1. Minor serde's code improvements
2. Javadoc added to serde-related code
* Serdes configuration docs WIP
* wip
* DeserializeResult.toString impl added
* SchemeRegistry.canDeserialize checks subject existence
* Update Serialization.md
* Update README.md
Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-11-25 14:35:06 +04:00
Boris Djurdjevic
e26f9787d8
Implement mTLS for KC ( #2833 )
...
* feat(kafka-connect): Implement kafka-connect mTLS configuration
* chore(kafka-connect): Use SecuredWebClient
* feat(ssl-demo): Configure SANs and recreate certificates
* feat(ssl-demo): Add docker-compose demo with TLS enabled components
2022-11-15 14:11:57 +04:00
Winnie Chiu
8011f60ccf
[Docs] Fix misplaced setting in compose files ( #2886 )
...
(cherry picked from commit de5b34a5c1c7ce6c727d980f7dba950ddbd528b4)
2022-11-07 19:54:11 +04:00
Jochen Schalanda
3277186fc0
Add support for multiple Protobuf files ( #2874 )
...
* Add support for multiple Protobuf files
* Ensure `protobufFiles` list is not empty
* Address review comments
2022-11-07 15:15:40 +04:00
Jochen Schalanda
62f35e3e29
Upgrade to Java 17 (Azul Zulu) ( #2879 )
...
* Feature: Upgrade to Java 17 (Azul Zulu)
Java/OpenJDK 13 has reached end of life (and end of security support)
more than 2.5 years ago: https://endoflife.date/java
This change set updates the build, CI pipeline, and Docker image of Kafka UI to Java 17 (LTS).
* Add back maven-compile-plugin to kafka-ui-e2e-checks
2022-11-07 14:57:59 +04:00
Winnie Chiu
a8ccbc96c8
Upgrade kafka in compose files ( #2601 ) ( #2777 )
...
* Upgrade kafka in compose files (#2601 )
* Fix compose files (#2601 )
* Fix arm64 compose file (#2601 )
2022-10-31 12:49:36 +04:00
Marat Gumarov
6dff8f105e
Support prometheus as a metrics interface ( #2190 )
...
* Unified internal metric representation (RawMetric)
* Prometheus retriever implemented
* Jmx metrics formatted to jmx-exporter openmentrics format
Co-authored-by: Mohamad Choukair <mchoukair@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: Daniel Ching <110129035+daching-provectus@users.noreply.github.com>
Co-authored-by: Daniel Ching <daching@provectus.com>
Co-authored-by: German Osin <germanosin@Germans-MacBook-Pro.local>
Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Ilya Kuramshin <iliax@proton.me>
2022-10-07 14:36:27 +04:00
Roman Zabaluev
69c6e9aeeb
Update docs considering docker plugin ( #2595 )
2022-09-16 19:59:49 +04:00
Roman Zabaluev
5db2c17994
Cognito logout + revamp ( #1063 )
...
* application yml example
* Update SSO guide
* Add cognito logout handler
* Fix annotations
* impl a separate config file for cognito
* cleanup
* Rollback auth.type change
* Add compose example
* Auth revamp
* Review suggestions
* Use configurationProperties, rename urls to uris
2022-09-15 05:36:21 +04:00
Jibin Philipose
9f32abcd09
New documentation for building and running without docker ( #2574 )
...
* New documentation for building and running without docker and some doc fixes and additions.
* Improvements in docs
* spell fix
* improvements in docs
* minor modification
* improvements
2022-09-15 04:38:05 +04:00
Shubham Jain
79c5f19758
Enhance documentation regarding arm/m1 ( #2453 )
...
* Update docs to help Apple Mac M1 Users
* Update missing information about
* building react app
* Update prerequisites docs
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-09-09 02:36:55 +04:00
Roman Zabaluev
ecb28f4dd5
Fix e2e compose paths
...
& update docs
2022-09-01 17:23:37 +04:00
Vlad Senyuta
9440c5e7af
[e2e] Get rid of 2nd cluster ( #2509 )
...
* add pageLoadTimeout
* upd readme
* upd readme
* upd readme
* upd readme
* rmv second cluster
* rmv second cluster
* rmv second cluster
* More renames
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-09-01 14:12:18 +04:00
Shubham Jain
b135594e3f
Add a compose file for arm64 ( #2503 )
...
Update documentation
Create scripts folder
Closes #2502
2022-08-30 16:11:50 +04:00
Roman Zabaluev
63a4514522
Update ssl compose example with
...
Add a property to disable CN verification
2022-08-29 14:04:27 +04:00
MichaelGonzalezMurillo
a665fb4d83
Add an ability to build api without frontend based on a property ( #2424 )
2022-08-17 19:17:37 +04:00
Roman Zabaluev
b6e9e43868
[e2e] Fix kafka-connect healthcheck ( #1921 )
2022-08-02 18:32:13 +04:00
Roman Zabaluev
541e4018ec
Active Directory support for LDAP ( #2056 )
...
* AD support for LDAP
* Review suggestions, documentation update
2022-07-15 18:06:49 +04:00
ㅅㅅㅇ
83222edc62
Update running.md ( #2248 )
...
Fix broken url link
2022-07-06 09:10:40 +04:00
sbritprovectus
e5f2df86dd
Fix e2e tests ( #1543 )
...
* Fix tests. Change folders structure
* Activate tests
* Update selenoid.yaml, broesers.json. Add pull of selenoid browser image.
* Change method of pulling chrome image
* Fix browsers.json and return the pulling of browser image
* Update pom.
* Update pom
* Fix pom
* Fix java version in pom
* Migrate to selenium containers
* Return selenoid container. Fix using of BASE_URL
* Delete video recording for Selenoid container
* expose kafka-ui port
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Fix tests run
* Delete comment from workflow file
* Stop using containers and migrate to WDM.
* Delete comments. Update dependencies.
* Comment using of TestConfiguration.BASE_URL.replace("localhost", "host.docker.internal");
* Retern changes
* Clean code
* Update e2e-checks.yaml
* Update e2e-checks.yaml
* Fix skip test parameter in tests project
* Delete host.docker.internal in base tests
* Return previous usage of testcontainers
* Delete base url configuration
* Revert versions in pom.xml
* Return some values
* Small fixes for stabilize tests
* Trying to fix tests running on github
* Update base test
* Fix selenide url
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Fix selenide url
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* test stuff
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* added extra-hosts host.docker.internal:host-gateway
* added extra-hosts host.docker.internal:host-gateway
* added extra-hosts host.docker.internal:host-gateway
* added command --add-host
* separate driver container
* small refactoring, change selenoid to BrowserWebDriverContainer
* changed in BaseTest port from 8678 to 8080
* changed tests with xpath and added additional actions (click javascrip, click with action)
* added issue to allure reporting
* provided fix for connectors tests
* deleted afterAll annotation
* fixed tests with ConnectorsTests
* fixed tests with Topics
* removed commented action with click
* Revert unnecessary changes
* Fix workflow
* added @Disabled for test- update topic dui to issue https://github.com/provectus/kafka-ui/issues/1500
* removed dependency, testconteiners version
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: kshpilchyna <kshpilchyna@provectus.com>
2022-06-06 22:05:43 +04:00
Lucas McCain
aab1f5cea4
Fix a link for setting up SSL for kafka ( #1758 )
...
* Fix #1752
Add guide for secure broker.
* Reference docker-compose file instead of copypaste
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-04-22 02:25:47 +03:00
Naresh Kumar Reddy Gaddam
8d10eb69e8
Kafka Connect: Implement basic auth ( #1754 )
...
* [TRIVIAL]: Enable basic auth for kafka connect cluster
* Update README.md
Co-authored-by: Naresh Kumar Reddy <naresh.kumar.reddy@dnb.no>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-04-06 15:56:43 +04:00
Joseph DeChicchis
1796fd519b
Implement deserializing binary protobuf encoded message keys ( #1729 )
...
* Add protobufMessageNameForKeyByTopic option to config. Message keys are
deserialized using a protobuf schema if the config is set. Otherwise
message keys are treated as strings.
Closes #1699
* Add documentation around kafkaui's protobuf support
* Add protobufMessageNameForKey config option
* Update README with info about default types
* Imeplement support for protobufMessageNameForKeyByTopic
* fallback to FALLBACK_FORMATTER
* Add ability to publish message with protobuf key
* Change log levels to debug and add @Nullable annotations
* Attempt at fixing documentation workflow
Co-authored-by: Ilya Kuramshin <ilia-2k@rambler.ru>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: Roman Zabaluev <github@haarolean.dev>
2022-04-05 16:26:57 +04:00
Roman Zabaluev
8908d6839c
Get rid of zookeeper ( #1726 )
...
* Get rid of zookeeper
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Fix build
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Cleanup
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Review fixes
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Skip unwanted domain check for documentation flow
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-03-23 18:23:37 +03:00
Roman Zabaluev
a67305a28e
Update readme logo
...
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-18 18:15:59 +03:00
Ottavio M. Hartman
efb8410bd6
Update documentation; fix links ( #1603 )
...
* closes 1602
* readme
* fixes
* docs
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-17 19:26:15 +03:00
Roman Zabaluev
6388153259
Update kafka-ssl example
2022-02-11 18:31:26 +03:00
Roman Zabaluev
136f12d76a
Add an example for SSL w/ kafka ( #1568 )
...
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: Ruslan Ibragimov <ruibragimov@provectus.com>
2022-02-09 12:22:51 +03:00
Roman Zabaluev
f4f8b4fa67
Documentation update ( #1381 )
...
* Update issue templates
* Update PULL_REQUEST_TEMPLATE.md
* Update PULL_REQUEST_TEMPLATE.md
* Update stale labels
* Move files, minor updates
* Moved files, minor updates
* Update bug_report.md
* Update feature_request.md
* Add roadmap document
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Update SECURITY.md
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Update stale periods
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Add a stub for guidelines for QA
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Extract building/running instructions from the readme
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* move testing guidelines
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Table of contents for contributing
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Add license and release badges
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Add info about actuator endpoints
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Update logotype and slogan
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Redone contributing guide
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
* Minor updates
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-02 21:03:48 +03:00
