Commit graph

210 commits

Author SHA1 Message Date
Mohamad Choukair
f3c07b15e5
Fix zookeeper thread leaking (#1574)
* Issue #1472 Added protection to prevent infinite loops when connection is in error with zookeeper

* Issue #1472 Added protection to prevent infinite loops when connection is in error with zookeeper

* Issue #1472 Added protection to prevent infinite loops when connection is in error with zookeeper

* Issue #1472 corrected checkStyle error

* Issue #1472 corrected bug

* Get rid of a successful test

this will always fail since it's not an integration test

* Fix test

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-03-10 14:13:39 +03:00
Ilya Kuramshin
a8fc61dcff
Integration tests structure unified, unnecessary imports removed (#1711)
Co-authored-by: iliax <ikuramshin@provectus.com>
2022-03-09 15:47:36 +03:00
Ilya Kuramshin
11c6ce25ff
KSQL SSE refactoring (#1479)
* sse refactoring

* checkstyle fix

* checkstyle fix

* refactor

* refactor

* api spec changes

* ReactiveAdminClient toMono fix

* fixes

* fixes

* fixes

* fixes

* small improvement

Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: Damir Abdulganiev <dabdulganiev@provectus.com>
2022-03-08 20:23:12 +03:00
Roman Zabaluev
8b07a332e6
Fix external resources loading for backend part (#1639)
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-03-08 15:07:42 +03:00
Ilya Kuramshin
48d3c3828e
Implement topic existence check for MessagesService (#1694)
Co-authored-by: iliax <ikuramshin@provectus.com>
2022-03-08 14:26:32 +03:00
Ilya Kuramshin
4651c5e308
Add "initializing" state to ServerStatus enum (#1652)
* * initializing status added to cluster
* MetricsCache initializing
* unuseful tests removed BrokerServiceTest

* checkstyle fix

Co-authored-by: iliax <ikuramshin@provectus.com>
2022-02-24 14:18:24 +03:00
German Osin
cd24ff631e
Code optimisations (#1648)
* Code optimisations

Co-authored-by: German Osin <germanosin@Germans-MacBook-Pro.local>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: German Osin <germanosin@Germans-MBP.lan>
2022-02-22 14:49:33 +03:00
German Osin
4cc4175ef2
Minor issues fixes (#1646)
* Fixed minor issues

* fixed review bug

* fixed bug

Co-authored-by: German Osin <germanosin@Germans-MacBook-Pro.local>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-21 15:13:48 +03:00
Mohamad Choukair
e2a3b7b263
Added host & port brokers endpoint (backend) (#1553)
* Added port to endpoint brokers - Issue:1521

* Added port to endpoint brokers - Issue:1521

* Fixed code style - Issue:1521

* Fixed checkStyle violations - Issue:1521

* Issue #1521 added port to mapper

Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: Ilya Kuramshin <ilia-2k@rambler.ru>
2022-02-17 19:23:15 +03:00
Roman Zabaluev
0e227bc0b0
Fix SR failing pending requests (#1609)
* Get rid of unnecessary duplicate check

* Fix typo

* Fix error code duplication

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Add defaultToGlobal, refactoring

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Fix response entity forwarding

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Additional fixes

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Revert unnecessary changes

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Fix error code value

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>

* Fix test

Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-17 18:46:52 +03:00
Ilya Kuramshin
b8c43f069e
RESTART_ALL_TASKS & RESTART_FAILED_TASKS actions implemented, small refactoring (#1601)
RESTART_ALL_TASKS & RESTART_FAILED_TASKS kafka connect actions implemented, small refactoring (#1601)
2022-02-17 13:25:46 +03:00
Ilya Kuramshin
b3ef8da446
ISSUE-897: SchemaRegistryAwareRecordSerDe refactor (#1593)
* SchemaRegistryAwareRecordSerDe refactor

Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-17 13:15:27 +03:00
Jonas Geiregat
ce8627ea59
Fix hanging schema delete (#1360)
* Fix hanging schema delete

Co-authored-by: Ilya Kuramshin <ilia-2k@rambler.ru>
Co-authored-by: Jonas Geiregat (31198) <jonas.geiregat@tvh.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-14 21:26:51 +03:00
ValentinPrischepa
edabfca966
Implement recreating a topic
* [ISSUE-998][backend] Add functionality to re-create topic in one click

* [ISSUE-998][backend] Add functionality to re-create topic in one click

* [ISSUE-998][backend] Add functionality to re-create topic in one click

Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-09 16:05:08 +03:00
ValentinPrischepa
529cd0bd6e
Implemented topics sorting by size (#1539)
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-09 15:35:33 +03:00
Ilya Kuramshin
772b878d90
ISSUE-943: Topic messages tailing implementation (#1515)
* Topic messages tailing implementation
2022-02-09 14:23:35 +03:00
Ilya Kuramshin
1699663bac
Smart filters: Groovy script messages filter implementation (reopened) (#1547)
* groovy script messages filter added
2022-02-09 13:28:27 +03:00
Ilya Kuramshin
6424a88aca
Console banner updated (#1319)
* banner changed to fix new name

* width adjusted to 80 to fit all terminals

Co-authored-by: iliax <ikuramshin@provectus.com>
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-02-08 12:42:15 +03:00
Ilya Kuramshin
81ecea1c91
Sanitizer disable property added (#1531) 2022-02-01 21:35:42 +03:00
ValentinPrischepa
42a004af1c
Implement sorting order for consumers page (#1526)
* [ISSUE-1383]Create parameters of request for sorting topics. (backend)

* [ISSUE-1383]Create parameters of request for sorting topics. (backend)

* Create parameters of request for sorting consumer groups. (backend)
2022-02-01 13:14:05 +03:00
ValentinPrischepa
7cdcacf5d5
Implement topics sorting (backend) (#1498)
* [ISSUE-1383]Create parameters of request for sorting topics. (backend)

* [ISSUE-1383]Create parameters of request for sorting topics. (backend)

Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-01-31 13:38:56 +03:00
ValentinPrischepa
540b8eb79b
Schema registry pagination and search
* [ISSUE-1191, ISSUE-1208] Implemented search and pagination for schema registry overview page.

* [ISSUE-1191, ISSUE-1208] Implemented search and pagination for schema registry overview page.

* [ISSUE-1191, ISSUE-1208] Implemented search and pagination for schema registry overview page.

* [ISSUE-1191, ISSUE-1208] fixed Checkstyle violation issue.

* WIP: Fixes some frontend issues just to build frontend

* WIP: Fixes fronted just to build it

* WIP: Fixes frontend to build it

* WIP: Schemas tests are failing

* WIP: List tests work

* WIP: Details test work

* WIP: Updates tests

* WIP: Fixes lint errors and comments

* WIP: Changes usePagination, some tests have warns

* WIP: Refreshes with query string works correctly

* cleanup

* WIP: cleanup

* WIP: cleanup

* WIP: Removes ThemeProvider from test as render function uses ThemeProvider

* WIP: Pagination + Search works correcly

* WIP: Cleanup

* WIP: Cleanup

* WIP: Cleanup

Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
Co-authored-by: Damir Abdulganiev <dabdulganiev@provectus.com>
Co-authored-by: Damir Abdulganiev <damupka@gmail.com>
2022-01-31 13:24:39 +03:00
aizerin
2c6a197cb3
LDAP: add admin auth and search filter (#1403)
Co-authored-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-01-28 15:33:27 +03:00
Roman Zabaluev
afca54d374
Add missing readonly field (#1506)
Signed-off-by: Roman Zabaluev <rzabaluev@provectus.com>
2022-01-28 15:24:50 +03:00
Ilya Kuramshin
f85a340b79
ISSUE-1444: Additional params added to connector restart endpoint to be able to… (#1445)
* Additional params added to connector restart endpoint to be able to restart not only connector instance, but all its tasks
* RESTARTING state added to ConnectorTaskStatus
* Restart connector button rename

Co-authored-by: iliax <ikuramshin@provectus.com>
2022-01-21 15:24:13 +03:00
bystnik
0a7d64de78
URL decoding for cluster names (#1378)
Co-authored-by: Nikita Bystrov <n.bysterov>
2022-01-18 13:32:41 +03:00
Ilya Kuramshin
6828a41242
Consumer groups pagination (Backend) (#1318)
* consumer-groups endpoint with pagination added
* consumer group dto mappings moved to controller
* consumer groups loading logic refactored
2022-01-17 13:58:33 +03:00
Ilya Kuramshin
b54b20aa48
bytesOutPerSec metric added to topic (#1358)
Co-authored-by: iliax <ikuramshin@provectus.com>
2022-01-10 11:13:38 +03:00
Roman Zabaluev
24f7e4c190
Fix NPE on producing a message (#1342) 2021-12-29 19:37:44 +03:00
Roman Zabaluev
108242aeb2
Fix healthchecks (#1339) 2021-12-29 13:20:34 +03:00
Roman Zabaluev
32a2e753b0
Allow producing messages with an empty key/value
* [ISSUE 1046] UI allows to submit message with empty key & value (#1264)

* [ISSUE 1046] UI allows to submit message with empty key & value

* Update Contract

(cherry picked from commit 4b730eb288)

* Backend fix

* Refactoring

* Fix nullable & checkstyle

* Fix jsonnullable get

* Remove unnecessary check and add a test

Co-authored-by: Oleg Shur <workshur@gmail.com>
2021-12-24 19:00:27 +03:00
ValentinPrischepa
49285bee9b
Improve exception message about schema registry back compatibility
* [ISSUE-1250] Introduced new exception for attempt at creating new PROTOBUF/JSON schemas via a deprecated version of Schema Registry.

* [issue-1250] fixed formatting

* [ISSUE-1250] updated exception message
2021-12-23 10:43:10 +03:00
Roman Zabaluev
e34520f6e9
Implement LDAP authentication (#1173)
* Sync container versions in examples

* Add ldap required dependencies

* Create a separate package for security configs

* Fix annoying checkstyle line length

* Refactor auth security configuration setup

* Implement ldap authentication. Closes #1023

* Review fixes

* Review fixes
2021-12-22 18:18:59 +03:00
Roman Zabaluev
b2586c41a0
Fix CVE-2021-44228 (#1231)
* Fix CVE-2021-44228

* Get rid of log4j

* Fix build

* Migration logging properties

* Get rid of json logs. Leave commented for local debugging purposes
2021-12-22 15:53:46 +03:00
Ilya Kuramshin
2ce80f6d80
cluster storage usage removed from SchemaRegistryService (#1262)
Co-authored-by: iliax <ikuramshin@provectus.com>
2021-12-17 11:51:31 +03:00
Ilya Kuramshin
ec42c37f24
Ksql functionality reimplemented (#1161)
* Ksql functionality reimplemented

Co-authored-by: iliax <ikuramshin@provectus.com>
2021-12-13 13:16:47 +03:00
Ilya Kuramshin
c9811e4f60
Kafka API timeout fixes + doc (#1187)
*  kafka.admin-client-timeout set to 30s by default
* unused zookeeper.connection-timeout removed
* documentation to admin-client-timeout param added
2021-12-08 13:23:10 +03:00
Ilya Kuramshin
062367efcc
1. stacktrace field added to error responses (#1145)
1. stacktrace field added to error responses (#1145)
2. lastError added to Cluster responce
3. blocking metrics initialization on startup changed to async init (returned to prev behavior)
2021-11-30 13:07:51 +03:00
Ilya Kuramshin
4d3db7a7de
AdminClient creation error handling fix
1. AdminClient error logging added (#1136)
2. MetricsService error handling fixed
2021-11-29 12:44:49 +03:00
Ilya Kuramshin
d674808f3c
ISSUE-1128: Add respect to cluster's order in ClustersStorage (#1130)
* add respect to cluster's order in ClustersStorage
* minor cleaning
2021-11-26 14:50:32 +03:00
Ilya Kuramshin
0cc39fcd7f
using AdminClient for offsets retrieval instead of Consumer (#1123)
Co-authored-by: iliax <ikuramshin@provectus.com>
2021-11-24 14:44:26 +03:00
Ilya Kuramshin
43a0e383cf
Metrics & cache refactoring (#1036)
Metrics retrieval refactoring:
* metrics data moved to separate class with its own storage
* topics list pagination logic changed for better actuality
* some mappings moved to InternalXXX classed
2021-11-17 13:10:55 +03:00
Roman Zabaluev
e021d97194
Enable spring boot actuators 2021-11-12 14:23:03 +03:00
Si Tang
c986bc178c
Sanitize credentials in Kafka Connector configs
* #980: sanitize credentials in Kafka Connector configs. Add a warning to edit page when people try to edit configs with obfuscated field

* #980: make sanitizer patterns configurable

* #980 fix using stream pattern

Co-authored-by: Si Tang <si@indeed.com>
2021-10-30 13:42:08 +03:00
Ilya Kuramshin
627885d0ec
Metrics collection refactoring
* BE Services split:
* Unrelated logic moved from ClusterService to proper services
* KafkaCluster existence check moved to controllers level
* useless interfaces removed

* ConsumingService merged into MessagesService

* sonar fix

* PR fixes

* methods rename

* checkstyle fix

* wip

* wip

* wip

* wip

* wip

* wip

* getTopicDetails partitions set returned

* compilation fix

* Disk usage fields fixes

* typo & unused removal

* features retrieval returned

Co-authored-by: Ilya Kuramshin <ikuramshin@provectus.com>
2021-10-28 11:55:43 +03:00
Ilya Kuramshin
c5e5717a98
Handling for 404 responses from connect added (#1025)
* handling for 404 responses from connect added
* connector name uniqness check added (#1026)
2021-10-27 21:31:15 +03:00
Si Tang
7bd3fdd152
Upgrade Spring Boot to 2.5.6
* #931: upgrade Spring Boot to 2.5.6

* #931: add spring-boot-starter-validation dependency
2021-10-26 14:38:25 +03:00
Ilya Kuramshin
83fba5aca3
onTopicXXX methods refactored to use only cluster name. This is important because cluster instance send as parameter can be outdated in case of simultaneous updates (#1012) 2021-10-25 21:10:59 +03:00
Ilya Kuramshin
335e0ea983
ISSUE-993: Cluster names validation moved to ClustersProperties, added default n… (#1010) 2021-10-25 14:51:01 +03:00
German Osin
4339841771
Changed consumer groups request logic for better performance (#965)
* Changed consumer groups request logic for better performance

* Fixed checkstyle
2021-10-14 11:52:58 +03:00