Commit graph

57 commits

Author SHA1 Message Date
github-actions
081695d5e6 Increased version in pom.xml 2021-06-09 11:37:21 +00:00
github-actions
54581bee63 Increased release 2021-06-09 11:37:18 +00:00
Ildar Almakaev
721cf0e750
[Kafka Connect] Fix getting list of connector's topics (#504)
* [Kafka Connect] Fix getting list of connector's topics

* Fix code style in KafkaConnectService.java

* Fix code style
2021-05-31 12:30:29 +03:00
German Osin
345e1e2b85
Use npm codegen instead of maven (#451) 2021-05-12 15:50:05 +03:00
Marat Adiyatullin
01213c96db Add object->any type mapping for openapi generator config 2021-05-12 13:28:00 +03:00
Marat Adiyatullin
16e63f2c35 Add connect views 2021-05-03 13:33:53 +03:00
Ramazan Yapparov
f3c0866940
#187 added search and sorting (#388)
* - added search by topic name
- added filter by internal topics
- added sort by multiple columns
- added tests

* moved sorting later in the stream
2021-04-27 13:32:08 +03:00
Ramazan Yapparov
eef45fc6ab
#356 added endpoint to fetch all cluster connectors (#361)
* added endpoint to fetch all cluster connectors

* - refactoring code
- refactoring schema
- added connector class property
2021-04-15 11:15:25 +03:00
German Osin
7bfae45162
ISSUE-121 Topic level consumer groups & consumerr group details (#360) 2021-04-08 11:49:44 +03:00
iliax
8d2f929a52
ISSUE-309: Topic create & update models split (#312)
* ISSUE-309: Topic create & update models split
2021-03-25 11:24:51 +03:00
Alexander Krivonosov
a153ce7b21
#289: Negative segment size (#295)
* Fix the negative segment size problem

* Fixed contract topic size bug

Co-authored-by: German Osin <german.osin@gmail.com>
2021-03-24 12:08:54 +03:00
iliax
106c42e4cc
ISSUE-257: ErrorResponse format description (#294)
* ISSUE-257: ErrorResponse format description & processing added
2021-03-23 16:44:57 +03:00
Ramazan Yapparov
217b363076
added features field to kafka cluster (#256) 2021-03-18 21:16:59 +03:00
Ramazan Yapparov
a8ed4ff37f
#188 added pagination for get topics api (#249)
* added pagination for get topics api

* frontend fix

* - fixed merge conflicts
- renamed pageSize to perPage
2021-03-15 21:37:36 +03:00
German Osin
909e196011
Feature refactoring, structure simplifying (#250) 2021-03-15 17:41:25 +03:00
Ramazan Yapparov
881a2167b7
#211 Feature/clear topic messages (#241)
* added delete action

* added test for delete action

* added 404 status in contract

* fixed typo

* added partition parameter

* big refactoring
2021-03-15 11:51:33 +03:00
Ildar Almakaev
e02dd84491
#200 Update schema subject object (#229)
* Return id, version, schema, and subject after creating a new schema

* Throw 422 error code if incoming new schema is unprocessable entity due to invalid fields

* Return 409/Conflict error code if schema is duplicate. Change endpoint of createNewSchema method

* Fix submitting new subject or new version if the subject already exists

* Include schema type to schema objects. By default it's AVRO

* [ISSUE-200] Update FE to support new version of api

* Add one more schema-registry with version 5.5.0 to docker-compose files and app properties

* Upgrade Confluent service versions in tests up to 5.5.0

* Set schemaType is required and ignore when submitting to Schema Registry if it's NULL

Co-authored-by: Oleg Shuralev <workshur@gmail.com>
2021-03-10 15:14:15 +03:00
Ramazan Yapparov
a9cb9567d6
#224 added deleteTopic operation api (#228)
* added deleteTopic operation

* added not found response
2021-03-04 17:51:18 +03:00
Oleg Shur
36a5bbf9b1
[CHORE] Add openapi-generator-cli to generate sources on FE side (#221)
* [CHORE] Add openapi generator to react app

* [CHORE] Use consistent config for gens in mvn & react app

* [CHORE] Update enums to follow new format

* [CHORE] Fix linter issue
2021-03-03 20:56:28 +03:00
Ramazan Yapparov
73f8991517
#164 readonly mode (#218)
* added ReadOnlyModeFilter

* added tests

* refactored kafka connect and schema registry tests
2021-03-01 19:51:58 +03:00
Ramazan Yapparov
377fa830c6
#163 Kafka connect crud (#182)
* - added kafka connect to docker-compose files
- added kafka connect property to application

* - added /connectors endpoint
- added /connectors/{name} endpoint

* - added /connectors/{name}/(restart/pause/resume) endpoints
- updated connector config model
- added FileStream connectors configs

* - added /connectors post endpoint

* - added all other kafka connect endpoints
- reverted config file

* - moved kafka connect related endpoints to separate tag/controller

* - added kafka connect container
- added simple test for kafka connect service
- added assertj dependency

* - added tests for KafkaConnectService

* - added more tests
- moved kafkaConnect rest client calls to separate class
- added validation

* - removed additional validation
- removed Client class
- removed client test and replaced with proper integration test
- added retries

* - cleanup

* moved to using generated kafka connect client

* updated rest schema

* added new maven clean plugin configuration so it deletes generated sources in kafka-ui-react-app/src directory

* changed restart/pause/resume action schema

* - added test
- refactoring

* added more error handling

* added more tests for error handling

* fixed schema registry tests

* changed /connect to /connects

* - fixed mutating connector object during getConnector request
- added new retrying kafka connect client with common retry logic
- fixed dependency scope

* - removed unnecessary `.cache()`

* - reverted testcontainers initialization code

* - added missing UNASSIGNED status to response enum

* - fixed configurations
- fixed testcontainers lifecycle management

* fixed application-local.yml
2021-02-26 15:48:04 +03:00
github-actions
655aa41a72 Increased version in pom.xml 2021-02-24 12:07:32 +00:00
Ildar Almakaev
6ec516345e
Issue-181 Refactor schema CRUD API (#192)
* Rename attribute 'schemaName' to 'subject' for consistency with Schema Registry terms

* Refactor 'GET /schemas' and 'GET .../schemas/{subject}/versions' to get the latest schemas data at once

* Fix getting error code from attributes in our custom GlobalErrorWebExceptionHandler
2021-02-15 11:52:44 +03:00
Ildar Almakaev
961d14454a
Add CRUD actions for Schema Registry service (#165)
* Map schema registry port to 8081 in kafka-clusters-only.yaml

* Add endpoint to retrieve subjects from schema registry by cluster name

* Add endpoint to retrieve subject versions from schema registry

* Get subject schema by version from schema-registry

* Add ability to create/delete schema subjects in/from schema-registry service

* Create WebClient bean and refactor its usage

* Refactor schema api contract

* Return 'Bad request' if there is 'Internal Server Error' from Schema Registry

* Add GET/PUT methods to get/update a schema compatibility level globally or only for a scepific schema

* Add SchemaRegistryContainer.java and testcontainers dependencies for integration tests

* Add junit-jupiter5 for testing

* Add GET/DELETE actions for the latest schema version

* Add endpoint to check schema compatibility

* Set up configuration for testing

* Add basic test for SchemaRegistryService

* Fix mapping from dto to web model

* Change createNewSchema action's endpoint

* Update tests in SchemaRegistryServiceTests

* Rename getLatestSchema method and update tests

* Add more test in SchemaRegistryServiceTests

* Include compatibility level info to schema response. Change Flux to Mono

* Update tests

* Pass schema-registry url for secondLocal cluster too

* Remove explicit returns of 404 status code (it'll be processed by global error handler)

* Add global error handler to response with exception details

* Autoconfigure WebTestClient in SchemaRegistryServiceTest
2021-02-09 10:40:11 +03:00
66632
e2ca126354 Increased version in pom.xml 2021-01-22 11:14:20 +00:00
66632
c701a1763a Increased version in pom.xml 2021-01-20 09:40:13 +00:00
soffest
494443bb08
OpenAPI integration for kafka-ui frontend (#120)
* Added react openapi gen

* Integrated Openapi to React

Co-authored-by: German Osin <german.osin@gmail.com>
Co-authored-by: Sofia Shnaidman <sshnaidman@provectus.com>
2020-11-20 14:16:27 +03:00
soffest
9fd3697062
Kafka ui contract: Updated schemas required props. (#119)
* Updated schemas required props.

* Renamed long to int64 params

Co-authored-by: Sofia Shnaidman <sshnaidman@provectus.com>
Co-authored-by: German Osin <german.osin@gmail.com>
2020-11-19 15:38:28 +03:00
German Osin
3c196587a3
Fixed metrics and stats endpoints (#115) 2020-11-09 20:25:48 +03:00
66632
e9a6b52f2a Increased version in pom.xml 2020-10-30 15:00:38 +00:00
66632
2a743a5bf4 Increased version in pom.xml 2020-10-30 14:52:14 +00:00
66632
944fb19cb9 Increased version in pom.xml 2020-09-16 12:42:09 +00:00
66632
dd5389cb1c Increased version in pom.xml 2020-08-05 13:26:31 +00:00
66632
7a4e7820f4 Increased version in pom.xml 2020-08-03 12:47:46 +00:00
German Osin
8057dc123f
Fixed bugs (#90)
* Fixed bugs

* More fixes
2020-08-03 15:40:58 +03:00
66632
bc2efc9bee Increased version in pom.xml 2020-08-03 07:50:57 +00:00
66632
cb0c1661da Increased version in pom.xml 2020-07-31 12:53:43 +00:00
Bulat Yusupov
ed08f278f9
moved tagging before creating release notes (#88)
Co-authored-by: Bulat Yusupov <byusupov@provectus.com>
2020-07-31 15:37:01 +03:00
66632
d96ec48cc3 Increased version in pom.xml 2020-07-31 12:08:47 +00:00
Bulat Yusupov
db6824d19e
Fixed ci/cd release notes (#87)
* initial commit

* added timestamps and email recipient

* Revert "added timestamps and email recipient"

This reverts commit 41085705

* added steps to Jenkins pipeline

* added steps to Jenkins pipeline

* added script for creating release notes

* fixed script for creating release notes

* fixed script for creating release notes

* Increased version

* Revert "Increased version"

This reverts commit 86949059

* added steps to Jenkins pipeline

* Increased version

* moved ci-cd from ci-cd branch to master

* Revert "Increased version"

This reverts commit 666ee948

* fixes in pipeline

* fixed version for docker

* fixed version for docker

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* added conditions to jenkinsfile

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* testing PR build

* fixed release_json.sh script

* fixed release_json.sh script

* fixed release_json.sh script

* testing PR build

* testing PR build

* testing PR build

Co-authored-by: Bulat Yusupov <byusupov@provectus.com>
Co-authored-by: 66632 <>
2020-07-31 14:46:08 +03:00
66632
f0559d0646 Increased version in pom.xml 2020-07-31 07:48:57 +00:00
Roman Nedzvetskiy
efc35a9cfb
Backend jmx metrics (#64)
* Start doing endpoint for jmx metrics

* Added endpoint for getting jmx metric per broker

* Cluster jmx metrics sum endpoit added

* Added endpoints for cluster metrics and broker metrics

* Cleared some code

* Fixed jmxmetrics names

* Changed to all values in metrics

* Removed redundant imports

* Renamed param constant

* Changed to calculate brokers and clusters metrics in one place

* Removed redundant imports

* Fixed some mistakes

* Replaced multiple method usage into single

* Fixed mulptiple call

* Removed cluster level metrics, now only broker-level metrics in cluster

* Just small fixes

* removed redundant variable

* Renamed method for cluster level metrics

* Fixed after PR and added sum for number cluster metrics by num and persec keywords in canonicalname

* Added metricdto object

* Added list of metrics to enum

* Renames and optimizings

* Renamed jmxmetrics objects param to metrics

Co-authored-by: Roman Nedzvetskiy <roman@Romans-MacBook-Pro.local>
2020-07-30 14:03:07 +03:00
Roman Nedzvetskiy
66afaa4971
Added begin and end offset param (#78)
* Added begin and end offset param

* moved consumer to try with resources block

* Fixed some problems

* Moved to gettopicdetails

* Cleanup code

Co-authored-by: Roman Nedzvetskiy <roman@Romans-MacBook-Pro.local>
Co-authored-by: German Osin <german.osin@gmail.com>
2020-07-24 14:16:49 +03:00
Anton Petrov
fd60bdafa8
Text search for topic messages (#62)
* Text search for topic messages

* Code optimization
2020-06-23 14:01:14 +03:00
Roman Nedzvetskiy
32a09faa95
Added bytesIn/OutPerSec params for clusterMetrics object (#56)
* Added bytesIn/OutPerSec params for clusterMetrics object

* Removed redundant todos, cleaned imports

* Jmx connections moved to pool, methods moved to separate classes

* Added pool handling and returning methods

* Fix after previous PR comments - fixed result map, configured pool, removed redundant methods and code

* Removed redundant imports and empty initialization

* Removed fill method

* Closing connection replaced to destroyObject method

* Try catch block while returning object to pool was fixed

* Removed redundant logs and try catch

Co-authored-by: Roman Nedzvetskiy <roman@Romans-MacBook-Pro.local>
2020-06-17 14:44:34 +03:00
Anton Petrov
b60f2a357e
Feature/10 record deserialization (#57)
* Record deserialization

* Check avro schema for topic

* Fix sdp docker config

* Code cleanup

* Code review changes

* Move Avro schema name template to cluster-level config
2020-06-11 14:07:31 +03:00
Anton Petrov
465a3726e8
Feature/10 seek offsets (#54)
* Seek offsets

* Seek and limit
2020-06-02 14:08:22 +03:00
Roman Nedzvetskiy
f3f6e74b1e
Backend for updating topics, fixed backend from another pr branches (#52)
* Backend for updating topics, fixed backend from another pr branches

* Changed caching of extendedAdminClient instances

Co-authored-by: Roman Nedzvetskiy <roman@Romans-MacBook-Pro.local>
2020-05-28 14:18:22 +03:00
Roman Nedzvetskiy
9a5ffc9eb5
Added segmentSize for topic level (#48)
* Added segmentSize for topic level

* Added segmentSize to clusterMetrics object

* Added internalClusterState for storing clusterMetrics linked to cluster's brokersIds

* InternalBrokersMetrics param added to InternalClusterMetrics

* Changed topic segment counting logic by finding the partition leader, added partition collecting in topic

* Added segmentSize info to cluster, broker, topic and topicPartition level

* Reduce to map moved to clusterutil, leader moved in cache

* Cleared some code

Co-authored-by: Roman Nedzvetskiy <roman@Romans-MacBook-Pro.local>
2020-05-28 13:11:50 +03:00
Anton Petrov
c725927261 Add key, headers and timestamp type fields 2020-05-13 17:28:27 +03:00