Commit graph

30 commits

Author SHA1 Message Date
Timur Davletov
29a3158df6
#707 Exposed broker log dirs (#720)
#707 Exposed broker log dirs
2021-07-27 13:03:21 +03:00
Ildar Almakaev
5f4de1313d
Handle errors when creating topics and return custom one as BAD_REQUEST (#670) 2021-07-14 15:35:45 +03:00
Ilya Kuramshin
73d4e48941
Json Schema support (#651)
* Json schema support implementation

* comments & checkstyle

* unused mapper removed

* Explicit validation added

Co-authored-by: Ilya Kuramshin <ikuramshin@provectus.com>
2021-07-12 13:41:38 +03:00
Ilya Kuramshin
dd19cc2eec
Msg sending improvements (#640)
* init

* Messages sending fixes & tests

* Messages sending fixes & tests

* Messages sending json test added

* PR comments fixes

* Setting getTopicMessages & sendTopicMessage key value type to string.

* admin-client-timeout default removed

* sonar fix

* messages filtering changes to String.contains

Co-authored-by: Ilya Kuramshin <ikuramshin@provectus.com>
2021-07-09 15:53:30 +03:00
German Osin
71e1370deb
Provide topic offsets in list (#647) 2021-07-07 22:31:45 +03:00
Ilya Kuramshin
66228b00d5
#234: Consumer group offsets reset implementation (#605)
* junit4 annotations replaced with jupiter

* Consumer group offsets reset implementation

* comments added, Sonar warn fix

* minor cleanup

* checkstyle fix

* PR comments fix

* API doc fix

* API path renamed

Co-authored-by: Ilya Kuramshin <ikuramshin@provectus.com>
2021-07-05 14:32:01 +03:00
Marsel
09ebd03e71
Issue#314 partitions count increase (#579)
* adding topic's partitions count increase

* adding tests

* making pull request fixes

* making pull request fixes

* adding checkstyle fix

* pull request fixes

* adding cluster update after increasing partitions

* pull requset fix

Co-authored-by: marselakhmetov <makhmetov@provectus.com>
2021-07-02 14:38:22 +03:00
German Osin
97ec512b00
#122 Fix emitter to consume records in right order (#598)
* #122 Fix emitter to consume records in right order

* Fixed naming
2021-06-29 09:52:18 +03:00
German Osin
1460fef68e
ISSUE-502 Send messages & avro key deserializer (#582)
* ISSUE-502 Send messages & avro key deserializer

* Fixed sonarcube issue

* Fixed sonarcube issues

* Schema endpoint & fixes

* fixed sonar issue
2021-06-28 09:54:20 +03:00
Ildar Almakaev
5dd3944faa
Add API to delete consumer group by id (#578)
* [issue-516] Add API to delete consumer groups by IDs

* Add more tests to check consumer groups deletions

* Refactor and fix code style

* Fix codestyle

* Rethrow OperationInterruptedException with 500 error code if a thread is interrupted

* Use SneakyTrhrows to handle InterruptedException. Refactor

* Change deletion of groups API to single group delete

* Fix codestyle

* Rollback changes in kafka-ui-react-app/package-lock.json
2021-06-23 20:47:32 +03:00
German Osin
d3bf65cfb6
#122 Seek direction backend support (#562) 2021-06-18 11:56:14 +03:00
German Osin
a1ee6a30c0
#518 Fixed backend page size (#523)
* #518 Fixed backend page size

* Fixedd tests
2021-06-03 19:44:42 +03:00
Ramazan Yapparov
f3c0866940
#187 added search and sorting (#388)
* - added search by topic name
- added filter by internal topics
- added sort by multiple columns
- added tests

* moved sorting later in the stream
2021-04-27 13:32:08 +03:00
Ramazan Yapparov
eef45fc6ab
#356 added endpoint to fetch all cluster connectors (#361)
* added endpoint to fetch all cluster connectors

* - refactoring code
- refactoring schema
- added connector class property
2021-04-15 11:15:25 +03:00
Ildar Almakaev
e9a0a1af91
Get only numeric values of JMX metrics (#318)
* Use only numeric JMX metrics
* Propagate all exceptions from JmxClusterUtil.getJmxMetric to upper call
2021-03-31 12:34:43 +03:00
iliax
8d2f929a52
ISSUE-309: Topic create & update models split (#312)
* ISSUE-309: Topic create & update models split
2021-03-25 11:24:51 +03:00
iliax
31fe89d1c9
ISSUE-297: Sonar warnings fixes (#298)
* ISSUE-297: Sonar warnings fixes
2021-03-23 21:54:03 +03:00
iliax
992e8b0898
#253: Get messages implementation improvement (#286)
* 1. End offsets check added to RecordEmitter
2. Tests for OffsetsSeek, RecordEmitter added

* Tests improvements

* ISSUE-257: checkstyle

* ISSUE-257: withSchemaType sonar fix

* ISSUE-257: withSchemaType sonar fix

Co-authored-by: Ilya Kuramshin <ikuramshin@provectus.com>
2021-03-23 14:55:04 +03:00
German Osin
590bdfb610
Added checkstyle (#296) 2021-03-23 10:55:28 +03:00
Ramazan Yapparov
a8ed4ff37f
#188 added pagination for get topics api (#249)
* added pagination for get topics api

* frontend fix

* - fixed merge conflicts
- renamed pageSize to perPage
2021-03-15 21:37:36 +03:00
German Osin
909e196011
Feature refactoring, structure simplifying (#250) 2021-03-15 17:41:25 +03:00
Ramazan Yapparov
881a2167b7
#211 Feature/clear topic messages (#241)
* added delete action

* added test for delete action

* added 404 status in contract

* fixed typo

* added partition parameter

* big refactoring
2021-03-15 11:51:33 +03:00
Ildar Almakaev
e02dd84491
#200 Update schema subject object (#229)
* Return id, version, schema, and subject after creating a new schema

* Throw 422 error code if incoming new schema is unprocessable entity due to invalid fields

* Return 409/Conflict error code if schema is duplicate. Change endpoint of createNewSchema method

* Fix submitting new subject or new version if the subject already exists

* Include schema type to schema objects. By default it's AVRO

* [ISSUE-200] Update FE to support new version of api

* Add one more schema-registry with version 5.5.0 to docker-compose files and app properties

* Upgrade Confluent service versions in tests up to 5.5.0

* Set schemaType is required and ignore when submitting to Schema Registry if it's NULL

Co-authored-by: Oleg Shuralev <workshur@gmail.com>
2021-03-10 15:14:15 +03:00
Ramazan Yapparov
58df6c1a7e
#222 Fix npe in consumer (#223)
* add null handling
* replaced hashmaps with immutable map in deserializer returns
* added simple test for previously failing case
2021-03-09 17:15:12 +03:00
Ramazan Yapparov
be96bbc381
Testcontainers speedup and stabilization (#220)
* - moved testcontainers lifecycle control back to singleton
- changed kafka connect container wait strategy from logs to http

* refactoring

* reduced startup timeout to 5 minutes
2021-03-03 13:08:51 +03:00
Ramazan Yapparov
73f8991517
#164 readonly mode (#218)
* added ReadOnlyModeFilter

* added tests

* refactored kafka connect and schema registry tests
2021-03-01 19:51:58 +03:00
Ramazan Yapparov
377fa830c6
#163 Kafka connect crud (#182)
* - added kafka connect to docker-compose files
- added kafka connect property to application

* - added /connectors endpoint
- added /connectors/{name} endpoint

* - added /connectors/{name}/(restart/pause/resume) endpoints
- updated connector config model
- added FileStream connectors configs

* - added /connectors post endpoint

* - added all other kafka connect endpoints
- reverted config file

* - moved kafka connect related endpoints to separate tag/controller

* - added kafka connect container
- added simple test for kafka connect service
- added assertj dependency

* - added tests for KafkaConnectService

* - added more tests
- moved kafkaConnect rest client calls to separate class
- added validation

* - removed additional validation
- removed Client class
- removed client test and replaced with proper integration test
- added retries

* - cleanup

* moved to using generated kafka connect client

* updated rest schema

* added new maven clean plugin configuration so it deletes generated sources in kafka-ui-react-app/src directory

* changed restart/pause/resume action schema

* - added test
- refactoring

* added more error handling

* added more tests for error handling

* fixed schema registry tests

* changed /connect to /connects

* - fixed mutating connector object during getConnector request
- added new retrying kafka connect client with common retry logic
- fixed dependency scope

* - removed unnecessary `.cache()`

* - reverted testcontainers initialization code

* - added missing UNASSIGNED status to response enum

* - fixed configurations
- fixed testcontainers lifecycle management

* fixed application-local.yml
2021-02-26 15:48:04 +03:00
Ildar Almakaev
6ec516345e
Issue-181 Refactor schema CRUD API (#192)
* Rename attribute 'schemaName' to 'subject' for consistency with Schema Registry terms

* Refactor 'GET /schemas' and 'GET .../schemas/{subject}/versions' to get the latest schemas data at once

* Fix getting error code from attributes in our custom GlobalErrorWebExceptionHandler
2021-02-15 11:52:44 +03:00
Ildar Almakaev
961d14454a
Add CRUD actions for Schema Registry service (#165)
* Map schema registry port to 8081 in kafka-clusters-only.yaml

* Add endpoint to retrieve subjects from schema registry by cluster name

* Add endpoint to retrieve subject versions from schema registry

* Get subject schema by version from schema-registry

* Add ability to create/delete schema subjects in/from schema-registry service

* Create WebClient bean and refactor its usage

* Refactor schema api contract

* Return 'Bad request' if there is 'Internal Server Error' from Schema Registry

* Add GET/PUT methods to get/update a schema compatibility level globally or only for a scepific schema

* Add SchemaRegistryContainer.java and testcontainers dependencies for integration tests

* Add junit-jupiter5 for testing

* Add GET/DELETE actions for the latest schema version

* Add endpoint to check schema compatibility

* Set up configuration for testing

* Add basic test for SchemaRegistryService

* Fix mapping from dto to web model

* Change createNewSchema action's endpoint

* Update tests in SchemaRegistryServiceTests

* Rename getLatestSchema method and update tests

* Add more test in SchemaRegistryServiceTests

* Include compatibility level info to schema response. Change Flux to Mono

* Update tests

* Pass schema-registry url for secondLocal cluster too

* Remove explicit returns of 404 status code (it'll be processed by global error handler)

* Add global error handler to response with exception details

* Autoconfigure WebTestClient in SchemaRegistryServiceTest
2021-02-09 10:40:11 +03:00
Yevgen Taran
498eb96bee
Feature/backend init (#3)
* move react-app to its own folder inside project

* move backend to kafka-ui repo

* setup react inside netty

* make application ready in 2 commands

* update readme

* update readme

* update readme

* update readme

* update profiles for application start inside (sdp profile) and outside docker (local profile)

* broker metrics endpoint

* topics endpoint start commit

* topics details endpoint start commit //
dependencies and versions fix

* small pom updates //
continue review fixes

* fix review issues //
save errors //
save connections and update connection logic //
save jmx, zookeeper, kafka statuses //
error with getting one topic doesn't fail others //
async metrics processing //
cluster data storage refactoring

* properties version extracting

* properties versions

* topic details

* remove jmx, topic details, topic configs

* create topic

* final fixes, topic creation

* topic creation ui fixes

* add check for cases when cluster is offline
2020-02-27 15:55:53 +03:00