Kafka for Administrator - siva-kumar-kvjf-20250108-093145

Visit the Kafka for Administrator course recordings page

                WEBVTT

00:00:34.540 --> 00:00:42.140
everybody have completed any questions on there are any question that you want to discuss

00:00:45.740 --> 00:00:53.160
then I have fixed the cluster on the topics yesterday like there is an issue with the

00:00:53.160 --> 00:01:04.040
WS where it is not able to accept the request now I will give you the link where you can try out

00:01:04.040 --> 00:01:21.240
the game once you try it out we'll check the monitoring and see how the topics are getting

00:01:21.240 --> 00:01:28.460
then we will go ahead and install the Grafana and permittance to see in our own system

00:01:39.040 --> 00:01:45.960
yeah if you are able to open the one which I have pinned you should be able to see dashboard like this

00:01:45.960 --> 00:01:52.080
while we go through I will check the topics and see first I will go and check the user

00:01:52.080 --> 00:02:03.220
game see the messages how they are coming in you see for every 20 seconds we get an

00:02:03.220 --> 00:02:10.240
user with the score and the lives and the weight the level is this is one of the topic

00:02:10.240 --> 00:02:20.700
which listens to the messages coming in from specific user so let's see the stream image

00:02:20.700 --> 00:02:31.580
how actually the whole workflow goes through you see there are 3 49 producers why there are

00:02:31.580 --> 00:02:40.260
3 49 producers basically like every time you do play there is an asynchronously happens

00:02:40.260 --> 00:02:48.260
using the lambda function like it triggers the serverless option and inside the serverless

00:02:48.260 --> 00:02:55.560
it will push the message to the topic these are all the different producers producing data

00:02:55.560 --> 00:03:03.000
so ideally what happens is in this case every time a new producer comes in he will actually

00:03:03.000 --> 00:03:09.300
produce only one message he will not produce multiple because every time it triggers a

00:03:09.300 --> 00:03:17.340
new event internally from the application point of view so then those producers

00:03:17.340 --> 00:03:24.460
whatever it is producing it is going to the topic user game and we can see the

00:03:24.460 --> 00:03:29.900
number of partitions here and what is the bytes in and out what is the bytes we are

00:03:29.900 --> 00:03:34.820
getting in and what is the bytes we are getting out and also the messages in and messages

00:03:34.820 --> 00:03:43.100
out from the topic and we can see the retention time how long we can keep as

00:03:43.100 --> 00:03:50.500
of now I kept it as 1 hour and retention size I have set it to infinite it depends

00:03:50.500 --> 00:03:58.820
by comparing whatever the value we want and we have the cleaner policy estimate

00:03:58.820 --> 00:04:10.140
here from here we have 2 ksql queries that we have added one is stats per user

00:04:10.140 --> 00:04:18.080
that means we need the details of every user highest score that we will take it off

00:04:18.080 --> 00:04:26.080
here and the summary stats is basically like the whole whoever has still played till now

00:04:28.700 --> 00:04:33.160
it will be a summary of all those details