Visit the Mongo DB for Administrators course recordings page
WEBVTT
-->
Faith, please log on again into the database, so you need to do Mongo SH first, then you can run that stuff.
-->
So Mongo SH, and then you can rerun the stuff.
-->
Mongo SH, yep, so you can rerun whatever you want to run.
-->
I can learn after the delete, do I need to...
-->
Ah, yes, continue.
-->
I had Austin saying that we needed to copy something from...
-->
No, no, no, when you drop the database.
-->
So the next one, number two, the last command.
-->
Yeah, it's the DB.
-->
When you drop the database, you go to 4.1, then you copy that and piece it again.
-->
So, yeah, so for you faith, you still can be able to do 2.1 up until the dropping of the database.
-->
After the dropping of the database, then you go back to, what do you go this, you go back to number 1, 4.1.
-->
then you'll be able to read the data.
-->
Just a quick question.
-->
Yes.
-->
Is course a collection?
-->
So if I was to want to sort or find all the subjects in a course,
-->
will it be db.course.
-->
So if you log into your database and show collections,
-->
was it made part of the collections?
-->
So what you can do is just go to your terminal.
-->
Yes, what are you asking for the story?
-->
Okay, so I...
-->
Sorry, I didn't realize I'm on mute.
-->
Yeah, you were asking, you can continue?
-->
I was asking a course was a collection.
-->
Let's say I wanted to list all the courses.
-->
Will it be db.course.find and then that's what?
-->
I don't think course is a collection, but what you can do is while this year in the university
-->
DB, just do show collections and then be able to see if it is actually a collection.
-->
It would actually be a data that's an array that's in part of a document, right?
-->
So there's courses, there's students, right?
-->
So now when you want to find a specific course, you say DB.com.
-->
which is the collection name and then find whatever it is cost that you want I'm trying
-->
to list to see what courses are available before okay so just do db dot courses
-->
dot find and then your what you call this your brackets so just do your brackets
-->
and then exclamation yeah oh sorry sorry sorry not exclamation by semicolon
-->
Okay.
-->
This means this is the data that's there.
-->
So it's got a course name, it's got the credits, got the duration.
-->
What's her name, Colorfellow?
-->
You don't need to type Mongo SH again because you're already in Mongo SH anyway.
-->
So you just need to go down and recreate that script.
-->
Don't have to worry about the user.
-->
Just go and just create the collection first.
-->
Just go up a bit.
-->
You probably need to create a collection first, which is students, and then run that script.
-->
So you're already in the database, so you don't have to worry.
-->
So just create that, yeah, and then you can run that script.
-->
Yeah, you can paste.
-->
Hi Kumbuland, do you need to complete the 2.4?
-->
2.4. Yeah, you can do it if you want.
-->
It's just creating a user. So you can do it, that's not a problem.
-->
Ah, okay.
-->
Yeah, we haven't activated authentication, so it's just like adding documents that you're doing.
-->
you're doing you won't really see the effect of the user because we haven't got into the point of
-->
authenticating okay you know after running that's what are we looking at it looks like matrix
-->
what is that uh just enlarge that and then let's go to the top so it will show you different
-->
information uh just uh just up a bit so any creation strings that have happened so any access
-->
patterns, any metadata. Some of those stuff is not really important unless you have to be
-->
troubleshooting. And this is when you're troubleshooting deep, deep stuff, right? So it says
-->
here it's wired tiger, which is the type of storage that came with MongoDB. And then it also
-->
shows things like your auto commit, your backups. Did you even do any backups? Was there any
-->
increment backups that were done uh you can see where it says backup total modified incremental
-->
blocks with compressed data because you didn't do any backup you didn't do any incremental backups
-->
then it will show a zero right so it's just information related to different things in essence
-->
right um your block managers how many blocks have been allocated how many blocks are free um
-->
you know things like that file allocation unit size which is four gig so it's just statistics about
-->
a whole lot of things in relation to the university database, right?
-->
And to whatever collection you'd have put to say, I want this, I want stats about this specific
-->
collection.
-->
I go down a bit, some transaction, did you do any rollbacks, you know, any robbacks
-->
in terms of stable history store keys that would have been swept in non-dry run node?
-->
So it's, you can use it while it's troubleshooting, wanting to see.
-->
probably what could be happening with the specific collection, right, in terms of is there any
-->
backups, for example, if there was backups when last was it done, how many backups were done,
-->
when it comes to transaction, right? How many transactions were done? How many stable keys were
-->
removed? How many stable keys were restored? You know, it's that type of stats. But this would be
-->
when you're now doing advanced MongoDB, because now you want to, besides the performance level,
-->
that you're looking at you've managed to index your fields and all that stuff now you want to
-->
look deeper into the collection why is it there's always a problem with this specific collection
-->
when we're collecting data let's go have a look at that that's when you can be able to use this
-->
sharded it's not shattered we're going to get to a point where it's shattered the size in
-->
terms of um what you call the size of the file and then the count how many are they i think
-->
if you remember we're adding 2 000 students um total indexes
-->
size, as we said, what indexing does is the one that you index, we are saying that's
-->
the frequently access data or frequently create data. It takes all that information and puts it
-->
on the side. It creates a separate file where you've got the indexes so that when you need
-->
to create that information that's got the index, it doesn't really need to go and look for
-->
that information in the whole documents, right? Because when there's no index,
-->
what the database does, what MongoDB does is it goes and scans the whole collection, right?
-->
But if you've got, let's say, ID number as your index, it will just go to the indexing,
-->
look for that specific ID number, and then pull out that information from that specific ID number.
-->
It's not going to need to go into the collection and scan the whole collection
-->
to actually look for that one specific ID number, if that makes sense.
-->
Happy with that?
-->
I think I'm done on my site.
-->
all right cool um it's 12 o'clock anyway so we can take our lunch break and then we'll continue at
-->
one o'clock for those that are done okay thank um when where is done after doing
-->
number yes number two up until you get to 2.4 you'll be done okay yeah so if you're done
-->
2.4 then, yeah, you can take your lunch, then we'll be back at one.
-->
Excessively, with regard to allow those particular developments,
-->
and we have taken the decision to abandon the bail application at the stage.
-->
We will not be proceeding with the bail application anymore,
-->
I'm not quite certain with regard to the investigations,
-->
we've got to this method of a far away,
-->
and we would like, if possible, if all investigations,
-->
concept of reading computer, that the disclosure be given to us for the difficult
-->
ourselves of our ocean.
-->
However, we have come to the realization that for us to pursue a bit of the state,
-->
whilst there's another method ending that other side, would honestly be a futile exercise
-->
for us to attend a big application at the stage version.
-->
That is...
-->
Oops, cool.
-->
And we must see it as a...
-->
So morning, I had...
-->
...students, because what the guys who are working at Chill Us Punch are doing
-->
is absolutely insane on the market.
-->
She's succeeding only because of his huge audience.
-->
But if you think that way to just check out my Natalie video,
-->
it's one of my latest ones.
-->
But you need to understand who your biggest fans are,
-->
who your core audience is, so that you can figure out.
-->
so that you can figure out what to sell to me actually understand that his followers the people
-->
that actually engaged with his content are loyal diamond fans that are to be honest yeah so he calls
-->
him chillers after his podcast he made something for the chillers which is why the brand naming
-->
before we get into the market so you already had the fan base but the product itself anyone could say it
-->
saturated because you're already competing with these ones because they're in the same for sponge so he understood
-->
his audience he knew i'm going mass market i'm not going to
-->
luxury, I'm not selling something too expensive. That's what Granger is for. But Chilla's
-->
Punch is going to be something that my fans can't hate. And honestly, from one of the people
-->
looking at the product, he has perfected everything about marketing. The Majority Code he has
-->
is his pot. Who he's interviewing is even when people cut out clips for the podcast, they post on
-->
his videos. So he has a ton of social media reach organically. And number two, even with people
-->
the influence with Chillas Punch. When you think of Oscar MBO, we know that he's sponsored by
-->
do say, a podcast with Chiller's Punch. That video has like 40,000 likes, meaning people
-->
are viewing, liking, and seeing one of their favorite artists with this product.
-->
Another major thing he's done is that he wasn't afraid to completely remove himself or not
-->
make himself the sense of attention for Chiller's Punch, unlike some other alcohol
-->
farm. It's not just MacG. There's new flavors dropping this way to succeed on its own.
-->
And it's across the world. Watch Big Brother puts Chilers Punch in the contestants. It
-->
It gives it reach, it gives it legs, where he himself doesn't have to be a part of it.
-->
You can just search Chilas Punch and you see people tasting it, tasting the different
-->
flavors, buying six packs cases, showing that it's something that people want to try,
-->
want to make content about.
-->
Even if Maggi is not paying them, they don't even know him personally.
-->
Just because of the popularity of Chiller's Punch, it's being omnipresent without centering
-->
it around here.
-->
So being able to remove himself from the center of the brand, but have people make content
-->
about the brand. Have this podcast in every single celebrity who comes on. If people are interested
-->
in the celebrity, we see the celebrity with chill as punch. Being able to make connections with
-->
big brother, it's in huge distribution deals with places like Picking Pay, all of these things
-->
that he has done to push his brand to the point where even if he stopped talking about,
-->
the sales would still come. Because when you think about it, who is some other's main
-->
influencer? Who is the founder of black label? You don't need to know them in huge numbers. And for
-->
There was something that was lodged just in 2023 to accomplish so much in two years.
-->
Clearly, MacGee, nobody's doing nozoy...
-->
...first things to do.
-->
...theircings to do.
-->
Nobody...
-->
...and we can't see this...
-->
...and we sit in law and we should in law...
-->
...an amazing entrepreneur that I bumped into Usul Mwengue from a township called
-->
D near Barberton in Pomerton in Bumman.
-->
...he thanked me for a video I made a couple of years ago saying,
-->
please look into flying.
-->
Because sometimes flying...
-->
can cost almost the same as catching a taxi of us.
-->
And just recently I realized that some of the cheapest flights from Johannesburg taking out about
-->
$470,000, $650, what do I need you do?
-->
These ladies, if you're only in this coming Valentine, there's one thing that you're not
-->
going to leave out on the things that you buy me in.
-->
Why you buy me a car, or do you buy me whatever?
-->
What you buy me?
-->
If you feel like, you know, I'm...
-->
responsible.
-->
First of all, my community to work is not very demure because tell me why I wake up so early
-->
the money and make me to get to work very late.
-->
Well, this is my biggest problem is that I have to take an Uber to the heart.
-->
When we say, date somebody who is kind, we mean somebody who knows how to manage their emotions
-->
and is not going to like snap it to or speak to you in an ill manner because they do not
-->
to manage their emotion.
-->
who's empathetic and has compassion and is able to put themselves in other
-->
people's shoes you mean someone who is honest my friend and I went to this
-->
other meeting then after the meeting I decided that to myself or something to eat
-->
with who might treat you almost hijacked of work I'm pointed at me so
-->
instinct take over the training comes to the fore so you were in the 30s and you were just
-->
not at work I was not a work I was just traveling all that I guess I went to toilet
-->
but it's one of those guys are we back thumbs up if you're back i can see austin is back uh faith
-->
colophila are you back okay let's give them probably five minutes are we all back yes okay faith
-->
is back color fella are you back okay cool um okay anyone with questions with what we did earlier on
-->
before we went for lunch okay take silence and
-->
There's no questions, no question, that's fine.
-->
So let's continue with some theory.
-->
So now we want to get into monitoring,
-->
we want to get into the few tools
-->
that you can be able to use to monitor our database,
-->
our MongoDB.
-->
So firstly, let's talk about Mongo stat,
-->
which is a command line tool
-->
where it sort of provides statistics
-->
about the MongoDB instance, right?
-->
It's basic usage is just MongoStat and you can be able to then add some options.
-->
So for example, you can add a remote host.
-->
You want to monitor a remote host.
-->
You can add a remote host.
-->
Be mindful that when you add a host, you need to add the specific port.
-->
And then you can set refreshal intervals.
-->
How often do you want the statistics to refresh?
-->
It could be five seconds, it would be 10 seconds, you know.
-->
And in the case of this example, it's connecting to a remote host and it's
-->
It's going to display five rows of statistics, and it's going to refresh every one,
-->
let me take the pen, every one second, but it's going to showcase five rows at a time.
-->
And then you can also customize your output, right?
-->
When you customize your output, if you put the dash-dash-discover, then you can be able to specify
-->
which fields you want to be able to do.
-->
But we're going to look at it and be able to see what is it that it's,
-->
shows us. But the key matrix is that you need to look at, the metrics that you need to look
-->
at when it comes to Mongo stats is your insert, because then it's your insert operations per
-->
second, your query operations per second, your updates operations per second, and your delete
-->
options per second. So this information in terms of your operations, right, you can be able to
-->
see if there's something that is delaying, right? Let's say it's an update. You can then be
-->
able to ask yourself why is it an update is delaying right or if it's just a normal query it's
-->
just a query that needs to pull information why is it delay is it probably it's going through
-->
a scan is it probably the query that's running doesn't really is not querying any field that's got
-->
an index to it right you can be able to see all that stuff you can be able to get a number of
-->
commands executed per second per second this is in the instance where you've got um
-->
an application that's running that's got high what you call this db quitting that's involved you can also see the percentage of dirty cash dirty in the cash which means there is information in the cash that is not really sort of used so you call that dirty cash the amount of percentage used in regards to cash and then the number of flashes to disk per second and then some network in and network out right all these metrics
-->
can be able to determine, so it can point you to a few pointers, right, of what is it
-->
that you need to look at. If it's a networking issue, then your network traffic in might not
-->
be making sense as compared to the network traffic out, for example, right? Percentage of cash
-->
that's being used if it's about to be full, which means probably it's going to be slower,
-->
you know, things like that. If your delete is taking forever to happen, why is it taking
-->
forever to happen. Is it because it's going through a whole scan or is it because the query
-->
is not sort of referencing data that's not index? So those are some of the things that you can
-->
look at. Or could it be an input or could it be a disk I.O. situation where probably the disc
-->
is failing. So now the I.O. is now bad. It's now very, very high because you also want an IW
-->
that's very low. The read and write should be very, very quick rather than taking for a
-->
So it can point you, this matrix can be able to point you to the different sort of directions that you can be able to look at.
-->
That's usually the word to use Mongo stats for, right?
-->
And then wireless on that, you know, there is memory, right, for caching data and indexes.
-->
And the key metrics to monitor there is the working set.
-->
So the working set is the portion of data and indexes actively used by the application, right?
-->
When you go to the memory usage, right, there's that part where it actually has the indexes and the data that's actively being used by the application.
-->
It's very, what you call, it needs to be ready, right?
-->
And then you've got the cash usage, right, which is percentage of memory used by the wired tiger cash.
-->
Wiretiger is more or less a storage type that is used by MongoDB.
-->
Usually when you install MongoDB it comes with wire tiger, but then in essence, when you
-->
see the percentage of memory used by that, you can be able to determine, okay, is there
-->
problems, is their blockage in regards to what you call this, in regards to what you call this,
-->
the usage, right?
-->
the percentage is it too high with the memory that's available.
-->
Now you need to look at is it that there's too much dirty cache
-->
or you need to increase the amount of memory available
-->
because probably is now failing to handle, right?
-->
You can also check memory usage by doing your DB server status
-->
and then look for the bytes, right?
-->
Currently in the cash and maximum bytes configured, right?
-->
If you configured 2,000 megabytes, right, and your cache is sitting at 1,900, then you need to worry.
-->
Then where is the dirty cache sitting?
-->
Is it sitting on?
-->
Out of that 1,900, is it 500 megabytes?
-->
You need to worry about a high level of dirty cash, right?
-->
You can also use MongoStat to monitor your cash.
-->
As I said on the previous slide, that the key metrics is that you be able to see.
-->
see things like your dirty cache, your used cache, so you can be able to use MongoStat
-->
to be able to see the dirty-induced fields, right?
-->
Then let's look at some I.O performance, which is your input output, which is your read-write,
-->
right? It's a very critical performance when it comes to MongoDB because MongoDB is meant
-->
for high read-write operations, right? So because this data is coming unstructured,
-->
are mostly applications that have unstructured data read and write a lot, right?
-->
Because it's very, very unstructured, right?
-->
So the key metrics is that you need to look at is disc latency, the time taken for any read or write operations, right?
-->
And then the input, output operations per second.
-->
So disc latency is, how long does it take to read?
-->
How long does it take to write?
-->
If it is taking long, then there might be an issue with the disk itself, right?
-->
that could be another issue. Number one, number two, it could be it's writing a whole lot of data, right?
-->
So it also these are metrics that sort of give you pointers to where to look at.
-->
Is our storage, right? Is our disk, the hardware part of it, right? Do we need to change disk and upgrade to SSDs?
-->
If they're SSDs, then what could be happening? Let's look at the out of, what do you call this, the disk in themselves, right?
-->
and how do you check I.O. performance, you can be able to run your DB status and then
-->
Wyatt Tiger.com concurrent transactions. So your concurrent transaction will be able to tell you
-->
if there's a delay in terms of the writing or the reading, right? So you look at your read
-->
and write ticket usage. That's a way that you can be able to look at. And then you've got IOS stats
-->
which you use to monitor your disk I.O. It can be used to track your disk activity.
-->
It is sure that the performance, how is it performing and whatnot.
-->
Any questions on the monitoring?
-->
We have a practical for some monitoring.
-->
So any questions on that?
-->
Okay, cool.
-->
Silen says, no questions.
-->
Now, let's talk about also monitoring with the web console, right?
-->
So MongoDB has two versions.
-->
There is the enterprise version, which is a paid one.
-->
and then there's the community one.
-->
Now the community one doesn't come with a web interface.
-->
So you have to look at open source web interfaces that are out there, right?
-->
But when it comes to the enterprise version of MongoDB, right?
-->
It comes with MongoDB ops manager, right?
-->
And MongoDB ops manager or whichever one that you'll be able to use, it comes with features like real-time
-->
monitor so you can be able to see metrics in terms of real time such as your operations your memory
-->
usage your network activity you can also set up alerts for critical metrics right your high
-->
cp usage your low memory and that it can also help in terms of analyzing query performance right
-->
you can be able to see uh the slow running queries you can identify them if they need to be
-->
improved if you need to do any indexing to be able to improve if there are any things like um
-->
any summing or any aggregation that needs to happen.
-->
How can you then improve?
-->
And then you can also be able to do your backup and restoration from the web console, right?
-->
The real-time monitoring, alerts and performance analysis,
-->
you can also be able to incorporate external tools, right?
-->
Like your Prometheus and Grafana, your Nogios, what else,
-->
your new relic off the top of my head.
-->
I think UpDynamics or Datadog actually does that.
-->
it's got that part where you can be able to install it.
-->
So those tools also can be also be incorporated to be part of monitoring,
-->
and you also have that web console that you can be able to take your real-time monitoring,
-->
your alerts, your performance analysis, and all that stuff.
-->
Any questions on monitoring?
-->
Let's talk about indexing, right?
-->
And what is in an index, right?
-->
It's basically a data structure that helps in improving the
-->
speed of data retrieval so what happens is when you create an index using let's say if a field let's say
-->
your ID right it takes that ID field and places it somewhere out of uh what you call this out of
-->
the whole set of data right and what it does is it references anything that actually that actually
-->
uh what you call this that actually is related to that field so if you've got let's say
-->
ID number for someone. You work in a bank and they index an ID number. It means it's going to be
-->
easy. It doesn't really need to go through the whole scan of the whole collection to be able to
-->
actually, what you call this, to get that ID number. It just goes into the indexes, which is
-->
a separate sort of file, right, where that data is placed and then it's easy to retrieve anything
-->
that's related to that specific index, right?
-->
So how does indexes work?
-->
So it uses a B3, so a balanced tree to store indexes.
-->
So you've got a mother index and then you've got child, right?
-->
So you've got the mother, which is the index field and then child, child being anything
-->
that you're going to be able to pull in relation to the mother, right?
-->
And it contains a value from the index field and a pointer to the corresponding document.
-->
right and then when a query is executed it uses the index to quickly locate the relevant documents so because you're using ID number right for kumblan the moment you punch in that ID number it quickly rushes and gets because already it has references that lead to that document as compared if you don't index the ID number then it means it needs to come through the whole collection to look for the ID number amongst all the documents that are in there right and
-->
And then there's types of indexes.
-->
So firstly, there's the single field index.
-->
You're going to talk about it.
-->
But basically, that's how you create it.
-->
So which means there's a single field in it.
-->
Right.
-->
And then there's the compound index where you can actually have what you got this.
-->
You can actually have many fields, right?
-->
Multiple foods being where you can be able to index that.
-->
And then at the same time, you can also create what you call a multi-key index.
-->
A multi-key index is there's nested documents or an array, right?
-->
For example, subjects, you can index a subject because there's, under it, there's a whole lot of subjects that are in it.
-->
I think it was Coloferlo that was trying to list the number of courses, if I'm not mistaken.
-->
So this can be an example where you index courses, right, and be able to, you know,
-->
sort of it will be easy for you to be able to actually reference the courses or it could be
-->
subjects right where you've got um you're studying IT and then the subjects below if you index
-->
subjects it means that you'll be able to index everything that's under it which is an array
-->
which is a list of subjects or a nested um documents where nested documents is where you've got
-->
sort of documents that speak to each other all right and then you've got your text text index
-->
Let's say you want, it supports search queries on string content.
-->
You've got the name, you want to index the name, which comes as a text, right?
-->
So it then becomes easy for you to be able to search your text.
-->
You also have geospatial indexes where it supports your geospatial crease or coordinates.
-->
These ones are used mainly by your Uber's and your boats, your e-healing platform,
-->
or anything that's delivery related, they store geospital coordinates, data, right?
-->
Data where you can be able to use or you can be able to index coordinates, right, to be
-->
able to use them maybe for delivery or for pickup or for whatever.
-->
Then you've got what you call hashed index, right?
-->
This hashed index is mainly used for sharding when you're doing sharding, right?
-->
so you hash the um you index the hash of the field right the contents of a field and it would be in a hushed format
-->
right that's when you especially when you're doing sharding it's very very good but we'll get to
-->
the point where we actually do the speak about the sharding and then be able to explain this
-->
hashed indexing and then you've got ttl index where automatically you remove documents for um
-->
after a specified time, right, the document should expire after 3,600 seconds or after so many
-->
years or whatever, right? But usually TTR index is where you don't really need to store the
-->
document for too long, right? And you don't want that manual part of you having to log into
-->
the database to actually do some manual work. Then you can be able to do your TTR index, right?
-->
how do you manage index so there's the creation of the index right so it's db dot the collection
-->
and then the index operation that you want to do right so in the example there you've got db
-->
dot students students being the collection and then you want to create an index of the name right
-->
of understudent so when you go into the collection student when you look at it at the table on the
-->
relational database side it's a table then what you call this you want to use name as an index right
-->
you can also be able to list index which is get indexes and then you can also be able to drop index right
-->
you can also be able to rebuild an index right let's say there's some complications that would have
-->
happened you had multi-key or you had confound now you have to break them and now you want to rebuild the whole
-->
collection right it's a matter of just running db dot the collection name and then dot re-index right now
-->
some tree structure so tree structure is used to store indexes as i say right um which is a balanced
-->
trees that allow efficient in session division and search operations right and each node in the
-->
B3 contains multiple keys right and pointers to child nodes right so your key is your
-->
index and then your node is the document that you need to reference to right so indexes are
-->
also stored in a separate data file within the DB path directory right so you can set that
-->
up you can actually where you want your indexes to be you can set that up right and and each
-->
index entry contains two things contains the index filled value right
-->
Let's say your indexing ID, right, and then a point to the corresponding document, which means to whatever document it is that it sort of the index value is connected to, right?
-->
Then there's index selectivity where it refers to how unique the values in the index field are.
-->
They are always unique, right?
-->
They are very, very highly selective, which means they are unique values, which obviously results in better query performance.
-->
and then there's the low selectivity which has got many duplicate values but what
-->
tries to do is it avoids mostly when it's indexing avoid low selectivity where you've got
-->
what you got those many duplicate values right don't use that rather use high
-->
selectivity where there's you use unique values right unique values could be your
-->
user ID for example right it's going to be unique or
-->
email it's meant to be unique there's no way that you find two people with the same email
-->
and then they receive mail differently I don't think there's any of that setup it might
-->
differ just one part you know and then there's also what you call index cardinality where
-->
it refers to the number of unique value in the indexed field right so high cardinality
-->
fields are very very good things like your email your ID number passport
-->
number because they are unique they are never the same you never find two people with the
-->
same email you never find two people with the same passport number one has to be fake if you find
-->
somebody two people with the same ID number one has to be fake so those are the high card
-->
high cardinal um what you call this uh candidates for indexing right law cardinal ones
-->
are your gender you've got a million male right and you don't want to be using that because if you're
-->
going to index your gender, then you won't benefit that much. It will still need to scroll
-->
through the whole, if there's 1,000 male, right, it needs to scroll through that whole 1,000
-->
may. And then indexes also consume size, right? They also consume, because remember,
-->
when you create an index, it creates a data, right, that it separates from the main DB, right?
-->
and when you then reference that index obviously it's pointing to date so indexes can actually consume
-->
additional storage because there is the issue of the size of the index filled in itself and then the
-->
number of documents in the collection what indexing is meant to do is it's meant to bring whatever
-->
data close by so that it doesn't have to go through a whole scan when it needs to find a document
-->
So you can imagine bringing that data close by it.
-->
So it creates that file layers, that index information, and all that stuff, right?
-->
Some best practices for indexing.
-->
Frequently create fields are very much important.
-->
That's number one.
-->
The ones that are frequently created is what you need to look at indexing, right?
-->
And also create compound indexing, right?
-->
For queries that need to filter on multiple fields, but also
-->
do it wisely don't just do it you can't just index a field because just because probably you feel
-->
like what you call this it will need in my multiple and whatnot but is it frequently
-->
accessed that's number one number two do you really need to index it because sometimes it's
-->
not wise it's going to impact performance when it comes to indexing compound indexes and
-->
there's a whole lot of documents that it needs to go through or a whole lot of fields that it needs to go through.
-->
So you need to be careful about that.
-->
The other thing is don't over index, right?
-->
Because over indexing slows down your right operations, right?
-->
And it also consumes storage, right?
-->
Then monitor your index, right?
-->
It's very, very much important to monitor index, whether you're going to run your index stats or you're going to be having a tool that is going to be doing the real-time monitoring.
-->
you need to be very, very careful of that.
-->
And then there's a query that is covered, you know.
-->
The query that's covered is, you are saying that this query is basically based on, it's optimized, right?
-->
And in essence, you can be able to, let's say, index it as a whole, create a compound index,
-->
but for the benefit of performance in essence.
-->
So you need to use covered queries where, in essence, you can be able to use.
-->
indexes in a very wise way.
-->
Now, single field index, where you've got a single field as an index in a collection, right?
-->
It helps speed up your queries where you need to filter or you need to sort, where you need
-->
to aggregate, and then how does it, obviously, it creates a B-3 structure from the index field
-->
and then points to the relevant corresponding document, right?
-->
An example there is when you create an index of the name in the collection students, right?
-->
One indicates ascending order, minus one indicates descending order, right?
-->
And then some use cases is when you need to filter or sort by a single field,
-->
finding what you got is, all students is a specific name or sorting students by age.
-->
Right?
-->
And then you also have compound index where it says it's a combination of,
-->
multiple fields in a collection.
-->
It helps when you do your filter sort and aggregate also.
-->
And the order of the fields in the index matter, right?
-->
Queries can use the index if they include a prefix of the indexed films, right?
-->
An example being an index on name and age, right?
-->
It can be used for queries on name or name and edge, but not just age, if that makes sense.
-->
So in this order, you've got name and age.
-->
so you can use a query to look using the name or using the name and the age right as your condition but you can't just use the age right and then use cases when you need to filter or sort by multiple fields obviously for example finding all students with a specific name and age range and then geospital which is more or less for maps mostly for
-->
used for your Uber's and whatnot.
-->
Supposed two types, which is the 2D for flat 2D coordinates,
-->
your latitude and longitude, and then you've got two-d sphere, right,
-->
for your spherical geometry, right?
-->
Your geo-jason objects and all that stuff.
-->
How it works, it uses, it utilizes specialized data structures,
-->
which is, for example, geishing,
-->
where it indexes the geospatial data,
-->
and then queries can find documents near a point, right,
-->
or an intersection line, right?
-->
And then an example is obviously creating
-->
a two-disphere index on a location field.
-->
For example, the DB dot places being,
-->
places being the collection and then create
-->
index on location two-dispheres,
-->
type to this fear, and then the use case is
-->
location-based queries, finding nearby restaurants,
-->
for example, or finding all
-->
places within a certain distance from a point.
-->
So when you do your Google search, these are some of the indexing that really happens.
-->
Some best practices, single field indexes use for queries that you need to filter or sort by
-->
a single field, right? Compound when you need to sort by multiple fields, just better when you
-->
use for location-based queries, and then avoid over-indexing.
-->
can be very difficult to manage right there is the part of it slowing down some operations
-->
there's the part of it consuming storage but also there's the part where it can if you're not
-->
very careful enough right um it consumes storage it will take bigger space right and then also
-->
monitor your your index usage right then there's something called equity profiler right
-->
a query profiler basically is used to be able to identify suboptimal queries and what are
-->
these suboptimal queries these are queries where they take long time to execute they consume
-->
access CPU memory or disk i.o they retain large amounts of data unnecessarily and do not use
-->
indexes effectively right and some common causes of this is where a query needs to do a full
-->
scan right it needs to go through a whole collection or whole database
-->
right but mainly a whole collection right weight quiz an entire collection instead of using
-->
indexes and then an example is where you have to create non-index field right and then inefficient
-->
index usage where you've got queries that use indexes but still perform poor due to low selectivity
-->
the type of field that you've chosen there's many duplicate values right it's number one incorrect
-->
index order in compound indexes right where you're starting with the
-->
age and then the name for example in the example that we had right now it's going to save through
-->
everybody that's got 20 years to look for a name um kumbulani on it right and yet if you do vice
-->
versa to look for kumblani that's index and then the age is 20 so it's it becomes easy you know
-->
it doesn't have to go through a whole lot and then large results says where you've got quiz that
-->
written a large number of documents right where then causes high network and memory usage
-->
That's also another common cause of suboptimal queries and then complex aggregation
-->
pipelines where you need to do a whole lot of summing and this and that and that, you know,
-->
when they are complex, usually they have what you call this, they need more power to be able to run that.
-->
So those are some of the sort of common causes of suboptimal, what you call those?
-->
sub-optimal queries right and then how to use the query profiler it's it's an inbuilt mongo db tool right
-->
so it helps you identify your slow inefficient queries collect detailed information about that
-->
your execution time your index usage the scan documents and all that stuff so you need to enable
-->
the profiler right and enabling the profiler it it sort of has three profiling levels zero it's
-->
of default one is log slow operations right which are threshold in milliseconds and then two
-->
which is log operation so logging all operations would help in terms of when you really need to
-->
troubleshoot but you can leave it to say log slow uh operations and you need to set a profiling
-->
level right so for example here you've got db set profiling level to one and anything slower than a
-->
100 milliseconds. So you can set to say our target's time for query execution is 100 milliseconds,
-->
right? Then you run that and it should be able to tell you which ones are the queries that are
-->
running slow at the end of the day. And then you can also be able to verify profiling level
-->
by doing the DB get profiling status and then after that you can then be able to sort of change
-->
anything that you want to change or make changes, right? The profile data, right, is stored in
-->
the system profile collection. So if you want to see anything to be good to go into the DB.System.com
-->
profile. Find, and then you can be able to sort, right, with the limit of 10, in essence.
-->
Some key fields in profiler data, operation, what is the operation type?
-->
What is the namespace?
-->
Usually, its name space is constituted by the database and the collection.
-->
There was a time that we did statistics, I think it was Colorfellow.
-->
If you looked at the bottom, very bottom, there were lines, a few lines that were not
-->
part of the brackets.
-->
You would find that there is somewhere where it says NS, and it's.
-->
It will have the database name and the collection on it, right?
-->
Then execution time in milliseconds, and then some plan summary, the query plan, you know,
-->
call scan for collection scan, I-X scan for index scan.
-->
It will show you that keys examined, number of index keys examined or docs examined, the number of docs examined,
-->
and then number of documents retain.
-->
So you need to be mindful of that.
-->
Any questions when it comes to these topics?
-->
Okay, there's no questions.
-->
I think we can go to our practical and then move over to the next one,
-->
which is early.
-->
Where is my machine?
-->
Okay, there it is
-->
where is my window now?
-->
Cool.
-->
So we've done
-->
one, two, three
-->
as I said, it's going to be optional
-->
and then what we need to look at
-->
is the security part.
-->
Now, to do a bit of explaining,
-->
right, please be mindful of this.
-->
So this part, you don't need to do it
-->
because it's setting up of the Mongo
-->
db from sketch you've done that already right and then you come to the creation of mongo s h you've
-->
created a user earlier on but be also be mindful that on number two there's a time that you
-->
actually deleted the user as part of it are we together i think there's a time that you created
-->
the user and then it was deleted as of um exercise number two right so if you
-->
had deleted then you need to recreate now there's this part that I want to
-->
explain right so you see it says edit ETC mongo d.com right so what you then do
-->
is you go to your terminal log out of the DB if you are locked into the DB right
-->
and then what you need to do whether you use VIM or you use nano it's up to you
-->
I use VIM so now you need to go to the location ETC and then
-->
mongo dot mongo d.com right if you go into it it's got different subcategories so for example the db
-->
part is sorted already right this is where you store your data if you are using a specific
-->
engine you then can be able to put it there and obviously some details to it then where to write
-->
the logging data right the destination is the file append is equal to true so it doesn't need to
-->
remove some stuff, just needs to add on top. And then the path to it, right, which is your MongoD.
-->
log, usually most applications have got that log file sitting in VA log, right? So they'll
-->
have VA log and then create a file, a directory that will have a specific file, right? And you
-->
also have your network interfaces, and then how the process runs, process management. And then when
-->
you come here, when it comes to security, for you to enable security, you need to come and
-->
then move that part and then go below one space not one tab one space right and i think it's
-->
security if i'm not mistaken um authorization sorry authorization and then equals to two right this is how
-->
it should be right and then you save your file and then you need to
-->
to restart mongo for that to take effect right you need to restart mongo for that to take effect
-->
or else it would not take effect so you then come and then um be able to then restart that right which is
-->
good sometimes if it fails to restart all you need to do is just to do status because sometimes
-->
it would be an issue within your oops it actually failed okay that's weird
-->
MongoDB server let's see failed control error during global initialization
-->
okay status restart MongoDB let me have a look again
-->
Mongo.D so security is it true or enabled I think it's enabled yeah it's enabled that's why
-->
enabled and then restart that and then status okay it's happy now so just be mindful of that now
-->
after you restart it which means now anything that you're going to need to do is um is
-->
it's got authorization so you need to log on so be mindful i didn't create the user i didn't even
-->
create any user but in essence when you then need to log on right it's a matter of just um
-->
you don't need because you've already created the user admin you probably don't need to do this whole
-->
process um it's if you just do um this right authentication database admin it would be able to
-->
to ask you for admin because that's the only user that you have. But in the case where you
-->
have different users already in the database admin, in the admin database, then you need to do this
-->
and be mindful of the password that you used earlier on to be able to create the user. Happiness,
-->
understood. Are we understood with that? Are you?
-->
Hi, Kumbolani.
-->
Hi. Sorry. Well, no one's answering.
-->
You know, I joined later, so I'm trying to catch up, but there is a recording, so it's okay.
-->
Yeah, I'll check it out again.
-->
It's just I'm struggling to connect to where you are now.
-->
Oh, is it?
-->
Yeah.
-->
I'm sharing a screen.
-->
I'm not sure if you'll be able to see that.
-->
Okay, I can see that.
-->
Oh, we're not going on this link, eh?
-->
Only the other one that we're working on.
-->
Which one?
-->
The link that there's one that I sent in the chat, right?
-->
it's for you to create your virtual machine like the one that I have on the screen and then
-->
what you call this that GitHub link I'm going to share also so that you can be able to go
-->
and do it and then be able to follow number one two three as you go yeah okay okay then
-->
thanks all right no problem I actually wanted to explain number so when it comes to
-->
secure deployment this part really works when
-->
you've got three different machines.
-->
You're not using local.
-->
So the part of the enabling of the SS is the binding to local.
-->
It's already done.
-->
But then the enabling of SSL or TLS,
-->
you don't really need to do it because it's just one machine that we're using.
-->
So it wouldn't be right.
-->
But the rest of the stuff, I think people can be able to do it.
-->
Just there when you were explaining,
-->
I think you got quiet because of you're going too fast.
-->
when you're explaining where to authorize it, are those steps included as part of this?
-->
Say that again?
-->
No, I'm saying the part where you're showing us now whereby you're including,
-->
as part of the situation, you're including authorize, is this part of the steps that we have here?
-->
Yes, it's part of the steps.
-->
Now, the question would be on number one, there was creation of a user, right?
-->
on number two there was a part where you create a user and deleted it now if you created the
-->
user and deleted it on either number one where you created the user or created the second user and
-->
you deleted it on number two then you still need to recreate the user here right so you do your
-->
mongo sage you create your user and then you enable your security within the mongodd.com
-->
file right and then restart your mongo db and then you'll be able to use this
-->
got it okay now got it okay got it so for sorry just one thing for number three days so you're
-->
not doing that the web console the the web console takes a bit of time to install that's one thing
-->
yeah I would rather you guys can do it at your spare time whether it's later on in the week or
-->
whatever because you still have access to this virtual machine I think for a few weeks if I'm not
-->
mistaken. If it's not a month or two months, I'm not mistaken, but you can always do it on your
-->
own, on your virtual machine. It takes a lot of time to set up. That's why I sort of left it on the
-->
side. Yeah, that's why I said number three, don't really do it because it takes a bit of time. That's
-->
number one. Number two, ops manager is easy to install, but it's enterprise, which means
-->
there is pain that's involved. Now, when you then go to the community edition and you want
-->
install the open source of it they are not easy to install that's why i'm saying it takes take some
-->
time on number four the only one that you need to skip is this part the networking part why i say
-->
networking or this part is because when you use tls and ssl authentication you need to use it amongst
-->
two uh nodes right mongo db nodes where you've got it's it's not one machine like this it's not on local
-->
host it's got different other machines available and you want to add that machine to the cluster
-->
that's when you can be able to use your TLS because then there's that secure communication
-->
between the nodes in the cluster that's why I'm saying you don't really need to do it because
-->
you won't be able to see the effect of it if that makes sense okay I know that makes sense
-->
yeah but the rest of the stuff you should be able to do it so if you do number four and then
-->
When you go to number five, it's creation of what you call this indexes.
-->
Geospatal, you might not be able to get any results because we are not doing any location-based,
-->
but it's to help for information purposes to know how it is done.
-->
And then you can then be able to look at the rest of the staff, sub-optimal queries, looking at that, profiling,
-->
analyzing the profile data and some advanced techniques in terms of,
-->
covered indexes running cover query uh and that should be all and then after you're done with
-->
number four and number five you can then look at exercise day one i'm just going to use the bathroom
-->
quickly i'll be back in a sec uh quick question guys are we supposed to install um we're supposed
-->
to set up uh mongo dpubuntu because i'm getting an area yes you're supposed to
-->
but did you exit the database first if you're still showing you
-->
the city database you must exit the database first oh that will explain there how
-->
it yeah just say exit and you just do that should be fine oh okay yeah because i was just
-->
getting some error okay exit anyone with a question or getting errors or breaking stuff
-->
Faith, you don't need to set up again, MongoDB.
-->
Don't worry about that.
-->
Because it's already set up.
-->
So you can...
-->
Okay, should I...
-->
Yeah, you can go from the end, from number two.
-->
So he's setting up doesn't need...
-->
You don't need to worry about that.
-->
Okay, so we can skip the first...
-->
The number one, yes, and then go to number two.
-->
Where you've got the authentication part here.
-->
Who is it?
-->
Asanda?
-->
Dylendai said, don't be disturbed.
-->
About what?
-->
Go and sleep.
-->
Go and sleep, your time is slow.
-->
I'm getting an error D-212.
-->
Let's have a look.
-->
But after the security, right?
-->
Yeah.
-->
Enter, let's see something.
-->
Let me see where it's true.
-->
You press enter.
-->
I almost like it's fine.
-->
I didn't get that error.
-->
So you're able to quit and write?
-->
Why did you?
-->
Please press enter.
-->
Wait, are we supposed to unhash the security there?
-->
Yes.
-->
Oh, okay.
-->
And then do escape, quit without saving.
-->
I want to see something quickly.
-->
That's you, Colophila.
-->
Use the overwrite, so use the cue and exclamation.
-->
They are not pseudo.
-->
Yeah, that's why.
-->
It should be good to go now.
-->
When I get to, after starting.
-->
Who is that now?
-->
It's cool fellow.
-->
Okay.
-->
Let's see.
-->
see. Okay, go let's see. You've authorized that, that's fine. Have you been running, have
-->
you been running Mongo SH as student? All this while. Okay, I did sue our menacea, yeah.
-->
Did you, and I restarted it, right? Just go up, just go up again. Uh-huh, and the status
-->
says everything is okay, correct?
-->
So just do a status.
-->
Okay, everything is fine.
-->
Control C.
-->
And then try and log in from here.
-->
Let's see.
-->
Mongo S.H minus U something, something.
-->
Then enter.
-->
Authentication failed.
-->
Okay, just remove minus U admin minus P, secure admin.
-->
The password and that.
-->
Just, yeah, remove that.
-->
Even the user, yeah.
-->
Remove the user.
-->
Enter.
-->
So there's probably something.
-->
with your password?
-->
I'm assuming so.
-->
Good. Okay.
-->
Yeah. So if you created a user from number one, I think the password is actually different.
-->
The other password should be, I think, admin 1, 2, 3, if I'm not mistaken, on number 1.
-->
So if that user was still there, then it would definitely won't be able to log in.
-->
But now, because you're not specifying the password, then it's just going direct and just looking for the
-->
admin user.
-->
Just a reminder, Lisa, we can skip the network straight and right?
-->
Yeah, you can skip the networking part.
-->
And then go on to number one, so on.
-->
Yeah, and continue.
-->
I'm sorry.
-->
So after skipping network, we go away, we went to four, we go to four, right?
-->
Yes, number four.
-->
Yeah, because I'm by number four, then I've run the first one and
-->
let me just go through it again here.
-->
I've run the first one here whereby, uh,
-->
by the terminal.
-->
Yes.
-->
Right, open.
-->
Okay, I've run this one.
-->
The first one, where is it?
-->
Yeah, this one.
-->
I think this is an output, so, but it has a set
-->
as the output there.
-->
Are we supposed to run that output that has an inset in it?
-->
Inset screen is a date that command date used.
-->
No.
-->
So log out of your database.
-->
Look out of your database.
-->
Exit, just do exit.
-->
just to exit?
-->
Yeah.
-->
No, I'm typing.
-->
It's not showing.
-->
Okay.
-->
So just read it sometime.
-->
Refresh it.
-->
Disconnecting.
-->
Okay.
-->
Refresh the page and then exit from there and then run it.
-->
So what you can also do is remove the, what you call this, just run Mongo stats and that's it.
-->
You don't need to point it anyway.
-->
So as a start, just run Mongo stats and that's it.
-->
Stats and that's it.
-->
Okay.
-->
Yeah.
-->
Okay.
-->
So let me try.
-->
Let me refresh the piece.
-->
All right, that's fine.
-->
Colophila, you're not supposed to run that Mongo stats when you're in the database.
-->
Please exit and then run it.
-->
What's her name again?
-->
Zandri, let me share the link.
-->
You don't really need to log into GitHub.
-->
Okay, thanks.
-->
Are you able to type?
-->
So are you able to tap on mine here because I'm trying again.
-->
I just reset it.
-->
It's not doing anything.
-->
Okay, just one second.
-->
I'll get to you just now.
-->
If, Zandra, if you can just copy that and then paste it into the web browser for the virtual machine.
-->
Then you should be good to go.
-->
And let's see, interactive.
-->
Oopsie, where did I go now?
-->
Okay, there we are.
-->
Interactive and...
-->
Oops, what's going on?
-->
You broke a machine, my friend.
-->
Oh, but you said you can break it, eh?
-->
Yeah, you can break it.
-->
So...
-->
You can always restart it.
-->
Is it?
-->
Yeah, we can restart it.
-->
Just one second before we restart it.
-->
Let's see.
-->
you and then so we can close this terminal basically because I also don't know
-->
why it's hanging so I was saying this right here okay so I was saying this right
-->
Where is that now?
-->
So I was saying mongo stack.
-->
Why?
-->
Mongo stent.
-->
Wait.
-->
Did you reboot this machine at some point?
-->
No.
-->
That's weird.
-->
When has it been running?
-->
Yeah, it's been from 12.
-->
Yeah, 12 or 8?
-->
Yeah.
-->
But also remember, I think this servers are.
-->
These servers are two hours ahead, if I'm not mistaken.
-->
I think they use the UK time.
-->
Yeah, but if that's key that news, then it's just started now, yeah.
-->
It's two hours.
-->
Which is very weird.
-->
Is this the password?
-->
Is that the right password?
-->
Yeah.
-->
Okay, cool.
-->
It's the correct password.
-->
So what was the issue there?
-->
Why was it like that?
-->
Why was it hanging?
-->
Honestly, I don't know.
-->
I don't know.
-->
I can't tell you anything because I didn't do any troubleshooting to it.
-->
But yeah, so you can see number of insets, queries, update, delete, get more commands, dirty, used, flashes, you know, and all that stuff.
-->
Network, incoming, network outgoing.
-->
There is constant, so you can't really say you would worry that much about it.
-->
Yeah.
-->
But that's basically how it will show or you can do this and then.
-->
five rows, one second each.
-->
Oops.
-->
I need to see the time.
-->
I need to set two rows.
-->
It's doing one second, two, four, six.
-->
So you can be, oopsie.
-->
So version config, path to a configuration file, host,
-->
any SSO, any authentication,
-->
KBOROS, URI,
-->
starts option.
-->
So you've got minus O, field to show, or custom that.
-->
Row count, discover, discover nodes and display stats for all, if there are any.
-->
Use HTTP instead of raw DB connection.
-->
All option now fields.
-->
Output to JSON, it must be interactive.
-->
So let's say, if we do dash dash all.
-->
Okay, no release the same thing.
-->
to check if they're
-->
you're moving a little too fast now
-->
if I was a story. Okay, I'm coming there.
-->
Okay, I'm coming.
-->
So if let's say there was
-->
many nodes, right? If it was a cluster, then it would
-->
show each and every cluster, information about each and every
-->
cluster, if that makes sense, right?
-->
So here, because it's just local host, it's just going to show us local host.
-->
But let's say if there was three other servers, then it would list all those and be able to tell the stats of each of them, if that makes sense.
-->
Yeah.
-->
Who are saying they are stuck somewhere?
-->
The Mongo stats, the creation was a failing.
-->
So I'm getting an authorization failed.
-->
Okay.
-->
All right.
-->
I see faith got it wrong
-->
I tried to follow what you were helping
-->
Austin with I'm still
-->
sued all right that's fine
-->
I didn't really start yet
-->
okay but what
-->
I tried removing the credentials
-->
okay just hold on
-->
what you can also do is because
-->
we're not sure of your credentials
-->
remember
-->
oh it's on
-->
it's on view only
-->
let me change you to interactive
-->
remember
-->
there was a time that I said
-->
just use admin, just, okay, where is my scroller, just do it like this, okay, cool.
-->
Because the password, right, we're not sure if you use admin 1 or you used secure admin,
-->
we need to check that for the purpose of continue, okay, why is it failing now?
-->
mongo starts that authentication failed okay let me do this um i'm gonna switch off authentication first let me just go back