2:05:45
2024-05-20 09:46:48
2:09
2024-05-20 12:30:32
2:41:18
2024-05-20 12:33:23
1:36:58
2024-05-21 08:00:54
5:24:36
2024-05-21 10:06:11
3:24
2024-05-22 06:36:04
9:25
2024-05-22 08:03:05
40:22
2024-05-22 08:14:12
2:49
2024-05-22 09:47:03
1:48:29
2024-05-22 09:50:24
1:57:28
2024-05-22 12:09:49
Visit the Apache Nifi GROUP 2 course recordings page
WEBVTT
-->
I'll give you just a minute, but no rush.
-->
Let's see, are you able to type now?
-->
How's it looking now?
-->
Oh, there it is.
-->
Let me know if you're able to connect in and use your keyboard.
-->
While he's working on that, it looks like everyone else, I kind of followed along.
-->
Everyone is up and running on the 9.5 canvas, which is really great.
-->
Again, I was thinking about having it already installed, but I figured the install, the downloading of the zip,
-->
extracting the zip file, those types of things.
-->
I felt like we could walk through it, so it was good to see.
-->
I see some of you have already started to create processors and chain those together.
-->
That's amazing.
-->
We're already a little ahead of the curve, so we might get out of here a little early today.
-->
But definitely looking good.
-->
Well, Daris, if you can give me a thumbs up, just give me a shout when you're ready.
-->
Are you able to type yet?
-->
Right now, one of my coworkers is helping me troubleshoot.
-->
Okay, perfect, perfect.
-->
I'll give you just another minute.
-->
While we wait on that, did anyone have any issues installing NaPy?
-->
I know the zip was already downloaded, but you can go to the website, click download, and it will download.
-->
Did anybody have any issues getting everything up and running, going into the logs, finding your username and password?
-->
Again, I'd recommend you copy that username and password and put it in a separate document,
-->
just so you have it for later today and tomorrow and the next day.
-->
Yeah, I had an issue with a Java home environment variable issue, but I was able to power through that.
-->
Yeah, you're going to get that in the logs just because we're just using Java that is downloaded from Oracle's website.
-->
And so it gets installed, and it's a JRE, it's not even a JDK.
-->
And so it doesn't have Java home set properly and NaPy home set properly and all those.
-->
It should have complained about it, but still run.
-->
Okay, perfect.
-->
The Java home environment variable is not defined correctly.
-->
Instead, the path will be used to find the Java executable.
-->
If you are deploying this and building this into a reward environment, you would have a NaPy home, a Java home,
-->
those types of things.
-->
One thing to also keep in mind, NaPy supports Java 8 and Java 11.
-->
And so if you have a different version, like 13 or 9, it's not supported.
-->
But again, all of these instructions are on the administrator guide website.
-->
If you look at the administrator guide, it goes into how to start, stop, how to build,
-->
some of the port configuration that we talked about, zookeeper information,
-->
also some configuration best practices.
-->
And I usually include this link I send out as well because I've seen these best practices not followed.
-->
And NaPy likes to have a ton of open files and a ton of TCP sockets open.
-->
And so if you are running on a Linux machine and this is not configured properly,
-->
it will start spitting out some errors and some of your data flows will start having issues and things like that.
-->
Just some of those best practices.
-->
And again, it recommends anti-virus exclusion on these folders just because they're constantly running.
-->
And not only would anti-virus constantly running on that, it would take a ton of resources.
-->
NaPy isolates this pretty well.
-->
So the risk of not scanning those directories is very low.
-->
Now, there is processors.
-->
I know a couple of virus processors that when they ingest files,
-->
it will go through a Clam AV or similar processor to check for viruses.
-->
But yeah, all of that is in your administrator guide.
-->
If you are a sysadmin and you've set up Java homes and stuff like that previously,
-->
you'll see that as well.
-->
I am looking at mine. Let me pull.
-->
Well, there it is real quick.
-->
Actually, I can start your address.
-->
I can actually log into yours interactively as well and see if I can type.
-->
My typing works.
-->
Yes, I send a message to close the machine and try to open.
-->
Just connect to the other stuff with a different browser in general.
-->
Yeah, I'm thinking the same.
-->
Glad you're on the call for tech support, tech assistance.
-->
Give me another minute and see if you're able to get started.
-->
Give me one more minute and see if you need help.
-->
Just keep trying to get connected.
-->
If it doesn't work, I think it was suggested to shut it down
-->
and shut down the machine, close the browser, try a different browser
-->
and connect back in and start it up.
-->
Tom, are you there helping him?
-->
No, I'm teleworking.
-->
Oh, it looks like he's working now.
-->
Good deal.
-->
Now he'll join the call.
-->
I think he's good to go.
-->
Hopefully he's coming back on teams.
-->
Either way, we will get started.
-->
OK, so it looks like everyone else got their NIFI install up
-->
and running and working.
-->
Tonight or this afternoon when we are complete to exit NIFI,
-->
all we're going to do is bring this window up
-->
and we're going to hit Control C.
-->
We're not doing it here right now,
-->
but we're just going to hit Control C to exit the process
-->
and NIFI will go away.
-->
And then that way we're just, you know,
-->
if you don't have to, you can leave it running.
-->
These virtual machines are available to you
-->
for the next few days, you know, as much as you need them.
-->
So you don't want to shut it down
-->
and you just want to, you know, pause where you're at.
-->
You can just close your browser, come back tomorrow,
-->
connect and pick up right where you left off.
-->
So that being said, I am going to go ahead
-->
and finish building out my data flow.
-->
And you are more than welcome to follow along.
-->
I am, you know, it's going to be a lot of hands-on
-->
from this point forward.
-->
So, you know, feel free to, you know,
-->
build as much as you can.
-->
Some of the scenarios,
-->
I have a couple of scenarios for us to work through,
-->
you know, probably starting either later today
-->
or tomorrow morning.
-->
They will take a few hours to build,
-->
but you are going to be able to build a data flow
-->
that does ETL steps, controller steps,
-->
and those types of things from scratch.
-->
So, you know, that's always a good thing.
-->
But let's get started.
-->
So, you know, let's pick up where we were.
-->
Oh, gee.
-->
So, what I did is I, you know,
-->
I created a processor, a Git file processor.
-->
My goal is I want to take this data.
-->
This weather data zip file.
-->
Or I can take the other data.
-->
And I want to unzip this data.
-->
You know, I've already went through
-->
and extracted it manually,
-->
looked at the CSV, the JSON, stuff like that.
-->
But, you know, in this scenario,
-->
I am delivered a zip file to the file system,
-->
you know, every day.
-->
So, I need to...
-->
I just heard of a Teams notification.
-->
Oh, there it is.
-->
Good deal.
-->
Okay.
-->
And so in this scenario,
-->
I want to pick up, you know,
-->
some data from the file system.
-->
I want to identify the type of data it is
-->
because depending on that,
-->
you know, it's the way I might decompress this data.
-->
And so with that being said,
-->
I'm going to take my...
-->
I'm going to create a new folder.
-->
And all I'm going to do is call that weather data.
-->
And then I want to take the weather data zip file
-->
and I'll just copy it,
-->
move it into the weather data folder that I created.
-->
I'm doing this so, you know,
-->
we could actually pick it up from that directory,
-->
but then we have to start doing some file expressions
-->
or some regex and things like that.
-->
So to keep this easy,
-->
I want to work off of this directory in my git file
-->
and I am going to pick up this zip file.
-->
So I'm going to go back to my git file.
-->
I'm going to configure it.
-->
And I'm going to configure it
-->
to pick up everything in this weather data file.
-->
And we went into the configuration of this git file.
-->
So there is a file filter of regular expression.
-->
So if you are familiar with regex,
-->
especially Java regex,
-->
you can put in regex patterns to only pick up
-->
certain extensions, for instance.
-->
So we could do whatever dot star right now,
-->
but I can do whatever dot zip.
-->
Or if I know the name of that file coming in every day,
-->
I can put weather data dot zip if that was the name of it
-->
and that's the only file it's going to pick up.
-->
Right now I have it as a wild card
-->
where it will pick up any amount of data,
-->
any data that comes into this folder called weather data.
-->
And so right now I just have my one zip file there.
-->
What I want to do because I'm testing this
-->
is I want to keep that source file.
-->
Later on, if after I test this flow
-->
and things are working and that type of stuff,
-->
I may just have it where it doesn't keep the file.
-->
Because either I'm pushing this file to a database
-->
or a bigger file system or whatever.
-->
If you have that type of requirement,
-->
you may not want to keep the source file
-->
just to save on disk space and those types of things.
-->
But for testing purposes and because it's my initial flow
-->
I'm building, I'm going to go ahead and keep it.
-->
I'm really not messing with anything else.
-->
I have my input directory, my file filter.
-->
I'm not really worried about my batch size,
-->
the maximum number of files to pull in each iteration.
-->
Right now I have it set to 10.
-->
I've got one file so it really doesn't matter.
-->
But you can start to see some of the power
-->
and some of the capability in a single processor.
-->
And I can specify these things.
-->
I can build these things relatively easy.
-->
And again, I'm not opening anything code related
-->
like Visual Studio Code or Eclipse or any other IDE.
-->
I'm just using the non-fine canvas
-->
to build out my data flow.
-->
I don't want to ignore hidden files and folders.
-->
How often do I want to pull the folder,
-->
any kind of age or size.
-->
But honestly I think I got it the way I wanted.
-->
I got it the way I want it.
-->
I want to double check here.
-->
Yep, there's my weather data, file filter, batch size.
-->
Okay, everything that I want,
-->
I want to keep the resource file applied.
-->
So my goal with this is I want to pick up compressed files.
-->
Now, that could be zip files.
-->
That could be Gunzip.
-->
That could be TAR.
-->
There's numerous different compression type technologies.
-->
So we primarily use in our day-to-day life zip.
-->
But you may have others like it was mentioned with TAR,
-->
Gunzip, Rare.
-->
There's so many others out there.
-->
But in this scenario, you got zips
-->
and that's all I'm picking up.
-->
But this folder could have 100 different compressed files
-->
and all different compression technologies,
-->
different file names, different extensions, things like that.
-->
And so what I want to do is I want to pick that up.
-->
And because it could be anything but a zip file,
-->
I want to send it to another processor
-->
that identifies the MIME type.
-->
And so again, what I'm going to do is I'm going to go in here
-->
and just make this a lot easier to understand.
-->
So that way, if my colleagues are looking at this,
-->
they will easily can reference this
-->
and understand what's going on.
-->
Thomas, I think you got your hand up.
-->
Yes.
-->
What relationship did you use on the git file folder processor?
-->
The relationship?
-->
Try or terminate?
-->
Yeah.
-->
Yeah.
-->
No.
-->
Good question.
-->
So let's go back into the configure
-->
and let's go relationship.
-->
So this processor only has success.
-->
So if it can't pick it up, it can only pick up files it can read.
-->
And so if it can't read them, it doesn't know that they're there.
-->
Now, we will get into some additional relationships
-->
with other processors and where we send failures
-->
and things like that.
-->
But for this one, you only have success.
-->
So all files are routed to success.
-->
You don't need to do anything for terminate or retry.
-->
No.
-->
You don't need to.
-->
So the reason we have this here, previous NAFA,
-->
the NAFA that we built originally,
-->
you had to terminate or retry every single connection.
-->
So on a git file, I would have to create a new,
-->
push it to another processor that did terminate flow file.
-->
And that used to be a processor called terminate flow file.
-->
So what they did in some of the later versions
-->
in the last couple of years is have auto termination
-->
just because it makes sense.
-->
And it's less processors on the canvas.
-->
It just makes sense.
-->
But we'll get into more about the relationships
-->
because we're about to actually use those.
-->
But when you drag your arrow over to your next processor,
-->
it's only going to give you one option.
-->
So one of the things I like to do is log everything.
-->
So I am going to take, drag this over,
-->
and the only relationship I have is success, add, done.
-->
And then my log message, in this scenario,
-->
I'm going to have it automatically terminate.
-->
Oh, it actually is going to automatically terminate
-->
everything.
-->
Why?
-->
And that way, if there's any issue,
-->
it's just going to automatically terminate it
-->
right there.
-->
And the log message is another one of those
-->
that only has success.
-->
It's either going to log or not.
-->
And if you are not logging the information,
-->
you have a problem outside of NiFi
-->
just because that's this only job of this processor
-->
is to log whatever you send it.
-->
So what I'm going to do, actually,
-->
is because you asked that question,
-->
I'm going to have a log message.
-->
And that log message is going to log all of my success.
-->
So what I'm going to do.
-->
And you'll realize if you're working off your local machine,
-->
it's so much nicer not to deal with any of the latency.
-->
So I'm going to call this my success block.
-->
And when I have another relationship,
-->
it's either going to be success or failure,
-->
but I'm going to log it.
-->
So in the NiFi app log,
-->
you will see a success message for this log,
-->
for this file.
-->
And let me see.
-->
Let me figure this.
-->
All right.
-->
I want to change the color.
-->
Let's give it a green.
-->
Green of some sort.
-->
All right.
-->
I mean, I wanted to ask because you identified the file
-->
and the log message, I get a relationship success.
-->
Is it invalid because relationship success
-->
is not connected to any component,
-->
and it's not auto-terminated?
-->
Yeah.
-->
But it doesn't look like yours says that.
-->
OK.
-->
Just a second, and then we will.
-->
All right.
-->
I'll log through this, and Tom,
-->
I'll take a look at yours as well.
-->
So I have my Git file,
-->
and again, the only relationship on that one is success.
-->
There is no failure.
-->
So I'm sending the success to the identify mine type
-->
because in this scenario,
-->
I want to understand what type of compression is being used
-->
so I know how to decompress it.
-->
And so also I want to send all of my messages
-->
that are successful to a log message processor.
-->
And so this log message processor,
-->
all it's doing is logging all the messages.
-->
So I'm going to put logging all...
-->
logging all success messages.
-->
All right.
-->
So now my Git file has picked the file up.
-->
It is now sending it to log all success messages
-->
as well as identify mine type.
-->
I have now created a connection to the next processor.
-->
We can actually click on this connection,
-->
right-click and configure as well,
-->
and go to details.
-->
You can actually name this as well to make it more readable.
-->
You can say, you know, connection as well as prioritizers.
-->
So, you know, I can send files to this connection
-->
and instead of a FIFO first in, first out method,
-->
maybe I want to process, you know, a priority attribute
-->
or I want to do all the older files first
-->
before I do any of the newer files.
-->
Or I really just don't care and I want to send...
-->
The first one that comes in is the first one that goes out.
-->
So there is some configurations there.
-->
You can configure this.
-->
Most of the time I see a lot of people just leaving it as is.
-->
But it is there.
-->
Okay.
-->
So now I've picked the file up.
-->
I've built a connection to my next processor
-->
that is a success.
-->
And that processor is identify mine type.
-->
Well, that one, when I go to configure it,
-->
by looking at relationships, it's also a success as well.
-->
And we'll get into more of why that is.
-->
So, you know, if it identifies the mine type,
-->
it will send it, you know, where I want it to be sent.
-->
But if it doesn't, it's still identifying it.
-->
It just didn't identify it.
-->
So the processor itself still worked.
-->
Still done what it's supposed to do.
-->
It just couldn't identify unknown files.
-->
And I'll show you how we deal with that.
-->
So I've got my identify mine type.
-->
And that looks like it's going to be a zip file.
-->
You know, I'm expecting zip files.
-->
I'm expecting tar files, gun zip, rare files.
-->
So I'm going to need to decompress this file.
-->
Okay.
-->
You see we have a failure and a success now in our terminate.
-->
But let's go to the properties first.
-->
So instead of doing a compress, I want to decompress.
-->
I'm going to tell it to use the mine type attribute.
-->
So when it goes through, it is going to identify the mine type.
-->
And in this case, it's going to look at that file and say,
-->
hey, this is a zip file.
-->
So it's going to give that attribute.
-->
Remember, it's a key pair value.
-->
So it's going to use the mine type attribute.
-->
And it's going to say zip.
-->
And then if I want, I can update the file name.
-->
You know, I can deal with removing extensions, stuff like that.
-->
But for this scenario, I'm just saying, you know, false on the update file name.
-->
All right.
-->
So I need to drag my connection.
-->
I'm only going to identify mine type, you know, has that one success.
-->
There's only a success on the connection in this scenario.
-->
So, you know, name success.
-->
And so the success coming from identify mine type is going to compress content.
-->
You know, just for sake of use, I want to also want to log all success.
-->
And what I'm also going to do is I'm going to do a control C, control V.
-->
So I did a copy and paste on this processor.
-->
And I want to say control C, control V on my label.
-->
And I want to name this one failure.
-->
And I want to change the color of this one to red.
-->
All right.
-->
And then I want to change this processor to make sense because it's no longer log
-->
all success.
-->
It's now logging all failure.
-->
And you'll see why I put this in as well, you know, towards the end of this flow.
-->
For design, you know, following design principles and not thought and, you know,
-->
knowing what I know about building flows, I like to have a few log messages on my
-->
canvas while I'm building my flow files.
-->
The reason being is I usually actually don't even turn them on.
-->
But this is a catch-all method of seeing what is successful, what is failure,
-->
and what those look like.
-->
And we, you know, this exercise is to show you, you know, some of those
-->
additional, you know, menus and capabilities and stuff like that.
-->
One second.
-->
Okay.
-->
Sorry about that.
-->
My roofer was calling me.
-->
So what I like to do is leave those messages in there.
-->
And then as I'm building this flow, I can just send the, I have somewhere to
-->
send success, I have somewhere to send failure, you know, not only to the
-->
next processor but to a message itself just so I can see what is going on.
-->
So, anyway, so identify MIME type only has success.
-->
We brought it down to compress content.
-->
So what we've done is we've picked the zip file up.
-->
We're identifying that's a .zip.
-->
And then we're extracting, you know, then we're going to send it to
-->
this processor to extract that data.
-->
And so what we did is we went in, we configured this.
-->
We told it that we want to decompress the compression format.
-->
We can specify the compression format.
-->
We can put, you know, Gzip or zip or, you know, they're snappy.
-->
There's so many different compression formats.
-->
So, you know, I can specify that it's a zip.
-->
But who knows, right?
-->
Maybe I'm looking at a folder that is getting, you know, data from all over
-->
the place and different formats and different compressions and those
-->
types of things.
-->
And instead of building a flow for each format, right now I can just
-->
send it to identify the file type.
-->
And depending on that file type is how the next processor will
-->
decompress it.
-->
And so, you know, I'm going to set to decompress.
-->
I'm using the file type.
-->
Let me get a good name for this.
-->
So let's do decompress content.
-->
And we'll work on relationships next.
-->
So now I've got it where I'm sending it to my decompress.
-->
I'm expected a bunch of flow files, you know, from that
-->
one zip file.
-->
What I want to do is if it fails to decompress, because, you know,
-->
identify my type is just going to identify the file type.
-->
If it's unknown, that's a file type for identify my own type.
-->
So it doesn't have failure, but this one does.
-->
So if the file is corrupt or it doesn't know how to handle it or,
-->
you know, those types of things, it needs to know where to go.
-->
So we're going to send all of our failures to our general log
-->
message here.
-->
All right.
-->
So now that we've got that, where are we going to send the
-->
success, right?
-->
So I'm expected after this process to get the contents of
-->
that zip file.
-->
So what am I going to do with that?
-->
So let's do a put file.
-->
So now that I've unzipped it, I've got my files.
-->
I'm going to just put them back on the file system.
-->
So here we can actually do a put file, connect it, and if it
-->
successfully extracts it, we want to use the put file.
-->
And, you know, because I want to log all of my successes and
-->
failures, I want to add success as well.
-->
Now, put file is invalid, right?
-->
I've got the yellow exclamation mark.
-->
It's not in a stopped state and ready to go.
-->
It's because I need to define my relationships as well as the
-->
directory it's going in.
-->
So if I look at my properties, I need a directory to put this
-->
file.
-->
So what I want to do here is I want to go to my downloads.
-->
I'm going to create another folder called unzipped weather
-->
data.
-->
And I always go on the address bar because Windows likes to
-->
use its fancy names like this PC downloads.
-->
You click on the address bar, it gives you the full path.
-->
And then what I'm going to do on this is paste that path
-->
back in and everything should go into unzipped weather
-->
data.
-->
Now, I didn't put it back in my original weather data
-->
folder, right?
-->
Because if I did, it would take everything it decompressed
-->
and send it right back through the flow file.
-->
So, you know, I'll be writing this flow file and it would
-->
send it right back through.
-->
So what I like to do is definitely put it in a
-->
different folder because I don't want to have to
-->
reprocess those files over and over and over and create
-->
this infinite loop.
-->
So it's going in a different directory because I don't
-->
want to pick those up by this process group.
-->
So anyway, so I have to put file.
-->
Now I've entered the call on the directory.
-->
Now I need to terminate this.
-->
And I can go in here.
-->
I can configure this processor.
-->
And I can say auto terminate on failure or auto retry
-->
on failure.
-->
I can tell it to auto terminate on success or auto
-->
terminate on retry.
-->
Meaning if it writes it to file and it was successful,
-->
it's going to automatically terminate and be done.
-->
And, you know, I don't know because the file landed
-->
on the file system.
-->
But this is another one of those examples where,
-->
you know, I kind of want to see success, you know,
-->
instead of just trusting it's going to write to the
-->
file system.
-->
And then if there's a failure, you know, I want to
-->
see what that failure.
-->
And so what I like to do is put those, you know,
-->
the nice thing here as well is, you know, if I'm
-->
not mistaken, I should be able to auto terminate
-->
failure as well.
-->
No, we cannot because another relationship exists.
-->
So you can't auto terminate if you already have a
-->
relationship built.
-->
But you can have multiple failures, multiple successes.
-->
You know, like to get file from system.
-->
I had a success here and I have a success here.
-->
You know, I'm going to identify mine type.
-->
I have two successes.
-->
You know, same throughout the flow.
-->
So what this does is as I'm building this flow, it
-->
gives me more insight into what's going on.
-->
So if I need to make changes or if there's errors
-->
or something else like that, you know, I can make
-->
those changes before the flow actually gets
-->
deployed.
-->
So now I have this get file from system.
-->
Identify mine type, decompress and put file.
-->
Everything is in a stopped position.
-->
There's no yields.
-->
There's right now no errors.
-->
And so, you know, to me, this data flow looks
-->
good.
-->
So, NetFi is very, very quick.
-->
If we were to put a thousand zip files in that
-->
folder, it would pick it up before we would have
-->
time to stop it.
-->
Luckily, we only have one zip file and we
-->
kept the source.
-->
So what I like to do when I'm building a data
-->
flow is I like to kind of, you know, think
-->
through it logically a little bit.
-->
I like to sit down and I'm like, okay, well,
-->
I'm going to get a file.
-->
I need to decompress that file.
-->
I need, you know, for me to do that, I'm
-->
going to have to identify the mine type.
-->
You know, I'm going to build those things in.
-->
And then when I'm done with this file, I may
-->
send it to another processor that extracts the
-->
data.
-->
But for this use case, in this scenario, we
-->
are just going to write it back to file
-->
system.
-->
So what I like to do is kind of think
-->
through it and start building out my flow.
-->
I have my success messages.
-->
I have my failure messages.
-->
You know, I think I am pretty good to just
-->
let this go.
-->
But instead of turning everything on, what I
-->
like to do is I like to click on each
-->
individual processor, now that I've got the
-->
flow built, and say run once.
-->
So what's that going to do is go and pick
-->
up that file.
-->
And if my next processor is stopped, it's
-->
going to just queue up on this connection.
-->
So you can see I ran it one time.
-->
So I got one file in the connection.
-->
It's 2.19 kilobytes.
-->
And it's waiting to go to the next
-->
processor.
-->
Now that I've got that file, though, I can
-->
actually go and list my queue.
-->
Because all this data is automatically being
-->
queued up.
-->
So let me move this over.
-->
So with all the data queued up, I can
-->
now start looking at those attributes.
-->
I can start looking at data governance
-->
and provenance has happened.
-->
Because, you know, we have a file now in
-->
So what I like to do is I got that queued
-->
file.
-->
I will go and I will list the queue.
-->
It's going to tell me that I have this
-->
file name weatherdata.zip in the queue.
-->
Here's the file size.
-->
Here's how long it's been queued.
-->
The lineage duration.
-->
If it was penalized, I can actually
-->
download the content, which is the zip
-->
file.
-->
I can view the content.
-->
Or I can look at the provenance for it.
-->
So if I try to view the content, I'm
-->
going to complain that there is no
-->
viewer registered for this content type.
-->
This is what I was saying since content
-->
type application zip.
-->
So this is what I was saying.
-->
If it's a JSON document, which we'll
-->
get into, there's a viewer built in.
-->
So I can actually look at the JSON
-->
document.
-->
But for zip, there is no viewer here
-->
for zip.
-->
If there was, I would be able to look
-->
at that zip file.
-->
So let me close this.
-->
Also, if I go right here at the
-->
beginning, I can view the details of
-->
this file.
-->
So this is where I can start looking
-->
at my attributes.
-->
So we have our attributes and we have
-->
our flow files.
-->
From earlier, we went through this a
-->
little earlier on some of the
-->
terminology.
-->
But from the details, it gets assigned
-->
a unique ID.
-->
Everything that comes through the
-->
system gets a UUID.
-->
What is the file name?
-->
When it picked it up, it read the
-->
file name.
-->
Now that is an attribute.
-->
The reason you want that as an
-->
attribute is you may already have
-->
some rules set up that if it ends
-->
in .zip, it goes to processor X.
-->
If it ends in .json, it goes to
-->
processor Y.
-->
So you can actually read the
-->
attributes and route on attribute,
-->
which is a processor in our list.
-->
You should see route on attribute.
-->
The file size.
-->
If there was a queue position,
-->
which there wasn't.
-->
How long it's been queued.
-->
How long lineage has been the
-->
duration of the lineage.
-->
Some other identifiers.
-->
Offset.
-->
If you're looking at the Kafka
-->
type stuff.
-->
I can download the content,
-->
which would be a zip file.
-->
If I click that, I'm about to
-->
get a zip file downloaded.
-->
Whether it is.
-->
Then if I hit view, I can, you
-->
know, if I, again, if I have a,
-->
you know, if I have a viewer for
-->
this, I can, you know, if I
-->
have a viewer for this, I can view
-->
it.
-->
But the main thing I care about
-->
is these attributes.
-->
So, you know, git file is
-->
giving me these attributes.
-->
So git file said, hey, here's
-->
the absolute path.
-->
Here is the file creation
-->
time.
-->
Here is the file last access
-->
time, last modified time.
-->
You know, you may do some
-->
sorting and filtering of data
-->
based upon time.
-->
You know, if maybe who was
-->
the owner of the file, right?
-->
This is, you know, the owner
-->
as described in the Windows
-->
file system.
-->
What was the file name that I
-->
grabbed?
-->
It's a path that from when I
-->
picked it up, not the full
-->
path, the absolute, but just
-->
the path where, so if I, you
-->
know, had a recursive
-->
directory, right, it may have,
-->
you know, weather data two,
-->
like folder and then the
-->
file name.
-->
And then, of course, view ID
-->
that was assigned to this.
-->
Okay?
-->
So that is our connection.
-->
We have a file in the key.
-->
We ran this one time.
-->
And now we're trying to send
-->
it to the identify file
-->
MIME type, which is
-->
identifying the files.
-->
So let's run this just one
-->
time.
-->
I do a manual click to refresh.
-->
Like I said, you know, it's
-->
set up for five-minute
-->
refresh, but, you know, if
-->
you want to do it quicker,
-->
it's just easier to hit,
-->
right-click, refresh.
-->
And then you can also, if
-->
you did refresh the browser,
-->
you can do something like
-->
that as well, but right-click
-->
and refresh.
-->
But, anyways, identify MIME
-->
type.
-->
So I got the success.
-->
There's only success.
-->
Even if it doesn't identify
-->
it, it's still attempted to.
-->
And the relationships with this
-->
processor is success or
-->
nothing.
-->
And so that should also tell
-->
you, like, well, if it
-->
doesn't identify, if it's
-->
unknown, it's still going to
-->
go to success.
-->
It's just going to go to
-->
success is unknown.
-->
But let's look at the queue.
-->
So we can actually list this
-->
queue.
-->
And if we look at the
-->
attributes, we should have a
-->
new one.
-->
MIME extension.
-->
So it did identify the MIME
-->
type.
-->
And it identified it as a
-->
.zip.
-->
And the MIME type is, well,
-->
it identified the extension as
-->
.zip with the MIME type of
-->
application slash zip.
-->
If that was a different type of
-->
file, like, you know, like
-->
tar or gunzip, it would have
-->
recognized it as application
-->
slash gz or application slash
-->
tar.
-->
And so in this scenario, it
-->
is a zip file.
-->
So it identified it as an
-->
application slash zip.
-->
And now we have that
-->
attribute.
-->
So from here, right, I could
-->
say, well, I don't really want
-->
to decompress all my tar files.
-->
I want to send them somewhere
-->
else.
-->
So I can actually do a route
-->
on that attribute.
-->
Excuse me.
-->
I can do a route on an
-->
attribute and say, you know,
-->
something to the effect of, you
-->
know, send tar this way and
-->
zip to the decompress.
-->
But for in this scenario, I'm
-->
going to send it to success,
-->
and I'm going to let the
-->
decompressed content determine,
-->
you know, what type it is.
-->
So the MIME type detected it
-->
in my decompressed content.
-->
I did have my property set to
-->
use the MIME type attribute.
-->
So, you know, you saw the
-->
MIME.type attribute.
-->
So it's going to use that
-->
attribute to determine the
-->
decompression method.
-->
So this decompressed content
-->
is a Java application, right?
-->
And, you know, the source
-->
code of this would know how to
-->
handle how to decompress or
-->
compress all these formats.
-->
And so, you know, built into
-->
this processor, it already
-->
knows that if it's a zip, this
-->
is the method I'm going to use
-->
to unzip or to zip.
-->
So for this scenario, we're
-->
going to run it one time.
-->
It should be successful
-->
because, you know, it is a
-->
proper zip file.
-->
I bet everyone on this call has
-->
downloaded a zip file before and
-->
it got corrupted or something,
-->
and you couldn't extract it.
-->
So if the processor could not
-->
extract this file, it would have
-->
went to failure.
-->
And then I would like, okay,
-->
well, why am I getting
-->
failures?
-->
And this gives me this run
-->
once way of doing things gives
-->
me the capability to just run
-->
it one time and see how things
-->
are going.
-->
And I like to do that a few
-->
times before I fully turn
-->
things on.
-->
So anyway, so now it is
-->
success.
-->
We're going to list the queue.
-->
I don't think there's any,
-->
yeah, there's no new
-->
attributes.
-->
The decompress doesn't add
-->
attributes.
-->
All right.
-->
We're going to run put file
-->
one time.
-->
Let's see here.
-->
I'm looking at why it didn't
-->
actually do the decompress.
-->
All right.
-->
And I want to actually,
-->
I want to test this another
-->
way, too.
-->
I want to put
-->
this file back into its
-->
original spot for now.
-->
All right.
-->
Let's see here.
-->
Let's see fail.
-->
So decompress content.
-->
All right.
-->
We should have got three files
-->
out of that because that zip
-->
file has three.
-->
So I'm adding in a little bit
-->
more logic here to see
-->
where it's failing.
-->
So automatically terminate
-->
and automatically terminate
-->
apply.
-->
So it's good to go because I
-->
don't need to worry about
-->
that.
-->
And
-->
run them once.
-->
That should clear my
-->
queues.
-->
Give me just a second.
-->
For some reason it is being
-->
penalized.
-->
Let's see.
-->
Let's see.
-->
And some of these checks and
-->
balances to begin with.
-->
Empty my queue.
-->
Still there.
-->
That's not corrupted.
-->
It's really weird.
-->
It was penalized but there was
-->
no reason to penalize it.
-->
But I may have clicked something
-->
wrong.
-->
Let me see here.
-->
Put this file here.
-->
Did I use the wrong processor?
-->
Let's see.
-->
I'm just looking at why this is
-->
not
-->
Even when I have a problem
-->
I do the same thing.
-->
All right.
-->
Supports don't see it.
-->
This is really weird.
-->
Because it's supposed to decompress.
-->
This is one of those things
-->
where
-->
Is it possible to test as you
-->
go?
-->
For example, you put together
-->
those first couple processors
-->
could you have tested
-->
that some way right off the bat
-->
and then moved on and then
-->
you know what I mean?
-->
So that kind of
-->
test as you go?
-->
That is
-->
exactly why
-->
you see me just saying
-->
run once.
-->
So I run it one time
-->
and then what I like to do
-->
and it's weird
-->
this literally was not working
-->
but it was just working.
-->
What I like to do is run it one time
-->
and then I'll look at my queue.
-->
Let me go
-->
and listen to my queue.
-->
Am I like
-->
I know that I'm expecting
-->
one zip file.
-->
So I look at the queue
-->
and that's what I have is one
-->
zip file.
-->
And then I'll run
-->
let me finish building
-->
this relationship.
-->
What I was thinking was like
-->
doing it that way as opposed to putting all the
-->
processes on the canvas
-->
at one time and connecting them
-->
and doing all that work.
-->
I get that you can do the run
-->
once at each processor but
-->
I guess it's
-->
up.
-->
It depends on how you
-->
work I suppose.
-->
Exactly.
-->
If you remember when I opened up
-->
there's many ways to skin a cat
-->
with NAFA.
-->
That's some of the power of NAFA
-->
but that's also some of the
-->
negative because you don't have
-->
you like strict boundaries.
-->
And so I mean
-->
there we go.
-->
So
-->
yeah what I like to do
-->
is you know
-->
but you don't if your question is
-->
do you have to complete the flow
-->
file to test? You don't.
-->
You can
-->
I can have just this and
-->
this and a log
-->
message right? And
-->
you know I can
-->
run this flow.
-->
It will not
-->
continue until the
-->
it will not go through the next
-->
processor until
-->
you know until
-->
you have everything resolved.
-->
You have the relationships
-->
and things like that. So that's why I run
-->
this once. I'm going to run
-->
identify MIME type once.
-->
So I have it successfully
-->
sent to the decompressed content
-->
the unpacked content.
-->
Yep and I'm going to take and put
-->
the original
-->
put the original back in here
-->
even.
-->
The latency when you're
-->
like I said when you
-->
when you're doing this on your own laptop
-->
and not a VM
-->
it's a lot more it's a lot better
-->
user experience.
-->
But I'm going to drag this here and
-->
I'm going to put the original
-->
where I want the unpacking to go.
-->
And then I'll go through
-->
and clean this up. So
-->
I'm going to run the
-->
decompressed content once.
-->
And I'm just
-->
testing out because I'm trying to figure out why.
-->
So one came in three
-->
came out.
-->
There's the
-->
decompressed. Okay.
-->
There we go.
-->
For some reason
-->
this processor is not
-->
doing its job but it's okay
-->
we can fix this.
-->
We'll start
-->
deleting the connections.
-->
And this
-->
is a good example of you know
-->
if a processor is not a good fit
-->
not you know there's something
-->
going on it's not that there's a bug
-->
or something there's something going on with my
-->
configuration.
-->
You're not empty.
-->
But if a processor is not
-->
a good fit
-->
and you have those connections
-->
have data in the queue
-->
you got to clear the queue before you
-->
can replace
-->
the processor. So what we did is we just
-->
put it to unpacked content instead
-->
of decompress.
-->
So you may want
-->
to update your flow. I'll go
-->
back and check why the
-->
decompress is not working
-->
because it didn't decompress anything.
-->
So in this scenario though we've
-->
got our file coming in. We've
-->
identified the mom type. We're now
-->
using the unpacked content
-->
processor.
-->
It unpacks the contents of
-->
flow file
-->
and
-->
gave me
-->
this one I like as well because
-->
it gives me three different relationships
-->
more than just
-->
yes or no right. So
-->
it gives me a failure relationship
-->
original relationship success.
-->
So you know in this scenario
-->
I have unpacked
-->
it. I'm sending all of my
-->
success to
-->
log messages for instance
-->
but I could send it you know
-->
I need to put it in this folder.
-->
So what I want to do is take
-->
the original
-->
and we'll put it there.
-->
Notice that that file
-->
is still in the queue
-->
and I'm now changing
-->
direction of where it's going.
-->
And now I'm going to take
-->
my success and put it here.
-->
Okay perfect.
-->
So now what I've
-->
done is well I fixed
-->
my error and I used the
-->
correct processor but
-->
I've picked the file up. I've
-->
identified the mom type unpacked
-->
the content. I sent any
-->
failures. So
-->
if it's a corrupted
-->
zip or similar it's
-->
going to log all failure messages.
-->
I'm sending the
-->
original back to
-->
the place I picked
-->
it up you know just for testing
-->
purposes. Well actually
-->
I'm sending it to unzipped weather data
-->
which I need to change.
-->
This one is going to
-->
I'm going to change it and I'm going to
-->
put it back exactly where I have it.
-->
It's going to fail because
-->
the name you know I kept the original
-->
but
-->
doesn't matter.
-->
Okay.
-->
We'll go ahead and empty my
-->
queues.
-->
In
-->
this scenario and this
-->
these issues I'm running into these
-->
are issues you're going to deal with.
-->
And you know
-->
how you overcome those.
-->
So I
-->
think I've got my
-->
I think I've
-->
got my flow
-->
kind of figured out.
-->
Let me empty my queues and I'm
-->
going to run it one more time.
-->
And empty.
-->
Okay.
-->
So I see nothing in my queues.
-->
I see all my connections
-->
are good. I'm at a stopped
-->
state. If there would have been an
-->
error and I'm trying to generate
-->
an error. If there's an
-->
error we're going to get a
-->
little red box in
-->
the top right of your processor
-->
and it
-->
will give you a brief description
-->
of the error.
-->
That's another reason I recommend
-->
logging as much as possible
-->
so you can get a better
-->
detailed message.
-->
And I'm trying to cause
-->
an error and we will
-->
during our building of flow files.
-->
But you know as soon as
-->
I can cause an error and we see
-->
it I'll let you know. But anyway
-->
so I've built my
-->
my flow. I want
-->
to get it. Then if I mine
-->
tight I'm going to unpack it.
-->
Any failures I'm going to
-->
log those.
-->
I'm going to take all the success
-->
and put them in
-->
the unzipped weather
-->
data folder. And then I
-->
take the original and I'm going to put it
-->
right back to where it came from.
-->
And so this
-->
should work now.
-->
So I'm going to tell it to run once.
-->
Refresh.
-->
It's there. Run once again.
-->
And Tom I think you were kind of asking
-->
this. Let me do this. Delete.
-->
So
-->
so this
-->
processor is not complete anymore because
-->
I deleted that connection.
-->
But I can still run
-->
the previous.
-->
Well that also goes back to what you were saying.
-->
You need to make sure you resolve all those little
-->
warning things on
-->
your processors right?
-->
And so now it's just going to
-->
stop. And so when I resolve
-->
the relationship success
-->
it's
-->
going to
-->
stop.
-->
red and wait for me to either start
-->
it or start once.
-->
So let's just run one time.
-->
And I should get
-->
five files. The original
-->
and four in the
-->
zips. Okay so we're working great
-->
now. And then from there
-->
I am going to put
-->
actually I just hit start
-->
instead of run once. It's going to put all
-->
four files where it goes.
-->
And then also it's just
-->
logging everything. So let me go ahead and
-->
I can turn on my log.
-->
I can turn on all the logging.
-->
Good.
-->
So now let's go check
-->
our folders.
-->
Should have
-->
there we go.
-->
And then also through the original
-->
back in there.
-->
So
-->
now I have
-->
my flow. My flow is built.
-->
I've tested it. It looks like
-->
it's working. I'm being
-->
penalized. I don't know if it's system
-->
resources or what.
-->
Maybe too many successes.
-->
I have to
-->
take a look at why I'm being penalized.
-->
yeah so I've got my
-->
original. I've got my success.
-->
I've got my failures. I have
-->
a operational flow
-->
now.
-->
So what I want to do now
-->
Well, if that is success
-->
I already know I can take away
-->
that one. So this is
-->
where I like to go back and
-->
delete, delete,
-->
put the file success.
-->
Alright.
-->
And I'll
-->
show you
-->
you just
-->
double click where you want.
-->
This was a question I had in the
-->
last class.
-->
It's actually not
-->
in the documentation but if you double
-->
click right where you want
-->
you can bend the line
-->
to make it more
-->
easier to read.
-->
So you can mess with those.
-->
Okay, so I think I am
-->
good.
-->
I'm going to empty my queue.
-->
Alright, so I've got the file
-->
that's coming in. It's unpacking. Any
-->
failures go there. Any failures
-->
go there.
-->
Success
-->
goes here in
-->
the original.
-->
Oh, I don't need
-->
the success because I just put the file
-->
I'm testing my flow out.
-->
I'm going to auto terminate success.
-->
Any failures I've got to go into my
-->
log. Any unpacking
-->
failures I've got to go into my log.
-->
And then I'm going to send the original
-->
back to its original place.
-->
Yeah, weather data. You know what?
-->
I want to make a copy of
-->
it
-->
for demonstration purposes. So I'm
-->
going to actually create weather data
-->
one where it's going to put it.
-->
It should come back and complain.
-->
Oh, I don't have any more successes.
-->
Where's my weather data?
-->
Copy.
-->
Alright.
-->
My goal is to be able to
-->
pick up the weather data.
-->
Push it back. Oh, perfect.
-->
And it's
-->
me.
-->
So I want to pick up
-->
from weather data.
-->
Once I have
-->
that, I am going to
-->
grab it here.
-->
I like to go through and double check.
-->
It's going to pick up from weather data.
-->
It's going to unpack it.
-->
It's going to put all the
-->
extracted files in
-->
unzipped weather data.
-->
I have that
-->
created unzipped weather data.
-->
I can go ahead and control A
-->
and delete all these.
-->
My folders are empty.
-->
Everything's good
-->
to go.
-->
And then I'm going to take the
-->
original
-->
and I'm going to put it in a
-->
separate folder.
-->
So I just have a copy of the original.
-->
And then that way I can tell
-->
the git file to
-->
keep, to not keep
-->
the source.
-->
Apply.
-->
Okay.
-->
So technically this, and I'm not
-->
using this, as you can see
-->
anymore.
-->
So I can actually delete that.
-->
My flow file is
-->
pretty simple now.
-->
I've taken out some logging.
-->
I have
-->
run through this once or twice
-->
to make sure everything works.
-->
So what I like to do is
-->
work backwards in starting
-->
this. If all of these,
-->
and we're about to do that too,
-->
if all of these are in a
-->
processor group, I could
-->
start and stop the whole
-->
processor group from
-->
my operating palette.
-->
Alright, we can unpack content.
-->
Go there.
-->
Identify mine type.
-->
Start.
-->
And here's why I use the run once.
-->
So, let me
-->
open my folders again.
-->
Unzip.
-->
Alright.
-->
It is empty.
-->
As soon as I turn this
-->
on,
-->
I should have files here.
-->
There they are.
-->
It's that quick.
-->
And that's what I was saying
-->
earlier, if I had a thousand
-->
files, it would
-->
process extremely quickly.
-->
Alright, so I built
-->
my flow.
-->
My flow was successful.
-->
It did exactly like
-->
I wanted it to do.
-->
You know, so with that being said,
-->
any questions
-->
on, you know, kind of
-->
walking through our original flow?
-->
We did have a couple of errors.
-->
We had to work through those.
-->
We had to, you know,
-->
touch up a few things.
-->
It looks like a spider web.
-->
So it's definitely not
-->
how I like it to look.
-->
So what I like to do, as a,
-->
you know, as an engineer,
-->
is I'll go through and start
-->
cleaning this up.
-->
And, you know, I'll start
-->
putting this where
-->
it makes logical sense to me.
-->
And I'll start cleaning all this up.
-->
And you can see
-->
when you're dealing with
-->
these, you know, these types of
-->
operations,
-->
you know, having
-->
connections everywhere,
-->
you know,
-->
let me read that line.
-->
There you go. Having
-->
connections everywhere, you know,
-->
it can look like a spider web.
-->
And so what I like to do is
-->
as my flow is
-->
developing and as I'm building this out,
-->
I go through
-->
and start
-->
cleaning it up. I start applying
-->
labels. I apply colors.
-->
I'll go through and rename
-->
these to make sense.
-->
So with that being said,
-->
any questions
-->
on this
-->
flow and how
-->
we built it, any of the visualization,
-->
we still haven't went into the
-->
Babanauts, and we're going to work on that
-->
after the break.
-->
But, you know, any questions
-->
so far?
-->
I missed a part
-->
about the group.
-->
Also, I'm not sure what
-->
the parts
-->
like small boxes are.
-->
Oh, okay.
-->
And I'll look at
-->
your screen in a minute and make sure
-->
everything looks good.
-->
But, you know, the connections
-->
are the relationships
-->
coming out of that
-->
processor. So for instance,
-->
getting the file only
-->
has success. So
-->
I send the success to identify
-->
the MIND type.
-->
Identify MIND type only has
-->
success, so I will send it to the
-->
unpacked content.
-->
And then
-->
the unpacked content has
-->
three relationships.
-->
The one is your original
-->
file that it got.
-->
The success,
-->
which is the unzipped files,
-->
and failure in case
-->
it can't unzip the file.
-->
Or
-->
extract it from a tar or whatever.
-->
So those are the three
-->
relationships, three connections
-->
I need to build.
-->
I could go to
-->
unpacked content and
-->
configure it.
-->
And say auto
-->
terminate on failure,
-->
for instance. But
-->
what I like to do for now
-->
is I like to log that message.
-->
So I can actually, while it's running,
-->
I can just say stop and configure.
-->
And I can say auto terminate failure.
-->
And it would
-->
auto terminate this failure.
-->
And, you know,
-->
and it would be complete.
-->
But I like to send it, I like to
-->
especially send my failures
-->
to a message. That way
-->
I know what's going on. But later
-->
you can take this out
-->
if you don't need that.
-->
And maybe you just turn on
-->
your bulletin level down to like
-->
info or warn. And
-->
try to catch that message.
-->
And that way it's in the log.
-->
I personally like to make sure
-->
that that failure is
-->
logged. But then that's how your
-->
connections are made. So when I come out
-->
of this unpacked content, I needed
-->
three different relationships.
-->
So I sent my original
-->
back to the original location
-->
properties.
-->
I sent it
-->
to a different folder.
-->
The original
-->
speed, the original speed in the zip, right?
-->
The original being the zip, yes.
-->
Success
-->
was the
-->
extracted files.
-->
And any failures was
-->
if it's a corrupt zip.
-->
And I remember now
-->
I looked at my put file.
-->
I can
-->
So I was getting files from
-->
weather data. Weather data
-->
is empty, right?
-->
I was putting
-->
the original in weather data one.
-->
So if I hit copy
-->
Go back into the
-->
the first get file
-->
and I say paste.
-->
Oh, luckily I have this stopped.
-->
It would have already run through.
-->
Oh, actually it did
-->
create an error. Good.
-->
The file name already exists
-->
in the where it's trying
-->
to put the files.
-->
And so it threw an error.
-->
Which I'm glad
-->
because I was trying to figure out how
-->
to quickly generate an error for
-->
us to see.
-->
So again, this is your top right corner.
-->
You'll see that little red box.
-->
It's penalizing it because
-->
the file with the same name already exists.
-->
But yes, so the original
-->
I was putting in weather data one.
-->
I'm picking up from weather data.
-->
The put file for the
-->
extract
-->
is going into the unzipped
-->
weather data folder.
-->
And we're good to go.
-->
You do see now I have a failure
-->
because
-->
the same name exists
-->
and my naming strategy
-->
has it throw an error.
-->
I can say ignore.
-->
And it would just keep writing.
-->
Okay.
-->
I want to pause here.
-->
We actually don't have much time
-->
but I definitely need something to drink.
-->
Any questions
-->
on the initial flow
-->
that we kind of worked together
-->
to build?
-->
I'm going to change this to
-->
replace.
-->
Is there
-->
an easier way to replace a processor?
-->
Instead of building all your relations
-->
and stuff, just change it?
-->
I guess not.
-->
Great question, Travis.
-->
Swap it out.
-->
Yeah, it would be amazing
-->
and keep those connections.
-->
So for you to
-->
replace a processor
-->
you have to
-->
delete the connections
-->
before you can delete the processor.
-->
So unfortunately
-->
there is not.
-->
But here's what I will do.
-->
I got three or four
-->
great suggestions from the
-->
last class
-->
and I've already submitted them
-->
to be built into the next version
-->
of NAFA.
-->
So I am making a note
-->
to make
-->
replacing a processor easier.
-->
That would be extremely cool.
-->
It would be.
-->
I'm trying to figure out how...
-->
It was, wasn't it?
-->
Imagine
-->
your flow is
-->
100 processors in length.
-->
Right?
-->
Exactly.
-->
That's why I'm not a developer.
-->
So that's why
-->
I also
-->
recommend
-->
let's build
-->
and test while we go through it
-->
because you do not
-->
want to get 100 processors deep
-->
and you've got to jerk
-->
20 of them out
-->
and replace them with another
-->
10 or 15.
-->
It's just chaotic.
-->
This is the beauty
-->
of workflow-based programming
-->
though, right? We don't have to
-->
code it to do this
-->
but there are nuances
-->
with this.
-->
But
-->
great suggestion and I will
-->
actually take that back
-->
to the
-->
NAFA committee to see if we can
-->
get that worked into some future versions.
-->
I'm trying to think through how
-->
you can do that but we'll figure it out.
-->
Any other
-->
questions on our first
-->
flow?
-->
Alright, well we're getting close
-->
to the end of the day. I need to go get something
-->
to drink and use the restroom real quickly.
-->
So instead of a 15 minute
-->
break, can we do 10
-->
and
-->
then we'll come back, kind of wrap
-->
up for the day and then
-->
I will see everyone tomorrow morning.
-->
So I have
-->
4.02. Let's come back
-->
at
-->
which is
-->
2.02, y'all's time. So let's do
-->
2.17
-->
and 4.
-->
We're only taking 10 minutes.
-->
12. Okay.
-->
I will see you all in 10 minutes.
-->
Perfect.
-->
And then let's wrap the day up.
-->
Let's get some questions. I do want to
-->
if you all went through your flows
-->
I'm going to bring them up
-->
and just kind of walk through them real quickly.
-->
Tomorrow we're going to be
-->
doing real flows with
-->
real controller services and those
-->
types of things. But I'll be back
-->
in just a minute.
-->
Awesome, thank you.
-->
Any suggestions
-->
you all have though, like
-->
I wish this was easier, that was easier.
-->
Please let me know.
-->
Those
-->
the last class
-->
had a few
-->
really good ones that I
-->
wasn't even thinking about.
-->
While I wait
-->
I'm going to pull up everyone's
-->
screen and kind of
-->
some high achievers.
-->
I was told this was
-->
a non-technical class.
-->
You guys are rocking it.
-->
Give everybody a couple more minutes
-->
to get back and then
-->
we will go through our first
-->
flow.
-->
Alright.
-->
Sorry, I'm just reading the
-->
decompressed and compressed
-->
content processor and seeing what
-->
changed because it
-->
should have decompressed but it looks like
-->
the MIME type is
-->
automatically detected and the data is
-->
decompressed if necessary.
-->
But there's no
-->
relationship.
-->
Okay.
-->
So we have
-->
a few minutes left of the day.
-->
We might be able to get out here a few minutes
-->
early but what I like to do is
-->
kind of go around the room.
-->
See how your first data flow
-->
went. Any
-->
major issues except for following
-->
my incorrect steps.
-->
Which kind of worked out because then
-->
we got to see how you replace
-->
these things and we also generated
-->
an error which I was hoping to do
-->
just so I can show
-->
where those errors look like
-->
and where they come from.
-->
So we will start with
-->
Tom.
-->
How did your first
-->
flow go, Tom?
-->
You were
-->
great at going through the steps
-->
and this was my first one.
-->
I've never even seen or touched
-->
well, I've seen it but I haven't done
-->
anything with it. This is my first
-->
experience hands on
-->
with it doing something with
-->
it.
-->
I think you explained things very well. It made it
-->
pretty easy. I still
-->
have some work to do on mine but I think
-->
I'll clean mine up and probably
-->
make mine more like yours
-->
and mess around with it after
-->
we're done for the day or whatever.
-->
That is
-->
what I want to hear. I'd love to hear
-->
that you've never
-->
touched this before.
-->
In your opinion,
-->
picking data up,
-->
decompressing it, putting in
-->
another spot,
-->
in my opinion, as someone
-->
who
-->
has never touched it before, I think this
-->
would be a lot easier
-->
than trying to
-->
write a script or something else
-->
but
-->
what is your overall take?
-->
What did you
-->
like? What did you dislike?
-->
How did it work out for you?
-->
I
-->
liked it a lot
-->
except for the thing that Travis
-->
brought up.
-->
I did the same thing as you. I replaced
-->
the decompress with the unpack
-->
content processor
-->
and you had to delete those
-->
relationships first and all that.
-->
It wasn't too bad but I could
-->
see where that would be
-->
tedious if you had to do that for a lot more than just
-->
one.
-->
I'm semi familiar with
-->
workflow type
-->
technologies like
-->
I did SharePoint a lot that had
-->
a workflow type thing tied behind
-->
it and even the stuff we use with
-->
power apps has a workflow, has a flow
-->
component to it.
-->
Like you mentioned, the
-->
flow way of doing things
-->
isn't anything new but
-->
this is definitely pretty cool.
-->
I can see a lot of value
-->
in this.
-->
Wait until tomorrow
-->
when we are taking the
-->
CSVs and JSON that you extracted
-->
and we're actually extracting
-->
CSV data and making
-->
it a JSON document.
-->
I don't give as much guidance
-->
on that flow
-->
because that one
-->
I kind of want it open to
-->
interpretation because there's a couple of different ways
-->
to do that.
-->
But no, overall, I think
-->
your flow looks great.
-->
If you're going to go through
-->
and start cleaning up, renaming
-->
the processors,
-->
you may even put in there
-->
put file into
-->
this directory or something
-->
as a name.
-->
You could see already
-->
with mine how I had
-->
10, 15 processors
-->
starting to stack up
-->
and
-->
it starts running together.
-->
So if you name it
-->
and you do that beautification,
-->
it just makes it a lot easier.
-->
I like hearing from people
-->
who have never touched
-->
NIFI and so
-->
great job.
-->
I think your flow looks great.
-->
I can't wait to see it cleaned up
-->
and then I think tomorrow
-->
you'll get into some deeper dive of this.
-->
We'll look at some products, some lineage
-->
and you're going to
-->
be a data engineer in the next couple
-->
of days.
-->
I was worried that this was going to be
-->
overkill with the slides and
-->
not enough hands on, but this has been
-->
to my surprise a lot better than
-->
what I was expecting.
-->
I don't know how else.
-->
I could go through the slides
-->
and swap
-->
back and forth between NIFI and that.
-->
So what I like to do
-->
and also, like I was in the army,
-->
I briefed generals and stuff
-->
all the time, so
-->
I understand PowerPoint
-->
by PowerPoint.
-->
So I like to do is just run through
-->
the terminology and then kind
-->
of repeat it as we go
-->
along.
-->
I promise
-->
we have only a little bit more PowerPoint
-->
to do, but most of the rest
-->
of the time we are going to be working
-->
because that's the best way to learn it.
-->
I agree. I love it.
-->
Thank you so much.
-->
All right.
-->
Darius, how are you doing?
-->
I had to restart
-->
my computer earlier, so I missed
-->
some of it.
-->
I pretty much just tried to
-->
mimic, I guess, what you were doing.
-->
I didn't get to all of it.
-->
Don't worry.
-->
What you want to do
-->
is git file.
-->
Do you have the
-->
zip file and everything
-->
you need? If you go to that folder,
-->
let's look at that.
-->
Go to downloads. Awesome.
-->
Yeah, you missed
-->
the part where we copied over.
-->
Actually, if you go to your desktop,
-->
and you
-->
go to your uploads,
-->
and just right click and say copy
-->
on the weather data, and then you go back
-->
to your downloads,
-->
and paste it.
-->
You can right click and paste.
-->
You'll have to do it more towards the bottom.
-->
There you go.
-->
Those are the
-->
and you can create any kind
-->
of folders you want, but if you
-->
extract weather data, just say extract all.
-->
There you go.
-->
That'll work.
-->
It's going to pop open a new window.
-->
If you click
-->
up in the address bar, instead of
-->
saying this PC downloads is going to give you
-->
the real path, now copy that
-->
path,
-->
and then
-->
go to your get file,
-->
right click, and do
-->
your properties, and then put
-->
that input directory, paste that in
-->
on your input directory, on the value
-->
of the input directory. First line.
-->
There you go. Paste that in.
-->
Okay.
-->
And apply. Alright.
-->
You see how it went from yellow to red?
-->
The processor?
-->
It was
-->
yellow like your log message.
-->
So, you know,
-->
if you can, just keep trying to work through
-->
this flow.
-->
I'll be happy to come back after I
-->
go through the others and work
-->
with you. You want
-->
to identify that MIME type.
-->
You want to unpack the content.
-->
And then you probably want to put
-->
the file.
-->
You identify the MIME type.
-->
Do you have the original?
-->
Or you auto
-->
terminate an original
-->
on the unpacked content?
-->
Let's look at your relationship.
-->
Because you have a relationship or something
-->
off because it's still yellow.
-->
Alright, hit cancel.
-->
Go to your unpacked
-->
content processor, not the
-->
connection. There you go.
-->
And let's look at relationships
-->
with that.
-->
And we need the original.
-->
So,
-->
you don't have it auto
-->
terminating. You can actually just say cancel.
-->
And let's, you got a put file
-->
for your success.
-->
Right click on your put file.
-->
Say copy. And then right click beside
-->
it until it's a paste. There you go.
-->
And then on your unpacked content,
-->
drag another connection
-->
to that and do
-->
the original.
-->
And then,
-->
you know, say original.
-->
And now it turned red
-->
and it's ready to go.
-->
So, the only thing you have left to do
-->
is figure out where you want
-->
to put the original file.
-->
Because you decided on the get
-->
files to
-->
not leave, you know,
-->
you're picking it up so it's not
-->
staying where it's at.
-->
So, you want to copy it
-->
to a new folder. And then
-->
also create a folder to put the
-->
weather data that is extracted.
-->
And so you want to put that in your
-->
put file. If you right click on
-->
your put file, you'll see, you know,
-->
directory again. And so you want
-->
to go back to that downloads folder.
-->
Create a new folder called,
-->
you know, original file.
-->
And put that directory in there.
-->
And create a new folder called,
-->
you know, unzipped weather data.
-->
And put that directory in
-->
the other put file. And then your
-->
flow should be good.
-->
You know, and then your long
-->
message, you might want to just auto terminate
-->
success.
-->
And then,
-->
you got the concept, I think.
-->
Try to plug that in. I'll come
-->
back to you at the end.
-->
See how, you know, how much
-->
progress you made. You know, the only
-->
other thing is you might want to think about
-->
like, you know, naming the
-->
processors, you know,
-->
real names that we can all
-->
understand, you know, mutifying
-->
and stuff like that. But, you
-->
know, overall, you know, I think you
-->
got it. So I'll come back to you and see how
-->
you're doing.
-->
All right, thanks. Yep, yep.
-->
All right.
-->
How are you?
-->
Hi, you doing good?
-->
How did your flow go?
-->
I think it's going fine.
-->
Everything went
-->
is working. The only thing
-->
I want to see is, like, if I
-->
make it fail once
-->
the file's unzipped,
-->
where do I see the log messages
-->
for that? Oh, yeah.
-->
We had that, if you remember.
-->
So it's set to
-->
failure. And so you can
-->
actually have that
-->
file send failure messages
-->
to your log message.
-->
But also, in that top right corner
-->
of the processor, remember mine
-->
had the little red box?
-->
In that little
-->
red box. Maybe that's why it's hiding.
-->
Well, you don't have it yet
-->
because you haven't had a failure.
-->
But it's,
-->
you know, in the
-->
actual processor. But I really,
-->
you know, I like what you're doing here, where you're
-->
sending the failure to the log
-->
message. And
-->
so, yeah,
-->
if you do have a failure because of
-->
the naming strategy, you have
-->
this failure. You can do ignore,
-->
you can do replace. But because you have
-->
failure, it should
-->
generate a little red box
-->
in the top right corner of the processor
-->
itself, letting you know
-->
that that file name already
-->
exists.
-->
And so, you know, to overcome
-->
that, you could
-->
rename the files in flight.
-->
And that way, when
-->
they go to a put file, it'll actually
-->
be a different file name. You could
-->
ignore the error.
-->
You can log it, you know, depending
-->
on your scenario and how you want to do
-->
things. Okay. So,
-->
so whatever icon
-->
or whatever message shows up
-->
will tell you like what actually happened
-->
because I haven't seen that yet.
-->
I'm just trying it right now. Let's run it.
-->
Let's do it. Let's do it. We got time.
-->
Let's see it.
-->
Not to put you on the spot. So you
-->
want to run the identify mine type once?
-->
Oh, okay.
-->
Refresh. It's kind of slow.
-->
Yeah, this virtual machine
-->
can
-->
you'll see me trying to adjust and
-->
click and everything else. Okay.
-->
So it's success.
-->
So let's unpack the content.
-->
So you have original. Okay.
-->
Run once. And now go to the
-->
canvas and say refresh.
-->
Sorry, refreshed.
-->
So the original message
-->
went to your put file
-->
on the left. The unzipped
-->
contents went to the bottom
-->
of the put file.
-->
And you should be able to say run once.
-->
Oh, you see the error?
-->
The top right button. There you go.
-->
Penalize because the same file name
-->
exists.
-->
Is this log saved somewhere
-->
else? Like in some
-->
log logging files?
-->
Yeah, it's in your nafi-at
-->
log that you opened up to get your username
-->
and password. So
-->
it's logging the messages
-->
there that are that is happening.
-->
And it's good.
-->
You know, you could
-->
if you're familiar with Linux, you can tell
-->
the log. You can also tell the log in
-->
Windows. But if you go back
-->
to your go back a directory that your bin
-->
directory. So that's where you
-->
your executables are. Go to logs.
-->
There you go.
-->
Open up logs.
-->
Go ahead and open up that nafi-app.
-->
Yeah, no worries. Go ahead and open up
-->
nafi-app. That one.
-->
Just open it.
-->
Scroll all the way down.
-->
Man, don't highlight it all.
-->
That's a pretty good size log already.
-->
You
-->
should see, scroll up
-->
a little bit, your warn right there.
-->
See, you have a warn.
-->
Yep.
-->
Standard put file with
-->
an ID. It's penalizing the standard
-->
flow. Scroll all the way to the right.
-->
There you go.
-->
Yeah, and it gives you
-->
a lot of information.
-->
A lot more information than just that
-->
little red box.
-->
But because you are sending
-->
that failure to the log
-->
message is the reason you have it.
-->
If you did not, it
-->
would just have a failure on the
-->
screen and it would not
-->
go to the log.
-->
Alright.
-->
So was this
-->
your first time of
-->
working
-->
with nafi or have you already
-->
played with it before?
-->
No, this is my first time too.
-->
Oh, nice, nice. And what were your
-->
thoughts?
-->
I think this could be really
-->
powerful.
-->
Because you can do
-->
things so granular and
-->
modify and
-->
control where data
-->
can go and how you log it.
-->
Yeah, I'm looking forward to it.
-->
Okay.
-->
You're going to get into
-->
some difficult ones tomorrow.
-->
Richard, how did it go?
-->
As you can tell,
-->
I was doing a lot of experimenting.
-->
I started
-->
off with some files, primarily
-->
images. For the most
-->
part, I did try to figure with the compressed
-->
content and I was
-->
trying to change some settings there.
-->
My first time,
-->
definitely aware of it.
-->
Having had to do the process of
-->
really moving any
-->
files. For the most part, I was
-->
experimenting to see
-->
the settings of each
-->
of these. If
-->
I changed it from zip, what would it do
-->
differently, I guess.
-->
So I guess it was more
-->
tinkering than anything.
-->
I know someone asked it, but I
-->
missed it. Unfortunately, I didn't
-->
see how you can draw in the background.
-->
I guess.
-->
Up on your toolbar,
-->
go up
-->
and the last one
-->
should be labeled.
-->
Up here?
-->
Yep. Click and drag it down.
-->
That's what I was
-->
expecting.
-->
Double click it.
-->
Then you can change
-->
the value. You can actually name it.
-->
You can put a name, you can put a font size,
-->
all those things.
-->
But I am glad
-->
this is also why
-->
for the
-->
class to run wild.
-->
Because you brought up a point
-->
that I can show right now.
-->
So non-fi is configured
-->
to
-->
handle a max queue size
-->
of 10,000 files
-->
or 1 gig.
-->
Now that is configurable
-->
in the properties.
-->
So you have 10,000 files
-->
in your queue on that
-->
put file, right?
-->
So what's going to happen is
-->
the queue that's before
-->
that previous processor
-->
is going to start filling up
-->
as well.
-->
And it's going to continue trying to
-->
process data as much as it
-->
can. So when that unpack
-->
content to put file,
-->
when that gets over 1 gig
-->
or 10,000 files,
-->
it's going to back up.
-->
And so what happens is it will start
-->
backing up the whole system.
-->
So
-->
you need to clear your queue
-->
so the rest of them will start clearing
-->
itself.
-->
But
-->
any questions,
-->
issues or anything?
-->
Any, what do you think?
-->
No, definitely
-->
kind of value here.
-->
I was just kind of experimenting
-->
with some use cases on my end.
-->
Actually, I started off with some images
-->
who first started.
-->
And then
-->
I started following
-->
the content more closely and then I started going down
-->
this road. But even then I was just grooming
-->
quite a bit. So more understanding
-->
of these settings and just
-->
trying to understand the behavior as well.
-->
But definitely love it. I could see
-->
a lot of use cases
-->
and looking forward to learning
-->
some more.
-->
All right.
-->
Good deal. Leroy.
-->
You have a
-->
failure. But it looks like
-->
everything is running.
-->
Are you there, Leroy? I will come back
-->
to Leroy. But it looks like
-->
he's got it going.
-->
Peter.
-->
Leroy has mic issues.
-->
All right. Perfect.
-->
We can come back. Peter,
-->
how did it go?
-->
I followed along with yours
-->
for the most part.
-->
Mine at the bottom, the put files
-->
were supposed to be putting the unzipped files
-->
into the folder. It has a triangle
-->
on it. When I right click that triangle
-->
it says invalid.
-->
Valid up here.
-->
I don't know what that means.
-->
Hit cancel.
-->
And then hover over
-->
the little yellow yield beside the
-->
put file. So you do not have
-->
a success relationship
-->
or a failure relationship
-->
on your put file.
-->
So either, you know,
-->
if it fails, you may want to send
-->
that to a log message
-->
and auto terminate.
-->
And you may want to just go ahead and
-->
just say auto terminate success.
-->
So right click on put file
-->
and say configure.
-->
And then you can say auto terminate
-->
on success.
-->
And then
-->
failure, you can terminate
-->
as well.
-->
Or you can send it to your
-->
log message.
-->
So if you want, click on the
-->
put file, drag it to your log message.
-->
And say failure.
-->
And say add.
-->
And your put file
-->
now is turned red.
-->
And it's ready to go.
-->
Perfect.
-->
And then same thing for the one over here.
-->
Yeah, hover over. Let's take a look.
-->
Yep.
-->
It's your relationships. And that's usually what
-->
it is. Or you're missing
-->
something that is required.
-->
But besides
-->
that, you know, what did you think?
-->
Yeah, it seems pretty straightforward.
-->
It's definitely easy to follow along with your class.
-->
I've had issues
-->
with the virtual environment,
-->
but no issues with the
-->
butterfly.
-->
The virtual environment crashed on me at one point.
-->
I had to restart that. And it's just been running
-->
really slowly.
-->
Yeah,
-->
that's the biggest thing I hear
-->
is the latency sometimes.
-->
Because you'll point and click and then it never
-->
responds. And then you'll get
-->
it clicked and then
-->
it responds after a second.
-->
And you're already working on something else.
-->
I totally get it.
-->
You know,
-->
some of the same issues crept up before.
-->
So if Maria's on the
-->
call, I think hopefully she's taking
-->
notes. But
-->
I totally get it. Was this
-->
your first time with
-->
NaFi?
-->
Oh, awesome. Awesome.
-->
Well, it looks like you kind of run through well.
-->
You know, you can continue working on
-->
this. I would just, you know, as tips,
-->
I would just clean things up.
-->
You know, make sure you name your
-->
processor or something easily understandable.
-->
You know, you know how to plot labels
-->
and things like that.
-->
You know, you get this ready for production.
-->
So any questions
-->
I can answer for you?
-->
No, not
-->
right now. Thank you.
-->
Travis with a great suggestion
-->
of replacing
-->
processors.
-->
That was probably my only major
-->
hang up. Sorry, I'm looking at
-->
Splunk related things.
-->
There's Splunk processors.
-->
There's a lot of good Splunk processors.
-->
Yeah, I'm the Splunk guy
-->
from YPG, so I was just looking.
-->
Oh, nice, nice.
-->
But your
-->
test, it
-->
worked well.
-->
I got to work, and no issue. Well, I had
-->
to do that first, but
-->
it was because I was not replacing
-->
files, so that was easy to fix.
-->
This is
-->
my first time using NaFi.
-->
I have watched my lead use it.
-->
So, a little bit
-->
starting knowledge with that.
-->
Yeah, it's great.
-->
I love it. I can't wait to use it for other things.
-->
Perfect, perfect. Yeah, there's
-->
you know, if you get home tonight
-->
and you come back tomorrow and say, hey, Josh
-->
do you know that there's a processor for
-->
this? Let me know.
-->
I like hearing that like, oh, I'm the Splunk
-->
guy, because we have Splunk processors.
-->
There's processors for
-->
all kinds of things. So
-->
I'd love to hear about some of the tech you're trying
-->
to connect to.
-->
Is there a
-->
I don't want to say third party, but
-->
like processes that you can
-->
load into NaFi that might not be
-->
like come with the install?
-->
Like a library somewhere?
-->
You
-->
could load up
-->
like some data flows,
-->
like pre-built data flows,
-->
and I know that like
-->
InfluxDB has
-->
a template
-->
that they ship with
-->
their processors as well
-->
that you could load up that
-->
would help you get data into Influx.
-->
But
-->
you know, I
-->
I don't think that there's
-->
you know, I've never seen anything like
-->
you know, if you're asking for
-->
like a pre-built flow where you just need to
-->
you know, manage the
-->
in and out. I haven't seen
-->
anything like that, but I will definitely
-->
take a look around and see.
-->
Yeah, I'll take a look around
-->
and see. I was just not sure you were seeking
-->
you up top here. No. Alright. Good deal.
-->
Alright.
-->
And then that was Travis.
-->
We are going back
-->
to
-->
Odaris? Was it
-->
were you having a microphone issue? Oh no, I think
-->
that was someone else.
-->
It wasn't you. Leroy, Leroy, right?
-->
Here it comes.
-->
You guys coming?
-->
I can. Yep. I remember now
-->
you had the mic issue. How did it go?
-->
It went
-->
well. I was able to keep up with the example.
-->
I appreciate having to work
-->
through the errors. So that
-->
was a good value add. Awesome.
-->
I had to anchor with an
-->
i5 flow, but it was like the first time
-->
building from scratch. Okay.
-->
Good deal. And I guess to be looking
-->
at the custom processors
-->
so we can kind of implement some machine
-->
learning models. Something crazy.
-->
Awesome.
-->
Let me note that there is
-->
a TensorFlow
-->
processors. I have a friend who
-->
develops
-->
a lot of custom processors and then
-->
puts them out on GitHub. So
-->
let me
-->
write that down. I'll throw you some
-->
points. You know, it's not
-->
necessarily part of the official
-->
NaFi package and training.
-->
But I definitely want to send you some
-->
stuff that would put you in the right direction.
-->
Yeah, I figured something that exists already where you can
-->
just kind of plug in your
-->
PyTorch modeler.
-->
Yeah, there's
-->
quite a bit. And there's ways
-->
to run Python and NaFi
-->
that we are going to get to tomorrow.
-->
If you look at your processors,
-->
there is an execute script
-->
and an execute process.
-->
So, you know,
-->
just a tidbit.
-->
I will, you know, this goes for everyone.
-->
Tomorrow during the, you know,
-->
some of the hands-on scenarios, I'm
-->
not going to tell you every processor
-->
you need to use.
-->
You know, the goal is to look
-->
through the processors and determine
-->
what best suits you.
-->
So, you know,
-->
if you get a free minute
-->
later today or tomorrow,
-->
it might, you know, it might be
-->
good and beneficial just to kind of skim
-->
the processor list and say, you
-->
know, kind of look at the groups of them
-->
and how they can help you.
-->
But any issues, Leroy,
-->
or, you know,
-->
any answers or
-->
any other questions I can answer?
-->
I think that's it for now.
-->
Okay, perfect.
-->
Alright, well,
-->
I think we had a great day.
-->
I think
-->
overall, you know, everyone
-->
got it installed. It was not,
-->
you know, I don't think there was a lot of
-->
difficulty getting it installed.
-->
I don't think there was many issues
-->
there. We got it up
-->
and running. We got our flows.
-->
You know, tomorrow
-->
we're going to get registry running.
-->
We're going to check our flows in
-->
and we're going to build some more. So,
-->
you know, a lot of cool things
-->
there. If you
-->
have any questions or anything else, let me know.
-->
Feel free to continue working
-->
on this. But,
-->
you know, I think
-->
you all are off to a
-->
good start. If you need anything,
-->
just let me know. But I'm going to go ahead
-->
and end the training for today
-->
and run to Costco
-->
real quick. And so,
-->
you know, if you need anything, let me know. But
-->
we will see each other again
-->
tomorrow morning.
-->
That's good. Thanks, Joshua.
-->
Yep. Hey, thanks, guys.
-->
Hey.
-->
Hey, I called you on Teams
-->
and your cell phone.
-->
Just giving you a call
-->
back. But if you can,
-->
give me a call back.
-->
Here on my cell phone.
-->
Yo.
-->
They look well.
-->
That's good.
-->
The class is like, damn,
-->
I really like this.
-->
If the crane company cannot,
-->
I have
-->
a backup.
-->
Get warm in here now.
-->
Towards the end of the day, it
-->
warms up.
-->
There is a dead
-->
baby bunny from your
-->
dog.
-->
He went out and went straight for it,
-->
in his mouth.
-->
And then I said, oh, Dodo.
-->
And he dropped it and he came back.
-->
He will keep going back to the same
-->
spot until he picks it up.
-->
So I'm going to pick it up and throw it away tonight.
-->
Dodo.
-->
That baby bunny, what was that?
-->
Oh, that baby bunny.
-->
Are you ready to go?
-->
Let's go.
-->
Let's go.
-->
Oh,
-->
thank you.
-->
I'm going to go to your house.
-->
Now,
-->
I'm going to go to your place.
-->
Nothing.
-->
Nothing.
-->
Nothing.
-->
Yeah.
-->
Yeah.
-->
Nothing.
-->
Yeah.
-->
Yeah.
-->
There you go.
-->
What should I eat?
-->
Oh.
-->
Oh.
-->
There you go.
-->
I want to get this back home and I want to get this home and I want to save it for the next week.
-->
And I want to change it to clear this bill.
-->
And I want to change it.
-->
Get all the way off, get it off.
-->
I love that.
-->
It's cool to be really happy with the car.
-->
Yeah, it's really happy.
-->
Oh!
-->
What? I didn't see the problem.
-->
Yeah, it's nice.
-->
All right.
-->
Give me your heart.
-->
Give me your heart.
-->
I got you heart.
-->
Okay
-->
Oh
-->
No, my my mistress brought me one my mistress
-->
Whoa, I told you I didn't kill you
-->
No
-->
No
-->
I found the steam bun in the freezer and then you wrap it into jelly. Yeah for lunch
-->
Oh
-->
And they eat
-->
Oh
-->
So
-->
So
-->
So
-->
So
-->
So
-->
So
-->
Oh
-->
So
-->
So
-->
Hey
-->
Oh
-->
Hey
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
It just needs to be picked up and put on a trailer and
-->
And that's all it needs to be the only problem is is there's a fence between the swim spa and the trailer
-->
And I really don't want to remove the fence unless unless a crane service is going to cost too much and then I'll have to remove
-->
But I don't think it should
-->
Let me get that for you
-->
If you're going on so if you're going on Google Earth to look at it
-->
Do know that the guy with the the trees have all been trimmed and the metal covering has been removed as well
-->
Just to let you know the DNS crane service delivered it two years ago, but that lady is not responding to me
-->
So that's not calling around
-->
So it's been cleared out I can send you pictures but the address is two two one six
-->
L O C H lock Haven Drive in Plano, Texas seven five zero two three
-->
And
-->
I have pictures of I have video of the delivery and I have pictures of the
-->
All around it and everything else if it would help
-->
Yeah, that covers been removed so that and that covers completely removed it's wide open
-->
And then the trees
-->
Have all been trimmed back
-->
And and really cut really really good. So it's a clear opening to pick it up over that fence and
-->
Put it on a trailer
-->
Also, the neighbors have given permission if you need to put like your operators down over onto their property
-->
But there's also an alleyway there something but you know, there's it's been it's been prepped and ready to go
-->
You know in our opinion
-->
Yes, yes
-->
Over the cup the so there's two main trees on each side of it and both have been drastically cut back
-->
That's a 20-foot
-->
Nineteen hundred pounds
-->
As soon as possible, so it's a basically up to you
-->
Thursday Thursday Friday. Yeah. Well, I mean I expect you guys to be open it quickly
-->
So Thursday or Friday works Saturday works Monday works of this week and Monday or next week
-->
Yeah, let's see for Friday
-->
Yeah, how much well first of all how much you think this gonna cost me cuz I have no clue
-->
I
-->
Can send you the specs it like even the specs online it is 1900 pounds
-->
Well, it will be drained out I need so I'm so I'm in Austin area
-->
I don't live in Austin, but I'm living the area and I'm buying it from this guy
-->
so I have to range for transportation and pick up and so
-->
He's gonna leave the water in it and I'm gonna drive up with my guy who who has a trailer at a truck
-->
And I'm gonna look at it make sure it's running and then we got like two or three pumps
-->
So we can pump out all the water very quickly
-->
You know once I verify things are working
-->
So we would have it ready by like noon on that day whatever day we do it and it would be dry
-->
And yeah, it's 1900 pounds. I've got the full specs. I can tell you exactly how long how wide and everything else
-->
It's flat on the ground on concrete
-->
And
-->
Like I said, I have video of it being delivered by DNS crane service
-->
and I can see how they did it in
-->
Yeah, but I can figure something out if you guys you know can you or something I can figure out how to get the straps
-->
Under me where you just lift it up
-->
With like a jack or something
-->
It is I'm looking at it now and you got about a foot between the fence in the swim spa
-->
Yeah, I'm looking at picture right now
-->
Okay
-->
Yeah, I mean we can do military this
-->
We'll try but anyways, um, so so I mean
-->
But I think I think three hours you'll be I think three hours is gonna be plenty because I'm gonna be there early
-->
to make sure everything's ready and then
-->
Oh, oh I thought it was three hours on site
-->
No
-->
Okay
-->
Okay, and then if we do it you you can do it on a Friday
-->
Okay
-->
Well, let me let me let me verify my guy can pick up on Friday
-->
He I just talked to him and it sounded like he could
-->
And I can call you back. What is your name?
-->
We don't think I'm Josh
-->
All right, let me let me let me coordinate
-->
Getting the truck and trailer up there to pick it up and you guys are you know
-->
like I said, you're just gonna lift it up and put it on my goose neck trailer and
-->
Then and then everybody be done
-->
Oh
-->
Yeah, no, I just need to coordinate timing so I'll definitely be calling you back
-->
Thank you, I've got the crane service
-->
So we don't need anybody to remove the fence
-->
It's gonna cost a little bit more than having to pay someone to remove the fence
-->
But this way it's nice and easy and they just need to pick it up and move
-->
We will need to figure out how to lift it up so they can get their straps underneath
-->
Is there any
-->
Kind of clearance on the bottom
-->
They would need to get their straps underneath to lift it
-->
And it looks like it sits flat on the concrete
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh
-->
Oh