Hello guys.
So we are going to continue our Python series.
And in this video we are going to discuss about a new library which is called as pandas.
Again we are into the Data Analysis with Python series section.
And in our previous video we have discussed about numpy, which is an amazing library where you can
actually perform a lot of operations with using this.
Along with that, uh, in this video we are completely going to focus on pandas.
Uh, the first topic will be pandas data frame and series.
So pandas is also a very important library because it has a lot of data manipulation capabilities.
Right.
So we will be discussing step by step.
We'll be writing down the code in front of you.
And again I will discuss in such a way that we take multiple examples and we try to find out its capabilities.
Right.
So here you can see that pandas is a powerful data manipulation library in Python, widely used for
data analysis and cleaning.
It provides two primary data structures series and data frame.
We will understand the difference between series and data frame.
A series is one dimensional array like object, while a data frame is a two dimensional size, mutable
and potentially heterogeneous tabular data structure with labeled axes.
Okay, so the basic difference between series and data frame is a series is a one dimensional like array
object.
Like how we create a one dimensional array in numpy.
We can also convert that particular numpy array, which is one dimensional, into a series right and
data frame.
Altogether, whenever we have a two dimensional numpy array.
We can also convert that into data frames, because data frame is also a two dimensional, mutable,
and potentially heterogeneous means, it can have different different elements over there.
So let us go ahead and start.
Now, before we go ahead and start, the first thing that you really need to do is that go to your requirement
dot txt and go ahead and install pandas.
Okay.
So we need to have this library again.
As I showed you there are two ways.
Either you can do it from here or you can just right pip install pandas right.
So right now I will just try to show you from here.
Let me just go ahead and open my terminal.
Okay.
Now, inside this terminal, I'll go ahead and open my command prompt.
Or my V and V is in.
Make sure that your V and V environment is activated.
Now I'll just go ahead and run pip install minus r requirement dot txt.
Okay.
So here you can see that automatically my requirement dot txt is going to install uh everything that
is required.
You know uh like uh pandas numpy.
So numpy is already done.
Now the pandas installation is basically happening.
So it is going to take time based on that.
Uh, all the other libraries that is already installed, I don't have to do any changes for that.
So it is going to take some amount of time with respect to the installation.
And remember now you can see the installation is done.
That basically means I can actually use this particular, uh, pandas library from here.
So just to check what I'm going to do, I'll just go ahead and write import pandas as PD.
Okay.
So now if I go ahead and execute it.
So let's see.
So here you can see perfectly.
My pandas library is basically working okay.
As I said, the first topic that we are going to talk about is series.
And what did we define a series.
Uh, it is just like a pandas series is a one dimensional.
So here I have written a definition.
A pandas series is a one dimensional array like object that can hold any data type.
It is similar to a column in a table.
Right.
So let us go ahead and create this particular series.
And then you'll be able to understand.
So first of all we will go ahead and import pandas.
And this time whenever we are importing pandas I'm going to use the alias which is called as PD.
Okay.
So that I don't have to write pandas dot pandas dot.
Like let's say this is my data.
And here I'm going to basically use 12345.
Okay.
Let me quickly go ahead and create a series.
So it is nothing but PD dot series.
Series is an inbuilt function.
And here I can give my single dimension data over here and automatically I'll be able to get the series
itself.
Now, if I go ahead and print this series.
So here you'll be able to see.
Series slash N okay.
And here is nothing but my series.
So here you can see this is how a series look like all the elements one, 234, five.
These are the default indexes okay.
So don't worry about this 012345.
This is the index with respect to the series.
But you can just imagine like this is one column one single column.
And that is what series is all about okay.
So this is one way of creating series just from a list.
The other way that I am actually going to show you is from dictionary.
So let's go ahead and create a series from Dictionary elements.
Okay.
Uh, so for this, what I'm actually going to do and why I'm saying this is as a series if I go ahead
and print type of series okay.
So type of this particular series variable.
So here you can see that it is a pandas code dot series uh data type Okay, now create a series from
a dictionary.
So let's say this is my data.
And here I'm just going to use some of the elements like a colon.
One next element will be b colon two.
The third element will be c colon c colon three.
And these are my elements.
Now in order to use this particular data and convert this into series I will just go ahead and write
PD dot series.
And here we are specifically going to use the data okay.
Now if I go ahead and check this out series series underscore dictionary okay.
This is my variable.
If I go ahead and just print it okay.
So let's go ahead and print.
And here I'm going to basically go ahead and write series underscore dict.
Here you can see this becomes my index this key what I have actually written a b and c this becomes
my index.
Before the default index was 01234.
But here you can see that these are my default indexes now.
And that is what is important for creating series from a dictionary.
Whatever will be the keys that will be considered as addition as the index.
And all the values will be considered as the series values.
Okay.
Now I can also make sure that I can give a different index over here based on my requirement.
So for that what I will do let's say this is my data.
So let's say I will go ahead and write ten 2030.
Now my pandas series and all this will be required.
See, whenever we try to open an Excel file right there will be finding a lot of rows, columns, you
know different different data will be there.
And to read that do some manipulation.
This pandas will be very much important.
Important.
I will also show you how to read a particular data and show you amazing things over there, how you
can explore it in the form of data frame and series itself.
So let's take this particular data and I will just go ahead and create my indexes.
Let's say my indexes will be something like this a b, B.
Okay.
So these are my indexes that I want to do.
And I have to always use uh based on the number of elements that many number of indexes needs to be
there right now in order to probably use this index over here right before without using any like any
key value pairs.
Right.
How do I write it down.
So here I will write PD dot series okay PD dot series.
And here specifically with respect to the series I will require data.
And then the second parameter that I'm actually going to give is my index.
And this is what index I'm actually going to give okay.
So if I go ahead and execute this.
So here you can see that this will basically be my index.
And these are my elements.
So this way also you can probably create your series.
And.
All right uh as said what is the series.
It is a one dimensional array object that can hold any data type.
It is similar to a column in a table.
Okay.
So here, uh, these were all the things.
Now, uh, let's talk about data frame.
Okay.
So let's talk about data frame.
So here uh data frame.
As I said it is a two dimensional mutable and potential and heterogeneous tabular data structured with
label axes, that is rows and columns.
It is similar to a table in an, uh, database or in an Excel.
Let's say that this is my Excel sheet I am going to use for this particular example.
So here you will be able to see I'll be having some column names.
I'll be having so many different rows.
Right.
This is a multi-dimensional array right.
Or multi-dimensional data itself.
Right.
I have so many things I can directly read this, convert this into a data frame, and with the help
of data frame, what is data frame that it will be having multiple rows and columns.
That's it.
In series it just has a single column in data frame.
It can have multiple uh rows and columns.
Okay.
So let's go ahead and create one by one.
First we will try to see how to create a data frame completely from basics.
And then uh we will also try to read a specific CSV file and see how a dataframe is basically created.
So here you will be able to see create a data frame from a dictionary.
Dictionary of list.
Okay, so I will consider a I'll create a list a dictionary with the key value pairs.
And in that key value pairs values will be in the form of list.
Since I really need to have a multiple data.
So here I will go ahead and create my data.
So this is how we basically create a dictionary.
And here we are going to use name value pairs right.
A key value pairs.
Let's say my first column is name okay.
And in name I'm going to use multiple names right.
Considering multiple rows first name will be Krish.
Second name will be John okay.
Third name will be Jack.
Okay.
So these all names are specifically there.
Then let's go ahead and use this age inside this.
Let's go ahead and create some values like let's say 25, 30 and 45.
Okay.
So these are my second column, right.
Since I'm making sure that I have three records.
So all the fields, all the column names that I'm specifically giving is also having three different
values.
Let's say city I'm going to use over here.
City will be let's say over here I go ahead and write Bangalore okay.
Or let's say here, I'm going to write New York.
Okay.
And the third one, let's say I'm going to basically write Chicago, Chicago or Texas anything.
Okay.
Texas.
So this okay.
This is a city.
This is a state.
Right.
So let's say I will go ahead and write Florida okay.
So these are some of the elements that I've actually written.
But in short what is this.
It has basically three columns.
And it has so many rows.
Right.
This is my first row.
This is my second row right.
Not first row sorry.
First row has the elements.
Second row has the element.
John.
Jack.
Chris age is 25.
He lives in the city of Bangalore.
Similarly, John name is over here.
His age is 30 and he lives in New York.
Similarly, Jack is over here.
His name age is 45 and he lives in Florida.
Okay, so all those specific elements is basically present.
Now what I'm actually going to do, I'm going to basically go and write PD dot data frame.
And this will basically get created.
How to create a data frame I'll use this entire data and basically create it.
Right.
So if I go ahead and print now you'll be able to see that I will be having rows and columns.
Right.
So this will be my DF okay.
Now let me just go ahead and print type of DF okay.
So DF and so this is my DF.
This is my zeroth index.
You can see three columns name age and city.
I have crushed 25 Bangalore John 30, New York Jack 45 Florida.
Like this is just like how we have the data in the Excel sheet, right?
And clearly you can probably see with respect to this.
Okay.
Um, and if I'm going to right type of DF, I can actually see that all this information is there now.
So it's not like I always have to keep it in the form of data frame.
I can also use numpy and I can convert this into NP dot array.
Okay.
So if I go ahead and write NP dot array with respect to df, I'll be able to get this array.
Okay.
There I will just skip this index and it will just skip this column names like this.
Also you can basically do okay.
So this was one example like how you can actually use a dictionary from a list of dictionary you can
specifically create or from a list of uh from a list, right.
Nested list.
Also is there this list is basically inside a dictionary.
You can also see that okay, now what I will do I will create a data frame from a list of dictionaries.
Create a data frame from a list of dictionaries Okay, I'll do the same thing over here.
Now, let me do one thing.
Let me go ahead and create my data.
And here I will just go ahead and create a list.
And with respect to the list what I will do, I will just copy this entire thing, this three fields.
Let's say I'm just copying it.
Okay.
Now instead of writing like this, right where name is equal to all the names, age is equal to all
the names.
So what I will specifically do, I will say, hey, uh, let's create a key or let's create a dictionary
over here, and this single dictionary will be this one.
Uh, first of all, I will just going to use name is equal to crush.
Right.
So name colon crush.
The second field that I will probably go ahead with is nothing but age.
So age is colon with 32 okay.
And the third column that I'm specifically going to use is nothing but city okay.
And city I am just using it as colon Bangalore.
Okay.
Bangalore.
And here I will be using this.
Okay.
So this becomes my one row data.
Okay.
And I will remove all these things.
Now similarly I can go ahead and create with respect to any number of key value pairs.
So let's say I go and paste the same thing.
Go ahead and paste the same thing just to make sure that I have this entire thing in the similar way.
Okay.
So what this is this is the list of dictionary key value pairs.
So this becomes my first row like name 32 Bangalore.
Right.
Like this.
Let's say I'm going to change something over here 34.
Let's say I'm going to write John over here.
Right.
And here I'm going to write, uh, Bobby.
Okay.
Here I'm going to write Jack.
Okay.
All these elements are there.
Let's go ahead and execute it.
And I will go ahead and use the same thing.
PD dot data frame.
Okay.
And let me just go ahead and execute it.
So here you can see Krish 32, Bangalore John 34, Bangalore 32, Bangalore.
Jack.
32.
Bangalore.
So all these values are specifically there with respect to this particular data frame.
Right.
So this is also how you can actually go ahead and create it.
Uh, it is up to you how you really want to go ahead whether you want to go ahead with this.
But most of the time, what we specifically do is that whenever we are creating something, right,
whenever we are creating, uh, we mostly work with bigger data set, right, like CSV file, Excel
file and all.
So let me now just go ahead and show you how to probably read any kind of data set with respect to this.
And uh, let's say if I am having a CSV file, how can I probably read it?
You know, I will be showing you multiple examples that I when I go ahead, you know, with Excel and
all.
But right now I'm just going to use this online sales data dot CSV which is available over here.
So for this what I will do we'll go ahead and write PD dot read underscore csv.
There are a lot of function like read, underscore excel and all.
But I'm actually going to use this okay.
so read underscore CSV.
And here I'm going to just call this CSV file.
Let's rename it.
Okay I will just go ahead and write this as sales data.
Let's see whether this will get converted.
Sales underscore data dot csv.
Okay.
So I will just go ahead and rename it.
And let's go ahead and write sales uh underscore data dot csv.
Okay.
And this basically becomes my data frame.
Now if I want to go ahead and see the top five elements of data frame, I'll just go ahead and write
DF dot head.
Right.
So DF dot head is basically giving me this entire information.
If I remove this print also you'll also be able to see much more clearly.
So this is how it looks like.
This is my first column, second column, third column and all.
And DF dot head of five is basically giving me top five records 01234.
Similarly, if I want to see the last five records, I can actually use DF dot tail and it basically
gives me the last five records.
Right.
And these are the default indexes that we are specifically getting.
Right.
Um, so guys now let's go ahead and see how we can access the data from this particular data frame which
is important.
So here I'm going to basically write accessing data from data frame.
Okay.
So right now this is my data okay.
Like all this information is specifically data over here that is available.
Um or let me just go ahead and create this one.
This also looks good.
Okay.
So I will just go ahead and use this particular data.
Let's see.
So this is my data right now okay.
Uh, crush uh age and all.
So if I go ahead uh, also use DF right.
So DF is basically this one.
Okay.
Now after I go ahead and create DF.
Now let's say that I want to pick up all the elements with respect to name.
Right.
So here what I can go ahead and do, I can just go ahead and write DF of name.
Right.
So whenever we are specifically using anything related to columns, we can directly go ahead and call
the column name.
So if I go ahead and write DF of name, I'm actually able to get this right.
Now very important thing that you'll see if I'm picking up one column right.
It actually becomes what.
It becomes a series.
Right.
So if I go ahead and see type of DF of name, you will be able to see that I'm actually able to get
series over here.
Right.
So since I'm just picking up one, uh, one entire column itself.
Okay.
Now this is one way.
Now there is also another way which I will be showing you.
I will be using DF of look, look of zero.
Look of zero basically means I am just going to pick up my zeroth row, right?
If I really want to go with row index that time, I will be specifically using this lock operation.
Okay.
Lock lock lock lock lock.
Okay.
Lock operation.
That basically means, uh, row index.
Okay.
Let's say I want to go ahead with, uh, um, column index also.
So I can also go ahead and write index location eye lock.
Right.
And here if I go ahead and give zero that basically means I'm picking it up.
Chris 25, Bangalore.
So here you can see name, age.
City.
Name, age.
City.
Chris 25 Bangalore.
Right.
Let's say I will just want crush.
I can go ahead and write zero comma zero.
Right.
Uh, so I'm getting an error over here.
Oh no, it is not an error.
It is a warning.
It says zero comma zero zero comma.
Zero is nothing but this specific element.
Zero comma zero.
Right.
So first is something related to you can just consider location of zero comma zero is basically giving
you this thing uh row and index okay.
So let's say I go ahead and execute this.
Let's try with iloc of zero.
So here you can see I'm getting this.
If I go ahead and zero of one I'll be able to see this 25 if I go ahead and put zero comma two I will
be able to see this.
Right.
So all the information is basically there.
If I go ahead and write zero colon.
Right.
So here you can see there is a key error because it is not able to find out the error.
So zero colon two is basically giving me this particular value Bangalore okay.
So you can also use index location to probably get some kind of information over here.
Okay.
Now let's say there is one more function which has been never discussed anywhere.
Probably I guess I've never seen I've hardly seen in any tutorials I have discussed test.
It is nothing but accessing a specified element.
Okay, so I hope you got a difference between location and index location.
Index location.
Over here.
Let's say if I go ahead and write zero of two specifically working with index location, this is only
with respect to lower index.
And whenever I really want to work with column I can just go ahead and write this like that directly.
The column name.
These are the three ways how we probably access.
Okay.
So let's say that accessing a specified element if I'm specifically using what does this basically mean.
Right.
So if I go ahead and write df of okay df of name okay.
So here you'll be able to see that I'm able to get the all the names in this specific dataframe.
Right column.
Chris.
John.
Jack.
So how many records are there with respect to that particular information?
I'm getting it.
And if I go ahead and just write DF dot head or DF right.
So if I go ahead and right here, I'll be able to see okay.
In my zeroth row it is Chris first John.
And this is there right now let me do one thing, something like this okay I will write DF dot at okay.
This is very much important.
See at let's say I give one comma and here I am just going to give h.
I am able to get this particular value.
Why I am getting this value.
Let's see okay.
So if I go ahead and write DF okay I'm saying hey from from a specific element you have to specify,
you have to take it.
Right.
So from the age, which is my first index, my first index is this one.
Right.
So I am actually able to get 30 right.
If I similarly go ahead and write two of H which I will be able to get it right.
45 if I go ahead and write DF of at and let's say instead of two, I will oh, I will just go ahead
and go ahead and write name.
So I want the second index name.
So it will be nothing but Jack.
Right.
So this is also very amazing at search.
Uh, there is also something called as I at okay.
So I will go ahead and write accessing uh specified element okay using I at okay.
Now what is this I at.
So here you can actually give you a row index and column index.
Let's say if I go ahead and write df of two comma two okay.
That basically means uh let's see what is the error.
So here okay I did not write I at I at operation of two comma two.
So here you will be able to see second second element.
Second second element is nothing but 0012 okay.
And 012 column wise.
This is my second index.
So I want to pick up Florida.
So I am able to get Florida over here right.
So I at here you are basically giving your both row index and your column index.
So I hope you are able to understand all these things in an amazing way.
Now, uh, let me do one thing.
Okay.
So right now this is my DF.
Okay.
Now let's perform some data manipulation with data frames.
This will basically give you an idea okay.
What are data manipulation with data frames?
Data manipulation with data frames.
Okay.
Now what I'm actually going to do.
Okay.
So you know this is my DF okay.
Now what I'm actually going to do I will be adding a new column.
Okay.
And let's say uh how to add a new column I want to add a new column which is called as salary okay.
So let's say how do I do it.
So I will go ahead and write DF off my salary name column.
Okay.
So here you can see salary column okay, which will be equal to the salary that I am probably planning
to add over here.
Right.
DF of salary okay.
And then what I will do is that I will just go ahead and use the specific list.
Let's say I will go ahead and write 50,000 60,070 thousand okay.
So this basically becomes an algorithm.
So if I go ahead and print the DF.
Now here you can see a new salary is basically added.
So with the help of this we I am adding a column right over here.
Perfect.
So this is great.
Let's say I want to remove a column.
Remove a column okay.
So what I can do I can write DF dot drop.
And here I'm going to basically say salary okay.
Now when I'm doing DF dot drop I'll get see.
If I execute this I will get an error salary not found in the excess okay.
But you can see over here salary is basically present.
Now what does this axis mean?
Axis has two values okay.
One is the row value.
It will probably go ahead and check with respect to row index.
First of all it is checking row index.
It is not even going to the column index because by default your axis value will be zero.
Okay.
So if I am executing, it is basically checking by default in the row index.
And row index does not have a name like salary.
So if you want to make this entire code, check the column index like name, age, salary, city, salary.
You have to convert this into axis is equal to one.
Okay.
So if I go ahead and execute this here, you will be able to see that I am not having this salary column.
But still if I go ahead and again check DF again, the salary column came.
What has happened even though I dropped it but still in my DF after some time.
Like if I go ahead and execute it again, salary is coming.
So this is very much important when this drop is basically happening right with axis equal to one,
it is not a permanent operation, right?
There is just a temporary operation, and it does not even save the state of the data frame.
So in order to make it save it, you have another parameter which is called as in place is equal to
true okay.
Or in place is equal to true.
So once you do in place is equal to true, then what it is going to do whatever drop operation you are
doing, it is going to do it permanently.
So if I go ahead and execute it, and if I go ahead and see DF here, you can see now I do not have
that specific column.
Okay.
And this is how you remove a column okay.
Remove a column.
Now let's say that uh, I want to do some kind of operation where I will add age to the column.
Let's say after one year what happens?
Everybody's age increases by one.
So here I'm going to write DF of age okay.
Plus one.
Right.
And this is basically going to get applied to all the values inside the age.
And this should be replaced in DF of age itself Okay.
So now what will happen is that initially I had 25, 30, 45.
Now the age is going to be 26, 31 and 45.
So if I go ahead and display it now.
So here you can see 2631 and 46.
Right.
2631 and 46.
So one it has got incremented by one okay.
Now you you can add any number of rows as you like or any number of changes or any manipulation that
you specifically want to do.
Okay.
And this is how you will be able to do it.
Let's say I want to go ahead and drop based on row.
So what I will do I will just go ahead and write DF of drop.
And I can also give my index.
So if I go ahead and write like this again, this is again not a permanent operation.
Because if I go ahead and say DF, I'll again back it.
So in order to make it permanent, I will just go ahead and write in place is equal to true.
Okay.
Now if I go ahead and execute it I just have two records.
That is John and Jack.
Okay.
Uh, this was mostly about how you can probably read how you can probably do things, you know, uh,
and some of the information regarding data frame and, uh, series.
But as we go ahead, we will be discussing more about manipulation.
Now, let me do one thing.
Let me show you again, uh, with respect to some of the things that you really need to know about data
frames, I can also say it as a attributes which will be important.
Right?
So if I go ahead and write DF dot head of five, I'm going to get the first five elements.
Uh, along with this, uh, I'm just going to copy and paste some more additional things that you can
do.
You can see the data types.
You can if you okay.
If you go ahead and write DF dot describe you'll basically get the statistical summary okay.
So let's do one thing.
Let's remove this and let's see the statistical summary df dot data df dot describe.
Or if you don't want to see like that I will go ahead and write DF dot describe.
If I go ahead and write it.
Here you can see these are some statistical analysis like count how many number of records are there
mean?
what is the mean of transaction ID, which is not important, but mean of unit sold is important.
Mean of unit price is important.
Total revenue standard deviation.
Also you can probably see over here minimum maximum values 25 percentiles 50 percentile 75 percentile
seven Max.
All those values will be able to see with the help of uh df.describe.
Okay.
Now, uh, what I'm actually going to specifically do over here is that, uh, let's, uh, use one
code.
Okay.
So what I'll do, I will just say, hey, uh, do one thing.
Okay.
Um, based on this unit price.
Okay.
Based on this unit price, or let me just show you that in the next session, because I'm going to probably
do some more operations over here and do perform some data manipulation.
In this video, I wanted to cover about data frames and series, get a basic hands on.
And now in the next video we'll talk more about the manipulation.
So I hope you like this particular video.
This was it from my side.
I will see you in the next video.
Thank you.