Hi, welcome back. In this video, you will learn how to download historical stock data via Python.
Here on my screen, you can see an example of a CSV file which contains historical stock data, these
data are for "Apple", the company.
So some companies allow their stocks to be sold to everyone in the world.
So basically, you can buy shares of a company.
And so every day there is a price of the share.
It opens at a certain price in dollars or euros or other currencies.
It reaches a peak the price every day, and then it reaches perhaps a low.
And then when the day closes at 4pm or 5pm, the closing price is also registered and also the volume
how many shares were sold that day?
So these are data for every day for a particular company.
These are for Apple.
So you can download any data for any company which is public, which means they allow you to buy their
shares.
And the website that has these data is finance.yahoo.com.
So you need to go to finance.yahoo.com and then understand how the data are made available.
So this is how it works.
Each company has a ticker symbol, so basically a code that identifies the company in the stock markets.
For Apple,
that is AAPL.
For Microsoft, for example, it's MSFT, and so you can find these ticker symbols
somewhere on the internet to know what company you want.
So for Apple, you'd want to enter AAPL and then search, that button, then you'd end up in the Apple Web
page for Yahoo Finance.
So this is how you do it manually.
Then you'd go to Historical Dot so you can do all these clicks manually, but also with selenium.
And then this is the historical data web page.
But then you can click here and then click on Max, so Max means you get all the data since the company
started to sell stocks.
So by clicking that and clicking apply.
You should get data from...
In the case of Apple, that would be from 1980.
So 40 years ago.
That's a lot of data.
And you can see the data here.
You can scrape them from this web page.
Or even better, you can use this download button.
So you can click with selenium in the download button.
And that will give us a CSV file.
You see here.
It's downloaded.
And yeah, those are the data.
So from 1980, up to row 10361.
So that's 2022.
January 12.
Now.
This is how you do it manually.
How do we do it with the Selenium now? Well, you can do it by repeating those clicks with
Selenium as you learned previously.
So you find the element, you click it using the dot click method.
And so one step by step, or you can be smarter and not use selenium at all, because what happens
is that if you click this download button, what's happening is that you are clicking a link.
You can get that link by right clicking on the download button and then go to copy, link, address
or copy link, depending on your browser.
I'm on Chrome, so I have this copy link address. With that copy link address.
You can go to Python and paste that link there.
So that link is the link that if visited, it downloads a CSV file.
So instead of using Selenium, we can use this link because this link has embedded the period.
So from that period,
that's in POSIX time, we will convert it, so that first period is the first date in the year 1980 when
the data became available.
And then the second period is the last day.
So yesterday. Because the stock today is not open yet, so that is from yesterday.
So those are the periods.
So this is basically a query, and we can modify this URL by inputting other periods here to get any
period that we want with Data.
Then we have the interval "1d", so we want data for every day.
One day the interval, and then we have the ticker symbol.
So the codes of the company, we want data from Apple in this case.
So we write Apple there, after the download, so download, slash Apple, or MSFT for Microsoft or other ticker
symbols for other companies.
So there's a pattern that can be followed, and you can download any CSV data, any stock data
with Python.
So let's do this.
To download files with Python, you can use requests.
Then once you have a URL as a string, so let's say you URL is equal to that string, so enclose the
URL in quotes and then content is equal to requests.get. Get is a method which expects a URL.
So the URL is that one.
And then you say dot content. Content is a property.
So that is the requests objects and then you grab the content of that request object.
Let's see what content is.
By printing it out.
So it's a bite object.
And currently it's forbidden.
That is because we need the headers.
So, the web page, the Yahoo Finance web page doesn't allow scripts to download files.
You can fix that by writing some headers.
So that's a variable.
It has this dictionary.
You can find this attached in this lecture, in the lecture resources.
So this will impersonate a browser, so the scripts will behave like a browser, and so it will be allowed
to download the file.
But this header symbol now you have to provide it to the get method as a second argument.
So headears is equal to to headers.
The headers arguments is equal to that variable.
Let's try again.
And this time it seems to be working.
So you see some dates here.
With the price, the opening price, the closing price and so on.
So those are the data, but it's just a byte object.
What we need to do now is to create an empty file and then write those bytes in the file so that we
have an actual file on disk. To create a file we use with open.
And set a name for the file such as data.csv. Open it in write byte mode.
You can also use, for CSV files you can also use write, and then change that content to text to deal
with simple text.
But when you use content here and 'wb' here, this is a more generalized approach.
You can download binary files as well, such as audio files, MP3, MP4 files, zip files.
So any binary files which are not text files.
So this method includes everything non-binary and binary files.
So I'll use this method. With open data.csv open as binary in write mode, as file.
So in this file mode, which opened in write mode, we can write what's the content.
So that's content now is written in this empty file, and so we will end up with a file on disk.
First, it's printed out from this print content, you can remove that. And then,
this code is executed.
So we we end up with this data.csv file.
Yep, that's the file. You can download it.
by going to here and you can open it with Excel or any other spreadsheet programs.
So we have this header, the date, the open, the high, the low, the close, the adjusted close and the volume
and it runs up to yesterday, that date, right?
Now, let's further improve our scripts by letting the user choose whatever period they want and whatever
ticker symbol they want data for.
For that, I would like to implement an input function. Actually two input functions.
The first would be from date eq,ual to input and enter a message for the users something like:
"Enter start date".
In this format.
So the user is supposed to answer the start date from when they want data for, and that should be in
year slash month slash day, that format. And then we will parse that format and and convert it into
a timestamp into this POSIX time, EPOCH time.
So this here is seconds.
And this is a time system.
It is seconds from the year 1970, 1st of January of that year.
And so we can convert that into human readable date time.
Actually, we get the human readable date time in this format, and then we convert it to this timestamp
so that we can supply this to this URL because this works with this kind of format.
So I would like to duplicate this input function and renamed the variable to _date and end_date
in that format again.
We're going to use a chain of methods to convert these data type formats into Epoch time format.
It's simple
actually. I'll show you how I can do it.
I'll show you first in the interactive shell and then implement it here.
So let's suppose that from date given by the user is 2022, one for January and 10 for the date.
So 10th of January 2022.
So that's a strength that the user will enter and that is stored in this variable.
Now, how can we convert it into this format?
Well, first we need to convert it from a string into a date time object.
For that, we need from datetime, import datetime.
So we're going to need to import that as well in here.
So past it there so that we can have it for later.
And then, you use this date time here.
And use its strptime method, which gets two arguments, the first argument is the string.
So that is the string.
And the second argument is the format that this string contains.
So this will give us a specialized datetime object.
You will see what that is.
So the format of this string is percentage Y.
So that means the year is the first one and then there is the slash.
And then there is the month the percentage lowercase m, and then a slash again.
You see, I'm following this, and then percentage d. So these are format codes, percentage, capital Y.
Percentage lowercase m, and percentage d. So I'm going to store this in a variable D and then D is
this datetime format.
So it's the same date, 10th of January 2022.
Now, having this means we can convert it into epoch time.
For that, we need the time module.
So inport time, and then epoch would be time.mktime.
And then you point to the d variable, and then you point to the time tuple variable which converts
these datetime object into yet another similar object.
So this mktime needs that other object so execute, and epoch finally is in that format similar
to that and that.
So let's copy this code we get from time we convert it into d. Let's see from datetime.
Same, to datetime, so we converted the end date as well into this, so paste it there, change these too, to_date.
So that string goes here and that goes in here of and then we need to convert into epoch, so import time.
And write, creates a variable from_epoch is equal to
time.mktime().
And this is from_datetime.timetuple() that time to pull so from_epoch, and then let's duplicate this.
This is to_datetime.
To_epoch.
And that's it, so once we have this, we can inject them into the URL, how can we do that?
Well, using an f string.
So add F in front of the url string, and then instead of period here,
instead of this number, period is equal to curly brackets and then goes from epoch, that variable.
So that number will be placed in here.
So careful here.
After that comes this and operator.
Then we have period_to and then we have the other
number, it's to_periods.
And of course, you can also make the ticker symbol dynamic.
For example, ticker is equal to input.
Enter the ticker symbol.
So, for example, the user will enter AAPL for Apple, et cetera.
Therefore, we need to replace AAPL to
these curly brackets and then enter ticker there, that variable.
So, for example, this time I'll try MSFT for Microsoft's, so let's run the code, enter the ticker
symbol, it was MSFT, FT, enter the start date, 2021.
Let's say, November for, 11 and then the 20th of November.
And then, the end date, 2021, December, the 20th, enter. I've got an error, to _period is not defined.
Obviously, I placed here to_period, but it should be to_epoch.
So let's try again.
MSFT.
2021, 11, 20, 12, 20, enter.
And so we have another error, but this time is the error is not from the Python interpreter.
It is from the server.
Why?
Well, because this print function is printing out the content and this is the content we got.
And in the content, which is written by the server, we get this message that the parameter period
is of an invalid type.
That is because this is tricky, but if you remember our from_epoch and the to_epoch values were in
in a float format, so dot something dot zero or something else and this URL does not accept this
kind of decimal point
number.
So we need to convert it into an integer and we can do that just here.
So we create from_epoch and then we convert it into an int and we convert to_epoch into an int
as well.
And that should do it, so run again.
MSFT.
2021, 11, 20th and until December.
Enter and yeah, those are the data.
Finally.
And you can also see them in here.
So that's one month of data.
It's 19 rows in December.
So that excludes holidays and Saturdays and Sundays.
That's why we don't have thirty one rows, but we have only nineteen, so lots of holidays in December.
And so it starts from the 22nd of November, obviously the 20th and the 21st of November,
where a Saturday and Sundays, probably.
And so we don't have those data there.
And that's the final code, so I thank you, and I'll see you later.