2 years ago
#73156
a7dc
How to speed up data download from Alpha Advantage API?
Forgive my ignorance but I'm struggling with how to approach this. I'm trying to write a program which pulls back historical gappers and runs backtesting strategies on the data.
First I'm getting a list of tickers from IEX cloud:
import requests #The requests library for HTTP requests in Python
import csv
import pandas as pd
from secrets import IEX_CLOUD_API_TOKEN_TEST
from secrets import IEX_CLOUD_API_TOKEN_LIVE
api_url = f'https://cloud.iexapis.com/stable/ref-data/iex/symbols?format=csv&token={IEX_CLOUD_API_TOKEN_LIVE}'
tickers = []
with requests.Session() as s:
download = s.get(api_url)
decoded_content = download.content.decode('utf-8')
cr = csv.reader(decoded_content.splitlines(), delimiter=',')
my_list = list(cr)
for row in my_list[1:]:
tickers.append(row[0])
print(tickers)
Then I'm looping over the tickers and using the Alpha Vantage API to pull 2 years worth of full market data:
stocksIntraDay = []
dates = ['year1month1', 'year1month2', 'year1month3', 'year1month4', 'year1month5', 'year1month6', 'year1month7', 'year1month8', 'year1month9', 'year1month10', 'year1month11', 'year1month12',
'year2month1', 'year2month2', 'year2month3', 'year2month4', 'year2month5', 'year2month6', 'year2month7', 'year2month8', 'year2month9', 'year2month10', 'year2month11', 'year2month12']
def getIntraday():
# cycle through our symbols (only the first one for testing)
for i in tickers[1]:
ticker = i
print(ticker)
apiKey = 'MY_KEY'
# alpha splits out the dates it gives us into 24 segments
# cycle through the segments
for i in dates:
date = i
# api string where we pass in the symbol and the current date
api_url = f'https://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY_EXTENDED&symbol={ticker}&outputsize=full&interval=1min&slice={date}&apikey={apiKey}'
# download the csv file for each date
with requests.Session() as s:
download = s.get(api_url)
decoded_content = download.content.decode('utf-8')
cr = csv.reader(decoded_content.splitlines(), delimiter=',')
my_list = list(cr)
for row in my_list:
print(row)
getIntraday()
The download is really slow. I've seen this article here but I can't make sense of it. How can I speed up the download of the files and improve this code?
python
stock
0 Answers
Your Answer