2 years ago

#73156

test-img

a7dc

How to speed up data download from Alpha Advantage API?

Forgive my ignorance but I'm struggling with how to approach this. I'm trying to write a program which pulls back historical gappers and runs backtesting strategies on the data.

First I'm getting a list of tickers from IEX cloud:

import requests #The requests library for HTTP requests in Python
import csv
import pandas as pd
from secrets import IEX_CLOUD_API_TOKEN_TEST
from secrets import IEX_CLOUD_API_TOKEN_LIVE

api_url = f'https://cloud.iexapis.com/stable/ref-data/iex/symbols?format=csv&token={IEX_CLOUD_API_TOKEN_LIVE}'
tickers = []
with requests.Session() as s:
    download = s.get(api_url)

    decoded_content = download.content.decode('utf-8')

    cr = csv.reader(decoded_content.splitlines(), delimiter=',')
    my_list = list(cr)
    for row in my_list[1:]:
        tickers.append(row[0])

print(tickers)

Then I'm looping over the tickers and using the Alpha Vantage API to pull 2 years worth of full market data:

stocksIntraDay = []
dates = ['year1month1', 'year1month2', 'year1month3', 'year1month4', 'year1month5', 'year1month6', 'year1month7', 'year1month8', 'year1month9', 'year1month10', 'year1month11', 'year1month12',
'year2month1', 'year2month2', 'year2month3', 'year2month4', 'year2month5', 'year2month6', 'year2month7', 'year2month8', 'year2month9', 'year2month10', 'year2month11', 'year2month12']

def getIntraday():
    # cycle through our symbols (only the first one for testing)
    for i in tickers[1]:
        ticker = i
        print(ticker)
        apiKey = 'MY_KEY'

        # alpha splits out the dates it gives us into 24 segments
        # cycle through the segments
        for i in dates:
            date = i

            # api string where we pass in the symbol and the current date
            api_url = f'https://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY_EXTENDED&symbol={ticker}&outputsize=full&interval=1min&slice={date}&apikey={apiKey}'

            # download the csv file for each date
            with requests.Session() as s:
                download = s.get(api_url)
                decoded_content = download.content.decode('utf-8')
                cr = csv.reader(decoded_content.splitlines(), delimiter=',')
                my_list = list(cr)
                for row in my_list:
                    print(row)


getIntraday()

The download is really slow. I've seen this article here but I can't make sense of it. How can I speed up the download of the files and improve this code?

python

stock

0 Answers

Your Answer

Accepted video resources