我是 python 新手,正在尝试了解多线程
这是我到目前为止所拥有的:
d_thread = {0:(('instrumentType', 'OPTSTK'), ('symbol', 'INFY'),
('expiryDate', 'select'), ('optionType', 'PE'),
('strikePrice', '2800'), ('dateRange', 'day'),
('fromDate', '11-04-2012'),('toDate', '12-04-2012'),
('segmentLink', '9'), ('symbolCount', '')),
12:(('instrumentType', 'OPTSTK'), ('symbol', 'INFY'),
('expiryDate', 'select'), ('optionType', 'PE'),
('strikePrice', '2400'), ('dateRange', 'day'),
('fromDate', '27-04-2012'), ('toDate', '28-04-2012'),
('segmentLink', '9'), ('symbolCount', ''))}
这本词典大约有500 个键
。 键
是pandas dataframe
的索引
。
我想创建 10 个工作线程
来发出请求
,然后将数据放入数据帧
中。我不知道如何让工作人员在一个线程完成时选择 next
键。
到目前为止我所拥有的:
import threading
from queue import Queue
import requests
hist_lock = threading.Lock()
def opthist_job(worker,d_thread):
headers = {
'Pragma': 'no-cache',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'en-US,en;q=0.9',
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36',
'Accept': '*/*',
'Referer': 'https://www.nseindia.com/products/content/derivatives/equities/historical_fo.htm',
'X-Requested-With': 'XMLHttpRequest',
'Connection': 'keep-alive',
'Cache-Control': 'no-cache',
}
params = d_threading[0] # This is where I need to get the value of key
opthistdf = requests.get('https://www.nseindia.com/products/dynaContent/common/productsSymbolMapping.jsp', headers=headers, params=params)
with hist_lock: # I am not sure if this is required in this instance.
#### Some more functions ####
def threader():
while True:
worker = q.get()
opthist_job(worker)
q.task_done()
q = Queue()
for th in range(len(d_threading.keys())):
t=threading.Thread(target=threader)
t.daemon = True
t.start()
提前致谢。
最佳答案
您可能需要使用 mulitprocessing.pool 提供的东西图书馆。
让我们尝试使用map功能:
from multiprocessing import Pool
def f(parameter): #in parameter you have a tuple (key, value) from your dict
result = requests.get('https://www.nseindia.com/products/dynaContent/common/productsSymbolMapping.jsp', headers=headers, params=parameter[1])
return (parameters[0], result)
if __name__ == '__main__':
with Pool() as pool:
result = pool.map(f, d_thread.items())
print(result) #this should show you the results as a list of (key, result)
print(dict(result)) #here you have a dict of your results
关于python - 如何使用字典作为输入进行多线程?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/54768801/