python异步和httpx

Question

我对异步编程非常陌生，我正在玩 httpx。我有以下代码，我确定我做错了什么 - 只是不知道它是什么。有两种方法，一种是同步的，一种是异步的。他们都来自谷歌金融。在我的系统上，我看到的时间如下:

异步:5.015218734741211
同步:5.173618316650391

代码如下:

import httpx
import asyncio
import time



#
#--------------------------------------------------------------------
#
#--------------------------------------------------------------------
#
def sync_pull(url):
  r = httpx.get(url)
  print(r.status_code)


#
#--------------------------------------------------------------------
#
#--------------------------------------------------------------------
#
async def async_pull(url):
  async with httpx.AsyncClient() as client:
    r = await client.get(url)
    print(r.status_code)


#
#--------------------------------------------------------------------
#
#--------------------------------------------------------------------
#
if __name__ == "__main__":

  goog_fin_nyse_url = 'https://www.google.com/finance/quote/'
  tickers = ['F', 'TWTR', 'CVX', 'VZ', 'GME', 'GM', 'PG', 'AAL', 
             'MARK', 'AAP', 'THO', 'NGD', 'ZSAN', 'SEAC',
             ]  

  print("Running asynchronously...")
  async_start = time.time()
  for ticker in tickers:
    url = goog_fin_nyse_url + ticker + ':NYSE'
    asyncio.run(async_pull(url))
  async_end = time.time()
  print(f"Time lapsed is: {async_end - async_start}")


  print("Running synchronously...")
  sync_start = time.time()
  for ticker in tickers:
    url = goog_fin_nyse_url + ticker + ':NYSE'
    sync_pull(url)
  sync_end = time.time()
  print(f"Time lapsed is: {sync_end - sync_start}")

我曾希望异步方法所需的时间只是同步方法所需时间的一小部分。我做错了什么？

Answer 1

当您说 asyncio.run(async_pull) 时，您说的是运行 'async_pull' 并等待结果返回。由于您对循环中的每个代码都执行一次此操作，因此您实际上是在使用 asyncio 同步运行事物并且不会看到性能优势。

您需要做的是创建多个异步调用并同时运行它们。有几种方法可以做到这一点，最简单的方法是使用 asyncio.gather (参见 https://docs.python.org/3/library/asyncio-task.html#asyncio.gather )，它接收一系列协程并同时运行它们。调整你的代码相当简单，你创建一个异步函数来获取一个 url 列表，然后在每个 URL 上调用 async_pull ，然后将其传递给 asyncio.gather 和等待结果。使您的代码适应这种情况如下所示:

import httpx
import asyncio
import time

def sync_pull(url):
    r = httpx.get(url)
    print(r.status_code)

async def async_pull(url):
    async with httpx.AsyncClient() as client:
        r = await client.get(url)
        print(r.status_code)


async def async_pull_all(urls):
    return await asyncio.gather(*[async_pull(url) for url in urls])

if __name__ == "__main__":

    goog_fin_nyse_url = 'https://www.google.com/finance/quote/'
    tickers = ['F', 'TWTR', 'CVX', 'VZ', 'GME', 'GM', 'PG', 'AAL',
           'MARK', 'AAP', 'THO', 'NGD', 'ZSAN', 'SEAC',
           ]

    print("Running asynchronously...")
    async_start = time.time()
    results = asyncio.run(async_pull_all([goog_fin_nyse_url + ticker + ':NYSE' for ticker in tickers]))
    async_end = time.time()
    print(f"Time lapsed is: {async_end - async_start}")


    print("Running synchronously...")
    sync_start = time.time()
    for ticker in tickers:
        url = goog_fin_nyse_url + ticker + ':NYSE'
        sync_pull(url)
    sync_end = time.time()
    print(f"Time lapsed is: {sync_end - sync_start}")

以这种方式运行，异步版本对我来说大约需要一秒钟，而不是同步运行七秒钟。