使用 aiohttp 的 Python lib 美丽的汤

Question

有人知道怎么做:

import html5lib
import urllib
from bs4 import BeautifulSoup

soup = BeautifulSoup(urllib.request.urlopen('http://someWebSite.com').read().decode('utf-8'), 'html5lib')

使用 aiohttp 代替 urllib ?

谢谢^^

Answer 1

你可以这样做:

import asyncio
import aiohttp
import html5lib
from bs4 import BeautifulSoup

SELECTED_URL = 'http://someWebSite.com'

async def get_site_content():
    async with aiohttp.ClientSession() as session:
        async with session.get(SELECTED_URL) as resp:
            text = await resp.read()

    return BeautifulSoup(text.decode('utf-8'), 'html5lib')

loop = asyncio.get_event_loop()
sites_soup = loop.run_until_complete(get_site_content())
print(sites_soup)
loop.close()