python - 使用 BeauitifulSoup 进行网页抓取错误 : [Errno 10061]

Question

尝试使这段代码正常工作:(使用 BeautifulSoup 的网络抓取示例)

import urllib2    
wiki = "https://en.wikipedia.org/wiki/List_of_state_and_union_territory_capitals_in_India"
page = urllib2.urlopen(wiki)
from bs4 import BeautifulSoup
soup = BeautifulSoup(page)

我收到此错误:-

URLError: <urlopen error [Errno 10061] No connection could be made because the target machine actively refused it>

我想这与一些防火墙/安全相关的问题有关，有人可以帮助解决应该做什么吗？

Answer 1

您可以尝试使用 requests 这样的方法:

import requests
from bs4 import BeautifulSoup 

wiki = "https://en.wikipedia.org/wiki/List_of_state_and_union_territory_capitals_in_India"
page = requests.get(wiki).content
soup = BeautifulSoup(page)

<小时/>

如果你想获取表格，你可以像这样使用 pandas:

import pandas as pd

wiki = "https://en.wikipedia.org/wiki/List_of_state_and_union_territory_capitals_in_India"
df = pd.read_html(wiki)[1]
df2 = df.copy()
df2.columns = df.iloc[0]
df2.drop(0, inplace=True)
df2.drop('No.', axis=1, inplace=True)
df2.head()

输出: