python - 提取 CSS 类中的文本

Question

尝试将网页中的数据提取到表中。例如

Block Number XXX
Building Name YYY
Street Name zzz
Pin Code 123456789

我正在尝试使用此代码以表格形式获取公司的所有详细信息...

html_doc='https://s3.amazonaws.com/todel162/test.html'

from urllib.request import urlopen
from bs4 import BeautifulSoup

soup = BeautifulSoup(urlopen(html_doc), 'html.parser')

mydivs = soup.findAll("div", {"class": "row"})

for i in mydivs:
    for x in i.findAll("div", {"class": "col-md-3 col-sm-3"}):
        print (x)
        print ('new x')

但我不知道如何提取特定 CSS 类的 HTML 标记中的文本。

Answer 1

您需要的所有元素都在 <div class="col-md-3 col-sm-3"> 内标签。并且，除一般信息之外的所有项目均遵循特定格式:first div tag是标签，第二个是对应的文本。因此，您可以简单地找到它们，然后将它们交替打印为标签及其文本。

因此，除了一般信息之外的所有内容都可以在 <div id="fldFirm"> 中找到。标签。由于一般信息不遵循上述格式，因此您必须先单独抓取它。您可以使用 limit=2 获取前两个标签.

import requests
from bs4 import BeautifulSoup

r = requests.get('https://s3.amazonaws.com/todel162/test.html')
soup = BeautifulSoup(r.text, 'lxml')

gen_info = [x.text.strip() for x in soup.find('div', id='divInfoType').find_all('div', class_='col-md-3 col-sm-3', limit=2)]
print('{}: {}'.format(gen_info[0], gen_info[1]))

all_info = [x.text.strip() for x in soup.find('div', id='fldFirm').find_all('div', class_='col-md-3 col-sm-3')]

for i in range(0, len(all_info), 2):
    print('{}: {}'.format(all_info[i], all_info[i+1]))

输出:

Information Type: Other Than Individual
Name: Bellissimo Developers Thane Private Limited (Lodha Developers Pvt Ltd)
Organization Type: Company
Description For Other Type Organization: NA
Do you have any Past Experience ?: No
Block Number: 412, Floor 4 17G
Building Name: Vardhaman Chamber
Street Name: Cawasji Patel Road
Locality: Fort
Land mark: Horniman Circle
State/UT: MAHARASHTRA
Division: Konkan
District: Mumbai City
Taluka: Mumbai City
Village: Mumbai City
Pin Code: 400001
Office Number: 02261334263
Website URL: www.lodhagroup.com

编辑:

要创建数据框，您可以使用:

gen_info = [x.text.strip() for x in soup.find('div', id='divInfoType').find_all('div', class_='col-md-3 col-sm-3', limit=2)]
all_info = gen_info + [x.text.strip() for x in soup.find('div', id='fldFirm').find_all('div', class_='col-md-3 col-sm-3')]
df = pd.DataFrame({'A': all_info[0::2], 'B': all_info[1::2]})