python - 使用 Python Beautiful Soup 抓取数据 - 无法提取字段

Question

我正在尝试使用 Python Beautiful Soup 从 IG 索引页面中提取股票代码 (South Africa 40) 字段，但无法检索它。

我试图从中获取数据的网页是 https://www.ig.com/uk/ig-indices/south-africa-40?siteId=igm

带有股票代码数据的 HTML 代码:

<div class="ma-content title">
    <h1>South Africa 40</h1>

        <p>
            .........some text..........
        </p>

</div>

我已经尝试过这个:

name = soup.select('div.ma-content title h1')[0].text

但收到错误消息:

Traceback (most recent call last): File "IGIndexDataScrape_Minute_v0.1.py", line 30, in name = soup.select('div.ma-content title h1')[0].text IndexError: list index out of range

对上述内容的任何建议/代码更正都会非常有帮助。

这是直接复制和粘贴的完整代码:

import urllib2
from bs4 import BeautifulSoup

import csv
from datetime import datetime

from lxml import html
import requests

quote_page = ['https://www.ig.com/uk/ig-indices/south-africa-40?siteId=igm']

data = []
for pg in quote_page:
page = urllib2.urlopen(pg)

soup = BeautifulSoup(page, 'html.parser')

name = soup.select('div.ma-content title h1')[0].text

sell_price = soup.find('span', attrs={'class':'price', 'id':'bid'}).text
data.append(sell_price)

buy_price = soup.find('span', attrs={'class':'price', 'id':'ofr'}).text
data.append(buy_price)

print sell_price + "\t\t" + buy_price + name

#    data.append(name, sell_price, buy_price)
#    print name + "\t\t" + sell_price + "\t\t" + buy_price

Answer 1

您是否尝试过find_all而不是select？像这样的东西:

name_div = soup.find_all('div', {'class': 'ma-content title'})[0]
name = name_div.find('h1').text