python - 如何从 HTML 文件中的某一行开始抓取数据

Question

我正在尝试从 HTML 文件中抓取数据。它看起来像这样:

from bs4 import BeautifulSoup as bs
import urllib
redditPage1 = "http://redditlist.com/sfw"
r=urllib.urlopen(redditPage1).read()
soup = bs(r)

现在我想按照订阅者数量的顺序将 Reddit 版主(或 Reddit 子版主)放入列表中。为此，我只需要查看这行代码之后的数据:

<h3 class="listing-header">Subscribers</h3>

此行之前的所有内容都是无关紧要的，此行之后有关 subredditor 的所有条目如下所示:

<div class="listing-item" data-target-filter="sfw" data-target-subreddit="funny">
<div class="offset-anchor" id="funny-subscribers"></div>
<span class="rank-value">1</span>
<span class="subreddit-info-panel-toggle sfw"> <div>i</div> </span>
<span class="subreddit-url">
<a class="sfw" href="http://reddit.com/r/funny" target="_blank">funny</a>
</span>
<span class="listing-stat">18,197,786</span>
</div>

我应该怎么做才能提取该行之后而不是之前的 subredditor 名称？

Answer 1

尝试查找 <h3 class="listing-header">Subscribers</h3> ，然后获取父项 div ，范围将限制为Subscribers分区然后找到所有class为listing-item的div ，循环它们以获取内部元素 <a> 的文本(名称) :

from bs4 import BeautifulSoup as bs
import urllib
redditPage1 = "http://redditlist.com/sfw"
r=urllib.urlopen(redditPage1).read()
soup = bs(r,'lxml')
for sub_div in soup.find("h3", text="Subscribers").parent.find_all('div',{ "class" : "listing-item" }):
    print(sub_div.find('a').getText())