我在 html 中有以下代码
<section>
<section>
<h2>Title1</h2>
<p>Text1</p>
<p>Text1</p>
</section>
<section>
<h2>Title2</h2>
<p>Text2</p>
<p>Text2</p>
</section>
<section>
<h2>Title3</h2>
<p>Text3</p>
<p>Text3</p>
</section>
</section>
<section>
<h2>Title2-1</h2>
<p>Text2-1</p>
<p>Text2-1</p>
</section>
<section>
<h2>Title3-1</h2>
<p>Text3-1</p>
<p>Text3-1</p>
</section>
class RUSpider(BaseSpider):
name = "ru"
allowed_domains = ["http://127.0.0.1:8000/"]
start_urls = [
"http://127.0.0.1:8000/week2/1_am/#/",
"http://127.0.0.1:8000/week1/1/",
"http://127.0.0.1:8000/week3/1_am/"
]
rules = [
Rule(SgmlLinkExtractor(), follow=True)
]
def parse(self, response):
filename = response.url.split("/")[3]
hxs = HtmlXPathSelector(response)
divs = hxs.select('//div')
sections = divs.select('//section').extract()
# print sections.extract
#class definition for scrapy and html selector
for each in sections: #iterate over loop [above sections]
soup = BeautifulSoup(each)
sp= soup.prettify()
elements = soup.findAll("section".split())
print len(elements),'sublength'
if len(elements ) > 1:
for element in elements:
for subelement in element:
print subelement,'element'
else:
item = RItem() # create Index Item
item['html_content'] = each
print each
yield item
尽管某些没有小节的部分被分解为单独的元素,但某些结果的格式正确。
我想要单独的每个部分。我的意思是因为 1 个部分还有其他部分。我想循环这些部分并单独获取它们,以便我可以跟踪循环。由于某些部分没有子部分,因此无需循环遍历它们。
在 BeautifulSoup 中有更好的方法吗? 我想要以下输出
<section>
<h2>Title1</h2>
<p>Text1</p>
<p>Text1</p>
</section>
<section>
<h2>Title2</h2>
<p>Text2</p>
<p>Text2</p>
</section>
<section>
<h2>Title3</h2>
<p>Text3</p>
<p>Text3</p>
</section>
<section>
<h2>Title2-1</h2>
<p>Text2-1</p>
<p>Text2-1</p>
</section>
<section>
<h2>Title3-1</h2>
<p>Text3-1</p>
<p>Text3-1</p>
</section>
最佳答案
检查此方法。这是您提供的数据的通用数据。
data = """
<section>
<section>
<h2>Title1</h2>
<p>Text1</p>
<p>Text1</p>
</section>
<section>
<h2>Title2</h2>
<p>Text2</p>
<p>Text2</p>
</section>
<section>
<h2>Title3</h2>
<p>Text3</p>
<p>Text3</p>
</section>
</section>
<section>
<h2>Title2-1</h2>
<p>Text2-1</p>
<p>Text2-1</p>
</section>
<section>
<h2>Title3-1</h2>
<p>Text3-1</p>
<p>Text3-1</p>
</section>
"""
from bs4 import BeautifulSoup
soup = BeautifulSoup(data)
sections = soup.find_all('section')
for each in sections: #iterate over loop [above sections]
if each.find('section'):
continue
else:
print each.prettify()
关于python - Beautiful Soup 迭代 html 标签,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/26624894/