python - 提供 scrapy xpath 的备用路径

Question

我刚刚开始使用Scrapy，并试图拉低主队、客队并从http://www.bbc.com/sport/football/results/得分作为一种学习方式。

一切正常，除了我用来获取团队的 xpath 依赖于“a”标签:

match.xpath('.//*[@class="team-home teams"]/a/text()').extract_first()

某些团队没有链接，因此查询偶尔会返回 None。

以下 xpath 删除/a/并选取未链接的团队，但也包含许多换行符字符串:

match.xpath('.//*[@class="team-home teams"]/text()').extract_first()

如果没有返回，如何修改代码以提供替代 xpath？或者是否有一个更智能的 xpath 无论是否存在“a”标签都会返回正确的结果？

import scrapy


class FootballresultsSpider(scrapy.Spider):
    name = "footballResults"
    start_urls = ['http://www.bbc.com/sport/football/results/']

    def parse(self, response):

        for match in response.xpath('//td[@class="match-details"]'):
            yield {
                'home_team': match.xpath('.//*[@class="team-home teams"]/a/text()').extract_first(),
                'score': match.xpath('.//span[@class="score"]/abbr/text()').extract_first(),
                'away_team': match.xpath('.//*[@class="team-away teams"]/a/text()').extract_first(),
            }

* 编辑*

下面是尝试使用“|”的代码在 xpath 之间，但对于任何没有 anchor 标记的条目仍返回 None。为了简洁起见，我仅使用单个条目 home 进行演示。

import scrapy

class ResultsSpider(scrapy.Spider):
    name = "results"
    #allowed_domains = ["www.bbc.com"]
    start_urls = ['http://www.bbc.com/sport/football/results/']

    def parse(self, response):

        match_details = response.xpath('//td[@class="match-details"]')

        for match in match_details:

            a_xpath = './/span[@class="team-home teams"]/a/text()'
            text_xpath = './/span[@class="team-home teams"]/a/text()'


            home = match.xpath(a_xpath + ' | ' + text_xpath).extract_first()

            yield {
                'Home': home
            }

下面是有效的代码，虽然有点啰嗦，但我确信有一种更简洁的方法来实现它。

import scrapy


class ResultsSpider(scrapy.Spider):
    name = "results"
    #allowed_domains = ["www.bbc.com"]
    start_urls = ['http://www.bbc.com/sport/football/results/']

    def parse(self, response):

        match_details = response.xpath('//td[@class="match-details"]')

        for match in match_details:

            if match.xpath('.//span[@class="team-home teams"]/a/text()').extract_first() == None:
                home = match.xpath('.//span[@class="team-home teams"]/text()').extract_first().strip()
            else:
                home = match.xpath('.//span[@class="team-home teams"]/a/text()').extract_first()

            yield {
                'Home': home,
            }

Answer 1

您可以使用 | xpath 中的 运算符:

first_xpath = './/*[@class="team-home teams"]/a/text()'
second_xpath = ... # The alternative xpath
match.xpath(first_xpath + ' | ' + second_xpath).extract_first()