excel - VBA Selenium FindElementByXPath 找不到元素

Question

我编写了一个 VBA，它使用 selenium chrome web 驱动程序打开一个 web 链接来抓取数据，我遇到了几个问题，我需要你的建议。

代码示例和结果 1:
错误激活

    Sub test_supplements_store()
    Dim driver As New ChromeDriver
    Dim post As Object

    i = 1

    driver.Get "https://www.thesupplementstore.co.uk/brands/optimum_nutrition?page=4"
On Error Resume Next
    For Each post In driver.FindElementsByClass("desc")
        Cells(i, 1) = post.FindElementByTag("a").Attribute("title")
        Cells(i, 2) = Trim(Split(post.FindElementByClass("size").Text, ":")(1))
        Cells(i, 3) = post.FindElementByXPath(".//span[@class='now']//span[@class='pricetype-purchase-unit multi-price']//span[@class='blu-price blu-price-initialised']").Text
        Cells(i, 4) = post.FindElementByTag("a").Attribute("href")
        i = i + 1
    Next post
End Sub

代码示例和结果 2:错误停用

Sub test_supplements_store()
    Dim driver As New ChromeDriver
    Dim post As Object

    i = 1

    driver.Get "https://www.thesupplementstore.co.uk/brands/optimum_nutrition?page=4"
'On Error Resume Next
    For Each post In driver.FindElementsByClass("desc")
        Cells(i, 1) = post.FindElementByTag("a").Attribute("title")
        Cells(i, 2) = Trim(Split(post.FindElementByClass("size").Text, ":")(1))
        Cells(i, 3) = post.FindElementByXPath(".//span[@class='now']//span[@class='pricetype-purchase-unit multi-price']//span[@class='blu-price blu-price-initialised']").Text
        Cells(i, 4) = post.FindElementByTag("a").Attribute("href")
        i = i + 1
    Next post
End Sub

代码示例和结果 3:错误激活

Sub test_supplements_store()
    Dim driver As New ChromeDriver
    Dim post As Object

    i = 1

    driver.Get "https://www.thesupplementstore.co.uk/brands/optimum_nutrition"
On Error Resume Next
    For Each post In driver.FindElementsByClass("desc")
        Cells(i, 1) = post.FindElementByTag("a").Attribute("title")
        Cells(i, 2) = Trim(Split(post.FindElementByClass("size").Text, ":")(1))
        Cells(i, 3) = post.FindElementByXPath(".//span[@class='now']//span[@class='pricetype-purchase-unit multi-price']//span[@class='blu-price blu-price-initialised']").Text
        Cells(i, 4) = post.FindElementByTag("a").Attribute("href")
        i = i + 1
    Next post
End Sub

第一个示例返回网站上的所有 74 项商品，除了价格，但在很长一段时间内(大约两分钟)。

第二个示例仅将标题返回到工作表的第一个单元格并弹出错误。

第三个示例仅返回 21，但未返回那些没有 now 标签的商品的价格。脚本运行速度非常快，不到 10 秒。

请就如何退回所有 74 件商品以及标题、尺寸、价格、href 提供建议。

Answer 1

您正在处理的页面已应用了lay-loading 方法。这是因为不是一次加载所有项目；相反，它会在您向下滚动时加载其余部分。我在代码中使用了一个小的 javascript 函数，它解决了这个问题。我希望这是您正在寻找的结果。

Sub test_supplements_store()
    Dim driver As New ChromeDriver
    Dim post As Object

    driver.Get "https://www.thesupplementstore.co.uk/brands/optimum_nutrition"
    On Error Resume Next

    Do While EndofPage = False
        PrevPageHeight = CurrentPageHeight
        CurrentPageHeight = driver.ExecuteScript("window.scrollTo(0, document.body.scrollHeight);var CurrentPageHeight=document.body.scrollHeight;return CurrentPageHeight;")
        driver.Wait 3000
        If PrevPageHeight = CurrentPageHeight Then
            EndofPage = True
        End If
    Loop

    For Each post In driver.FindElementsByXPath("//li[contains(@class,'prod')]")
        i = i + 1: Cells(i, 1) = post.FindElementByXPath(".//a").Attribute("title")
        Cells(i, 2) = Split(post.FindElementByXPath(".//p[@class='size']").Text, ": ")(1)
        Cells(i, 3) = post.FindElementByXPath(".//p[@class='price']//span[@class='now']//span|.//p[@class='price']//span[@class='dynamictype-single']").Text
        Cells(i, 4) = post.FindElementByXPath(".//a").Attribute("href")
    Next post
End Sub