VBA 网页抓取 : Difference between internet explorer and XMLHTTP request

Question

我正在学习 VBA 中的网页抓取。我无法理解通过 Internet Explorer 和 XMLHTTP 请求发送的请求结果的差异。

这是网站--http://www.scstrade.com/stockscreening/SS_CompanySnapShotQF.aspx?symbol=LUCK

通过 IE 进行抓取会在文本文件中写入完美的结果，但 XMLHTTP 请求会在文本文件中写入空字符串。我不明白其中的原因。每个方法的代码如下:

Dim ie As SHDocVw.InternetExplorer
Dim ele As MSHTML.IHTMLElement
Dim page As MSHTML.HTMLDocument
Dim fso As Scripting.FileSystemObject
Dim st As Scripting.TextStream
   
Set ie = New SHDocVw.InternetExplorer
ie.Visible = True
ie.navigate "http://www.scstrade.com/stockscreening/SS_CompanySnapShotQF.aspx?symbol=DGKC"

Do While ie.Busy Or ie.readyState <> READYSTATE_COMPLETE
Loop

Set ele = ie.document.getElementById("listisq1")

Set fso = New Scripting.FileSystemObject
Set st = fso.CreateTextFile(ThisWorkbook.Path & "\myFile.txt")
st.Write ele.innerText

XMLHTTP请求的代码如下

Dim page As MSHTML.HTMLDocument
Dim req As New MSXML2.XMLHTTP60
Dim ele As MSHTML.IHTMLElement
Dim fso As Scripting.FileSystemObject
Dim st As Scripting.TextStream

req.Open "GET", "http://www.scstrade.com/stockscreening/SS_CompanySnapShotQF.aspx?symbol=LUCK", False
req.send

If req.Status <> 200 Then
    MsgBox "req failed"
    Exit Sub
End If

Set page = New MSHTML.HTMLDocument

page.body.innerHTML = req.responseText

Set ele = page.getElementById("listisq1")

Set fso = New Scripting.FileSystemObject
Set st = fso.CreateTextFile(ThisWorkbook.Path & "\XMLHTTPscrape.txt")
st.Write ele.innerText

这个方法(XMLHTTP)没有在文本文件中写入任何内容。如果代码有问题或者两种方法是否存在固有差异，请指导我。谢谢

Answer 1

以下内容应该为您提供 this portion 的内容在 json 响应中。

Sub GetContent()
    Const link = "http://www.scstrade.com/stockscreening/SS_CompanySnapShotQF.aspx/isq4"
    Dim payload As Variant

    payload = "{'sym':'LUCK','_search':false,'nd':1600525542589,'rows':30,'page':1,'sidx':'Year / Quarter','sord':'desc'}"
    
    With CreateObject("MSXML2.XMLHTTP")
        .Open "POST", link, False
        .setRequestHeader "Content-Type", "application/json"
        .send payload
        MsgBox .responseText
    End With
End Sub