我正在尝试拉取 https://www.askebsa.dol.gov/epds/default.asp 上的内容使用 rvest 或 RSelenium 但当 javascript 页面以搜索框开头时没有找到指导？将所有这些内容放入一个简单的 CSV 文件中会很棒。
之后，从个人文件中提取数据，例如 https://www.askebsa.dol.gov/mewaview/View/Index/6219似乎有可能..但我也很感激一个干净的建议来做到这一点。谢谢
最佳答案
对于问题的第一部分，这种使用 rvest 的方法应该可行。我在最后一步收到错误消息，找不到所需的名称标签。
这是我的方法-
# open a html-session
web_session <- html_session("https://www.askebsa.dol.gov/epds/default.asp")
# get the form
test_search <- html_form(read_html("https://www.askebsa.dol.gov/epds/default.asp"))[[2]]
# set the required values for fields such as company_name, ein_number etc
# pass that info and submit the form - here i am getting an error
# it cannot recognize the 'search button' name
# if that is resolved it should work
set_values(test_search, 'm1company' = "Bend", 'm1ein' = '81-6268978' ) %>%
submit_form(web_session, ., submit = "cmdSubmitM1") %>%
read_html(.) -> some_html
如果我有时间，我会尝试做更多的研究并回复您。我发现了一些关于类似主题的教程和 SO 问题 here和 here .它们有点旧但仍然有用。
第二部分比较容易，因为您不涉及任何动态元素。我能够使用“selector-gadget”检索表单中的所有地址，并将所有节点名称复制粘贴到 html_nodes() 函数中。
# read the file and save it into a nested list
test_file_with_address <- read_html("https://www.askebsa.dol.gov/mewaview/View/Index/6219")
# copy paste all the css node names and get the text from the html file
test_file_with_address %>%
html_nodes(".border-top:nth-child(19) code , .border-top:nth-child(18) code , .border-top:nth-child(14) code , .border-top:nth-child(13) code , .border-top:nth-child(12) code , .border-top:nth-child(11) code , .border-top:nth-child(9) code , .section-header+ .border-top code
") %>% html_text()
[1] "\r\n Bend Chamber of Commerce Benefit Plan and Trust for Wood Products Employers\r\n 777 N.W. Wall Street, Suite 200\r\n Bend, OR 97703\r\n \r\n "
[2] "(541) 382-3221"
[3] "81-6268978"
[4] "501"
[5] "\r\n Bend Chamber of Commerce\r\n 777 N.W. Wall Street, Suite 200\r\n Bend, OR 97703\r\n \r\n "
[6] "(541) 382-3221"
[7] "93-0331932"
[8] "\r\n Katy Brooks\r\n Bend Chamber of Commerce\r\n 777 N.W. Wall Street, Suite 200\r\n Bend, OR 97703\r\n \r\n "
[9] "(541) 382-3221"
[10] "katy@bendchamber.org"
[11] "\r\n Deb Oster\r\n Scott Logging/Scott Transportation\r\n 400 S.W. Bluff Drive, #101\r\n Bend, OR 97702\r\n \r\n "
[12] "(541) 536-3636"
[13] "debo@scotttransport.com"
[14] "\r\n Karen Gibbons\r\n Allen & Gibbons Logging\r\n P.O. Box 754\r\n Canyonville, OR 97417\r\n \r\n "
[15] "(541) 839-4294"
[16] "agibbonslog@frontiernet.net"
[17] "\r\n Cascade East Benefits\r\n dba Johnson Benefit Planning\r\n 777 N.W. Wall Street, Suite 100\r\n Bend, OR 97703\r\n \r\n "
[18] "(541) 382-3571"
[19] "del@johnsonbenefitplanning.com"
[20] "93-1130374"
[21] "\r\n PacificSource Health Plans\r\n P.O. Box 7068\r\n Springfield, OR 97475-0068\r\n \r\n "
[22] "(541) 686-1242"
[23] "george.sherwood@pacifcsource.com"
[24] "93-0245545"
[25] "\r\n PacificSource Health Plans\r\n P.O. Box 7068\r\n Springfield, OR 97475-0068\r\n \r\n "
[26] "(541) 686-1242"
[27] "george.sherwood@pacificsource.com"
[28] "93-0245545"
[29] "N/A"
这需要更多的 regex 魔法来清理并将它们放入 data.frame 中，但可以看到基本的构建 block 。
关于javascript - 使用 R 在搜索后面抓取 asp javascript 分页表，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/51794400/