这段代码从这里抓取 http://www.bls.gov/schedule/news_release/2015_sched.htm发布列下包含就业情况的每个日期。
pg <- read_html("http://www.bls.gov/schedule/news_release/2015_sched.htm")
# target only the <td> elements under the bodytext div
body <- html_nodes(pg, "div#bodytext")
# we use this new set of nodes and a relative XPath to get the initial <td> elements, then get their siblings
es_nodes <- html_nodes(body, xpath=".//td[contains(., 'Employment Situation for')]/../td[1]")
# clean up the cruft and make our dates!
nfpdates2015 <- as.Date(trimws(html_text(es_nodes)), format="%A, %B %d, %Y")
###thanks @hrbrmstr for this###
我想对包含其他年份的其他 URL 重复这一点,以相同的方式命名,只是年份数字发生变化。特别是对于以下 URL:
#From 2008 to 2015
http://www.bls.gov/schedule/news_release/2015_sched.htm
http://www.bls.gov/schedule/news_release/2014_sched.htm
...
http://www.bls.gov/schedule/news_release/2008_sched.htm
我对rvest
、HTML
和XML
几乎一无所知。我想用 for 循环应用相同的代码,但我的努力是徒劳的。当然,我可以将 2015 年的代码重复八次以获得所有年份,既不会花费太长也不会占用太多空间。然而,我非常想知道如何以更有效的方式做到这一点。谢谢。
最佳答案
在循环中,您可以使用 paste0
语句更改 url
字符串
for(i in 2008:2015){
url <- paste0("http://www.bls.gov/schedule/news_release/", i, "_sched.htm")
pg <- read_html(url)
## all your other code goes here.
}
或者使用 lapply
返回结果列表。
lst <- lapply(2008:2015, function(x){
url <- paste0("http://www.bls.gov/schedule/news_release/", x, "_sched.htm")
## all your other code goes here.
pg <- read_html(url)
# target only the <td> elements under the bodytext div
body <- html_nodes(pg, "div#bodytext")
# we use this new set of nodes and a relative XPath to get the initial <td> elements, then get their siblings
es_nodes <- html_nodes(body, xpath=".//td[contains(., 'Employment Situation for')]/../td[1]")
# clean up the cruft and make our dates!
nfpdates <- as.Date(trimws(html_text(es_nodes)), format="%A, %B %d, %Y")
return(nfpdates)
})
返回
lst
[[1]]
[1] "2008-01-04" "2008-02-01" "2008-03-07" "2008-04-04" "2008-05-02" "2008-06-06" "2008-07-03" "2008-08-01" "2008-09-05"
[10] "2008-10-03" "2008-11-07" "2008-12-05"
[[2]]
[1] "2009-01-09" "2009-02-06" "2009-03-06" "2009-04-03" "2009-05-08" "2009-06-05" "2009-07-02" "2009-08-07" "2009-09-04"
[10] "2009-10-02" "2009-11-06" "2009-12-04"
## etc...
关于html - 使用 for 循环或 lapply 将 Web 抓取到具有相似 URL 的 R 多个链接,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/36963583/