r - 使用 rvest 抓取带有跨度的 html 表

Question

我正在使用 rvest 提取以下页面中的表格:

https://en.wikipedia.org/wiki/List_of_United_States_presidential_elections_by_popular_vote_margin

以下代码有效:

URL <- 'https://en.wikipedia.org/wiki/List_of_United_States_presidential_elections_by_popular_vote_margin'
table <- URL %>%  
  read_html %>% 
  html_nodes("table")  %>% 
  .[[2]] %>% 
  html_table(trim=TRUE)

但是边距和总裁姓名列有一些奇怪的值。原因是源代码有以下几点:

<td><span style="display:none">00.001</span>−10.44%</td>

所以我得到的不是 -10.44%，而是 00.001∼10.44%

我怎么能解决这个问题？

Answer 1

一种选择是单独定位和替换问题列。

可以使用 xpath 定位边距列

# get the html
html <- URL %>%  
  read_html()

# Example using the first margin column (column # 6)
html %>%
  html_nodes(xpath = '//table[2]') %>%       # get table 2
  html_nodes(xpath = '//td[6]/text()') %>%   # get column 6 using text()
  iconv("UTF-8", "UTF-8")                    # to convert "−" to "-"
# [1] "−10.44%" "−3.00%"  "−0.83%"  "−0.51%"  "0.09%"   "0.17%"   "0.57%"  
# [8] "0.70%"   "1.45%"   "2.06%"   "2.46%"   "3.01%"   "3.12%"   "3.86%"  
#[15] "4.31%"   "4.48%"   "4.79%"   "5.32%"   "5.56%"   "6.05%"   "6.12%"  
#[22] "6.95%"   "7.27%"   "7.50%"   "7.72%"   "8.51%"   "8.53%"   "9.74%"  
#[29] "9.96%"   "10.08%"  "10.13%"  "10.85%"  "11.80%"  "12.20%"  "12.25%" 
#[36] "14.20%"  "14.44%"  "15.40%"  "17.41%"  "17.76%"  "17.81%"  "18.21%" 
#[43] "18.83%"  "22.58%"  "23.15%"  "24.26%"  "25.22%"  "26.17%"

对另一个边距列执行相同操作。我用过 iconv转换 −至 - ，因为这是一个编码问题，但您可以改用基于替换的解决方案(例如使用 sub )。

要以总统姓名为目标列，您可以再次使用 xpath:

html %>%
  html_nodes(xpath = '//table[2]') %>% 
  html_nodes(xpath = '//td[3]/a/text()') %>%
  html_text()
# [1] "John Quincy Adams"      "Rutherford Hayes"       "Benjamin Harrison"     
# [4] "George W. Bush"         "James Garfield"         "John Kennedy"          
# [7] "Grover Cleveland"       "Richard Nixon"          "James Polk"            
#[10] "Jimmy Carter"           "George W. Bush"         "Grover Cleveland"      
#[13] "Woodrow Wilson"         "Barack Obama"           "William McKinley"      
#[16] "Harry Truman"           "Zachary Taylor"         "Ulysses Grant"         
#[19] "Bill Clinton"           "William Henry Harrison" "William McKinley"      
#[22] "Franklin Pierce"        "Barack Obama"           "Franklin Roosevelt"    
#[25] "George H. W. Bush"      "Bill Clinton"           "William Taft"          
#[28] "Ronald Reagan"          "Franklin Roosevelt"     "Abraham Lincoln"       
#[31] "Abraham Lincoln"        "Dwight Eisenhower"      "Ulysses Grant"         
#[34] "James Buchanan"         "Andrew Jackson"         "Martin Van Buren"      
#[37] "Woodrow Wilson"         "Dwight Eisenhower"      "Herbert Hoover"        
#[40] "Franklin Roosevelt"     "Andrew Jackson"         "Ronald Reagan"         
#[43] "Theodore Roosevelt"     "Lyndon Johnson"         "Richard Nixon"         
#[46] "Franklin Roosevelt"     "Calvin Coolidge"        "Warren Harding"