r - 错误 : 'NA' does not exist in current working directory (Webscraping)

标签 r web-scraping na

我正在尝试从以下网址抓取数据:
https://university.careers360.com/colleges/list-of-degree-colleges-in-India
我想点击每所大学的名称并获取每所大学的特定数据。

首先我所做的是在一个向量中收集所有大学网址:

#loading the package:
library(xml2)
library(rvest)
library(stringr)
library(dplyr)

#Specifying the url for desired website to be scrapped
baseurl <- "https://university.careers360.com/colleges/list-of-degree-colleges-in-India"

#Reading the html content from Amazon
basewebpage <- read_html(baseurl)

#Extracting college name and its url
scraplinks <- function(url){
   #Create an html document from the url
   webpage <- xml2::read_html(url)
   #Extract the URLs
   url_ <- webpage %>%
   rvest::html_nodes(".title a") %>%
   rvest::html_attr("href")  
   #Extract the link text
   link_ <- webpage %>%
   rvest::html_nodes(".title a") %>%
   rvest::html_text()
   return(data_frame(link = link_, url = url_))
}

#College names and Urls
allcollegeurls<-scraplinks(baseurl)

现在工作正常,但是当我为每个 url 使用 read_html 时,它显示错误。
#Reading the each url
for (i in allcollegeurls$url) {
  clgwebpage <- read_html(allcollegeurls$url[i])
}

Error: 'NA' does not exist in current working directory ('C:/Users/User/Documents').



我什至使用了“break”命令,但仍然出现相同的错误-:
#Reading the each url
for (i in allcollegeurls$url) {
  clgwebpage <- read_html(allcollegeurls$url[i])
  if(is.na(allcollegeurls$url[i]))break
}

请帮忙。

根据要求发布所有大学网址的 str-:
> str(allcollegeurls)
Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   30 obs. of  2 variables:
 $ link: chr  "Netaji Subhas Institute of Technology, Delhi" "Hansraj 
College, Delhi" "School of Business, University of Petroleum and Energy 
Studies, D.." "Hindu College, Delhi" ...
 $ url : chr  "https://www.careers360.com/university/netaji-subhas- 
 university-of-technology-new-delhi" 
"https://www.careers360.com/colleges/hansraj-college-delhi" 
"https://www.careers360.com/colleges/school-of-business-university-of- 
 petroleum-and-energy-studies-dehradun" 
"https://www.careers360.com/colleges/hindu-college-delhi" ...

最佳答案

这项工作,

purrr::map(allcollegeurls$url, read_html)

map 函数:map 函数通过对每个元素应用一个函数并返回一个与输入长度相同的向量来转换它们的输入。我喜欢避免for在 R 中使用。

关于r - 错误 : 'NA' does not exist in current working directory (Webscraping),我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/54513282/

相关文章:

r - 加快 write.table 的性能

r - 用 R 中的线条按组绘制多个数据集

r - GA包的ga功能实现

javascript - 抓取谷歌词典

r - 根据值填充 NA

r - data.table-按除一列以外的所有元素分组

python - 如何获取每个标签的数据?

html - 使用 BeautifulSoup 或 XPATH 获取内容属性值对

r - 检测向量是否至少有 1 个 NA 的最快方法?

r - 按组计算总缺失值?