我正在尝试运行基本的搜寻器。从NutchTutorial获得命令:bin / crawl urls -dir crawl -depth 3 -topN 5
(在完成所有预设之后)
我从Windows运行,所以我已经将cygwin64安装为运行环境
从nutch主目录运行bin / nutch时,我没有看到任何问题,但是当我尝试如上所述运行爬网时,出现以下错误:
Injector: starting at 2014-11-29 11:31:35
Injector: crawlDb: -dir/crawldb
Injector: urlDir: urls
Injector: Converting injected urls to crawl db entries.
Injector: java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Er
an\mapred\staging\Eran996102549\.staging to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:691)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:664)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSys
tem.java:514)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.jav
a:349)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:19
3)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmi
ssionFiles.java:126)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
tion.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:9
36)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1353)
at org.apache.nutch.crawl.Injector.inject(Injector.java:324)
at org.apache.nutch.crawl.Injector.run(Injector.java:380)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Injector.main(Injector.java:370)
在本教程中没有提及该错误。
我该怎么办 ?
最佳答案
这是一个权限问题。您应该为文件夹设置“读取,写入和执行”权限(Hadoop配置文件core-site.xml中的<name>hadoop.tmp.dir</name>
值)。
希望有帮助
Le Quoc Do
关于java - Nutch Crawler错误:权限被拒绝,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/27200873/