hadoop - hadoop中的Jobtracker无法运行

Question

我正在使用Cygwin启动namenode和jobtracker。
Namenode运行成功，但是当我尝试打开作业跟踪器时，失败。

我拿出了日志，PFB。
您能否帮助我通过日志了解错误的确切位置。

2017-05-23 11:19:05,862 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting JobTracker
STARTUP_MSG:   host = ADMINIB-QKB8E8D/9.78.115.117
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.2.1
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
STARTUP_MSG:   java = 1.7.0_80
************************************************************/
2017-05-23 11:19:05,980 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2017-05-23 11:19:05,993 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
2017-05-23 11:19:05,994 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2017-05-23 11:19:05,994 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics system started
2017-05-23 11:19:06,113 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source QueueMetrics,q=default registered.
2017-05-23 11:19:06,395 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
2017-05-23 11:19:06,396 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2017-05-23 11:19:06,397 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
2017-05-23 11:19:06,399 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
2017-05-23 11:19:06,400 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list
2017-05-23 11:19:06,401 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
2017-05-23 11:19:06,401 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
2017-05-23 11:19:06,412 INFO org.apache.hadoop.mapred.JobTracker: Starting jobtracker with owner as IBM
2017-05-23 11:19:06,441 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
2017-05-23 11:19:06,444 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort50001 registered.
2017-05-23 11:19:06,445 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort50001 registered.
2017-05-23 11:19:06,526 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2017-05-23 11:19:06,724 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2017-05-23 11:19:06,727 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50030
2017-05-23 11:19:06,728 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50030 webServer.getConnectors()[0].getLocalPort() returned 50030
2017-05-23 11:19:06,728 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50030
2017-05-23 11:19:06,728 INFO org.mortbay.log: jetty-6.1.26
2017-05-23 11:19:07,061 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50030
2017-05-23 11:19:07,064 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered.
2017-05-23 11:19:07,064 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source JobTrackerMetrics registered.
2017-05-23 11:19:07,074 INFO org.apache.hadoop.mapred.JobTracker: JobTracker up at: 50001
2017-05-23 11:19:07,074 INFO org.apache.hadoop.mapred.JobTracker: JobTracker webserver: 50030
2017-05-23 11:19:07,102 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2017-05-23 11:19:07,102 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50001: starting
2017-05-23 11:19:07,103 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 50001: starting
2017-05-23 11:19:07,103 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 50001: starting
2017-05-23 11:19:07,104 INFO org.apache.hadoop.mapred.JobTracker: Setting safe mode to true. Requested by : IBM
2017-05-23 11:19:07,190 INFO org.apache.hadoop.mapred.JobTracker: Setting safe mode to false. Requested by : IBM
2017-05-23 11:19:07,392 INFO org.apache.hadoop.mapred.JobTracker: Cleaning up the system directory
2017-05-23 11:19:07,461 INFO org.apache.hadoop.mapred.JobHistory: Creating DONE folder at file:/C:/cygwin64/home/hadoop-1.2.1/logs/history/done
2017-05-23 11:19:07,464 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-05-23 11:19:07,465 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:IBM cause:java.io.IOException: Failed to set permissions of path: C:\cygwin64\home\hadoop-1.2.1\logs\history\done to 0755
2017-05-23 11:19:07,479 FATAL org.apache.hadoop.mapred.JobTracker: java.io.IOException: Failed to set permissions of path: C:\cygwin64\home\hadoop-1.2.1\logs\history\done to 0755
    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:691)
    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:672)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514)
    at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349)
    at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:193)
    at org.apache.hadoop.mapred.JobHistory.initDone(JobHistory.java:564)
    at org.apache.hadoop.mapred.JobHistory.initDone(JobHistory.java:540)
    at org.apache.hadoop.mapred.JobTracker$4.run(JobTracker.java:1823)
    at org.apache.hadoop.mapred.JobTracker$4.run(JobTracker.java:1821)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.JobTracker.initialize(JobTracker.java:1820)
    at org.apache.hadoop.mapred.JobTracker.offerService(JobTracker.java:2147)
    at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4711)

2017-05-23 11:19:07,480 INFO org.apache.hadoop.mapred.JobTracker: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down JobTracker at ADMINIB-QKB8E8D/9.78.115.117
************************************************************/

Answer 1

它已失败，因为hadoop尝试写入的日志目录没有权限。只需更改文件夹的文件权限(即提供写访问权限)，它就可以工作。