hadoop - HBase 0.98.5 无法在 Windows 上启动

Question

我已经在 Windows 7 PC 上下载并安装了 HBase 0.98.5。我按照 Apache 的入门步骤修改了 base-site.xml 如下(配置元素为空 OOB):

<configuration>
    <property>
    <name>hbase.rootdir</name>
    <value>file:///c:/datastore/hbase</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>c:/datastore/zookeper</value>
  </property>
</configuration>

启动 HBase 时出现以下错误:

2014-08-13 14:37:26,827 DEBUG [main-EventThread] master.ActiveMasterManager: A master is now available
2014-08-13 14:37:26,828 WARN  [M:0;rzm01:57477] hbase.ZNodeClearer: Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!)
2014-08-13 14:37:26,829 INFO  [M:0;rzm01:57477] master.ActiveMasterManager: Registered Active Master=rzm01.self,57477,1407929845469
2014-08-13 14:37:26,836 INFO  [M:0;rzm01:57477] Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS
2014-08-13 14:37:26,909 FATAL [M:0;rzm01:57477] master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010)
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:404)
        at org.apache.hadoop.util.Shell.run(Shell.java:379)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
        at org.apache.hadoop.util.Shell.execCommand(Shell.java:678)
        at org.apache.hadoop.util.Shell.execCommand(Shell.java:661)
        at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:639)
        at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:468)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:424)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:905)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:886)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:783)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:772)
        at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:651)
        at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:629)
        at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:587)
        at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:461)
        at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileS
ystem.java:152)
        at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:128)
        at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:790)
        at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:603)
        at org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.run(HMasterCommandLine.java:263)
        at java.lang.Thread.run(Thread.java:744)
2014-08-13 14:37:26,912 INFO  [M:0;rzm01:57477] master.HMaster: Aborting
2014-08-13 14:37:26,912 DEBUG [M:0;rzm01:57477] master.HMaster: Stopping service threads
2014-08-13 14:37:26,913 INFO  [M:0;rzm01:57477] ipc.RpcServer: Stopping server on 57477
2014-08-13 14:37:26,914 INFO  [M:0;rzm01:57477] master.HMaster: Stopping infoServer
2014-08-13 14:37:26,918 INFO  [RpcServer.listener,port=57477] ipc.RpcServer: RpcServer.listener,port=57477: stopping
2014-08-13 14:37:26,952 INFO  [main] regionserver.ShutdownHook: Installed shutdown hook thread: Shutdownhook:RS:0;rzm01:57505

通过查看抛出异常的 ProcessBuilder#start() 方法，似乎缺少一些参数:

public Process start() throws IOException {
        // Must convert to array first -- a malicious user-supplied
        // list might try to circumvent the security check.
        String[] cmdarray = command.toArray(new String[command.size()]);
        cmdarray = cmdarray.clone();

        for (String arg : cmdarray)
            if (arg == null)
                throw new NullPointerException(); // this is line 1010
        // Throws IndexOutOfBoundsException if command is empty
        String prog = cmdarray[0];

知道这里可能有什么问题吗？

Answer 1

问题是 Windows 的 HBase 发行版缺少一些 Hadoop 文件。 您可以从以下博客下载这些文件:http://www.srccodes.com/p/article/39/error-util-shell-failed-locate-winutils-binary-hadoop-binary-path (感谢 Abhijit Ghosh 为 Windows 构建这些文件并共享它们 - 否则您将不得不自己构建它们)。

所以按照以下步骤来解决问题:

1. 下载或构建适用于 Windows 的 Hadoop 二进制文件和库；
2. 将这些文件(例如 winutils.exe、winutils.dll 等)复制到 %HBASE_HOME%\bin

3. 编辑 %HBASE_HOME%\conf\hbase-env.cmd:找到行:

   set HBASE_OPTS="-XX:+UseConcMarkSweepGC" "-Djava.net.preferIPv4Stack=true"
   

   并在末尾添加以下选项:"-Dhadoop.home.dir=c:\hbase-0.98.5-hadoop2" 其中 c:\hbase-0.98.5-hadoop2 是你的 HBase Home 目录，这样它就会变成如下:

   set HBASE_OPTS="-XX:+UseConcMarkSweepGC" "-Djava.net.preferIPv4Stack=true" "-Dhadoop.home.dir=c:\hbase-0.98.5-hadoop2"`
   

祝你好运