我想编写一个UDF来从HBase获取一些东西,我用它来将 token 设置为hiveconf,但是我无法使用hiveconf连接到HBase,它会抛出NullPointException
我尝试了很多方法,例如:https://www.programcreek.com/java-api-examples/index.php?api=org.apache.hadoop.hbase.security.token.TokenUtil
但它仍然抛出 NullPointException
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.security.User;
import org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier;
import org.apache.hadoop.hbase.security.token.TokenUtil;
import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.ql.hooks.ExecuteWithHookContext;
import org.apache.hadoop.hive.ql.hooks.HookContext;
import org.apache.hadoop.security.UserGroupInformation;
import org.apache.hadoop.security.token.Token;
public class HbaseTokenFetcherHook implements ExecuteWithHookContext{
private static final Log LOG = LogFactory.getLog(HbaseTokenFetcherHook.class);
@Override
public void run(HookContext hookContext) throws Exception {
HiveConf hiveConf = hookContext.getConf();
/* If required */
hiveConf.set(“zookeeper.znode.parent”, "/hbase-secure");
try {
UserGroupInformation.setConfiguration(hiveConf);
Connection tokenConnection = ConnectionFactory.createConnection(hiveConf);
Token<AuthenticationTokenIdentifier> token = TokenUtil.obtainToken(tokenConnection, User.getCurrent());
String urlString = token.encodeToUrlString();
hiveConf.set(“HBASE_AUTH_TOKEN”, urlString);
} catch (IOException | InterruptedException e) {
LOG.error("Error while fetching token for hbase"
+ e.getMessage(), e);
}
}
}
它抛出异常:
Token<AuthenticationTokenIdentifier> token = TokenUtil.obtainToken(tokenConnection, User.getCurrent());
错误信息: 引起原因:java.lang.NullPointException 在 org.apache.hadoop.hbase.zookeeper.ZookeeperWatcher.getMetaReplicaNodes(ZookeeperWatcher.java:497) 在 org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:558)
最佳答案
尝试Propolis第三方Hive UDF,它具有各种HBase get函数来获取值或整个家族。有关如何构建和使用的说明,请参阅自述文件,了解有关特定 UDF 类型describe
和函数名称的更多信息。我在使用 Kerberos 的 Hadoop 集群上进行了测试,效果很好。
关于java - 如何使用 Kerberos 在 hive UDF 中获得 HBase 连接?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/56641135/