根据Running Apache Hive with Alluxio的教程,我将Alluxio
设置在Ceph
和Hive
之间的中间层,我试图从Alluxio提供服务于HDFS中存储的现有表,因为我现在使用external table
来访问数据。
这里的关键步骤是将表位置从HDFS和Ceph之类的分布式存储系统修改为alluxio:
4.2. Move an External Table from HDFS to Alluxio Assume there is an existing external table u_user in Hive with location set to hdfs://namenode_hostname:port/ml-100k. You can use the following HiveQL statement to check its “Location” attribute:
hive> desc formatted u_user;
Then use the following HiveQL statement to change the table data location from HDFS to Alluxio:hive> alter table u_user set location "alluxio://master_hostname:port/ml-100k";
我使用的语句是:
alter table call_center set location "alluxio://alluxio_master:19998/tpcds_text_1000.db/call_center";
但是,我得到如下错误:
ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. null
通过在WARN日志级别中打开配置单元,我们获得了更多的异常详细信息:
WARN [ main] metastore.RetryingMetaStoreClient: MetaStoreClient lost connection. Attempting to reconnect (1 of 1) after 1s. alter_table_with_environmentContext
org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) ~[hive-exec-3.1.1.jar:3.1.1]
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) ~[hive-exec-3.1.1.jar:3.1.1]
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[hive-exec-3.1.1.jar:3.1.1]
当前的
call_center
表信息如下:# Detailed Table Information
Database: tpcds_text_1000
OwnerType: USER
Owner: root
Retention: 0
Location: s3a://tpcds/user/root/tpcds/1000/call_center
Table Type: EXTERNAL_TABLE
任何评论都欢迎,谢谢。
最佳答案
Running Apache Hive with Alluxio中的某些修改可能仅在重新启动metastore之后才生效。
我杀死了metastore并通过hive --service metastore
重新启动了它,然后成功修改了表位置。
对于可能遇到相同情况的任何人,请在此处保留此线程。
关于java - 无法从Hive更改表格位置,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/57642673/