问题
我正在尝试从 Hive 表中读取,但收到以下错误:
[error] (run-main-0) org.apache.spark.sql.AnalysisException: Table or view not found: tags; line 1 pos 14
我已将 hive-site.xml
放在 $SPARK_HOME/conf
和 $HIVE_HOME/conf
中。同样,我可以毫不费力地使用 sqoop 从 mysql 中获取数据并将其导入到 hive 中。我的 Scala 代码有问题吗?或者这是一个配置错误?
Scala 代码:
package test1
import java.io.File
import org.apache.spark.sql.Row
import org.apache.spark.sql.SparkSession
case class Movie(movieid: String, title: String, genres: String)
case class Tag(userid: String, title: String, tag: String)
object SparkHiveTest {
def main(args: Array[String]) {
val warehouseLocation = new File("spark-warehouse").getAbsolutePath
val spark = SparkSession
.builder()
.master("local")
.appName("SparkHiveExample")
.config("spark.sql.warehouse.dir", warehouseLocation)
.enableHiveSupport()
.getOrCreate()
spark.sql("SELECT * FROM tags").show()
spark.stop()
}
}
hive 站点.xml:
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true</value>
<description>metadata is stored in a MySQL server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>MySQL JDBC driver class</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hiveuser</value>
<description>user name for connecting to mysql server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hivepass</value>
<description>password for connecting to mysql server</description>
</property>
</configuration>
最佳答案
确保您的 Hive Metastore 配置正确:
<configuration>
<property>
<name>hive.metastore.uris</name>
<value>HIVE METASTORE URI(S) HERE</value>
<description>URI for client to contact metastore server</description>
</property>
</configuration>
关于mysql - 使用 Spark 读取 Hive,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/43285668/