mysql - 如何使用sparklyr spark_write_jdbc连接MySql

Question

spark_write_jdbc(members_df,
    name = "Mbrs",
    options = list(
      url = paste0("jdbc:mysql://",mysql_host,":",mysql_port,"/",dbname),
      user = mysql_user,
      password = mysql_password),
    mode = "append")

导致以下异常:

Error: java.sql.SQLException: No suitable driver
  at java.sql.DriverManager.getDriver(DriverManager.java:315)

.jar 文件位于运行 RStudio 的服务器上的一个文件夹中，配置详情如下。我们能够通过 RMySql 包访问 MySql，因此 MySql 可以正常工作并可以访问。

config$`spark.sparklyr.shell.driver-class-path` <- "/dev/shm/temp/mysql-connector-java-5.1.44-bin.jar"

Answer 1

即使问题不同，我认为答案仍然适用于此:

How to use a predicate while reading from JDBC connection?

通过 sparklyr 连接到 JDBC MySQL 的代码(我做了一些小改动以简化代码，由 Jake Russ 编写)

library(sparklyr)
library(dplyr)

config <- spark_config()
#config$`sparklyr.shell.driver-class-path` <- "E:\\spark232_hadoop27\\jars\\mysql-connector-java-5.1.47-bin.jar"
#in my case, using RStudio and Sparkly this seemed to be optional

sc <- spark_connect(master = "local")

db_tbl <- spark_read_jdbc(sc,
                  name    = "table_name",  
                  options = list(url      = "jdbc:mysql://localhost:3306/schema_name",
                                 user     = "root",
                                 password = "password",
                                 dbtable  = "table_name"))