首先感谢您抽出时间回答下一个问题:)
我正在尝试将 Databricks Scala 应用程序与 Azure 表存储连接，但出现以下错误:
Azure 表 Scala APP
Error:
NoSuchMethodError: reactor.netty.http.client.HttpClient.resolver(Lio/netty/resolver/AddressResolverGroup;)Lreactor/netty/transport/ClientTransport;
at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:94) at com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:18) at com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58) at com.azure.core.http.HttpClient.createDefault(HttpClient.java:50) at com.azure.core.http.HttpClient.createDefault(HttpClient.java:40) at com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62) at com.azure.data.tables.BuilderHelper.buildPipeline(BuilderHelper.java:122) at com.azure.data.tables.TableServiceClientBuilder.buildAsyncClient(TableServiceClientBuilder.java:161) at com.azure.data.tables.TableServiceClientBuilder.buildClient(TableServiceClientBuilder.java:93)
我附上代码:
val clientCredential: ClientSecretCredential = new ClientSecretCredentialBuilder()
.tenantId(tenantID)
.clientId(client_san_Id)
.clientSecret(client_san_Secret)
.build()
val tableService = new TableServiceClientBuilder()
.endpoint("https://<Resource-Table>.table.core.windows.net")
.credential(clientCredential)
.buildClient()
非常感谢您抽出时间!
最佳答案
首先，您需要在 Azure databricks 上安装存储。
然后使用下面的代码来挂载表存储。
dbutils.fs.mount(
source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/<directory-name>",
mountPoint = "/mnt/<mount-name>",
extraConfigs = Map("<conf-key>" -> dbutils.secrets.get(scope = "<scope-name>", key = "<key-name>")))
使用以下代码访问表存储:
// scala
val df = spark.read.text("/mnt/<mount-name>/...")
val df = spark.read.text("dbfs:/<mount-name>/...")
可以引用这个notebook
另请参阅此 article作者:高里·马哈詹
关于azure - 使用 Azure 表存储的 Databricks Scala 应用程序出现错误，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/71485239/