apache-drill - 如何在 apache drill 中以编程方式添加更多存储插件？

Question

我尝试了钻取 JDBC 驱动程序以编程方式进行查询。

代码的有用部分:

Connection conn = new Driver().connect("jdbc:drill:zk=local", getDefaultProperties());
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery("show databases");
        while (rs.next())
        {
            String SCHEMA_NAME = rs.getString("SCHEMA_NAME");
            System.out.println(SCHEMA_NAME);
        }

public static Properties getDefaultProperties()
  {
    final Properties properties = new Properties();
    properties.setProperty(ExecConstants.HTTP_ENABLE, "false");
    return properties;
  }

一切正常，直到我有了 cp 和 dfs 存储插件。以上查询的输出:

INFORMATION_SCHEMA
cp.default
dfs.default
dfs.root
dfs.tmp
sys

但是当我添加 mongo 作为带有配置的存储插件时:

{
  "type": "mongo",
  "connection": "mongodb://localhost:27017/",
  "enabled": false
}

我遇到以下异常:

java.sql.SQLException: Failure in starting embedded Drillbit: java.lang.RuntimeException: Unable to deserialize "/tmp/drill/sys.storage_plugins/mongo.sys.drill"
    at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:109)
    at org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:66)
    at org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
    at net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
    at org.apache.drill.jdbc.Driver.connect(Driver.java:78)
    at com.mkyong.App.main(App.java:37)
Caused by: java.lang.RuntimeException: Unable to deserialize "/tmp/drill/sys.storage_plugins/mongo.sys.drill"
    at org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:140)
    at org.apache.drill.exec.store.sys.local.FilePStore$Iter$DeferredEntry.getValue(FilePStore.java:219)
    at org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:168)
    at org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:132)
    at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:244)
    at org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:100)
    ... 5 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type id 'mongo' into a subtype of [simple type, class org.apache.drill.common.logical.StoragePluginConfig]
 at [Source: [B@21318883; line: 2, column: 3]
    at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
    at com.fasterxml.jackson.databind.DeserializationContext.unknownTypeException(DeserializationContext.java:849)
    at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:167)
    at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:99)
    at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:84)
    at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:132)
    at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:41)
    at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1269)
    at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:912)
    at org.apache.drill.exec.store.sys.serialize.JacksonSerializer.deserialize(JacksonSerializer.java:44)
    at org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:138)
    ... 10 more

另外，如何以编程方式添加插件配置？

Edit: Similar behaviour for hive.

Answer 1

Drill 提供了一个 REST API。我使用了 docs 中显示的 curl 命令:

curl -X POST -/json" -d '{"name":"myplugin", "config": {"type": "file", "enabled": false, "connection": "file:///", "workspaces": { "root": { "location": "/", "writable": false, "defaultInputFormat": null}}, "formats": null}}' http://localhost:8047/storage/myplugin.json