scala - 登录模块控制标志在 JAAS 配置中不可用 - Scala Kafka

Question

尝试使用 kerberos 身份验证连接到 Kafka 时遇到问题。使用 scala 和我的 jaas.config看起来像这样。

 KafkaClient { <br />
        com.sun.security.auth.module.Krb5LoginModule required <br />
        useKeyTab=true<br />
        keyTab="/etc/security/keytabs/storm.service.keytab"<br />
        storeKey=true<br />
        useTicketCache=false<br />
        serviceName="kafka"<br />
        principal="storm@EXAMPLE.COM";<br />
        debug=true<br />
        client=true; };<br />

Exception in thread "main" org.apache.kafka.common.KafkaException: Failed to construct kafka consumer  
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:799)  
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:615)  
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:596)  
    at org.apache.spark.sql.kafka010.SubscribeStrategy.createConsumer(ConsumerStrategy.scala:62)  
    at org.apache.spark.sql.kafka010.KafkaOffsetReader.createConsumer(KafkaOffsetReader.scala:314)  
    at org.apache.spark.sql.kafka010.KafkaOffsetReader.<init>(KafkaOffsetReader.scala:78)  
    at
 org.apache.spark.sql.kafka010.KafkaSourceProvider.createContinuousReader(KafkaSourceProvider.scala:130)
at org.apache.spark.sql.kafka010.KafkaSourceProvider.createContinuousReader(KafkaSourceProvider.scala:43)  
    at org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:185)
    at com.gm.SparkDataIngest.Main$.main(Main.scala:119)
    at com.gm.SparkDataIngest.Main.main(Main.scala)  

Caused by: java.lang.IllegalArgumentException: Login module control flag is not available in the JAAS config  
    at org.apache.kafka.common.security.JaasConfig.loginModuleControlFlag(JaasConfig.java:85)
    at org.apache.kafka.common.security.JaasConfig.parseAppConfigurationEntry(JaasConfig.java:111)
    at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:63)
    at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:148)
    at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:142)
    at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:119)
    at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65)
    at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:713)

Answer 1

如果您的 JAAS 配置确实是您所显示的，那么您有一个 语法错误 -- 具体来说，一个额外的分号 ; 这使解析器失败，因为它找到了没有意义的第二个配置条目。

JAAS 配置应该是简单的(并注意分号的位置)......

Blurb {
  some.login.module.class status
  option1=value1
  option2="value2"
  ;
};

...或复杂(现在您了解分号的含义...

Blurb {
  some.login.module.class status
  option1=value1
  option2="value2"
  ;
  other.login.module.class status
  option3=value3
  ;
};
DahDah {
  some.login.module.class status
  option1=value99
  option2="value88"
  ;
};

顺便说一句，您可以通过设置 -Djava.security.debug=configparser 将 JAAS 配置解析器切换到 Debug模式。

就我个人而言，我总是使用组合 -Dsun.security.krb5.debug=true -Djava.security.debug=gssloginconfig,configfile,configparser,logincontext在对 Kerberos 问题进行故障排除时。