当我启动我的Web应用程序时，我收到此错误，我想使用spark从mongodb获取一些数据并执行一些spark-sql，如代码:
我返回了一个空列表，因为我正在测试，但它不应该出现错误，因为问题是当我用 mongospark 填充数据集时。
我使用了spark文件夹中的jar，所以我不认为我的库有问题 我已经在 java 应用程序中尝试过了，它可以工作
public List<temp> test(){
List<temp> listtmp = new ArrayList<>();
SparkSession spark = SparkSession.builder()
.master("spark://192.168.1.103:7077")
.appName("MongoSparkConnectorIntro")
.config("spark.mongodb.input.uri", "mongodb://127.0.0.1/temperature.templog")
.config("spark.mongodb.output.uri", "mongodb://127.0.0.1/temperature.templog")
.getOrCreate();
// Create a JavaSparkContext using the SparkSession's SparkContext object
JavaSparkContext jsc = new JavaSparkContext(spark.sparkContext());
/*Start Example: Read data from MongoDB************************/
Dataset<Row> df = MongoSpark.load(jsc).toDF();
/*End Example**************************************************/
df.createOrReplaceTempView("temp");
Dataset<Row> data = spark.sql("select * from temp");
// Analyze data from MongoDB
return listtmp;
}
错误:
java.lang.ExceptionInInitializerError
at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
at org.apache.spark.rdd.RDD.map(RDD.scala:370)
at com.mongodb.spark.sql.MongoInferSchema$.apply(MongoInferSchema.scala:87)
at com.mongodb.spark.MongoSpark.toDF(MongoSpark.scala:587)
at com.mongodb.spark.rdd.MongoRDD.toDF(MongoRDD.scala:72)
at com.mongodb.spark.rdd.api.java.JavaMongoRDD.toDF(JavaMongoRDD.scala:57)
at model.SparkService.test(SparkService.java:48)
at controller.FiltrageView.getGetresult(FiltrageView.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.el.BeanELResolver.getValue(BeanELResolver.java:363)
at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176)
at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203)
at com.sun.el.parser.AstValue.getValue(AstValue.java:140)
at com.sun.el.parser.AstValue.getValue(AstValue.java:204)
at com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:226)
at com.sun.faces.facelets.el.TagValueExpression.getValue(TagValueExpression.java:109)
at javax.faces.component.ComponentStateHelper.eval(ComponentStateHelper.java:194)
at javax.faces.component.ComponentStateHelper.eval(ComponentStateHelper.java:182)
at javax.faces.component.UIData.getValue(UIData.java:732)
at org.primefaces.component.api.UIData.getDataModel(UIData.java:792)
at javax.faces.component.UIData.getRowCount(UIData.java:356)
at org.primefaces.component.datatable.DataTableRenderer.encodeTbody(DataTableRenderer.java:1141)
at org.primefaces.component.datatable.DataTableRenderer.encodeTbody(DataTableRenderer.java:1120)
at org.primefaces.component.datatable.DataTableRenderer.encodeRegularTable(DataTableRenderer.java:445)
at org.primefaces.component.datatable.DataTableRenderer.encodeMarkup(DataTableRenderer.java:376)
at org.primefaces.component.datatable.DataTableRenderer.encodeEnd(DataTableRenderer.java:95)
at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:920)
at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1863)
at javax.faces.render.Renderer.encodeChildren(Renderer.java:176)
at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:890)
at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1856)
at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1859)
at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1859)
at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:458)
at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:134)
at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:120)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:219)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:659)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:318)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:416)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:283)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:459)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:167)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:206)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:180)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:283)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:200)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:132)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:111)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:536)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:591)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:571)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Jackson version is too old 2.5.1
at com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonModule.scala:56)
at com.fasterxml.jackson.module.scala.JacksonModule.setupModule$(JacksonModule.scala:51)
at com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:651)
at org.apache.spark.rdd.RDDOperationScope$.<init>(RDDOperationScope.scala:82)
at org.apache.spark.rdd.RDDOperationScope$.<clinit>(RDDOperationScope.scala)
... 70 more
最佳答案
我假设您至少使用 Spark 2.0，因为您正在使用 SparkSession
spark 2.0支持的Jackson版本是2.6.5
https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.0.2
检查以 com.fasterxml.jackson
旁白:异常来自 jackson-module-scala_2.11 依赖项中的 JacksonModule.scala，因此 Spark 对最低 Jackson 版本进行了代码级别检查，我发现这非常有趣!
关于java.lang.ExceptionInInitializerError 引起的 : com. fastxml.jackson.databind.JsonMappingException : Jackson version is too old 2. 5.1，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/56402953/