java - 当我在github中运行spring-hadoop-sample并发现我的yarn应用程序以失败状态开始时

标签 java spring-boot hadoop sample

我是Hadoop的新手,现在我在一台机器上运行Pseudo-Distributed模型,并且我想在GitHub中运行yarn示例。 https://github.com/spring-projects/spring-hadoop-samples/tree/master/boot/yarn-boot-simple
但是,当我按照指导运行该广口瓶时,发现我的 yarn 应用程序在应用程序列表中失败。

我已经运行了start-all.sh,看来一切正常,当我使用jps -l命令时,节点正在运行

[root@server25 container_1550477585755_0003_02_000001]# jps|grep Node
24731 SecondaryNameNode
25286 NodeManager
24018 NameNode
24275 DataNode

当我运行jar时,这是日志:
[root@server25 jar_repo]# java -jar yarn-boot-simple-client-0.1.0.jar 

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.2.3.RELEASE)

[2019-02-18 17:40:49.706] boot - 11970  INFO [main] --- ClientApplication: Starting ClientApplication on server25 with PID 11970 (/opt/HADOOP/jar_repo/yarn-boot-simple-client-0.1.0.jar started by root in /opt/HADOOP/jar_repo)
[2019-02-18 17:40:49.805] boot - 11970  INFO [main] --- AnnotationConfigApplicationContext: Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@2ca0cbe4: startup date [Mon Feb 18 17:40:49 CST 2019]; root of context hierarchy
[2019-02-18 17:40:53.187] boot - 11970  INFO [main] --- ConfiguringBeanFactoryPostProcessor: No bean named 'taskScheduler' has been explicitly defined. Therefore, a default ThreadPoolTaskScheduler will be created.
[2019-02-18 17:40:53.195] boot - 11970  INFO [main] --- ConfiguringBeanFactoryPostProcessor: No bean named 'taskExecutor' has been explicitly defined. Therefore, a default SyncTaskExecutor will be created.
[2019-02-18 17:40:53.210] boot - 11970  INFO [main] --- AutowiredAnnotationBeanPostProcessor: JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
[2019-02-18 17:40:53.891] boot - 11970  INFO [main] --- ThreadPoolTaskExecutor: Initializing ExecutorService  'metricsExecutor'
[2019-02-18 17:40:56.060] boot - 11970  INFO [main] --- SpringYarnConfiguration: Enabling CLIENT for Yarn
[2019-02-18 17:40:56.072] boot - 11970  INFO [main] --- SpringYarnConfiguration: We couldn't figure out if we could use existing configuration
[2019-02-18 17:40:56.072] boot - 11970  INFO [main] --- SpringYarnConfiguration: Building configuration for bean 'yarnConfiguration'
[2019-02-18 17:40:56.284] boot - 11970  INFO [main] --- SpringYarnConfigBuilder: Existing yarnConfiguration: null
[2019-02-18 17:40:57.117] boot - 11970  INFO [main] --- ConfigurationFactoryBean: Overwriting fsUri=[file:///] with fsUri=[hdfs://localhost:8020]
[2019-02-18 17:40:57.118] boot - 11970  INFO [main] --- ConfigurationFactoryBean: Overwriting rmAddress=[0.0.0.0:8032] with rmAddress=[localhost:8032]
[2019-02-18 17:40:58.083] boot - 11970  WARN [main] --- NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[2019-02-18 17:40:58.365] boot - 11970  INFO [main] --- ConfigurationFactoryBean: Executing with tokens:
[2019-02-18 17:40:58.399] boot - 11970  INFO [main] --- SpringYarnConfigBuilder: Setting configuration for SpringYarnConfigs:  fs.defaultFS=hdfs://localhost:8020 yarn.resourcemanager.address=localhost:8032 Configuration: core-default.xml, core-site.xml, yarn-default.xml, yarn-site.xml, mapred-default.xml, mapred-site.xml
[2019-02-18 17:40:58.409] boot - 11970  INFO [main] --- EnvironmentFactoryBean: Adding CLASSPATH=./*
[2019-02-18 17:40:59.181] boot - 11970  INFO [main] --- ThreadPoolTaskScheduler: Initializing ExecutorService  'taskScheduler'
[2019-02-18 17:40:59.191] boot - 11970  INFO [main] --- AnnotationMBeanExporter: Registering beans for JMX exposure on startup
[2019-02-18 17:40:59.203] boot - 11970  INFO [main] --- DefaultLifecycleProcessor: Starting beans in phase 0
[2019-02-18 17:40:59.612] boot - 11970  INFO [main] --- ClientApplication: Started ClientApplication in 11.105 seconds (JVM running for 12.658)
[2019-02-18 17:40:59.818] boot - 11970  INFO [main] --- AbstractYarnClient: submitApplication, got applicationId=[application_1550477585755_0004]
[2019-02-18 17:40:59.818] boot - 11970  INFO [main] --- AbstractResourceLocalizer: Setting stagingId=application_1550477585755_0004
[2019-02-18 17:40:59.818] boot - 11970  INFO [main] --- AbstractResourceLocalizer: Marking distributed state false
[2019-02-18 17:40:59.819] boot - 11970  INFO [main] --- AbstractResourceLocalizer: About to distribute localized files
[2019-02-18 17:41:03.150] boot - 11970  INFO [Thread-2] --- AnnotationConfigApplicationContext: Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@2ca0cbe4: startup date [Mon Feb 18 17:40:49 CST 2019]; root of context hierarchy
[2019-02-18 17:41:03.163] boot - 11970  INFO [Thread-2] --- DefaultLifecycleProcessor: Stopping beans in phase 0
[2019-02-18 17:41:03.166] boot - 11970  INFO [Thread-2] --- ThreadPoolTaskScheduler: Shutting down ExecutorService 'taskScheduler'
[2019-02-18 17:41:03.171] boot - 11970  INFO [Thread-2] --- AnnotationMBeanExporter: Unregistering JMX-exposed beans on shutdown
[2019-02-18 17:41:03.172] boot - 11970  INFO [Thread-2] --- ThreadPoolTaskExecutor: Shutting down ExecutorService 'metricsExecutor'

和下面的/opt/HADOOP/hadoop-2.9.2/logs/userlogs/application_1550477585755_0004/container_1550477585755_0004_02_000001/Appmaster.stdout日志:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.2.3.RELEASE)

[2019-02-18 17:41:11.945] boot - 12386  INFO [main] --- AppmasterApplication: Starting AppmasterApplication on server25 with PID 12386 (/tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1550477585755_0004/filecache/10/yarn-boot-simple-appmaster-0.1.0.jar started by root in /tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1550477585755_0004/container_1550477585755_0004_02_000001)
[2019-02-18 17:41:12.038] boot - 12386  INFO [main] --- AnnotationConfigApplicationContext: Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@2ca0cbe4: startup date [Mon Feb 18 17:41:12 CST 2019]; root of context hierarchy
[2019-02-18 17:41:14.261] boot - 12386  INFO [Thread-2] --- AnnotationConfigApplicationContext: Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@2ca0cbe4: startup date [Mon Feb 18 17:41:12 CST 2019]; root of context hierarchy
[2019-02-18 17:41:14.266] boot - 12386  WARN [Thread-2] --- AnnotationConfigApplicationContext: Exception thrown from ApplicationListener handling ContextClosedEvent
java.lang.IllegalStateException: ApplicationEventMulticaster not initialized - call 'refresh' before multicasting events via the context: org.springframework.context.annotation.AnnotationConfigApplicationContext@2ca0cbe4: startup date [Mon Feb 18 17:41:12 CST 2019]; root of context hierarchy
        at org.springframework.context.support.AbstractApplicationContext.getApplicationEventMulticaster(AbstractApplicationContext.java:344)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:331)
        at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:869)
        at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:804)
[2019-02-18 17:41:14.273] boot - 12386  WARN [Thread-2] --- AnnotationConfigApplicationContext: Exception thrown from LifecycleProcessor on context close
java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' before invoking lifecycle methods via the context: org.springframework.context.annotation.AnnotationConfigApplicationContext@2ca0cbe4: startup date [Mon Feb 18 17:41:12 CST 2019]; root of context hierarchy
        at org.springframework.context.support.AbstractApplicationContext.getLifecycleProcessor(AbstractApplicationContext.java:357)
        at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:877)
        at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:804)

我唯一能确定的是 jar 先启动然后自行关闭,有人可以帮助我吗?非常感谢。

最佳答案

如果您想帮助我解决这个问题,很抱歉,当我根据此处的Java版本指南https://wiki.apache.org/hadoop/HadoopJavaVersions.Finaly将我的JDK版本从jdk8切换到jdk7时,我就知道了,状态和FinalStatus已完成且成功。感谢试图帮助我的 friend 。

关于java - 当我在github中运行spring-hadoop-sample并发现我的yarn应用程序以失败状态开始时,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/54744405/

相关文章:

java - 方法的递归实现

java - 设置传入 httpServletRequest 的 cookie 版本

java - 多模块组件扫描在 Spring Boot 中不起作用

hadoop - 尝试将 Windows pentaho 连接到虚拟机 HDFS 时出错

java - hadoop mapreduce IntWritable范围有多长?

Hadoop 2.6.1 单节点设置 : Data Node is not up

java - 获取 http 响应 PHP 和 ANDROID

java - 在动态 Web 应用程序中使用 JDBC

java - 如何在 Spring Boot Rest 响应中将 boolean 值序列化为字符串?

java - 如何使用存储库模式更新 mongodb 中的嵌入文档