我正在使用 YarnClient 以编程方式开始一项工作。我正在运行的集群已经过 kerberos 化。
法线贴图减少通过“yarn jar examples.jar wordcount ...”提交的工作。
我尝试以编程方式提交的作业没有。我收到此错误:
14/09/04 21:14:29 ERROR client.ClientService: Error happened during application submit: Application application_1409863263326_0002 failed 2 times due to AM Container for appattempt_1409863263326_0002_000002 exited with exitCode: -1000 due to: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "yarn-c1-n1.clouddev.snaplogic.com/10.184.28.108"; destination host is: "yarn-c1-cdh.clouddev.snaplogic.com":8020; .Failing this attempt.. Failing the application. 14/09/04 21:14:29 ERROR client.YClient: Application submission failed
代码看起来像这样:
ClientContext context = createContextFrom(args);
YarnConfiguration configuration = new YarnConfiguration();
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(configuration);
ClientService client = new ClientService(context, yarnClient, new InstallManager(FileSystem.get(configuration)));
LOG.info(Messages.RUNNING_CLIENT_SERVICE);
boolean result = client.execute();
我想也许添加一些东西的效果是:
yarnClient.getRMDelegationToken(new Text(InetAddress.getLocalHost().getHostAddress()));
也许可以减轻我的痛苦,但这似乎也无济于事。任何帮助将不胜感激。
最佳答案
好吧,经过数小时又数小时的时间,我们已经弄清楚了。对于所有后代的编码人员,永远被 hadoop 缺乏文档所困扰:
您必须通过调用获取凭据从 UserGroupInformation 对象中获取 token 。然后,您必须在 ContainerLaunchContext 上设置 token 。
关于hadoop - 客户端无法通过 :[TOKEN, KERBEROS 进行身份验证],我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/25755479/