如何设置 Spark 集群而不使用无密码访问设置 ssh,因为 document提到:
If you do not have a password-less setup, you can set the environment variable SPARK_SSH_FOREGROUND and serially provide a password for each worker.
我想知道如何使用 SPARK_SSH_FOREGROUND 变量??
最佳答案
环境变量SPARK_SSH_FOREGROUND
可以设置为yes
或y
之类的东西,以在前台执行ssh
正如您在源代码中看到的:
https://github.com/apache/spark/pull/2444/commits/1bba8a959ea9ecf21eb0783d3ce302065bf1a154
if [ "${SPARK_SSH_FOREGROUND}" = "y" ] || [ "${SPARK_SSH_FOREGROUND}" = "yes" ]; then
ssh $SPARK_SSH_OPTS "$slave" $"${@// /\\ }" \
关于ssh - 创建spark集群,无需ssh无密码访问,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/37246119/