hadoop - MapReduce编程模型-Mapper可以在 map 处理过程中相互通信吗？

Closed. This question needs details or clarity。它当前不接受答案。

想改善这个问题吗？添加详细信息，并通过editing this post阐明问题。

6年前关闭。

Improve this question

我知道reduce任务必须独立且独立地运行。但是对于Mapper来说，映射器似乎有机会相互交流吗？如果是这样，请解释。

最佳答案

映射器不会相互通信。这样做是为了确保每个映射任务的可靠性仅由运行该映射任务的机器的可靠性来控制。

请参阅YDN教程的摘录，以获得更好的理解:

If Mappers and Reducers had individual identities and communicated with one another or the outside world, then restarting a task would require the other nodes to communicate with the new instances of the map and reduce tasks, and the re-executed tasks would need to reestablish their intermediate state. This process is notoriously complicated and error-prone in the general case. MapReduce simplifies this problem drastically by eliminating task identities or the ability for task partitions to communicate with one another. An individual task sees only its own direct inputs and knows only its own outputs, to make this failure and restart process clean and dependable.

附注:请问您是怎么想的？

关于hadoop - MapReduce编程模型-Mapper可以在 map 处理过程中相互通信吗？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/21416131/

上一篇：docker - 已安装PHP，Composer和Docker的Docker镜像

下一篇：hadoop - HDFS block 顺序如何

variables - 通过参数初始化Hadoop中的公共(public)静态变量

hadoop - Hadoop 的 TaskStatus 类中的 getProgress() 方法是什么？

hadoop - 如何从S3获取数据并将其用于Elastic map reduce/在哪里写代码？

scala - Apache Spark - 生成对列表

c++ - 将 QWidget 映射到变量

Hadoop:数据节点没有启动

c# - EmitMapper 的集合列表映射问题

hadoop - 具有身份映射器的 MapReduce 作业失败，返回 "Type mismatch in key from map"

hadoop - 在 Hive 中将字符串值转换为十进制时为 NULL