我的目标是让HDFS的replica block的位置节点自己定。 查看HDFS的代码后,我在这里找到了调用轨迹(hadoop 2.2版本)
BlockManager.getDatanodeDescriptors(List<String>) line: 1400
BlockManager.chooseTarget(String, int, DatanodeDescriptor, HashMap<Node,Node>, long, List<String>) line: 1378
FSNamesystem.getAdditionalBlock(String, long, String, ExtendedBlock, HashMap<Node,Node>, List<String>) line: 2477
NameNodeRpcServer.addBlock(String, String, ExtendedBlock, DatanodeInfo[], long, String[]) line: 555
ClientNamenodeProtocolServerSideTranslatorPB.addBlock(RpcController, ClientNamenodeProtocolProtos$AddBlockRequestProto) line: 387
ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(Descriptors$MethodDescriptor, RpcController, Message) line: 59582
ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(RPC$Server, String, Writable, long) line: 585
ProtobufRpcEngine$Server(RPC$Server).call(RPC$RpcKind, String, Writable, long) line: 928
我发现 BlockManager.java 有一个 chooseTarget
方法:
public DatanodeDescriptor[] chooseTarget(final String src,
final int numOfReplicas, final DatanodeDescriptor client,
final HashMap<Node, Node> excludedNodes,
final long blocksize, List<String> favoredNodes) throws IOException {
我想如果我可以设置 List<String> favoredNodes
的值(value),那么我的目标就可以实现。
最佳答案
我从 DistributedFileSystem.java(hadoop 2.2 版本)找到 API
/**
* Same as
* {@link #create(Path, FsPermission, boolean, int, short, long,
* Progressable)} with the addition of favoredNodes that is a hint to
* where the namenode should place the file blocks.
* The favored nodes hint is not persisted in HDFS. Hence it may be honored
* at the creation time only. HDFS could move the blocks during balancing or
* replication, to move the blocks from favored nodes. A value of null means
* no favored nodes for this create
*/
public HdfsDataOutputStream create(final Path f,
final FsPermission permission, final boolean overwrite,
final int bufferSize, final short replication, final long blockSize,
final Progressable progress, final InetSocketAddress[] favoredNodes)
throws IOException {
关于hadoop - 如何让HDFS的replica block的位置自己设置?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/32779439/