我对 Hadoop 和 Docker 很陌生，我正在按照本教程使用 docker 容器运行 Hadoop 集群。
来源:https://clubhouse.io/developer-how-to/how-to-set-up-a-hadoop-cluster-in-docker/
当我键入以下内容时，就像在教程中一样:
docker ps
所以容器似乎起作用了......但是当我去 http://localhost:9870 或 http://localhost:50070 从namenode查看系统的当前状态时，什么也没有发生(该站点不能达到)。
有谁知道如何解决这个问题？
操作系统:Windows
谢谢
编辑:
有两个“docker-compose”文件:
-docker-compose.yml
-docker-compose-v3.yml
这里是
docker-compose.yml的内容:version: "2"
services:
namenode:
build: ./namenode
image: bde2020/hadoop-namenode:1.1.0-hadoop2.7.1-java8
container_name: namenode
volumes:
- hadoop_namenode:/hadoop/dfs/name
environment:
- CLUSTER_NAME=test
env_file:
- ./hadoop.env
ports:
- "9870:50070"
resourcemanager:
build: ./resourcemanager
image: bde2020/hadoop-resourcemanager:1.1.0-hadoop2.7.1-java8
container_name: resourcemanager
depends_on:
- namenode
- datanode1
- datanode2
env_file:
- ./hadoop.env
ports:
- "8089:8088"
historyserver:
build: ./historyserver
image: bde2020/hadoop-historyserver:1.1.0-hadoop2.7.1-java8
container_name: historyserver
depends_on:
- namenode
- datanode1
- datanode2
volumes:
- hadoop_historyserver:/hadoop/yarn/timeline
env_file:
- ./hadoop.env
nodemanager1:
build: ./nodemanager
image: bde2020/hadoop-nodemanager:1.1.0-hadoop2.7.1-java8
container_name: nodemanager1
depends_on:
- namenode
- datanode1
- datanode2
env_file:
- ./hadoop.env
datanode1:
build: ./datanode
image: bde2020/hadoop-datanode:1.1.0-hadoop2.7.1-java8
container_name: datanode1
depends_on:
- namenode
volumes:
- hadoop_datanode1:/hadoop/dfs/data
env_file:
- ./hadoop.env
datanode2:
build: ./datanode
image: bde2020/hadoop-datanode:1.1.0-hadoop2.7.1-java8
container_name: datanode2
depends_on:
- namenode
volumes:
- hadoop_datanode2:/hadoop/dfs/data
env_file:
- ./hadoop.env
datanode3:
build: ./datanode
image: bde2020/hadoop-datanode:1.1.0-hadoop2.7.1-java8
container_name: datanode3
depends_on:
- namenode
volumes:
- hadoop_datanode3:/hadoop/dfs/data
env_file:
- ./hadoop.env
volumes:
hadoop_namenode:
hadoop_datanode1:
hadoop_datanode2:
hadoop_datanode3:
hadoop_historyserver:
docker run -d -p 80:80 --name myserver nginx
EDIT2:
docker 日志名称节点容器的输出:
$ docker logs f8e90944fedf
Configuring core
- Setting hadoop.proxyuser.hue.hosts=*
- Setting fs.defaultFS=hdfs://namenode:9000
- Setting hadoop.http.staticuser.user=root
- Setting io.compression.codecs=org.apache.hadoop.io.compress.SnappyCodec
- Setting hadoop.proxyuser.hue.groups=*
Configuring hdfs
- Setting dfs.namenode.datanode.registration.ip-hostname-check=false
- Setting dfs.webhdfs.enabled=true
- Setting dfs.permissions.enabled=false
- Setting dfs.namenode.name.dir=file:///hadoop/dfs/name
Configuring yarn
- Setting yarn.timeline-service.enabled=true
- Setting yarn.scheduler.capacity.root.default.maximum-allocation-vcores=4
- Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
- Setting yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
- Setting yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage=98.5
- Setting yarn.log.server.url=http://historyserver:8188/applicationhistory/logs/
- Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
- Setting yarn.timeline-service.generic-application-history.enabled=true
- Setting yarn.log-aggregation-enable=true
- Setting yarn.resourcemanager.hostname=resourcemanager
- Setting yarn.scheduler.capacity.root.default.maximum-allocation-mb=8192
- Setting yarn.nodemanager.aux-services=mapreduce_shuffle
- Setting yarn.resourcemanager.resource_tracker.address=resourcemanager:8031
- Setting yarn.timeline-service.hostname=historyserver
- Setting yarn.resourcemanager.scheduler.address=resourcemanager:8030
- Setting yarn.resourcemanager.address=resourcemanager:8032
- Setting mapred.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec
- Setting yarn.nodemanager.remote-app-log-dir=/app-logs
- Setting yarn.resourcemanager.scheduler.class=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
- Setting mapreduce.map.output.compress=true
- Setting yarn.nodemanager.resource.memory-mb=16384
- Setting yarn.resourcemanager.recovery.enabled=true
- Setting yarn.nodemanager.resource.cpu-vcores=8
Configuring httpfs
Configuring kms
Configuring mapred
- Setting mapreduce.map.java.opts=-Xmx3072m
- Setting mapreduce.reduce.java.opts=-Xmx6144m
- Setting mapreduce.reduce.memory.mb=8192
- Setting yarn.app.mapreduce.am.env=HADOOP_MAPRED_HOME=/opt/hadoop-3.2.1/
- Setting mapreduce.map.memory.mb=4096
- Setting mapred.child.java.opts=-Xmx4096m
- Setting mapreduce.reduce.env=HADOOP_MAPRED_HOME=/opt/hadoop-3.2.1/
- Setting mapreduce.framework.name=yarn
- Setting mapreduce.map.env=HADOOP_MAPRED_HOME=/opt/hadoop-3.2.1/
Configuring for multihomed network
remove lost+found from /hadoop/dfs/name
rm: cannot remove '/hadoop/dfs/name/lost+found': No such file or directory
2020-08-10 10:53:49,592 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = f8e90944fedf/172.22.0.2
STARTUP_MSG: args = []
STARTUP_MSG: version = 3.2.1
STARTUP_MSG: build = https://gitbox.apache.org/repos/asf/hadoop.git -r b3cbbb467e22ea829b3808f4b7b01d07e0bf3842; compiled by 'rohithsharmaks' on 2019-09-10T15:56Z
STARTUP_MSG: java = 1.8.0_232
************************************************************/
2020-08-10 10:53:50,063 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2020-08-10 10:53:55,494 INFO namenode.NameNode: createNameNode []
2020-08-10 10:54:01,381 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2020-08-10 10:54:05,032 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2020-08-10 10:54:05,056 INFO impl.MetricsSystemImpl: NameNode metrics system started
2020-08-10 10:54:05,617 INFO namenode.NameNodeUtils: fs.defaultFS is hdfs://namenode:9000
2020-08-10 10:54:05,623 INFO namenode.NameNode: Clients should use namenode:9000 to access this namenode/service.
2020-08-10 10:54:10,545 INFO util.JvmPauseMonitor: Starting JVM pause monitor
2020-08-10 10:54:11,572 INFO hdfs.DFSUtil: Starting Web-server for hdfs at: http://0.0.0.0:9870
2020-08-10 10:54:11,964 INFO util.log: Logging initialized @37617ms
2020-08-10 10:54:15,166 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2020-08-10 10:54:15,510 INFO http.HttpRequestLog: Http request log for http.requests.namenode is not defined
2020-08-10 10:54:15,814 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2020-08-10 10:54:15,898 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
2020-08-10 10:54:15,899 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2020-08-10 10:54:15,903 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2020-08-10 10:54:16,378 INFO http.HttpServer2: Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
2020-08-10 10:54:16,386 INFO http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
2020-08-10 10:54:16,704 INFO http.HttpServer2: Jetty bound to port 9870
2020-08-10 10:54:16,720 INFO server.Server: jetty-9.3.24.v20180605, build timestamp: 2018-06-05T17:11:56Z, git hash: 84205aa28f11a4f31f2a3b86d1bba2cc8ab69827
2020-08-10 10:54:17,880 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@7d0b7e3c{/logs,file:///opt/hadoop-3.2.1/logs/,AVAILABLE}
2020-08-10 10:54:17,910 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@4b741d6d{/static,file:///opt/hadoop-3.2.1/share/hadoop/hdfs/webapps/static/,AVAILABLE}
2020-08-10 10:54:21,077 INFO handler.ContextHandler: Started o.e.j.w.WebAppContext@7486b455{/,file:///opt/hadoop-3.2.1/share/hadoop/hdfs/webapps/hdfs/,AVAILABLE}{/hdfs}
2020-08-10 10:54:21,250 INFO server.AbstractConnector: Started ServerConnector@6de0a3f9{HTTP/1.1,[http/1.1]}{0.0.0.0:9870}
2020-08-10 10:54:21,268 INFO server.Server: Started @46920ms
2020-08-10 10:54:29,608 WARN namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!
2020-08-10 10:54:29,609 WARN namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of data loss due to lack of redundant storage directories!
2020-08-10 10:54:34,440 INFO namenode.FSEditLog: Edit logging is async:true
2020-08-10 10:54:35,312 INFO namenode.FSNamesystem: KeyProvider: null
2020-08-10 10:54:35,490 INFO namenode.FSNamesystem: fsLock is fair: true
2020-08-10 10:54:35,570 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2020-08-10 10:54:36,027 INFO namenode.FSNamesystem: fsOwner = root (auth:SIMPLE)
2020-08-10 10:54:36,034 INFO namenode.FSNamesystem: supergroup = supergroup
2020-08-10 10:54:36,035 INFO namenode.FSNamesystem: isPermissionEnabled = false
2020-08-10 10:54:36,035 INFO namenode.FSNamesystem: HA Enabled: false
2020-08-10 10:54:37,237 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2020-08-10 10:54:37,572 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2020-08-10 10:54:37,576 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=false
2020-08-10 10:54:37,758 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2020-08-10 10:54:37,774 INFO blockmanagement.BlockManager: The block deletion will start around 2020 Aug 10 10:54:37
2020-08-10 10:54:37,816 INFO util.GSet: Computing capacity for map BlocksMap
2020-08-10 10:54:37,822 INFO util.GSet: VM type = 64-bit
2020-08-10 10:54:37,849 INFO util.GSet: 2.0% max memory 239.8 MB = 4.8 MB
2020-08-10 10:54:37,854 INFO util.GSet: capacity = 2^19 = 524288 entries
2020-08-10 10:54:38,122 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
2020-08-10 10:54:38,126 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
2020-08-10 10:54:38,293 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS
2020-08-10 10:54:38,295 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
2020-08-10 10:54:38,298 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2020-08-10 10:54:38,299 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2020-08-10 10:54:38,301 INFO blockmanagement.BlockManager: defaultReplication = 3
2020-08-10 10:54:38,311 INFO blockmanagement.BlockManager: maxReplication = 512
2020-08-10 10:54:38,312 INFO blockmanagement.BlockManager: minReplication = 1
2020-08-10 10:54:38,313 INFO blockmanagement.BlockManager: maxReplicationStreams = 2
2020-08-10 10:54:38,313 INFO blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms
2020-08-10 10:54:38,317 INFO blockmanagement.BlockManager: encryptDataTransfer = false
2020-08-10 10:54:38,317 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000
2020-08-10 10:54:39,701 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
2020-08-10 10:54:39,705 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
2020-08-10 10:54:39,706 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
2020-08-10 10:54:39,707 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
2020-08-10 10:54:40,102 INFO util.GSet: Computing capacity for map INodeMap
2020-08-10 10:54:40,116 INFO util.GSet: VM type = 64-bit
2020-08-10 10:54:40,172 INFO util.GSet: 1.0% max memory 239.8 MB = 2.4 MB
2020-08-10 10:54:40,210 INFO util.GSet: capacity = 2^18 = 262144 entries
2020-08-10 10:54:40,322 INFO namenode.FSDirectory: ACLs enabled? false
2020-08-10 10:54:40,328 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
2020-08-10 10:54:40,330 INFO namenode.FSDirectory: XAttrs enabled? true
2020-08-10 10:54:40,331 INFO namenode.NameNode: Caching file names occurring more than 10 times
2020-08-10 10:54:40,638 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
2020-08-10 10:54:40,737 INFO snapshot.SnapshotManager: SkipList is disabled
2020-08-10 10:54:40,953 INFO util.GSet: Computing capacity for map cachedBlocks
2020-08-10 10:54:40,968 INFO util.GSet: VM type = 64-bit
2020-08-10 10:54:40,969 INFO util.GSet: 0.25% max memory 239.8 MB = 613.8 KB
2020-08-10 10:54:40,973 INFO util.GSet: capacity = 2^16 = 65536 entries
2020-08-10 10:54:41,321 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2020-08-10 10:54:41,322 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2020-08-10 10:54:41,322 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2020-08-10 10:54:41,597 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
2020-08-10 10:54:41,599 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2020-08-10 10:54:41,637 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2020-08-10 10:54:41,641 INFO util.GSet: VM type = 64-bit
2020-08-10 10:54:41,644 INFO util.GSet: 0.029999999329447746% max memory 239.8 MB = 73.7 KB
2020-08-10 10:54:41,647 INFO util.GSet: capacity = 2^13 = 8192 entries
2020-08-10 10:54:42,357 INFO common.Storage: Lock on /hadoop/dfs/name/in_use.lock acquired by nodename 360@f8e90944fedf
2020-08-10 10:54:43,559 INFO namenode.FileJournalManager: Recovering unfinalized segments in /hadoop/dfs/name/current
2020-08-10 10:54:44,466 INFO namenode.FileJournalManager: Finalizing edits file /hadoop/dfs/name/current/edits_inprogress_0000000000000000032 -> /hadoop/dfs/name/current/edits_0000000000000000032-0000000000000000032
2020-08-10 10:54:45,218 INFO namenode.FSImage: Planning to load image: FSImageFile(file=/hadoop/dfs/name/current/fsimage_0000000000000000029, cpktTxId=0000000000000000029)
2020-08-10 10:54:47,482 INFO namenode.FSImageFormatPBINode: Loading 18 INodes.
2020-08-10 10:54:48,937 INFO namenode.FSImageFormatProtobuf: Loaded FSImage in 3 seconds.
2020-08-10 10:54:48,943 INFO namenode.FSImage: Loaded image for txid 29 from /hadoop/dfs/name/current/fsimage_0000000000000000029
2020-08-10 10:54:48,999 INFO namenode.FSImage: Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@16c63f5 expecting start txid #30
2020-08-10 10:54:49,007 INFO namenode.FSImage: Start loading edits file /hadoop/dfs/name/current/edits_0000000000000000030-0000000000000000030 maxTxnsToRead = 9223372036854775807
2020-08-10 10:54:49,050 INFO namenode.RedundantEditLogInputStream: Fast-forwarding stream '/hadoop/dfs/name/current/edits_0000000000000000030-0000000000000000030' to transaction ID 30
2020-08-10 10:54:51,152 INFO namenode.FSImage: Loaded 1 edits file(s) (the last named /hadoop/dfs/name/current/edits_0000000000000000030-0000000000000000030) of total size 1048576.0, total edits 1.0, total load time 569.0 ms
2020-08-10 10:54:51,165 INFO namenode.RedundantEditLogInputStream: Fast-forwarding stream '/hadoop/dfs/name/current/edits_0000000000000000031-0000000000000000031' to transaction ID 30
2020-08-10 10:54:51,190 INFO namenode.RedundantEditLogInputStream: Fast-forwarding stream '/hadoop/dfs/name/current/edits_0000000000000000032-0000000000000000032' to transaction ID 30
2020-08-10 10:54:51,209 INFO namenode.FSNamesystem: Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
2020-08-10 10:54:51,220 INFO namenode.FSEditLog: Starting log segment at 33
2020-08-10 10:54:53,045 INFO namenode.NameCache: initialized with 0 entries 0 lookups
2020-08-10 10:54:53,048 INFO namenode.FSNamesystem: Finished loading FSImage in 11158 msecs
2020-08-10 10:54:55,085 INFO namenode.NameNode: RPC server is binding to 0.0.0.0:9000
2020-08-10 10:54:55,728 INFO ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 1000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2020-08-10 10:54:55,929 INFO ipc.Server: Starting Socket Reader #1 for port 9000
2020-08-10 10:54:59,335 INFO namenode.FSNamesystem: Registered FSNamesystemState, ReplicatedBlocksState and ECBlockGroupsState MBeans.
2020-08-10 10:54:59,515 INFO namenode.LeaseManager: Number of blocks under construction: 0
2020-08-10 10:54:59,690 INFO hdfs.StateChange: STATE* Safe mode ON.
The reported blocks 0 needs additional 1 blocks to reach the threshold 0.9990 of total blocks 2.
The minimum number of live datanodes is not required. Safe mode will be turned off automatically once the thresholds have been reached.
2020-08-10 10:55:00,177 INFO ipc.Server: IPC Server Responder: starting
2020-08-10 10:55:00,209 INFO ipc.Server: IPC Server listener on 9000: starting
2020-08-10 10:55:00,265 INFO namenode.NameNode: NameNode RPC up at: namenode/172.22.0.2:9000
2020-08-10 10:55:00,330 INFO namenode.FSNamesystem: Starting services required for active state
2020-08-10 10:55:00,336 INFO namenode.FSDirectory: Initializing quota with 4 thread(s)
2020-08-10 10:55:00,709 INFO namenode.FSDirectory: Quota initialization completed in 370 milliseconds
name space=18
storage space=75
storage types=RAM_DISK=0, SSD=0, DISK=0, ARCHIVE=0, PROVIDED=0
2020-08-10 10:55:00,807 INFO blockmanagement.CacheReplicationMonitor: Starting CacheReplicationMonitor with interval 30000 milliseconds
2020-08-10 10:55:13,692 INFO hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(172.22.0.5:9866, datanodeUuid=bce9d643-ec21-425e-a7b1-937090c2d19b, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038) storage bce9d643-ec21-425e-a7b1-937090c2d19b
2020-08-10 10:55:13,706 INFO net.NetworkTopology: Adding a new node: /default-rack/172.22.0.5:9866
2020-08-10 10:55:13,708 INFO blockmanagement.BlockReportLeaseManager: Registered DN bce9d643-ec21-425e-a7b1-937090c2d19b (172.22.0.5:9866).
2020-08-10 10:55:14,185 INFO hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(172.22.0.3:9866, datanodeUuid=48c6289d-41b9-4d4e-a39f-f01fc6a16d4e, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038) storage 48c6289d-41b9-4d4e-a39f-f01fc6a16d4e
2020-08-10 10:55:14,190 INFO net.NetworkTopology: Adding a new node: /default-rack/172.22.0.3:9866
2020-08-10 10:55:14,196 INFO blockmanagement.BlockReportLeaseManager: Registered DN 48c6289d-41b9-4d4e-a39f-f01fc6a16d4e (172.22.0.3:9866).
2020-08-10 10:55:14,455 INFO hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(172.22.0.4:9866, datanodeUuid=7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038) storage 7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd
2020-08-10 10:55:14,457 INFO net.NetworkTopology: Adding a new node: /default-rack/172.22.0.4:9866
2020-08-10 10:55:14,458 INFO blockmanagement.BlockReportLeaseManager: Registered DN 7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd (172.22.0.4:9866).
2020-08-10 10:55:14,786 INFO blockmanagement.DatanodeDescriptor: Adding new storage ID DS-9aa7c881-0925-4ef4-917a-2a2c9ec7b935 for DN 172.22.0.5:9866
2020-08-10 10:55:14,845 INFO blockmanagement.DatanodeDescriptor: Adding new storage ID DS-e6deaf47-6082-4c65-8b98-89edac2df8f8 for DN 172.22.0.3:9866
2020-08-10 10:55:14,991 INFO blockmanagement.DatanodeDescriptor: Adding new storage ID DS-6359f398-c37d-43eb-adc2-4978cbdff889 for DN 172.22.0.4:9866
2020-08-10 10:55:15,259 INFO BlockStateChange: BLOCK* processReport 0xf1b19f021e1b1e0f: Processing first storage report for DS-9aa7c881-0925-4ef4-917a-2a2c9ec7b935 from datanode bce9d643-ec21-425e-a7b1-937090c2d19b
2020-08-10 10:55:15,337 INFO blockmanagement.BlockManager: initializing replication queues
2020-08-10 10:55:15,344 INFO hdfs.StateChange: STATE* Safe mode extension entered.
The reported blocks 1 has reached the threshold 0.9990 of total blocks 2. The minimum number of live datanodes is not required. In safe mode extension. Safe mode will be turned off automatically in 29 seconds.
2020-08-10 10:55:15,353 INFO BlockStateChange: BLOCK* processReport 0xf1b19f021e1b1e0f: from storage DS-9aa7c881-0925-4ef4-917a-2a2c9ec7b935 node DatanodeRegistration(172.22.0.5:9866, datanodeUuid=bce9d643-ec21-425e-a7b1-937090c2d19b, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038), blocks: 2, hasStaleStorage: false, processing time: 94 msecs, invalidatedBlocks: 0
2020-08-10 10:55:15,386 INFO blockmanagement.BlockManager: Total number of blocks = 2
2020-08-10 10:55:15,398 INFO blockmanagement.BlockManager: Number of invalid blocks = 0
2020-08-10 10:55:15,402 INFO blockmanagement.BlockManager: Number of under-replicated blocks = 1
2020-08-10 10:55:15,403 INFO blockmanagement.BlockManager: Number of over-replicated blocks = 0
2020-08-10 10:55:15,403 INFO blockmanagement.BlockManager: Number of blocks being written = 0
2020-08-10 10:55:15,403 INFO hdfs.StateChange: STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 55 msec
2020-08-10 10:55:15,404 INFO BlockStateChange: BLOCK* processReport 0x5e59b4fa3cbbe2af: Processing first storage report for DS-6359f398-c37d-43eb-adc2-4978cbdff889 from datanode 7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd
2020-08-10 10:55:15,405 INFO BlockStateChange: BLOCK* processReport 0x5e59b4fa3cbbe2af: from storage DS-6359f398-c37d-43eb-adc2-4978cbdff889 node DatanodeRegistration(172.22.0.4:9866, datanodeUuid=7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038), blocks: 2, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0
2020-08-10 10:55:15,411 INFO BlockStateChange: BLOCK* processReport 0x829ff257cefe75bb: Processing first storage report for DS-e6deaf47-6082-4c65-8b98-89edac2df8f8 from datanode 48c6289d-41b9-4d4e-a39f-f01fc6a16d4e
2020-08-10 10:55:15,448 INFO BlockStateChange: BLOCK* processReport 0x829ff257cefe75bb: from storage DS-e6deaf47-6082-4c65-8b98-89edac2df8f8 node DatanodeRegistration(172.22.0.3:9866, datanodeUuid=48c6289d-41b9-4d4e-a39f-f01fc6a16d4e, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038), blocks: 2, hasStaleStorage: false, processing time: 37 msecs, invalidatedBlocks: 0
2020-08-10 10:55:24,291 INFO ipc.Server: IPC Server handler 0 on default port 9000, call Call#0 Retry#0 org.apache.hadoop.hdfs.protocol.ClientProtocol.mkdirs from 172.22.0.6:42424: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /rmstate/FSRMStateRoot/RMDTSecretManagerRoot. Name node is in safe mode.
The reported blocks 2 has reached the threshold 0.9990 of total blocks 2. The minimum number of live datanodes is not required. In safe mode extension. Safe mode will be turned off automatically in 21 seconds. NamenodeHostName:namenode
2020-08-10 10:55:35,378 INFO hdfs.StateChange: STATE* Safe mode ON, in safe mode extension.
The reported blocks 2 has reached the threshold 0.9990 of total blocks 2. The minimum number of live datanodes is not required. In safe mode extension. Safe mode will be turned off automatically in 9 seconds.
2020-08-10 10:55:45,389 INFO hdfs.StateChange: STATE* Safe mode is OFF
2020-08-10 10:55:45,389 INFO hdfs.StateChange: STATE* Leaving safe mode after 45 secs
2020-08-10 10:55:45,390 INFO hdfs.StateChange: STATE* Network topology has 1 racks and 3 datanodes
2020-08-10 10:55:45,390 INFO hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks
最佳答案
你能提供 docker-compose.yml 文件吗？
但我认为您的问题与端口转发有关。
您必须在 docker-compose.yml 上启用它。
例子:
services:
hadoop:
ports:
- "9870:9870"
关于docker - 容器正在运行但无法访问本地主机，我该如何解决这个问题？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/63337975/