networking - Docker 容器在 localhost 上可见,但在具有自定义 bridge0 的其他主机上不可见

标签 networking docker ipv6 iptables connection-timeout

在 RHEL 7.1 上安装新的 Docker 1.7.0

因此,我安装了最新的 Docker 1.7.0,但我无法让这台新服务器向外界提供服务。

[root@pppdc9prd8ok eea.docker.jenkins]# uname -a
Linux pppdc9prd8ok 3.10.0-229.4.2.el7.x86_64 #1 SMP Fri Apr 24 15:26:38 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

[root@pppdc9prd8ok eea.docker.jenkins]# docker --version
Docker version 1.7.0, build 0baf609
[root@pppdc9prd8ok eea.docker.jenkins]# docker info
Containers: 10
Images: 110
Storage Driver: devicemapper
 Pool Name: docker-253:0-4374531-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: extfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 4.398 GB
 Data Space Total: 107.4 GB
 Data Space Available: 99.18 GB
 Metadata Space Used: 7.029 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.14 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Data loop file: /app_local/var-lib-docker/devicemapper/devicemapper/data
 Metadata loop file: /app_local/var-lib-docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93-RHEL7 (2015-01-28)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.0-229.4.2.el7.x86_64
Operating System: Red Hat Enterprise Linux
CPUs: 4
Total Memory: 15.52 GiB
Name: pppdc9prd8ok
ID: 3M2F:QYY7:Z5DI:YTVI:RAV4:SHPM:C3RC:CWIY:FHFA:ZYAS:SNHG:CMTY

使用 bridge0 而不是 docker0 设置 Docker

我关注了 Docker 文档中的高级网络主题到 change my default docker bridge from docker0 to bridge0由于与我们的内部网络发生冲突。

我使用 docker-compose.yml 启动了一个在端口 80 上运行的 docker 容器,如下所示:
[root@pppdc9prd8ok eea.docker.jenkins]# docker ps
CONTAINER ID        IMAGE                   COMMAND                CREATED             STATUS              PORTS                                            NAMES
a9f5637552ba        eeacms/jenkins:master   "/usr/local/bin/jenk   4 seconds ago       Up 4 seconds        0.0.0.0:50000->50000/tcp, 0.0.0.0:80->8080/tcp   eeadockerjenkins_master_1
c6fcac33b044        yorkshirekev/postfix    "/bin/bash -c '/star   7 seconds ago       Up 6 seconds                                                         eeadockerjenkins_postfix_1
199ad3d48dfe        eeacms/jenkins:slave    "/bin/sh -c /bin/jen   5 minutes ago       Up 47 seconds                                                        eeadockerjenkins_worker_1
3a8057253b7d        eeacms/jenkins:slave    "/bin/sh -c /bin/jen   5 minutes ago       Up 47 seconds                                                        eeadockerjenkins_worker_2
fced8be92258        eeacms/jenkins:slave    "/bin/sh -c /bin/jen   5 minutes ago       Up 46 seconds                                                        eeadockerjenkins_worker_3
7cb4cfabd3c2        mongo                   "/entrypoint.sh mong   2 weeks ago         Up 20 seconds       0.0.0.0:27017->27017/tcp                         mongodb-dotci

无法从 Internet 访问端口 80 上的服务器

从“ps”打印的内容来看,80 端口上运行的服务非常好,绑定(bind)到 80 端口上的所有 ip 地址 0.0.0.0。但是,试图从另一个位置访问这台机器我无法。
Marcello-New2015:~ mdesales$ curl http://docker.corp.intuit.net/
curl: (7) Failed to connect to docker.corp.intuit.net port 80: Operation timed out

有来自本地主机的内部路由

很奇怪,因为我已经检查过容器是否可以从内部访问并且它正在工作。
[root@pppdc9prd8ok eea.docker.jenkins]# curl localhost | grep html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 26791  100 26791    0     0   110k      0 --:    <!DOCTYPE html><html><head resURL="/static/9ebca566">
--:-- --:--:-- --:--:--  110k

Netstat 还显示它绑定(bind)到 ipv6。

我猜 RHEL 7.1 已经配置了开箱即用的 ipv6,因为我在安装过程中没有设置它。无论如何,这向我展示了......我正在挖掘,ipv6的:::* 与 ipv4 的 0.0.0.0 相同。
[root@pppdc9prd8ok eea.docker.jenkins]#  netstat -tulnp | grep docker
tcp6       0      0 :::27017                :::*                    LISTEN      18271/docker-proxy
tcp6       0      0 :::80                   :::*                    LISTEN      18498/docker-proxy
tcp6       0      0 :::50000                :::*                    LISTEN      18490/docker-proxy

iptables 显示将调用正确转发到接口(interface)的规则。

ifconfig 和 iptables 的所有接口(interface)都正确显示
[root@pppdc9prd8ok eea.docker.jenkins]# ifconfig
bridge0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.5.1  netmask 255.255.252.0  broadcast 192.168.7.255
        ether 1e:dd:74:96:b1:c5  txqueuelen 0  (Ethernet)
        RX packets 10551  bytes 10704512 (10.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 9986  bytes 10375991 (9.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.42.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 00:00:00:00:00:00  txqueuelen 0  (Ethernet)
        RX packets 54772  bytes 61032436 (58.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 53436  bytes 61653718 (58.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.132.52.146  netmask 255.255.252.0  broadcast 10.132.55.255
        ether 00:50:56:01:0e:ba  txqueuelen 1000  (Ethernet)
        RX packets 117543  bytes 12322742 (11.7 MiB)
        RX errors 0  dropped 626  overruns 0  frame 0
        TX packets 21044  bytes 3662343 (3.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ip 表有以下规则:
[root@pppdc9prd8ok eea.docker.jenkins]# iptables -t nat -nxvL
Chain PREROUTING (policy ACCEPT 82 packets, 10381 bytes)
    pkts      bytes target     prot opt in     out     source               destination
      23     1412 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 52 packets, 6951 bytes)
    pkts      bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 330 packets, 29005 bytes)
    pkts      bytes target     prot opt in     out     source               destination
       0        0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 330 packets, 29005 bytes)
    pkts      bytes target     prot opt in     out     source               destination
      21     1548 MASQUERADE  all  --  *      !bridge0  192.168.4.0/22       0.0.0.0/0
      15     1028 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0
       0        0 MASQUERADE  tcp  --  *      *       192.168.4.5          192.168.4.5          tcp dpt:27017
       0        0 MASQUERADE  tcp  --  *      *       192.168.4.8          192.168.4.8          tcp dpt:50000
       0        0 MASQUERADE  tcp  --  *      *       192.168.4.8          192.168.4.8          tcp dpt:8080

Chain DOCKER (2 references)
    pkts      bytes target     prot opt in     out     source               destination
       0        0 DNAT       tcp  --  !bridge0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:27017 to:192.168.4.5:27017
       0        0 DNAT       tcp  --  !bridge0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:50000 to:192.168.4.8:50000
       8      512 DNAT       tcp  --  !bridge0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 to:192.168.4.8:8080

不知道去哪里:(帮助...

最佳答案

简短的回答:安装“bridge0”时删除“docker0”网桥!

好的,所以越来越多地挖掘我发现docker0的存在以某种方式干扰了网络......

长答案:逐步验证

我首先验证了 bridge0 实际上正在使用中。它不是!

[root@pppdc9prd8ok eea.docker.jenkins]# systemctl status docker
docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─http-proxy.conf
   Active: active (running) since Fri 2015-07-10 07:23:14 UTC; 30min ago
     Docs: https://docs.docker.com
 Main PID: 18034 (docker)
   CGroup: /system.slice/docker.service
           ├─18034 /usr/bin/docker -d -H fd://
           ├─18271 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 27017 -container-ip 192.168.4.5 -container-port 27017
           ├─18490 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 50000 -container-ip 192.168.4.8 -container-port 50000
           └─18498 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 192.168.4.8 -container-port 8080

Jul 10 07:23:57 pppdc9prd8ok docker[18034]: time="2015-07-10T07:23:57.124143415Z" level=info msg="GET /v1.18/containers/json?all=0&limit=-1&trunc_cmd=0&filters=%7B%22label%22%3A+%5...5D%7D&size=0"
Jul 10 07:23:57 pppdc9prd8ok docker[18034]: time="2015-07-10T07:23:57.126520912Z" level=info msg="GET /v1.18/containers/c6fcac33b04480970aa3606f86e5ed9571a320b6ff5cdc8ecdf81edfb416720a/json"
Jul 10 07:23:57 pppdc9prd8ok docker[18034]: time="2015-07-10T07:23:57.128362232Z" level=info msg="GET /v1.18/containers/json?all=1&limit=-1&trunc_cmd=0&filters=%7B%22label%22%3A+%5...5D%7D&size=0"
Jul 10 07:23:57 pppdc9prd8ok docker[18034]: time="2015-07-10T07:23:57.130940471Z" level=info msg="POST /v1.18/containers/create?name=eeadockerjenkins_master_1"
Jul 10 07:23:57 pppdc9prd8ok docker[18034]: time="2015-07-10T07:23:57.299140678Z" level=info msg="GET /v1.18/containers/a9f5637552bad2d608f838cdb2a263452f5e98962c45ebe759ed0904211d6962/json"
Jul 10 07:23:57 pppdc9prd8ok docker[18034]: time="2015-07-10T07:23:57.301413002Z" level=info msg="POST /v1.18/containers/a9f5637552bad2d608f838cdb2a263452f5e98962c45ebe759ed0904211d6962/start"
Jul 10 07:23:57 pppdc9prd8ok docker[18034]: time="2015-07-10T07:23:57.504799799Z" level=info msg="DELETE /v1.18/containers/0665b35b4f1df8e8d098a429ae4a057a91c36cc341d33f710b00cc3c4...alse&v=False"
Jul 10 07:23:58 pppdc9prd8ok docker[18034]: time="2015-07-10T07:23:58.657884948Z" level=info msg="GET /v1.18/containers/json?all=0&limit=-1&trunc_cmd=0&filters=%7B%22label%22%3A+%5...5D%7D&size=0"
Jul 10 07:24:01 pppdc9prd8ok docker[18034]: time="2015-07-10T07:24:01.793020916Z" level=info msg="GET /v1.19/containers/json"
Jul 10 07:43:25 pppdc9prd8ok docker[18034]: time="2015-07-10T07:43:25.850272360Z" level=info msg="GET /v1.19/info"
Hint: Some lines were ellipsized, use -l to show in full.

事实证明,RHEL 7.1 使用服务 Upstart 安装 Docker,而没有指向环境变量。
[root@pppdc9prd8ok eea.docker.jenkins]# cat /etc/sysconfig/docker
# /etc/sysconfig/docker
#
# Other arguments to pass to the docker daemon process
# These will be parsed by the sysv initscript and appended
# to the arguments list passed to docker -d

other_args="-b=bridge0"

我必须添加行 环境文件=-/etc/sysconfig/docker 在以下文件中并将环境变量添加到“docker -d”命令中:
[root@pppdc9prd8ok eea.docker.jenkins]# cat /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket

[Service]
EnvironmentFile=-/etc/sysconfig/docker
ExecStart=/usr/bin/docker -d $other_args -H fd://
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity

[Install]
WantedBy=multi-user.target

重新启动 docker 服务现在会在系统中显示 docker0 参数。
[root@pppdc9prd8ok eea.docker.jenkins]# systemctl status docker
docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─http-proxy.conf
   Active: active (running) since Fri 2015-07-10 07:23:14 UTC; 30min ago
     Docs: https://docs.docker.com
 Main PID: 18034 (docker)
   CGroup: /system.slice/docker.service
           ├─18034 /usr/bin/docker -d -b=bridge0 -H fd://
           ├─18271 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 27017 -container-ip 192.168.4.5 -container-port 27017
           ├─18490 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 50000 -container-ip 192.168.4.8 -container-port 50000
           └─18498 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 192.168.4.8 -container-port 8080

但是,该服务仍然无法正常工作......我检查并让它工作的最后一件事是删除网桥“docker0”。它成功了!
[root@pppdc9prd8ok eea.docker.jenkins]# ip link set docker0 down
[root@pppdc9prd8ok eea.docker.jenkins]# brctl delbr docker0
[root@pppdc9prd8ok eea.docker.jenkins]# ifconfig
bridge0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.5.1  netmask 255.255.252.0  broadcast 192.168.7.255
        ether 16:1b:b8:42:5c:9e  txqueuelen 0  (Ethernet)
        RX packets 6550  bytes 6542448 (6.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6133  bytes 6585941 (6.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.132.52.146  netmask 255.255.252.0  broadcast 10.132.55.255
        ether 00:50:56:01:0e:ba  txqueuelen 1000  (Ethernet)
        RX packets 114644  bytes 11944039 (11.3 MiB)
        RX errors 0  dropped 626  overruns 0  frame 0
        TX packets 19671  bytes 2808015 (2.6 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

从另一台主机测试它现在可以正常工作了!
Marcello-New2015:~ mdesales$ curl http://docker.corp.intuit.net/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 26804  100 26804    0     0  60458      0 --:--:-- --:--:-- --:--:-- 60505

关于networking - Docker 容器在 localhost 上可见,但在具有自定义 bridge0 的其他主机上不可见,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/31335347/

相关文章:

android - 替代 NetworkInterface.getHardwareAddress()?

java - 在单台计算机上模拟 p2p 网络流量

python - 如何使用带有 Tensorflow 的 Docker 在 Mac 终端上运行 Python 脚本?

c++ - 使用普通套接字的 IPv6 多播有哪些要求?

linux - Python 3 - 无法接收 IPv6 数据包(UDP - linux)

java - SocketChannel : Why if I write msgs quickly the latency of each message is low, 但是当我每 30 秒写一个 msg 时延迟很高?

networking - Arduino 射频传感器网络

docker - 如何使用 docker 和 deepstream.io 设置项目?

Docker Compose 作为 CI 管道

java - Java 中的 ipv6 套接字权限被拒绝