我的DevOps技能较弱，对Docker的了解也不多。我需要和Ceph成为Java friend 。首先，我正在尝试在我的Docker容器中运行CEPH。我查看了Intellij Idea，并了解到并非所有容器都在运行。我的 docker 组成看起来像这样:
version: '2.1'
services:
mon1:
image: ceph/daemon:${CEPH_CONTAINER_VERSION}
command: "mon"
environment:
MON_IP: ${MON1_IP}
CEPH_PUBLIC_NETWORK: ${MON1_CEPH_PUBLIC_NETWORK}
volumes:
- ${VOLUMES_PATH}/ceph:/etc/ceph
- ${VOLUMES_PATH}/lib/:/var/lib/ceph/
networks:
- ceph-cluster-net
mgr:
image: ceph/daemon:${CEPH_CONTAINER_VERSION}
command: "mgr"
volumes:
- ${VOLUMES_PATH}/ceph:/etc/ceph
- ${VOLUMES_PATH}/lib/:/var/lib/ceph/
depends_on:
- mon1
networks:
- ceph-cluster-net
ports:
- ${DASHBOARD_PORT}:7000
expose:
- ${DASHBOARD_PORT}
osd1:
pid: host
privileged: true
image: ceph/daemon:${CEPH_CONTAINER_VERSION}
command: "osd"
volumes:
- ${VOLUMES_PATH}/ceph:/etc/ceph
- ${VOLUMES_PATH}/lib/:/var/lib/ceph/
- /dev/:/dev/
environment:
OSD_DEVICE: ${OSD1_DEVICE}
OSD_TYPE: disk
OSD_FORCE_ZAP: 1
depends_on:
- mon1
networks:
- ceph-cluster-net
osd2:
pid: host
privileged: true
image: ceph/daemon:${CEPH_CONTAINER_VERSION}
command: "osd"
volumes:
- ${VOLUMES_PATH}/ceph:/etc/ceph
- ${VOLUMES_PATH}/lib/:/var/lib/ceph/
- /dev/:/dev/
environment:
OSD_DEVICE: ${OSD2_DEVICE}
OSD_TYPE: disk
OSD_FORCE_ZAP: 1
depends_on:
- mon1
networks:
- ceph-cluster-net
osd3:
pid: host
privileged: true
image: ceph/daemon:${CEPH_CONTAINER_VERSION}
command: "osd"
volumes:
- ${VOLUMES_PATH}/ceph:/etc/ceph
- ${VOLUMES_PATH}/lib/:/var/lib/ceph/
- /dev/:/dev/
environment:
OSD_DEVICE: ${OSD3_DEVICE}
OSD_TYPE: disk
OSD_FORCE_ZAP: 1
depends_on:
- mon1
networks:
- ceph-cluster-net
rgw1:
image: ceph/daemon:${CEPH_CONTAINER_VERSION}
command: "rgw"
volumes:
- ${VOLUMES_PATH}/ceph:/etc/ceph
- ${VOLUMES_PATH}/lib/:/var/lib/ceph/
depends_on:
- osd1
- osd2
- osd3
networks:
- ceph-cluster-net
ports:
- ${RGW_PORT}:8080
mds1:
image: ceph/daemon:${CEPH_CONTAINER_VERSION}
command: "mds"
environment:
CEPHFS_CREATE: 1
volumes:
- ${VOLUMES_PATH}/ceph:/etc/ceph
- ${VOLUMES_PATH}/lib/:/var/lib/ceph/
depends_on:
- osd1
- osd2
- osd3
networks:
- ceph-cluster-net
networks:
ceph-cluster-net:
driver: bridge
external: true
我有一个env-example文件:
CEPH_CONTAINER_VERSION=v3.1.0-stable-3.1-luminous-ubuntu-16.04-x86_64
VOLUMES_PATH=~/volumes_data/ceph
DASHBOARD_PORT=6000
RGW_PORT=6080
OSD1_DEVICE=/dev/sdb
OSD2_DEVICE=/dev/sdc
OSD3_DEVICE=/dev/sdd
MON1_IP=172.18.0.2
MON1_CEPH_PUBLIC_NETWORK=172.18.0.0/24
在运行docker-compose之前，我运行以下命令:
docker network create --driver bridge ceph-cluster-net
cp env-example .env
docker-compose up -d
未运行图像的日志如下所示:
MON1:
importing contents of /var/lib/ceph/bootstrap-osd/ceph.keyring into /etc/ceph/ceph.mon.keyring
importing contents of /var/lib/ceph/bootstrap-mds/ceph.keyring into /etc/ceph/ceph.mon.keyring
importing contents of /var/lib/ceph/bootstrap-rgw/ceph.keyring into /etc/ceph/ceph.mon.keyring
importing contents of /var/lib/ceph/bootstrap-rbd/ceph.keyring into /etc/ceph/ceph.mon.keyring
importing contents of /etc/ceph/ceph.client.admin.keyring into /etc/ceph/ceph.mon.keyring
2019-12-24 12:18:53.293858 7ff156f2a080 -1 unable to find any IP address in networks '172.18.0.0/24' interfaces ''
MGR:
2019-12-24 12:18:54 /entrypoint.sh: static: does not generate config
2019-12-24 12:18:54 /entrypoint.sh: static: does not generate the admin key, so we can not get it.
2019-12-24 12:18:54 /entrypoint.sh: static: make it available with the help of your configuration management system.
2019-12-24 12:18:54 /entrypoint.sh: static: ceph-ansible is a good candidate to deploy a containerized version of Ceph.
2019-12-24 12:18:54 /entrypoint.sh: static: ceph-ansible will help you fetching the keys and push them on the right nodes.
2019-12-24 12:18:54 /entrypoint.sh: static: if you're interested, please visit: https://github.com/ceph/ceph-ansible
2019-12-24 12:23:54.568294 7f3373429700 0 monclient(hunting): authenticate timed out after 300
2019-12-24 12:23:54.568338 7f3373429700 0 librados: client.admin authentication error (110) Connection timed out
[errno 110] error connecting to the cluster
OSD1:
2019-12-24 12:18:50 /entrypoint.sh: static: does not generate config
2019-12-24 12:19:00 /entrypoint.sh: Timed out while trying to reach out to the Ceph Monitor(s).
2019-12-24 12:19:00 /entrypoint.sh: Make sure your Ceph monitors are up and running in quorum.
2019-12-24 12:19:00 /entrypoint.sh: Also verify the validity of client.bootstrap-osd keyring.
OSD2和OSD3:
2019-12-24 12:10:44 /entrypoint.sh: static: does not generate config
2019-12-24 12:10:44 /entrypoint.sh: ERROR- The device pointed by OSD_DEVICE (/dev/sdc) doesn't exist !
2019-12-24 12:18:48 /entrypoint.sh: static: does not generate config
2019-12-24 12:18:48 /entrypoint.sh: ERROR- The device pointed by OSD_DEVICE (/dev/sdc) doesn't exist !
RGW1:
2019-12-24 12:18:52 /entrypoint.sh: static: does not generate config
2019-12-24 12:19:02 /entrypoint.sh: Timed out while trying to reach out to the Ceph Monitor(s).
2019-12-24 12:19:02 /entrypoint.sh: Make sure your Ceph monitors are up and running in quorum.
2019-12-24 12:19:02 /entrypoint.sh: Also verify the validity of client.bootstrap-rgw keyring.
MDS1:
2019-12-24 12:18:53 /entrypoint.sh: static: does not generate config
创建docker-compose时，我使用了这篇文章:https://github.com/VasiliyLiao/ceph-docker-compose
最佳答案
看日志
unable to find any IP address in networks '172.18.0.0/24' interfaces ''
我将从检查网络开始。甚至更好，如果您不擅长使用devops，请尝试ceph-ansible并进行少量更改，即可准备好带有docker的ceph。
关于docker - 在Docker中运行CEPH，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/59468904/