hadoop - 在OpenStack Sahara中生成Hadoop集群时的“No valid host was found”

Question

我想使用OpenStack Sahara创建Hadoop集群。我有一个 Controller 节点和一个计算节点，它们位于不同的物理计算机上，并具有以下IP:

Controller :172.18.xxx.x1；计算:172.18.xxx.x2

首先，我创建一个具有 float IP的网络:

neutron net-create ext-net --router:external=True
neutron subnet-create ext-net --allocation-pool start=192.168.0.2,end=192.168.0.254  --enable_dhcp=False 192.168.0.0/24
neutron floatingip-create ext-net

然后创建节点组模板和集群模板:

openstack dataprocessing node group template create     --name vanilla-default-master-floating --plugin vanilla     --version 2.7.1 --processes namenode resourcemanager     --flavor 2 --auto-security-group --floating-ip-pool c7c879b9-5695-43cc-88b0-9dce957497dc
openstack dataprocessing node group template create     --name vanilla-default-worker-floating --plugin vanilla     --version 2.7.1 --processes datanode nodemanager     --flavor 2 --auto-security-group  --floating-ip-pool c7c879b9-5695-43cc-88b0-9dce957497dc
openstack dataprocessing cluster template create     --name vanilla-default-cluster-floating     --node-groups vanilla-default-master-floating:1 vanilla-default-worker-floating:3

最后，我尝试启动集群:

openstack dataprocessing cluster create --name my-cluster-1 --cluster-template vanilla-default-cluster-floating --user-keypair key --neutron-network ext-net --image sahara-mitaka-vanilla-hadoop

那是sahara-engine.log的日志

2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [req-9cd4a763-48aa-4123-96ae-5ae9aa626d85 136b5723b10d4e89afabb86adca1ed89 598f0ed79f6d461589df0124123eaf16 - - -] [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf] Error during operating on cluster (reason: Heat stack failed with status Resource CREATE failed: ResourceInError: resources.vanilla-default-master-floating.resources[0].resources.inst: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
Error ID: 9e87d53c-e2bd-4e34-b5a7-89048c7a1ba0)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf] Traceback (most recent call last):
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/sahara/service/ops.py", line 192, in wrapper
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     f(cluster_id, *args, **kwds)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/sahara/service/ops.py", line 285, in _provision_cluster
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     INFRA.create_cluster(cluster)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/sahara/service/heat/heat_engine.py", line 63, in create_cluster
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     self._launch_instances(cluster, target_count, CREATE_STAGES)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/sahara/service/heat/heat_engine.py", line 224, in _launch_instances
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     cluster, target_count, update_stack, disable_rollback)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/sahara/utils/cluster_progress_ops.py", line 139, in handler
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     add_fail_event(instance, e)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     self.force_reraise()
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     six.reraise(self.type_, self.value, self.tb)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/sahara/utils/cluster_progress_ops.py", line 136, in handler
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     value = func(*args, **kwargs)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/sahara/service/heat/heat_engine.py", line 215, in _create_instances
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     last_updated_time=stack.last_updated_time)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]   File "/usr/lib/python2.7/site-packages/sahara/utils/openstack/heat.py", line 112, in wait_stack_completion
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf]     raise ex.HeatStackException(stack.stack_status_reason)
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf] HeatStackException: Heat stack failed with status Resource CREATE failed: ResourceInError: resources.vanilla-default-master-floating.resources[0].resources.inst: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf] Error ID: 9e87d53c-e2bd-4e34-b5a7-89048c7a1ba0
2016-06-23 16:34:49.224 15603 ERROR sahara.service.ops [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf] 
2016-06-23 16:35:06.997 15603 WARNING sahara.service.heat.heat_engine [req-9cd4a763-48aa-4123-96ae-5ae9aa626d85 136b5723b10d4e89afabb86adca1ed89 598f0ed79f6d461589df0124123eaf16 - - -] [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf] Cluster creation rollback (reason: Heat stack failed with status Resource CREATE failed: ResourceInError: resources.vanilla-default-master-floating.resources[0].resources.inst: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
Error ID: 9e87d53c-e2bd-4e34-b5a7-89048c7a1ba0)
2016-06-23 16:35:07.420 15603 INFO sahara.utils.cluster [req-9cd4a763-48aa-4123-96ae-5ae9aa626d85 136b5723b10d4e89afabb86adca1ed89 598f0ed79f6d461589df0124123eaf16 - - -] [instance: none, cluster: 9ec5f346-1321-4b75-a96a-e3f84bd28acf] Cluster status has been changed. New status=Error

因此，我的猜测是我的虚拟网络和子网的创建是错误的，或者我在那里缺少某些东西。

如果这有帮助，那就是openstack-status的输出:

== Glance services ==
openstack-glance-api:                   active
openstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     active
== Horizon service ==
openstack-dashboard:                    404
== neutron services ==
neutron-server:                         active    (disabled on boot)
== Cinder services ==
openstack-cinder-api:                   active
openstack-cinder-scheduler:             active
openstack-cinder-volume:                inactive  (disabled on boot)
== Sahara services ==
openstack-sahara-api:                   active
openstack-sahara-engine:                active
== Support services ==
mariadb:                                inactive  (disabled on boot)
openvswitch:                            active
dbus:                                   active
rabbitmq-server:                        active
memcached:                              active
== Keystone users ==
/usr/lib/python2.7/site-packages/keystoneclient/shell.py:64: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient.
  'python-keystoneclient.', DeprecationWarning)
WARNING: unsupported identity-api-version 3, falling back to 2.0
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning: Constructing an instance of the keystoneclient.v2_0.client.Client class without a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
  'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning: Using the 'tenant_name' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_name' argument instead
  super(Client, self).__init__(**kwargs)
/usr/lib/python2.7/site-packages/debtcollector/renames.py:45: DeprecationWarning: Using the 'tenant_id' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_id' argument instead
  return f(*args, **kwargs)
/usr/lib/python2.7/site-packages/keystoneclient/httpclient.py:371: DeprecationWarning: Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
  'the 2.0.0 release.', DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/session.py:140: DeprecationWarning: keystoneclient.session.Session is deprecated as of the 2.1.0 release in favor of keystoneauth1.session.Session. It will be removed in future releases.
  DeprecationWarning)
/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py:56: DeprecationWarning: keystoneclient auth plugins are deprecated as of the 2.1.0 release in favor of keystoneauth1 plugins. They will be removed in future releases.
  'in future releases.', DeprecationWarning)
Authorization Failed: The resource could not be found. (HTTP 404) (Request-ID: req-2b286118-c025-4359-a7a4-d5a25f505070)
== Glance images ==
+--------------------------------------+------------------------------+
| ID                                   | Name                         |
+--------------------------------------+------------------------------+
| c202991b-78b3-4fcd-bffa-9590c3208e69 | cirros                       |
| d02adcdb-b202-4147-a88a-cea51ce438b1 | sahara-mitaka-vanilla-hadoop |
+--------------------------------------+------------------------------+

那是openstack user list的输出:

+----------------------------------+-------------------+
| ID                               | Name              |
+----------------------------------+-------------------+
| 0bf4d4c8089c4991abd777dc57729d13 | glance            |
| 136b5723b10d4e89afabb86adca1ed89 | admin             |
| 674fbea90b7b411d82e1785dcbf9052b | sahara            |
| 82d455369bb14007a5c12eea7c38654d | cinder            |
| 920243b74273437a93632012cdc1c45e | neutron           |
| d0823e2eb5844d80822b8684437f2459 | heat_domain_admin |
| e93399b678074c9abe6b42f6b9fec6f5 | heat              |
| f338a775ab5c43f09adac18e807f257b | nova              |
| f75772b9af5c42338cbff05c13202c22 | demo              |
+----------------------------------+-------------------+

顺便说一句:我正在openSUSE 42.1上运行OpenStack Mikita。

Answer 1

“未找到有效主机”表示您没有足够的资源。

例如，Sahara将尝试在只有3GO的计算节点中创建具有4GO RAM的实例。