python - 在当前进程完成其引导阶段之前尝试启动一个新进程

Question

我是 dask 的新手，我发现拥有一个可以轻松实现并行化的模块真是太好了。我正在做一个项目，我能够在一台机器上并行化一个循环为 you can see here .但是，我想转到 dask.distributed .我对上面的类应用了以下更改:

diff --git a/mlchem/fingerprints/gaussian.py b/mlchem/fingerprints/gaussian.py
index ce6a72b..89f8638 100644
--- a/mlchem/fingerprints/gaussian.py
+++ b/mlchem/fingerprints/gaussian.py
@@ -6,7 +6,7 @@ from sklearn.externals import joblib
 from .cutoff import Cosine
 from collections import OrderedDict
 import dask
-import dask.multiprocessing
+from dask.distributed import Client
 import time


@@ -141,13 +141,14 @@ class Gaussian(object):
         for image in images.items():
             computations.append(self.fingerprints_per_image(image))

+        client = Client()
         if self.scaler is None:
-            feature_space = dask.compute(*computations, scheduler='processes',
+            feature_space = dask.compute(*computations, scheduler='distributed',
                                          num_workers=self.cores)
             feature_space = OrderedDict(feature_space)
         else:
             stacked_features = dask.compute(*computations,
-                                            scheduler='processes',
+                                            scheduler='distributed',
                                             num_workers=self.cores)

             stacked_features = numpy.array(stacked_features)

这样做会产生此错误:

 File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py", line 136, in _check_not_importing_main
    is not going to be frozen to produce an executable.''')
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

我尝试了不同的添加方式 if __name__ == '__main__':没有任何成功。这可以是 reproduced by running this example .如果有人可以帮助我解决这个问题，我将不胜感激。我不知道应该如何更改代码以使其正常工作。

谢谢。

编辑:示例是 cu_training.py .

Answer 1

Client命令启动新进程，因此它必须在 if __name__ == '__main__': 内 block 如本 SO question 中所述或者这个 GitHub issue

这与多处理模块相同