python - Poetry 如何处理二进制依赖项? (尤其是 NumPy )

标签 python numpy anaconda conda python-poetry

到目前为止,我一直使用 conda 作为虚拟环境和依赖项管理。但是,将我的 environment.yml 文件从我的开发机器传输到生产服务器时,有些东西无法按预期工作。现在,我想研究替代方案。诗歌看起来不错,尤其是因为

poetry also maintains a lock file, and it has a benefit over pipenv because it keeps track of which packages are subdependencies. (https://realpython.com/effective-python-environment/#poetry)


这可能会大大提高稳定性。但是,我正在处理 science-heavy projects (矩阵、数据科学、机器学习),所以在实践中我需要 scipy 堆栈(例如 numpy、pandas、scitkit-learn)。

Python became too slow for some pure computational workloads so numpy and scipy were born. [...] They are written in C and just wrapped as a python library.

Compiling such libraries brings a set of challenges since they (more or less) have to be compiled on your machine for maximum performance and proper linking with libraries like glibc.

Conda was introduced as an all-in-one solution to manage python environments for the scientific community.

[...] Instead of using a fragile process of compiling libraries on your machine, libraries are precompiled and just downloaded when you request them. Unfortunately, the solution comes with a caveat - conda does not use PyPI, the most popular index of python packages.

(https://modelpredict.com/python-dependency-management-tools#fnref:conda-compiling-challenges)


据我所知,这甚至不符合 Conda 的标准,因为它做了相当多的优化以充分利用我的 CPU/GPU/numpy 架构。 ( https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/#Myth-#6:-Now-that-pip-uses-wheels,-conda-is-no-longer-necessary )
https://numpy.org/install/本身建议使用 conda,但也说可以通过 pip 安装(而诗歌使用 pypi)

For users who know, from personal preference or reading about the main differences between conda and pip below, they prefer a pip/PyPI-based solution, we recommend:

[...] Use Poetry as the most well-maintained tool that provides a dependency resolver and environment management capabilities in a similar fashion as conda does.


我想获得诗歌设置的稳定性和 conda 设置的速度。
诗歌如何处理二进制依赖?
它是否也像 conda 一样考虑我的硬件?

如果诗歌在这方面没有提供,我可以将它与 conda 结合起来吗?

最佳答案

numpy提供多个wheel不同操作系统、cpu 架构和 python 版本的文件。 wheel包是预编译的,所以目标系统不必编译包。poetry能够根据您的系统为您选择合适的车轮。
说到这里,我建议使用 poetry ,只要你只需要python包,pypi也有。一旦您需要其他非 Python 工具,请坚持使用 conda。 (免责声明:我是 poetry 的维护者之一)。
还相关:https://github.com/python-poetry/poetry/issues/1904

关于python - Poetry 如何处理二进制依赖项? (尤其是 NumPy ),我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/62513005/

相关文章:

python - 使用 numpy 将矩阵乘以点数组?

python - 创建两个 numpy.ndarray 的字典?

python - numpy数组中小于(<)运算符的作用是什么?

python - 使用 python 从 javascript 文本中提取 url

python - 如何在 Windows 64 位上安装 ImageMagick 和 Anaconda?

python - cv2.VideoCapture错误

python - 用于医学成像的 TensorFlow tf.data.Dataset API

python - 删除最后打印的字符python

javascript - 将图像从base64转换为图像并保存在django的数据库中

python - 无法导入matplotlib.pyplot,与字体相关的IOError