python - 使用 pyenv 和 virtualenv 在 apache 后面部署 Flask 应用程序时找不到 numpy、pandas 模块

Question

我为一个相当详细的应用程序构建了一个 Flask 前端，经过大量编码后它在本地完美运行。所以现在我想让它对公众可见，所以已经读到它需要一个 apache 或 nginx 前端。我计算出了所需实例的大小，并在 AWS 上将其打开，然后运行了 these两个tutorials这证明基本情况是有效的。 Apache 设置完成后，当我导航到 AWS 实例的 IP 时，会显示 Apache 欢迎页面。然后，当我暂时用 Flask 应用程序替换“Hello World”代码并使用 IP 刷新浏览器时，就会呈现出 hello world。另外，如果我替换here中的flasskapp.py代码它完美地跨apache服务到在浏览器中输入pip找到的模块列表的IP，包括Numpy和Pandas，否则它会提示找不到!然而，通过在顶部添加 import numpy 可以轻松杀死该脚本，但它会给出以下错误消息:

ImportError: cannot import name 'multiarray'

和

Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` 
(removes all files not under version control).  
Otherwise reinstall numpy.

都在 apache 日志中。

当我放回我的应用程序而不是 hello world 时，就会出现问题。它只是拒绝导入 pandas 这是 Apache 日志的最后一部分:

Mon Jul 03 19:42:14.691081 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560] Traceback (most recent call last):
[Mon Jul 03 19:42:14.691126 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/var/www/html/flaskapp/flaskapp.wsgi", line 30, in <module>
[Mon Jul 03 19:42:14.691130 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     from similar_items_frontend import app as application
[Mon Jul 03 19:42:14.691137 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/var/www/html/flaskapp/similar_items_frontend.py", line 4, in <module>
[Mon Jul 03 19:42:14.691140 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     from item_similar_to_doc import similar_to_doc
[Mon Jul 03 19:42:14.691145 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/home/ubuntu/find-similar/frontend/../item_similar_to_doc.py", line 1, in <module>
[Mon Jul 03 19:42:14.691148 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     from PullVectors import p
[Mon Jul 03 19:42:14.691153 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/home/ubuntu/find-similar/frontend/../RunSimilarity.py", line 5, in <module>
[Mon Jul 03 19:42:14.691156 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     import pandas as pd
[Mon Jul 03 19:42:14.691161 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]   File "/home/ubuntu/.pyenv/versions/miniconda3-latest/envs/find_similarProject/lib/python3.6/site-packages/pandas/__init__.py", line 19, in <module>
[Mon Jul 03 19:42:14.691164 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560]     "Missing required dependencies {0}".format(missing_dependencies))
[Mon Jul 03 19:42:14.691180 2017] [wsgi:error] [pid 12300:tid 140131159766784] [client 93.21.05.132:52560] ImportError: Missing required dependencies ['numpy']

我觉得我已经尝试了一切，经历了很多痛苦的时间，拉扯头发的时间..

这是000-default.conf:

WSGIPythonPath /home/ubuntu/.pyenv/versions/miniconda3-latest/envs/vtenv4YTproject/lib/python3.6/site-packages/

<VirtualHost *:80>
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        WSGIDaemonProcess flaskapp threads=5
        WSGIScriptAlias / /var/www/html/flaskapp/flaskapp.wsgi

        <Directory flaskapp>
              WSGIScriptReloading On
              WSGIProcessGroup %{GLOBAL}
              WSGIApplicationGroup %{GLOBAL}
              Order allow,deny
              Allow from all
        </Directory>



        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

我应该提到，当我通过 ssh 进入 AWS 实例时，我的 Flask 应用程序在其 virtualenv 中完美运行(达到了 Flask 提供页面服务的程度)，当然需要进行一些修补才能使其正常工作，但它找到它的所有依赖项，不会在 pandas 导入时崩溃。

为了尝试修复它，我查看了 here , here ，添加了来自 here 的一些代码，浏览了大量文档 here ，我的flaskapp.wsgi 文件如下所示:

import sys
import logging
import os.path
sys.path.append(os.path.join(os.path.dirname(os.path.realpath(__file__)), os.pardir))
import config
import site

prev_sys_path = list(sys.path) 

site.addsitedir('/home/ubuntu/.pyenv/versions/miniconda3-latest/envs/find_similarProject/lib/python3.6/site-packages')

new_sys_path = []

for item in list(sys.path): 
    if item not in prev_sys_path:
        new_sys_path.append(item) 
        sys.path.remove(item)
sys.path[:0] = new_sys_path 

logging.basicConfig(stream=sys.stderr)
sys.path.insert(0, "/var/www/html/flaskapp")

from similar_vids_frontend import app as application
application.secret_key = config.WSGI_KEY

我没有理由可以找到为什么 pandas/numpy 导入在 apache 中失败，但在 virtualenv 中运行时工作正常。我想知道是否与权限和组有关，根据文档:

Do be aware that the user that Apache runs your code as will need to be able to access the Python virtual environment. On some Linux distributions, the home directory of a user account is not accessible to other users. Rather than change the permissions on your home directory, it might be better to consider locating your WSGI application code and any Python virtual environment outside of your home directory.

但是 Ubuntu 是那些发行版之一吗？那么，将虚拟环境和项目安装到 / 文件夹中呢？

感谢任何提示或解决方案。

Answer 1

这现在有效，主要归功于格雷厄姆的食谱here还有this answer关于将模块加载到 apache 中。它有效的证明是 hello world 应用程序的以下变体:

from flask import Flask
import numpy
app = Flask(__name__)

@app.route('/')
def hello_from_np():
  a = numpy.array([4,5,6])
  return str(a)

if __name__ == '__main__':
  app.run()

当您访问 IP 时，会在浏览器中显示此信息:

...而不是 apache 日志中关于 numpy 的一堆错误。 一切都已针对 Python 3.6 设置完毕:

AH00489: Apache/2.4.7 (Ubuntu) mod_wsgi/4.5.15 Python/3.6 configured

想记录一些细节作为给自己的注释，以防万一有人遇到麻烦并尝试重现此内容，也许可以让您免于数日找错树。

因此，我从 Ubuntu Server 16.04 切换到 14.04.5 LTS，以防这是问题的一部分，同时也是因为 pyenv性能很差，在我测试的许多旧版 Python 的安装过程中崩溃了。还选择忽略 pyenv-virtualenv，因为没有为此实例计划多个项目，而且也少了一件出错的事情。

从 Ubuntu 的全新安装中，除了安装 apache2、pyenv、libapache2-mod-wsgi 并使用 a2enmod 激活它之外，我还特意在使用 pip 安装需求之前运行了以下几行，其中包括 numpy，所以它不会不要提示:

sudo apt-get install gcc
sudo apt-get install g++

这也是需要的:

sudo apt-get install apache2-dev

然后在环境内部，满足要求后，

pip install mod_wsgi
mod_wsgi-express module-config

第二行返回:

LoadModule wsgi_module "/home/ubuntu/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so"
WSGIPythonHome "/home/ubuntu/.pyenv/versions/miniconda3-latest"

然后加载模块，docs鼓励您粘贴到文件 httpd.conf 中，该文件至少在我的 apache 和 ubuntu 版本中不存在。经过大量阅读后，我避免创建一个，也避免将其粘贴到 apache2.conf 中，这似乎是 related 。相反，按照顶部链接的 ubuntu 说明进行操作，

编辑此文件:

sudo vi /etc/apache2/mods-available/wsgi.load

将此行粘贴到其中并保存:

LoadModule wsgi_module /path/to/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so

然后运行此命令并在必要时重新启动 apache 服务器:

sudo a2enmod wsgi

000-default.conf 如下所示:

WSGIPythonHome "/home/ubuntu/.pyenv/versions/miniconda3-latest"

<VirtualHost *:80>
    # The ServerName directive sets the request scheme, hostname and port that
    # the server uses to identify itself. This is used when creating
    # redirection URLs. In the context of virtual hosts, the ServerName
    # specifies what hostname must appear in the request's Host: header to
    # match this virtual host. For the default virtual host (this file) this
    # value is not decisive as it is used as a last resort host regardless.
    # However, you must set it for any further virtual host explicitly.
    #ServerName www.example.com

    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html
        WSGIDaemonProcess helloapp threads=5 python-path=/var/www/html/frontend/  
        WSGIScriptAlias / /var/www/html/frontend/helloapp.wsgi

        <Directory flaskapp>
        WSGIProcessGroup helloapp
        WSGIApplicationGroup %{GLOBAL}
        Order allow,deny
        Allow from all
        </Directory>

    # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
    # error, crit, alert, emerg.
    # It is also possible to configure the loglevel for particular
    # modules, e.g.
    #LogLevel info ssl:warn

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    # For most configuration files from conf-available/, which are
    # enabled or disabled at a global level, it is possible to
    # include a line for only one particular virtual host. For example the
    # following line enables the CGI configuration for this host only
    # after it has been globally disabled with "a2disconf".
    #Include conf-available/serve-cgi-bin.conf
</VirtualHost>