python - scrapy 无法在我的 Mac 上加载 libxslt.1.dylib

Question

我使用pip install scrapy来安装scrapy

但是我运行 scrapy 出现以下错误:

➜  source  scrapy      
Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 7, in <module>
    from scrapy.cmdline import execute
  File "/Library/Python/2.7/site-packages/scrapy/__init__.py", line 48, in <module>
    from scrapy.spiders import Spider
  File "/Library/Python/2.7/site-packages/scrapy/spiders/__init__.py", line 10, in <module>
    from scrapy.http import Request
  File "/Library/Python/2.7/site-packages/scrapy/http/__init__.py", line 11, in <module>
    from scrapy.http.request.form import FormRequest
  File "/Library/Python/2.7/site-packages/scrapy/http/request/form.py", line 9, in <module>
    import lxml.html
  File "/Library/Python/2.7/site-packages/lxml/html/__init__.py", line 42, in <module>
    from lxml import etree
ImportError: dlopen(/Library/Python/2.7/site-packages/lxml/etree.so, 2): Library not loaded: /opt/local/lib/libxslt.1.dylib
  Referenced from: /Library/Python/2.7/site-packages/lxml/etree.so
  Reason: image not found

在我的/opt/floder 中对 libxslt.1.dylib 进行奇怪的 scrapy 查找...

我运行pip install scrapy，得到这个:

➜  ~  pip install scrapy
Requirement already satisfied (use --upgrade to upgrade): scrapy in /Library/Python/2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /Library/Python/2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): queuelib in /Library/Python/2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /Library/Python/2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.8.0 in /Library/Python/2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in /Library/Python/2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): Twisted>=10.0.0 in /Library/Python/2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /Library/Python/2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): service-identity in /Library/Python/2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): cryptography>=0.7 in /Library/Python/2.7/site-packages (from pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0 in /Library/Python/2.7/site-packages (from Twisted>=10.0.0->scrapy)
Requirement already satisfied (use --upgrade to upgrade): characteristic>=14.0.0 in /Library/Python/2.7/site-packages (from service-identity->scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules in /Library/Python/2.7/site-packages (from service-identity->scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyasn1 in /Library/Python/2.7/site-packages (from service-identity->scrapy)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/site-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): enum34 in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): ipaddress in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): idna>=2.0 in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): cffi>=1.1.0 in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): pycparser in /Library/Python/2.7/site-packages (from cffi>=1.1.0->cryptography>=0.7->pyOpenSSL->scrapy)

救命!!

2015/11/06更新

嗨，乔·杨。

我已经尝试过这个brew install libxml2 libxslt。但是它不起作用，看来我的python的查找路径是错误的？

➜  haimi git:(V3.2.0) brew install libxml2 libxslt
Warning: libxml2-2.9.2 already installed
Warning: libxslt-1.1.28_1 already installed
➜  haimi git:(V3.2.0) ✗ scrapy
Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 7, in <module>
    from scrapy.cmdline import execute
  File "/Library/Python/2.7/site-packages/scrapy/__init__.py", line 48, in <module>
    from scrapy.spiders import Spider
  File "/Library/Python/2.7/site-packages/scrapy/spiders/__init__.py", line 10, in <module>
    from scrapy.http import Request
  File "/Library/Python/2.7/site-packages/scrapy/http/__init__.py", line 11, in <module>
    from scrapy.http.request.form import FormRequest
  File "/Library/Python/2.7/site-packages/scrapy/http/request/form.py", line 9, in <module>
    import lxml.html
  File "/Library/Python/2.7/site-packages/lxml/html/__init__.py", line 42, in <module>
    from lxml import etree
ImportError: dlopen(/Library/Python/2.7/site-packages/lxml/etree.so, 2): Library not loaded: /opt/local/lib/libxslt.1.dylib
  Referenced from: /Library/Python/2.7/site-packages/lxml/etree.so
  Reason: image not found
➜  haimi git:(V3.2.0) ✗ 

Answer 1

听起来您需要 lxml python 包所需的 libxslt 和 libxml 库，而 scrapy 也需要这些库。

如果您安装“X-code 命令行工具包”，它应该包含这些库。这里有一个指南:http://quantgreeks.com/how-to-install-xcode-command-line-tools-in-osx-yosemite/

本质上，它归结为运行:

xcode-select --install