javascript - 使用 scrapy python 从 javascript 获取数据到 python

Question

如何使用 scrapy python 从 javascript 内容中获取数据？ JavaScript 看起来像这样

<script type="text/javascript">
  var ad_reply_url = "http://www2.mudah.my/ar/send/0?ca=3_s&id=49825097&l=0";
  var mcvl = "";
  var images = [
     'http://img.rnudah.com/images/13/133608119523265.jpg', 
     'http://img.rnudah.com/images/13/135608116569903.jpg', 
     'http://img.rnudah.com/images/13/137608113616541.jpg', 
     'http://img.rnudah.com/images/13/139608119186498.jpg'
  ];
 var thumbnails = [
    'http://img.rnudah.com/thumbs/13/133608119523265.jpg',
    'http://img.rnudah.com/thumbs/13/135608116569903.jpg',
    'http://img.rnudah.com/thumbs/13/137608113616541.jpg',
    'http://img.rnudah.com/thumbs/13/139608119186498.jpg'
 ];</script>

所以，我想要的是。我想要来自 var images 的数据并像这样打印该数据

['http://img.rnudah.com/images/13/133608119523265.jpg','http://img.rnudah.com/images/13/135608116569903.jpg', 'http://img.rnudah.com/images/13/137608113616541.jpg','http://img.rnudah.com/images/13/139608119186498.jpg' ];

有人可以帮助我吗？谢谢。

Answer 1

我没有使用 Scrapy Python，只是使用常规 Python。 但这非常简单:

代码示例:

import ast
import re

page_source = '''
<script type="text/javascript">
  var ad_reply_url = "http://www2.mudah.my/ar/send/0?ca=3_s&id=49825097&l=0";
  var mcvl = "";
  var images = [
     'http://img.rnudah.com/images/13/133608119523265.jpg',
     'http://img.rnudah.com/images/13/135608116569903.jpg',
     'http://img.rnudah.com/images/13/137608113616541.jpg',
     'http://img.rnudah.com/images/13/139608119186498.jpg'
  ];
 var thumbnails = [
    'http://img.rnudah.com/thumbs/13/133608119523265.jpg',
    'http://img.rnudah.com/thumbs/13/135608116569903.jpg',
    'http://img.rnudah.com/thumbs/13/137608113616541.jpg',
    'http://img.rnudah.com/thumbs/13/139608119186498.jpg'
 ];</script>
'''

variables = re.findall('(?si)var(.*?);', page_source)

var_collection = {}
for var in variables:
    var = var.strip()
    var_key = var.split(' = ')[0]
    var_value = ast.literal_eval(var.split(' = ')[1])
    var_collection.update({var_key: var_value})

print(var_collection['images'])

输出:

['http://img.rnudah.com/images/13/133608119523265.jpg', 'http://img.rnudah.com/images/13/135608116569903.jpg', 'http://img.rnudah.com/images/13/137608113616541.jpg', 'http://img.rnudah.com/images/13/139608119186498.jpg']

相关: https://stackoverflow.com/a/18108644/295246