python - 从 html <script> 中提取 JSON 对象

Question

我有 JSON 对象:

{
  "review_body": "Beef noodles realism weathered modem tanto hotdog dolphin long-chain hydrocarbons 8-bit euro-pop tank-traps Tokyo narrative.-space j-pop franchise otaku faded RAF girl artisanal hotdog denim ablative systemic smart-Kowloon. Man construct dome smart-computer pen monofilament beef noodles rain garage geodesic bicycle San Francisco wonton soup dissident nodal point tower. Boat uplink film dead man modem warehouse. Nodal point jeans euro-pop render-farm nano-fetishism semiotics hacker gang. Futurity narrative youtube otaku Kowloon free-market drugs. Fluidity assassin Tokyo bicycle media assault concrete industrial grade ablative lights boat BASE jump A.I. post-stimulate carbon. Physical computer narrative city youtube math-neural assassin modem.",
  "link": "http://www.getlost.com/store/acme/review/10607787#comment10607787",
  "seller_id": "104523",
  "survey_id": "9933447",
  "loggedin_user": 0,
  "store_rating": "8.02",
  "store_thumb": "http://www.getlost.com/store/thumbnail/acme.jpg",
  "store_name": "acme",
  "username": "ronin666",
  "rating": "1",
  "ref": "RR,acme,104523"
}

嵌入在

<script LANGUAGE="javascript">
window.commentShare = $.extend((window.commentShare || {}), {
    10607787: {
        "review_body": "Beef noodles realism weathered modem tanto hotdog dolphin long-chain hydrocarbons 8-bit euro-pop tank-traps Tokyo narrative.-space j-pop franchise otaku faded RAF girl artisanal hotdog denim ablative systemic smart-Kowloon. Man construct dome smart-computer pen monofilament beef noodles rain garage geodesic bicycle San Francisco wonton soup dissident nodal point tower. Boat uplink film dead man modem warehouse. Nodal point jeans euro-pop render-farm nano-fetishism semiotics hacker gang. Futurity narrative youtube otaku Kowloon free-market drugs. Fluidity assassin Tokyo bicycle media assault concrete industrial grade ablative lights boat BASE jump A.I. post-stimulate carbon. Physical computer narrative city youtube math-neural assassin modem.",
        "link": "http:\/\/www.getlost.com\/store\/acme\/review\/10607787#comment10607787",
        "seller_id": "104523",
        "survey_id": "9933447",
        "loggedin_user": 0,
        "store_rating": "8.02",
        "store_thumb": "http:\/\/www.getlost.com\/store\/thumbnail\/acme.jpg",
        "store_name": "acme",
        "username": "ronin666",
        "rating": "1",
        "ref": "RR,acme,104523"
    }
});
</script>

我想提取上述 JSON 对象。如何实现？我应该使用正则表达式吗？

如何获取此类对象(通过 Ipython、python 2.7):

我基本上是在使用 BeautifulSoup 抓取评论网站 resellerratings.com 上的任意一家商店。我获得了 soup 对象并注意到有一些有用的 JSON 对象包含所选商店的每条评论的信息。然而，在调用 soup.find("script", language = "javascript") 时，我仍然保留了嵌入在脚本标签中的 JSON 对象。

from mechanize import Browser
import bs4
from bs4 import BeautifulSoup

br = Browser()
br.set_handle_robots(False)
br.set_handle_refresh(False)

example_url = 'http://www.resellerratings.com/store/My_Digital_Palace'

response = br.open(example_url)
soup = BeautifulSoup(response)
soup.find("script", language = "javascript")

这应该返回:

<script language="javascript">
window.commentShare = $.extend(
    (window.commentShare || {}), {
        375015: {
            "review_body": "I bought a Kodak LS443 form My Digital Palace in 2004.  I also purchased a 5 year warranty.  Now the camera does not work and I am unable to contact them.  What do I do???  Am I just screwed???<br><br>Margaret Fuller<br>margaret_fuller@sbcglobal.net",
            "link": "http:\/\/www.resellerratings.com\/store\/My_Digital_Palace\/review\/375015#comment375015",
            "seller_id": "6930",
            "survey_id": "385176",
            "loggedin_user": 0,
            "store_rating": "1.00",
            "store_thumb": "http:\/\/www.resellerratings.com\/store\/thumbnail\/My_Digital_Palace.jpg",
            "store_name": "My Digital Palace",
            "username": "maf1059",
            "rating": "1",
            "ref": "RR,My_Digital_Pala,6930"
        }
    }
);
</script>

Answer 1

非常简单，只需去掉包装器和无关的行即可获得丰富多彩的 JSON 本身。下面删除了你的 javscript 片段的前四行和最后三行(同时还把最初的 { 放回丢失的地方):

import json

raw = "{" + "\n".join(str(soup.find("script")).split("\n")[4:-3])

如果<script>页面上的对象不是以统一的方式编写的(也就是说，它并不总是前四行和最后两行是无关的)，您可能不得不求助于正则表达式或其他匹配。之后，您可以继续访问 JSON。

json_obj = json.loads(raw)

您的问题只是一个正则表达式/拆分问题。我认为人们对 Javascript 有点反感。 :)