javascript - 使用 Json 和 BS4 抓取 HTML 中的脚本标签

Question

我希望能够从网页上抓取此链接的代码 https://secure.ewaypayments.com/sharedpage/sharedpayment?AccessCode=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==

我目前正在 python 中使用 json 和 bs4。

整页源码 https://pastebin.com/iU5c9GBF

<div class="Actions">
                <input class="action" type="submit" id="submit-button" value="Place Order" title="Place Order" onclick="return showModal()" disabled="disabled" />
              <input type="hidden" id="EWAY_TransactionID" name="EWAY_TransactionID" value="" />
              <script src="https://secure.ewaypayments.com/scripts/eCrypt.js"> </script>
              <script type="text/javascript">
                var eWAYConfig = {
                  sharedPaymentUrl: "https://secure.ewaypayments.com/sharedpage/sharedpayment?AccessCode=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=="
                };
                function showModal()
                {
                  // verify captcha

                  // show modal
                  return eCrypt.showModalPayment(eWAYConfig, resultCallback);
                }
                function resultCallback(result, transactionID, errors) {
                  if (result == "Complete") {
                    document.getElementById("EWAY_TransactionID").value = transactionID;
                    document.getElementById("Form_PaymentForm").submit();
                    //Please wait until we process your order, James at 9/10/2017
                    document.getElementById("overlay").style.display = "block";
                  }
                  else if (errors != "")
                  {
                    alert("There was a problem completing the payment: " + errors);
                  }
                }
              </script>

以前使用过的代码

s = requests.session()
orderurl = s.get('https://www.supplystore.com.au/shop/checkout/submit.aspx')
soup = bs(orderurl.text, 'html.parser')
find = soup.find("div", {"class": "Actions"}).find("script")[1]

Answer 1

您不能使用 BeautifulSoup 来解析 Javascript 数据，但您可以使用 re 模块(data 是您的 HTML 代码):

import re    
from bs4 import BeautifulSoup

soup = BeautifulSoup(data, 'lxml')    
txt = soup.select('.Actions script')[1].text

print(re.search(r'sharedPaymentUrl:\s*"(.*?)"', txt)[1])

打印:

https://secure.ewaypayments.com/sharedpage/sharedpayment?AccessCode=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==