python - 如何从网页下载代码，beautifulsoup 没有获取所有内容

Question

我想从这个网页获取代码值 https://www.oslobors.no/markedsaktivitet/#/list/shares/quotelist/ob/all/all/false

然而，当使用 Beautifulsoup 时，我似乎并没有得到所有的内容，而且我不太明白如何更改我的代码以实现我的目标

import urllib3  
from bs4 import BeautifulSoup  
def oslobors():
http=urllib3.PoolManager()  
url = 'https://www.oslobors.no/markedsaktivitet/#/list/shares/quotelist/ob/all/all/false'  
response = http.request('GET', url)  
soup=BeautifulSoup(response.data, "html.parser")  
print(soup)  
return  
print(oslobors())

Answer 1

您要解析的内容动态生成。您可以使用任何浏览器模拟器，如 selenium，也可以尝试以下包含 json 响应的 url。以下是简单的方法。

import requests

url = 'https://www.oslobors.no/ob/servlets/components?type=table&generators%5B0%5D%5Bsource%5D=feed.ob.quotes.EQUITIES%2BPCC&generators%5B1%5D%5Bsource%5D=feed.merk.quotes.EQUITIES%2BPCC&filter=&view=DELAYED&columns=PERIOD%2C+INSTRUMENT_TYPE%2C+TRADE_TIME%2C+ITEM_SECTOR%2C+ITEM%2C+LONG_NAME%2C+BID%2C+ASK%2C+LASTNZ_DIV%2C+CLOSE_LAST_TRADED%2C+CHANGE_PCT_SLACK%2C+TURNOVER_TOTAL%2C+TRADES_COUNT_TOTAL%2C+MARKET_CAP%2C+HAS_LIQUIDITY_PROVIDER%2C+PERIOD%2C+MIC%2C+GICS_CODE_LEVEL_1%2C+TIME%2C+VOLUME_TOTAL&channel=a66b1ba745886f611af56cec74115a51'

res = requests.get(url)
for ticker in res.json()['rows']:
    ticker_name = ticker['values']['ITEM']
    print(ticker_name)

您可能会得到的结果(部分):

APP
HEX
APCL
ODFB
SAS NOK
WWI
ASC