python - Pandas read_html 返回原始 HTML 内容[对于某些行/单元格/等]

Question

足够简单的问题，但我猜答案是“不”:

我有一个 HTML 表格，我正在用 pandas.read_html 阅读它就好了。但是某些单元格(列)中包含图像、列表或其他明显 read_html 丢弃的格式。我显然不希望 Pandas 解析其中的任何内容，但是有什么方法可以让它返回原始 HTML，例如 DataFrame 单元格中的一个字符串，以便我可以自己解析它？

例子:

<table>
    <th>Column 1</th> 
    <th>Column 2</th>
    <tr>
        <td>Cell1</td> 
        <td>Cell2 <img src="http://www.link.com/image.jpg /></td>
    </tr>
    <tr>
        <td>Cell3</td> 
        <td>Cell4 <img src="http://www.website.com/picture.gif /></td>
    </tr>          
</table>

如果 Pandas 要解析这个，我可能只会从 column2 中得到“Cell2”和“Cell4”。我想做的是以某种方式获取单元格的全部内容，包括 <img> 的 [raw?] HTML 内容。标签。然后我可以自己解析它们。

Answer 1

pd.read_html 没有选项做你想做的功能。因此，当我尝试从加利福尼亚州国务院网站获取企业名称列表时，当 Pandas 自动解析 HTML 时，我得到了除名称之外的所有内容:

>>> bizname = 'poss'
>>> url = f'https://businesssearch.sos.ca.gov/CBS/SearchResults?filing=&SearchType=CORP&SearchCriteria={bizname}&SearchSubType=Begins'
>>> df = pd.read_html(url)[0]
>>> df
  Entity Number Registration Date         Status                                        Entity Name Jurisdiction      Agent for Service of Process
0      C2645412        04/02/2004         ACTIVE  View details for entity number 02645412  POSSU...      GEORGIA   ERESIDENTAGENT, INC. (C2702827)
1      C0786330        09/22/1976      DISSOLVED  View details for entity number 00786330  POSSU...   CALIFORNIA                        I. HALPERN
2      C2334141        03/01/2001  FTB SUSPENDED  View details for entity number 02334141  POSSU...   CALIFORNIA                   CLAIR G BURRILL
3      C0658630        11/08/1972  FTB SUSPENDED  View details for entity number 00658630  POSSU...   CALIFORNIA                               NaN
4      C1713121        09/23/1992  FTB SUSPENDED  View details for entity number 01713121  POSSU...   CALIFORNIA                LAWRENCE J. TURNER
5      C1207820        08/05/1983      DISSOLVED  View details for entity number 01207820  POSSU...   CALIFORNIA                          R L CARL
6      C3921531        06/27/2016         ACTIVE  View details for entity number 03921531  POSSU...   CALIFORNIA  REGISTERED AGENTS INC (C3365816)

该网站将企业名称隐藏在按钮后面。
但是你可以使用 requests下载原始html。
那么你可以使用bs4提取原始 HTML 表格以及您想要的任何特定行 ( <tr> ) 或单元格 ( <td> )。

>>> soup = bs4.BeautifulSoup(requests.get(url).text)
>>> table = soup.find('table').findAll('tr')
>>> names = []
... for row in table:
...     names.append(getattr(row.find('button'), 'contents', [''])[0].strip())
>>> names
['',
 'POSSUM FILMS, INC',
 'POSSUM INC.',
 'POSSUM MEDIA, INC.',
 'POSSUM POINT PRODUCTIONS, INC.',
 'POSSUM PRODUCTIONS, INC.',
 'POSSUM-BILITY EXPRESS, INCORPORATED',
]
>>> df['Entity Name'] = names[1:]
>>> df['Entity Name'] = names[1:]
>>> df
  Entity Number Registration Date         Status                          Entity Name Jurisdiction      Agent for Service of Process
0      C2645412        04/02/2004         ACTIVE                    POSSUM FILMS, INC      GEORGIA   ERESIDENTAGENT, INC. (C2702827)
1      C0786330        09/22/1976      DISSOLVED                          POSSUM INC.   CALIFORNIA                        I. HALPERN
2      C2334141        03/01/2001  FTB SUSPENDED                   POSSUM MEDIA, INC.   CALIFORNIA                   CLAIR G BURRILL
3      C0658630        11/08/1972  FTB SUSPENDED       POSSUM POINT PRODUCTIONS, INC.   CALIFORNIA                               NaN
4      C1713121        09/23/1992  FTB SUSPENDED             POSSUM PRODUCTIONS, INC.   CALIFORNIA                LAWRENCE J. TURNER
5      C1207820        08/05/1983      DISSOLVED  POSSUM-BILITY EXPRESS, INCORPORATED   CALIFORNIA                          R L CARL
6      C3921531        06/27/2016         ACTIVE                      POSSUMS WELCOME   CALIFORNIA  REGISTERED AGENTS INC (C33658

这样做不会正确处理标题，所以如果需要，不要忘记忽略第一行。