python - Pandas 抓取的数据在 Pandas 中不起作用

标签 python python-3.x pandas selenium screen-scraping

为什么当我手动将数据输入到 Excel 中时,pandas 可以正常工作。然而,当我抓取数据时,将其放入 csv 中。它给了我:

    zz = df1.WE=np.where(df3.AL.isin(df1.EW),df1.WE,np.nan)
ValueError: operands could not be broadcast together with shapes (148,) (537,) () 

其他网站还没有发生过。我在这里遗漏了一些明显的东西吗?是 excel 格式不正确还是这里的数据不一样?

df3

df3 = pd.DataFrame(columns=['DAT', 'G', 'TN', 'O1', 'L1', 'TN2', 'O2', 'L2', 'D', 'AJ', 'AK', 'AL'])

df1

                                         EW    WE  \
0             Ponte Preta U20 v Cruzeiro U20  2.10   
1  Fluminense RJ U20 v Defensor Sporting U20  2.00   
2              Gremio RS U20 v Palmeiras U20  3.30   
3                       Barcelona v Sporting  1.33   

                                                  DA  
0  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
1  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
2  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
3  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  

代码:

df3 = pd.DataFrame(columns=['DAT', 'G', 'TN', 'O1', 'L1', 'TN2', 'O2', 'L2', 'D', 'AJ', 'AK', 'AL'])

df3['DAT'] = df2['AA']



zz = df1.WE=np.where(df3.AL.isin(df1.EW),df1.WE,np.nan)
print(zz)

我已经提供了创建数据框 1、2 和 pandas 代码的所有脚本,直到它创建错误 here

我一直在努力

错误:

    zz = df1.WE=np.where(df3.AL.isin(df1.EW),df1.WE,np.nan)
ValueError: operands could not be broadcast together with shapes (0,) (4,) ()

从抓取中创建并作为数据框加载的错误文件:

如果这还不够,我还按原样加载了文件。

File 1 , File 2

手动创建的工作文件:

工作:

File 1 , File 2

知道如何解决这个问题吗?

最佳答案

我认为你需要改变:

df1.WE=np.where(df3.AL.isin(df1.EW),df1.WE,np.nan)

df1.WE=np.where(df1.EW.isin(df2.AL),df1.WE,np.nan)

问题是 DataFrame 的长度与实际数据不同。因此需要使用另一个数据更改来自 df1 的数据 - comapring 返回与 df1 长度相同且没有错误的 maks。

使用您的数据:

df1 = pd.read_csv('df1.csv', names=['a','b','c'])
print (df1.head())
                                           a     b  \
0             Ponte Preta U20 v Cruzeiro U20  2.10   
1  Fluminense RJ U20 v Defensor Sporting U20  2.00   
2              Gremio RS U20 v Palmeiras U20  3.30   
3                       Barcelona v Sporting  1.33   
4                        Bayern Munich v PSG  2.40   

                                                   c  
0  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
1  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
2  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
3  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
4  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  

df2 = pd.read_csv('df2.csv', names=['a','b','c', 'd', 'e'])
print (df2.head())
                 a                    b                  c     d  \
0          In-Play      CSKA Moscow U19        Man Utd U19  1.14   
1          In-Play  Atletico Madrid U19        Chelsea U19  1.01   
2          In-Play         Juventus U19     Olympiakos U19  1.40   
3  Starting in 22'       Paris St-G U19  Bayern Munich U19  2.24   
4      Today 21:00         Man City U19       Shakhtar U19  2.66   

                                                   e  
0  https://www.betfair.com.au/exchange/plus/footb...  
1  https://www.betfair.com.au/exchange/plus/footb...  
2  https://www.betfair.com.au/exchange/plus/footb...  
3  https://www.betfair.com.au/exchange/plus/footb...  
4  https://www.betfair.com.au/exchange/plus/footb...  

comapre 数字列,此处为 bd:

df1.b=np.where(df1.b.isin(df2.d),df1.b,np.nan)
#first 5 values is NaNs
print (df1.head())
                                           a   b  \
0             Ponte Preta U20 v Cruzeiro U20 NaN   
1  Fluminense RJ U20 v Defensor Sporting U20 NaN   
2              Gremio RS U20 v Palmeiras U20 NaN   
3                       Barcelona v Sporting NaN   
4                        Bayern Munich v PSG NaN  

                                                   c  
0  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
1  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
2  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
3  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
4  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  

#check if some not NaNs values in b column
print (df1[df1.b.notnull()])
                                       a      b  \
23                Swindon v Forest Green   1.40   
50       Sportivo Barracas v Canuelas FC  13.00   
80                              FC Nitra   1.53   
81                                   0-0   1.40   
83       Cape Town City v Maritzburg Utd   1.53   
84         Mamelodi Sundowns v Baroka FC   3.75   
90  Dorking Wanderers v Tonbridge Angels   1.53   
95             Coalville Town v Stamford   1.40   

                                                    c  
23  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
50  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
80  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
81  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
83  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
84  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
90  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  
95  https://www.bet365.com.au/#/AC/B1/C1/D13/E40/F...  

您的测试数据的另一个问题是行数相同 (4),因此没有错误。

关于python - Pandas 抓取的数据在 Pandas 中不起作用,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/47658860/

相关文章:

python-2.7 - Pandas 根据不同的时间窗口计算特征

python - 具有多个特征的 train_test_split

python - 如何用for循环改进子图部分?

python - 从一个微型 Python 网络框架发出 header

python - Django(Gunicorn)在生产中没有看到 ENV VARS,但 Django-shell 有

python - 您可以在 Keras 中编写自定义学习率调度程序吗?

postgresql - 使用 GeoAlchemy2 按距离选择和排序。错误的 ST_AsBinary 换行

python - 我怎样才能加快 python 中的迭代?

android - 有没有办法在Android上运行Python?

python - 附魔错误。错误: Don't pass bytestrings to pyenchant-python