python - 为什么当我使用 pd.update 时 pandas 抛出 "Data overlaps."？

Question

我有我的主数据框 df_main，它的形状如下:

df_main = pd.DataFrame({
    'GroupID': ([1]*7) + ([2]*2) + ([3]*6),
    'GroupName': (['String 1']*7) + (['String 2']*2) + (['String 3']*6),
    'FirstName': (['Matthew']*7) + (['Mark']*2) + (['Luke']*6),
    'LastName': (['Smith']*7) + (['Jones']*2) + (['Roberts']*6),
    'StartDate': (['2020-01-01']*7) + (['1998-01-01']*2) + (['N/A']*6),
}).replace('N/A',np.NaN)

我有一个辅助数据框 df_update，我想用它来更新主数据框。此数据框缺少 Luke Roberts 的开始日期:

df_update = pd.DataFrame({
    'GroupID': [1, 2, 3],
    'GroupName': ['String 1', 'String 2', 'String 3'],
    'FirstName': ['Matthew', 'Mark', 'Luke'],
    'LastName': ['Smith', 'Jones', 'Roberts'],
    'StartDate': ['2020-01-01', '1998-01-01', '2005-01-01'],
})

我为两个数据帧设置了相同的索引:

df_main = df_main.set_index(['GroupID', 'GroupName', 'FirstName', 'LastName'])
df_update = df_update.set_index(['GroupID', 'GroupName', 'FirstName', 'LastName'])

我尝试更新 df_main:

df_main.update(df_update, overwrite=False, errors='raise')

但它不起作用:

ValueError: Data overlaps.

为什么会这样，我怎样才能优雅地完成这个更新？

编辑:这是 Python 3.6.7 和 Pandas 0.25.0。

Answer 1

对于我在 pandas 1.1.3 中使用您的解决方案:

df_main.update(df_update)
print (df_main)
                                       StartDate
GroupID GroupName FirstName LastName            
1       String 1  Matthew   Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
2       String 2  Mark      Jones     1998-01-01
                            Jones     1998-01-01
3       String 3  Luke      Roberts   2005-01-01
                            Roberts   2005-01-01
                            Roberts   2005-01-01
                            Roberts   2005-01-01
                            Roberts   2005-01-01
                            Roberts   2005-01-01

如果只需要替换缺失值，请使用 DataFrame.fillna :

df_main = df_main.set_index(['GroupID', 'GroupName', 'FirstName', 'LastName'])
df_update = df_update.set_index(['GroupID', 'GroupName', 'FirstName', 'LastName'])
df = df_main.fillna(df_update)
print (df)
                                       StartDate
GroupID GroupName FirstName LastName            
1       String 1  Matthew   Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
2       String 2  Mark      Jones     1998-01-01
                            Jones     1998-01-01
3       String 3  Luke      Roberts   2005-01-01
                            Roberts   2005-01-01
                            Roberts   2005-01-01
                            Roberts   2005-01-01
                            Roberts   2005-01-01
                            Roberts   2005-01-01

如果更改数据，可能会看到解决方案的差异:

df_update = pd.DataFrame({
    'GroupID': [1, 2, 3],
    'GroupName': ['String 1', 'String 2', 'String 3'],
    'FirstName': ['Matthew', 'Mark', 'Luke'],
    'LastName': ['Smith', 'Jones', 'Roberts'],
    'StartDate': ['1990-01-01', '1991-01-01', '1992-01-01'],
})

df_main = df_main.set_index(['GroupID', 'GroupName', 'FirstName', 'LastName'])
df_update = df_update.set_index(['GroupID', 'GroupName', 'FirstName', 'LastName'])
df_main.update(df_update)
print (df_main)
                                       StartDate
GroupID GroupName FirstName LastName            
1       String 1  Matthew   Smith     1990-01-01
                            Smith     1990-01-01
                            Smith     1990-01-01
                            Smith     1990-01-01
                            Smith     1990-01-01
                            Smith     1990-01-01
                            Smith     1990-01-01
2       String 2  Mark      Jones     1991-01-01
                            Jones     1991-01-01
3       String 3  Luke      Roberts   1992-01-01
                            Roberts   1992-01-01
                            Roberts   1992-01-01
                            Roberts   1992-01-01
                            Roberts   1992-01-01
                            Roberts   1992-01-01

---

df_main = df_main.set_index(['GroupID', 'GroupName', 'FirstName', 'LastName'])
df_update = df_update.set_index(['GroupID', 'GroupName', 'FirstName', 'LastName'])
df = df_main.fillna(df_update)
print (df)
                                       StartDate
GroupID GroupName FirstName LastName            
1       String 1  Matthew   Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
                            Smith     2020-01-01
2       String 2  Mark      Jones     1998-01-01
                            Jones     1998-01-01
3       String 3  Luke      Roberts   1992-01-01
                            Roberts   1992-01-01
                            Roberts   1992-01-01
                            Roberts   1992-01-01
                            Roberts   1992-01-01
                            Roberts   1992-01-01