python - Pandas groupby 根据 2 个组仅选择一个值并将其余值转换为 0

Question

我有一个 pandas 数据框，它的日期时间索引如下所示:

df=

           Fruit    Quantity
01/02/10    Apple   4
01/02/10    Apple   6
01/02/10    Pear    7
01/02/10    Grape   8
01/02/10    Grape   5
02/02/10    Apple   2
02/02/10    Fruit   6
02/02/10    Pear    8
02/02/10    Pear    5

现在，对于每个日期和每种水果，我只想要一个值(最好是顶部的值)，而日期的其余水果保持为零。所以期望的输出如下:

           Fruit    Quantity
01/02/10    Apple   4
01/02/10    Apple   0
01/02/10    Pear    7
01/02/10    Grape   8
01/02/10    Grape   0
02/02/10    Apple   2
02/02/10    Fruit   6
02/02/10    Pear    8
02/02/10    Pear    0

这只是一个小例子，但我的主数据框有超过 300 万行，并且水果不一定按日期顺序排列。

谢谢

Answer 1

您可以使用:

• set_index索引名称
• groupby与 cumcount每组计数器
• 通过 eq 获取第一组的 bool 掩码
• 通过where设置0和面具

m = df.rename_axis('Date').groupby(['Date', 'Fruit']).cumcount().eq(0)
df['Quantity'] = df['Quantity'].where(m, 0)
print (df)
          Fruit  Quantity
01/02/10  Apple         4
01/02/10  Apple         0
01/02/10   Pear         7
01/02/10  Grape         8
01/02/10  Grape         0
02/02/10  Apple         2
02/02/10  Fruit         6
02/02/10   Pear         8
02/02/10   Pear         0

另一个解决方案 reset_index ，但有必要通过 values 将 bool 掩码转换为 numpy 数组，因为不同的索引:

m = df.reset_index().groupby(['index', 'Fruit']).cumcount().eq(0)
df['Quantity'] = df['Quantity'].where(m.values, 0)
print (df)
          Fruit  Quantity
01/02/10  Apple         4
01/02/10  Apple         0
01/02/10   Pear         7
01/02/10  Grape         8
01/02/10  Grape         0
02/02/10  Apple         2
02/02/10  Fruit         6
02/02/10   Pear         8
02/02/10   Pear         0

时间:

np.random.seed(1235)

N = 10000
L = ['Apple','Pear','Grape','Fruit']
idx = np.repeat(pd.date_range('2017-010-01', periods=N/20).strftime('%d/%m/%y'), 20)
df = (pd.DataFrame({'Fruit': np.random.choice(L, N),
                   'Quantity':np.random.randint(100, size=N), 'idx':idx})
      .sort_values(['Fruit','idx'])
      .set_index('idx')
      .rename_axis(None))             

#print (df)

<小时/>

def jez1(df):
    m = df.rename_axis('Date').groupby(['Date', 'Fruit']).cumcount().eq(0)
    df['Quantity'] = df['Quantity'].where(m, 0)
    return df

def jez2(df):
    m = df.reset_index().groupby(['index', 'Fruit']).cumcount().eq(0)
    df['Quantity'] = df['Quantity'].where(m.values, 0)
    return df

def rnso(df):
    df['date_fruit'] = df.index+df.Fruit # new column with date and fruit merged
    dflist = pd.unique(df.date_fruit)    # find its unique values
    dfv = df.values                      # get rows as list of lists
    for i in dflist:                     # for each unique date-fruit combination
        done = False
        for c in range(len(dfv)): 
            if dfv[c][2] == i:           # check each row
                if done: 
                    dfv[c][1] = 0        # if not first, make quantity as 0
                else: 
                    done = True

    # create new dataframe with new data: 
    newdf = pd.DataFrame(data=dfv, columns=df.columns, index=df.index)
    return newdf.iloc[:,:2] 

<小时/>

print (jez1(df))      
print (jez2(df))      
print (rnso(df))      

In [189]: %timeit (rnso(df))
1 loop, best of 3: 6.27 s per loop

In [190]: %timeit (jez1(df))
100 loops, best of 3: 7.56 ms per loop

In [191]: %timeit (jez2(df))
100 loops, best of 3: 8.77 ms per loop

编辑另一个答案:

存在问题，您需要按列 Fruit 和 index 重复调用。 所以有两种可能的解决方案:

• 从索引创建列reset_index并调用DataFrame.duplicated ，最后将输出转换为 numpy 数组 values
• 将 Fruit 列添加到 index by set_index并调用Index.duplicated

#solution1
mask = df.reset_index().duplicated(['index','Fruit']).values
#solution2
#mask = df.set_index('Fruit', append=True).index.duplicated()
df.loc[mask, 'Quantity'] = 0

时间1

def jez1(df):
    m = df.rename_axis('Date').groupby(['Date', 'Fruit']).cumcount().eq(0)
    df['Quantity'] = df['Quantity'].where(m, 0)
    return df

def jez3(df):
    mask = df.reset_index().duplicated(['index','Fruit']).values
    df.loc[mask, 'Quantity'] = 0
    return df

def jez4(df):
    mask = df.set_index('Fruit', append=True).index.duplicated()
    df.loc[mask, 'Quantity'] = 0
    return df

print (jez1(df))
print (jez3(df))
print (jez4(df))

In [268]: %timeit jez1(df)
100 loops, best of 3: 6.37 ms per loop

In [269]: %timeit jez3(df)
100 loops, best of 3: 3.82 ms per loop

In [270]: %timeit jez4(df)
100 loops, best of 3: 4.21 ms per loop