python - pandas 将一个 df 中的日期与另一个 df 中的时间范围进行匹配，然后进行 groupby-sum

Question

我有两个数据帧，test1 和 test2。对于 test2 中的每个 ID 值，我想检查 test2 中的date 并将其与日期范围进行比较对于 test1 中的相同 ID 值。如果 test2 中的任何一个 date 位于 test1 中的日期范围内，则对 amount 列求和，将该总和指定为 test1 中的附加列。

输出:

因此，新的 test1 df 将有一列 amount_sum，它是 test2 中所有金额的总和，其中 date 位于 test1 - 该 ID

的日期范围内

import random
import string

test1 = pd.DataFrame({
    'ID':[''.join(random.choice(string.ascii_letters[0:4]) for _ in range(3)) for n in range(100)],
    'date1':[pd.to_datetime(random.choice(['01-01-2018','05-01-2018','06-01-2018','08-01-2018','09-01-2018'])) + pd.DateOffset(int(np.random.randint(0, 100, 1))) for n in range(100)],
    'date2':[pd.to_datetime(random.choice(['01-01-2018','05-01-2018','06-01-2018','08-01-2018','09-01-2018'])) + pd.DateOffset(int(np.random.randint(101, 200, 1))) for n in range(100)]
})

test2 = pd.DataFrame({
    'ID':[''.join(random.choice(string.ascii_letters[0:4]) for _ in range(3)) for n in range(100)],
    'amount':[random.choice([1,2,3,5,10]) for n in range(100)],
    'date':[pd.to_datetime(random.choice(['01-01-2018','05-01-2018','06-01-2018','08-01-2018','09-01-2018'])) + pd.DateOffset(int(np.random.randint(0, 100, 1))) for n in range(100)]
})

Answer 1

用途:

#outer join both df by ID columns
df = test1.merge(test2, on='ID', how='outer')
#filter by range
df = df[(df.date > df.date1) & (df.date < df.date2)]
#thank you @Abhi for alternative
#df = df[df.date.between(df.date1, df.date2, inclusive=False)]
#aggregate sum
s = df.groupby(['ID','date1','date2'])['amount'].sum()
#add new column to test1
test = test1.join(s, on=['ID','date1','date2'])

示例:

#https://stackoverflow.com/q/21494489
np.random.seed(123)

#https://stackoverflow.com/a/50559321/2901002
def gen(start, end, n):
    start_u = start.value//10**9
    end_u = end.value//10**9

    return pd.to_datetime(np.random.randint(start_u, end_u, n), unit='s')

n = 10
test1 = pd.DataFrame({
    'ID':np.random.choice(list('abc'), n),
    'date1': gen(pd.to_datetime('2010-01-01'),pd.to_datetime('2010-03-01'), n).floor('d'),
    'date2':gen(pd.to_datetime('2010-03-01'),pd.to_datetime('2010-06-01'), n).floor('d')
})

m = 5
test2 = pd.DataFrame({
    'ID': np.random.choice(list('abc'), m),
    'amount':np.random.randint(10, size=m),
    'date':gen(pd.to_datetime('2010-01-01'), pd.to_datetime('2010-06-01'), m).floor('d')
})

<小时/>

print (test1)
  ID      date1      date2
0  c 2010-01-15 2010-05-22
1  b 2010-02-08 2010-04-16
2  c 2010-01-24 2010-04-12
3  c 2010-02-01 2010-04-09
4  a 2010-01-19 2010-05-20
5  c 2010-01-27 2010-05-24
6  c 2010-02-23 2010-03-15
7  b 2010-01-31 2010-05-09
8  c 2010-02-23 2010-03-29
9  b 2010-01-08 2010-03-07

print (test2)
  ID  amount       date
0  a       4 2010-05-15
1  b       6 2010-03-26
2  a       1 2010-01-07
3  b       5 2010-02-07
4  a       6 2010-04-13

#outer join both df by ID columns
df = test1.merge(test2, on='ID', how='outer')
#filter by range
df = df[(df.date > df.date1) & (df.date < df.date2)]
print (df)
   ID      date1      date2  amount       date
6   b 2010-02-08 2010-04-16     6.0 2010-03-26
8   b 2010-01-31 2010-05-09     6.0 2010-03-26
9   b 2010-01-31 2010-05-09     5.0 2010-02-07
11  b 2010-01-08 2010-03-07     5.0 2010-02-07
12  a 2010-01-19 2010-05-20     4.0 2010-05-15
14  a 2010-01-19 2010-05-20     6.0 2010-04-13

<小时/>

#thank you @Abhi for alternative
#df = df[df.date.between(df.date1, df.date2, inclusive=False)]
#aggregate sum
s = df.groupby(['ID','date1','date2'])['amount'].sum()
#add new column to test1
test = test1.join(s, on=['ID','date1','date2'])
print (test)
  ID      date1      date2  amount
0  c 2010-01-15 2010-05-22     NaN
1  b 2010-02-08 2010-04-16     6.0
2  c 2010-01-24 2010-04-12     NaN
3  c 2010-02-01 2010-04-09     NaN
4  a 2010-01-19 2010-05-20    10.0
5  c 2010-01-27 2010-05-24     NaN
6  c 2010-02-23 2010-03-15     NaN
7  b 2010-01-31 2010-05-09    11.0
8  c 2010-02-23 2010-03-29     NaN
9  b 2010-01-08 2010-03-07     5.0