python - Pandas 从非统一的日期列表中查找一年前的日期

Question

我需要一些关于项目的更多帮助。我正在尝试分析 450 万行数据。我已将数据读入数据框，组织了数据，现在有 3 列:1) 日期作为日期时间 2) 唯一标识符 3) 价格

我需要计算每件商品价格的逐年变化，但日期不统一且每件商品不一致。例如:

date      item  price
12/31/15   A     110
12/31/15   B     120
12/31/14   A     100
6/24/13    B     100

我想要找到的结果是:

date      item  price  previousdate   % change
12/31/15   A     110   12/31/14       10%
12/31/15   B     120   6/24/13        20%
12/31/14   A     100
6/24/13    B     100

编辑 - 更好的数据示例

 date   item    price
6/1/2016    A   276.3457646
6/1/2016    B   5.044165645
4/27/2016   B   4.91300186
4/27/2016   A   276.4329163
4/20/2016   A   276.9991265
4/20/2016   B   4.801263717
4/13/2016   A   276.1950213
4/13/2016   B   5.582923328
4/6/2016    B   5.017863509
4/6/2016    A   276.218649
3/30/2016   B   4.64274783
3/30/2016   A   276.554653
3/23/2016   B   5.576438253
3/23/2016   A   276.3135836
3/16/2016   B   5.394435443
3/16/2016   A   276.4222986
3/9/2016    A   276.8929462
3/9/2016    B   4.999951262
3/2/2016    B   4.731349423
3/2/2016    A   276.3972068
1/27/2016   A   276.8458971
1/27/2016   B   4.993033132
1/20/2016   B   5.250379701
1/20/2016   A   276.2899864
1/13/2016   B   5.146639666
1/13/2016   A   276.7041978
1/6/2016    B   5.328296958
1/6/2016    A   276.9465891
12/30/2015  B   5.312301356
12/30/2015  A   256.259668
12/23/2015  B   5.279105491
12/23/2015  A   255.8411198
12/16/2015  B   5.150798234
12/16/2015  A   255.8360529
12/9/2015   A   255.4915183
12/9/2015   B   4.722876886
12/2/2015   A   256.267146
12/2/2015   B   5.083626167
10/28/2015  B   4.876177757
10/28/2015  A   255.6464653
10/21/2015  B   4.551439655
10/21/2015  A   256.1735769
10/14/2015  A   255.9752668
10/14/2015  B   4.693967392
10/7/2015   B   4.911797443
10/7/2015   A   256.2556707
9/30/2015   B   4.262994526
9/30/2015   A   255.8068691
7/1/2015    A   255.7312385
4/22/2015   A   234.6210132
4/15/2015   A   235.3902076
4/15/2015   B   4.154926102
4/1/2015    A   234.4713827
2/25/2015   A   235.1391496
2/18/2015   A   235.1223471

我所做的(在其他用户的一些帮助下)没有起作用，但如下。感谢你们提供的任何帮助或为我指明了正确的方向!

import pandas as pd
import datetime as dt
import numpy as np

df = pd.read_csv('...python test file5.csv',parse_dates =['As of Date'])

df = df[['item','price','As of Date']]

def get_prev_year_price(x, df):
    try:
        return df.loc[x['prev_year_date'], 'price']
        #return np.abs(df.time - x)
    except Exception as e:
        return x['price']

#Function to determine the closest date from given date and list of all dates
def nearest(items, pivot):
    return min(items, key=lambda x: abs(x - pivot))

df['As of Date'] = pd.to_datetime(df['As of Date'],format='%m/%d/%Y')
df = df.rename(columns = {df.columns[2]:'date'})

# list of dates
dtlst = [item for item in df['date']]

data = []
data2 = []
for item in df['item'].unique():
    item_df = df[df['item'] == item] #select based on items
    select_dates = item_df['date'].unique()
    item_df.set_index('date', inplace=True) #set date as key index

    item_df = item_df.resample('D').mean().reset_index() #fill in missing date
    item_df['price'] = item_df['price'].interpolate('nearest') #fill in price with nearest price available
    # use max(item_df['date'] where item_df['date'] < item_df['date'] - pd.DateOffset(years=1, days=1))
        #possible_date = item_df['date'] - pd.DateOffset(years=1)
        #item_df['prev_year_date'] = max(df[df['date'] <= possible_date])

    item_df['prev_year_date'] = item_df['date'] - pd.DateOffset(years=1) #calculate 1 year ago date
    date_df = item_df[item_df.date.isin(select_dates)] #select dates with useful data
    item_df.set_index('date', inplace=True)

    date_df['prev_year_price'] = date_df.apply(lambda x: get_prev_year_price(x, item_df),axis=1)
    #date_df['prev_year_price'] = date_df.apply(lambda x: nearest(dtlst, x),axis=1)

    date_df['change'] = date_df['price'] / date_df['prev_year_price']-1
    date_df['item'] = item
    data.append(date_df)
    data2.append(item_df)
summary = pd.concat(data).sort_values('date', ascending=False)
#print (summary)

#saving the output of the CSV file to see how data looks after being handled 
filename = '...python_test_file_save4.csv'
summary.to_csv(filename, index=True, encoding='utf-8')

Answer 1

根据当前用例假设，这适用于该特定用例

In [2459]: def change(grp):
      ...:     grp['% change'] = grp.price.diff()
      ...:     grp['previousdate'] = grp.date.shift(1)
      ...:     return grp

按日期排序，然后groupby并应用change函数，然后对索引进行排序。

In [2460]: df.sort_values('date').groupby('item').apply(change).sort_index()
Out[2460]:
        date item  price  % change previousdate
0 2015-12-31    A    110      10.0   2014-12-31
1 2015-12-31    B    120      20.0   2013-06-24
2 2014-12-31    A    100       NaN          NaT
3 2013-06-24    B    100       NaN          NaT