我有一个像这样的数据框 df
project_ID country prj_start prj_end revenue profit
2131 USA 201603 201703 100000 30000
5124 UK 201502 201606 1500 1000
1245 UK 201010 201710 1800 1000
我想找出每个月和每个国家/地区的活跃项目数量，并计算它们的收入和利润。输出看起来像这样
Month country active_projects revenue profit
201603 USA 15 500000 100000
201603 UK 20 150000 100000
201604 Germany 30 1000000 500000
我的第一门编程语言是 C++，所以我倾向于使用循环来做事。我几乎成功地找到了一个解决方案，我创建了这样的月份时段。
#making a monthlist dataframe with count column to hold no. of active projects
monthlist = pd.DataFrame(columns= ["months","count"])
#making a new dataframe to insert the results into
newdf = pd.DataFrame(columns=["month", "country","active_prj_count","rev","gp"])
#making the month slots, not concerned with future values
monthlist['months']=pd.date_range(start = min(df['prj_start']), end =datetime.date.today(), freq='M').map(lambda x: 100*x.year + x.month)
monthlist['count']=0
#traversing through the original dataframe and monthlist to insert a new row into newdf
#everytime the project start is less than and prj end is greater than the month slot
i=0
for y in range(len(df)):
for x in range(len(monthlist)):
if(df.loc[y,'prj_start']<=monthlist.loc[x,'months'] & df.loc[y,'prj_end']>=monthlist.loc[x,'months']):
monthlist.loc[x,'count']=monthlist.loc[x,'count']+1
newdf.loc[i] = [monthlist.loc[x,'months'],df.loc[y,'country']
,monthlist.loc[x,'count'],df.loc[y,'revenue'],df.loc[y,'profit']]
i=i+1
这个解决方案可行，但我不得不承认它不是很智能且计算效率不高。需要一段时间来处理。有人想通过可能使用 pandas 或 numpy 函数来改进代码吗？
最佳答案
您可以对每一行应用函数并提取每个项目出现的日期，然后按月份和国家/地区汇总。
>>> df
project_ID country prj_start prj_end revenue profit
0 2131 USA 201603 201703 100000 30000
1 5124 UK 201502 201606 1500 1000
2 1245 UK 201010 201710 1800 1000
让我们添加更多样本，每个月有不同的国家/地区:
>>> df_new = pd.DataFrame([
[1111, 'Germany',201603, 201703,1000, 4000],
[4111, 'Germany',201603, 201703,4000, 6000],
[3112, 'Germany',201010, 201703,4000, 6000],
[2112, 'Germany',201603, 201703,4000, 6000],
[2116, 'Germany',201502, 201710,4000, 6000]],
columns=df.columns)
>>> df_new
project_ID country prj_start prj_end revenue profit
0 1111 Germany 201603 201703 1000 4000
1 4111 Germany 201603 201703 4000 6000
2 3112 Germany 201010 201703 4000 6000
3 2112 Germany 201603 201703 4000 6000
4 2116 Germany 201502 201710 4000 6000
>>> df_ = pd.concat([df,df_new],axis=0,ignore_index=True)
project_ID country prj_start prj_end revenue profit
0 2131 USA 201603 201703 100000 30000
1 5124 UK 201502 201606 1500 1000
2 1245 UK 201010 201710 1800 1000
3 1111 Germany 201603 201703 1000 4000
4 4111 Germany 201603 201703 4000 6000
5 3112 Germany 201010 201703 4000 6000
6 2112 Germany 201603 201703 4000 6000
7 2116 Germany 201502 201710 4000 6000
将prj_start和prj_end转换为datetime，并指明要解析的格式format="%Y%m":
>>> df_[['prj_start','prj_end']] = df_[['prj_start','prj_end']].apply(pd.to_datetime, format="%Y%m")
>>> df_
project_ID country prj_start prj_end revenue profit
0 2131 USA 2016-03-01 2017-03-01 100000 30000
1 5124 UK 2015-02-01 2016-06-01 1500 1000
2 1245 UK 2010-10-01 2017-10-01 1800 1000
3 1111 Germany 2016-03-01 2017-03-01 1000 4000
4 4111 Germany 2016-03-01 2017-03-01 4000 6000
5 3112 Germany 2010-10-01 2017-03-01 4000 6000
6 2112 Germany 2016-03-01 2017-03-01 4000 6000
7 2116 Germany 2015-02-01 2017-10-01 4000 6000
现在让我们定义一个函数来转换行并应用它:
def transform_row(row):
date_index = pd.date_range(row['prj_start'].min(),
row['prj_end'].max(), freq='MS')
row_out = pd.DataFrame(np.repeat(row.values,
len(date_index.values),axis=0),
index=date_index, columns=row.columns)
row_out.index.name = 'date'
return row_out.reset_index()
df_transformed = pd.concat([transform_row(row.to_frame().T)
for i,row in df_.iterrows()],axis=0)
然后，最后应用 pivot_table 按国家和日期聚合值:
df1 = pd.pivot_table(df_transformed,
index=['date','country'],
values=['revenue','profit'],
aggfunc=np.sum,fill_value=0)
df2 = pd.pivot_table(df_transformed,
index=['date','country'],
values=['project_ID'],
aggfunc=len,fill_value=0)
最后拼接datafame得到按月的数据:
pd.concat([df1,df2],axis=1)
profit revenue project_ID
date country
2010-10-01 Germany 6000 4000 1
UK 1000 1800 1
2010-11-01 Germany 6000 4000 1
UK 1000 1800 1
2010-12-01 Germany 6000 4000 1
UK 1000 1800 1
2011-01-01 Germany 6000 4000 1
UK 1000 1800 1
2011-02-01 Germany 6000 4000 1
UK 1000 1800 1
2011-03-01 Germany 6000 4000 1
UK 1000 1800 1
2011-04-01 Germany 6000 4000 1
UK 1000 1800 1
2011-05-01 Germany 6000 4000 1
UK 1000 1800 1
2011-06-01 Germany 6000 4000 1
UK 1000 1800 1
2011-07-01 Germany 6000 4000 1
UK 1000 1800 1
2011-08-01 Germany 6000 4000 1
UK 1000 1800 1
2011-09-01 Germany 6000 4000 1
UK 1000 1800 1
2011-10-01 Germany 6000 4000 1
UK 1000 1800 1
2011-11-01 Germany 6000 4000 1
UK 1000 1800 1
2011-12-01 Germany 6000 4000 1
UK 1000 1800 1
... ... ... ...
2016-10-01 USA 30000 100000 1
2016-11-01 Germany 28000 17000 5
UK 1000 1800 1
USA 30000 100000 1
2016-12-01 Germany 28000 17000 5
UK 1000 1800 1
USA 30000 100000 1
2017-01-01 Germany 28000 17000 5
UK 1000 1800 1
USA 30000 100000 1
2017-02-01 Germany 28000 17000 5
UK 1000 1800 1
USA 30000 100000 1
2017-03-01 Germany 28000 17000 5
UK 1000 1800 1
USA 30000 100000 1
2017-04-01 Germany 6000 4000 1
UK 1000 1800 1
2017-05-01 Germany 6000 4000 1
UK 1000 1800 1
2017-06-01 Germany 6000 4000 1
UK 1000 1800 1
2017-07-01 Germany 6000 4000 1
UK 1000 1800 1
2017-08-01 Germany 6000 4000 1
UK 1000 1800 1
2017-09-01 Germany 6000 4000 1
UK 1000 1800 1
2017-10-01 Germany 6000 4000 1
UK 1000 1800 1
关于时间序列的Python聚合，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/48785833/