我有一个数据框访问
=
visit_occurrence_id visit_start_date person_id
1 2016-06-01 1
2 2019-05-01 2
3 2016-01-22 1
4 2017-02-14 2
5 2018-05-11 3
和另一个数据框测量
=
measurement_date person_id visit_occurrence_id
2017-09-04 1 Nan
2018-04-24 2 Nan
2018-05-22 2 Nan
2019-02-02 1 Nan
2019-01-28 3 Nan
2019-05-07 1 Nan
2018-12-11 3 Nan
2017-04-28 3 Nan
我想根据 person_id 和可能的最近日期,用访问表的visit_occurrence_id 填充测量表的visit_occurrence_id。
我已经写了一个代码,但是花了很多时间。
measurement has 7*10^5 rows.
Note: visit_start_date and measurement_date are object types
my code -
import datetime as dt
unique_person_list = measurement['person_id'].unique().tolist()
def nearest_date(row,date_list):
date_list = [dt.datetime.strptime(date, '%Y-%m-%d').date() for date in date_list]
row = min(date_list, key=lambda x: abs(x - row))
return row
modified_measurement = pd.DataFrame(columns = measurement.columns)
for person in unique_person_list:
near_visit_dates = visit[visit['person_id']==person]['visit_start_date'].tolist()
if near_visit_dates:
near_visit_dates = list(filter(None, near_visit_dates))
near_visit_dates = [i.strftime('%Y-%m-%d') for i in near_visit_dates]
store_dates = measurement.loc[measurement['person_id']== person]['measurement_date']
store_dates= store_dates.apply(nearest_date, args=(near_visit_dates,))
modified_measurement = modified_measurement.append(store_dates)
我的代码的执行时间相当长。您能否帮助我降低时间复杂度或使用其他解决方案。
编辑 - 添加数据框构造函数。
import numpy as np
measurement = {'measurement_date':['2017-09-04', '2018-04-24', '2018-05-22', '2019-02-02',
'2019-01-28', '2019-05-07', '2018-12-11','2017-04-28'],
'person_id':[1, 2, 2, 1, 3, 1, 3, 3],'visit_occurrence_id':[np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan,np.nan]}
visit = {'visit_occurrence_id':[1, 2, 3, 4, 5],
'visit_start_date':['2016-06-01', '2019-05-01', '2016-01-22', '2017-02-14', '2018-05-11'],
'person_id':[1, 2, 1, 2, 3]}
# Create DataFrame
measurement = pd.DataFrame(measurement)
visit = pd.DataFrame(visit)
最佳答案
您可以执行以下操作:
df=pd.merge(measurement[["person_id", "measurement_date"]], visit, on="person_id", how="inner")
df["dt_diff"]=df[["visit_start_date", "measurement_date"]].apply(lambda x: abs(datetime.datetime.strptime(x["visit_start_date"], '%Y-%m-%d').date() - datetime.datetime.strptime(x["measurement_date"], '%Y-%m-%d').date()), axis=1)
df=pd.merge(df, df.groupby(["person_id", "measurement_date"])["dt_diff"].min(), on=["person_id", "dt_diff", "measurement_date"], how="inner")
res=pd.merge(measurement, df, on=["measurement_date", "person_id"], suffixes=["", "_2"])[["measurement_date", "person_id", "visit_occurrence_id_2"]]
输出:
measurement_date person_id visit_occurrence_id_2
0 2017-09-04 1 1
1 2018-04-24 2 2
2 2018-05-22 2 2
3 2019-02-02 1 1
4 2019-01-28 3 5
5 2019-05-07 1 1
6 2018-12-11 3 5
7 2017-04-28 3 5
关于python - 如何使用另一个数据框中最近的日期填充一个数据框中的日期列,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/59408809/