我有不规则间隔的时间序列数据。我有总能源使用量和能源使用的持续时间。
Start Date Start Time Duration (Hours) Usage(kWh)
1/3/2016 12:28:00 PM 2.233333333 6.23
1/3/2016 4:55:00 PM 1.9 11.45
1/4/2016 6:47:00 PM 7.216666667 11.93
1/4/2016 7:00:00 AM 3.45 9.45
1/4/2016 7:26:00 AM 1.6 7.33
1/4/2016 7:32:00 AM 1.6 4.54
我想计算 15 分钟窗口内所有负载曲线的总和。我可以在必要时四舍五入(例如,最近的 1 分钟)。我不能立即使用重新采样,因为它会将使用量平均到下一个时间戳中,在第一个条目 1/3 12:28 PM 的情况下,将需要 6.23 kWh 并均匀分布到下午 4:55,这是不准确的。 6.23 kWh 应该分布到 12:28 PM + 2.23 hrs ~= 2:42 PM。
最佳答案
这是一个简单的实现,它只设置了一个系列,
result
,其索引具有分钟频率,然后循环遍历行
df
(使用 df.itertuples
)并为每个添加适当的功率
关联区间中的行:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame({'Duration (Hours)': [2.233333333, 1.8999999999999999, 7.2166666670000001, 3.4500000000000002, 1.6000000000000001, 1.6000000000000001], 'Start Date': ['1/3/2016', '1/3/2016', '1/4/2016', '1/4/2016', '1/4/2016', '1/4/2016'], 'Start Time': ['12:28:00 PM', '4:55:00 PM', '6:47:00 PM', '7:00:00 AM', '7:26:00 AM', '7:32:00 AM'], 'Usage(kWh)': [6.2300000000000004, 11.449999999999999, 11.93, 9.4499999999999993, 7.3300000000000001, 4.54]} )
df['duration'] = pd.to_timedelta(df['Duration (Hours)'], unit='H')
df['start_date'] = pd.to_datetime(df['Start Date'] + ' ' + df['Start Time'])
df['end_date'] = df['start_date'] + df['duration']
df['power (kW/min)'] = df['Usage(kWh)']/(df['Duration (Hours)']*60)
df = df.drop(['Start Date', 'Start Time', 'Duration (Hours)'], axis=1)
result = pd.Series(0,
index=pd.date_range(df['start_date'].min(), df['end_date'].max(), freq='T'))
power_idx = df.columns.get_loc('power (kW/min)')+1
for row in df.itertuples():
result.loc[row.start_date:row.end_date] += row[power_idx]
# The sum of the usage over 15 minute windows is computed using the `resample/sum` method:
usage = result.resample('15T').sum()
usage.plot(kind='line', label='usage')
plt.legend(loc='best')
plt.show()
关于性能的注意事项:遍历 df
的行不是很方便
快,尤其是 len(df)
很大的时候。为了获得更好的性能,您可能需要一个
more clever method , 它处理
以矢量化方式“同时”处理所有行:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
# Here is an example using a larger DataFrame
N = 10**3
dates = pd.date_range('2016-1-1', periods=N*10, freq='H')
df = pd.DataFrame({'Duration (Hours)': np.random.uniform(1, 10, size=N),
'start_date': np.random.choice(dates, replace=False, size=N),
'Usage(kWh)': np.random.uniform(1,20, size=N)})
df['duration'] = pd.to_timedelta(df['Duration (Hours)'], unit='H')
df['end_date'] = df['start_date'] + df['duration']
df['power (kW/min)'] = df['Usage(kWh)']/(df['Duration (Hours)']*60)
def using_loop(df):
result = pd.Series(0,
index=pd.date_range(df['start_date'].min(), df['end_date'].max(), freq='T'))
power_idx = df.columns.get_loc('power (kW/min)')+1
for row in df.itertuples():
result.loc[row.start_date:row.end_date] += row[power_idx]
usage = result.resample('15T').sum()
return usage
def using_cumsum(df):
result = pd.melt(df[['power (kW/min)','start_date','end_date']],
id_vars=['power (kW/min)'], var_name='usage', value_name='date')
result['usage'] = result['usage'].map({'start_date':1, 'end_date':-1})
result['usage'] *= result['power (kW/min)']
result = result.set_index('date')
result = result[['usage']].resample('T').sum().fillna(0).cumsum()
usage = result.resample('15T').sum()
return usage
usage = using_cumsum(df)
usage.plot(kind='line', label='usage')
plt.legend(loc='best')
plt.show()
len(df)
等于 1000 时,using_cumsum
比 using_loop
快 10 倍以上:
In [117]: %timeit using_loop(df)
1 loop, best of 3: 545 ms per loop
In [118]: %timeit using_cumsum(df)
10 loops, best of 3: 52.7 ms per loop
关于python - 在 Pandas 中重采样和归一化不规则时间序列数据,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/39078835/