我在使用 pandas 重新采样功能时遇到问题。我有分钟采样数据,我正在尝试以 0.7 秒的频率重新采样。我尝试使用带有“700L”选项的重新采样,但行为不符合预期。 我举了一个例子:
import pandas as pd
from datetime import datetime
import pytz
import numpy as np
import matplotlib.pyplot as plt
def convert_2_datetime(timestamp, timezoneid):
"""
:param timestamp: UTC format in milliseconds (data.index.values)
:param timezoneid: timezone object from CTX (for example pytz.timezone(ctx.inp.assets[0].properties['timezoneid']))
:return: vector of datetimes
"""
if isinstance(timestamp,int) or isinstance(timestamp,float):
utctime = datetime.utcfromtimestamp(timestamp / 1000).replace(tzinfo=pytz.utc)
output = utctime.astimezone(pytz.timezone(timezoneid.zone))
else:
utctime = [datetime.utcfromtimestamp(i / 1000).replace(tzinfo=pytz.utc) for i in timestamp]
output = [i.astimezone(pytz.timezone(timezoneid.zone)) for i in utctime]
return output
# minute sampled data
v1 = [0.0,0.0,0.0,0.0,0.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,0.0,0.0,0.0,0.0,0.0]
data = pd.DataFrame({'v1':np.array(v1)},index=np.arange(start=1,stop=len(v1)+1)*60000)
data['ts']= convert_2_datetime(timestamp=data.index.values,timezoneid=pytz.timezone('UTC'))
data.set_index('ts',inplace=True)
data07 = data.resample(rule='700L',closed={'right','left'}).interpolate(method='linear')
data06 = data.resample(rule='600L',closed={'right','left'}).interpolate(method='linear')
data11 = data.resample(rule='1100L',closed={'right','left'}).interpolate(method='linear')
plt.show()
data07.v1.plot(style='.',label='700 ms')
data06.v1.plot(style='.', label = '600 ms')
data11.v1.plot(style='.', label = '1500 ms')
data.v1.plot(style='x',label='original')
plt.legend()
print('Finish!')
如果我用“600L”重新采样,最终结果是正确的(示例中的data06); “700L”(示例中的 data07)不正确。见下图:
我缺少有关重新采样函数的信息?
非常感谢大家!
最佳答案
解决方法
就您而言,我认为您应该在插值之前对方法重新采样,例如在 mean
上重新采样。我相信这与 resample
的输出以及 interpolate
的读取方式有关。例如,以下内容似乎有效:
data07 = data.resample('700L').mean().interpolate()
data06 = data.resample('600L').mean().interpolate()
data10 = data.resample('1000L').mean().interpolate()
此图表明它有效:
data07.v1.plot(style='.',label='700 ms', alpha=0.75, ms=3,zorder=2)
data06.v1.plot(style='^',label='600 ms', alpha=0.5, zorder=1)
data10.v1.plot(style='^',label='1000 ms', alpha=0.5, zorder=0, ms=10)
data.v1.plot(style='x',label='original', ms=10)
plt.legend()
解释(有点...):
当您使用任何方法(包括 mean()
)对数据进行重新采样时,无论您的数据在何处重新采样,您都会得到 NaN
:
>>> data.resample('700L').mean().head()
v1
ts
1970-01-01 00:00:59.500000+00:00 0.0
1970-01-01 00:01:00.200000+00:00 NaN
1970-01-01 00:01:00.900000+00:00 NaN
1970-01-01 00:01:01.600000+00:00 NaN
1970-01-01 00:01:02.300000+00:00 NaN
当您调用interpolate
时,它将用适当的线性插值填充NaN
。
>>> data.resample('700l').mean().interpolate().head()
v1
ts
1970-01-01 00:00:59.500000+00:00 0.0
1970-01-01 00:01:00.200000+00:00 0.0
1970-01-01 00:01:00.900000+00:00 0.0
1970-01-01 00:01:01.600000+00:00 0.0
1970-01-01 00:01:02.300000+00:00 0.0
当您直接在 resample
的输出上调用 interpolate
时,interpolate
的行为似乎不符合预期,给出了一堆开头为 NaN,然后是从最大值 (1) 向下逐渐倾斜。不太确定为什么:
>>> data.resample('700l').interpolate().head()
v1
ts
1970-01-01 00:00:59.500000+00:00 NaN
1970-01-01 00:01:00.200000+00:00 NaN
1970-01-01 00:01:00.900000+00:00 NaN
1970-01-01 00:01:01.600000+00:00 NaN
1970-01-01 00:01:02.300000+00:00 NaN
关于python - Pandas 重采样函数问题从分钟到毫秒的重新采样,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/50724231/