python - 防止 Pandas read_csv 截断完整时间戳

Question

我在 Mac OS X 上使用 Pandas 0.11。我尝试使用 pandas read_csv 导入 csv 文件，文件中的一列是完整时间戳，其值如下:

fullts
1374087067.357464
1374087067.256206
1374087067.158231
1374087067.074162

我有兴趣获取后续时间戳之间的时间差，因此我导入它并指定 dtype:

    data = read_csv(fn, dtype={'fullts': float64})

但是，pandas 似乎将数字截断为整数部分:

    data.fullts.head(4)

产量:

1374087067
1374087067
1374087067
1374087067

有什么建议吗？

谢谢!

添加:尝试按照建议使用pd.to_datetime，并收到此错误:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-37ed0da45608> in <module>()
---> 1 pd.to_datetime(sd1.fullts)

/Users/user/anaconda/lib/python2.7/site-packages/pandas-0.11.0-py2.7-macosx-10.5-x86_64.egg/pandas/tseries/tools.pyc in to_datetime(arg, errors, dayfirst, utc, box, format)
    102         values = arg.values
    103         if not com.is_datetime64_dtype(values):
--> 104             values = _convert_f(values)
    105         return Series(values, index=arg.index, name=arg.name)
    106     elif isinstance(arg, (np.ndarray, list)):

/Users/user/anaconda/lib/python2.7/site-packages/pandas-0.11.0-py2.7-macosx-10.5-x86_64.egg/pandas/tseries/tools.pyc in _convert_f(arg)
     84             else:
     85                 result = tslib.array_to_datetime(arg, raise_=errors == 'raise',
---> 86                                                  utc=utc, dayfirst=dayfirst)
     87             if com.is_datetime64_dtype(result) and box:
     88                 result = DatetimeIndex(result, tz='utc' if utc else None)
/Users/user/anaconda/lib/python2.7/site-packages/pandas-0.11.0-py2.7-macosx-10.5-x86_64.egg/pandas/tslib.so in pandas.tslib.array_to_datetime (pandas/tslib.c:15411)()

TypeError: object of type 'float' has no len()

Answer 1

从 csv 读取时不需要指定数据类型(默认情况下应使用 float64)。

在 pandas 0.12 中，您可以使用 to_datetime 的单位参数将整数或 float (纪元时间)列转换为 pandas 时间戳。 :

In [11]: df
Out[11]:
         fullts
0  1.374087e+09
1  1.374087e+09
2  1.374087e+09
3  1.374087e+09

In [12]: pd.to_datetime(df.fullts)  # default unit is ns
Out[12]:
0   1970-01-01 00:00:01.374087067
1   1970-01-01 00:00:01.374087067
2   1970-01-01 00:00:01.374087067
3   1970-01-01 00:00:01.374087067
Name: fullts, dtype: datetime64[ns]

In [13]: pd.to_datetime(df.fullts, unit='s')
Out[13]:
0   2013-07-17 18:51:07.357464
1   2013-07-17 18:51:07.256206
2   2013-07-17 18:51:07.158231
3   2013-07-17 18:51:07.074162
Name: fullts, dtype: datetime64[ns]

文档字符串状态:

unit : unit of the arg (D,s,ms,us,ns) denote the unit in epoch
              (e.g. a unix timestamp), which is an integer/float number