python - 向 dask 数据帧添加新列会引发 ValueError : Length of values does not match length of index

Question

我知道这个回溯

ValueError:值的长度与索引的长度不匹配

源于这样一个事实:一个dataframe is longer or shorter than the other ddf.assign(new_col=ts_col) 期间的 dataframe 或 ddf['ts_col'] = ts_col 中的相同操作。

问题是，我看不出长度有何不同 - 用代码解释:

from dask import dataframe as dd

# Read data
ddf = dd.read_csv(csv_path)
ddf.persist()


# Convert to unixtimestamp to pandas timestamp
ts_col = pd.to_datetime(ddf.ts_unixtime_sec_prec, unit='s', errors='coerce')
ts_col.fillna()

# Check data
> ts_col[0:2]
< DatetimeIndex(['2019-05-23 09:09:56', '2019-05-23 09:09:56'], dtype='datetime64[ns]', freq=None)

# Checking length
> len(ddf.index) 
< 11227296

> len(ts_col)
< 11227296

# Try to assign it to dataframe.
> ddf['ts_col'] = ts_col
< ValueError: Length of values does not match length of index <<< Error

Answer 1

让它与 lambda/map 函数一起使用:

df['ts'] = df['ts'].map(lambda x: pd.to_datetime(x, errors='coerce'))

Source