python - 如何将 pyarrow timestamp dtype 转换为 time64 类型？

Question

我正在尝试将 pyarrow 时间戳类型转换为 time64 类型。但它显示转换错误。

import pyarrow as pa
from datetime import datetime

dt = datetime.now()
table = pa.Table.from_pydict({'ts': pa.array([dt, dt])})
new_schema = table.schema.set(0, pa.field('ts', pa.time64('us')))
table.schema
# ts: timestamp[us]
new_schema
# ts: time64[us]

table.cast(new_schema)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/table.pxi", line 1329, in pyarrow.lib.Table.cast
  File "pyarrow/table.pxi", line 277, in pyarrow.lib.ChunkedArray.cast
  File "/home/inspiron/.virtualenvs/par/lib/python3.7/site-packages/pyarrow/compute.py", line 243, in cast
    return call_function("cast", [arr], options)
  File "pyarrow/_compute.pyx", line 446, in pyarrow._compute.call_function
  File "pyarrow/_compute.pyx", line 275, in pyarrow._compute.Function.call
  File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 105, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: Unsupported cast from timestamp[us] to time64 using function cast_time64

有什么办法可以让这个选角成为可能吗？

Answer 1

time64[us] 是一天中的某个时间。它表示自午夜以来的微秒数。它不与任何特定日期绑定(bind)，并且无法转换为时间戳。

Arrow 文档有点稀疏，但 parquet docs更好地解释一下:

TIME

TIME is used for a logical time type without a date with millisecond or microsecond precision. The type has two type parameters: UTC adjustment (true or false) and unit (MILLIS or MICROS, NANOS).

TIME with unit MILLIS is used for millisecond precision. It must annotate an int32 that stores the number of milliseconds after midnight.

TIME with unit MICROS is used for microsecond precision. It must annotate an int64 that stores the number of microseconds after midnight.

TIME with unit NANOS is used for nanosecond precision. It must annotate an int64 that stores the number of nanoseconds after midnight.

The sort order used for TIME is signed.