Python 优化 - 我可以避免双重 for 循环吗？

Question

我有一个如下所示的 df:

import datetime as dt
import pandas as pd
import pytz

cols = ['utc_datetimes', 'zone_name']
data = [
    ['2019-11-13 14:41:26,2019-12-18 23:04:12', 'Europe/Stockholm'],
    ['2019-12-06 21:49:04,2019-12-11 22:52:57,2019-12-18 20:30:58,2019-12-23 18:49:53,2019-12-27 18:34:23,2020-01-07 21:20:51,2020-01-11 17:36:56,2020-01-20 21:45:47,2020-01-30 20:48:49,2020-02-03 21:04:52,2020-02-07 20:05:02,2020-02-10 21:07:21', 'Europe/London']
]

df = pd.DataFrame(data, columns=cols)
print(df)


#                             utc_datetimes             zone_name
# 0 2019-11-13 14:41:26,2019-12-18 23:04:12             Europe/Stockholm
# 1 2019-12-06 21:49:04,2019-12-11 22:52:57,2019-1...   Europe/London

我想计算行本地时间的夜晚数和星期三数，df 中的日期表示。这是所需的输出:

                              utc_datetimes             zone_name        nights wednesdays
0   2019-11-13 14:41:26,2019-12-18 23:04:12             Europe/Stockholm    0   1
1   2019-12-06 21:49:04,2019-12-11 22:52:57,2019-1...   Europe/London      11   2

我提出了以下双 for 循环，但对于相当大的 df 来说，它的效率不如我希望的那样:

# New columns.
df['nights'] = 0
df['wednesdays'] = 0

for row in range(df.shape[0]):
    date_list = df['utc_datetimes'].iloc[row].split(',')
    user_time_zone = df['zone_name'].iloc[row]

    for date in date_list:
        datetime_obj = dt.datetime.strptime(
            date, '%Y-%m-%d %H:%M:%S'
        ).replace(tzinfo=pytz.utc)
        local_datetime = datetime_obj.astimezone(pytz.timezone(user_time_zone))

        # Get day of the week count:
        if local_datetime.weekday() == 2:
            df['wednesdays'].iloc[row] += 1

        # Get time of the day count:
        if (local_datetime.hour >17) & (local_datetime.hour <= 23):
            df['nights'].iloc[row] += 1 

如有任何建议，我们将不胜感激:)

PD。忽略“夜晚”的定义，只是一个例子。

Answer 1

一种方法是首先通过展开 utc_datetimes 列来创建引用 df，然后获取每个区域的 TimeDelta:

df = pd.DataFrame(data, columns=cols)

s = (df.assign(utc_datetimes=df["utc_datetimes"].str.split(","))
       .explode("utc_datetimes"))

s["diff"] = [pd.Timestamp(a, tz=b).utcoffset() for a,b in zip(s["utc_datetimes"],s["zone_name"])]

使用这个助手 df，您可以计算星期三和晚上的数量:

df["wednesdays"] = (pd.to_datetime(s["utc_datetimes"])+s["diff"]).dt.day_name().eq("Wednesday").groupby(level=0).sum()
df["nights"] = ((pd.to_datetime(s["utc_datetimes"])+s["diff"]).dt.hour>17).groupby(level=0).sum()

print (df)

#
                                       utc_datetimes         zone_name  wednesdays  nights
0            2019-11-13 14:41:26,2019-12-18 23:04:12  Europe/Stockholm         1.0     0.0
1  2019-12-06 21:49:04,2019-12-11 22:52:57,2019-1...     Europe/London         2.0    11.0