python - 如何改进Python中的for循环迭代

Question

我需要添加一个新列，我将在该列中对导出温度是否较低、正常或较高进行分类(我的数据框中有 100 万行)。

为此，我使用带有 multif 的 for 迭代来对每一行进行分类，如下所示:低温(低于 61)、正常温度(61 到 64 之间)和高温(高于 64)。然而，这 100 万次迭代太多了，而且花费的时间太长，我认为我的电脑卡住了，需要关闭 spy IDE。

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import xlrd
import plotly.offline as py
import plotly.graph_objects as go
import warnings
warnings.filterwarnings('ignore')
def read_date(date):
    return xlrd.xldate.xldate_as_datetime(date, 0)
data1 = pd.read_csv(r'C:\Dados1_14a26_maio.txt', sep=r'\t', engine='python')
data1 = data1.drop('Descartar',axis=1)
data2 = pd.read_csv(r'C:\Dados2_14a26_maio.txt', sep=r'\t', engine='python')
data3 = pd.read_csv(r'C:\Dados3_14a26_maio.txt', sep=r'\t', engine='python')
data3 = data3.drop(['Descartar','Descartar.1','Descartar.2','Descartar.3','Descartar.4','Descartar.5','Descartar.6','Descartar.7','Descartar.8'],axis=1)
DataHora = pd.DataFrame(data1, columns=['Hora'])
DataHora['Hora'] = pd.to_datetime(DataHora['Hora'].apply(read_date), errors='coerce')
data_in = [data1.TentHT, data2.NumVentOn, data3.Tamb]
entrada = pd.concat(data_in, axis=1)
data_out = [data1.TsaidaHT]
saida = pd.concat(data_out, axis=1)
pca_matriz = pd.concat([entrada, saida], axis=1)
cond = pd.DataFrame()
Status = pd.Series([])
for x in saida.index:
    if saida.TsaidaHT[x] < 61: 
        Status[x] = "Low"
    elif saida.TsaidaHT[x] >= 61 and saida.TsaidaHT[x] <= 64: 
        Status[x] = "Normal"
    elif saida.TsaidaHT[x] > 64: 
        Status[x] = "High"
cond.insert(0,"Status",Status)

我想知道是否有办法使 for 和 multi if 的迭代速度更快，以免卡住我的电脑。

Answer 1

您可以使用numpy.select来实现这一点 - 它得到了更好的优化。我花了大约 2-3 秒的时间运行下面的 100 万条虚拟记录。

import pandas as pd
import numpy as np
from random import randint

d = {"temp":[randint(1,100) for _ in range(1000000)]}

df = pd.DataFrame(d)

df["Status"] = np.select([df["temp"]<61, df["temp"]>64],
                         ["Low","High"],
                         default="Normal")
df.tail()

结果:

        temp  Status
999995     8     Low
999996    62  Normal
999997    40     Low
999998     3     Low
999999    48     Low