python - Pandas 根据函数返回单独的 DataFrame 值

Question

我有两个 DataFrame，df1 是地点位置，df2 是车站位置。我试图找到一种更有效的方法来应用距离函数来查找哪些车站在一定范围内并返回车站的名称。如果距离函数是 +/- 1 的纬度差，这是我的预期结果:

# df1
   Lat  Long 
0   30    31    
1   37    48    
2   54    62    
3   67    63     

# df2
   Station_Lat  Station_Long Station
0           30            32     ABC    
1           43            48     DEF    
2           84            87     GHI    
3           67            62     JKL    

# ....Some Code that compares df1 and df2....

# result
   Lat  Long  Station_Lat  Station_Long Station
    30    31           30            32     ABC
    67    63           67            62     JKL

我有一个使用 cartesian product 的解决方案/Cross Join 在单个 DataFrame 上应用函数。这个解决方案有效，但是我在真实数据集中有数百万行，这使得笛卡尔积非常慢。

import pandas as pd
df1 = pd.DataFrame({'Lat' : [30, 37, 54, 67],
                    'Long' : [31, 48, 62, 63]})

df2 = pd.DataFrame({'Station_Lat' : [30, 43, 84, 67],
                    'Station_Long' : [32, 48, 87, 62],
                    'Station':['ABC', 'DEF','GHI','JKL']})

# creating a 'key' for a cartesian product
df1['key'] = 1
df2['key'] = 1

# Creating the cartesian Join
df3 = pd.merge(df1, df2, on='key')

# some distance function that returns True or False
# assuming the distance function I want is +/- 1 of two values
def some_distance_func(x,y):
    return x-y >= -1 and x-y <= 1

# applying the function to a column using vectorized approach
# https://stackoverflow.com/questions/52673285/performance-of-pandas-apply-vs-np-vectorize-to-create-new-column-from-existing-c
df3['t_or_f'] =  list(map(some_distance_func,df3['Lat'],df3['Station_Lat']))

# result
print(df3.loc[df3['t_or_f']][['Lat','Long','Station_Lat','Station_Long','Station']].reset_index(drop=True))

我还尝试了 iterrows() 的循环方法，但这比交叉连接方法慢。有没有更Pythonic/更有效的方法来实现我正在寻找的东西？

Answer 1

您可以使用pd.cut函数来指定包含纬度的适当间隔，并简单地合并两个数据帧以获得结果:

bins = [(i-1,i+1) for i in df1['Lat']]
bins = [item for subbins in bins for item in subbins]

df1['Interval'] = pd.cut(df1['Lat'], bins=bins)
df2['Interval'] = pd.cut(df2['Station_Lat'], bins=bins)

pd.merge(df1,df2)

此解决方案比您的解决方案稍快。 10.2 ms ± 201 µs per loop与 12.2 ms ± 1.34 ms per loop 。