gr = []
for i in range(12000): gr.extend([i] * 2)
np.random.seed(0)
df = pd.DataFrame({'gr': gr,
'col1': np.random.choice(200, 24000)})
anyOfThese = np.array([50, 60]) #randomly chosen
t = time()
out = df[df.groupby('gr')['col1'].transform(lambda x: np.any(np.in1d(np.array(x), anyOfThese))).astype(bool)].gr.unique()
print(round(time() - t,2))
>>> 1.87
我需要获取在 col1
中具有任何两个所需值的所有组。
有没有办法更快地实现这一点?我需要重复相同的过程 ~100k 次。
最佳答案
使用boolean indexing
按 isin
过滤:
out = df.loc[df['col1'].isin(anyOfThese), 'gr'].unique()
或通过 numpy.in1d
查看成员(member)资格:
out = df.loc[np.in1d(df['col1'], anyOfThese), 'gr'].unique()
时间:
np.random.seed(218)
gr = []
for i in range(12000):
gr.extend([i] * 2)
np.random.seed(0)
df = pd.DataFrame({'gr': gr,
'col1': np.random.choice(200, 24000)})
anyOfThese = np.array([50, 60]) #randomly chosen
a = df[df.groupby('gr')['col1'].transform(lambda x: np.any(np.in1d(np.array(x), anyOfThese))).astype(bool)].gr.unique()
out = df.loc[df['col1'].isin(anyOfThese), 'gr'].unique()
print ((a == out).all())
True
In [314]: %timeit df[df.groupby('gr')['col1'].transform(lambda x: np.any(np.in1d(np.array(x), anyOfThese))).astype(bool)].gr.unique()
2.9 s ± 79.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [315]: %timeit df.loc[df['col1'].isin(anyOfThese), 'gr'].unique()
746 µs ± 32.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
In [316]: %timeit df.loc[np.in1d(df['col1'], anyOfThese), 'gr'].unique()
325 µs ± 14.9 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
关于python - 获取具有任何所需值的组,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/51155624/