我有一个超过 15000 行的数据框对象 df
,例如:
anime_id name genre rating
1234 Kimi no nawa Romance, Comedy 9.31
5678 Stiens;Gate Sci-fi 8.92
我正在尝试查找具有特定 anime_id 的行。
a_id = "5678"
temp = (df.query("anime_id == "+a_id).genre)
我只是想知道这个搜索是在常数时间(如字典)还是线性时间(如列表)中完成的。
最佳答案
这是一个非常有趣的问题!
我觉得取决于以下几个方面:
按索引访问单行(索引已排序且唯一)应该有运行时 O(m)
其中 m << n_rows
按索引访问单行(索引不唯一且未排序)应该有运行时 O(n_rows)
按索引访问单行(索引不唯一且已排序)应该有运行时 O(m)
其中 m < n_rows
)
通过 bool 索引访问行(独立于索引)应该有运行时 O(n_rows)
演示:
索引是有序且唯一的:
In [49]: df = pd.DataFrame(np.random.rand(10**5,6), columns=list('abcdef'))
In [50]: %timeit df.loc[random.randint(0, 10**4)]
The slowest run took 27.65 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 331 µs per loop
In [51]: %timeit df.iloc[random.randint(0, 10**4)]
1000 loops, best of 3: 275 µs per loop
In [52]: %timeit df.query("a > 0.9")
100 loops, best of 3: 7.84 ms per loop
In [53]: %timeit df.loc[df.a > 0.9]
100 loops, best of 3: 2.96 ms per loop
索引未排序且不唯一:
In [54]: df = pd.DataFrame(np.random.rand(10**5,6), columns=list('abcdef'), index=np.random.randint(0, 10000, 10**5))
In [55]: %timeit df.loc[random.randint(0, 10**4)]
100 loops, best of 3: 12.3 ms per loop
In [56]: %timeit df.iloc[random.randint(0, 10**4)]
1000 loops, best of 3: 262 µs per loop
In [57]: %timeit df.query("a > 0.9")
100 loops, best of 3: 7.78 ms per loop
In [58]: %timeit df.loc[df.a > 0.9]
100 loops, best of 3: 2.93 ms per loop
索引不是唯一的且已排序:
In [64]: df = pd.DataFrame(np.random.rand(10**5,6), columns=list('abcdef'), index=np.random.randint(0, 10000, 10**5)).sort_index()
In [65]: df.index.is_monotonic_increasing
Out[65]: True
In [66]: %timeit df.loc[random.randint(0, 10**4)]
The slowest run took 9.70 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 478 µs per loop
In [67]: %timeit df.iloc[random.randint(0, 10**4)]
1000 loops, best of 3: 262 µs per loop
In [68]: %timeit df.query("a > 0.9")
100 loops, best of 3: 7.81 ms per loop
In [69]: %timeit df.loc[df.a > 0.9]
100 loops, best of 3: 2.95 ms per loop
关于python - Pandas DataFrame 搜索是线性时间还是常数时间?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/45240803/