python - python 中的 next() 真的那么快吗？

Question

通过这里的一篇帖子，我了解到使用 next() 来搜索和检索列表中第一次出现的元素可能会很快。然而，我很惊讶地看到传统的 for-if-break 语法在相当长的一段时间内表现得更好。如果我在分析中犯了错误，请纠正我。 这是我尝试过的片段:

>>> def compare_2():
...     device = 'a'
...     l = ['a', 'b', 'c', 'd']
...     z = next((device for x in l if x==device), None)

>>> def compare_1():
...     device = 'a'
...     l = ['a', 'b', 'c', 'd']
...     z = None
...     for x in l:
...             if x == device:
...                     z = device
...                     break

>>> import timeit
>>> t = timeit.Timer(setup='from __main__ import compare_2', stmt='compare_2()')
>>> t.timeit()
1.5207240581512451
>>> t = timeit.Timer(setup='from __main__ import compare_1', stmt='compare_1()')
>>> t.timeit()
0.46623396873474121

我认为这可能会发生，因为我试图搜索和检索列表中的第一个元素作为示例。我还尝试了最后一个元素，并注意到 next() 的性能并没有比以前更好。

>>> def compare_2():
...     device = 'd'
...     l = ['a', 'b', 'c', 'd']
...     z = next((device for x in l if x==device), None)
...
>>>
>>> def compare_1():
...     device = 'd'
...     l = ['a', 'b', 'c', 'd']
...     z = None
...     for x in l:
...             if x == device:
...                     z = device
...                     break
...

>>>
>>> t = timeit.Timer(setup='from __main__ import compare_2', stmt='compare_2()')
>>> t.timeit()
1.6903998851776123
>>> t = timeit.Timer(setup='from __main__ import compare_1', stmt='compare_1()')
>>> t.timeit()
0.66585493087768555

很想知道在优化代码方面何时实际使用 next() 以及何时不使用。 谢谢!

更新: if device in l 肯定会更快。 我实际上只是想制作一个简单案例的原型(prototype)。我在尝试根据属性匹配从对象列表中检索对象时遇到此问题。例如: obj = next(obj for obj in obj_list if obj.value == 1)

Answer 1

我想知道是否还有其他事情发生。创建生成器会产生一些开销，但我认为将条件 if x==device 放入生成器会强制生成整个列表，并在 next() 之前创建一个新列表能跑。

请参阅此示例，比较强制创建新列表的列表推导式和惰性且不强制创建的生成器:

>>> from timeit import Timer
>>> # List comprehension forces a new list to be created in memory
>>> def f1():
...     q = [x for x in xrange(1000)]
...     r = q[1]
...     return r
... 
>>> # Generator comprehension does 'lazy' iteration, only when needed
>>> def f2():
...     q = (x for x in xrange(1000))
...     r = next(q)
...     return r
... 
>>> Timer(f1).timeit()
47.420308774268435
>>> Timer(f2).timeit()
1.346566078497844

看到列表推导速度很慢，生成器惰性方法意味着它仅在您调用 next() 时开始迭代，获取一个值并停止。

现在这个例子，唯一的变化是都用if x = 999取最后一个元素:

>>> # List comprehension still forces creation of a new list
>>> # although the list only ends up with one element
>>> # nb. it's the last element
>>> def f1():
...     q = [x for x in xrange(1000) if x == 999]
...     r = q[0]
...     return r
... 
>>> # Generator comprehension is lazy
>>> # nb. it also only returns the last element
>>> def f2():
...     q = (x for x in xrange(1000) if x == 999)
...     r = next(q)
...     return r
... 
>>> Timer(f1).timeit()
37.279105355189984
>>> Timer(f2).timeit()
37.46816399778598

看他们现在基本一样了。发电机已经减速了。条件迫使它做与列表理解相同的事情，它不能在不评估整个列表的情况下懒惰地只接受一个匹配的东西。

所以我认为在您的示例中，您不是只是看到创建生成器然后在其他人回答时调用它的开销，正如我最初的评论所说。

我认为通过包含 if x==device 条件，您将强制生成器构造迭代整个列表，并创建一个新的列表对象，< em>并用所有结果填充它，然后在那个新列表上创建一个生成器，然后调用它来获取结果。

因此，与遍历现有列表的 for 循环相比，很多的开销要大，这并不是因为 next() 本身就很慢。

编辑:您可以在将生成器表达式添加到 Python 的提案中看到它:PEP-0289 - Generator Expressions , 在关于 Early Binding vs Late Binding 的部分

Asked to summarize the reasoning for binding the first expression, Guido offered [5] :

Consider sum(x for x in foo()). Now suppose there's a bug in foo() that raises an exception, and a bug in sum() that raises an exception before it starts iterating over its argument. Which exception would you expect to see? I'd be surprised if the one in sum() was raised rather the one in foo(), since the call to foo() is part of the argument to sum(), and I expect arguments to be processed before the function is called.

OTOH, in sum(bar(x) for x in foo()), where sum() and foo() are bugfree, but bar() raises an exception, we have no choice but to delay the call to bar() until sum() starts iterating -- that's part of the contract of generators. (They do nothing until their next() method is first called.)

换句话说，如果 x==device 将抛出异常，因为无法比较列表中的一项，例如来自自定义对象的类型错误，您可能希望在调用 next() 之前看到该异常，从而强制迭代整个列表，失去您可能希望看到的生成器惰性的保存，并创建更多列表对象创建开销与 for 循环相比。