python - StopIteration 会使 python 变慢吗？

Question

<分区>

Answer 1

虽然异常监控在通常情况下有一些小的开销，但在迭代器的情况下，处理 StopIteration 异常似乎没有任何开销。 Python 将迭代器作为一种特殊情况进行优化，因此 StopIteration 不涉及任何异常处理程序。 (我还会观察到——我可能会遗漏一些东西——很难想出一个不隐式使用迭代器的 Python for 循环。

下面是一些示例，首先使用内置的 range 函数和一个简单的 for 循环:

Python 2.7.5
>>> import dis
>>> def x():
...   for i in range(1,11):
...     pass
...
>>> dis.dis(x)
  2           0 SETUP_LOOP              23 (to 26)
              3 LOAD_GLOBAL              0 (range)
              6 LOAD_CONST               1 (1)
              9 LOAD_CONST               2 (11)
             12 CALL_FUNCTION            2
             15 GET_ITER
        >>   16 FOR_ITER                 6 (to 25)
             19 STORE_FAST               0 (i)

  3          22 JUMP_ABSOLUTE           16
        >>   25 POP_BLOCK
        >>   26 LOAD_CONST               0 (None)
             29 RETURN_VALUE

请注意，范围本质上被视为迭代器。

现在，使用一个简单的生成器函数:

>>> def g(x):
...   while x < 11:
...     yield x
...     x = x + 1
...
>>> def y():
...   for i in g(1):
...     pass
...
>>> dis.dis(y)
  2           0 SETUP_LOOP              20 (to 23)
              3 LOAD_GLOBAL              0 (g)
              6 LOAD_CONST               1 (1)
              9 CALL_FUNCTION            1
             12 GET_ITER
        >>   13 FOR_ITER                 6 (to 22)
             16 STORE_FAST               0 (i)

  3          19 JUMP_ABSOLUTE           13
        >>   22 POP_BLOCK
        >>   23 LOAD_CONST               0 (None)
             26 RETURN_VALUE
>>> dis.dis(g)
  2           0 SETUP_LOOP              31 (to 34)
        >>    3 LOAD_FAST                0 (x)
              6 LOAD_CONST               1 (11)
              9 COMPARE_OP               0 (<)
             12 POP_JUMP_IF_FALSE       33

  3          15 LOAD_FAST                0 (x)
             18 YIELD_VALUE
             19 POP_TOP

  4          20 LOAD_FAST                0 (x)
             23 LOAD_CONST               2 (1)
             26 BINARY_ADD
             27 STORE_FAST               0 (x)
             30 JUMP_ABSOLUTE            3
        >>   33 POP_BLOCK
        >>   34 LOAD_CONST               0 (None)
             37 RETURN_VALUE

注意这里的y与上面的x基本相同，区别在于一条LOAD_CONST指令，因为x 引用数字 11。同样，我们的简单生成器基本上等同于 while 循环:

>>> def q():
...   x = 1
...   while x < 11:
...     x = x + 1
...
>>> dis.dis(q)
  2           0 LOAD_CONST               1 (1)
              3 STORE_FAST               0 (x)

  3           6 SETUP_LOOP              26 (to 35)
        >>    9 LOAD_FAST                0 (x)
             12 LOAD_CONST               2 (11)
             15 COMPARE_OP               0 (<)
             18 POP_JUMP_IF_FALSE       34

  4          21 LOAD_FAST                0 (x)
             24 LOAD_CONST               1 (1)
             27 BINARY_ADD
             28 STORE_FAST               0 (x)
             31 JUMP_ABSOLUTE            9
        >>   34 POP_BLOCK
        >>   35 LOAD_CONST               0 (None)
             38 RETURN_VALUE

同样，没有特定的开销来处理迭代器或生成器(range 可能比生成器版本更优化，只是因为它是内置的，而不是因为 Python 的方式处理它)。

最后，让我们看一下使用 StopIteration

编写的实际显式迭代器

>>> class G(object):
...   def __init__(self, x):
...     self.x = x
...   def __iter__(self):
...     return self
...   def next(self):
...     x = self.x
...     if x >= 11:
...       raise StopIteration
...     x = x + 1
...     return x - 1
...
>>> dis.dis(G.next)
  7           0 LOAD_FAST                0 (self)
              3 LOAD_ATTR                0 (x)
              6 STORE_FAST               1 (x)

  8           9 LOAD_FAST                1 (x)
             12 LOAD_CONST               1 (11)
             15 COMPARE_OP               5 (>=)
             18 POP_JUMP_IF_FALSE       30

  9          21 LOAD_GLOBAL              1 (StopIteration)
             24 RAISE_VARARGS            1
             27 JUMP_FORWARD             0 (to 30)

 10     >>   30 LOAD_FAST                1 (x)
             33 LOAD_CONST               2 (1)
             36 BINARY_ADD
             37 STORE_FAST               1 (x)

 11          40 LOAD_FAST                1 (x)
             43 LOAD_CONST               2 (1)
             46 BINARY_SUBTRACT
             47 RETURN_VALUE

现在，在这里我们可以看到生成器函数涉及的指令比这个简单的迭代器少一些，主要与实现的差异和一些与引发 StopIteration 异常相关的指令有关。尽管如此，使用此迭代器的函数完全等同于上面的 y:

>>> def z():
...   for i in G(1):
...     pass
...
>>> dis.dis(z)
  2           0 SETUP_LOOP              20 (to 23)
              3 LOAD_GLOBAL              0 (G)
              6 LOAD_CONST               1 (1)
              9 CALL_FUNCTION            1
             12 GET_ITER
        >>   13 FOR_ITER                 6 (to 22)
             16 STORE_FAST               0 (i)

  3          19 JUMP_ABSOLUTE           13
        >>   22 POP_BLOCK
        >>   23 LOAD_CONST               0 (None)
             26 RETURN_VALUE

当然，这些结果是基于这样一个事实，即 Python for 循环将优化迭代器以消除对 StopIteration 异常的显式处理程序的需要。毕竟，StopIteration 异常本质上构成了 Python for 循环操作的正常部分。

---

关于为什么以这种方式实现，参见PEP-234它定义了迭代器。这专门解决了异常费用的问题:

• It has been questioned whether an exception to signal the end of the iteration isn't too expensive. Several alternatives for the StopIteration exception have been proposed: a special value End to signal the end, a function end() to test whether the iterator is finished, even reusing the IndexError exception.

  • A special value has the problem that if a sequence ever contains that special value, a loop over that sequence will end prematurely without any warning. If the experience with null-terminated C strings hasn't taught us the problems this can cause, imagine the trouble a Python introspection tool would have iterating over a list of all built-in names, assuming that the special End value was a built-in name!

  • Calling an end() function would require two calls per iteration. Two calls is much more expensive than one call plus a test for an exception. Especially the time-critical for loop can test very cheaply for an exception.

  • Reusing IndexError can cause confusion because it can be a genuine error, which would be masked by ending the loop prematurely.