python - Python中的 `await`是否会屈服于事件循环？

Question

我想知道当我们用异步Python代码await一个协程时到底发生了什么，例如:

await send_message(string)

(1)将send_message添加到事件循环中，并且调用协程放弃对事件循环的控制，或者

(2)我们直接跳到o​​jit_code

我读过的大多数explanations都指向(1)，因为它们将调用协程描述为退出。但是我自己的实验表明情况(2)是这样的:我试图在调用者之后但在被调用者之前运行协程，但无法实现这一点。

Answer 1

免责声明:自从我到达这里寻找自己的答案以来，有待纠正(尤其是关于细节和正确的术语)。尽管如此，下面的研究指出了一个相当具有决定性的“要点”结论:
正确的OP答案:不，await(本身)不屈服于事件循环，yield屈服于事件循环，因此对于给定的情况:“(2)我们直接跳入send_message”。特别是，某些yield表达式是底部实际上可以切换出异步任务的唯一点(就确定可以暂停Python代码执行的确切位置而言)。
有待证明:1)理论/文档； 2)实现代码； 3)示例。
通过理论/文献
PEP 492:具有async和await语法的协程

While the PEP is not tied to any specific Event Loop implementation, it is relevant only to the kind of coroutine that uses yield as a signal to the scheduler, indicating that the coroutine will be waiting until an event (such as IO) is completed. ...

[await] uses the yield from implementation [with an extra step of validating its argument.] ...

Any yield from chain of calls ends with a yield. This is a fundamental mechanism of how Futures are implemented. Since, internally, coroutines are a special kind of generators, every await is suspended by a yield somewhere down the chain of await calls (please refer to PEP 3156 for a detailed explanation). ...

Coroutines are based on generators internally, thus they share the implementation. Similarly to generator objects, coroutines have throw(), send() and close() methods. ...

The vision behind existing generator-based coroutines and this proposal is to make it easy for users to see where the code might be suspended.

在上下文中，“使用户容易看到代码可能被挂起的地方”似乎是指在同步代码中yield是可以在允许其他代码运行的例程中“挂起”执行的地方，现在扩展到异步上下文，其中yield(如果其值未在正在运行的任务中使用，而是传播到调度程序中)是“切换到调度程序的信号”。
更简洁地说:发电机的产量控制在哪里？在yield处。协程(包括使用async和await语法的协程)是生成器，因此也是如此。
它不仅是类比，在实现中(见下文)，任务“进入”和“离开”协程的实际机制并不是异步世界的新事物，魔幻事物或独特事物，而仅仅是通过调用科罗的 <generator>.send() 方法。这是(据我所理解的文字)PEP 492背后的“愿景”的一部分:async和await不会提供任何新的代码暂停机制，而只是将异步糖放到Python已经广受欢迎的强大生成器上。
和
PEP 3156:“异步”模块

The loop.slow_callback_duration attribute controls the maximum execution time allowed between two yield points before a slow callback is reported [emphasis in original].

也就是说，从异步的角度来看，不间断的代码段被划分为两个连续的yield点(其值达到了运行的Task级别(通过await/yield from隧道)的值，而没有在其中使用)。
和这个:

The scheduler has no public interface. You interact with it by using yield from future and yield from task.

异议:“那是'yield from'，但是您试图争辩说该任务只能在yield本身上切换!yield from和yield是不同的东西，我的 friend ，而且yield from本身不会暂停代码!”
答:不是矛盾。 PEP表示您通过使用yield from future/task与调度程序进行交互。但是，正如上面在PEP 492中指出的那样，任何yield from(〜aka await)链最终都会到达yield(“底乌龟”)。特别是(请参阅下文)，yield from future实际上在完成一些包装工作之后就将yield等同于相同的future，并且yield是实际的“转出点”，在该点上，另一项任务将接管。但是，将代码直接yield直到当前的Future直接对Task进行编码是不正确的，因为您将绕过必要的包装程序。
我已经从上面的引用中提出了反对意见，并指出了它的实际编码注意事项:在Python异步代码中合适的yield最终是一件事，它以标准方式暂停执行代码，任何其他yield都会执行的操作，现在进一步使调度程序参与可能的任务切换。
通过实现代码
asyncio/futures.py

class Future:
...
    def __await__(self):
        if not self.done():
            self._asyncio_future_blocking = True
            yield self  # This tells Task to wait for completion.
        if not self.done():
            raise RuntimeError("await wasn't used with future")
        return self.result()  # May raise too.

    __iter__ = __await__  # make compatible with 'yield from'.

释义:yield self行是告诉正在运行的任务暂时搁置并让其他任务运行的过程，在self完成后的某个时间返回到这一行。asyncio世界中几乎所有可等待的对象都是Future周围的包装(多层包装)。事件循环对于所有更高级别的await awaitable表达式始终完全是盲目的，直到代码执行滴加到await future或yield from future，然后(如此处所示)调用yield self，然后生成的self被“捕获”了除以外的Task。当前协程堆栈正在运行，从而向任务发出信号以暂停。
在yield self上下文中，上述“代码在await future内的asyncio处挂起”规则的唯一且唯一的异常(exception)是在yield中可能使用裸asyncio.sleep(0)。而且由于sleep函数是本文评论中的主题，因此让我们来看一下。
asyncio/tasks.py

@types.coroutine
def __sleep0():
    """Skip one event loop run cycle.
    This is a private helper for 'asyncio.sleep()', used
    when the 'delay' is set to 0.  It uses a bare 'yield'
    expression (which Task.__step knows how to handle)
    instead of creating a Future object.
    """
    yield


async def sleep(delay, result=None, *, loop=None):
    """Coroutine that completes after a given time (in seconds)."""
    if delay <= 0:
        await __sleep0()
        return result

    if loop is None:
        loop = events.get_running_loop()
    else:
        warnings.warn("The loop argument is deprecated since Python 3.8, "
                      "and scheduled for removal in Python 3.10.",
                      DeprecationWarning, stacklevel=2)

    future = loop.create_future()
    h = loop.call_later(delay,
                        futures._set_result_unless_cancelled,
                        future, result)
    try:
        return await future

    finally:
        h.cancel()

注意:这里有两种有趣的情况，控制可以转移到调度程序:
(1)yield中的裸__sleep0(通过await调用时)。
(2)yield self就在await future之内。
asyncio/tasks.py中的关键行(对于我们而言)是当Task._step通过result = self._coro.send(None)运行其顶级协程并识别以下四种情况:
(1)result = None由coro(再次是生成器)生成:任务“放弃对一个事件循环迭代的控制”。
(2)result = future是在coro内生成的，进一步的魔术成员领域证据表明， future 是从Future.__iter__ == Future.__await__之外以适当的方式产生的:任务将控制权交给事件循环，直到将来完成。
(3)由coro发出StopIteration，指示协程完成(即，作为生成器，它用尽了所有yield):任务的最终结果(本身就是Future)被设置为协程返回值。
(4)发生其他任何Exception:相应地设置了任务的set_exception。
模数细节，我们关心的重点是asyncio事件循环中的协程段最终通过coro.send()运行。除了最初的启动和最终终止，send()精确地从它生成的最后一个yield值进行到下一个。
举个例子

import asyncio
import types

def task_print(s):
    print(f"{asyncio.current_task().get_name()}: {s}")

async def other_task(s):
    task_print(s)

class AwaitableCls:
    def __await__(self):
        task_print("    'Jumped straight into' another `await`; the act of `await awaitable` *itself* doesn't 'pause' anything")
        yield
        task_print("    We're back to our awaitable object because that other task completed")
        asyncio.create_task(other_task("The event loop gets control when `yield` points (from an iterable coroutine) propagate up to the `current_task` through a suitable chain of `await` or `yield from` statements"))

async def coro():
    task_print("  'Jumped straight into' coro; the `await` keyword itself does nothing to 'pause' the current_task")
    await AwaitableCls()
    task_print("  'Jumped straight back into' coro; we have another pending task, but leaving an `__await__` doesn't 'pause' the task any more than entering the `__await__` does")

@types.coroutine
def iterable_coro(context):
    task_print(f"`{context} iterable_coro`: pre-yield")
    yield None # None or a Future object are the only legitimate yields to the task in asyncio
    task_print(f"`{context} iterable_coro`: post-yield")

async def original_task():
    asyncio.create_task(other_task("Aha, but a (suitably unconsumed) *`yield`* DOES 'pause' the current_task allowing the event scheduler to `_wakeup` another task"))

    task_print("Original task")
    await coro()
    task_print("'Jumped straight out of' coro. Leaving a coro, as with leaving/entering any awaitable, doesn't give control to the event loop")
    res = await iterable_coro("await")
    assert res is None
    asyncio.create_task(other_task("This doesn't run until the very end because the generated None following the creation of this task is consumed by the `for` loop"))
    for y in iterable_coro("for y in"):
        task_print(f"But 'ordinary' `yield` points (those which are consumed by the `current_task` itself) behave as ordinary without relinquishing control at the async/task-level; `y={y}`")
    task_print("Done with original task")

asyncio.get_event_loop().run_until_complete(original_task())

在python3.8中运行产生

Task-1: Original task

Task-1: 'Jumped straight into' coro; the await keyword itself does nothing to 'pause' the current_task

Task-1: 'Jumped straight into' another await; the act of await awaitable itself doesn't 'pause' anything

Task-2: Aha, but a (suitably unconsumed) yield DOES 'pause' the current_task allowing the event scheduler to _wakeup another task

Task-1: We're back to our awaitable object because that other task completed

Task-1: 'Jumped straight back into' coro; we have another pending task, but leaving an __await__ doesn't 'pause' the task any more than entering the __await__ does

Task-1: 'Jumped straight out of' coro. Leaving a coro, as with leaving/entering any awaitable, doesn't give control to the event loop

Task-1: await iterable_coro: pre-yield

Task-3: The event loop gets control when yield points (from an iterable coroutine) propagate up to the current_task through a suitable chain of await or yield from statements

Task-1: await iterable_coro: post-yield

Task-1: for y in iterable_coro: pre-yield

Task-1: But 'ordinary' yield points (those which are consumed by the current_task itself) behave as ordinary without relinquishing control at the async/task-level; y=None

Task-1: for y in iterable_coro: post-yield

Task-1: Done with original task

Task-4: This doesn't run until the very end because the generated None following the creation of this task is consumed by the for loop

实际上，诸如以下的练习可以帮助人们将async/await的功能与“事件循环”等概念脱钩。前者有利于后者的良好实现和用法，但您可以将async和await用作特殊语法的生成器，而无需任何“循环”(无论是asyncio还是其他方式):

import types # no asyncio, nor any other loop framework

async def f1():
    print(1)
    print(await f2(),'= await f2()')
    return 8

@types.coroutine
def f2():
    print(2)
    print((yield 3),'= yield 3')
    return 7

class F3:
   def __await__(self):
        print(4)
        print((yield 5),'= yield 5')
        print(10)
        return 11

task1 = f1()
task2 = F3().__await__()
""" You could say calls to send() represent our
   "manual task management" in this script.
"""
print(task1.send(None), '= task1.send(None)')
print(task2.send(None), '= task2.send(None)')
try:
    print(task1.send(6), 'try task1.send(6)')
except StopIteration as e:
    print(e.value, '= except task1.send(6)')
try:
    print(task2.send(9), 'try task2.send(9)')
except StopIteration as e:
    print(e.value, '= except task2.send(9)')

产生

1

2

3 = task1.send(None)

4

5 = task2.send(None)

6 = yield 3

7 = await f2()

8 = except task1.send(6)

9 = yield 5

10

11 = except task2.send(9)