python - 如何在 Python 线程中中止/取消 HTTP 请求？

Question

我正在寻找中止/取消 Python 线程中的 HTTP 请求。我必须坚持使用线程。我不能使用 asyncio 或标准库之外的任何东西。
此代码适用于套接字:

"""Demo for Canceling IO by Closing the Socket

Works!

"""

import socket
import time

from concurrent import futures

start_time = time.time()

sock = socket.socket()


def read():
    "Read data with 10 second delay."
    sock.connect(('httpbin.org', 80))
    sock.sendall(b'GET /delay/10 HTTP/1.0\r\n\r\n')
    while True:
        data = sock.recv(1024)
        if not data:
            break
        print(data.decode(), end='')


with futures.ThreadPoolExecutor() as pool:
    future = pool.submit(read)
    futures.wait([future], timeout=5)
    sock.close()  # <-- Interrupt sock.recv(1024) in Thread:read().

end_time = time.time()
print(f'Duration: {end_time - start_time:.3f}')

# Duration is ~5s as expected.

关闭主线程中的socket用于中断执行器池线程中的recv()。 HTTP 请求应该需要 10 秒，但我们只等待 5 秒然后关闭套接字(有效地取消 HTTP 请求/响应)。
现在我尝试使用 http.client:

"""Demo for Canceling IO in Threads with HTTP Client

Doesn't work!

"""

import time

from concurrent import futures

from http.client import HTTPConnection


def get(con, url):
    con.request('GET', url)
    response = con.getresponse()
    return response


start_time = time.time()

with futures.ThreadPoolExecutor() as executor:
    con = HTTPConnection('httpbin.org')
    future = executor.submit(get, con, '/delay/10')
    done, not_done = futures.wait([future], timeout=5)
    con.sock.close()

end_time = time.time()
print(f'Duration: {end_time - start_time:.3f}')

# Duration is ~10s unfortunately.

不幸的是，这里的总持续时间约为 10 秒。关闭套接字不会中断客户端中的 recv_into()。
好像我做出了一些错误的假设。如何从单独的线程中断 http 客户端中使用的套接字？

Answer 1

您所描述的是预期的有据可查的行为:

Note close() releases the resource associated with a connection but does not necessarily close the connection immediately. If you want to close the connection in a timely fashion, call shutdown() before close().

在 CPython howto docs 中仍然可以找到有关此行为的更多详细信息:

Strictly speaking, you're supposed to use shutdown on a socket before you close it. The shutdown is an advisory to the socket at the other end. Depending on the argument you pass it, it can mean "I'm not going to send anymore, but I'll still listen", or "I'm not listening, good riddance!". Most socket libraries, however, are so used to programmers neglecting to use this piece of etiquette that normally a close is the same as shutdown(); close(). So in most situations, an explicit shutdown is not needed.

One way to use shutdown effectively is in an HTTP-like exchange. The client sends a request and then does a shutdown(1). This tells the server "This client is done sending, but can still receive." The server can detect "EOF" by a receive of 0 bytes. It can assume it has the complete request. The server sends a reply. If the send completes successfully then, indeed, the client was still receiving.

Python takes the automatic shutdown a step further, and says that when a socket is garbage collected, it will automatically do a close if it's needed. But relying on this is a very bad habit. If your socket just disappears without doing a close, the socket at the other end may hang indefinitely, thinking you're just being slow. Please close your sockets when you're done.

解决方案
在关闭之前调用关闭。
例子

with futures.ThreadPoolExecutor() as executor:
    con = HTTPConnection('httpbin.org')
    future = executor.submit(get, con, '/delay/10')
    done, not_done = futures.wait([future], timeout=5)
    con.sock.shutdown()
    con.sock.close()

引用
Python Socket 对象 - 关闭:https://docs.python.org/3/library/socket.html#socket.socket.close
CPython Howto 套接字 - 断开连接:https://github.com/python/cpython/blob/65460565df99fbda6a74b6bb4bf99affaaf8bd95/Doc/howto/sockets.rst#disconnecting