我正在尝试在 Python 中编写一个充当生成器对象的类,特别是当它被垃圾收集时 .close()
被调用时。这很重要,因为这意味着当生成器被中断时,我可以确保它会自行清理,例如关闭文件或释放锁。
这是一些解释性代码:
如果您中断生成器,那么当它被垃圾回收时,Python 会对生成器对象调用 .close()
,这会向生成器抛出一个 GeneratorExit
错误,该错误可以被捕获允许清理,如下所示:
from threading import Lock
lock = Lock()
def CustomGenerator(n, lock):
lock.acquire()
print("Generator Started: I grabbed a lock")
try:
for i in range(n):
yield i
except GeneratorExit:
lock.release()
print("Generator exited early: I let go of the lock")
raise
print("Generator finished successfully: I let go of the lock")
for i in CustomGenerator(100, lock):
print("Received ", i)
time.sleep(0.02)
if i==3:
break
if not lock.acquire(blocking=False):
print("Oops: Finished, but lock wasn't released")
else:
print("Finished: Lock was free")
lock.release()
Generator Started: I grabbed a lock
Received 0
Received 1
Received 2
Received 3
Generator exited early: I let go of the lock
Finished: Lock was free
但是,如果您尝试通过继承 collections.abc.Generator
来实现自己的生成器对象,Python 似乎没有注意到在收集对象时它应该调用 close:
from collections.abc import Generator
class CustomGeneratorClass(Generator):
def __init__(self, n, lock):
super().__init__()
self.lock = lock
self.lock.acquire()
print("Generator Class Initialised: I grabbed a lock")
self.n = n
self.c = 0
def send(self, arg):
value = self.c
if value >= self.n:
raise StopIteration
self.c += 1
return value
def throw(self, type, value=None, traceback=None):
print("Exception Thrown in Generator: I let go of the lock")
self.lock.release()
raise StopIteration
for i in CustomGeneratorClass(100, lock):
print("Received ", i)
time.sleep(0.02)
if i==3:
break
if not lock.acquire(blocking=False):
print("Oops: Finished, but lock wasn't released")
else:
print("Finished: Lock was free")
lock.release()
Generator Class Initialised: I grabbed a lock
Received 0
Received 1
Received 2
Received 3
Oops: Finished, but lock wasn't released
我认为继承 Generator
足以让 python 相信我的 CustomGeneratorClass 是一个生成器,并且应该在垃圾收集时调用 .close()
。
我认为这与“生成器对象”是某种特殊的生成器
这一事实有关:
from types import GeneratorType
c_gen = CustomGenerator(100)
c_gen_class = CustomGeneratorClass(100)
print("CustomGenerator is a Generator:", isinstance(c_gen, Generator))
print("CustomGenerator is a GeneratorType:",isinstance(c_gen, GeneratorType))
print("CustomGeneratorClass is a Generator:",isinstance(c_gen_class, Generator))
print("CustomGeneratorClass is a GeneratorType:",isinstance(c_gen_class, GeneratorType))
CustomGenerator is a Generator: True
CustomGenerator is a GeneratorType: True
CustomGeneratorClass is a Generator: True
CustomGeneratorClass is a GeneratorType: False
我可以创建一个 GeneratorType
的用户定义类对象吗?
关于 python 如何决定调用 .close()
的内容,有什么我不明白的吗?
如何确保在我的自定义生成器上调用 .close()
?
此问题与 How to write a generator class 不重复。 对于实际创建一个生成器类,该问题的可接受答案确实推荐了我在这里尝试的结构,它是一个生成器类,但没有正确地进行垃圾收集,如上面的代码所示。
最佳答案
PEP342 ,状态:
[generator].__del__()
is a wrapper for[generator].close()
. This will be called when the generator object is garbage-collected ...
collections.abc 中的 Generator 类不实现__del__
,它的父类(super class)或元类也不实现。
将 __del__
的实现添加到问题中的类中会导致锁被释放:
class CustomGeneratorClass(Generator):
...
def __del__(self):
self.close()
输出:
Generator Class Initialised: I grabbed a lock
Recieved 0
Recieved 1
Recieved 2
Recieved 3
Exception Thrown in Generator: I let go of the lock
Finished: Lock was free
警告:
我对 Python 中对象终结的复杂性没有经验,因此应该仔细检查此建议,并进行破坏测试。特别是 language reference 中有关 __del__
的警告应该考虑一下。
更高级别的解决方案是在上下文管理器中运行生成器
with contextlib.closing(CustomGeneratorClass(100, lock)):
# do stuff
但这很麻烦,并且依赖于代码的用户记得这样做。
关于python - 如何创建正确垃圾收集的自定义生成器类,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/58775283/