我正在使用 pygraphviz 为不同的数据配置创建大量图表。我发现无论在图形中放入什么信息,程序在绘制第 170 个图形后都会崩溃。程序停止时没有生成错误消息。如果绘制这么多图形,是否需要重新设置?
我在 Windows 10 机器、Pygraphviz 1.5 和 graphviz 2.38 上运行 Python 3.7
for graph_number in range(200):
config_graph = pygraphviz.AGraph(strict=False, directed=False, compound=True, ranksep='0.2', nodesep='0.2')
# Create Directory
if not os.path.exists('Graph'):
os.makedirs('Graph')
# Draw Graph
print('draw_' + str(graph_number))
config_graph.layout(prog = 'dot')
config_graph.draw('Graph/'+str(graph_number)+'.png')
最佳答案
我能够通过以下方式不断重现该行为:
我怀疑代码中某处存在未定义行为( UB ),即使行为是 与 相同:
做了一些调试(在 agraph.py 和 cgraph.dll (write.c) 中添加了一些 print(f) 语句)。 PyGraphviz 调用 Graphviz 的工具 (.exes) 进行许多操作。为此,它使用 subprocess.Popen 并通过其 3 个可用流(stdin、stdout、stderr)与子进程通信。
从一开始我就注意到
170 * 3 = 510
(非常接近 512 (0x200)),但直到后来才注意到我应该有的关注(主要是因为 Python 进程(运行下面的代码) 没有超过 ~150在任务管理器 (TM) 和进程资源管理器 (PE) 中打开句柄)。然而,一些谷歌搜索显示:
C run-time I/O now supports up to 8,192 files open simultaneously at the low I/O level. This level includes files opened and accessed using the _open, _read, and _write family of I/O functions. By default, up to 512 files can be open simultaneously at the stream I/O level. This level includes files opened and accessed using the fopen, fgetc, and fputc family of functions. The limit of 512 open files at the stream I/O level can be increased to a maximum of 8,192 by use of the _setmaxstdio function.
以下是我为调试和重现错误而修改的代码。它需要(为了代码简洁,因为同样的事情可以通过 CTypes 实现)PyWin32 包(
python -m pip install pywin32
)。代码00.py:
#!/usr/bin/env python
import sys
import os
#import time
import pygraphviz as pgv
import win32file as wfile
def handle_graph(idx, dir_name):
graph_name = "draw_{0:03d}".format(idx)
graph_args = {
"name": graph_name,
"strict": False,
"directed": False,
"compound": True,
"ranksep": "0.2",
"nodesep": "0.2",
}
graph = pgv.AGraph(**graph_args)
# Draw Graph
img_base_name = graph_name + ".png"
print(" {0:s}".format(img_base_name))
graph.layout(prog="dot")
img_full_name = os.path.join(dir_name, img_base_name)
graph.draw(img_full_name)
graph.close() # !!! Has NO (visible) effect, but I think it should be called anyway !!!
def main(*argv):
print("OLD max open files: {0:d}".format(wfile._getmaxstdio()))
# 513 is enough for your original code (170 graphs), but you can set it up to 8192
wfile._setmaxstdio(513) # !!! COMMENT this line to reproduce the crash !!!
print("NEW max open files: {0:d}".format(wfile._getmaxstdio()))
dir_name = "Graph"
# Create Directory
if not os.path.isdir(dir_name):
os.makedirs(dir_name)
#ts_global_start = time.time()
start = 0
count = 169
#count = 1
step_sleep = 0.05
for i in range(start, start + count):
#ts_local_start = time.time()
handle_graph(i, dir_name)
#print(" Time: {0:.3f}".format(time.time() - ts_local_start))
#time.sleep(step_sleep)
handle_graph(count, dir_name)
#print("Global time: {0:.3f}".format(time.time() - ts_global_start - step_sleep * count))
if __name__ == "__main__":
print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
main(*sys.argv[1:])
print("\nDone.")
输出 :e:\Work\Dev\StackOverflow\q060876623>"e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code00.py Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32 OLD max open files: 512 NEW max open files: 513 draw_000.png draw_001.png draw_002.png ... draw_167.png draw_168.png draw_169.png Done.
结论 :
3 * (graph_count + 1) <= max_fds
,您可以了解数字。从那里,如果您将限制设置为 8192(我没有对此进行测试),您应该能够处理 2729 个图形(假设代码没有打开额外的 fds)旁注 :
关于python - Pygraphviz 绘制 170 个图形后崩溃,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/60876623/