我正在尝试在生成输出时读取通过 subprocess.Popen
运行的“pipenv install”(pipenv==2018.11.26,Python 3.6.0)命令的输出,即整个过程结束后不会,因为根据需要下载的数据量、连接速度等,可能需要很长时间。
我正在打印所有 stdout 和 stderr 消息并为其添加前缀,但我仍然无法理解这种“旋转器”消息:“[== ] 创建虚拟环境...”来自何处.
这是我正在运行的完整代码
import subprocess
import threading
cmd = ['pipenv', 'install','--ignore-pipfile']
cwd = 'test'
def run_cmd(cmd,cwd):
popen = subprocess.Popen(cmd, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,bufsize=1)
a = threading.Thread(target=printTh,args=("stdout",iter(popen.stdout.readline, b"")))
a.start()
b = threading.Thread(target=printTh,args=("stderr",iter(popen.stderr.readline, b"")))
b.start()
while popen.poll() is not None:
a.join()
b.join()
def printTh(pipe_name,iter):
for line in iter:
print(pipe_name+"->"+line.rstrip().decode("utf-8"), end = "\r\n",flush =True)
run_cmd(cmd,cwd)
在我的 GUI 控制台上,我可以看到除了微调器“[ ===] 创建虚拟环境..
”消息之外的所有内容都有一个前缀,该消息既不属于 stderr 也不属于 stdout:
控制台打印:
stderr->Creating a virtualenv for this project…
stderr->Pipfile: C:\Users\ahadu\test\Pipfile
stderr->Using C:/Program Files (x86)/Anaconda3/python.exe (3.6.0) to create virtualenv…
[ ===] Creating virtual environment...Running virtualenv with interpreter C:/Program Files (x86)/Anaconda3/python.exe
stderr->Already using interpreter C:\Program Files (x86)\Anaconda3\python.exe
stderr->Using base prefix 'C:\\Program Files (x86)\\Anaconda3'
stderr-> No LICENSE.txt / LICENSE found in source
stderr->New python executable in C:\Users\ahadu\test\.venv\Scripts\python.exe
stderr->Installing setuptools, pip, wheel...
stderr->done.
stderr->
stderr-Successfully created virtual environment!
stderr->Virtualenv location: C:\Users\ahadu\test\.venv
stdout->Installing dependencies from Pipfile.lock (d34422)…
stdout->To activate this project's virtualenv, run pipenv shell.
stdout->Alternatively, run a command inside the virtualenv with pipenv run.
Process finished with exit code 0
这条消息从哪里来?这也是可能需要一些时间的地方,因此在生成此消息时无法阅读该消息会破坏整个工作。
换句话说,我期望在控制台上看到以下打印内容:
some-source->[= ] Creating virtual environment...
some-source->[ =] Creating virtual environment...
some-source->[= ] Creating virtual environment...
some-source->[ =] Creating virtual environment...
直到完成。
有人知道这个问题的原因以及如何解决吗?
最佳答案
我相信这是因为,对于那些旋转器输出,pipenv 在写入下一行之前将流(stdout/stderr)重置回行的开头。因此,您的前缀被清除。
最初我以为你只需要设置 PIPENV_NOSPIN
和 PIPENV_HIDE_EMOJIS
运行脚本之前的环境变量。虽然这会禁用微调器和表情符号,但有些行仍然没有前缀。 (此外,您不会看到需要时间的操作的任何进度):
temp$ export PIPENV_NOSPIN=1
temp$ export PIPENV_HIDE_EMOJIS=1
temp$ python3.8 test.py
...
stderr>>>>>Pipfile.lock not found, creating...
stderr>>>>>Locking [dev-packages] dependencies...
stderr>>>>>Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
Success!>>>
stderr>>>>>Updated Pipfile.lock (49bf85)!
stdout>>>>>Installing dependencies from Pipfile.lock (49bf85)...
stdout>>>>>To activate this project's virtualenv, run pipenv shell.
stdout>>>>>Alternatively, run a command inside the virtualenv with pipenv run.
如果我们按照“解决依赖关系...”日志到 venv_resolve_deps
function in utils.py ,然后按照 write
method in spin.py 操作,我们会看到这样的几行:
stdout.write(decode_output(u"\r", target_stream=stdout))
现在写入 "\r"
可能会导致如下结果:
>>> print('abc\rdef')
def
先前的输出被清除。由于您的前缀位于开头,因此它也会被清除。请注意,我可能错了,这就是原因,无法真正理解这些行是如何打印出来的,但这是我能找到/想到的唯一解释。
现在,我只需将 text=True
传递到 Popen
即可使您的脚本正常工作。调用,这意味着:
stdin, stdout and stderr will be opened in text mode using the encoding and errors specified in the call or the defaults for io.TextIOWrapper
def run_cmd(cmd, cwd):
popen = subprocess.Popen(
cmd, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize=1,
text=True) # <==== add param here
a = threading.Thread(
target=printTh,
args=("stdout", iter(popen.stdout.readline, ""))) # <===== remove 'b'
a.start()
b = threading.Thread(
target=printTh,
args=("stderr", iter(popen.stderr.readline, ""))) # <===== remove 'b'
b.start()
while popen.poll() is not None:
a.join()
b.join()
def printTh(pipe_name, iter):
for line in iter:
# =================== remove .decode ==============
print(pipe_name + ">>>>>" + line.rstrip(), end="\r\n", flush=True)
run_cmd(cmd, cwd)
结果似乎是你想要的:
stderr>>>>>⠋ Creating virtual environment...
stderr>>>>>⠙ Creating virtual environment...
stderr>>>>>⠹ Creating virtual environment...
stderr>>>>>⠸ Creating virtual environment...
stderr>>>>>⠼ Creating virtual environment...
...
stderr>>>>>
stderr>>>>✔ Successfully created virtual environment!
stderr>>>>>Virtualenv location: /Users/me/.venvs/temp-9JdZcEvf
stderr>>>>>Creating a Pipfile for this project...
stdout>>>>>Installing flask...
stderr>>>>>
stderr>>>>>⠋ Installing...
stderr>>>>>⠙ Installing flask...
stderr>>>>>⠹ Installing flask...
stderr>>>>>⠋ Installing flask...
stderr>>>>>⠙ Installing flask...
stderr>>>>>⠹ Installing flask...
stderr>>>>>⠸ Installing flask...
stderr>>>>>⠼ Installing flask...
stderr>>>>>Adding flask to Pipfile's [packages]...
stderr>>>>✔ Installation Succeeded
stderr>>>>>Pipfile.lock not found, creating...
stderr>>>>>Locking [dev-packages] dependencies...
stderr>>>>>Locking [packages] dependencies...
stderr>>>>>
stderr>>>>>⠋ Locking...
stderr>>>>>Building requirements...
stderr>>>>>
stderr>>>>>Resolving dependencies...
stderr>>>>>
stderr>>>>>⠙ Locking...
stderr>>>>>⠹ Locking...
stderr>>>>>⠸ Locking...
stderr>>>>>⠙ Locking...
stderr>>>>>⠦ Locking...
stderr>>>>>⠧ Locking...
stderr>>>>>⠇ Locking..✔ Success!
stderr>>>>>Updated Pipfile.lock (9536c4)!
stdout>>>>>Installing dependencies from Pipfile.lock (9536c4)...
stdout>>>>>To activate this project's virtualenv, run pipenv shell.
stdout>>>>>Alternatively, run a command inside the virtualenv with pipenv run.
测试使用:
- pipenv,版本 2020.11.15
- Python 3.8
- macOS 10.15.7,带有 bash 5.x 的终端
关于python - 如何使用子进程拦截 "pipenv install"spinner-like 输出消息?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/63342602/