寻找替代方案,因为 OpenCV 不会为实时相机流(在 Windows 上) 提供时间戳,而这在我的计算机视觉算法中是必需的,我找到了 ffmpeg 和这个优秀文章https://zulko.github.io/blog/2013/09/27/read-and-write-video-frames-in-python-using-ffmpeg/ 该解决方案使用 ffmpeg,访问其标准输出 (stdout) 流。我扩展它以读取标准错误 (stderr) 流。
在 Windows 上运行 python 代码,虽然我从 ffmpeg stdout 接收到视频帧,但在为第一帧提供 showinfo videofilter 详细信息(时间戳)后 stderr 卡住。
我记得在 ffmpeg 论坛的某处看到重定向时会绕过像 showinfo 这样的视频过滤器。这就是以下代码无法按预期工作的原因吗?
预期:它应该将视频帧写入磁盘并打印时间戳详细信息。
实际:它写入视频文件但没有获取时间戳(showinfo)详细信息。
这是我试过的代码:
import subprocess as sp
import numpy
import cv2
command = [ 'ffmpeg',
'-i', 'e:\sample.wmv',
'-pix_fmt', 'rgb24',
'-vcodec', 'rawvideo',
'-vf', 'showinfo', # video filter - showinfo will provide frame timestamps
'-an','-sn', #-an, -sn disables audio and sub-title processing respectively
'-f', 'image2pipe', '-'] # we need to output to a pipe
pipe = sp.Popen(command, stdout = sp.PIPE, stderr = sp.PIPE) # TODO someone on ffmpeg forum said video filters (e.g. showinfo) are bypassed when stdout is redirected to pipes???
for i in range(10):
raw_image = pipe.stdout.read(1280*720*3)
img_info = pipe.stderr.read(244) # 244 characters is the current output of showinfo video filter
print "showinfo output", img_info
image1 = numpy.fromstring(raw_image, dtype='uint8')
image2 = image1.reshape((720,1280,3))
# write video frame to file just to verify
videoFrameName = 'Video_Frame{0}.png'.format(i)
cv2.imwrite(videoFrameName,image2)
# throw away the data in the pipe's buffer.
pipe.stdout.flush()
pipe.stderr.flush()
那么如何仍然从 ffmpeg 中获取帧时间戳到 python 代码中,以便它可以在我的计算机视觉算法中使用...
最佳答案
重定向 stderr 在 python 中有效。
所以不是这个 pipe = sp.Popen(command, stdout = sp.PIPE, stderr = sp.PIPE)
这样做 pipe = sp.Popen(command, stdout = sp.PIPE, stderr = sp.STDOUT)
我们可以通过添加异步调用来读取 ffmpeg 的标准流(stdout 和 stderr)来避免重定向。这将避免视频帧和时间戳的任何混合,从而避免容易出错的分离。
所以修改原始代码以使用 threading
模块将如下所示:
# Python script to read video frames and timestamps using ffmpeg
import subprocess as sp
import threading
import matplotlib.pyplot as plt
import numpy
import cv2
ffmpeg_command = [ 'ffmpeg',
'-nostats', # do not print extra statistics
#'-debug_ts', # -debug_ts could provide timestamps avoiding showinfo filter (-vcodec copy). Need to check by providing expected fps TODO
'-r', '30', # output 30 frames per second
'-i', 'e:\sample.wmv',
'-an','-sn', #-an, -sn disables audio and sub-title processing respectively
'-pix_fmt', 'rgb24',
'-vcodec', 'rawvideo',
#'-vcodec', 'copy', # very fast!, direct copy - Note: No Filters, No Decode/Encode, no quality loss
#'-vframes', '20', # process n video frames only. For Debugging
'-vf', 'showinfo', # showinfo videofilter provides frame timestamps as pts_time
'-f', 'image2pipe', 'pipe:1' ] # outputs to stdout pipe. can also use '-' which is redirected to pipe
# seperate method to read images on stdout asynchronously
def AppendProcStdout(proc, nbytes, AppendList):
while proc.poll() is None: # continue while the process is alive
AppendList.append(proc.stdout.read(nbytes)) # read image bytes at a time
# seperate method to read image info. on stderr asynchronously
def AppendProcStderr(proc, AppendList):
while proc.poll() is None: # continue while the process is alive
try: AppendList.append(proc.stderr.next()) # read stderr until empty
except StopIteration: continue # ignore stderr empty exception and continue
if __name__ == '__main__':
# run ffmpeg command
pipe = sp.Popen(ffmpeg_command, stdout=sp.PIPE, stderr=sp.PIPE)
# 2 threads to talk with ffmpeg stdout and stderr pipes
framesList = [];
frameDetailsList = []
appendFramesThread = threading.Thread(group=None, target=AppendProcStdout, name='FramesThread', args=(pipe, 1280*720*3, framesList), kwargs=None, verbose=None) # assuming rgb video frame with size 1280*720
appendInfoThread = threading.Thread(group=None, target=AppendProcStderr, name='InfoThread', args=(pipe, frameDetailsList), kwargs=None, verbose=None)
# start threads to capture ffmpeg frames and info.
appendFramesThread.start()
appendInfoThread.start()
# wait for few seconds and close - simulating cancel
import time; time.sleep(2)
pipe.terminate()
# check if threads finished and close
appendFramesThread.join()
appendInfoThread.join()
# save an image per 30 frames to disk
savedList = []
for cnt,raw_image in enumerate(framesList):
if (cnt%30 != 0): continue
image1 = numpy.fromstring(raw_image, dtype='uint8')
image2 = image1.reshape((720,1280,3)) # assuming rgb image with size 1280 X 720
# write video frame to file just to verify
videoFrameName = 'video_frame{0}.png'.format(cnt)
cv2.imwrite(videoFrameName,image2)
savedList.append('{} {}'.format(videoFrameName, image2.shape))
print '### Results ###'
print 'Images captured: ({}) \nImages saved to disk:{}\n'.format(len(framesList), savedList) # framesList contains all the video frames got from the ffmpeg
print 'Images info captured: \n', ''.join(frameDetailsList) # this contains all the timestamp details got from the ffmpeg showinfo videofilter and some initial noise text which can be easily removed while parsing
关于python - 如何在 Windows 上从 ffmpeg 获取实时视频帧和时间戳到 python,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/42230269/