c# - 如何从USB摄像头获取的每一帧中提取时间戳？

Question

这种场景在实时视频处理中很常见。我需要时间戳来与其他设备同步。

我已尝试cv::VideoCapture，但它无法从视频流中提取时间戳。

所以我有两个问题:

1. USB摄像头提供的视频流是否确实包含时间戳信息？
2. 如果有的话。我应该怎么做才能提取它？ C# 解决方案是最好的，而 C++ 也可以。

添加:

使用这两个属性不起作用:

secCounter = (long) cap.get(CAP_PROP_POS_MSEC);
frameNumber = (long) cap.get(CAP_PROP_POS_FRAMES);

它总是给出以下结果:

VIDEOIO ERROR: V4L2: getting property #1 is not supported

msecCounter = 0

frameNumber = -1

Answer 1

OpenCV 的 VideoCapture 类是一个非常高级的接口(interface)，用于从相机检索帧，因此它“隐藏”了连接到相机、从相机检索帧并将这些帧解码为有用的颜色空间所需的许多细节，例如BGR。这很好，因为您不必担心抓取帧的细节，但缺点是您无法直接访问您可能需要的其他数据，例如帧编号或帧时间戳。但这并不意味着不可能获得您想要的数据!

这是一个示例帧抓取循环，它大致基于 example code from here 为您提供所需的内容。 。这是在 C++ 中。

#include "opencv2/opencv.hpp"
using namespace cv;
int main(int, char**)
{
    VideoCapture cap(0); // open the default camera
    if(!cap.isOpened())  // check if we succeeded
        return -1;

    // TODO: change the width, height, and capture FPS to your desired
    // settings.
    cap.set(CAP_PROP_FRAME_WIDTH, 1920);
    cap.set(CAP_PROP_FRAME_HEIGHT, 1080);
    cap.set(CAP_PROP_FPS, 30);

    Mat frame;
    long msecCounter = 0;
    long frameNumber = 0;

    for(;;)
    {            
        // Instead of cap >> frame; we'll do something different.
        //
        // VideoCapture::grab() tells OpenCV to grab a frame from
        // the camera, but to not worry about all the color conversion
        // and processing to convert that frame into BGR.
        //
        // This means there's less processing overhead, so the time
        // stamp will be more accurate because we are fetching it
        // immediately after.
        //
        // grab() should also wait for the next frame to be available
        // based on the capture FPS that is set, so it's okay to loop
        // continuously over it.

        if(cap.grab())
        {
            msecCounter = (long) cap.get(CAP_PROP_POS_MSEC);
            frameNumber = (long) cap.get(CAP_PROP_POS_FRAMES);

            // VideoCapture::retrieve color converts the image and places
            // it in the Mat that you provide.
            if(cap.retrieve(&frame))
            {
                // Pass the frame and parameters to your processing
                // method.
                ProcessFrame(&frame, msecCounter, frameNumber);
            }
        }

        // TODO: Handle your loop termination condition here
    }
    // the camera will be deinitialized automatically in VideoCapture destructor
    return 0;
}

void ProcessFrame(Mat& frame, long msecCounter, long frameNumber)
{
    // TODO: Make a copy of frame if you are going to process it
    // asynchronously or put it in a buffer or queue and then return
    // control from this function. This is because the reference Mat
    // being passed in is "owned" by the processing loop, and on each
    // iteration it will be destructed, so any references to it will be
    // invalid. Hence, if you do any work async, you need to copy frame.
    //
    // If all your processing happens synchronously in this function,
    // you don't need to make a copy first because the loop is waiting
    // for this function to return.

    // TODO: Your processing logic goes here.
}

如果您使用 C# 和 Emgu CV它看起来会有点不同。我还没有测试过这段代码，但它应该可以工作或者非常接近解决方案。

using System;
using Emgu.CV;
using Emgu.CV.CvEnum;
static class Program
{
    [STAThread]
    static void Main()
    {
        VideoCapture cap = new VideoCapture(0);
        if(!cap.IsOpened)
        {
            return;
        }

        cap.SetCaptureProperty(CapProp.FrameWidth, 1920);
        cap.SetCaptureProperty(CapProp.FrameHeight, 1080);
        cap.SetCaptureProperty(CapProp.Fps, 30);

        Mat frame = new Mat();            
        long msecCounter = 0;
        long frameNumber = 0;

        for(;;)
        {
            if(cap.Grab())
            {
                msecCounter = (long) cap.GetCaptureProperty(CapProp.PosMsec);
                frameNumber = (long) cap.GetCaptureProperty(CapProp.PosFrames);

                if(cap.Retrieve(frame))
                {
                    ProcessFrame(frame, msecCounter, frameNumber);
                }
            }

            // TODO: Determine when to quit the processing loop
        }
    }

    private static void ProcessFrame(Mat frame, long msecCounter, long frameNumber)
    {
        // Again, copy frame here if you're going to queue the frame or
        // do any async processing on it.

        // TODO: Your processing code goes here.
    }
}

Emgu 的 VideoCapture实现还允许异步Grab为您完成的操作，以及当抓取的帧准备好与Retrieve一起使用时的通知。看起来像这样:

using System;
using Emgu.CV;
using Emgu.CV.CvEnum;
static class Program
{
    private static Mat s_frame;
    private static VideoCapture s_cap;
    private static object s_retrieveLock = new object();

    [STAThread]
    static void Main()
    {
        s_cap = new VideoCapture(0);
        if(!s_cap.IsOpened)
        {
            return;
        }

        s_frame = new Mat();

        s_cap.SetCaptureProperty(CapProp.FrameWidth, 1920);
        s_cap.SetCaptureProperty(CapProp.FrameHeight, 1080);
        s_cap.SetCaptureProperty(CapProp.Fps, 30);

        s_cap.ImageGrabbed += FrameIsReady;

        s_cap.Start();

        // TODO: Wait here until you're done with the capture process,
        // the same way you'd determine when to exit the for loop in the
        // above example.

        s_cap.Stop();
        s_cap.ImageGrabbed -= FrameIsReady;
    }

    private static void FrameIsReady(object sender, EventArgs e)
    {
        // This function is being called from VideoCapture's thread,
        // so if you rework this code to run with a UI, be very careful
        // about updating Controls here because that needs to be Invoke'd
        // back to the UI thread.

        // I used a lock here to be extra careful and protect against
        // re-entrancy, but this may not be necessary if Emgu's
        // VideoCapture thread blocks for completion of this event
        // handler.
        lock(s_retrieveLock)
        {
            msecCounter = (long) s_cap.GetCaptureProperty(CapProp.PosMsec);
            frameNumber = (long) s_cap.GetCaptureProperty(CapProp.PosFrames);

            if(s_cap.Retrieve(s_frame))
            {
                ProcessFrame(s_frame, msecCounter, frameNumber);
            }
        }
    }

    private static void ProcessFrame(Mat frame, long msecCounter, long frameNumber)
    {
        // Again, copy frame here if you're going to queue the frame or
        // do any async processing on it.

        // TODO: Your processing code goes here.
    }
}