java - 使用 JavaCV 库的图像旋转问题

Question

我正在编写一个 Android 应用程序，用于在指定的时间内录制视频。如果我使用智能手机的后置摄像头进行录制，一切正常。该应用程序具有暂停/记录功能，就像在 Vine 应用程序中一样。使用设备的前置摄像头录制时会出现问题。存储/播放视频倒置时，视频表面框架看起来很好。到处都有很多关于这个问题的讨论。但我没有找到任何有效的解决方案。

看看下面提到的代码和图像。

这是从前置摄像头拍摄的原始图像。为了更好地查看，我将其倒置。

这是轮换后我实际得到的:

方法:

     IplImage copy = cvCloneImage(image);
     IplImage rotatedImage = cvCreateImage(cvGetSize(copy), copy.depth(), copy.nChannels());
     //Define Rotational Matrix
     CvMat mapMatrix = cvCreateMat(2, 3, CV_32FC1);

     //Define Mid Point
     CvPoint2D32f centerPoint = new CvPoint2D32f();
     centerPoint.x(copy.width() / 2);
     centerPoint.y(copy.height() / 2);

     //Get Rotational Matrix
     cv2DRotationMatrix(centerPoint, angle, 1.0, mapMatrix);

     //Rotate the Image
     cvWarpAffine(copy, rotatedImage, mapMatrix, CV_INTER_CUBIC + CV_WARP_FILL_OUTLIERS, cvScalarAll(170));
     cvReleaseImage(copy);
     cvReleaseMat(mapMatrix);

我试过

     double angleTemp = angle;

     angleTemp= ((angleTemp / 90)%4)*90;       
     final int number = (int) Math.abs(angleTemp/90);

     for(int i = 0; i != number; ++i){            
         cvTranspose(rotatedImage, rotatedImage);
         cvFlip(rotatedImage, rotatedImage, 0);           
     }

最终抛出异常，指出源和目标与列数和行数不匹配。

更新:

视频就是这样录制的。

IplImage newImage = null;
if(cameraSelection == CameraInfo.CAMERA_FACING_FRONT){
    newImage = videoRecorder.rotate(yuvIplImage, 180);
    videoRecorder.record(newImage);
}
else
    videoRecorder.record(yuvIplImage);  

旋转是这样完成的:

    IplImage img = IplImage.create(image.height(), image.width(),
            image.depth(), image.nChannels());

    for (int i = 0; i < 180; i++) {
        cvTranspose(image, img);
        cvFlip(img, img, 0);
    }

如果您以前遇到过这种情况，谁能指出这里出了什么问题？

Answer 1

看到您已经有了一个 IplImage，您可能会发现它很有用。我修改了这个Open Source Android Touch-To-Record library的onPreviewFrame方法对捕获的帧进行转置和调整大小。

我在我的 setCameraParams() 方法中定义了“yuvIplImage”。

IplImage yuvIplImage = IplImage.create(mPreviewSize.height, mPreviewSize.width, opencv_core.IPL_DEPTH_8U, 2);

也像这样初始化录像机，发送宽度作为高度，反之亦然:

//call initVideoRecorder() method like this to initialize videoRecorder object of FFmpegFrameRecorder class.
initVideoRecorder(strVideoPath, mPreview.getPreviewSize().height, mPreview.getPreviewSize().width, recorderParameters);

//method implementation
public void initVideoRecorder(String videoPath, int width, int height, RecorderParameters recorderParameters)
{
    Log.e(TAG, "initVideoRecorder");

    videoRecorder = new FFmpegFrameRecorder(videoPath, width, height, 1);
    videoRecorder.setFormat(recorderParameters.getVideoOutputFormat());
    videoRecorder.setSampleRate(recorderParameters.getAudioSamplingRate());
    videoRecorder.setFrameRate(recorderParameters.getVideoFrameRate());
    videoRecorder.setVideoCodec(recorderParameters.getVideoCodec());
    videoRecorder.setVideoQuality(recorderParameters.getVideoQuality());
    videoRecorder.setAudioQuality(recorderParameters.getVideoQuality());
    videoRecorder.setAudioCodec(recorderParameters.getAudioCodec());
    videoRecorder.setVideoBitrate(1000000);
    videoRecorder.setAudioBitrate(64000);
}

这是我的 onPreviewFrame() 方法:

@Override
public void onPreviewFrame(byte[] data, Camera camera)
{

    long frameTimeStamp = 0L;

    if(FragmentCamera.mAudioTimestamp == 0L && FragmentCamera.firstTime > 0L)
    {
        frameTimeStamp = 1000L * (System.currentTimeMillis() - FragmentCamera.firstTime);
    }
    else if(FragmentCamera.mLastAudioTimestamp == FragmentCamera.mAudioTimestamp)
    {
        frameTimeStamp = FragmentCamera.mAudioTimestamp + FragmentCamera.frameTime;
    }
    else
    {
        long l2 = (System.nanoTime() - FragmentCamera.mAudioTimeRecorded) / 1000L;
        frameTimeStamp = l2 + FragmentCamera.mAudioTimestamp;
        FragmentCamera.mLastAudioTimestamp = FragmentCamera.mAudioTimestamp;
    }

    synchronized(FragmentCamera.mVideoRecordLock)
    {
        if(FragmentCamera.recording && FragmentCamera.rec && lastSavedframe != null && lastSavedframe.getFrameBytesData() != null && yuvIplImage != null)
        {
            FragmentCamera.mVideoTimestamp += FragmentCamera.frameTime;

            if(lastSavedframe.getTimeStamp() > FragmentCamera.mVideoTimestamp)
            {
                FragmentCamera.mVideoTimestamp = lastSavedframe.getTimeStamp();
            }

            try
            {
                yuvIplImage.getByteBuffer().put(lastSavedframe.getFrameBytesData());

                IplImage bgrImage = IplImage.create(mPreviewSize.width, mPreviewSize.height, opencv_core.IPL_DEPTH_8U, 4);// In my case, mPreviewSize.width = 1280 and mPreviewSize.height = 720
                IplImage transposed = IplImage.create(mPreviewSize.height, mPreviewSize.width, yuvIplImage.depth(), 4);
                IplImage squared = IplImage.create(mPreviewSize.height, mPreviewSize.height, yuvIplImage.depth(), 4);

                int[] _temp = new int[mPreviewSize.width * mPreviewSize.height];

                Util.YUV_NV21_TO_BGR(_temp, data, mPreviewSize.width,  mPreviewSize.height);

                bgrImage.getIntBuffer().put(_temp);

                opencv_core.cvTranspose(bgrImage, transposed);
                opencv_core.cvFlip(transposed, transposed, 1);

                opencv_core.cvSetImageROI(transposed, opencv_core.cvRect(0, 0, mPreviewSize.height, mPreviewSize.height));
                opencv_core.cvCopy(transposed, squared, null);
                opencv_core.cvResetImageROI(transposed);

                videoRecorder.setTimestamp(lastSavedframe.getTimeStamp());
                videoRecorder.record(squared);
            }
            catch(com.googlecode.javacv.FrameRecorder.Exception e)
            {
                e.printStackTrace();
            }
        }

        lastSavedframe = new SavedFrames(data, frameTimeStamp);
    }
}

这段代码使用了一个方法“YUV_NV21_TO_BGR”，我从这个 link 中找到了它

基本上我遇到了和你一样的问题，“Android 上的绿色魔鬼问题”。在我刚刚转置 YuvIplImage 时添加“YUV_NV21_TO_BGR”方法之前，更重要的是转置、翻转(有或没有调整大小)的组合，结果视频中有绿色输出，几乎像你的一样。这种“YUV_NV21_TO_BGR”方法消除了绿色输出问题。感谢来自上述 google groups 线程的@David Han。

您还应该知道，在 onPreviewFrame 中的所有这些处理(转置、翻转和调整大小)都需要很长时间，这会严重影响您的每秒帧数 (FPS) 率。当我在 onPreviewFrame 方法中使用此代码时，录制视频的最终 FPS 从 30fps 降至 3 帧/秒。