c++ - 如何在ubuntu上的opencv和dlib上裁剪图像

Question

我想在 ubuntu 上使用 c++ 在 opencv 和 dlib 上执行这些操作。

1. 使用dlib检测人脸。(我已经做过了。)
2. 仅裁剪嘴部周围的图像。

这是我的代码。它基于 dlib 示例代码。

#include <dlib/image_processing/frontal_face_detector.h>
#include <dlib/image_processing/render_face_detections.h>
#include <dlib/image_processing.h>
#include <dlib/gui_widgets.h>
#include <dlib/image_io.h>
#include <iostream>
#include <opencv2/opencv.hpp>
#include <highgui.h>


using namespace dlib;
using namespace std;

// ----------------------------------------------------------------------------------------

int main(int argc, char** argv)
{  
    try
    {
        // This example takes in a shape model file and then a list of images to
        // process.  We will take these filenames in as command line arguments.
        // Dlib comes with example images in the examples/faces folder so give
        // those as arguments to this program.
        if (argc == 1)
        {
            cout << "Call this program like this:" << endl;
            cout << "./face_landmark_detection_ex shape_predictor_68_face_landmarks.dat faces/*.jpg" << endl;
            cout << "\nYou can get the shape_predictor_68_face_landmarks.dat file from:\n";
            cout << "http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2" << endl;
            return 0;
        }

        // We need a face detector.  We will use this to get bounding boxes for
        // each face in an image.
        frontal_face_detector detector = get_frontal_face_detector();
        // And we also need a shape_predictor.  This is the tool that will predict face
        // landmark positions given an image and face bounding box.  Here we are just
        // loading the model from the shape_predictor_68_face_landmarks.dat file you gave
        // as a command line argument.
        shape_predictor sp;
        deserialize(argv[1]) >> sp;
    cv::Mat cimg = cv::imread(argv[1]);


        image_window win, win_faces;
        // Loop over all the images provided on the command line.
        for (int i = 2; i < argc; ++i)
        {
            cout << "processing image " << argv[i] << endl;
            array2d<rgb_pixel> img;
            load_image(img, argv[i]);
/*
            // Make the image larger so we can detect small faces.
            pyramid_up(img);
*/
            // Now tell the face detector to give us a list of bounding boxes
            // around all the faces in the image.
            std::vector<rectangle> dets = detector(img);
            cout << "Number of faces detected: " << dets.size() << endl;

            // Now we will go ask the shape_predictor to tell us the pose of
            // each face we detected.
            std::vector<full_object_detection> shapes;
            for (unsigned long j = 0; j < dets.size(); ++j)
            {
                full_object_detection shape = sp(img, dets[j]);
                cout << "number of parts: "<< shape.num_parts() << endl;
                cout << "pixel position of first part:  " << shape.part(0) << endl;
                cout << "pixel position of second part: " << shape.part(1) << endl;
                // You get the idea, you can get all the face part locations if
                // you want them.  Here we just store them in shapes so we can
                // put them on the screen.
                shapes.push_back(shape);
            }

        // Crop the original image to the defined ROI */
        cv::Rect roi;
        roi.x = 0;
        roi.y = 0;
        roi.width = 200;
        roi.height = 200;

        cv::Mat crop = cimg(roi);
        cv::imshow("crop", crop);

            // Now let's view our face poses on the screen.
/*
            win.clear_overlay();
            win.set_image(img);
            win.add_overlay(render_face_detections(shapes));

            // We can also extract copies of each face that are cropped, rotated upright,
            // and scaled to a standard size as shown here:
            //dlib::array<array2d<rgb_pixel> > face_chips;
            //extract_image_chips(img, get_face_chip_details(shapes), face_chips);
            //win_faces.set_image(tile_images(face_chips));
*/

            cout << "Hit enter to process the next image..." << endl;
            cin.get();
        }
    }
    catch (exception& e)
    {
        cout << "\nexception thrown!" << endl;
        cout << e.what() << endl;
    }
}

但它给了我这个错误。

OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in Mat, file /home/bigadmin/opencv-3.1.0/modules/core/src/matrix.cpp, line 508

exception thrown!
/home/bigadmin/opencv-3.1.0/modules/core/src/matrix.cpp:508: error: (-215) 0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows in function Mat

请教我解决此问题的方法。

谢谢。

Answer 1

您的代码中有很多错误:

1. cv::Mat cimg = cv::imread(argv[1]); - argv[1] 是检测器文件，而不是图像 - 你将得到空图像，这就是你的程序崩溃的原因

2. 您没有迭代图像。尝试这样的事情:

   for (int i = 2; i < argc; ++i)
   {
       cout << "processing image " << argv[i] << endl;
       cv::Mat cvimg = cv::imgread(argv[i]);
       dlib::cv_image<rgb_pixel> img(cvimg);
   ...
   

   在这里您只需读取一次文件即可检测人脸

3. 您应该根据人脸检测器功能(甚至更好 - 基于形状预测器)指定裁剪区域

...

std::vector<rectangle> dets = detector(img);

这里dets中的每一项都是一个矩形，描述人脸，你可以这样裁剪:

dlib::rectangle r = dets[j];
cv::Rect roi(r.left(), r.top(), r.width(), r.height());
cv::Mat face = cvimg(roi);

但这将是全脸图像。如果你只想裁剪嘴巴，你应该使用形状预测器的输出(未测试 - 请检查是否编译良好):

full_object_detection shape = sp(img, dets[j]);
auto mouth_left = shape.part(45);
auto mouth_right = shape.part(54);
unsigned long mouth_width = (mouth_right - mouth_left).length();
double padding = 0.2;
cv::Rect roi(mouth_left.x() - mouth_width * padding, mouth_left.y() - mouth_width*0.5, mouth_width * (1 + padding * 2), mouth_width);
cv::Mat mouth = cvimg(roi);

这将产生未对齐的嘴部图像