java - Opencv - 检测眼睛是闭着还是睁着

Question

我正在做一个项目，我们试图检测图片中眼睛是闭着还是睁着。到目前为止，我们所做的是检测到面部，然后是眼睛。然后我们应用霍夫变换，希望当眼睛睁开时虹膜是唯一的圆圈。问题是当眼睛闭合时，它也会产生一个圆圈:

代码如下:

import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.core.Point;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.highgui.Highgui;
import org.opencv.objdetect.CascadeClassifier;
import org.opencv.imgproc.Imgproc;



 
public class FaceDetector {
 
    public static void main(String[] args) {
        
    
 
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
        System.out.println("\nRunning FaceDetector");
 
        CascadeClassifier faceDetector = new CascadeClassifier("D:\\CS\\opencv\\sources\\data\\haarcascades\\haarcascade_frontalface_alt.xml");
        CascadeClassifier eyeDetector = new CascadeClassifier("D:\\CS\\opencv\\sources\\data\\haarcascades\\haarcascade_eye.xml");
       
        Mat image = Highgui.imread("C:\\Users\\Yousra\\Desktop\\images.jpg");
        Mat gray = Highgui.imread("C:\\Users\\Yousra\\Desktop\\eyes\\E7.png");
        
        String faces;
        String eyes;
        
 
        MatOfRect faceDetections = new MatOfRect();
        MatOfRect eyeDetections = new MatOfRect();
        
        Mat face;
        Mat crop = null;
        Mat circles = new Mat();
        faceDetector.detectMultiScale(image, faceDetections);
        
   for (int i = 0; i< faceDetections.toArray().length; i++){
            
            faces = "Face"+i+".png";
            
             face = image.submat(faceDetections.toArray()[i]);
             crop = face.submat(4, (2*face.width())/3, 0, face.height());
            Highgui.imwrite(faces, face);
             eyeDetector.detectMultiScale(crop, eyeDetections, 1.1, 2, 0,new Size(30,30), new Size()); 
         
             if(eyeDetections.toArray().length ==0){
                 
                 System.out.println(" Not a face" + i);
             }else{
                 
                 System.out.println("Face with " + eyeDetections.toArray().length + "eyes" );
                 
                 for (int j = 0; j< eyeDetections.toArray().length ; j++){
                     
                    System.out.println("Eye" );
                    Mat eye = crop.submat(eyeDetections.toArray()[j]);
                    eyes = "Eye"+j+".png";
                    Highgui.imwrite(eyes, eye);
                     
                 }
             }
         }
             
             
          
        
            
             Imgproc.cvtColor(gray, gray, Imgproc.COLOR_BGR2GRAY);
             System.out.println("1 Hough :" +circles.size());
 float circle[] = new float[3];

             for (int i = 0; i < circles.cols(); i++)
             {
                     circles.get(0, i, circle);
                 org.opencv.core.Point center = new org.opencv.core.Point();
                 center.x = circle[0];
                 center.y = circle[1];
                 Core.circle(gray, center, (int) circle[2], new Scalar(255,255,100,1), 4);
                 }
             
             
             Imgproc.Canny( gray, gray, 200, 10, 3,false);  
             
             Imgproc.HoughCircles( gray, circles, Imgproc.CV_HOUGH_GRADIENT, 1, 100, 80, 10, 10, 50 );
             System.out.println("2 Hough:" +circles.size());
             
             for (int i = 0; i < circles.cols(); i++)
             {
                     circles.get(0, i, circle);
                 org.opencv.core.Point center = new org.opencv.core.Point();
                 center.x = circle[0];
                 center.y = circle[1];
                 Core.circle(gray, center, (int) circle[2], new Scalar(255,255,100,1), 4);
                 }
             Imgproc.Canny( gray, gray, 200, 10, 3,false);  
             
             Imgproc.HoughCircles( gray, circles, Imgproc.CV_HOUGH_GRADIENT, 1, 100, 80, 10, 10, 50 );
             System.out.println("3 Hough" +circles.size());
             
             //float circle[] = new float[3];

             for (int i = 0; i < circles.cols(); i++)
             {
                     circles.get(0, i, circle);
                 org.opencv.core.Point center = new org.opencv.core.Point();
                 center.x = circle[0];
                 center.y = circle[1];
                 Core.circle(gray, center, (int) circle[2], new Scalar(255,255,100,1), 4);
                 }

            String hough = "afterhough.png";
            Highgui.imwrite(hough, gray);
   }     
}

如何让它更准确？

Answer 1

在大多数情况下，即眼睛部分打开或关闭的情况下，圆形霍夫变换不太可能很好地工作。您最好隔离眼睛周围的矩形区域(边界框)并根据像素强度(灰度级)计算度量。例如，区域内像素的方差可以很好地区分睁眼和闭眼。使用 OpenCV Haar 级联检测到的面部周围边界框的相对位置，可以非常可靠地获得眼睛周围的边界框。本文中的图 3 给出了定位过程的一些概念。

http://personal.ee.surrey.ac.uk/Personal/J.Collomosse/pubs/Malleson-IJCV-2012.pdf