python - 将纹理特征集成到基于颜色的视频帧目标检测中

Question

我正在做一个项目，在这个项目中我必须从鸟瞰 Angular 检测有色车辆。请参阅下面的图片框

因为我认为对于这种情况，深度学习是过大的，所以我决定采用直方图反投影方法来检测红色车辆，从而选择经典的检测管道。
我使用的代码如下所示

#import packages
import cv2 
import numpy as np
import matplotlib.pyplot as plt

#defining disk for the convolution
disc = cv2.getStructuringElement(cv2.MORPH_RECT,(10,10))
#defining Kernel
kernel=cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
kernel1 = np.ones((5,5),np.uint8)



#preparing the object model by cropping the target object and reading it
Red_Model=cv2.imread(r'C:/Users/kjbaili/.spyder-py3/webcam_calib/red_new_uniform.png')

#convert object_model to hsv
hsv_red =  cv2.cvtColor(Red_Model, cv2.COLOR_BGR2HSV)
cv2.imshow('HSV_Model',hsv_red)

#calculating histogram of object_model
M = cv2.calcHist([hsv_red],[0,1],None,[180,256],[0,180,0,256])

#reading the image where we want to search for the target object
img=cv2.imread(r'C:/Users/kjbaili/.spyder-py3/Mean_Shift/MST_with_3D_projection/messigray1.png')

#converting the image to hsv
hsv_img =  cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

#calculating histogram of image
I= cv2.calcHist([hsv_img],[0,1],None,[180,256],[0,180,0,256])

#calculating the ration histogram
R=M.copy() /I.copy()

#extracting the image channels
h,s,v = cv2.split(hsv_img.copy())


 
#backprojecting the R_red hist
B1 = R[h.ravel(),s.ravel()]

#reshaping the resulted vector into image and normalize the values between 0 and 1
B2=B1.reshape(hsv_img.copy().shape[:2])
B = np.minimum(B1.copy(),1)
B = B.reshape(hsv_img.copy().shape[:2])
 



#removing noises
open_B_red=cv2.morphologyEx(B.copy(),cv2.MORPH_OPEN,kernel)     
Filter_red=cv2.filter2D(open_B_red.copy(),-1,disc,open_B_red)

#displaying results
cv2.imshow('Filter',Filter_red)



cv2.waitKey(0)
cv2.destroyAllWindows()

该算法适用于所示的效果(请参见结果蒙版图像)

但是，当对象的颜色与背景颜色相似(如下面的图片)时，它也会检测到我的T恤，它会检测到它们两者，因此无法仅区分目标对象

因此，我想到与颜色功能一起使用附加功能，以增强检测能力，以便尽管颜色相似度很小，它仍然可以检测到目标对象，因此得出了所谓的“ Local Binary Pattern-LBP”。 “，用于计算对象的纹理。实际上，我可以计算出对象和图像的LBP，但是不知道将其与上述方法集成在一起，以便通过颜色纹理检测来检测红色车辆。
因此，我非常欣赏在使用LBP的T exture功能或什至使用其他附加功能的情况下，如何改善检测的想法。
有关LBP的信息，请参见:https://towardsdatascience.com/face-recognition-how-lbph-works-90ec258c3d6b
有关直方图反向投影的工作原理的信息:https://theailearner.com/2019/04/18/histogram-backprojection/
这是原始图像:

这是结果面具

提前致谢

Answer 1

这是在Python / OpenCV中执行此操作的一种方法。颜色的第一阈值。然后应用一些形态来清洁无关区域。然后获取轮廓并在每个轮廓上循环。计算每个矩形的旋转矩形及其面积和纵横比(最大尺寸/最小尺寸)。过滤两个轮廓上的轮廓，并在输入图像上绘制白色轮廓和白色旋转矩形。
输入:

import cv2
import numpy as np

image = cv2.imread("red_car.png")
hh, ww = image.shape[:2]

# convert to HSV
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

# create a binary thresholded image
lower = (80,100,180)
upper = (255,255,255)
thresh = cv2.inRange(hsv, lower, upper)


# apply morphology
kernel = np.ones((5,5), np.uint8)
clean = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# get external contours
contours = cv2.findContours(clean, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]

area_thresh1 = 500
area_thresh2 = 1000
aspect_thresh1 = 2
aspect_thresh2 = 4
result1 = image.copy()
result2 = image.copy()
for c in contours:
    
    # get rotated rectangle from contour
    # get its dimensions
    # get angle relative to horizontal from rotated rectangle
    rotrect = cv2.minAreaRect(c)
    box = cv2.boxPoints(rotrect)
    box = np.int0(box)
    (center), (dim1,dim2), angle = rotrect
    maxdim = max(dim1,dim2)
    mindim = min(dim1,dim2)
    area = dim1 * dim2
    if area > 0:
        aspect = maxdim / mindim
        #print(area, aspect)

    if area > area_thresh1 and area < area_thresh2 and aspect > aspect_thresh1 and aspect < aspect_thresh2:
        # draw contour on input
        cv2.drawContours(result1,[c],0,(255,255,255),1)
        # draw rectangle on input
        cv2.drawContours(result2,[box],0,(255,255,255),1)
        print(area, aspect)

# save result
cv2.imwrite("red_car_thresh.png",thresh)
cv2.imwrite("red_car_clean.png",clean)
cv2.imwrite("red_car_thresh_result1.png",result1)
cv2.imwrite("red_car_thresh_result2.png",result2)

# display result
cv2.imshow("thresh", thresh)
cv2.imshow("clean", clean)
cv2.imshow("result1", result1)
cv2.imshow("result2", result2)
cv2.waitKey(0)
cv2.destroyAllWindows()

阈值图片:

形态清除图像:

汽车轮廓:

汽车旋转边界框: