python - 使用 Python 自动裁剪图像以提取内部黑色边框 ROI

Question

我一直在研究 OpenCV 和 Pillow(以及 Python 之外的 ImageMagick，尤其是 Fred's Image Magick Scripts)来实现以下目标:

自动识别扫描图像中的内部黑色边框，并将图像裁剪到该边框。这是一个涂黑的示例图像，第一个是“原始”，第二个是我想要实现的目标，黑色边框周围有红色突出显示:

问题在于边框不在图像的外部，并且扫描的质量差异很大，这意味着边框永远不会在同一位置，并且无法通过像素进行裁剪。

编辑:我正在寻找一种裁剪图像的方法，仅将所有内容保留在黑色边框内(现在模糊)

我正在寻求帮助，以了解如何 a) 是否可以进行此类裁剪以及 b) 如何最好使用 Python 进行此操作。

谢谢!

Answer 1

这是在 Imagemagick 中执行此操作的一种非常简单的方法。

Get the center coordinates

Clone the image and do the following on the clone

Threshold the image so that the inside of the black lines is white. 

(If necessary use -connected-components to merge smaller black features into the white in the center)

Apply some morphology open to make sure that the black lines are continuous

Floodfill the image with red starting in the center

Convert non-red to black and red to white

Put the processed clone into the alpha channel of the input

输入:

center=$(convert img.jpg -format "%[fx:w/2],%[fx:h/2]\n" info:)

convert img.jpg \
\( +clone -auto-level -threshold 35% \
-morphology open disk:5 \
-fill red -draw "color $center floodfill" -alpha off \
-fill black +opaque red -fill white -opaque red \) \
-alpha off -compose copy_opacity -composite result.png

这里是与上面等效的 Python Wand 代码:

#!/bin/python3.7

from wand.image import Image
from wand.drawing import Drawing
from wand.color import Color
from wand.display import display

with Image(filename='black_rect.jpg') as img:
    with img.clone() as copied:
        copied.auto_level()
        copied.threshold(threshold=0.35)
        copied.morphology(method='open', kernel='disk:5')
        centx=round(0.5*copied.width)
        centy=round(0.5*copied.height)
        with Drawing() as draw:
            draw.fill_color='red'
            draw.color(x=centx, y=centy, paint_method='floodfill')
            draw(copied)
        copied.opaque_paint(target='red', fill='black', fuzz=0.0, invert=True)
        copied.opaque_paint(target='red', fill='white', fuzz=0.0, invert=False)
        display(copied)
        copied.alpha_channel = 'copy'
        img.composite(copied, left=0, top=0, operator='copy_alpha')
        img.format='png'
        display(img)
        img.save(filename='black_rect_interior.png')

对于 OpenCV，我建议以下处理可能是一种方法。抱歉，我不熟悉 OpenCV

Threshold the image so that the inside of the black lines is white. 

Apply some morphology open to make sure that the black lines are continuous

Get the contours of the white regions.

Get the largest interior contour and fill the inside with white

Put that result into the alpha channel of the input

添加:

对于那些有兴趣的人，这里有一个更长的方法，将有助于透视校正。我做的事情与 nathancy 所做的类似，但是是在 Imagemagick 中。

首先，对图像进行阈值处理并打开形态学以确保黑线是连续的。

然后做连通分量，得到最大白色区域的ID号

然后提取该区域

id=$(convert img.jpg -auto-level -threshold 35% \
-morphology open disk:5 -type bilevel \
-define connected-components:mean-color=true \
-define connected-components:verbose=true \
-connected-components 8 null: | grep "gray(255)" | head -n 1 | awk '{print $1}' | sed 's/[:]*$//')
echo $id

convert img.jpg -auto-level -threshold 35% \
-morphology open disk:5 -type bilevel \
-define connected-components:mean-color=true \
-define connected-components:keep=$id \
-connected-components 8 \
-alpha extract -morphology erode disk:5 \
region.png

现在进行 Canny 边缘检测和霍夫线变换。这里我保存了canny图像、霍夫线作为红线以及叠加在图像上的线和线信息，这些信息保存在.mvg文件中。

convert region.png \
\( +clone -canny 0x1+10%+30% +write region_canny.png \
-background none -fill red -stroke red -strokewidth 2 \
-hough-lines 9x9+400 +write region_lines.png +write lines.mvg \) \
-compose over -composite region_hough.png

convert region_lines.png -alpha extract region_bw_lines.png

# Hough line transform: 9x9+400
viewbox 0 0 2000 2829
# x1,y1  x2,y2 # count angle distance
line 0,202.862 2000,272.704  # 763 92 824
line 204.881,0 106.09,2829  # 990 2 1156
line 1783.84,0 1685.05,2829  # 450 2 2734
line 0,2620.34 2000,2690.18  # 604 92 3240

接下来我使用我编写的脚本来进行角点检测。这里我使用 Harris 探测器。

corners=$(corners -m harris -t 40 -d 5 -p yes region_bw_lines.png region_bw_lines_corners.png)
echo "$corners"

pt=1 coords=195.8,207.8
pt=2 coords=1772.8,262.8
pt=3 coords=111.5,2622.5
pt=4 coords=1688.5,2677.5

接下来，我按顺时针方向提取并排序角。以下是我编写的一些代码，是我从 here 转换而来的

list=$(echo "$corners" | sed -n 's/^.*=\(.*\)$/\1/p' | tr "\n" " " | sed 's/[ ]*$//' )
echo "$list"
195.8,207.8 1772.8,262.8 111.5,2622.5 1688.5,2677.5

# sort on x
xlist=`echo "$list" | tr " " "\n" | sort -n -t "," -k1,1`
leftmost=`echo "$xlist" | head -n 2`
rightmost=`echo "$xlist" | tail -n +3`
rightmost1=`echo "$rightmost" | head -n 1`
rightmost2=`echo "$rightmost" | tail -n +2`
# sort leftmost on y
leftmost2=`echo "$leftmost" | sort -n -t "," -k2,2`
topleft=`echo "$leftmost2" | head -n 1`
btmleft=`echo "$leftmost2" | tail -n +2`
# get distance from topleft to rightmost1 and rightmost2; largest is bottom right
topleftx=`echo "$topleft" | cut -d, -f1`
toplefty=`echo "$topleft" | cut -d, -f2`
rightmost1x=`echo "$rightmost1" | cut -d, -f1`
rightmost1y=`echo "$rightmost1" | cut -d, -f2`
rightmost2x=`echo "$rightmost2" | cut -d, -f1`
rightmost2y=`echo "$rightmost2" | cut -d, -f2`
dist1=`convert xc: -format "%[fx:hypot(($topleftx-$rightmost1x),($toplefty-$rightmost1y))]" info:`
dist2=`convert xc: -format "%[fx:hypot(($topleftx-$rightmost2x),($toplefty-$rightmost2y))]" info:`
test=`convert xc: -format "%[fx:$dist1>$dist2?1:0]" info:`
if [ $test -eq 1 ]; then
btmright=$rightmost1
topright=$rightmost2
else
btmright=$rightmost2
topright=$rightmost1
fi
sort_corners="$topleft $topright $btmright $btmleft"
echo $sort_corners

195.8,207.8 1772.8,262.8 1688.5,2677.5 111.5,2622.5

最后，我使用角坐标在黑色背景上绘制一个白色填充的多边形，并将该结果放入输入图像的 Alpha channel 中。

convert img.jpg \
\( +clone -fill black -colorize 100 \
-fill white -draw "polygon $sort_corners" \) \
-alpha off -compose copy_opacity -composite result.png