问题
所以总的来说，我对计算机视觉还很陌生。我目前正在尝试通过分析 2 个图像来计算单应性。我想使用单应性来校正 1 个图像的视角以匹配另一个。但我得到的比赛既糟糕又错误。所以我所做的单应扭曲完全关闭。
当前状态
我正在使用 EmguCV 在 C# 中包装 opencv。
我知道我的代码似乎“正常”工作。
我加载我的两个图像并声明一些变量来存储计算工件。
(Image<Bgr, byte> Image, VectorOfKeyPoint Keypoints, Mat Descriptors) imgModel = (new Image<Bgr, byte>(imageFolder + "image0.jpg").Resize(0.2, Emgu.CV.CvEnum.Inter.Area), new VectorOfKeyPoint(), new Mat());
(Image<Bgr, byte> Image, VectorOfKeyPoint Keypoints, Mat Descriptors) imgTest = (new Image<Bgr, byte>(imageFolder + "image1.jpg").Resize(0.2, Emgu.CV.CvEnum.Inter.Area), new VectorOfKeyPoint(), new Mat());
Mat imgKeypointsModel = new Mat();
Mat imgKeypointsTest = new Mat();
Mat imgMatches = new Mat();
Mat imgWarped = new Mat();
VectorOfVectorOfDMatch matches = new VectorOfVectorOfDMatch();
VectorOfVectorOfDMatch filteredMatches = new VectorOfVectorOfDMatch();
List<MDMatch[]> filteredMatchesList = new List<MDMatch[]>();
请注意，我使用了
ValueTuple<Image,VectorOfKeyPoint,Mat>直接用它们各自的关键点和描述符存储图像。在此之后，使用 ORB 检测器和 BruteForce 匹配器来检测、描述和匹配关键点:
ORBDetector detector = new ORBDetector();
BFMatcher matcher = new BFMatcher(DistanceType.Hamming2);
detector.DetectAndCompute(imgModel.Image, null, imgModel.Keypoints, imgModel.Descriptors, false);
detector.DetectAndCompute(imgTest.Image, null, imgTest.Keypoints, imgTest.Descriptors, false);
matcher.Add(imgTest.Descriptors);
matcher.KnnMatch(imgModel.Descriptors, matches, k: 2, mask: null);
在此之后，我申请了 ratio test并使用匹配距离阈值进行进一步过滤。
MDMatch[][] matchesArray = matches.ToArrayOfArray();
//Apply ratio test
for (int i = 0; i < matchesArray.Length; i++)
{
MDMatch first = matchesArray[i][0];
float dist1 = matchesArray[i][0].Distance;
float dist2 = matchesArray[i][1].Distance;
if (dist1 < ms_MIN_RATIO * dist2)
{
filteredMatchesList.Add(matchesArray[i]);
}
}
//Filter by threshold
MDMatch[][] defCopy = new MDMatch[filteredMatchesList.Count][];
filteredMatchesList.CopyTo(defCopy);
filteredMatchesList = new List<MDMatch[]>();
foreach (var item in defCopy)
{
if (item[0].Distance < ms_MAX_DIST)
{
filteredMatchesList.Add(item);
}
}
filteredMatches = new VectorOfVectorOfDMatch(filteredMatchesList.ToArray());
禁用这些过滤器方法中的任何一个都不会真正使我的结果变得更好或更糟(只是保留所有匹配项)，但它们似乎有意义，所以我保留它们。
最后，我从找到的和过滤的匹配项中计算出我的单应性，然后用这个单应性扭曲图像并绘制一些调试图像:
Mat homography = Features2DToolbox.GetHomographyMatrixFromMatchedFeatures(imgModel.Keypoints, imgTest.Keypoints, filteredMatches, null, 10);
CvInvoke.WarpPerspective(imgTest.Image, imgWarped, homography, imgTest.Image.Size);
Features2DToolbox.DrawKeypoints(imgModel.Image, imgModel.Keypoints, imgKeypointsModel, new Bgr(0, 0, 255));
Features2DToolbox.DrawKeypoints(imgTest.Image, imgTest.Keypoints, imgKeypointsTest, new Bgr(0, 0, 255));
Features2DToolbox.DrawMatches(imgModel.Image, imgModel.Keypoints, imgTest.Image, imgTest.Keypoints, filteredMatches, imgMatches, new MCvScalar(0, 255, 0), new MCvScalar(0, 0, 255));
//Task.Factory.StartNew(() => ImageViewer.Show(imgKeypointsModel, "Keypoints Model"));
//Task.Factory.StartNew(() => ImageViewer.Show(imgKeypointsTest, "Keypoints Test"));
Task.Factory.StartNew(() => ImageViewer.Show(imgMatches, "Matches"));
Task.Factory.StartNew(() => ImageViewer.Show(imgWarped, "Warp"));
tl;博士 : ORBDetector->BFMatcher->FilterMatches->GetHomography->WarpPerspective
输出
算法示例
测试投影是否出错
匹配时使用交叉检查
原始图像每张为 2448x3264，缩放比例为 0.2 之前 对它们进行任何计算。
题
基本上它既简单又复杂:我做错了什么？
正如你从上面的例子中看到的，我检测特征并匹配它们的方法似乎工作得非常糟糕。所以我在问是否有人可以在我的代码中发现错误。或者当互联网上有数百个示例显示它是如何工作的以及它是多么“容易”时，就为什么我的结果如此糟糕给出建议。
到目前为止我尝试过的:
我使用的例子:
原始图片:
原始图像可以从这里下载:
https://drive.google.com/open?id=1Nlqv_0sH8t1wiH5PG-ndMxoYhsUbFfkC
自问以来的进一步实验
所以我在询问后做了一些进一步的研究。大多数更改已包含在上面，但我想为此单独创建一个部分。
因此，在遇到这么多问题并且似乎无处可下之后，我决定谷歌搜索 original paper on ORB .在此之后，我决定尝试复制他们的一些结果。尝试后，我意识到即使我尝试将匹配图像旋转一定程度，匹配看起来也不错，但转换完全崩溃了。
我试图复制对象视角的方法有可能是错误的吗？
MCVE
https://drive.google.com/open?id=17DwFoSmco9UezHkON5prk8OsPalmp2MX
(没有包，但 nuget restore 就足以让它编译)
最佳答案
我遇到了同样的问题并找到了一个合适的解决方案:github Emgu.CV.Example DrawMatches.cs一切正常。
我修改了代码和方法FindMatch 看起来像这样:
public static void FindMatch(Mat modelImage, Mat observedImage, out VectorOfKeyPoint modelKeyPoints, out VectorOfKeyPoint observedKeyPoints, VectorOfVectorOfDMatch matches, out Mat mask, out Mat homography)
{
int k = 2;
double uniquenessThreshold = 0.80;
homography = null;
modelKeyPoints = new VectorOfKeyPoint();
observedKeyPoints = new VectorOfKeyPoint();
using (UMat uModelImage = modelImage.GetUMat(AccessType.Read))
using (UMat uObservedImage = observedImage.GetUMat(AccessType.Read))
{
var featureDetector = new ORBDetector(9000);
Mat modelDescriptors = new Mat();
featureDetector.DetectAndCompute(uModelImage, null, modelKeyPoints, modelDescriptors, false);
Mat observedDescriptors = new Mat();
featureDetector.DetectAndCompute(uObservedImage, null, observedKeyPoints, observedDescriptors, false);
using (var matcher = new BFMatcher(DistanceType.Hamming, false))
{
matcher.Add(modelDescriptors);
matcher.KnnMatch(observedDescriptors, matches, k, null);
mask = new Mat(matches.Size, 1, DepthType.Cv8U, 1);
mask.SetTo(new MCvScalar(255));
Features2DToolbox.VoteForUniqueness(matches, uniquenessThreshold, mask);
int nonZeroCount = CvInvoke.CountNonZero(mask);
if (nonZeroCount >= 4)
{
nonZeroCount = Features2DToolbox.VoteForSizeAndOrientation(modelKeyPoints, observedKeyPoints,
matches, mask, 1.5, 20);
if (nonZeroCount >= 4)
homography = Features2DToolbox.GetHomographyMatrixFromMatchedFeatures(modelKeyPoints,
observedKeyPoints, matches, mask, 2);
}
}
}
}
使用:
var model = new Mat(@"image0.jpg");
var scene = new Mat(@"image1.jpg");
Mat result = new Mat();
VectorOfKeyPoint modelKeyPoints;
VectorOfKeyPoint observedKeyPoints;
var matches = new VectorOfVectorOfDMatch();
Mat mask;
Mat homography;
FindMatch(model, scene, out modelKeyPoints, out observedKeyPoints, matches, out mask, out homography);
CvInvoke.WarpPerspective(scene, result, homography, model.Size, Inter.Linear, Warp.InverseMap);
结果:
如果你想看过程使用下一个代码:
public static Mat Draw(Mat modelImage, Mat observedImage)
{
Mat homography;
VectorOfKeyPoint modelKeyPoints;
VectorOfKeyPoint observedKeyPoints;
using (VectorOfVectorOfDMatch matches = new VectorOfVectorOfDMatch())
{
Mat mask;
FindMatch(modelImage, observedImage, out modelKeyPoints, out observedKeyPoints, matches, out mask, out homography);
Mat result = new Mat();
Features2DToolbox.DrawMatches(modelImage, modelKeyPoints, observedImage, observedKeyPoints,
matches, result, new MCvScalar(255, 0, 0), new MCvScalar(0, 0, 255), mask);
if (homography != null)
{
var imgWarped = new Mat();
CvInvoke.WarpPerspective(observedImage, imgWarped, homography, modelImage.Size, Inter.Linear, Warp.InverseMap);
Rectangle rect = new Rectangle(Point.Empty, modelImage.Size);
var pts = new PointF[]
{
new PointF(rect.Left, rect.Bottom),
new PointF(rect.Right, rect.Bottom),
new PointF(rect.Right, rect.Top),
new PointF(rect.Left, rect.Top)
};
pts = CvInvoke.PerspectiveTransform(pts, homography);
var points = new Point[pts.Length];
for (int i = 0; i < points.Length; i++)
points[i] = Point.Round(pts[i]);
using (var vp = new VectorOfPoint(points))
{
CvInvoke.Polylines(result, vp, true, new MCvScalar(255, 0, 0, 255), 5);
}
}
return result;
}
}
使用:
var model = new Mat(@"image0.jpg");
var scene = new Mat(@"image1.jpg");
var result = Draw(model, scene);
结果:
关于c# - EmguCV (OpenCV) ORBDetector 只发现坏匹配，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/53079167/