ios - Objective C 中 iOS 的图像模糊检测

Question

我试图确定从相机捕获的图像是否为 iOS 模糊。
我在拍照前已经检查了相机对焦，但如果图像模糊，这似乎有所不同。
我使用 Open CV 在 Android 上完成了这项工作，
OpenCV with Laplacian formula to detect image is blur or not in Android
这最终是，

int soglia = -6118750;
if (maxLap <= soglia) { // blurry

我玩了一下，然后减少到 -6718750。
对于 iOS，关于这样做的信息似乎较少。我看到一些人试图为此在 iOS 上使用 Open CV 的帖子，但他们似乎没有成功。
我看到这篇文章在 iOS 上使用 Metal 来做到这一点，
https://medium.com/better-programming/blur-detection-via-metal-on-ios-16dd02cb1558
这是在 Swift 中，所以我手动将其逐行转换为 Objective C。
我认为 code 可能是正确的翻译，但不确定原始代码是否正确或是否可以在相机捕获的图像上正常工作？
基本在我的测试中它总是给我 2 的结果，无论是平均值还是方差，这如何用于检测模糊图像或任何其他想法？

- (BOOL) detectBlur: (CGImageRef)image {
NSLog(@"detectBlur: %@", image);
// Initialize MTL
device = MTLCreateSystemDefaultDevice();
queue = [device newCommandQueue];

// Create a command buffer for the transformation pipeline
id <MTLCommandBuffer> commandBuffer = [queue commandBuffer];
// These are the two built-in shaders we will use
MPSImageLaplacian* laplacian = [[MPSImageLaplacian alloc] initWithDevice: device];
MPSImageStatisticsMeanAndVariance* meanAndVariance = [[MPSImageStatisticsMeanAndVariance alloc] initWithDevice: device];
// Load the captured pixel buffer as a texture
MTKTextureLoader* textureLoader = [[MTKTextureLoader alloc] initWithDevice: device];
id <MTLTexture> sourceTexture = [textureLoader newTextureWithCGImage: image options: nil error: nil];
// Create the destination texture for the laplacian transformation
MTLTextureDescriptor* lapDesc = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: sourceTexture.pixelFormat width: sourceTexture.width height: sourceTexture.height mipmapped: false];
lapDesc.usage = MTLTextureUsageShaderWrite | MTLTextureUsageShaderRead;
id <MTLTexture> lapTex = [device newTextureWithDescriptor: lapDesc];

// Encode this as the first transformation to perform
[laplacian encodeToCommandBuffer: commandBuffer sourceTexture: sourceTexture destinationTexture: lapTex];
// Create the destination texture for storing the variance.
MTLTextureDescriptor* varianceTextureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: sourceTexture.pixelFormat width: 2 height: 1 mipmapped: false];
varianceTextureDescriptor.usage = MTLTextureUsageShaderWrite | MTLTextureUsageShaderRead;
id <MTLTexture> varianceTexture = [device newTextureWithDescriptor: varianceTextureDescriptor];
// Encode this as the second transformation
[meanAndVariance encodeToCommandBuffer: commandBuffer sourceTexture: lapTex destinationTexture: varianceTexture];
// Run the command buffer on the GPU and wait for the results
[commandBuffer commit];
[commandBuffer waitUntilCompleted];
// The output will be just 2 pixels, one with the mean, the other the variance.
NSMutableData* result = [NSMutableData dataWithLength: 2];
void* resultBytes = result.mutableBytes;
//var result = [Int8](repeatElement(0, count: 2));
MTLRegion region = MTLRegionMake2D(0, 0, 2, 1);
const char* bytes = resultBytes;
NSLog(@"***resultBytes: %d", bytes[0]);
NSLog(@"***resultBytes: %d", bytes[1]);
[varianceTexture getBytes: resultBytes bytesPerRow: 1 * 2 * 4 fromRegion: region mipmapLevel: 0];
NSLog(@"resultBytes: %d", bytes[0]);
NSLog(@"resultBytes: %d", bytes[1]);

int variance = (int)bytes[1];

return variance < 2;
}

Answer 1

您的代码意味着您假设有 4 个 channel ，每个 channel 一个字节的差异纹理。但是对于您的varianceTextureDescriptor，您可能希望使用浮点值，这也是由于差异的值范围，请参阅下面的代码。此外，您似乎想与 OpenCV 进行比较并具有可比较的值。
无论如何，让我们从 MPSImageLaplacian 的 Apple 文档开始:

This filter uses an optimized convolution filter with a 3x3 kernel with the following weights:

在 Python 中可以这样做，例如喜欢:

import cv2
import np
from PIL import Image

img = np.array(Image.open('forrest.jpg'))
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
laplacian_kernel = np.array([[0, 1, 0], [1, -4, 1], [0, 1, 0]])

print(img.dtype)
print(img.shape)

laplacian = cv2.filter2D(img, -1, laplacian_kernel)
print('mean', np.mean(laplacian))
print('variance', np.var(laplacian, axis=(0, 1)))

cv2.imshow('laplacian', laplacian)
key = cv2.waitKey(0)

请注意，我们完全使用 Apple 文档中给出的值。
这为我的测试图像提供了以下输出:

uint8
(4032, 3024)
mean 14.531123203525237
variance 975.6843631756923

MPSImageStatisticsMeanAndVariance
我们现在想要使用 Apple 的 Metal Performance Shader MPSImageStatisticsMeanAndVariance 获得相同的值。
将输入图像转换为灰度图像很有用。然后应用 MPSImageLaplacian 图像内核。
一个字节也只能有 0 到 255 之间的值。因此，对于产生的均值或方差值，我们希望有浮点值。我们可以独立于输入图像的像素格式来指定它。所以我们应该使用 MTLPixelFormatR32Float 如下:

 MTLTextureDescriptor *varianceTextureDescriptor = [MTLTextureDescriptor
                                                       texture2DDescriptorWithPixelFormat:MTLPixelFormatR32Float
                                                       width:2
                                                       height:1
                                                       mipmapped:NO];

然后我们想将结果纹理中的 8 个字节解释为两个浮点数。我们可以通过工会很好地做到这一点。这可能是这样的:

union {
  float f[2];
  unsigned char bytes[8];
} u1;
MTLRegion region = MTLRegionMake2D(0, 0, 2, 1);
[varianceTexture getBytes:u1.bytes bytesPerRow:2 * 4 fromRegion:region mipmapLevel: 0];

最后，我们需要知道计算是用 0 到 1 之间的浮点值完成的，这实际上意味着我们想要乘以 255 或 255*255 以获得方差，以使其进入可比较的值范围:

NSLog(@"mean: %f", u1.f[0] * 255);
NSLog(@"variance: %f", u1.f[1] * 255 * 255);

为了完整起见，整个Objective-C代码:

id<MTLDevice> device = MTLCreateSystemDefaultDevice();
id<MTLCommandQueue> queue = [device newCommandQueue];
id<MTLCommandBuffer> commandBuffer = [queue commandBuffer];

MTKTextureLoader *textureLoader = [[MTKTextureLoader alloc] initWithDevice:device];
id<MTLTexture> sourceTexture = [textureLoader newTextureWithCGImage:image.CGImage options:nil error:nil];


CGColorSpaceRef srcColorSpace = CGColorSpaceCreateDeviceRGB();
CGColorSpaceRef dstColorSpace = CGColorSpaceCreateDeviceGray();
CGColorConversionInfoRef conversionInfo = CGColorConversionInfoCreate(srcColorSpace, dstColorSpace);
MPSImageConversion *conversion = [[MPSImageConversion alloc] initWithDevice:device
                                                                   srcAlpha:MPSAlphaTypeAlphaIsOne
                                                                  destAlpha:MPSAlphaTypeAlphaIsOne
                                                            backgroundColor:nil
                                                             conversionInfo:conversionInfo];
MTLTextureDescriptor *grayTextureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatR16Unorm
                                                                                                 width:sourceTexture.width
                                                                                                height:sourceTexture.height
                                                                                             mipmapped:false];
grayTextureDescriptor.usage = MTLTextureUsageShaderWrite | MTLTextureUsageShaderRead;
id<MTLTexture> grayTexture = [device newTextureWithDescriptor:grayTextureDescriptor];
[conversion encodeToCommandBuffer:commandBuffer sourceTexture:sourceTexture destinationTexture:grayTexture];


MTLTextureDescriptor *textureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:grayTexture.pixelFormat
                                                                                             width:sourceTexture.width
                                                                                            height:sourceTexture.height
                                                                                         mipmapped:false];
textureDescriptor.usage = MTLTextureUsageShaderWrite | MTLTextureUsageShaderRead;
id<MTLTexture> texture = [device newTextureWithDescriptor:textureDescriptor];

MPSImageLaplacian *imageKernel = [[MPSImageLaplacian alloc] initWithDevice:device];
[imageKernel encodeToCommandBuffer:commandBuffer sourceTexture:grayTexture destinationTexture:texture];


MPSImageStatisticsMeanAndVariance *meanAndVariance = [[MPSImageStatisticsMeanAndVariance alloc] initWithDevice:device];
MTLTextureDescriptor *varianceTextureDescriptor = [MTLTextureDescriptor
                                                   texture2DDescriptorWithPixelFormat:MTLPixelFormatR32Float
                                                   width:2
                                                   height:1
                                                   mipmapped:NO];
varianceTextureDescriptor.usage = MTLTextureUsageShaderWrite;
id<MTLTexture> varianceTexture = [device newTextureWithDescriptor:varianceTextureDescriptor];
[meanAndVariance encodeToCommandBuffer:commandBuffer sourceTexture:texture destinationTexture:varianceTexture];


[commandBuffer commit];
[commandBuffer waitUntilCompleted];

union {
    float f[2];
    unsigned char bytes[8];
} u;

MTLRegion region = MTLRegionMake2D(0, 0, 2, 1);
[varianceTexture getBytes:u.bytes bytesPerRow:2 * 4 fromRegion:region mipmapLevel: 0];

NSLog(@"mean: %f", u.f[0] * 255);
NSLog(@"variance: %f", u.f[1] * 255 * 255);

最终输出为 Python 程序提供了类似的值:

mean: 14.528159
variance: 974.630615

Python 代码和 Objective-C 代码也为其他图像计算类似的值。
即使这不是直接问的，也应该注意方差值当然也非常依赖于motif。如果您有一系列具有相同主题的图像，那么该值肯定是有意义的。为了说明这一点，这里有一个小测试，两个不同的图案都很清晰，但在方差值上显示出明显的差异:

在应用拉普拉斯滤波器后，您可以在上部区域和下部区域看到相应的图像转换为灰色。可以在图像之间的中间看到相应的中值或方差值。