swift - VNClassificationObservation 不起作用？

Question

我一直在努力尝试使用 CoreML 和 Vision 在 swift 中创建一个简单的计算机视觉应用程序。我已经训练了我自己的 Keras 网络，该网络拍摄分辨率为 64x64 的彩色图像，然后告诉您它是字母表中的哪个字母。当我在手机上执行此应用程序并拍摄图像并点击“使用此图像”时，代码在这段代码处崩溃:

//send a request to the network to identify the image
            let request = VNCoreMLRequest(model: model) { (request, error) in
                guard let results = request.results as? [VNClassificationObservation] else {
                    fatalError("Model failed to load image")
            }

在过去的三个小时里，我一直卡在这个错误上，希望你们能帮我找出问题所在!下面是我使用的其余代码。

class ViewController: UIViewController, UIImagePickerControllerDelegate, UINavigationControllerDelegate {

    @IBOutlet weak var imageView: UIImageView!

    let imagePicker = UIImagePickerController()

    override func viewDidLoad() {
        super.viewDidLoad()

        imagePicker.delegate = self
        imagePicker.sourceType = .camera
        imagePicker.allowsEditing = false

    }

    //function to chose an image from your library or take one with your camera
    func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {

        if let userPickedImage = info[UIImagePickerControllerOriginalImage] as? UIImage {

            imageView.image = userPickedImage

            //transform the image from UIImage to a CIIMage type
            guard let ciImage = CIImage(image: userPickedImage) else {
                fatalError("Couldn't transform image to CIImage type")
            }

            //call the detect function to pass the new ciImage into the network
            detect(image: ciImage)

        }

        imagePicker.dismiss(animated: true, completion: nil)

    }

    //function to classify the image that is taken with the camera or chosen from the library
    func detect(image: CIImage) {

        //try to load the model, if not throw an error
        guard let model = try? VNCoreMLModel(for: chars74k().model) else {
            fatalError("Loading coreML model failed")
        }

        //send a request to the network to identify the image
        let request = VNCoreMLRequest(model: model) { (request, error) in
            guard let results = request.results as? [VNClassificationObservation] else {
                fatalError("Model failed to load image")
        }

            print(results)

    }
        //create handler for image
        let handler = VNImageRequestHandler(ciImage: image)

        do{
            try handler.perform([request])
        }
        catch {
            print(error)
        }

Answer 1

您转换的结果类型错误。要检查结果的类型，您可以在 fatalError 行中放置一个断点并使用调试器执行此操作。但我假设你是初学者所以试试这个。替换:

guard let results = request.results as? [VNClassificationObservation] else {
    fatalError("Model failed to load image")
}

与:

print("your results are \(type(of: results))")

if let results = request.results as? [VNClassificationObservation] {
    print("your results are of type VNClassificationObservation")
}

if let results = request.results as? [VNPixelBufferObservation] {
    print("your results are of type VNPixelBufferObservation")
}

if let results = request.results as? [VNCoreMLFeatureValueObservation] {
    print("your results are of type VNCoreMLFeatureValueObservation")
}

docs对 VNCoreMLRequest 的可能结果非常清楚，所以你应该阅读它们(我强烈推荐这个，即使你没有那个就成功了，它并不多而且它们很简单)。

您请求的大多数案例输出都是将分类与模型的其他输出相结合的东西，因此您的失败闭包很可能应该替换为:

guard let results = request.results as? [VNCoreMLFeatureValueObservation] else {
    fatalError("Model failed to load results")
}

请记住相应地修改引用results 的代码。 VNCoreMLFeatureValueObservation 协议(protocol)包含有关可能值的信息。您还可以使用以下命令将它们从您的模型打印到调试器:

print(model.modelDescription.outputDescriptionsByName)

这应该让您明白您应该期待什么。