java - 谷歌探戈/opencv Android 相机应用程序崩溃/挂起

Question

我正在开发一个原型(prototype)应用程序，利用探戈深度相机进行模板匹配。到目前为止，我已经能够借助已经回答的问题、文档等解决所有问题。

但是，现在我似乎停了下来。我所做的最后一个主要更改是在显示相机预览图像之前绘制轮廓。无论我现在做什么，应用程序都会运行一段时间，然后就会崩溃。

我查看了 Android Studio 中的堆转储和分配跟踪。唯一可能奇怪的事情是堆转储中的 FinalizerReference 对象上可能有大量内存...

我还尝试将处理移至 AsyncTask，并跳过每个颜色帧，直到任务完成(以便一次只运行一个任务)并显示处理后的帧，但问题仍然存在。

我使用 Google Tango 获取颜色和深度相机数据，并使用 java 中的 opencv 来分析数据并进行模板匹配。

有人知道这些 logcat 消息的含义吗？

Logcat 错误:

E/lowmemorykiller: Error opening /proc/10173/oom_score_adj; errno=2
E/mm-camera-isp2: abf40_trigger_update:587 aec_ratio.ratio = 0.039062
W/ActivityManager: Scheduling restart of crashed service com.lenovo.lsf.device/com.lenovo.lsf.push.service.PushService in 68068ms
W/ActivityManager: Scheduling restart of crashed service com.qualcomm.qti.modemtestmode/.MbnSystemService in 88020ms
E/InputDispatcher: channel '478a66c com.android.documentsui/com.android.documentsui.DocumentsActivity (server)' ~ Channel is unrecoverably broken and will be disposed!
E/lowmemorykiller: Error writing /proc/10113/oom_score_adj; errno=22
E/JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 76)
E/lowmemorykiller: Error writing /proc/32408/oom_score_adj; errno=22
E/InputDispatcher: channel '85c4188 com.android.launcher3/com.android.launcher3.Launcher (server)' ~ Channel is unrecoverably broken and will be disposed!
E/mm-camera-isp2: abf40_trigger_update:587 aec_ratio.ratio = 0.000000
E/ConnectivityService: RemoteException caught trying to send a callback msg for NetworkRequest [ id=332, legacyType=-1, [ Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED] ]
E/mm-camera: mct_util_timer_handler:Error Backend stuck during HAL Command Raising SIGABRT
E/JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 76)
E/lowmemorykiller: Error writing /proc/32437/oom_score_adj; errno=22
E/JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 308)
E/JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 76)
E/lowmemorykiller: Error writing /proc/32437/oom_score_adj; errno=22
E/JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 76)
E/mm-camera: mct_util_timer_handler:Error Backend stuck during HAL Command Raising SIGABRT E/Surface: queueBuffer: error queuing buffer to SurfaceTexture, -32
E/Surface: queueBuffer: error queuing buffer to SurfaceTexture, -32
E/mm-camera: cpp_module_send_buf_divert_event:545] buffer event received with no divert config
E/mm-camera: cpp_module_send_buf_divert_event:545] buffer event received with no divert config
E/Camera3-OutputStream: getBufferLocked: Stream 0: Can't dequeue next output buffer: Broken pipe (-32)
E/Camera3-OutputStream: returnBufferCheckedLocked: Stream 0: Error queueing buffer to native window: Broken pipe (-32)
E/Camera3-Device: RequestThread: Can't get output buffer, skipping request: Broken pipe (-32)
E/Camera3-Device: Can't return buffer to its stream: Broken pipe (-32)
E/Camera3-OutputStream: getBufferLocked: Stream 0: Can't dequeue next output buffer: Broken pipe (-32)
E/Camera3-Device: RequestThread: Can't get output buffer, skipping request: Broken pipe (-32)

显示相机预览的代码:

mTango.experimentalConnectOnFrameListener(TangoCameraIntrinsics.TANGO_CAMERA_COLOR, new Tango.OnFrameAvailableListener() {
    byte[] imageByteArray = new byte[colorCameraIntrinsics.height * colorCameraIntrinsics.width * 3 / 2];  
    Mat yuvMat = new Mat( colorCameraIntrinsics.height + colorCameraIntrinsics.height/2, colorCameraIntrinsics.width, CvType.CV_8UC1 );
    Bitmap bitmapDisplay = Bitmap.createBitmap( colorCameraIntrinsics.width, colorCameraIntrinsics.height, Bitmap.Config.ARGB_8888 );
    Mat colorMatDisplay;

    @Override
    public void onFrameAvailable(TangoImageBuffer imageBuffer, int cameraId) {
        Log.d(TAG, "onFrameAvailable: color frame available");
        if (colorFrameCounter < 3) { //only use every 3rd frame
            Log.d(TAG, "onFrameAvailable: skipping frame " + colorFrameCounter);
            colorFrameCounter++;
        } else {
            colorFrameCounter = 0;
            if (!stopCameraView.get()) { // Only view frame if camera view is not stopped

                // convert image buffer data to byte array
                imageBuffer.data.get(imageByteArray);

                // byte array to Mat object with YUV coding (NV21)
                yuvMat.put(0, 0, imageByteArray);

                // locking access to lastColorFrameMat
                synchronized (lockVar) {
                    Log.d(TAG, "onFrameAvailable: converting to bitmap");
                    // convert from YUV (NV21) Mat to RGBA Mat and place in lastColorFrameMat (global)
                    Imgproc.cvtColor(yuvMat, lastColorFrameMat, Imgproc.COLOR_YUV2RGBA_NV21, 4);
                    colorMatDisplay = lastColorFrameMat.clone();
                }

                if (templateContours != null) {
                    Imgproc.drawContours(colorMatDisplay, templateContours, templateContourMaxIdx, new Scalar(0, 255, 0, 255), 5);
                }

                // convert colorMatDisplay to bitmap, for display in imageview
                Utils.matToBitmap(colorMatDisplay, bitmapDisplay);

                Log.d(TAG, "onFrameAvailable: view lastColorFrameMat on phone display");

                // View colorImage in imageViewer on UI thread
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        imageViewer.setImageBitmap(bitmapDisplay);
                    }
                });
            }
        }

    }
});

Answer 1

这里可能发生两件事:

1. 相机图像的缓冲区超出范围。

在当前的 Tango SDK 中，onFrameAvailable 回调的被调用方只能控制回调范围内的 imageBuffer。这意味着如果您在 AsynTask 中引用 imageBuffer，您可能会得到一个空缓冲区，并导致崩溃。 Tango解决这个问题的方法是始终从回调中深拷贝出数据，并在另一个线程中处理它。

该进程阻塞 Tango 线程的时间过长。

如果没有 AsyncTask，可能会发生处理阻塞 Tango 线程太长时间的情况，而这对 Tango 来说也是不利的。