ios - Replaykit，startCaptureWithHandler() 不在 captureHandler 中发送 Video 类型的 CMSampleBufferRef

Question

我已经实现了一个 RPScreenRecorder，它可以记录屏幕和麦克风音频。完成多个录制后，我停止录制并使用 AVMutableComposition 将音频与视频合并，然后合并所有视频以形成单个视频。

对于屏幕录制和获取视频和音频文件，我正在使用

- (void)startCaptureWithHandler:(nullable void(^)(CMSampleBufferRef sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable error))captureHandler completionHandler:

用于停止录制。我调用这个函数:

- (void)stopCaptureWithHandler:(void (^)(NSError *error))handler;

这些都非常简单。

大多数情况下，它工作得很好，我收到了视频和音频 CMSampleBuffers。 但有时 startCaptureWithHandler 只向我发送音频缓冲区而不是视频缓冲区。 一旦我遇到这个问题，我会重新启动我的设备并重新安装应用程序。这使得我的应用程序对用户来说非常不可靠。 我认为这是一个重播工具包问题，但无法与其他开发人员一起找出相关问题。 如果你们中有人遇到过这个问题并找到了解决方案，请告诉我。

我检查了很多次，但没有发现任何配置问题。 但无论如何它就在这里。

NSError *videoWriterError;
videoWriter = [[AVAssetWriter alloc] initWithURL:fileString fileType:AVFileTypeQuickTimeMovie
                                           error:&videoWriterError];


NSError *audioWriterError;
audioWriter = [[AVAssetWriter alloc] initWithURL:audioFileString fileType:AVFileTypeAppleM4A
                                           error:&audioWriterError];

CGFloat width =UIScreen.mainScreen.bounds.size.width;
NSString *widthString = [NSString stringWithFormat:@"%f", width];
CGFloat height =UIScreen.mainScreen.boNSString *heightString = [NSString stringWithFormat:@"%f", height];unds.size.height;

NSDictionary  * videoOutputSettings= @{AVVideoCodecKey : AVVideoCodecTypeH264,
                                       AVVideoWidthKey: widthString,
                                       AVVideoHeightKey : heightString};
videoInput  = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeVideo outputSettings:videoOutputSettings];

videoInput.expectsMediaDataInRealTime = true;

AudioChannelLayout acl;
bzero( &acl, sizeof(acl));
acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono;
NSDictionary * audioOutputSettings = [ NSDictionary dictionaryWithObjectsAndKeys:
                                      [ NSNumber numberWithInt: kAudioFormatAppleLossless ], AVFormatIDKey,
                                      [ NSNumber numberWithInt: 16 ], AVEncoderBitDepthHintKey,
                                      [ NSNumber numberWithFloat: 44100.0 ], AVSampleRateKey,
                                      [ NSNumber numberWithInt: 1 ], AVNumberOfChannelsKey,
                                      [ NSData dataWithBytes: &acl length: sizeof( acl ) ], AVChannelLayoutKey,
                                      nil ];

audioInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeAudio outputSettings:audioOutputSettings];

[audioInput setExpectsMediaDataInRealTime:YES];

[videoWriter addInput:videoInput];
    [audioWriter addInput:audioInput];
    
    [[AVAudioSession sharedInstance] setCategory: AVAudioSessionCategoryPlayAndRecord withOptions:AVAudioSessionCategoryOptionDefaultToSpeaker error:nil];

[RPScreenRecorder.sharedRecorder startCaptureWithHandler:^(CMSampleBufferRef  _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable myError) {

Block

}

startCaptureWithHandler 函数也有非常简单的功能:

[RPScreenRecorder.sharedRecorder startCaptureWithHandler:^(CMSampleBufferRef  _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable myError) {
                    
                    dispatch_sync(dispatch_get_main_queue(), ^{
                        
                        
                        if(CMSampleBufferDataIsReady(sampleBuffer))
                        {
                            
                            if (self->videoWriter.status == AVAssetWriterStatusUnknown)
                            {
                                    self->writingStarted = true;
                                    [self->videoWriter startWriting];
                                    [self->videoWriter startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp(sampleBuffer)];
                                    
                                    [self->audioWriter startWriting];
                                    [self->audioWriter startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp(sampleBuffer)];
                            }
                            if (self->videoWriter.status == AVAssetWriterStatusFailed) {
                                return;
                            }
                            
                            if (bufferType == RPSampleBufferTypeVideo)
                            {
                                
                                if (self->videoInput.isReadyForMoreMediaData)
                                {
                                        [self->videoInput appendSampleBuffer:sampleBuffer];
                                }
                            }
                            else if (bufferType == RPSampleBufferTypeAudioMic)
                            {
                                //                                printf("\n+++ bufferAudio received %d \n",arc4random_uniform(100));
                                if (writingStarted){
                                    if (self->audioInput.isReadyForMoreMediaData)
                                    {
                                            [self->audioInput appendSampleBuffer:sampleBuffer];
                                    }
                                }
                            }
                            
                        }
                    });
                    
                }

此外，当这种情况发生时，系统屏幕录像机也会损坏。单击系统记录器时，出现此错误:

错误显示“屏幕录制已停止，原因是:由于媒体服务错误导致录制失败”。

肯定有两个原因:

1. iOS Replay 套件处于测试阶段，这就是为什么它有时会在使用后出现问题。
2. 我已经实现了任何导致 replaykit 崩溃的有问题的逻辑。

如果是问题号。 1、那就没问题了。 如果这是问题号。 2 那么我必须知道我可能哪里错了？

我们将不胜感激您的意见和帮助。

Answer 1

因此，我遇到过一些 Replay 套件完全崩溃并且系统记录器每次都显示错误的情况，除非您重新启动设备。

第一个场景

当您开始录制并在完成处理程序中停止时

[RPScreenRecorder.sharedRecorder startCaptureWithHandler:^(CMSampleBufferRef  _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable error) {
    printf("recording");
} completionHandler:^(NSError * _Nullable error) {
    [RPScreenRecorder.sharedRecorder stopCaptureWithHandler:^(NSError * _Nullable error) {
        printf("Ended");
    }];
}];

第二种情况

当您开始录制并直接在捕获处理程序中停止时

__block BOOL stopDone = NO;
[RPScreenRecorder.sharedRecorder startCaptureWithHandler:^(CMSampleBufferRef  _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable error) {
    if (!stopDone){
        [RPScreenRecorder.sharedRecorder stopCaptureWithHandler:^(NSError * _Nullable error) {
            printf("Ended");
        }];
        stopDone = YES;
    }
    printf("recording");
} completionHandler:^(NSError * _Nullable error) {}];

More Scenarios are yet to be discovered and I will keep updating the answer

更新 1

It is true that the system screen recorded gives error when we stop recording right after the start, but it seem to work alright after we call startcapture again.

I have also encountered a scenario where I don't get video buffer in my app only and the system screen recorder works fine, will update the solution soon.

更新 2

So here is the issue, My actual app is old and it is being maintained and getting updated timely. When the replaykit becomes erroneous, My original app can't receive video buffers, I don't know if there is a configuration that is making this happen, or what?

But new sample app seem to work fine and after replay kit becomes erroneous. when I call startCapture next time, the replay kit becomes fine. Weird

更新 3

我发现了新问题。当出现权限警报时，应用程序将进入后台。由于我编写了代码，每当应用程序进入后台时，都会发生一些 UI 更改并且录制将停止。 这导致了错误

Recording interrupted by multitasking and content resizing

我还不确定是哪个特定的 UI 更改导致了此失败，但它只会在出现权限警报并进行 UI 更改时出现。 如果有人注意到此问题的任何特殊情况，请告诉我们。