iphone - 使用 AVAssetReader 从远程 Assets 读取(流)

Question

我的主要目标是从服务器流式传输视频，并在流式传输时逐帧剪切它(以便 OpenGL 使用)。为此，我使用了在互联网上随处可见的这段代码(我记得它来自 Apple 的 GLVideoFrame 示例代码):

NSArray * tracks = [asset tracks];
NSLog(@"%d", tracks.count);

for(AVAssetTrack* track in tracks) {

    NSLog(@"type: %@", [track mediaType]);

    initialFPS = track.nominalFrameRate;
    width = (GLuint)track.naturalSize.width;
    height = (GLuint)track.naturalSize.height;


    NSError * error = nil;

    // _movieReader is a member variable
    @try {
        self._movieReader = [[[AVAssetReader alloc] initWithAsset:asset error:&error] autorelease];
    }
    @catch (NSException *exception) {
        NSLog(@"%@ -- %@", [exception name], [exception reason]);
        NSLog(@"skipping track");

        continue;
    }


    if (error)
    {
        NSLog(@"CODE:%d\nDOMAIN:%@\nDESCRIPTION:%@\nFAILURE_REASON:%@", [error code], [error domain], error.localizedDescription, [error localizedFailureReason]);                                          
        continue;
    }

    NSString* key = (NSString*)kCVPixelBufferPixelFormatTypeKey;
    NSNumber* value = [NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA];
    NSDictionary* videoSettings = [NSDictionary dictionaryWithObject:value forKey:key]; 
    [_movieReader addOutput:[AVAssetReaderTrackOutput assetReaderTrackOutputWithTrack:track
                                                                       outputSettings:videoSettings]];
    [_movieReader startReading];
    [self performSelectorOnMainThread:@selector(frameStarter) withObject:nil waitUntilDone:NO];
}

但我总是在 [[AVAssetReader alloc] initWithAsset:error:] 处遇到此异常。

NSInvalidArgumentException -- *** -[AVAssetReader initWithAsset:error:] Cannot initialize an instance of AVAssetReader with an asset at non-local URL 'http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8'

所以我的两个问题是:

1. 异常是否真的告诉我 AVAssetReader 必须有本地 URL？它可以用于流式传输(就像 AVFoundation 类的其余部分一样)吗？
2. 如果 AVFoundation 方法不起作用，对于流式传输视频并同时分割其帧还有哪些其他建议？

非常感谢您的帮助。

Answer 1

AVFoundation 似乎并没有像区分所使用的文件或协议(protocol)的种类那样区分本地文件和非本地文件。使用 mp4/mov 与通过 m3u8 使用 HTTP Live 流协议(protocol)之间有非常明显的区别，但使用本地或远程 mp4 的差异有点模糊。

扩展上述内容:

a) 如果您的“远程” Assets 是 M3U8(即您正在使用 HTTP“实时”流媒体)，那么就没有任何机会。无论 M3U8 位于本地文件系统还是远程服务器上，由于多种原因，AVAssetReader 和所有 AVAsset 相关功能都无法工作。 However, AVPlayer, AVPlayerItem etc would work just fine.

b) 如果是 MP4/MOV，则需要进一步调查。 Local MP4/MOV's work flawlessly.虽然在远程 MP4/MOV 的情况下，我能够创建(或从 AVPlayerItem 或 AVPlayer 或 AVAssetTracks 检索)AVURLAsset，有时我可以使用它成功初始化 AVAssetReader (我将扩展“有时”)也很快)。然而，copyNextSampleBuffer always returns nil in case of remote MP4's 。由于调用 copyNextSampleBuffer 之前有几件事起作用，所以我不能 100% 确定是否:

i) copyNextSampleBuffer not working for remote mp4's, after all the other steps having been successful, is intended/expected functionality.

ii) That the 'other steps' seem to work at all for remote MP4's is an accident of Apple's implementation, and this incompatibility is simply coming to the fore when we hit copyNextSampleBuffer..............what these 'other steps' are, I'll detail shortly.

iii) I'm doing something wrong when trying to invoke copyNextSampleBuffer for remote MP4's.

所以@Paula，您可以尝试使用远程 MOV/MP4 进行进一步调查。

作为引用，以下是我尝试从视频中捕获帧的方法:

a)

Create an AVURLAsset directly from the video URL.

Retrieve the video track using [asset tracksWithMediaType:AVMediaTypeVideo]

Prepare an AVAssetReaderTrackOutput using the video track as the source.

Create an AVAssetReader using the AVURLAsset.

Add AVAssetReaderTrackOutput to the AVAssetReader and startReading.

Retrieve images using copyNextSampleBuffer.

b)

Create an AVPlayerItem from the video URL, and then an AVPlayer from it (or create the AVPlayer directly from the URL).

Retrieve the AVPlayer's 'asset' property and load its 'tracks' using "loadValuesAsynchronouslyForKeys:".

Separate the tracks of type AVMediaTypeVideo (or simply call tracksWithMediaType: on the asset once the tracks are loaded), and create your AVAssetReaderTrackOutput using the video track.

Create AVAssetReader using the AVPlayer's 'asset', 'startReading' and then retrieve images using copyNextSampleBuffer.

c)

Create an AVPlayerItem+AVPlayer or AVPlayer directly from the video URL.

KVO the AVPlayerItem's 'tracks' property, and once the tracks are loaded, separate the AVAssetTracks of type AVMediaTypeVideo.

Retrieve the AVAsset from AVPlayerItem/AVPlayer/AVAssetTrack's 'asset' property.

Remaining steps are similar to approach (b).

d)

Create an AVPlayerItem+AVPlayer or AVPlayer directly from the video URL.

KVO the AVPlayerItem's 'tracks' property, and once the tracks are loaded, separate the ones of type AVMediaTypeVideo.

Create an AVMutableComposition, and initialize an associated AVMutableCompositionTrack of type AVMediaTypeVideo.

Insert the appropriate CMTimeRange from video track retrieved earlier, into this AVMutableCompositionTrack.

Similar to (b) and (c), now create your AVAssetReader and AVAssetReaderTrackOutput, but with the difference that you use the AVMutableComposition as the base AVAsset for initializing your AVAssetReader, and AVMutableCompositionTrack as the base AVAssetTrack for your AVAssetReaderTrackOutput.

'startReading' and use copyNextSampleBuffer to get frames from the AVAssetReader.

P.S: I tried approach (d) here to get around the fact that the AVAsset retrieved directly from AVPlayerItem or AVPlayer was not behaving. So I wanted to create a new AVAsset from the AVAssetTracks I already had in hand. Admittedly hacky, and perhaps pointless (where else would the track information be ultimately retrieved from if not the original AVAsset!) but it was worth a desperate try anyway.

以下是不同类型文件的结果摘要:

1) Local MOV/MP4's - All 4 approaches work flawlessly.

2) Remote MOV/MP4's - The asset and tracks are retrieved correctly in approaches (b) through (d), and the AVAssetReader is initialized as well but copyNextSampleBuffer always returns nil. In case of (a), creation of the AVAssetReader itself fails with an 'Unknown Error' NSOSStatusErrorDomain -12407.

3) Local M3U8's (accessed through an in-app/local HTTP server) - Approaches (a), (b) and (c) fail miserably as trying to get an AVURLAsset/AVAsset in any shape or form for files streamed via M3U8's is a fools errand.

In case of (a), the asset is not created at all, and the initWithURL: call on AVURLAsset fails with an 'Unknown Error' AVFoundationErrorDomain -11800.

In case of (b) and (c), retrieving the AVURLAsset from the AVPlayer/AVPlayerItem or AVAssetTracks returns SOME object, but accessing the 'tracks' property on it always returns an empty array.

In case of (d), I'm able to retrieve and isolate the video tracks successfully, but while trying to create the AVMutableCompositionTrack, it fails when trying to insert the CMTimeRange from the source track into the AVMutableCompositionTrack, with an 'Unknown Error' NSOSStatusErrorDomain -12780.

4) 远程 M3U8 的行为与本地 M3U8 的行为完全相同。

我并不完全了解为什么这些差异存在，或者苹果无法减轻这些差异。但就这样吧。