ios - FFmpegKit 多个覆盖过滤器导致内存错误 - Flutter(仅适用于 iOS)

我需要将叠加图像放到视频中。它在Android上运行没有问题。
但是在 iOS 平台上，如果我尝试 23-24 叠加图像，它可以正常工作。
如果我尝试使用 30 多张图像，则会出现内存分配错误。
过滤时出错:无法分配内存
无法将帧注入(inject)过滤器网络:无法分配内存
每个叠加图像大约 50 kb
视频大约 250 MB
我尝试使用较小的图像，所以我可以毫无问题地使用 40 多个图像，所以它与计数无关，它与文件大小有关。
我认为复杂的过滤器流有 1MB 之类的限制。
我尝试了很多想法，但没有运气。我有两个问题:

我的 ffmpeg 命令是否正确？

你能给我建议任何改进，替代方案吗？

更新:我想做什么？
我正在尝试制作刻录的字幕视频。但我也需要支持表情符号。所以我想出了这些步骤:

将所有字幕项创建为 .png 图像。

以正确的时间将这些图像叠加到视频中。

FFmpeg 命令:

-i video.mov -i image1.png -i image1.png -i ... image43.png
-filter_complex "[0][1]overlay=x=0:y=0:enable='between(t,10.9,16.4)'[out];[out][2]overlay=x=0:y=0:enable='between(t,16.8,20.0)'[out];[out][3]overlay=x=0:y=0:enable='between(t,20.0,22.4)'[out];[out][4]overlay=x=0:y=0:enable='between(t,22.4,26.5)'[out];[out][5]overlay=x=0:y=0:enable='between(t,26.5,29.8)'[out];[out][6]overlay=x=0:y=0:enable='between(t,30.0,33.7)'[out];[out][7]overlay=x=0:y=0:enable='between(t,33.7,38.4)'[out];[out][8]overlay=x=0:y=0:enable='between(t,38.6,41.5)'[out];[out][9]overlay=x=0:y=0:enable='between(t,41.5,44.2)'[out];[out][10]overlay=x=0:y=0:enable='between(t,44.3,46.5)'[out];[out][11]overlay=x=0:y=0:enable='between(t,47.2,48.4)'[out];[out][12]overlay=x=0:y=0:enable='between(t,49.9,52.6)'[out];[out][13]overlay=x=0:y=0:enable='between(t,52.6,54.2)'[out];[out][14]overlay=x=0:y=0:enable='between(t,54.2,57.0)'[out];[out][15]overlay=x=0:y=0:enable='between(t,57.0,57.6)'[out];[out][16]overlay=x=0:y=0:enable='between(t,63.0,65.6)'[out];[out][17]overlay=x=0:y=0:enable='between(t,66.5,71.5)'[out];[out][18]overlay=x=0:y=0:enable='between(t,72.2,73.7)'[out];[out][19]overlay=x=0:y=0:enable='between(t,74.4,75.7)'[out];[out][20]overlay=x=0:y=0:enable='between(t,77.9,78.7)'[out];[out][21]overlay=x=0:y=0:enable='between(t,83.5,85.4)'[out];[out][22]overlay=x=0:y=0:enable='between(t,87.3,91.0)'[out];[out][23]overlay=x=0:y=0:enable='between(t,91.0,96.1)'[out];[out][24]overlay=x=0:y=0:enable='between(t,96.1,100.8)'[out];[out][25]overlay=x=0:y=0:enable='between(t,100.8,103.5)'[out];[out][26]overlay=x=0:y=0:enable='between(t,103.5,106.9)'[out];[out][27]overlay=x=0:y=0:enable='between(t,106.9,109.1)'[out];[out][28]overlay=x=0:y=0:enable='between(t,109.2,110.6)'[out];[out][29]overlay=x=0:y=0:enable='between(t,110.6,113.7)'[out];[out][30]overlay=x=0:y=0:enable='between(t,114.8,117.6)'[out];[out][31]overlay=x=0:y=0:enable='between(t,117.6,119.8)'[out];[out][32]overlay=x=0:y=0:enable='between(t,120.5,122.8)'[out];[out][33]overlay=x=0:y=0:enable='between(t,123.5,125.0)'[out];[out][34]overlay=x=0:y=0:enable='between(t,128.3,129.1)'[out];[out][35]overlay=x=0:y=0:enable='between(t,129.7,130.5)'[out];[out][36]overlay=x=0:y=0:enable='between(t,131.4,136.3)'[out];[out][37]overlay=x=0:y=0:enable='between(t,137.1,138.6)'[out];[out][38]overlay=x=0:y=0:enable='between(t,139.3,139.9)'[out];[out][39]overlay=x=0:y=0:enable='between(t,143.2,143.6)'[out];[out][40]overlay=x=0:y=0:enable='between(t,148.8,152.2)'[out];[out][41]overlay=x=0:y=0:enable='between(t,152.2,155.3)'[out];[out][42]overlay=x=0:y=0:enable='between(t,155.3,158.2)'[out];[out][43]overlay=x=0:y=0:enable='between(t,158.2,159.2)'[out]"
-map [out] -map 0:a? -vsync vfr -vcodec libx264 -x264-params crf=23 -preset superfast -acodec aac output.mp4

日志:

ffmpeg version v4.5-dev-3393-g30322ebe3c Copyright (c) 2000-2021 the FFmpeg developers
built with Apple clang version 13.0.0 (clang-1300.0.29.30)
configuration: --cross-prefix=arm64-ios-darwin- --sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.2.sdk --prefix=/Users/taner/Projects/ffmpeg-kit/prebuilt/apple-ios-arm64/ffmpeg --pkg-config=/opt/homebrew/bin/pkg-config --enable-version3 --arch=aarch64 --cpu=armv8 --target-os=darwin --enable-neon --enable-asm --ar=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar --cc=clang --cxx=clang++ --as='/Users/taner/Projects/ffmpeg-kit/.tmp/gas-preprocessor.pl -arch aarch64 -- clang -arch arm64 -target arm64-apple-ios12.1 -march=armv8-a+crc+crypto -mcpu=generic -DFFMPEG_KIT_ARM64 -Wno-unused-function -Wno-deprecated-declarations -fstrict-aliasing -fembed-bitcode -DIOS -DFFMPEG_KIT_BUILD_DATE=20220114 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.2.sdk -Oz -miphoneos-version-min=12.1 -I/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.2.sdk/usr/include' --ranlib=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib --strip=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip --nm=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm --extra-ldflags='-miphoneos-version-min=12.1' --disable-autodetect --enable-cross-compile --enable-pic --enable-inline-asm --enable-optimizations --enable-swscale --enable-shared --disable-static --install-name-dir='@rpath' --enable-pthreads --disable-v4l2-m2m --disable-outdev=v4l2 --disable-outdev=fbdev --disable-indev=v4l2 --disable-indev=fbdev --enable-small --disable-xmm-clobber-test --disable-debug --disable-neon-clobber-test --disable-programs --disable-postproc --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-sndio --disable-schannel --disable-securetransport --disable-xlib --disable-cuda --disable-cuvid --disable-nvenc --disable-vaapi --disable-vdpau --disable-alsa --disable-cuda --disable-cuvid --disable-nvenc --disable-vaapi --disable-vdpau --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gmp --enable-gnutls --enable-libmp3lame --enable-libass --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxml2 --enable-libopencore-amrnb --enable-libshine --enable-libspeex --enable-libdav1d --enable-libkvazaar --enable-libx264 --enable-libxvid --enable-libx265 --enable-libvidstab --enable-libilbc --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libtwolame --disable-sdl2 --enable-libvo-amrwbenc --enable-libzimg --disable-openssl --enable-zlib --enable-audiotoolbox --disable-outdev=audiotoolbox --enable-bzlib --enable-videotoolbox --enable-avfoundation --enable-iconv --disable-coreimage --disable-appkit --disable-opencl --disable-opengl --enable-gpl
libavutil 57. 13.100 / 57. 13.100
libavcodec 59. 15.102 / 59. 15.102
libavformat 59. 10.100 / 59. 10.100
libavdevice 59. 1.100 / 59. 1.100
libavfilter 8. 21.100 / 8. 21.100
libswscale 6. 1.102 / 6. 1.102
libswresample 4. 0.100 / 4. 0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mov':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2022-03-19T16:24:25.000000Z
encoder : Lavf58.20.100
Duration: 00:02:47.77, start: 0.000000, bitrate: 12431 kb/s
Stream #0:00x1: Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 248 kb/s (default)
Metadata:
creation_time : 2022-03-19T16:24:25.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
Stream #0:10x2: Video: h264 (avc1 / 0x31637661), yuv420p(progressive), 1080x1920 [SAR 1:1 DAR 9:16], 12174 kb/s, 29.99 fps, 30 tbr, 90k tbn (default)
Metadata:
creation_time : 2022-03-19T16:24:25.000000Z
handler_name : Core Media Video
vendor_id : FFMP
Input #1, png_pipe, from 'image1.png':
Duration: N/A, bitrate: N/A
Stream #1:0: Video: png, rgba(pc), 1080x1920, 25 fps, 25 tbr, 25 tbn
Input #2, png_pipe, from 'image2.png':
Duration: N/A, bitrate: N/A
Stream #2:0: Video: png, rgba(pc), 1080x1920, 25 fps, 25 tbr, 25 tbn
.
.
Input #43, png_pipe, from 'image43.png':
Duration: N/A, bitrate: N/A
Stream #43:0: Video: png, rgba(pc), 1080x1920, 25 fps, 25 tbr, 25 tbn
[aac @ 0x1047f4480] The "sub_text_format" option is deprecated: Deprecated, does nothing
[h264 @ 0x15f2a70f0] The "sub_text_format" option is deprecated: Deprecated, does nothing
[png @ 0x15f47f710] The "sub_text_format" option is deprecated: Deprecated, does nothing
[png @ 0x1059da740] The "sub_text_format" option is deprecated: Deprecated, does nothing
.
.
[png @ 0x15f2a8780] The "sub_text_format" option is deprecated: Deprecated, does nothing
Stream mapping:
Stream #0:1 (h264) -> scale (graph 0)
Stream #1:0 (png) -> overlay:overlay (graph 0)
Stream #2:0 (png) -> overlay:overlay (graph 0)
Stream #3:0 (png) -> overlay:overlay (graph 0)
Stream #4:0 (png) -> overlay:overlay (graph 0)
Stream #5:0 (png) -> overlay:overlay (graph 0)
Stream #6:0 (png) -> overlay:overlay (graph 0)
Stream #7:0 (png) -> overlay:overlay (graph 0)
Stream #8:0 (png) -> overlay:overlay (graph 0)
Stream #9:0 (png) -> overlay:overlay (graph 0)
Stream #10:0 (png) -> overlay:overlay (graph 0)
Stream #11:0 (png) -> overlay:overlay (graph 0)
Stream #12:0 (png) -> overlay:overlay (graph 0)
Stream #13:0 (png) -> overlay:overlay (graph 0)
Stream #14:0 (png) -> overlay:overlay (graph 0)
Stream #15:0 (png) -> overlay:overlay (graph 0)
Stream #16:0 (png) -> overlay:overlay (graph 0)
Stream #17:0 (png) -> overlay:overlay (graph 0)
Stream #18:0 (png) -> overlay:overlay (graph 0)
Stream #19:0 (png) -> overlay:overlay (graph 0)
Stream #20:0 (png) -> overlay:overlay (graph 0)
Stream #21:0 (png) -> overlay:overlay (graph 0)
Stream #22:0 (png) -> overlay:overlay (graph 0)
Stream #23:0 (png) -> overlay:overlay (graph 0)
Stream #24:0 (png) -> overlay:overlay (graph 0)
Stream #25:0 (png) -> overlay:overlay (graph 0)
Stream #26:0 (png) -> overlay:overlay (graph 0)
Stream #27:0 (png) -> overlay:overlay (graph 0)
Stream #28:0 (png) -> overlay:overlay (graph 0)
Stream #29:0 (png) -> overlay:overlay (graph 0)
Stream #30:0 (png) -> overlay:overlay (graph 0)
Stream #31:0 (png) -> overlay:overlay (graph 0)
Stream #32:0 (png) -> overlay:overlay (graph 0)
Stream #33:0 (png) -> overlay:overlay (graph 0)
Stream #34:0 (png) -> overlay:overlay (graph 0)
Stream #35:0 (png) -> overlay:overlay (graph 0)
Stream #36:0 (png) -> overlay:overlay (graph 0)
Stream #37:0 (png) -> overlay:overlay (graph 0)
Stream #38:0 (png) -> overlay:overlay (graph 0)
Stream #39:0 (png) -> overlay:overlay (graph 0)
Stream #40:0 (png) -> overlay:overlay (graph 0)
Stream #41:0 (png) -> overlay:overlay (graph 0)
Stream #42:0 (png) -> overlay:overlay (graph 0)
Stream #43:0 (png) -> overlay:overlay (graph 0)
overlay (graph 0) -> Stream #0:0 (libx264)
Stream #0:0 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
Error while filtering: Cannot allocate memory
Failed to inject frame into filter network: Cannot allocate memory
Error while processing the decoded data for stream #43:0
[aac @ 0x15f23b480] Qavg: 12511.496
[aac @ 0x15f23b480] 2 frames left in the queue on closing
Conversion failed!

最佳答案

您正在经历的是大型过滤器图的性质。过滤器之间的每个链接都需要一个帧缓冲区(至少 6 MB)，过滤操作本身可能需要额外的内存空间。因此，它必须用完您的 iDevice 的内存(必须比 Android 小)。
因此，解决方案必须是最小化过滤器数量的解决方案，您可以使用 concat 来做到这一点。解复用器，因此您的所有图像都来自一个(虚拟)源，并使用 overlay更复杂的 enable选项。
png_list.txt

ffconcat version 1.0

# starts at 0 and ends sometime between 16.4 and 16.8 seconds (picked 16.5)
file image1.png
duration 16.5

# starts at 16.5 and ends at 20
file image2.png
duration 3.5

# starts at 20.0 and ends at 22.4
file image3.png
duration 2.4

...

图像将在没有任何暂停的情况下连接，因此持续时间的总和必须至少与最后一个叠加图像的结束时间一样长，即 159.2 秒

确保正确选择持续时间，以便每个图像的实际开始和结束时间包含在总显示持续时间

内

FFmpeg 通话

ffmpeg -i video.mov -f concat -i png_list.txt \
  -filter_complex "[0][1]overlay=enable=\
    'between(t,10.9,16.4) + 
     between(t,10.9,16.4) + 
     between(t,16.8,20.0) + ... + 
     between(t,158.2,159.2)'[out]"
  -map [out] -map 0:a? ... output.mp4

我不能保证这会避免这个问题，但这在内存上应该容易得多。
刻录字幕
您可能已经知道这一点，并且您使用基于位图的字幕一定是有原因的，但是有 subtitles和 ass如果您有文本格式的字幕，过滤器可将字幕文本刻录到视频上。

关于ios - FFmpegKit 多个覆盖过滤器导致内存错误 - Flutter(仅适用于 iOS)，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/71540219/

ios - FFmpegKit 多个覆盖过滤器导致内存错误 - Flutter(仅适用于 iOS)

上一篇：audio - 如何使用 Libav API 合并多个音频文件？

下一篇：ffmpeg 输出比例不适用于帧率