node.js - 如何将缩略图/屏幕截图序列从 ffmpeg 传输到 nodejs 以进行进一步处理？

标签 node.js ffmpeg stdout spawn

我想要实现的是连续截图并将它们传递给nodejs应用程序，以便单独处理每个截图并做一些其他事情。我只需要这个用于 Linux 环境。我选择了 ffmpeg 和 x11grab 作为屏幕截图提供程序。以下命令可以正常工作:

ffmpeg -t 10 -s 1366x768 -f x11grab -i :0.0+0,0 -vf fps=30 output_%d.png -y

它在 10 秒内创建了 300 个屏幕后续帧。然后我想将输出重定向到我的 Nodejs 应用程序，而不仅仅是在硬盘驱动器上写入文件。所以我从 Node 调用 ffmpeg:

var spawn = require('child_process').spawn,
    fps = 30,
    duration = 10,
    screenSize = {w: 1366, h: 768},
    args = [
        '-t', 
        duration, 
        '-s', 
        screenSize.w + 'x' + screenSize.h, 
        '-f', 
        'x11grab', 
        '-i', 
        ':0.0', 
        '-vf',
        'fps=' + fps,
        '-f',
        'mjpeg',
        'pipe:1'
    ],
    ff = spawn('ffmpeg', args);

ff.stdout.on('data', function (data) {
    console.log('Data size: ' + data.length);
});

ff.stdout.on('end', function (data) {
    console.log('Stream end');
});

ff.stderr.on('data', function (data) {
    console.log('ff error: ' + data);
});

很抱歉日志很长，但这很重要:

ff error: ffmpeg version N-77455-g4707497 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04)
  configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libdcadec --enable-libfreetype --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvo-aacenc --enable-libvidstab
  libavutil      55. 11.100 / 55. 11.100
  libavcodec     57. 20.100 / 57. 20.100
  libavformat    57. 20.100 / 57. 20.100
  libavdevice    57.  0.100 / 57.  0.100
  libavfilter     6. 21.101 /  6. 21.101
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100

ff error: Input #0, x11grab, from ':0.0':
  Duration: N/A, start: 1451414448.216650, bitrate: N/A
    Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1366x768, 29.97 fps, 29.97 tbr, 1000k tbn, 
ff error: 29.97 tbc

ff error: [swscaler @ 0x34238a0] deprecated pixel format used, make sure you did set range correctly

ff error: Output #0, mjpeg, to 'pipe:1':
  Metadata:
    encoder         : 
ff error: Lavf57.20.100
    Stream #0:0: Video: mjpeg, yuvj444p(pc), 1366x768, q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc
    Metadata:
      encoder         : Lavc57.20.100 mjpeg
    Side data:
      unknown side data type 10 (24 bytes)
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> mjpeg (native))
Press [q] to stop, [?] for help

ff error: [swscaler @ 0x34238a0] Warning: data is not aligned! This can lead to a speedloss

Data size: 65536
Data size: 65536
Data size: 21413
Data size: 65536
Data size: 65536
Data size: 45581
Data size: 65536
Data size: 65536
Data size: 62377
Data size: 65536
Data size: 65536
Data size: 45581
Data size: 65536
Data size: 65536
Data size: 21413
Data size: 65536
Data size: 60933
Data size: 65536
Data size: 49550
Data size: 65536
Data size: 36709
Data size: 65536
Data size: 27035
Data size: 65536
Data size: 20131
Data size: 65536
Data size: 15887
Data size: 65536
Data size: 15887
Data size: 65536
Data size: 15887
Data size: 65536
Data size: 15911
Data size: 65536
Data size: 15911
Data size: 65536
Data size: 15911
ff error: frame=   16 fps=0.0 q=24.8 size=    1819kB time=00:00:00.53 bitrate=27935.6kbits/s speed=1.04x    
Data size: 65536
Data size: 15911
Data size: 65536
Data size: 15911
Data size: 65536
Data size: 15919
Data size: 65536
Data size: 15919
Data size: 65536
Data size: 15919
Data size: 65536
Data size: 15919
Data size: 65536
Data size: 15919
Data size: 65536
Data size: 15911
Data size: 65536
Data size: 15911
Data size: 65536
Data size: 15906
Data size: 65536
Data size: 15906
Data size: 65536
Data size: 15917
Data size: 65536
Data size: 15999
Data size: 65536
Data size: 15949
Data size: 65536
Data size: 15997
Data size: 65536
Data size: 15965
ff error: frame=   32 fps= 30 q=24.8 size=    3092kB time=00:00:01.06 bitrate=23743.7kbits/s speed=1.01x    
Data size: 65536
Data size: 16025
Data size: 65536
Data size: 15978
Data size: 65536
Data size: 15963
Data size: 65536
Data size: 16028
Data size: 65536
Data size: 15976
Data size: 65536
Data size: 15958
Data size: 65536
Data size: 15940
Data size: 65536
Data size: 15992
Data size: 65536
Data size: 15962
Data size: 65536
Data size: 16010
Data size: 65536
Data size: 15941
Data size: 65536
Data size: 15941
Data size: 65536
Data size: 15973
Data size: 65536
Data size: 15943
Data size: 65536
Data size: 15947
Data size: 65536
Data size: 15947
ff error: frame=   48 fps= 30 q=24.8 size=    4365kB time=00:00:01.60 bitrate=22349.6kbits/s speed=1.01x    
Data size: 65536
Data size: 15982
Data size: 65536
Data size: 15982
Data size: 65536
Data size: 15956
Data size: 65536
Data size: 15956
Data size: 65536
Data size: 15956
Data size: 65536
Data size: 16001
Data size: 65536
Data size: 15930
Data size: 65536
Data size: 15922
Data size: 65536
Data size: 15924
Data size: 65536
Data size: 15924
Data size: 65536
Data size: 15924
Data size: 65536
Data size: 15924
Data size: 65536
Data size: 15911
Data size: 65536
Data size: 15924
Data size: 65536
Data size: 15985
Data size: 65536
Data size: 15985
ff error: frame=   64 fps= 30 q=24.8 size=    5638kB time=00:00:02.13 bitrate=21651.3kbits/s speed=1.01x    
Data size: 65536
Data size: 15985
Data size: 65536
Data size: 15924
Data size: 65536
Data size: 15976
Data size: 65536
Data size: 15976
Data size: 65536
Data size: 15958
Data size: 65536
Data size: 16319
Data size: 65536
Data size: 16558
Data size: 65536
Data size: 16576
Data size: 65536
Data size: 16564
Data size: 65536
Data size: 16582
Data size: 65536
Data size: 16589
Data size: 65536
Data size: 16587
Data size: 65536
Data size: 16446
Data size: 65536
Data size: 16450
Data size: 65536
Data size: 16450
Data size: 65536
Data size: 16450
ff error: frame=   80 fps= 30 q=24.8 size=    6918kB time=00:00:02.66 bitrate=21251.0kbits/s speed=1.01x    
Data size: 65536
Data size: 16568
Data size: 65536
Data size: 16575
Data size: 65536
Data size: 16585
Data size: 65536
Data size: 18182
Data size: 65536
Data size: 17203
Data size: 65536
Data size: 16769
Data size: 65536
Data size: 16734
Data size: 65536
Data size: 16823
Data size: 65536
Data size: 16338
Data size: 65536
Data size: 16455
Data size: 65536
Data size: 16406
Data size: 65536
Data size: 16645
Data size: 65536
Data size: 16800
Data size: 65536
Data size: 16800
Data size: 65536
Data size: 16800
Data size: 65536
Data size: 16806
ff error: frame=   96 fps= 30 q=24.8 size=    8204kB time=00:00:03.20 bitrate=21001.8kbits/s speed=   1x    
Data size: 65536
Data size: 16795
Data size: 65536
Data size: 16804
Data size: 65536
Data size: 16770
Data size: 65536
Data size: 16760
Data size: 65536
Data size: 16813
Data size: 65536
Data size: 16445
Data size: 65536
Data size: 16259
Data size: 65536
Data size: 16260
Data size: 65536
Data size: 16265
Data size: 65536
Data size: 16284
Data size: 65536
Data size: 16233
Data size: 65536
Data size: 16233
Data size: 65536
Data size: 16182
Data size: 65536
Data size: 16058
Data size: 60561
ff error: frame=  111 fps= 30 q=24.8 size=    9384kB time=00:00:03.70 bitrate=20776.1kbits/s speed=   1x    
Data size: 61813
Data size: 61813
Data size: 61813
Data size: 61813
Data size: 61813
Data size: 61781
Data size: 61784
Data size: 61796
Data size: 61842
Data size: 61839
Data size: 61793
Data size: 61810
Data size: 61844
Data size: 61844
Data size: 61850
Data size: 61841
ff error: frame=  127 fps= 30 q=24.8 size=   10350kB time=00:00:04.23 bitrate=20027.8kbits/s speed=   1x    
Data size: 61858
Data size: 61853
Data size: 61833
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
ff error: frame=  142 fps= 30 q=24.8 size=   11256kB time=00:00:04.73 bitrate=19480.5kbits/s speed=   1x    
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
ff error: frame=  158 fps= 30 q=24.8 size=   12223kB time=00:00:05.26 bitrate=19011.4kbits/s speed=   1x    
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61867
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
ff error: frame=  173 fps= 30 q=24.8 size=   13129kB time=00:00:05.76 bitrate=18650.3kbits/s speed=   1x    
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
ff error: frame=  188 fps= 30 q=24.8 size=   14035kB time=00:00:06.26 bitrate=18346.6kbits/s speed=   1x    
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
ff error: frame=  203 fps= 30 q=24.8 size=   14941kB time=00:00:06.76 bitrate=18087.8kbits/s speed=   1x    
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
ff error: frame=  219 fps= 30 q=24.8 size=   15907kB time=00:00:07.30 bitrate=17850.8kbits/s speed=   1x    
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
ff error: frame=  234 fps= 30 q=24.8 size=   16813kB time=00:00:07.80 bitrate=17658.1kbits/s speed=   1x    
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
Data size: 61850
ff error: frame=  250 fps= 30 q=24.8 size=   17780kB time=00:00:08.33 bitrate=17478.0kbits/s speed=   1x    
Data size: 61850
Data size: 61712
Data size: 61712
Data size: 61712
Data size: 61712
Data size: 61712
Data size: 61712
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
ff error: frame=  265 fps= 30 q=24.8 size=   18675kB time=00:00:08.83 bitrate=17319.4kbits/s speed=   1x    
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
ff error: frame=  280 fps= 30 q=24.8 size=   19563kB time=00:00:09.33 bitrate=17171.2kbits/s speed=   1x    
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
ff error: frame=  295 fps= 30 q=24.8 size=   20452kB time=00:00:09.83 bitrate=17038.0kbits/s speed=   1x    
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
Data size: 60638
ff error: frame=  300 fps= 30 q=24.8 Lsize=   20748kB time=00:00:10.00 bitrate=16996.6kbits/s speed=0.998x    
video:20748kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

Stream end

错误消息也很奇怪，但目前还不是最大的问题，所以我们忽略它们。

查看来自 ffmpeg 的所有数据，我假设图像作为单个流而来，没有任何分离。我不认为研究二进制数据并寻找某些标志字节是解决问题和分离图像的好主意。我在帖子顶部提到的命令行会对硬盘驱动器进行单独的写入，这意味着它会为每个屏幕截图生成单独的可写流。对于从 Nodejs 生成的进程，它不会这样做。

我错过了什么？

最佳答案

解决方案是使用 png-streamer并稍微调整 ffmpeg args 。 png-streamer 解析传入的流并通过 PNG 二进制 header 识别 PNG 图像。这是最终的代码:

var ffmpegArgs = [
        '-t', 
        options.duration, 
        '-s', 
        options.width + 'x' + options.height, 
        '-f', 
        'x11grab', 
        '-i', 
        ':' + options.display + '+' + options.offsetX + ',' + options.offsetY, 
        '-vf',
        'fps=' + options.fps,
        '-f',         //<<
        'image2pipe', //<<
        '-vcodec',    //<<
        'png',        //<<
        'pipe:1'
    ], 
    ffmpeg = spawn('ffmpeg', ffmpegArgs);

new pngStreamer(ffmpeg, callback);

除了评论中的有用注释外，还需要使用 -vcodec png 的技巧。否则，缓冲区采用 LAVC 格式。

关于node.js - 如何将缩略图/屏幕截图序列从 ffmpeg 传输到 nodejs 以进行进一步处理？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/34517234/

上一篇：node.js - Node mongoose 解析 request.body 中的 JSON 负载

下一篇：javascript - Express 应用程序具有两个版本的客户端

相关文章：

node.js - 如何为 Azure 上托管的 Web 应用程序指定 node.exe.config？

node.js - 如何在聚焦于另一个程序后打开 Electron 窗口时聚焦于它

Laravel FFMPEG 视频转换无需等待队列

Bash:如何将一个进程的 stdout 和 stderr 传递给两个不同的进程？

python asyncio如何读取StdIn并写入StdOut？

c++ - VCL 形成写入 stdout 的应用程序

javascript - 如何从经纬度点获取城市名称？

node.js - Mongoose 种群忽略选项

batch-file - ffmpeg:使用递归输入输出到与源相同的文件夹？

php - 如何在视频上传时创建缩略图？ [php/codeigniter]

©2024 IT工具网  联系我们