At Muxable, we use FFmpeg to transcode WebRTC streams with our transcoder. The transcoder receives an RTP stream over cell networks with Pion and also uses Pion to write the transcoded RTP stream to the client. However, piping an RTP stream in memory to FFmpeg is a bit undocumented. I asked around and haven't been able to find a documented answer leading me to think that it wasn't actually possible with the current FFmpeg APIs. Last week I got an email from YCChiang suggesting that it might be possible! In the interest of documenting this path better, this post describes how it's done.
The first step is to create an SDP. In
libavformat, an SDP
input triggers an RTP source. You can create a minimal SDP that looks
v=0 o=- 0 0 IN IP4 127.0.0.1 s=No Name c=IN IP4 127.0.0.1 t=0 0 a=tool:libavformat 58.29.100 m=video 5000 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
Inputting this file with ffplay
ffplay -protocol_whitelist file,rtp,udp -i video.sdp will
listen on port 5000 for RTP packets.
Therefore, the first step is to generate a temporary SDP file. Our code to do that can be found here. Since we will be piping the RTP packets over memory, the choice of port and IPs doesn't matter. However, the payload type must match.
The SDP file that we generate is passed to
avformat_open_input() and we follow the typical flow to
AVIOContext. Crucially, we must also set the flag
RTSP_FLAG_CUSTOM_IO to indicate that the RTSP receiver will
also use the custom IO provided by
AVInputFormat *file_iformat = av_find_input_format("sdp"); AVFormatContext *ic = avformat_alloc_context(); AVDictionary *format_opts = NULL; av_dict_set(&format_opts, "sdp_flags", "custom_io", 0); avformat_open_input(&ic, "video.sdp", file_iformat, &format_opts); uint8_t *readbuf = (uint8_t *)av_malloc(4096); AVIOContext * avio_in = avio_alloc_context(readbuf, 4096, 1, video_queue, &read_packet, &write_packet, NULL); ic->pb = avio_in;
In the above example, we configure
libavformat to use a
custom i/o stream and then also set the
RTSP_FLAG_CUSTOM_IO flag to indicate that we should use the
custom i/o. It's unclear to me why this is not the default, but the
av_dict_set(&format_opts, "sdp_flags", "custom_io", 0); line
Interestingly, there is also a bug in the FFmpeg code right now so a
write_packet function must be passed in. See
this ticket for more