Back to the blog

In p

April 14th, 2021

IRL streaming is difficult. Maintaining a connection fast enough to stream video in variable environments causes many issues. In this post, I cover the ~3 months of work I've done so far to build a stable IRL streaming rig that outperforms many existing solutions. This post is also intended to be a technical document detailing all the research I've done so it will be quite opaque. A more accessible version (probably a YouTube video) is coming soon, but until then please only contact me for support if you have gone through the guide and understand what's going on.

Note for new streamers: You probably don't need this guide. This note is really important and I've put it right at the top intentionally. You don't need to have zero disconnects. Get your phone, boot up Streamlabs, and start streaming. Or grab your GoPro and use the app to stream. Your first priority is streaming, not finding a perfect rig.

Background

This document assumes some familiarity with how IRL streaming works today. Current IRL streaming solutions boil down to the following options, in what I believe descending order of market share.

  1. Streamlabs's mobile app or GoPro's app
  2. LiveU Solo or UnlimitedIRL
  3. Teradek Vidiu Go
  4. Peplink MAX Transit Duo + Encoder
  5. BELABOX + Encoder

There are some other options with alternative cameras, but I will focus on the above five as they work with the current most popular camera set: either the GoPro, the Sony AS300, or a high quality phone camera. The primary issues I find with these solutions are

A mininal backpack setup looks like the following

Perhaps not huge, but we can do better.

The Muxbox: Overview

Let's break down what we need in an IRL streaming rig.

  1. Camera. There are basically three options: your phone, a GoPro, or a Sony AS300. Most streamers seem to use a phone due to ease of use or the Sony AS300 due to GoPro's terrible software. But the GoPro has a big benefit over the other two that I personally care a lot about: exceptional stabilization. Being a bike streamer, the shakiness of the Sony AS300 is difficult to watch and even the stabilization of the Pixel 5 is not great. Of course, this depends on use case.
  2. Encoder. This converts the camera video feed into a H.264/265 stream to be sent over the network.
  3. Cell bonder. One way to improve stability is to bond multiple cell connections together. When one connection goes down, another can take over if it's available. Alternatively, if two connections are operating at 3 Mbps, we can bond them together to achieve 6 Mbps.

In the Muxbox, I use a GoPro, which supports RTMP streaming. Notably, however, it's not very good. When the GoPro runs into any sort of network problem it disconnects the stream. This is particularly annoying, but when I ran off of this setup, I measured disconnects between 1-2 per hour, depending on where I was located. This is actually surprisingly good, which is why I advocate that new streamers use this setup and not worry about a backpack. It's quite hard to do better than 1-2 disconnects per hour.

To stabilize the network connection, I stream the Muxbox to a wifi access point broadcasted by a Jetson Nano. This Jetson Nano ingests the stream immediately, so the GoPro never sees any further downstream network problems.

On the Jetson Nano, I transmux the RTMP stream from the GoPro to an SRT stream. SRT provides more reliability than RTMP. However, in practice I've found that RTMP is not actually that much worse, but SRT recovers a bit faster from poor network connections. My understanding is that the LiveU actually uses RTMP successfully, which corroborates that RTMP is not actually as bad as many IRL streamers believe.

The Jetson Nano runs some custom bonding code and I use three Motorola Moto E's as modems, one per network (T-Mobile, Verizon, AT&T). The SRT stream is sent through these bonded connections and received on a remote server, which reassembles the stream and pipes it to OBS. OBS adds the overlays and then sends the completed stream to Twitch.

Here's what the Muxbox looks like when all set up in a messenger bag. Plenty of room for activities!

Pros and Cons

The biggest benefit to this setup is fewer components. The Jetson Nano is reasonably small, but more significantly I do not require a fragile HDMI cable. Each component is individually powered (even the modems). The only single point of failure in my setup is the power to the Jetson Nano. In fact, having the GoPro wirelessly transmit to the Jetson Nano leads to unique streaming opportunities, such as being able to throw the GoPro or passing it between streamers, as well as allowing multiple cameras trivially. MooseDoesStuff is a streamer that exemplifies these benefits, as he's not able to be tethered to his computer or carry a large backpack while working. An unintentional benefit of wireless streaming is also that the setup is quite water resistant. I'm generally not concerned at all streaming in the rain.

Second, I leverage the GoPro's encoder, so a separate encoder isn't necessary. Since the encoder is often expensive and placed in the backpack, this ends up being a decent cost savings and dramatically reduces the amount of heat generated. The GoPro is typically in open air, so it's much easier to bleed off heat.

Third, the setup uses very little battery. I've found that the GoPro's encoder is notoriously efficient. I haven't run a full test, but a single 20000 mAh battery should be enough to power a 12 hour stream. The longest stream I've run is about six hours with half the battery used. A MAXOAK battery would be equivalent to streaming for a full 36 hours. This also translates to weight savings. The Muxbox typically weighs 2-3 lbs.

Lastly, I use mostly easy-to-find parts. Even finding a Sony AS300 is getting difficult these days, let alone trying to actually acquire a LiveU Solo. The Jetson Nano is probably the hardest component to find, but other boards can also work.

These benefits are not without some cons, however. The biggest con is a fixed bitrate. The GoPro streams at ~4500 kbps, which is much lower than what the LiveU outputs. I've chosen to accept this because I believe that value can both be configured with some firmware hacking and that GoPro will one day change the cap. However, subjectively it seems that most viewers still deem ~4500 kbps sufficiently high quality.

Second, the community around IRL streaming seems to have discounted wireless streaming and the GoPro entirely. I hope to change the perception, but currently there is not much community support around such efforts.

Components

So this is the good stuff. Here's the full hardware bill of materials. In total, it comes to ~$820 for a setup with three modems.

Maybe get a Jetson Nano case too. There are a bunch off Amazon. I use this one, but they all suck. Another reason I don't like the Jetson Nano.

Assembly

  1. Install the Intel AC8265 into the Jetson Nano and attach the antennas. Use this guide if you opted to buy a version with no instructions.
  2. Build your hacky power delivery cable to go from USB-C to DC barrel jack.
  3. Optional: Follow this guide to compile a custom kernel module to enable TTL mangling. This gets around some providers' hotspot throttling. This can also be set in /etc/sysctl.conf however it didn't seem to work for me.
  4. Configure your Android phones to always tether. Under Developer Options, change Default USB configuration to USB tethering. Maybe turn on Do not Disturb permanently. I also change the background color to the providers' colors so I know which SIM is in which phone at a glance.
  5. Optional: Install the GoPro labs firmware so you can get QR code support. It makes starting the stream a lot easier.
  6. Boot up the Jetson Nano and install Docker.
  7. On the Jetson Nano, run sudo docker run --detach --restart always --port 1935:1935 --add-host host.docker.internal:host-gateway --env SRT_RELAY_SERVER=host.docker.internal:1985 muxfd/muxbox-nginx-rtmp-srt-transmuxer.
  8. On the Jetson Nano, run sudo docker run --detach --restart always --network host muxfd/multipath-udp-sender -i 0.0.0.0:1985 <your server ip>:1985.
  9. Set up a wifi access point on the Jetson Nano.
    1. Click the wifi networks icon in the menu bar
    2. Click Edit Connections...
    3. Add a new connection
    4. Set it to AP/Hotspot mode
    5. Set your SSID and password
    6. Restart your Jetson Nano
  10. On your VPS, install Docker.
  11. On your VPS, run sudo docker run --detach --restart always --port 1985:1985/udp --add-host host.docker-internal:host-gateway muxfd/multipath-udp-receiver -i 0.0.0.0:1985 host.docker.internal:1935.
  12. On your VPS, run sudo docker run --detach --restart always --port 1935:1935/udp muxfd/srt-relay.
  13. That's it! Try connecting your GoPro to the Jetson Nano's AP and stream to rtmp://10.42.0.1:1935/live/mooo. In OBS, you should be able to receive this as srt://<your server ip>:1935?streamid=play/mooo.

Development History

In the process of developing the above setup, I've gone through a number of iterations. I use the metric of disconnects/hr which represents the number of times I need to stop and touch/reset the setup while streaming under typical conditions. A packet loss of greater than two seconds is also considered a disconnect, even if it's able to autorecover.

First, I started with RTMP streaming through a single-SIM hotspot. This results in ~1-2 disconnects/hr. Again, remarkably difficult to do better than, so I recommend all streamers start with this setup.

Next, I attempted to use a Raspberry Pi loaded with OpenMPTCPRouter. The issue with this approach is that OpenMPTCPRouter is not aware that the payload is video, so it does not actively discard packets when they are too late. This results in flooding the available bandwidth when bandwidth is recovered, extending downtime. This could potentially be worked around by cleverly setting the TTL on packets, however this would expose the GoPro to downstream network fluctuations as well. This setup results in ~6 disconnects/hr.

To attempt to proxy the connection so the GoPro doesn't see the network fluctuations, I attempted to use a Raspberry Pi to run nginx with a USB modem, the ZTE MF833V. This results in ~4 disconnects/hr.

To improve reliability, I had the GoPro transmux the stream to SRT before sending it over the network. This results in ~3.5 disconnects/hr.

I then learned that the GoPro doesn't really like the Raspberry Pi's wifi adapter. I tried various boards until landing on the Jetson Nano + Intel AC8265 with a single USB modem. This results in ~2 disconnects/hr.

To bond multiple modems together, I then used srtla for aggregation. This results in ~1.5 disconnects/hr.

I hypothesized that the USB modems were not strong enough to maintain connection with the cell towers, so I switched to using a single Moto E. This results in ~1 disconnect/hr. With srtla and multiple modems, I am not able to see an improvement. I hypothesize that this is due to srtla relying on congestion control, whereas in my case it's better to push packets aggresively, as the bitrate cannot be adjusted upstream.

In order to improve further, I wrote my custom bonding code to bond the modems together more aggressively to accommodate constant bitrates. This achieves an estimated ~0.3 disconnects/hr, which is the current state.

Further Research

As you can see, the road to zero disconnects/hr is a long one. I had anticipated I'd accomplish this in a month, however three months later, I'm still working on improving. Turns out, actually getting to zero is substantially more difficult than I thought, and getting to an acceptable rate of disconnects is much easier than I thought.

I'm proud to say though, based on watching other streamers' LiveU setups, they achieve ~0.75 disconnects/hr so this setup is capable of being more stable than a LiveU with equivalent bonded connections. I'm continuing to improve the stability of the bonding code and hope to further lower the disconnect rate.

Furthermore, I'm currently investigating how to get rid of the Jetson Nano. Its form factor is rather annoying due to being totally overkill for what I'm asking it to do and the power delivery trigger can be a bit fragile. However, many other boards can't seem to maintain the wifi connection to the GoPro. I'm not sure why the Intel AC8265 specifically can maintain that connection. Even more peculiarly, I've run the Intel AC8265 in a Latte Panda with no success.

Additionally, I would also like to find a smaller form factor modem. The Motorola Moto E's have been great because they are slim, but they are comparably heavier than ZTE MF833V's which I've used as well due to carrying around a cell phone battery. However, I have not found a lighter modem that can maintain connection as well as a cell phone.

Lastly, more long term, I'd like to package the hardware and software into a much smaller form factor. There's no reason the entire setup needs to be larger than the size of a cell phone. I envision a product that allows people to buy a single device to stream off of, including multiple modems and a built in transmuxer. This could help reduce costs and further improve waterproofing.

Acknowledgments and Retrospective

Thanks to everyone who has tuned into my streams. I basically use them as a testing ground and all my viewers are indirectly guinea pigs subjected to frequent F's and LSD. I guess my content is interesting enough that it's worth sitting through, but thanks everyone for helping me get this far.

Additionally, I want to thank rational_sail specifically with his BELABOX project of which I used srtla as a basis for my own bonding code. He's done a great deal of work and BELABOX is a good alternative if you're looking for a solution that more closely mirrors the LiveU.

Lastly, one of my viewers, gregwire, has been particularly helpful in telling me about modem tech and antennas, which I formed a few more hypotheses around why the GoPro might be disconnecting and why my modems weren't strong enough.

Looking back, it's clear my lack of hardware knowledge was the biggest impediment. I likely could have iterated much quicker investing in more expensive modem tech first and attempting to lower costs afterwards. I ended up losing a good amount of time to trying out cheaper boards that didn't make much progress whereas I should have started with a more expensive platform of stronger modems first. I got very lucky in trying the Jetson Nano not too late and I should've done that with my modems as well. Additionally, I continue to have a lack of insight into why the GoPro doesn't seem to like other boards, but unfortunately lack the knowledge to debug this further. This remains an open point that I'm not sure I'll be able to resolve.

If you're interested in helping with development and experimentation, I'd love to hear from you. Join my discord!


Comment on this post on Twitter and follow me on Twitch!