IRL streaming is difficult. Maintaining a connection fast enough to stream video in variable environments causes many issues. In this post, I cover the ~3 months of work I've done so far to build a stable IRL streaming rig that outperforms many existing solutions. This post is also intended to be a technical document detailing all the research I've done so it will be quite opaque. A more accessible version (probably a YouTube video) is coming soon, but until then please only contact me for support if you have gone through the guide and understand what's going on.
Note for new streamers: You probably don't need this guide. This note is really important and I've put it right at the top intentionally. You don't need to have zero disconnects. Get your phone, boot up Streamlabs, and start streaming. Or grab your GoPro and use the app to stream. Your first priority is streaming, not finding a perfect rig.
This document assumes some familiarity with how IRL streaming works today. Current IRL streaming solutions boil down to the following options, in what I believe descending order of market share.
There are some other options with alternative cameras, but I will focus on the above five as they work with the current most popular camera set: either the GoPro, the Sony AS300, or a high quality phone camera. The primary issues I find with these solutions are
A mininal backpack setup looks like the following
Perhaps not huge, but we can do better.
Let's break down what we need in an IRL streaming rig.
In the Muxbox, I use a GoPro, which supports RTMP streaming. Notably, however, it's not very good. When the GoPro runs into any sort of network problem it disconnects the stream. This is particularly annoying, but when I ran off of this setup, I measured disconnects between 1-2 per hour, depending on where I was located. This is actually surprisingly good, which is why I advocate that new streamers use this setup and not worry about a backpack. It's quite hard to do better than 1-2 disconnects per hour.
To stabilize the network connection, I stream the Muxbox to a wifi access point broadcasted by a Jetson Nano. This Jetson Nano ingests the stream immediately, so the GoPro never sees any further downstream network problems.
On the Jetson Nano, I transmux the RTMP stream from the GoPro to an SRT stream. SRT provides more reliability than RTMP. However, in practice I've found that RTMP is not actually that much worse, but SRT recovers a bit faster from poor network connections. My understanding is that the LiveU actually uses RTMP successfully, which corroborates that RTMP is not actually as bad as many IRL streamers believe.
The Jetson Nano runs some custom bonding code and I use three Motorola Moto E's as modems, one per network (T-Mobile, Verizon, AT&T). The SRT stream is sent through these bonded connections and received on a remote server, which reassembles the stream and pipes it to OBS. OBS adds the overlays and then sends the completed stream to Twitch.
Here's what the Muxbox looks like when all set up in a messenger bag. Plenty of room for activities!
The biggest benefit to this setup is fewer components. The Jetson Nano is reasonably small, but more significantly I do not require a fragile HDMI cable. Each component is individually powered (even the modems). The only single point of failure in my setup is the power to the Jetson Nano. In fact, having the GoPro wirelessly transmit to the Jetson Nano leads to unique streaming opportunities, such as being able to throw the GoPro or passing it between streamers, as well as allowing multiple cameras trivially. MooseDoesStuff is a streamer that exemplifies these benefits, as he's not able to be tethered to his computer or carry a large backpack while working. An unintentional benefit of wireless streaming is also that the setup is quite water resistant. I'm generally not concerned at all streaming in the rain.
Second, I leverage the GoPro's encoder, so a separate encoder isn't necessary. Since the encoder is often expensive and placed in the backpack, this ends up being a decent cost savings and dramatically reduces the amount of heat generated. The GoPro is typically in open air, so it's much easier to bleed off heat.
Third, the setup uses very little battery. I've found that the GoPro's encoder is notoriously efficient. I haven't run a full test, but a single 20000 mAh battery should be enough to power a 12 hour stream. The longest stream I've run is about six hours with half the battery used. A MAXOAK battery would be equivalent to streaming for a full 36 hours. This also translates to weight savings. The Muxbox typically weighs 2-3 lbs.
Lastly, I use mostly easy-to-find parts. Even finding a Sony AS300 is getting difficult these days, let alone trying to actually acquire a LiveU Solo. The Jetson Nano is probably the hardest component to find, but other boards can also work.
These benefits are not without some cons, however. The biggest con is a fixed bitrate. The GoPro streams at ~4500 kbps, which is much lower than what the LiveU outputs. I've chosen to accept this because I believe that value can both be configured with some firmware hacking and that GoPro will one day change the cap. However, subjectively it seems that most viewers still deem ~4500 kbps sufficiently high quality.
Second, the community around IRL streaming seems to have discounted wireless streaming and the GoPro entirely. I hope to change the perception, but currently there is not much community support around such efforts.
So this is the good stuff. Here's the full hardware bill of materials. In total, it comes to ~$820 for a setup with three modems.
Maybe get a Jetson Nano case too. There are a bunch off Amazon. I use this one, but they all suck. Another reason I don't like the Jetson Nano.
/etc/sysctl.conf however it didn't seem to work for me.
sudo docker run --detach --restart always --port 1935:1935 --add-host
host.docker.internal:host-gateway --env
SRT_RELAY_SERVER=host.docker.internal:1985
muxfd/muxbox-nginx-rtmp-srt-transmuxer.
sudo docker run --detach --restart always --network host
muxfd/multipath-udp-sender -i 0.0.0.0:1985 <your server
ip>:1985.
Edit Connections...AP/Hotspot modesudo docker run --detach --restart always --port 1985:1985/udp
--add-host host.docker-internal:host-gateway
muxfd/multipath-udp-receiver -i 0.0.0.0:1985
host.docker.internal:1935.
sudo docker run --detach --restart always --port 1935:1935/udp
muxfd/srt-relay.
rtmp://10.42.0.1:1935/live/mooo. In OBS, you should be
able to receive this as
srt://<your server ip>:1935?streamid=play/mooo.
In the process of developing the above setup, I've gone through a number of iterations. I use the metric of disconnects/hr which represents the number of times I need to stop and touch/reset the setup while streaming under typical conditions. A packet loss of greater than two seconds is also considered a disconnect, even if it's able to autorecover.
First, I started with RTMP streaming through a single-SIM hotspot. This results in ~1-2 disconnects/hr. Again, remarkably difficult to do better than, so I recommend all streamers start with this setup.
Next, I attempted to use a Raspberry Pi loaded with OpenMPTCPRouter. The issue with this approach is that OpenMPTCPRouter is not aware that the payload is video, so it does not actively discard packets when they are too late. This results in flooding the available bandwidth when bandwidth is recovered, extending downtime. This could potentially be worked around by cleverly setting the TTL on packets, however this would expose the GoPro to downstream network fluctuations as well. This setup results in ~6 disconnects/hr.
To attempt to proxy the connection so the GoPro doesn't see the network fluctuations, I attempted to use a Raspberry Pi to run nginx with a USB modem, the ZTE MF833V. This results in ~4 disconnects/hr.
To improve reliability, I had the GoPro transmux the stream to SRT before sending it over the network. This results in ~3.5 disconnects/hr.
I then learned that the GoPro doesn't really like the Raspberry Pi's wifi adapter. I tried various boards until landing on the Jetson Nano + Intel AC8265 with a single USB modem. This results in ~2 disconnects/hr.
To bond multiple modems together, I then used srtla for aggregation. This results in ~1.5 disconnects/hr.
I hypothesized that the USB modems were not strong enough to maintain connection with the cell towers, so I switched to using a single Moto E. This results in ~1 disconnect/hr. With srtla and multiple modems, I am not able to see an improvement. I hypothesize that this is due to srtla relying on congestion control, whereas in my case it's better to push packets aggresively, as the bitrate cannot be adjusted upstream.
In order to improve further, I wrote my custom bonding code to bond the modems together more aggressively to accommodate constant bitrates. This achieves an estimated ~0.3 disconnects/hr, which is the current state.
As you can see, the road to zero disconnects/hr is a long one. I had anticipated I'd accomplish this in a month, however three months later, I'm still working on improving. Turns out, actually getting to zero is substantially more difficult than I thought, and getting to an acceptable rate of disconnects is much easier than I thought.
I'm proud to say though, based on watching other streamers' LiveU setups, they achieve ~0.75 disconnects/hr so this setup is capable of being more stable than a LiveU with equivalent bonded connections. I'm continuing to improve the stability of the bonding code and hope to further lower the disconnect rate.
Furthermore, I'm currently investigating how to get rid of the Jetson Nano. Its form factor is rather annoying due to being totally overkill for what I'm asking it to do and the power delivery trigger can be a bit fragile. However, many other boards can't seem to maintain the wifi connection to the GoPro. I'm not sure why the Intel AC8265 specifically can maintain that connection. Even more peculiarly, I've run the Intel AC8265 in a Latte Panda with no success.
Additionally, I would also like to find a smaller form factor modem. The Motorola Moto E's have been great because they are slim, but they are comparably heavier than ZTE MF833V's which I've used as well due to carrying around a cell phone battery. However, I have not found a lighter modem that can maintain connection as well as a cell phone.
Lastly, more long term, I'd like to package the hardware and software into a much smaller form factor. There's no reason the entire setup needs to be larger than the size of a cell phone. I envision a product that allows people to buy a single device to stream off of, including multiple modems and a built in transmuxer. This could help reduce costs and further improve waterproofing.
Thanks to everyone who has tuned into my streams. I basically use them as a testing ground and all my viewers are indirectly guinea pigs subjected to frequent F's and LSD. I guess my content is interesting enough that it's worth sitting through, but thanks everyone for helping me get this far.
Additionally, I want to thank rational_sail specifically with his BELABOX project of which I used srtla as a basis for my own bonding code. He's done a great deal of work and BELABOX is a good alternative if you're looking for a solution that more closely mirrors the LiveU. If you're interested in a complete, no tech solution, ours can be found at Muxable. Your orders contribute to my research and development around IRL technology.
Lastly, one of my viewers, gregwire, has been particularly helpful in telling me about modem tech and antennas, which I formed a few more hypotheses around why the GoPro might be disconnecting and why my modems weren't strong enough.
Looking back, it's clear my lack of hardware knowledge was the biggest impediment. I likely could have iterated much quicker investing in more expensive modem tech first and attempting to lower costs afterwards. I ended up losing a good amount of time to trying out cheaper boards that didn't make much progress whereas I should have started with a more expensive platform of stronger modems first. I got very lucky in trying the Jetson Nano not too late and I should've done that with my modems as well. Additionally, I continue to have a lack of insight into why the GoPro doesn't seem to like other boards, but unfortunately lack the knowledge to debug this further. This remains an open point that I'm not sure I'll be able to resolve.
If you're interested in helping with development and experimentation, I'd love to hear from you. Join my discord!