If you click out of a YouTube video when it fails to buffer, you’re not alone. With recent research suggesting that nearly 70 percent of web content is streaming video and audio, whether a stream has to buffer or not is more important than ever. Luckily, researchers at MIT are breaking new ground with AI-powered streaming algorithms that could spell the death of the buffer wheel and bring us boldly into the future of video streaming.
Video streaming has exploded over the past five years. Streaming giants like Netflix, Amazon, Google, and Hulu are pouring resources into their original and licensed streaming content in an attempt to win audiences over from traditional TV networks. And the bet is working; consumers continue to leave their high-priced cable contracts to go with streaming options.
With a potentially massive chunk of the entertainment industry on the table, these services are investing in solving what researchers are calling “the video problem.” Streaming is heavily taxing on internet bandwidth, and audiences have increasingly lower tolerances for annoying pixelation, buffering, stalled videos, and long load times. If a video stalls out, audiences are more likely than ever to simply click away.
Why your streaming video stalls
Video is one of the most data-intensive internet uses, and loading videos for every internet user would vastly exceed available bandwidth if each video was loaded in full. Several years ago, streaming services adopted a clever algorithm technology called adaptive bitrate (ABR). ABR algorithms selectively load videos according to network conditions; that’s why the load bar on your YouTube video never moves too far ahead of where you’re currently viewing.
There are two primary kinds of ABR algorithms that keep your Stranger Things marathon going strong. Rate-based algorithms measure the connection speed and change video loading quality accordingly. Buffer-based ABR algorithms keep a certain percentage of unwatched video loaded ahead.
Together, these ABR algorithms have mostly kept pace with video streaming traffic, but with demand now topping 1 billion streaming hours per day on YouTube alone, the next generation of streaming algorithms are long overdue.
AI-powered streaming services
Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are answering the global call for better streaming quality with a new algorithm that leverages artificial intelligence to improve load rates and reduce buffering. The technology, called Pensieve, could revolutionize the streaming entertainment market.
Everyone’s network conditions are slightly different and constantly changing. Your internet speed is affected by the activity of everyone in your neighborhood, creating a chaotic environment for predictive algorithms like ABR. Pensieve works smarter rather than harder, using machine learning to react to changing conditions in real time.
Pensieve works based on a system of “rewards” and “penalties.” By registering a reward when videos load smoothly and a penalty when streaming interrupts, Pensieve can mimic a neural network and learn how to load your videos better. The system improves video streaming by 10 percent to 30 percent, and viewers rated its results 10 percent to 25 percent higher in overall quality over current algorithms’.
Traditional algorithms like ABR have relied on expert knowledge from human beings to function. MIT’s research team is confident that Pensieve can do the job on its own, and put the tool through robot boot camp to prove it. The team sent its AI-powered algorithm manager through a gauntlet of challenging real-world scenarios and purposefully exposed it to previously unknown network conditions to test its adaptive capabilities.
Pensieve pulled through every scenario, maintaining the same video resolution as the best traditional streaming algorithms but with less rebuffering. “This sort of stress test shows that it can generalize well for new scenarios out in the real world,” said Hongzi Mao, author of a related paper and member of the MIT team. Pensieve is also a first effort; as deep-learning algorithms progress, improvements will only increase over time.
Netflix digs deep into deep learning
MIT’s algorithm is a game changer, but they’re not alone in the battle against buffering. In early 2017, Netflix rolled out its own AI-powered algorithm. The new system, called Dynamic Optimizer, analyzes video frame-by-frame in real time and selectively compresses each scene for higher image quality on slow connections. “We’re allergic to rebuffering,” said Netflix vice president of product innovation Todd Yellin.
The new algorithm is smart enough to differentiate types of video content. Action-packed sequences from the latest superhero flicks get an increase in bitrate, while simpler animated content is eased back. The result is a steady stream for all users, especially those on slow connections. Bingers, rejoice!
Artificial intelligence is also revolutionizing the way streaming video reaches audiences. Not all Parks and Recreation viewers love The Office, and Netflix’s newest AI algorithms aim to find out why. Netflix is developing algorithms for its home pages and “suggested titles” that leverage machine learning to get to know your preferences like never before. Through the use of deep learning and simulated neural networks, Netflix’s algorithms can pick up on new patterns, narrowing in on each user’s preferences and extending the average binge session.
The future of streaming
AI-powered algorithms are opening new doors for streaming video’s next generation of content. Virtual reality headsets and 4K content are on the horizon of mainstream entertainment, but the current streaming algorithms simply can’t keep pace with these technologies’ much higher bitrate requirements.
With AI-driven streaming management, efficiencies across the entire web will increase. Streaming is taking over the entertainment industry, and AI algorithms like MIT’s Pensieve are aiding the takeover in a big way. As streaming video accounts for a larger and larger portion of internet traffic, artificial intelligence is coming along just in time to help video streaming continue to grow.
Cassie Tolhurst is a freelance tech journalist who writes for Geek Girl Con and College Puzzle.