This article is part of the Technology Insight series, made possible with funding from Intel.
Last summer, as athletes broke world records at the Tokyo 2020 Olympic Games, another “one for the books” was quietly taking place. For 19 days, live coverage of many Olympic events was broadcast over the public Internet at 8K at 60 frames per second (Fps) with HDR (High Dynamic Range) to Brazil and Japan. (You didn’t miss it; U.S. viewing was limited to private, invitation-only Intel sites in Portland, Santa Clara, and Hollywood audiences in Los Angeles.). It was believed to be the first live, broadcast-quality transmission on an open IP network cloud, coming just months after a pioneering 8K/60FPSFps livestream (without HDR), also led by Intel, in December 2020.
Between those two landmark events, Intel and its partners conducted more than a dozen successful demonstrations of its prototype for Over The Top (OTT), standards-based delivery of live broadcast-quality, 8K (7680×4320), 60Fps, HDR content on existing 4K infrastructure. “We are way beyond proof of concept,” says Ravindra “Ravi” Velhal, global content technology strategist and 8K lead at Intel. The company, he says, is working full speed ahead to create a new category and market that will bring live, end-to-end 8K Internet streaming to billions across the globe.
Lifelike, “you are there” immersion
Intel believes that content consumers will jump at the chance to experience dazzling, immersive, “you are there” sports and entertainment events, including gaming, enhanced by 3D, VR, AR, with life-like 8K UHD and sound – all at stadium scale. Viewers will be able to see every blade of grass on a playing field or get close to favorite performers glistening under spotlights on giant screens. With the support of Olympic Broadcast Services (OBS), Brazil’s GloboTV, a South American media conglomerate, and NHK Technologies, the Japanese 8K pioneer, Intel’s plans to use its wide-ranging innovations to enable broadcasters eager to offer new, scalable services appear to be well, streaming along.
Guiding these efforts is Velhal, a 23-year Intel veteran who’s decidedly not a typical company engineer. A pioneer of cinema VR and expert in new media formats, he’s done high-profile projects with Warner Brothers, Universal Pictures, Paramount, and Sony Pictures. With the latter, Velhal collaborated to launch an innovation center and “Spiderman: Far From Home”, the world’s first multiplayer VRE gaming experience over 5G during MWC 2018. He’s twice won Hollywood’s prestigious Lumiere award, his two projects were nominated for an Emmy, holds several patents, chairs the 8K Association and advisor to film industry forums worldwide.
Here’s an edited, condensed version of his VentureBeat interview. Quotes are omitted for readability.
A new category and ecosystem
For the Olympics, Intel is not a traditional broadcaster, known as a “rights-holder”. There’s only per country. In the U.S., its NBC Universal. In Japan, it’s NHK. Intel is an exclusive world Olympic partner in 360 content development platforms, VR, 5G technology platforms, AI platforms, sports performance platforms, drones and of course processors. Under special permission of Olympic Broadcast Services, we licensed the video signal to do a limited prototype of 8K/60FPS/ HDR live streaming through the open IP network to selected sites. Intel is a leader enabling these technologies that help the industry enhance broadcasts and immersion, for sports, for entertainment, for anything. The whole effort here is to advance the category and ecosystem creation.
The first time the Olympics was televised in 8K was London in 2012 by NHK, which has been a pioneer of 8K since 1995. But NHK has been doing it using their fixed-function solution, which is not scalable. They’re doing strictly on broadcast; their signal is sent to the satellite, where a dish antenna grabs it, and you can watch it in Japan. (Japan is the only country that 8K TV channels right now.) It’s a specific signal; NHK cannot broadcast over the top, over the AWS or Google Cloud, for instance.
What we’re doing with OBS/NHK is to show that we can take the 8K signal and scale it to a larger area, beyond one city or country, over an open Internet cloud. That’s the big difference. We did this at the 2018 FIFA World Cup. But that was a dedicated link; you take the signal and send it to just one site, so it was not scalable. The technical feasibility we’re showing now is using agnostic cloud service provider, so that you can have millions of clients consuming content in 8K globally.
How it Works
From content capture to playback, streaming low-latency, high-quality 8K/60FPS/HDR over the public internet requires an incredible amount of processing power, highly efficient video coding, and reliable streaming technologies. Intel is uniquely positioned to scale this. Many servers on the cloud are Intel Xeon hosted. We have the interconnectivity, the 5G technologies, and the Internet interconnect technologies at the edge, including Ethernet, Fiber, and radio access networks and cameras coming into one place.
The major innovation here is Scalable Xeon based local encoding and delivery over the top. How do you get that high quality onto the Internet? You have to compress it, you have to encode it, you have to use AI to estimate how those frames should be compressed and encoded. You have to do that at the edge, in the broadcast studio. Then you use the cloud as a distribution vehicle.
Content is captured at 8K 60Fps HDR in big, fat 48 gigabit per second optical lines. That’s what we feed to our encoder server. The SDR server is a basically a file server where we host recorded files. It’s also a repeat to the 8K encoder server output of the 8K encoder server. It can be either 80-100 Mbps, which is what we call the distribution signal, or it could be 250 Mbps which is called a contribution signal. That’s of very high quality, so that the broadcaster can do anything with it. But right now, the web service provider cannot handle distribution of more than 100 Mbps. Basically, we’re delivering 8K on the existing 4k infrastructure.
Encoding happens on Intel Xeon platinum. If you want to process 50 gigabits per second data and compress it to 80-100 mbps, you have to use 112 core based Xeon Server. At the Tokyo Olympics, we used encoded servers equipped with four Xeon 8380H processors. Each had a total of 28 cores and 56 threads, a total of 112 cores and 224 threads and encoder was optimized to use AVX-512 instruction support. We also used an Optane 900P SSD with a capacity of 480 gigabytes and DDR4-3200 memory with a capacity of 384 gigabytes. During our advanced trials post Olympics, we were able to achieve similar results with 2 x 38 core Ice Lake Xeon Scalable Platforms.
Delivery to the open Internet cloud and from there either HLS or the RTP. We solve the bottlenecks of the 8K TV playback using single cable HDMI 2.1. The only non-Intel part we use is Nvidia graphics card, which handles color correction and outputting this to the HDMI 2.1 compliant TV. All the rest is done by CPU, both from the encoding side and the decoding side.
“More pixels, faster pixels, better pixels” for stunning visuals
Three things have to come together to give you that full, immersive 8K experience: More pixels, better pixels, and faster pixels. That’s what Intel is driving.
More pixels is easy. 8K is four times more resolution than your 4K and 16 times more resolution than HD transmission.
What we did for the Tokyo Olympics — 60 frames per second — is a faster pixel. If you watch Hollywood movies, their frame rate is 24FPS— and majority of content is it 30FPS. The more frames per second you have, the more fluid your content. For sports, you need a minimum of 60; 120 looks better for slow motion. If you watch a football match in 30 frames per second, you see a frame drop. On 4K or HDTV, you will not see the difference. When you have more resolution, like 8K, you need a higher frame rate that goes along with it. So when you watch, say, a 100-meter sprint, it will look really phenomenal.
And better pixel is the color gamut or dynamic range. High dynamic range means your eyes can see the maximum number of colors that your TV can produce. That’s because the bit depth of pixel is more than eight bit or 10 bit color space. With high dynamic range, your screen and the LED will really pop out in the best color possible. That’s what raises the level of emotion.
The other element is immersive audio. We can do 24 speaker audio in a room, but are only doing 5.1 or stereo right now for the broadcasting. Our encoder processes 22.2 Channels and we have done that in the past. Industry needs to standardize audio format for broadcasting and support various audio profiles from 22.2, 7.2.4, 5.1, stereo as well as Dolby Atmos to achieve full audio immersion.
Quality of service, thanks to low CPU Latency
Right now, from a 50 Gbps gigabit input signal encoding to produce 200-250 Mbps contribution signal and 80 mbps distribution signal for delivery over OTT pipeline, our CPU latency is 200-400 milliseconds. That’s a world record in itself, though we yet to get an Emmy Award for it.
We divide the 8K screen into multiple horizontal bands, each with a dedicated Intel Xeon core processor. That’s how we do a lot of metrics calculation, add, multiply, add, because there’s huge amount of vector data or scalar data. Quality of service is important for broadcasting industry standards, because we are doing a lot of this parallelism here. That’s how we’re able to achieve 200-400 millisecond latency from input to output.
We’re using typical TCP types of hardware error correction – all the things that just work with what the Internet has figured out how to prioritize video. And because it’s a standard internet type streaming protocols, that all gets managed from the far edge all the way to the client to make sure that you have the best experience. It has a lot of the standard retry stuff, because it uses RTP and TLS, RTP and HLS, if it doesn’t get the packet it needs it can recover.
The future is 8K
Look at history, Intel has been at the forefront of media formats and delivery since Standard DVD definition started around 1996 with the collaboration between the tech companies and the service companies. Around 2003, FHD, which is Blu-ray, arrived. The transition was seven years. The transition to UltraHD Blue Ray in 4K quality was also about seven years. So right now, we are at the beginning of another seven-year cycle for a new TV 8K format. What I call the “format momentum” of the last 30 years is what you’re seeing now.
Plus, the TV industry follows the spirit of Moore’s law. Every few years, speed doubles, then quadruples at roughly the same price. Right now, you can buy a 4K TV for $300. The 8K TV you’ll be buying next Christmas will be probably under $1,000 or $2,000. It’s transitioning directly to 8K. YouTube has over 1 million 8K videos right now. Entire 8K workflow from Content capture, creation and post-production toolsets are available and are increasingly affordable. And you don’t need to wait for your cable carrier to start streaming their library in 8K, because it’s very expensive. For eSports and gaming community, gaming in 8K resolution brings out phenomenal details, vivid colors, higher frame rate and screen refresh rates reduces fatigue and provides immersion, like never been experience before. Experiences like this are transformative.
From first ever sports TV broadcast (in Black and White) during Tokyo 1964 Olympics to now live streaming 8K over OTT during Tokyo 2020 Olympics, industry relentlessly pushing the bounds of format frontiers. Sports and broadcast industry will continue to evolve to deliver live 8K over the OTT. For the Paris 2024 Summer Olympic Games, Intel technologies will continue to push pixel frontiers to even live 16K, multiple 8K TV channels or 8K with 120 frames per second over 5G. Technically, 16K is several times more than 8K 60FPS data rate. When the next platform comes we’ll continue to evolve and advance this technology and explore new frontiers, that’s Grovian culture in action.
The work we’re doing is the future of Olympic broadcasting, the future of sports broadcasting, and the future of live entertainment broadcasting. We are preparing the world for the democratization of 8K using open Internet. “Seeing is believing”, so we’re working to give you a chance to explore soon…