How CableLabs envisions learning with video walls, light field Holodecks, and AR/VR

As CEO of CableLabs, Phil McKinney’s job is to dream big. He gets paid by cable TV providers around the world to do research on the future and how to build the networks that the future will require, particularly to serve the needs of education.

Today, CableLabs is releasing a short film called “The Near Future: Ready for Anything,” the third in a series about how technology can help us in the future. CableLabs naturally sees wired and wireless networking as playing a big role in that future, which will require gigabytes and gigabytes of data flowing into and out of our homes and businesses. But I think “The Near Future” series does a particularly good job of showing how those technologies fit into the lifestyles of the future, whether it’s about being older or what’s the best way to teach kids.

This year’s video focuses on technologies for education such as augmented reality, virtual reality, video walls, Internet of Things measurement devices, and the “light field Holodeck” where you can interact with 3D objects in a virtual space. I spoke with McKinney about his latest vision for the future in an interview. You can also check out the embedded videos to look at the films CableLabs created.

Here’s an edited transcript of our interview.

Above: Phil McKinney is CEO of CableLabs.

Image Credit: CableLabs

Phil McKinney: This is the third film in the series. The 2016 film, “The Near Future: Bring It On,” was looking at the question of what consumers are going to do with a gigabit to the house. It has a bit of a gaming, entertainment look to it. Last year’s film, “The Near Future: A Better Place,” was looking at the role of technologies in the home for supporting people who want to age and play. Health care, the ability to get around in communities, living an independent life.

This year’s film has two elements to it. We’re calling it “The Near Future: Ready for Anything.” It’s looking at education. It has a bit of a work feel toward the end, but it’s really trying to address the question of: If you think about students today, and the role of innovation and technology, how do you prepare students today to use a technology that’s not yet been invented to solve problems we don’t know exist yet? How do we rethink that whole effort?

When you look across all three of the films, the underlying element, and the reason why CableLabs is doing these films, is really to talk about, or at least be an inspiration for, the innovators out there. We’re building the network of the future. Today more than 50 percent of all the U.S. homes served by cable have access to one gigabit. In 2016 it was less than five percent.

The industry is putting billions of dollars into the network. The question is, how are we going to inspire the innovators who create the innovations that build on this platform? We view the broadband network as a platform for innovation. For us, the role of the films is to inspire. What’s the future going to look like? What’s going to be possible? We want to inspire the innovators today to think about what part of that future they can go create, given the capabilities that we’re going to see continuing to increase in the network.

Above: The video wall can be used for collaboration in CableLabs vision of the future.

Image Credit: CableLabs

VentureBeat: It looks like you picked some technologies that are very prominent here, like augmented reality and AI assistance.

McKinney: AR, head-mounted displays, we’ve had them in all of the films. We think it’s important. But it’s more than just passive AR and VR. It’s about what we call social AR or mixed reality. In the case of this year’s film, we’re having students from the U.S. and Australia being able to work on projects together. They can feel like they’re not just looking at some rough avatar, but actually working together all in the same space on the same project at the same time. That social element asks the question: How do you make that happen, given the distance?

There are two parts to all of our films. We show the film, and then we have a video where I talk through it called “Behind the Tech.” In the 2016 films, we talked a bit about this concept of edge compute. While you have these networks passing bits, can you embed compute capabilities that are highly distributed into the network, so the rendering and the interactivity and the social aspects can be rendered extremely close to the person experiencing it? You can reduce that latency to the minimum and allow people to work and feel like they’re next to each other.

We shared our vision for edge compute, and now you’re starting to see a number of companies coming out with those approaches, HP Enterprise being one. They’ve had a big push into edge compute over the last couple of years. The technologies we picked for this year’s film are building on the commonality of the other Near Future films. All of our films are looking in a three to eight year time frame. We’re not trying to be Buck Rogers.

But we also introduce some new technologies, like the light field display: the light field that’s used in the fetch experiment in the science class, or the light field collaboration space shown at the end of the film. Light field display is a pretty exciting area of research now. It allows people to coexist in spaces, to participate. It allows for fully 3D volumetric displays. People can interact and feel like they’re working with a beaker in a science experiment or the bicycle at the end of the film.

What’s interesting about light field displays, in our current research and the work we’re doing right now, on a two foot by two foot light field display, in its current raw form, to maintain that image and allow for interactivity, you’re looking at a network feed of about 800 megabits. You start thinking about room-size displays —

Above: The vision of the Light Field Holodeck.

Image Credit: CableLabs

VentureBeat: Is that the one where they’re walking into the stage area?

McKinney: Yeah. What’s unique about light field displays is that the bandwidth does not grow with the square of the area. It actually grows with the square root of the size of the area. Starting with 800 megabits, it’s not like we’re going to have to eventually deliver terabits to cover that room. It’ll be somewhere in the six gigabit range. That’s before any efficiencies and compression and edge compute. That’s just raw pixels you have to push to get the full 3D volumetric experience.

In those areas, we’re not building light field displays. We’re doing research in those areas. It’s primarily for us to understand what are the requirements, what are the needs of the network to allow those innovations to become a reality. The cable network is rapidly deploying the gigabit networks, DOCSIS 3.1. It’s fully up and deployed. We’ll be able to support up to 10 gigabits symmetrical, because in a light field collaboration space you need upload and download speeds. We announced more than a year ago full duplex DOCSIS, which brings symmetrical gigabit speeds to the network. Consumers in their homes in the future can have 2, 5, 10 gigabit symmetrical.

Those kinds of experiences aren’t far off. There’s work being done in university research centers and stealth companies. Some companies are out there showing pretty little 3-inch light field displays. We’re looking at working on 2 foot by 2 foot with researchers. We think it has some interesting characteristics. When you watch the “Behind the Tech,” you’ll hear me talk about how, with that kind of technology, the science classes of today are a lot safer than when I was in school. As you can imagine, I was the kid throwing bombs in the pond in the middle of campus. [laughs]

Now you can let kids do experiments that you’d otherwise never allow them to do, let them have the discovery experiences and solve problems you can’t expose them to today. It’s about creating these environments to create and interact and work together, which is what we need students to have as far as the skill set to be the work force of the future that we need. It’s no longer about memorizing a formula. What can’t I Google in six seconds? It’s critical thinking skills and collaboration and creative thinking and problem solving. It’s the willingness to try and experiment. If the experiment fails, try another one and solve the problem.

VentureBeat: I read something not long ago about the “uploading lifestyle.” It goes against the idea that all we were ever going to do with the internet is download. Gamers are uploading video of themselves at the same time they download. Twitch and YouTube are making very good use of the upload capability in networks. The DOCSIS change is pretty timely in that respect. There’s a wholesale change in the way we’re using the internet.

McKinney: You’re seeing it in pockets. If you look at overall upload speeds across the U.S. market, it’s growing. It’s not growing as fast as download demand. You still see an asymmetry when it comes to what people are doing. For the longest time, you didn’t see a growth in the upload side. We’re starting to see an early trend there, whether it’s gamers who want to livestream or people working from home and using Skype more often.

Above: A video wall of the future.

Image Credit: CableLabs

VentureBeat: This week I did a story on how people in the second quarter uploaded more than 600 million hours of Fortnite gameplay on Twitch.

McKinney: The Fortnite phenomenon is another early indication. What’s the upload to YouTube now? How many hours of viewing time go there every day? It’s something astronomical. No one would ever be able to watch everything on YouTube. That all has to come from someplace. The content creation side is exploding. The tools for sharing are exploding. Fortnite is a good example, especially because it’s not just the upload. It’s the interactivity, the engagement, the community, the collaboration together. We see that all over the gaming space.

If you take a look at the first film from 2016, you’ll see a lot more about the upload side. You’ll see a young girl interacting with her grandmother in completely different parts of the country. We also showed a video wall in the home allowing the mom to work from home, but feel like she’s still a part of the office.

We don’t call out the upload and download piece specifically. Full duplex DOCSIS was an innovation we announced more than two years ago. The specifications were already done and vendors were already building them. Being able to deliver symmetrical gigabit networks is already a development as that silicon gets updated and goes into deployment. The upload side is something we saw four years ago.

It’s interesting. Our work from that is collaboration and innovation work that we did — we now fund about a dozen, a dozen and a half universities around the world. One that funded was Stanford, where we were funding wireless work. Our work with them looked at the ability to get symmetrical high speeds by simultaneously using the same channel. That’s what we use for full duplex DOCSIS. We’ve proved that it works at extremely high speeds in the existing network that people have in their homes. That solution is coming very quickly. Our timing with being able to offer symmetrical speeds is going to hit at just the right time, when we see this explosive growth in uploads.

VentureBeat: What version of that?

McKinney: It’s part of DOCSIS 3.1. DOCSIS 3.1 came out in 2015. That allowed for up to 10 gigabits download and one gigabit upload. Then a year later we announced an extension to DOCSIS 3.1 called full duplex. In some articles we refer to it as FDX. It’s DOCSIS 3.1 full duplex. That takes the upload from capped at 1 gig on up to 10 gig. You get full symmetrical. Your uploads and downloads are at the same speeds. We came out with that in the 2016 time frame.

We’ve wrapped up the specs. The silicon and the cable modem manufacturers are now in the process of building that spec into silicon and we have to get devices built. It’s still a little ways off, but the vendors are in active development to deploy cable modems that have full duplex capabilities.

When you’re doing 10 gig up and 10 gig down to consumers’ homes, the coax no longer becomes the bottleneck. What becomes the bottleneck is the fiber that feeds that connection; at the end we’ll call the node that feeds the coax into the home. Laying more fiber is very expensive and slows down deployment.

We have what I’d classify as one of the top fiber research teams anywhere. That team came up with a solution using a technology called coherent optics. When you use extreme long-haul fiber, like a fiber framework from New York to San Francisco, you use coherent. It has all the amplifiers and repeaters built into it. We developed an elegant way to use coherent in what we call the access networks. Think up to 80 or 100 kilometers, that kind of distance. A normal strand of fiber can support up to 40 gigabits. We now have shown coherent optics on single-strand fiber at 8 terabits, and we’re on a path to take that to 50 terabits.

You don’t have to change out any of the fiber in the ground. You change out the end points with a technology that’s one tenth the cost of all the current lasers. You can take your 40 gig network to eight terabits, and we’re on a path to take that to 50 terabits. To show how elegant the solution was, when we first demonstrated it to the CEOs of the biggest cable operators, we built the entire demo using parts we bought on eBay. Just to prove we weren’t doing some esoteric silicon. Through very elegant innovation, a way of looking at the problem completely differently, we could get this exponential growth in capacity with technology you can buy today, and therefore prove that it’s low cost and it’s real.

Given a combination of coherent with DOCSIS and all the fiber that’s in the ground and all the coax that’s already in people’s homes, the risk for deployment of gigabit services is pretty much taken away. That’s why today, most consumers aren’t even aware that more than 50 percent of all U.S. homes have access to gigabit service from their cable operators.

VentureBeat: I met recently with the TIP folks at Facebook.

McKinney: Yeah, we actually operate one of the TIP labs here at CableLabs.

Above: Mixed reality in the future.

Image Credit: CableLabs

VentureBeat: It’s an interesting question. There are 3.2 billion people with broadband in the world and maybe 4.3 billion without it. It seems like the natural capitalist incentive is to bring better broadband to those people who already have it. Economically, the argument for extending broadband to the other 4.3 billion is harder to justify. It makes me think about — we’re making broadband so much better and better for those of us who already have it, but there’s another conversation here about how to get it to the rest of the world.

McKinney: There’s a couple of points. One is, because of the elegance of our solution in DOCSIS that we’ve developed over a dozen years, and our work on coherent, the ability to upgrade the existing networks is a relatively low cost. You’re not digging ditches or putting holes in foundations. It’s not an apples to apples comparison to extending the network.

Now, in the case of the extending the network, if you look at the individual cable operators, a number of them are doing some interesting network extensions, like acquiring a wireless ISP. Midco, which is in Minnesota, North Dakota, and South Dakota, they just bought a wireless ISP to serve their rural markets, to find a way to extend the end of their network. They acquired a company they’d been partnered with.

They’re looking at how to get out there when they have one house per square mile. It’s hard to build a network out there, stringing cable on poles or digging ditches. But using point-to-point wireless or point-to-multipoint wireless as an extension to the end of the network can be viable and efficient. When you look at Windbreak in Nebraska, or Midco in the Dakotas, or Mediacom in Iowa, particularly in the rural areas, they’re all looking at how to extend the network. It’s an active effort.

At CableLabs we have active projects in that space to reduce, as much as possible, the cost for deployment. As we reduce that cost, our ability to extend the network grows. We have constant conversations about what we can do to bridge that digital divide. Every one of our members has a different strategy depending on what market they serve, where their network is at, and the best ways to reach and serve those customers who don’t have access to a 25-megabit connection, which is what the FCC defines as underserved.

VentureBeat: With the extension of networks on your mind, it seems like wireless is becoming the obvious solution compared to wired networks. Do you foresee less cable and more wireless in the future?

McKinney: As you know, with the laws of physics, wired networks will always be higher speed than wireless. At the same time, today at CableLabs we’re funded by 60 of the largest cable operators around the world. That includes Europe, China, Japan, Indonesia, Australia, and Latin and South America. Of those operators, 28 of them are also mobile network operators. Vodafone is part of CableLabs. Rogers and Shaw in Canada are part of CableLabs.

If you look at our research spend, the largest area for our work is actually wireless. You’re seeing some of the acquisitions. Vodafone bought Kabel Deutschland in Germany. They’re merging the wireless and the wired lines, because to do 5G networks you have to have fixed network assets deep in communities and streets and homes in residential areas. You have to backhaul off all those little cells you’re deploying. You need physical, hard network assets to make 5G a reality in the marketplace.

We’re seeing the MNOs, the mobile network side, and the fixed network side working very well together. We’ve already seen some mergers and acquisitions. Shaw bought WIND in Canada. Liberty sold its eastern Europe markets to Vodafone. They had the mobile side, but they didn’t have the fixed side there. It allowed for the eastern European markets under Vodafone to be served by wireless and wired in the same markets.

It’s an interesting set of dynamics happening in the marketplace right now. We don’t ever see where everything is going to go wireless and wired networks will be obsolete. People love the high speed they can get on their fixed network. But they want to be able to take some level of that broadband with them outside the house. If that’s Wi-Fi or CBRS or 3.8Ghz or 28Ghz, what’s the spectrum policy that allows consumers to take that broadband with them wherever they go? In our case, nearly half of our members that fund CableLabs have both a fixed network side and a mobile network side in their business models. We’ll see that happening more and more.

Above: Head-mounted displays of the future will help you understand the world around you.

Image Credit: CableLabs

VentureBeat: In the video, what do you think is the hardest thing to do, out of all the technologies you’re looking at?

McKinney: The VR, AR, mixed reality element, people have seen that. I think light field displays is one that hasn’t been talked about a lot yet. We’ll be able to build some exciting experiences with that. There’s a bit of IOT in there, which shows us the students in the garden using moisture probes in the earth, but that again has been pretty well shown.

VentureBeat: So we have the beginnings of all these things now, but the fidelity you’re showing here is pretty hard to do? AR glasses are here, but they’re still bulky and lower resolution.

McKinney: Right. Today you have avatars rather than literally seeing a person next to you. With light field displays, if you’re lucky enough to track down people who have working versions right now, they’re still pretty small. The resolution is not there yet.

We’re taking a look at Moore’s law, right? What was the resolution for digital cameras when Kodak did the first digital photographs, and what is it today with a digital SLR? If you look at those progressions of innovation, that’s what we’ve mapped. If I have VR goggles today and I can continue to improve performance in the network and edge compute — we’ve been improving the experience, the fidelity, because of embedded intelligence connecting to a remote cloud. What kind of resolution and interactivity can I get from that? What timeline will take us there?

When will we see the VR goggles we’re showing in the film? Best guess right now, probably four years or five years. The light field will be further out. That’s more like five to eight years, to get to the experimentation level of what we show in the school. The light field room collaboration with the bicycles, where people are projected into it, that’s eight years or more, that kind of time frame.

All of those are things we’re either actively doing ourselves or we have innovation partners we’re working with. Nothing we show is just us sitting around and Buck Rogering. It’s projects we’re engaged in and what we’re seeing as how they will progress. When do we think these things will become real, so we can talk intelligently about them and convey a vision and hopefully inspire researchers and startups to think about them. If you have a 1-gig symmetrical or a 10-gig symmetrical or even faster networks, what could you do with that? What would you invent? What would you put on that network to create something that’s never been dreamed of?

I think back to 1988 when CableLabs first got started. What was my broadband experience? It was dial-up. We’re 30 years old. You go from dial-up with your modem making its little chirping noises to today, where if I get less than 200 megabits I’m complaining about it.

VentureBeat: You showed the high school chemistry lab. Is the idea that you can do that lab work virtually instead of physically, where there’s dangerous chemicals and so on?

McKinney: Not only that. For years, up until I left HP, I served on the board of the Tech Museum in San Jose. I represented HP on the board. One of the shocking things I learned on that board is how few public schools in Silicon Valley have a wet lab, a chemistry lab. You want high school students to be able to walk into a lab and do some basic iodine stains or other simple chemistry experiments, and an amazingly large number of schools either don’t have a wet lab or don’t have teachers to properly staff it.

One of the things we did at the museum was create wet labs there. Teachers would bring their students to the museum to do chemistry experiments. If that’s happening in Silicon Valley, what’s happening in the rest of the world? So, could you create a virtual science experiment environment that allows this? My wife and I have been working on the construction of a P1 through P6 school in Rwanda. We’ve finally gotten broadband to the school. If I could have a virtual chemistry lab for those students, to be able to learn and experiment? Because there’s no way I’ll sell the school on a wet lab with Bunsen burners and everything else. It’s too much to maintain and supervise. If you think about a virtual science lab, what does that do for education when you’re no longer confined by the physicality of a lab?

Above: Soil sensors will give us lots of data about the ground.

Image Credit: CableLabs

VentureBeat: How have things changed in the last year that reminded you about the video from the last year, about helping older people? That caught my interest, because I’ve been having to help my 84-year-old mom.

McKinney: Part of the inspiration for that film was my grandmother. She died two years ago at 96 in Cincinnati. My brother lives in San Francisco and my cousins live on the East Coast, so who’s taking care of grandma? My wife happens to be the nurse in the family, so she deals with medical issues for her aging parents.

It’s funny. Everyone who’s seen that film, I get more people emailing or calling to say, “Where do I go to order that? I need that house for my grandmother.” If you look at some of our members — take Cox. Cox has gone out and bought a company that helps manage in-home health care. It’s not just technology. It’s not Cox Communications. It’s part of Cox Enterprise. But they’re working with Cox Communications to allow for remote monitoring, to take what we have in the film and make it more of a reality.

Cox decided to do that by acquisition, but they already had a long relationship with the Cleveland Clinic to support Cleveland Clinic’s remote tele-health program. They allowed specialists at the clinic to be able to remotely engage with patients and other doctors to provide remote support so that patients who couldn’t get to Cleveland could still get the benefit of the best and brightest. Cox’s acquisition, to get more engaged with in-home health, that’s just one example of where we’re seeing that become a reality.

VentureBeat: For my mother, I’ve figured out that what I need is a transcribing video phone, something I can write messages on as well. She has hearing problems.

McKinney: My grandma had hearing trouble as well, and she never wanted to wear her hearing aid. It’d squeal with feedback when she tried talking on the phone. She just didn’t like it. I tell this story in my book. We bought her an HP digital picture frame, and all my kids and my cousins and their kids, they loaded up pictures on this frame to put in my grandma’s house. A year later I went down for a visit and there’s an image on the digital picture frame, but it’s not moving, not showing the next picture. I’m still at HP at the time, so I think, “Oh, crap, our digital picture frame broke down.”

When she walked into the kitchen, I snuck over and picked up the frame from the end table, and what she had done was unplugged the picture frame and taped a physical picture on top of the glass. She did that because, one, she was worried it could start a fire, and two, she didn’t want to pay the electric bill to run it. She grew up in the Depression, you know? So here we are in 2009 or 2010, 80 years later, and that’s still the mindset.

Above: The AI agent Dot is hyper-personalized for you.

Image Credit: CableLabs

VentureBeat: It’s amazing to look at the ways people can still defeat technology.

McKinney: [laughs] Yep, yep. Again, part of the objective for the films — I don’t know if you remember, but I did a series of six videos for Carly [Fiorina] at HP which were all around mobile services. We showed that AR game running around San Francisco, Roku. What’s interesting is that the team I worked with to do those videos was the same team I worked with when we did the Corning video “A Day in the Life of Glass,” and it’s the team I worked with on this. They’ve been together 18 years creating these vision videos.

If you look at Roku, it’s basically Pokémon. That video came out in 2006, long before any of the smartphone capabilities we have today. When did Pokémon Go come out — 2014, something like that? Eight years after Roku, that vision video. Or you go back to Apple’s Dynabook. The Dynabook vision video, that lag time as far as painting the future — when does that future become real?

The videos, for me, are a forum for me and CableLabs to have conversations with community leaders, with other technology companies, about what the future could be if we all committed to making life better, to using technology for good. Enabling people to have those experiences, those interactions. Better education, better learning, better health, better family interactions. I guess I always swing to the optimistic side.