When Microsoft announced its Windows Mixed Reality platform last week, it talked about how the combination of virtual reality and augmented reality (which Microsoft calls mixed reality) would create a new era in computing.
Alex Kipman, technical fellow at Microsoft and the researcher behind Microsoft’s HoloLens AR headset, went on stage at an event in San Francisco and answered questions from a group of journalists. He did his entire presentation while wearing a new Samsung HMD Odyssey mixed reality headset, and he predicted that communications would be the killer app for mixed reality.
That’s why Microsoft acquired AltspaceVR, a social networking space in VR. AltspaceVR and Microsoft will try to figure out ways that people can meet each other in VR or AR. Developers will be able to make apps on Windows that can run on VR or AR headsets, and consumers will be able to use Windows for VR or AR apps, starting on October 17. There are plenty of other apps, as well, from Microsoft’s own Halo Recruit VR demo to the whole SteamVR library (coming later this year).
Kipman said, “We are standing at the threshold of the next revolution in computing — a revolution where computers empower us to expand our capabilities and transcend time, space, and devices, a revolution where we immerse ourselves in virtual worlds of our choosing, and we accomplish seemingly impossible things while making lasting memories with the people we love.”
After his presentation, Kipman took a bunch of questions. Here’s an edited transcript of the conversation.
Question: Any plans to make the Cliff House social, as in bringing friends into your personal home?
Alex Kipman: Absolutely. Maybe not in this current release of Windows, but Windows has a path of releases ahead of us and making the Cliff House social is part of our plan.
Question: Should we expect to hear anything about Xbox and VR soon?
Kipman: We’ve been pretty open with our VR strategy on Xbox. When you think about mixed reality or virtual reality in the living room—the living room is a communal space. You need wires to be in the right place. We just don’t believe, right now, that the state of the art of what virtual reality can do in the living room – from the headsets to the wires to the state of wireless – is such that it’s actually an enjoyable experience.
What you’ve seen us do is leverage this experience on PC first, where the cords are already in the right place, where the horsepower of the machines from the GPU and the CPU are in the right place. Over time, absolutely, we’ll have it in Xbox, but we’re not going to announce anything for that space today.
Question: You described Windows Mixed Reality as the first spatial operating system. Do you want to differentiate it from Windows 10 on its own, or do you still consider those the same thing? It seems like the way you’re presenting it is as its own operating system outside Windows 10.
Kipman: I’m glad you asked that question, because I’m going to disambiguate it. Absolutely all Windows 10. There is one operating system. It’s Windows 10 and we love it.
What I do want to differentiate is how that one operating system evolves into the future. As a matter of fact, in all your machines, on October 17, you’re going to download this version of the OS. You’ll have the same beautiful desktop, and when you plug in the headset there will be nothing to install. You’ll just go through the headset setup, download some updates perhaps, and you’re off to the races. The idea of being the first spatial operating system is just a natural evolution of the operating system to adhere to this immersive world that is mixed reality. So we’re absolutely not trying to differentiate from Windows 10. It is Windows 10.
Question: By October 17, how many mixed reality titles do you expect to be available? Is there anything you’d regard as a killer app?
Kipman: For me the killer app for mixed reality is Altspace communication, the communication fabric of being able to be with all the people you love in a family setting. Or as an employer, to be able to have my people be around the world in a physically present way without actually having to be physically present. The ability this provides—I mean, why are you here today? Think about that. You all have, I hope, put a premium on face-to-face interaction, reading each other’s facial expressions. There’s more to this experience than doing it over Skype, text messages, email.
We believe in the power of being present. The little con is that you have to be in the same place at the same time. Mixed reality gives you the superpower to transcend time and space, and now all of a sudden, in the very near future, you’ll be able to have communication that’s this immersive, this present, but maybe I’m standing on the surface of Mars. That’s the killer experience. It’s communication. You see that our acquisition today is part of the strategy of making sure that we start building a communication fabric created for mixed reality.
As far as the first half of your question, with how many titles are available in the store—it’s a store. More titles come in all the time. Out of the gate you’re getting more than 20,000 applications in that store. You saw that whole wall of immersive virtual reality experience, and that was a fraction of the titles. If I had to quote a number it would be 20,000, but it’s the Microsoft store. More titles are coming in all the time.
Question: Could you give a few more details on the AltspaceVR acquisition? How many of those team members are going to join Microsoft? Are you planning to rebrand that? Can you say anything about a purchase price?
Kipman: Most of those things we don’t talk about. My key objective with any acquisition is making sure that we stay true to the reason we bought them. To great extent it’s a partnership with people who’ve been pioneers in social mixed reality. The point of joining forces is to learn from each other and share the DNA that everyone has.
From that perspective, the majority of the team did come over as part of the acquisition. I don’t have anything else to talk about today in terms of branding, future direction, blah blah blah. The objective with the acquisition is to get the teams together and start creating, together, the mixed reality future centered on communication.
Question: Is the idea to build out AltspaceVR to keep what’s there alive, or do you want to start over and build a new social experience?
Kipman: In a very deliberate way: day one is keeping them precisely where they are. They have a very vibrant community in virtual reality. We love that community and want that community to keep going strong. They’re on many other platforms, and we continue to support that community on whatever platform they’re in, from the web to phones to everywhere in between. That’s the simplest answer I can give you. Day one, nothing happens on Altspace besides more building that community and keeping that community going forward.
To the question of whether people will be able to visit your Cliff House—over time, we’ll evolve the experiences. One can imagine that in the world of tomorrow, the minute you join any of these immersive experiences you’re going to be able to be with the people you love, the people you work with, and the people you’re trying to communicate with. Altspace is a very good foundation for that world of the future. But again, I have nothing more than to be excited about the fact that we acquired them. We love the community of Altspace and Altspace is going nowhere.
Question: We saw you interacting with a lot of legacy apps in 2D. What’s the timetable for some of those legacy apps evolving to a mixed reality interface?
Kipman: I think you’re going to see that it’s mixed. To a great extent, there’s a tremendous amount of value in our entire portfolio, in the Windows universe of apps. Part of my objective as a platform holder is to reduce friction for developers, so they can target a single unified platform that goes across an Xbox, a PC, a HoloLens, and our mixed reality capable devices.
Much like anything else on our universal Windows platform, developers are customizing for the device end point. In the same way you can get Spotify on the desktop and Spotify on the Xbox, the apps are a little bit different. One needs to be controlled with a game pad from about 10 feet away, the other needs to be controlled with a mouse from about two feet away. The folks at Spotify adjust for the end point within a single application that morphs in terms of both input and output to match that end point. It’s easy to imagine Spotify tomorrow doing the same thing, except within the context of mixed reality. Break the bezel and put music all around me, something like that.
The example I showed you with the picture of the little butterfly app, it’s an example of that. That’s a universal Windows application that had no bezel, where content is just popping out. You’re going to see that content start showing up in Windows Mixed Reality, and one expects that, over time, the majority of applications are going to be intelligent enough to say, “When I’m doing these types of things I’ll optimize for motion controllers. When I’m doing this I’ll customize more to not need a bezel.”
Question: When it comes to the Windows Mixed Reality platform, how do you see the early days of the platform addressing the mix of consumer and entertainment versus enterprise and commercial? There’s a lot of good business applications for mixed reality and a lot of them seem very bespoke. The consumer applications seem much more general and broad. What amount do you see being more business-focused or more consumer-focused in the early days? Do you see that shifting one way or the other over time?
Kipman: I think humans will be humans. Are you a consumer or are you working right now? We do all of these things. One of the beautiful things to embrace in the future is the fact that these devices will need to do all of these things. It’s not a question of how much time you spend on your laptop working versus watching a movie or playing a game. The answer is yes. It’ll vary by the day, by the person, by the time. You’ll see the same even in the early days of mixed reality.
Using something like HoloLens as an example, we have more early traction in modern workplaces, where it’s transforming people’s lives. I think you correctly call those applications more bespoke applications. You’re going to see ISBs building more evergreen applications, or out of box value, as the world moves forward. In the world of entertainment, gaming, things like that, there’s more content there today and we’re seeing that show up as well.
I wouldn’t answer with a proportion, though. I think these devices are the future of computing. We’re going to be wearing these devices most of the time going forward. The value is going to be in revolutionizing how we work, how we communicate, and how we play. It’s all of those things.
Question: How do you see yourselves encouraging more businesses to adopt these headsets specifically?
Kipman: The key premise here is choice for our customers and opportunity for our partners, in a sense. Connecting all of it with a single operating system that can target the whole spectrum.
We put out a demonstration with our partners at Cirque du Soleil at a conference we had for developers earlier this year. What we usually show there on stage—it’s a set of workers wearing HoloLens, a see-through device, and they’re in the middle of doing something. In this case they were building a show, moving objects around. But when they asked someone to teleport in, that person teleported in as an avatar by putting one of our Windows Mixed Eeality headsets on. If you’re teleporting somewhere in a business context, you don’t necessarily want your physical environment to be there. You want a virtual environment, because you’re trying to help someone. In this case the person was trying to say, “Hey, move that there. Not that prop, that’s different.”
The fact that I can connect these experiences—I can have the first-line workers, for example, in a HoloLens, and the knowledge workers on the virtual reality devices. A single experience, a single service that can target both is what businesses are looking for. To a great extent this is why I’m proud of the fact that Microsoft is the only company approaching the entire spectrum. We’re not trying to make augmented reality or virtual reality or anything else. We’re saying, “People have phones. They have PCs. They have opaque devices. They have see-through devices. How do I provide value for that entire value chain with a mixed reality commitment?”
Question: From a hardware perspective, the old MR headsets tightly followed a reference design from Microsoft. The displays were pretty much the same. With Odyssey it seems like a premium experience – OLED displays, the microphone, and so on. Does that fracture the platform a bit, when you have Windows Mixed Reality on integrated graphics, then Ultra, and you have two different classes of hardware?
Kipman: The way to look at it—I wouldn’t call any of these beautiful, amazing, powerful, delicious headsets old. They’re all brand new on October 17. What you’re seeing, again, is choice for customers. Choice on price point, choice on spec. They’re all based on our reference design. The perspective is, all of these headsets—I don’t care if you plug them into an Ultra machine, if you plug them into a Windows Mixed Reality machine. It just works.
To your point, fragmentation is bad. You’ll see headsets change over time. There’s opportunity for our partners. We have an amazingly creative ecosystem of partners and we’re working together with them to give value to their businesses and their customers. As we do that, we’re creating a single platform and working with our silicon partners – Intel, AMD, and Nvidia – to make sure we lower the total cost of ownership of these experiences and keep a seamless experience.
The only difference for a customer walking into our line this year—if they want something with discrete graphics, that’s a Windows Mixed Reality Ultra spec. If they want something with integrated graphics, that’s a Windows Mixed Reality experience. Plug any of these headsets into either spec, it just works.
Question: Is that going to be the case with all hardware you’re going to be able to buy going forward?
Kipman: You’re asking a futuristic question, which is hard for me to answer. I’ll tell you that our premise and our philosophy is one of making sure we create delicious customer experiences. From that perspective, creating a complex matrix of what plugs into where is not really that reasonable. I think you’ll see some content that just works on Ultra, some content that works on everything. That’s no different from gaming today. Do you want super high-end gaming and leave no pixel behind? Then you need a beefier GPU.
Content will freely flow into this, and our store is already super capable of clearly messaging to customers the type of specification they need. But our philosophy on the headsets is really, look, you buy a headset, plug it in, run the setup, and it just works.
Question: Where do you see Windows Mixed Reality working with phones?
Kipman: I don’t have anything more to share today on that topic. Today we’re here to talk about these headsets. I will say that at the end of the day, mixed reality does go from phones to PCs to headsets. If you closely track what we talk about, we talk about this idea of a Microsoft graph. The Microsoft graph is the ability we have to make sure we embrace the entire ecosystem, the plurality of devices out there. In this case, you could start browsing the web on your iPhone or Android phone and continue or finish that browsing on PC. Or you can start creating a Powerpoint on your PC and finish it on your iOS or Android or Windows phone.
It’s very easy to imagine a future, from a mixed reality perspective, where we’re going to continue to leverage this Microsoft graph we’re building to make sure that this value to our customers continues. I said it at the beginning: this thing is going to transcend time, space, and devices. I don’t know what you have in your pocket, but I’m going to bet it’s not a Windows phone. But what is the value that’s going to come to you from that experience? Let’s be people first. Let’s be customers first. With the Microsoft graph, you should expect that our experiences will start transcending devices.
I’m excited about the progress the entire industry has made toward embracing mixed reality this year. You guys have seen the developments on other platforms. I’m excited about it. It’s more sensors and more devices that will let the value of our AI-infused OS, of our mixed reality experiences, both in enterprise and in the consumer space, see value across the entire value chain.
Question: In your demo you showed both augmented reality and virtual reality, with the Samsung headset. Is the market breaking down already, in this early stage, where business use will be AR and consumer use will be VR?
Kipman: I really don’t think so. We’ve spoken before on this. That’s why we call it mixed reality. Without a shadow of a doubt, I can tell you that in the near future, devices will do both. What are we going to call those devices? As soon as I make a HoloLens go opaque, what is it? As soon as you pass through AR into virtual reality, what is it?
Most of the enterprises I talk to you, again, they’re using HoloLens for first-line worker scenarios — someone working on a jet engine, a manufacturing line, an oil refinery – but when they call a knowledge worker that’s sitting at a desk for help, for remote assistance or anything like that, more often than not that person just wants to put a VR headset on.
To some extent, what I think you’re seeing is that because the world has fragmented the platforms, enterprises have to pick. Consumers have to pick. Which is why we have, from the beginning in 2015, called this thing mixed reality and created a unified platform for both. As you give these gorgeous devices, with that same universal platform, to those same amazing enterprises and consumers, what you’re going to start seeing is you have the range to do both.
The real value comes in when you’re transporting people, places, and things in either a digital or analog fashion. These devices allow you to do that.
Question: You showed the Cliff House and a lot of ideas around productivity, which I always find interesting – the idea of multitasking in those windows. I use these systems a bit already. But one I’ve problem I’ve seen—how do you solve or rectify the problem of keyboards?
Kipman: Maybe I’m biased because I work with these things, but I’m using them more these days than I do my monitor — when I’m working, when I’m being productive. The beautiful thing about Windows is it supports all these inputs. Likely as not, when I’m sitting at my desk, instead of having a 30-inch monitor, I’m going to have all this data around me. I can sit in my chair and my keyboard and mouse are there. I can feel them.
Obviously you can click any input anywhere in the shell and get the same Windows 10 virtual keyboard coming up. You can ink on anything because it’s the same Windows 10 universal controls. You can use your game pad or your motion controllers. Typing on the virtual keyboard with the motion controllers, I find that to be faster for me than using the virtual controller with my fingers. I can double click-click-click. But when I’m reading and sending mail, I use my voice a lot, my speech for dictation. I just reply. I don’t know if you’ve used dictation with the latest version of Windows 10, but it is delicious. I’m typing paragraphs. Whenever I need to write some code, fine, I have my keyboard right in front of me.
If I look a little bit into the future, it’s not that far-fetched to think that this idea of mixed reality could entail actually seeing your keyboard in the virtual space and being able to manipulate it, with one key difference: superpowers. You could make your keys become anything. They don’t just have to be a QWERTY keyboard. Imagine the power in that context, from a creativity perspective. It’s a really powerful concept.
Question: Is the Cliff House suitable to be used seated, entirely, including navigation? Is there any consideration for accessibility throughout?
Kipman: Windows 10 accessibility is awesome. It is not a new operating system. It’s Windows 10. I’ve used it both ways. I use it sitting down. At that point I’m mostly teleporting. When I’m in my living room and I happen to be playing a game that has me moving around – Space Pirate Trainer, I’m moving around a lot – if I’m in the shell then I’ll find myself physically moving around to go from place to place as well as teleporting. We take lots of very detailed craftsmanship to make sure the experience is fantastic in both cases.
That includes–when you’re walking around physically, how do you keep it safe? How do you put a boundary around you, so you’re confident you’re not going to fall? Once you’re experienced you can set a boundary, so that if you are physically moving around, not sitting down, it will rez in the environment so you don’t get in trouble. As you’re going through, you’ll say, “Am I standing or am I sitting for this particular experience?” The experience will adjust to you.
Question: What about the duration you expect people to use these? With some of these previous headsets we’ve used, you jump in for a game and you jump out again. Maybe you play a few hours here and there. Now I could theoretically work through Windows with one of these headsets. Can I do that all day long, day after day? Are there any safety or health considerations there?
Kipman: This is back from what introduced the world to mixed reality in 2015. You could hear me talk about it then. I said, “The key thing that we put the highest premium on is comfort.” This could be comfort of the devices on your head, or the comfort of input going in your eyes. When we defined HoloLens in the beginning, all these other devices were making you nauseous. They were uncomfortable on your head. That’s why you’re seeing great traction with HoloLens in the workplace – not just to play games, but to get to jobs done. All the science fiction that we’ve turned into science fact starts from the premise of, “How can I make you have a very comfortable experience in these headsets?”
As we move into the virtual reality realm, that’s why we put HoloLens technology inside them, all of our know-how and learnings from HoloLens, to make sure these devices are comfortable. That comes from a vestibular perspective and a comfort on your head perspective. So no, nobody should be worried. There are no health issues. We have the devices in the back and you can play with them. Put them on and then put on other devices. I guarantee ours will feel much more comfortable for your eyes, and that’s by design.