Experiments with VR motion controllers show improving immersion increases the risk of VR sickness and that the ill effects are a varied and complex matter.
One of the joys of working in the R&D Labs at Tapptic is the excuse to spend a week playing with new gadgets, but all good things can turn sour. And our experiments testing new hand-held motion controller systems and pushing the boundaries of virtual reality motion to the limits isn’t one we’d recommend, because it caused some of our human guinea pigs to feel severe discomfort and nauseous.
This article will describe our experiments, explain how they made us feel sick, and how we tried to reduce nausea and other ill effects. It will also outline our consequent analysis of VR sickness syndrome and conclusions that both the causes and the symptoms of VR sickness are more complex, profound and varied than many VR motion studies suggest.
Setting the scene
While wandering around the Electronic Entertainment Expo Los Angeles last year, a motion controller system caught our attention. People were testing a VR game called Sprint Vector, clutching special handheld controllers that enabled them to run and jump in a VR simulation by swinging their arms back-and-forth, like a soldier on a speed march, and throwing both arms up in the air to make their avatar jump — while the players’ real legs remain stationary.
For those who haven’t played a VR game before, the usual method of in-game locomotion is teleportation, where the player looks and points with the controller toward where they want to go and presses a button to move there. Teleportation came about as a way to avoid motion sickness, commonly experienced when using a joystick or keyboard directional buttons in VR games, but this lacks the immersive nature of being able to actually walk or run and to move forward while looking right or left in your virtual world.
So let’s teleport to Poland. Michał Owsianko, the VR expert at Tapptic, built a simulation to test the HTC Vive headset with the handheld motion controllers. We conducted a number of experiments including walking and running using the handheld controllers, with the avatar’s direction of movement determined by, first, the way user’s head was facing, and subsequently, the way the torso was facing. We also examined what happens when the avatar walks/runs through virtual objects such as walls.
Ten people took part in the study. All were affected in some way, but some were unconcerned, while for others the effects of VR sickness were severe and prolonged, and in one case did not kick in until long after the experiment had finished.
Before we discuss the experiments, let’s explore the causes and symptoms of VR sickness.
The common symptoms of VR sickness are disorientation, lack of balance, headaches, and eye fatigue, as well as feeling sick, even retching and vomiting. These are similar to motion sickness, like car, seasickness, and simulator sickness (a long-time problem with Air Force flight simulators). Some of the causes are also similar, but with one major difference: You don’t need physical motion to experience VR sickness.
Like motion sickness, VR can cause nausea when there is a disconnection between your external sensory information (what you see and hear) and your internal sensors, known as the vestibular system. This means: if what you see and what you feel doesn’t match, you will feel ill and can actually vomit. Not everyone will be affected in this way, but it’s one of the main reasons why VR sickness happens.
But there are other causes of VR sickness that have nothing to do with motion. One of these is the eyes. Serious gamers claim that higher frame rate, such as 60 frames per second (FPS), delivers a much better gaming experience than 30 fps (for reference, the fps of a standard movie is shot at 24 fps, high-definition HD film doubles this, at least).
Perhaps there is biological reason for this: In order to minimize eye fatigue and disorientation, you need a smooth and consistently high frame rate. Expert opinion varies on what fps is acceptable for VR, but at Tapptic we believe 60 FPS per eye is the minimum requirement for VR (and 120 FPS for full HD resolution). This means you need powerful machines to run VR or settle for simpler simulations.
Another ingredient for the visual disorientation is field of view. Interestingly, this is more acute for women than men. Did you know that women tend to have better peripheral vision than men? So women see a more panoramic view, while men tend to have better straight-ahead distance vision. This means that women need a bigger field of view in VR to avoid feeling nauseous.
Then there’s the full array of proprioceptors in our body. These are muscle spindles that are located in muscle fibres throughout the body. They inform us where each limb is, how the joints are positioned and how much pressure each part of the body is experiencing, without the eyes needing to see them. If the messages stop or if the eyes and proprioceptors tell you different things, it may result in an “out of body experience.”
This mismatch between what the proprioceptors tell you is happening (real world) and what your eyes tell you is happening (VR world) can cause sickness. Our studies reveal this is particularly likely to occur when the VR simulation allows you to walk through objects. So if the VR avatar walks through a wall, the brain expects proprioceptors to report that you have hit a wall. And, we suspect, prior to impact the brain may warn the body to brace and/or prevent impact. When your real body feels no impact from the VR collision because there is no haptic feedback, it does funny things to your brain and stomach.
Most studies focus on the frame rate (vision) and motion orientation (vestibular system), suggesting that the impact of proprioceptors is not fully appreciated.
The psychological implications of VR sickness are often overlooked.
When writing his 1987 paper on flight simulator sickness, J.S. Crowley identified that airmen who had experienced physical symptoms of simulator sickness feared repeating training sessions in the simulator. While flight simulators are different to modern VR headsets, the physical symptoms are very similar to VR sickness i.e. eye fatigue, disorientation, nausea, vomiting, etc.
My own experience suggests there are psychological implications of VR also. It might sound silly, but after my severe and delayed reaction to the VR experiments, even a week on, I felt some fear about taking part in more VR testing.
Despite prolonged experimentation with the Vive VR system, the effects didn’t hit me right away. They appeared several hours after I finished, and when they kicked in, I felt terrible. I could no longer work, had to leave the office early, go home, and sleep off the effects for a couple hours. This delayed reaction is concerning. If the negative effects are triggered during the simulation, then it makes it extremely difficult to manage when to stop.
A generally good rule of thumb is this: If your face or ears are getting hot or you get disoriented, stop right away. If you ignore these warning signs and continue using VR, then you risk going to a “deep sickness” state, which can last for hours and give you a headache that strong painkillers won’t shift.
What we tested
It took Michał, our VR expert, a couple of hours to put together a demo for the HTC VIVE system, and a few more hours of refinement, polishing and testing, to have a working system. This enabled us to move around the VR world, holding the Vive handheld controllers, while swinging our arms to simulate the walking of the virtual avatar. The faster you swung your arms, the faster the avatar moved.
The two of us tested it for some time with no obvious ill effects. Then we invited some colleagues to take part, several of whom began to feel nauseous very quickly.
We tried a number of refinements:
- Move where your torso is pointing, not where your head is looking. This means you can walk around while moving your head from side-to-side without feeling “horizontal movement” nausea. However, problems can still arise when you don’t have both controllers in front of view.
- Walking upstairs. This was harder than it seemed: the rigid body forces of your virtual avatar and the colliding surfaces of the world caused weird accelerations inside the virtual world, including the avatar flying off or falling and causing discomfort for the user.
- Try actually walking. By attaching a Vive tracker to each leg, you can move your avatar by moving your real legs (while trying not to physically move). The biggest problem here came when the trackers lost contact with the base stations, which are infrared cameras. Disconnections happen very often, resulting in three possible outcomes:
- The avatar stops, but your real legs continue to move keep walking physically – the sensation is mildly uncomfortable, akin to walking wearing a blindfold.
- The avatar movement continues at the same speed while you stand still – this sensation is more uncomfortable, with strong disorientation and nausea and sometimes a fear of falling. Sitting down can relieve it.
- The avatar is propelled forwards or upwards by an invisible force at 200 km/h – this sensation is extremely uncomfortable. The sense of disorientation, nausea and vertigo (if flying), can be unbearable and sitting down doesn’t help.
Results on the next page:
When the tests were completed we asked the 10 participants three questions: How they felt using the new input system; how they felt when they walked through objects; and how they felt when they removed the headset.
All test subjects said they found using the controller a far more realistic interface for VR than standard teleporting.
When the experiment was completed, we asked each participant how using the input system made them feel:
- Only one person out of 10 had an excellent time testing the demo, and nothing that happened could make him even shake.
- Four felt fine with the experience.
- One felt fine at the time, but later experienced ill effects.
- Three of the testers had considerable issues with the system, only lasting a few seconds before they had to stop.
- One made it part of the way though before needing to sit down to recover.
We’d found little research that studied what happens when you collide or pass through virtual objects. In teleportation-based games and simulations, it isn’t possible. So in this demo you were able, and encouraged, to “run” at speed, crossing right through objects of all sizes.
On the whole, this is an unpleasant experience. The disconnect between the sight and proprioceptors feedback is quite strong, and in some cases it can cause temporary paralysis, give you shakes, or, with one test subject, caused extreme and instant unease.
Also it’s a weird feeling to hit something but feel no impact, due to the lack of haptic feedback.
Reactions to walk/running through virtual objects:
- One person loved it.
- Three weren’t bothered by the experience.
- Three felt the strangeness of the collision ruined the immersion of the simulation.
- It made two feel ill.
After using the system – which demanded some physical activity – we wanted to know how it felt to remove the headset.
- Three people felt ill effects immediately on removing the headset.
- One (myself) felt fine at the time but had an unpleasant reaction a few hours later.
- Three people, the ones who had experienced the most discomfort, were relieved to get back to reality.
- Two people, who’d found the VR experience more enjoyable than others, reported that real life now seemed fake, walking wasn’t as “easy” as it was before (e.g., it was more noticeable that walking needed a bit of effort).
Can we limit the effects?
Considering the similarities between motion sickness and VR sickness, it is possible that sea- or car-sickness pills, or sucking a ginger sweet (ginger root contains the same chemical component used in tablets and medicines), may have a calming effect on VR sickness.
The U.S. military developed a system that electro-stimulates the vestibular system to counter the motion of the VR simulation. Reportedly this tricks the brain sufficiently well that even combat pilots using the most advanced and roughest of flight simulations experience no motion sickness at all. This could also work with VR sickness, but it is a bit extreme for commercial, training or consumer applications.
Finally, introducing a fixed point of reference within the VR view may reduce the incidence of VR sickness, by diminishing issues with the vestibular system. For example, researchers at Purdue University, USA, recommend using a virtual nose mid-bottom screen as a constant reference point.
Our experimental group was only a small one, but it’s big enough to show there were too many issues for too many of the test group for this input system to appeal to a mass audience. This is a shame because all test subjects found immersion and experience with our new input system and the HTC Vive controllers superior to teleportation.
It is also important to note that the reasons why VR causes sickness, and the physical and psychological impacts it has on participants, are more complex and varied than many would have you believe.
Almar Suarez represents the R&D department of the digital european agency Tapptic, as R&D Project Manager.