Decoding virtual reality: Understanding the language of video game controls

This post has been edited by the GamesBeat staff. Opinions by GamesBeat community writers do not necessarily reflect those of the staff.

I never really "got" motion controls until I sat down with The Legend of Zelda: Skyward Sword and started slashing cuddly Bokoblins with erratic wrist gestures and tossing bomb flowers like bowling balls with an upward sweep of the remote.

That’s also when I realized I didn't actually like motion controls. The game wasn't acting like a game, it was mimicking real life. I remember playing for the first time and asking, "Which button swings my sword?"

A friend assured me I would figure it out, but it still wasn't obvious to me. This subtle shift in cognitive problem solving, from "press X to swing sword" to "pretend to swing sword to swing sword", is fundamentally changing the way our brains think while playing video games. As it stands now, I believe motion controls are an unnecessary distraction.

Motion controls in traditional games can make a fool out of anyone.


Playing video games is like learning to speak: For each new title, you need to re-learn how to communicate your will to its world. Traditional games use a "symbolic" language where you must understand each button's function, either through a manual or experimentation. Like our alphabet, the spidery letters act as symbols; they're absolute nonsense until you learn to decode them.

Motion controls speak in an "iconic" language, where the sweep of your arm resembles the sword strike on-screen. Like writing ":)" to denote a smile, the icon resembles its meaning similar to Egyptian Hieroglyphs.

Both are a means of communicating, but each have their own strengths. The non-literal nature of traditional video games is its greatest strength. Just like our language, when we code our thoughts into symbolic controls like gamepads, we are liberated to perform a wide variety actions with ease.

The strength of motion controls is a currently unrealized level of immersion, but by creating a hybrid in Skyward Sword, we actually dilute the overall experience until neither communication styles take advantage of their strengths. We cannot sword fight with the type of grace and control symbolic input provides, nor do we feel truly immersed in the reality of Skyward Sword because of the limitations of technology and breaks in the illusion. And why would you want to destroy an immersive virtual world with traditional gaming elements like hit points?

As Tom Bissell points out in his review of Dead Island, gamification is only powerful within certain contexts, and the immersion he's looking for does not require game-y conventions like levels and health bars.

"In a game about running from things that want to eat you, what is more important: the emotional experience of running from things that want to eat you, or knowing that the thing that wants to eat you is a Level 23 thing that wants to eat you? Knowing that the machete in your hand can take its head off, or knowing that the machete in your hand is capable of doing 320+ hit points of damage?"

There are two futures for both these technologies. Both are glorious, but paired together neither of these entertainment forms can reach their full potential. Ultimately, combining the immersive nature of motion controls and symbolic language of traditional video games is a hot, sticky mess and it's easy to see why. Can you imagine sitting down to read Oliver Twist, except every fourth word was a pictogram?

Oliver walked twenty (picture of a mile) that day; and all that (picture of a clock) tasted nothing but the (picture of a crust of bread) and a few draughts of (picture of a glass of water).

A classic indeed.

Screen Shot 2014-03-25 at 2.00.11 PMGamesBeat 2014 — VentureBeat’s sixth annual event on disruption in the video game market — is coming up on Sept 15-16 in San Francisco. Purchase one of the first 50 tickets and save $400!
blog comments powered by Disqus

GamesBeat is your source for gaming news and reviews. But it's also home to the best articles from gamers, developers, and other folks outside of the traditional press. Register or log in to join our community of writers. You can even make a few bucks publishing stories here! Learn more.

You are now an esteemed member of the GamesBeat community. That means you can comment on stories or post your own to GB Unfiltered (look for the "New Post" link by mousing over your name in the red bar up top). But first, why don't you fill out your via your ?

About GamesBeat