The very mention of the term “virtual reality” conjures images or the 1990s schlock Steve King film, “Lawnmower Man”: spandex body suits, human-sized gyroscopes, and of course, the seminal VR goggles. The promise of VR spawned arcade games, amusement park rides, and even a sequel to “Lawnmower Man,” but the technology just never truly materialized for the home user, much less the humble software developer.

While it’s been 20 years since VR was a buzzword, a new wave of products has brought the actual hardware into the realm of the plausible. Products like the Oculus Rift, Novint Falcon and the Virtuix Omni Treadmill put VR equipment into the three-digit price range, and with them comes the promise of developers building with these tools.

These products aren’t just toys, either. The Oculus Rift’s popularity has imbued a small team with a successful Kickstarter with the power and cachet to employ Doom creator John Carmack as CTO, and to win the favor of venture capitalists.

It’s unsurprising so many game developers have immediately expressed interest in the Oculus Rift. They’ve also supported existing devices, like the Novint Falcon; Valve Software, for example, released patches for Team Fortress 2 and Half-Life 2 to allow players to use the Novint Falcon or the Oculus Rift.

But VR has evolved in its 20-year hiatus, as have the hardware for it. An even more world-bending wave of equipment and developer access is on the horizon.

Release Valve
Valve Software has been a hotbed for innovation in the gaming space, and in 2011, the company hired a bevy of engineers intent on researching virtual reality and augmented reality projects. But in late 2013, the company closed down some of these skunkworks projects, resulting in a number of departures from the company.

One of those departures was Jeri Ellsworth, an electrical engineer with a penchant for designing computer processors in her free time. She and fellow ex-Valve employee Rick Johnson joined up after leaving the company, forming Technical Illusions.

And in May, they unveiled the CastAR, a pair of polarized sunglasses with a projector mounted on each side, and a camera in the middle. Using a piece of reflective fabric laid at a 90-degree angle, such as on a chair, the person wearing the CastAR can project a 3D image in front of him or herself.

The image can then be manipulated with either a special device or the human hand. Ellsworth and Johnson claim to have achieved accuracy of .07 millimeters at 3 meters’ distance from the US$10 cell-phone camera part they’re using.

But with all this innovation comes a central question: How do you even design software to take advantage of this and other new interface paradigms?

“What we realized early on is that this is such a new space to visualize data, games and interactions that we needed a mechanism to rapidly prototype and dynamically watch people interact with the device, and update very quickly,” said Johnson, who handles the software side of CastAR.

“That was one of my very early goals: to create game prototyping systems that were simplistic in graphics and abilities, but which  allow us to create interactions and start making adjustments. We realized that there’s so much we don’t know, and the only way to know was to do experiments and tweak things as we progress.”

Within those revisions, Johnson said that basic UI elements also have to be rethought from the ground up when you’re actually presenting them on top of the real world.

“UI is even more difficult to grasp in this experience,” said Johnson. “We’ve done experiments with 2D interfaces, we’ve thrown 3D text objects in the middle of the play-field. We’ve allowed people to see their score as part of the play-field, and we’ve tried versions where it’s projected onto a plane off to the side.”
#!
Feeding back
Such experimentations may one day form the basis for new expected behaviors in user interfaces, when such AR/VR equipment becomes commonplace. But the promise of virtual reality, as it was portrayed in the 1990s, is one of enhanced experience with more than just sounds and video. In those imaginative sci-fi days, VR participants were suspended in gyroscopes and hooked into what were hinted at as pleasure devices.

Such interactions are also becoming possible through other interesting interface devices, such as the Virtuix Omni Treadmill, a device designed to replace the directional controls in first-person shooter games. Players standing within this miniature octagonal ring can walk in any direction. When coupled with an Oculus Rift, the promise of 1990s arcade games and amusement park attractions threatens to be realized in homes around the world.

It’s not as if these devices are just popping up now, however. The Virtusphere, a giant white sphere in which users walk around, was on the market before 2010. The Novint Falcon 3D force feedback device has been available to home users for just as long.

What’s changed is the availability of millions of dollars in seed capital from Kickstarter; both the Oculus Rift and the Virtuix Omni Treadmill were started there. But there’s also the Foc.us, an independently funded device that never touched Kickstarter.

Then there’s the Foc.us, headgear that uses electrical current to (as the company claims) enhance concentration and learning. It is currently being marketed toward video game players.

The need for feedback from the computer to the human has even trickled into other consumer goods. Zac Nelson, a developer in the Ford college graduate program, has been applying data science and haptics to his stick shift.

Ford has been pushing the OpenXC platform as a method for developers to talk and interact with its cars through an open stream of information that can be fed from the on-board computer to an Android device.

Using OpenXC, Nelson and his crew created a “Haptic feedback shift knob. The idea came about last October when our research group sat down and said, ‘We’ve got this platform, and we’ve got the data. What’s the hardware extension we can design to handle the vehicle experience, particularly on a low-level vehicle?’ In most of the world, entry-level vehicles are manual transmission vehicles,” he said.

Thus, they created a simple in-car application and modification that gets the car’s RPMs from the OpenXC platform, then vibrates the stick when it’s time to upshift.

“The actual bulk of the computation is handled in the Android device, which takes pedal percentage and vehicle speed and how much the driver wants to accelerate, then goes through an algorithm to figure out what they should do,” said Nelson. “That code is straightforward: some basic Java. The firmware on the shifter itself is fairly simple as well.”

And that’s the spirit that embodies this new VR revolution more than anything else: the many-fold hackers and geeks around the world suddenly playing with combinations of hardware and software in their garages, or in their startups, or on Kickstarter.

The CastAR team, for example, is only two full-time people, for now. The eventual product, said Ellsworth, is hoped to be priced around $200, but there’s a lot of work that needs to be done just to make the device ready for production, let alone to create an entirely new interface paradigm (and to make it all fun to boot).

Said Johnson, an almost 20-year veteran of the video game industry and a founder of Raven Software: “My very first professional game project was four of us, and I was going to college. We had one artist who was a schoolteacher. Our evening hours were spent putting together this game experience. It’s what became Raven. Now it’s come full circle, from 50- to 100-man teams, back to a few people. This is unique. It’s the first time I’ve approached hardware design, and the way she [Ellsworth] approaches logic and such is completely different from how I would approach it. We have to sit and speak each other’s terms.”

And as with any new field in software, there are many late nights and long hours, said Johnson. “At Valve, we were putting in 12- to 14-hour days. We were super passionate. We’ve carried that over. Now it’s 14- to 16-hour days,” he said.

“For me, it’s one of those moments where I saw the future coming. I saw the invention of 3D [graphics] hardware, I saw the creation of massively multiplayer online games, and this, like those, was a new technology to express and experience and visualize games. It’s very exciting to be a part of.”