Blurring the line between virtual, real

Published in the News-Gazette online Sep 10, 2007

By Greg Kline

Swiss and British scientists recently reported creating an out-of-body experience for participants in studies, using cameras and video-display goggles to give the subjects a view of their own backs.

People even exhibited a physical response – they sweated nervously – when their virtual selves were threatened with a hammer.

"They cannot really distinguish any more between their own body and the virtual image," University of Illinois Professor Alfred Hubler said.

Similarly, some studies show that players who spend too much time on games like SimCity sometimes have a hard time remembering whether they saw something in the game or in real life, said Hubler, a UI physics professor.

Such "mixed reality" states may be more than just tricks of the mind, however. Hubler and UI colleagues are the first to show that the principles involved may apply to physical systems as well as cognitive, in this case a swinging pendulum.

Hubler at the UI Center for Complex Systems Research and graduate student Vadas Gintautas were able to get a virtual pendulum in a computer, one moving under time-tested equations of motion, and a real one outside swinging not just in sync but in concert.

When the pendulums – joined by hardware and software that had them almost instantaneously providing feedback on their states to each other – functioned out of concert, they eventually halted because of friction, real and programmed into the computer simulation.

"Both states die out," Hubler said.

But when he and Gintautas adjusted the swing frequencies to a point where the pendulums functioned as one, in a coherent state of mixed reality unison, they helped each other, in effect, defy the friction forces and they kept swinging.

"They just do it automatically," Hubler said. "It's surprising how sudden that happens. It comes with no warning."

Hubler said the key is to have instantaneous, bi-directional feedback, and that the systems being joined are sufficiently similar.

The work appeared in the journal Physical Review E of the American Physical Society and was highlighted in a society press release.

Hubler's lab looks something like Edison's workshop must have, filled with experiments involving everything from a wind tunnel to samples of cancer cells.

He studies complex systems – your car, traffic at rush hour, factories, the Internet, the weather are examples of those – with an eye to understanding, predicting and, where possible, controlling their behavior.

Beyond being an interesting experiment, techniques resulting from the mixed-reality research could have practical applications.

Faced with a real system possessing unknown qualities, for example, you might set up a virtual system, link the two and make adjustments on the computer until they shift from a dual to a mixed reality.

Since the two have to be similar to do so, the virtual system can serve as a probe or model of the real system's qualities.

Likewise, since the two can have an influence on each other, you might use the virtual system as a control mechanism for the real one.

"You change a virtual system and the real system follows to some extent," Hubler said.

Then there's time travel.

It's possible for the virtual system to leap into the future and back. The question is how that might affect the mixed-reality state and the real-world system. Hubler thinks it could strengthen the relationship.

The notion isn't as odd as it may sound. We often time travel – in our minds – and mix it with reality, driving to Chicago, for instance, and thinking ahead to the construction we will have to navigate and the exits we will need to take to reach our destinations.

"We anticipate what will happen and we are more in tune with the world," Hubler said. "We experience fewer emergencies, or fewer disconnects."