Angles Of View

Dr. James L. Davis is the President of GRADIENT Inc., a firm specializing in the design and creation of visual display systems for the simulation, training, and entertainment industries. Subsequent to receiving his doctorate in Electrical Engineering from MIT, Dr. Davis spent fourteen years in the simulation industry, working for Rediffusion Simulation and IVEX. He is a frequent lecturer on business and technology issues relating to entertainment and training. His Internet address is 74673.3342@compuserve.com. Dr. Davis is interviewed here on the subject of

Virtual Reality - How Real Is It?

Da-Lite: There’s a great deal of talk about virtual reality these days, but it’s not always easy to understand just what they’re talking about. What is virtual reality, anyway?

Davis: It depends on whom you ask. There is an accepted definition that virtual reality is a simulation of experiences recognizable by the human perceptual system (touch, taste, sight, hearing, and smell) which do not originate in the real world. You can envision many applications for this sort of thing. Historically, one of the first was teleoperation.

Imagine a robotic manipulator of some sort that’s far away, or maybe under the ocean, working on the base of an oil rig. But a human operator still needs to control it and there’s no economically feasible way to get a human that far down. So the concept is that you put in a couple of TV cameras and enough lights to illuminate everything and gear it all to a servo operated manipulator. However, you have to give the human operator enough information so that he can perform the task. So what you do is you provide him with a virtual environment which, in the simplest case, is just a TV monitor which portrays what the remote manipulator is seeing. And then you give him a control with tactile feedback so that in principle he could move it forward, clamp down on something, twist it or turn it. This is a very simplistic form of virtual reality.

Da-Lite: It gets a good bit more elaborate, doesn’t it?

Davis: Yes. With the help of science fiction novels and such people can get carried away. They want to do more than simple tasks. They want to amble around, they want to smell the flowers, they want to see the birds flying in the sky, feel the wind in their faces, and hear the brook gurgling in the background. Very quickly they find out how limiting and limited technology can be unless they have infinitely deep pockets.

I personally believe that flight simulators are the best examples of virtual reality you can encounter in the world today. Think about it. Here you have a flight simulator that contains a virtual environment using three million dollars worth of computer image generators and displays. You compliment that with this ten million dollar device that moves and tilts exactly like a real cockpit would and you’ve got a pretty good virtual experience. You’ve got tactile cuing, motion cuing; you’ve got twelve to eighteen speakers to provide aural cuing. And you’ve got visual cuing over some field-of-view that’s commensurate with the vehicle, typically. You don’t have taste, but taste isn’t part of the experience being sought anyway.

Da-Lite: Presumably the pilot could take a real cup of coffee in with him, couldn’t he?

Davis: Certainly. And I’m sure it would taste better than what they serve you on airlines. But the thing to notice is that a flight simulator is a device which costs about fifteen million dollars. The problem with virtual reality today is that people in the field are trying to be much more ambitious in terms of what they’re trying to undertake for a whole lot less money.

Da-Lite: Why must it be so expensive?

Davis: Because it’s very expensive to fool the human sensory apparatus. People train it from the day they’re born. So when you get a hold of a person who’s twenty years old , his visual sense is sharp. He knows what’s right and what looks wrong. And if you screw up in terms of a simulation, he’ll pick it up instantaneously.

And with flight simulators, even after spending millions of dollars, it’s still not perfect. But it turns out that for training psycho-motor skills and such it doesn’t have to be perfect. When you go to school, for instance, it isn’t a perfect simulation of the real world, yet through certain techniques they can teach you a great deal about the real world. In terms of training and teaching virtual reality is just another tool. And it’s better for some things than others.

Flight simulators work because their manufacturers don’t try to do too much, they haven’t been adamant about being perfectly virtual. They include actual aircraft parts so the touchy-feely stuff is very good. But notice: the other thing they’ve done is narrow the application so that they only bite off what they can chew.

The cockpit in which the pilot sits is really a buffer between him and reality. The application is not free motion through an infinite world. So you don’t have to simulate everything, you only have to simulate the sorts of interactions that you get in an aircraft.

Da-Lite: That means you don’t have to apply pressure to the pilot’s feet or put sensors on his fingers, doesn’t it?

Davis: Exactly. Your subject is enclosed in a rigid structure and all you have to do is buffet and tumble it in some aerodynamically correct way. Then, when the visual software displays an accurate rendition of, say, a runway at O’Hare airport you’ve got an overall setup that’s really very successful.

One of the reasons this aspect of virtual reality has been such a success is that flight simulators have been able to build off technologies that have largely been subsidized and driven by consumer products. The reason the visual displays in simulators are as good as they are is that there’s an entire world out there that needs 25-inch color CRTs for their TV sets. People need 9-inch projection tubes for their in-home theaters and the numerous corporate applications.

And you can now get screens from a company like Da-Lite that are really quite good; they have the right sort of gain profiles and such, and, best of all, they’re available in large enough sizes.

Da-Lite: And the reason you need big screens?

Davis: An important project in the virtual reality field is called the Cave.” This is a cubical chamber with a front projection screen for a floor and rear projection screens comprising three of its walls. In order to accommodate a human subject, those screens have obviously got to be big.

I think the key to success for virtual reality in the coming years will be to leverage technologies that are being developed for other, perhaps entirely unrelated applications. Because, finally, the challenge for virtual reality is not just technology; it’s cost reduction as well.

Da-Lite: Thus far you’ve been talking exclusively about large scale virtual reality systems. Isn’t there a whole other delivery system which we might describe as helmets?

Davis: Ah yes; the head mounted displays. There are two principle types: the see-through version and the totally immersive version. The former permits you to see through the imagery projected onto your display. Thus you can see the world around you. The immersive type is totally opaque to the outside world and thus is the configuration favored by the virtual reality purists.

At the upper end of this technology you find CRT driven head mounted displays which are either monochrome or are color via field sequential techniques. And at the lower end you find LCD driven head mounted displays. Now the trouble with these is that their resolution is limited. If you want color you’re usually limited to only about 60,000 pixels per eye.

Da-Lite: How many pixels would be enough?

Davis: It depends on how ambitious you are for what’s called instantaneous field-of-view.

Da-Lite: Does that mean that when you turn your head 30^o , your visual field shifts accordingly?

Davis: No; that’s field-of-regard. If you can turn your head around and see behind you, you’ve got a 360^o field-of-regard. instantaneous field-of-view is: if you hold your head steady, how wide are the blinders?

There have been human factors studies done which demonstrate that your sense of immersion in a virtual environment is really helped by the presence of peripheral vision. Thus the larger the instantaneous field-of-view, the more realistic it is. Ideally, then, you’d like to create a virtual reality visual system which provides about 170^o horizontal field-of-view by about 110^o vertically.

Let’s calculate how many pixels I would need to give to each eye if I want to have 20-20 vision over this large instantaneous field-of-view. To have 20-20 vision, you need 1 arc-minute resolution which means you need 60 pixels per degree (1 pixel/arc-minute). Multiply that out and you see that for each eye you need 67,000,000 pixels. Well, gee; that’s very difficult to do.

High-end computer image generators are happiest when they are working with about 1.2 million pixels per channel - max. So you see, they’re not even in the ballpark.

Da-Lite: What’s the compromise?

Davis: The best I’ve ever seen restricted the instantaneous field-of-view to about 12 by 16 degrees. Yes, it was tunnel vision, but the resolution was adequate.

The conundrum for the people who make helmets, then, is that they need to give you large fields-of-view yet they can’t afford to give the resolution necessary to make their virtual reality seem real. And it’s well nigh impossible technically to apply that amount of resolution with a head mounted display. You just can’t get that much equipment onto the head.

Da-Lite: And if you take the helmet off?

Davis: Then you’re back to the flight simulator visual system or the Cave” concept. Either one creates a stationary projection system which is much more like the real world. If you want to see a different part of the image, you turn your head and, sure enough, you’re seeing a different part of the visual field. Mind you, the available resolution in a the cave isn’t all that good. You couldn’t read a page of text, for example. But if your purpose is to interact with objects whose scale is sufficiently large, the outcome can be quite satisfactory.

Da-Lite: How, finally, can we measure the effectiveness of virtual reality?

Davis: I once saw a three axis graph which, depending how far out you were on each axis combined to measure the degree of achieved virtual reality. One of the axes was the feeling of immersion. Another was the degree of interactivity. And the third parameter was the level of realism.

Each scale goes from zero to one. To concoct just one example, you could put yourself into some sort of diorama which might rate a 1 on the realism scale but would get a 0 on the interactivity scale and likely only a very small number on the immersion scale. Most people would say that’s not virtual reality. I would argue that all three values have to be non-zero if genuine virtual reality is to be achieved.

And, yes, there are a number of focused applications which, given certain types of chambers, under the right conditions virtual reality can be fully actualized and the results are swell. But for any reasonable amount of money, those conditions have to be carefully limited. Night scenes are easier than trying to simulate broad daylight. Narrow fields-of-view are easier than wide ones. And so forth.

Virtual reality has a long way to go before it can truly fool us. Because when it comes to actual reality, we are, all of us, experts.