VR Ninety

At Siggraph97 (August 3-8, 1997 in LA), along with the Cave of Lascaux exhibit, Sense8 also debuted an experience base on data from NASA’s Mars Pathfinder mission. In the CAVE, visitors were able to step foot on the red planet and explore the area in vivid 3D.

I was excited to find this article in a 1999 issue of Journal of Geophysical Research.

Here is another paper from the NASA website. And one from the Fifth International Conference on Mars

I honestly don’t remember ever interacting with the researchers, but I have reached out to the people I could find on LinkedIn. I hope to hear from them and would love to share their memories/thoughts on the project. Here are their names and LinkedIn profiles:

• Carol R. Stoker
• Eric Zbinden
• Theodore T. Blackmon
• Bob Kanefsky –
• Joel Hagen
• Charles Neveu
• Daryl Rasmussen <– from the Press Release below, Daryl might have been our contact
• Kurt Schwehr
• Michael Sims

Clearly, this demo was “hot off the presses” and I recall how excited we were to be able to showcase this application/date . Prior to the landing on July 4, 1997, Sense8 released this press release (thanks to HPCWire). Siggraph was only 2 months later in August.

June 27, 1997

Mill Valley, CA -- As part of the objective of the Mars Pathfinder Mission, NASA will use simulation application software built with WorldToolKit from SENSE8 Corporation to create an interactive photo-realistic environment of Mars. When the Mars Pathfinder spacecraft lands on July 4, it will release a single vehicle microrover -- Sojourner -- equipped with a pair of stereoscopic cameras and other sensors onto the Mars surface. These instruments will allow the Sojourner to investigate the geology, surface morphology, rotational, and orbital dynamics of Mars.

  The dual cameras will take stereoscopic images of Mars and send them back to the Mission Control at NASA, where these images will be converted into 3D-Martian terrain geometry using a WorldToolKit-based application. The application will then texture-map these images onto the 3D terrain and create a virtual Martian environment. This WorldToolKit application will allow NASA scientists to interactively explore the terrain in real-time and send the Sojourner to specific areas for further investigation.

  "NASA has developed applications using WorldToolKit for several scientifc research projects in the past, including the Viking 1 mission, which is being displayed at the Smithsonian Institution's National Air & Space Museum," said Daryl Rasmussen, telepresence researcher who heads the Mars Virtual Control Center at NASA Ames Research Center. "WorldToolKit enables Mission Control scientists to become 'virtual astronauts'. The NASA developed application not only allows us to view 3D images taken from Mars, it also enables us to fully immerse ourselves into a virtual Martian environment."

  "SENSE8 is very excited to once again contribute our technology to the field of scientific research," said Tom Coull, president of SENSE8. "WorldToolKit offers developers a rich set of 3D graphics and sound capabilities, which has enabled NASA to quickly prototype and develop this mission-critical application."

Caterpillar, Inc.
Wheel Loader Design Assessment was developed through an industry/university/government partnership to help American industry advance in the use of computational science and engineering methods. This application helps Caterpillar engineers to evaluate new design parameters for wheel loaders in a safe and controlled environment. The Cave Automatic Virtual Environment (CAVE) and an operator platform with realistic machinery controls is used to simulate the actual wheel loader operation experience.
Platform(s): SGI (Onyx RE2) workstation
Peripheral Device(s): CAVE, head tracking system, operator platform with controls
Data Source(s): CAD Modeler, Pro/Engineer, vehicle simulation program

http://www.aaronjamesrogers.com/misc/hotmix16/vendors/sense8/success.html

NTT Software Corp.
Yokohama City, Japan
CyberCampus joins people by providing a common virtual space in which multiple users can meet together. Each user can move freely in the virtual space, interacting with other participants and the environment. Video and audio capabilities project the actual face and voice of each user into the 3D, virtual world. Users can participate in a “shopping mall” of retail, educational, and entertainment environments, including a virtual music store from Tower Records, and a virtual Ring Mountain from College of Marin.
Platform(s): Pentium Windows-based PCs
Peripheral Device(s): Video camera
Data Source(s): Live video, audio feeds

While researching the internet for Sense8 and WTK references, I came a cross some 1997 press coverage of our Pangea product, a networking/multi user extension to WorldToolKit. I have to say, I totally forgot that branding (I remember it as “World2World”). That is even more embarrassing when one of the articles I found is one of my few published interviews as Director of Product Marketing!

Interview:

Los Angeles, CA -- At SIGGRAPH '97, Sense8 Corporation, a provider of commercially viable 3D/Virtual Reality (3D/VR) tools and solutions, announced the upcoming fourth quarter release of a networking product named Pangea. The product, which supports Windows NT, SGI and Sun platforms will reportedly allow developers to quickly and efficiently create multi-user 3D/VR simulation applications for intranet and Internet deployment. To explore the enterprise-wide capabilities of this new product, HPCwire interviewed Tom Payne, Sense8's director of product marketing.

---

  HPCwire: Please explain the Pangea 3D multi-user server product and how it came about.

  PAYNE: "What Sense8 sells is software for doing virtual reality applications. Primarily the applications people have done with our products in that past are solitary, single user kind of things. You're immersed in a world, but you are alone. What we've had a number of customers ask for is a way to make building multi-user applications easier. We've always had a network capability in our software, but it's been low level. It's kind of intimidating to start out with, and we felt it was something that would stop customers from using it. So what we decided to do is develop a new product, a second generation product, that's a lot easier to use, a lot more aware of what the end user was going to try to do with the software and then hide as much of the networking stuff as we could from them. We also added the capability of working over the Internet. Right now our networking capability is designed primarily for doing things in the intranet world, like in the lab. It has a very high efficiency for lab uses, but if you want to go out on the Internet, the same techniques are not applicable, so we had to change the way we were working. Now we've got both, we've got Internet capability as well as the high speed local area network things you can do.

 "Basically what Pangea does is take all the information about your WorldToolKit application and literally share it with another version of that software. So you just say, I want the color and position of this object to be shared; so that all the other WorldToolKit applications in your simulation now know about the color and position of that object. If I change the color of it, all the other participants will see the color change. So it's just much easier now at the object level, where a lot of our users deal with things. For example, say I want to load in a model of that boat and I want to load in a model of that car. Many customers don't know anything about polygons, or textures or things like that. They just want to know about the boat and the car. If they want to share the boat's position, that's all they have to do instead of having to write low level packets and talk about TCBIV and routers and all those things. It's a much easier way to deal with the whole multi-user aspect of things.

  "When you're going out over the Internet you're concerned about bandwidth, so Pangea will only send information over the network when it changes. We aggregate data together so if the color and the position don't fit into the same packet, we'll tailor them so we only have to send out one packet rather than two separate ones. That way there's not excess data going over the net."

  HPCwire: Could you expand on Pangea's corporate features.

  PAYNE: "We have a lot of corporate customers and they require having firewalls set up. We're one of the only products that lets you go through those corporate firewalls in a secure way. The product is ideal for collaborative engineering activities, collaborative type experiences, Mutual Reality as people are calling it. For instance, NASA wants to use our products to do astronaut training. They'll have a guy in Houston suit up in a VR spacesuit and a guy in Florida suit up in the same thing, and they will be able to work on the space shuttle together in the same environment. We have a network manager test Computer Associates uses for their UniCenter product. They use WorldToolKit for the front end of that. We're envisioning in the future that while a systems administrator is flying around inside the 3D world, he can see other systems administrators working on other projects. It's effective in keeping them working only on the problem they need to be working on.

  "Primarily the corporate stuff is the bandwidth issue, the aggregate issue, so that it works over a number of different kinds of infrastructures like dial-in networks, a number of ATP's, the ATM network run over T1s, T3s etc. We made sure we tested at all those different configurations. We work over a standard infrastructure. Game people out there require that you dial in to their server and you have to use their technology. Our products are designed to work on any kind of infrastructure that's available, current legacy kind of systems. So it makes it a lot easier for you to integrate it into your world."

  HPCwire: Can you explain the interaction of the server with application such as WorlldToolKit or WorldUp?

  PAYNE: "Basically what you'll do is start up the application, it will log into the server and the server will track how long you've been on. That's another one of the features...in the external world, people that want to do virtual reality stuff want to be able to do things like billing, see how long people have been on, make sure only authorized users are on... the server has that kind of stuff built into it. It's able to do the acknowledgement back and forth between the applications.

  "The way that the server is set up is, it's a query/response kind of system so that if new features are added to the server, it all depends on what questions the server asks you and what your responses are. It's very easy to grow functionality to the server and maintain backwards compatibility with older clients. All of our platforms will run on the Sun, DEC, SGI and NT. I think at this point the servers will only be NT machines and then the next one will probably be Sun, because there's a lot of people out there using Sun now.

  "Once your application is up and running, you log into a server manager which will then hand off your application to a particularly suited server that will then do most of the communications. The server manager will know you are doing the engineering management application and this server over here is the one assigned to do that, it's configured properly for your application. The servers are configured by a text file that will have all the information about how many users can log in effectively, what kind of bandwidth you're going to provide, what kind of data groups you're going to provide/set up for different people. You can have your servers tuned for different applications. Once you establish communication with the server, your application will register entrance data...it'll say I want to share the color and location of this car, but I'm also interested in the color and location of all the other cars in the network. So then the server will send you color and location, and if you're also sharing orientation, but I'm not interested in that, the server will know that and it won't bother sending the orientation information. It filters that kind of stuff out.

  "Once you've established all that stuff you just start moving your objects around like it was a local application and everything else is basically handled for you. The server will transparently start changing the data, sending it back and forth between the client and the server. All the data is maintained on the server so if one of the clients goes down, it doesn't really matter because it's like a repository, it keeps all the simulations in sync. It's got a heart beat so if one of the simulations falls out, the server will know about it and notify the other participants if they've registered interest in knowing about it."

  HPCwire: Is there anything else out there comparable to this system?

  PAYNE: "The things that are out there that people are working with are primarily in the game space there's like r-time, the defense industry has their DIS (distributed interactive simulation), but that's specifically designed for military applications. Right in the protocol it talks about missile hits and explosions, so it's very focused on that application space. We're the only ones working in the business space offering general purpose simulations where custormers can do engineering etc. A lot of our customers are really excited about it, they can't wait to get there hands on it."

  HPCwire: Can you provide an example of how a large corporation might employ a multi-user 3D application?

  PAYNE: "A good example is here at a tradeshow. We have offices in Europe, the east coast and in San Francisco. We were trying to describe to people what the booth would look like, what orientation it would have, where the competitors would be etc. So what you can do very easily is build a 3D mockup of the booth, bring it up during meetings, have the people participate from Europe and the east coast, see the object, make comments interactively, they can say Let's move this thing over here. It's a lot more efficient than sending faxes.

  "If you were Webber bar-b-que and you wanted to put your latest bar-b-que up on your website, you can have a salesperson interactively show it off and have thousands of other people logged into the server watching, asking questions and interacting with the salesperson. It can increase the way you do sales on the internet as well.

-------
Steve Fisher is associate editor of HPCwire.

https://www.hpcwire.com/1997/08/15/sense8s-pangea-supports-multiplatform-3dvr-dvpmnt/

https://techmonitor.ai/technology/sense8_has_3dvr_multi_user_development_tool_for_businesses_1

Pangea was clearly another Sense8 technology that was ahead of its time (1997) with use cases only now (2022). I am sure a lot of people worked on it, but in my mind it will always be Arvind Suthar’s baby.