Virtual Reality Technology

phones as input devices Smart phone use in HMDs 3.4.2 Head-Mounted Displays Fixed-resolution HMD optics Foveated HMD optics Foveated HMDs which reduce native resolution Foveated HMDs which increase native resolution Head-Mounted Displays weight and weight distribution The FOVE 0 HMD The Oculus Quest 2 HMD The HTC VIVE Focus 3 The Pimax Vision 8K X HMD 3.4.3 Desk Supported Personal Displays Autostereoscopic monitors 3D Monitors 3.5 LARGE VOLUME DISPLAYS 3.5.1 Liquid Crystal Tiled Large-Volume Displays Tiled wall displays Tiled CAVE displays 3.5.2 Projector-based Large-Volume Displays Dome-type large volume displays Tiled Display Scene Continuity Geometrical continuity Visual continuity Temporal continuity 3.6 MICRO-LED WALLS AND HOLOGRAPHIC DISPLAYS 3.6.1 Micro-LED Walls 3.6.2 Holographic displays 3.7 CONCLUSIONS 3.8 REVIEW QUESTIONS REFERENCES 4. Output Devices: Three-Dimensional Sound, Haptic Displays, Olfactory Displays 4.1 THREE-DIMENSIONAL SOUND DISPLAYS 4.1.1 Localized vs. Non-Localized sound 4.1.2 The Human Auditory System Azimuth Cues Elevation Cues Range cues 4.1.3 Head-Related Transfer Function 4.1.4 Generic Convolvotron Architecture Spatialized sound for all-in-one HMDs 4.1.5 Speaker-based 3-D Sound 4.1.6 Wearable Sound Interfaces 4.2 HAPTIC DISPLAYS 4.2.1 The Human Haptic System Human Haptic Sensing System Human Sensory-Motor Control 4.2.2 Tactile Feedback Interfaces The CyberTouch II glove The HaptX DK2 touch feedback glove The Tactsuit X whole body tactile feedback Temperature Feedback Interfaces 4.2.3 Force Feedback Interfaces The Touch X Arm The Delta.3 Force Feedback Arm The Armeo Power SenseGlove Nova Force Feedback Glove 4.3 OLFACTORY DISPLAYS 4.3.1 The Human Olfactory System 4.3.2 Olfactory Systems for VR Ambient Olfactory Displays Air Cannon-based Olfactory Display Wearable Olfactory Displays 4.4 CONCLUSIONS 4.5 REVIEW QUESTIONS REFERENCES 5. Computing Architectures for VR 5.1 THE RENDERING PIPELINE 5.1.1 The Graphics Rendering Pipeline Traditional Graphics Rendering Pipeline Modern Graphics Rendering Pipeline A graphics pipeline example Anti-aliasing Graphics Pipeline Bottlenecks Graphics Pipeline Optimization 5.1.2 The Haptic Rendering Pipeline 5.2 GAMING DESKTOP ARCHITECTURES 5.2.1 The 12th Generation Core CPU and Chip Set 5.2.2 Cooling of Gaming PCs 5.2.3 Graphics Accelerator Cards for PCs The NVIDIA GTX 1080 Founder's Edition graphics card NVIDIA GeForce RTX 4090 graphics card The NVIDIA GeForce RTX 4090 graphics card 5.3 GRAPHICS BENCHMARKS 5.3.2 The SPECviewperf 2020 benchmarks 5.3.3 A Frame Capture Analysis Tool for Virtual Reality 5.4 DISTRIBUTED VR ARCHITECTURES 5.4.1 Split Rendering 5.4.2 Co-located Rendering Pipelines 5.4.3 Multi-pipeline synchronization Synchronization of haptic and graphics pipelines Synchronization of co-located graphics pipelines Pipeline synchronization using the NVIDIA Quadro Sync II card 5.4.4 Cloud Rendering A Cloud Rendering Architecture Example Cloud Rendering Latency 5.5 CONCLUSIONS 5.6 REVIEW QUESTIONS REFERENCES 6. Modeling Virtual Environments 6.1 GEOMETRIC MODELING 6.1.2 Virtual Object Shape Using a 3D Authoring Software Importing CAD files Creating surfaces with 3D scanners Using online 3D object databases 6.1.3 Object Visual Appearance Scene Illumination Texture Mapping 6.2 KINEMATICS MODELING 6.2.1 Homogeneous transformation matrices 6.2.2 Object Position 6.2.3 Transformation Invariants 6.2.4 Object hierarchies 6.2.5 Viewing the 3-D World 6.3 PHYSICAL MODELING 6.3.1 Collision Detection Collision detection for large virtual environments 6.3.2 Collision Response Involving Object Surfaces Topology-preserving collision response Topology-altering collision response Surface cutting and stitching Object morphing 6.3.3 Contact Force Modeling Contact Forces when Deforming Elastic Objects Contact Forces when Deforming Plastic Objects Contact Forces when Interacting with Virtual Walls 6.3.4 Force Smoothing and Mapping 6.3.5 Haptic Texturing Haptic textures produced by non-wearable interfaces Haptic textures produced by wearable interfaces 6.4 BEHAVIOR MODELING 6.4.1 Simple behavior models 6.4.2 Enhanced behavior models 6.4.4 Crowd behavior models 6.5 MODEL MANAGEMENT 6.5.1 Level-of-Detail Management Discrete Level-of-Detail Management Continuous Level-of-Detail Management Adaptive Level-of-Detail Management Using Foveated Rendering Adaptive Level-of-Detail Management Guaranteeing Frame Time 6.5.2 Cell Segmentation Automatic Cell Segmentation 3-D Cell Segmentation 6.6 CONCLUSION 6.7 REVIEW QUESTIONS REFERENCES 7. Virtual Reality Programming 7.1 SCENE GRAPHS AND TOOLKITS 7.1.1 Scene Graphs Internal scene graphs Distributed scene graphs 7.1.2 Toolkits 7.2 JAVA3D 7.2.1 Java 3D Model Geometry and Appearance 7.2.2 Java3D Scene Graph 7.2.3 Java3D Sensors and Behaviors 7.2.4 Java3D Networking 7.3 THE VIZARD TOOLKIT 7.3.1 Vizard Model Geometry and Appearance Model Geometry Model appearance 7.3.2 Vizard Scene Graph 7.3.3 Vizard Sensors and Behaviors Vizard Physics Engine Vizard OpenHaptics Plug-in 7.3.4 Vizard Networking 7.4 THE OPENHAPTICS TOOLKIT 7.4.1 OpenHaptics Integration with the Graphics Pipeline 7.4.2 OpenHaptics QuickHaptics Micro API 7.4.3 OpenHaptics Haptic Device to Screen Mapping 7.4.5 OpenHaptics Unity Plugin 7.5 UNITY 3D GAME ENGINE 7.5.1 The Game Engine 7.5.2 Game Production Pipeline The pre-production pipeline stage The production pipeline stage The post-production pipeline stage 7.5.3 Unity Game Programming Creating a New Project in Unity The Unity Editor Unity Game Objects Physics Programming in Unity Scripting in Unity 7.5.4 Artificial Intelligence in Unity Gaming Non-Player Characters, Finite State Machines and Machine Learning Unity Implementation 7.6 CONCLUSION 7.7 REVIEW QUESTIONS REFERENCES 8. Human Factors in Virtual Reality 8.1 METHODOLOGY AND TECHNOLOGY 8.1.1 The Experimental Protocol 8.1.2 Institutional Review and Participant Consent 8.1.2 Data Collection and Analysis Objective performance measures Modality-specific objective performance measures Task-specific objective performance measures Subjective performance criteria 8.1.3 Usability Engineering Methodology 8.2 USER PERFORMANCE STUDIES 8.2.1 Testbed Evaluation of Universal VR Tasks Influence of navigation and text display techniques on user performance Influence of locomotion techniques on user performance 8.2.3 User Performance Dependency on Feedback Modality User performance during object manipulation with substituted and redundant haptic feedback Participant sense of presence during tasks involving sensorial substitution Olfactory and haptic feedback contribution to participant quality of experience in VR 8.2.4 Sensorial Illusion Illusion of weight Illusion of Scent 8.3 HEALTH AND SAFETY ISSUES IN VIRTUAL ENVIRONMENTS 8.3.1 Direct Effects of Virtual Environments on Users Direct effects on the Visual System Direct Effects on the Auditory, Musculoskeletal and Olfactory Systems 8.3.2 Indirect Effects of Virtual Environments on Users Neural conflict between inner ear, visual and proprioceptive sensorial cues Ways to measure cybersickness severity Influence of user's characteristics Influence of VR system characteristics Temporal factors influencing cybersickness Adaptation and Aftereffects Guidelines for proper VR usage 8.4 SOCIETAL IMPLICATIONS OF VIRTUAL REALITY 8.4.1 Virtual Reality Impact on Professional Life 8.4.2 Virtual Reality Impact on Private Life 8.5 CONCLUSION 8.6 REVIEW QUESTIONS REFERENCES 9. Applications of Virtual Reality 9.1 MEDICAL APPLICATIONS OF VIRTUAL REALITY 9.1.1 Medical Education Anatomy trainers using Virtual Reality Surgical Skill Acquisition when Using VR Trainers 9.1.2 Virtual Rehabilitation Virtual Rehabilitation Game Design Principles Virtual Rehabilitation for Management of Chronic Pain Virtual Rehabilitation for Cognitive Impairments Autism Spectrum Disorder Dementias Remote Therapeutic Monitoring Using Home Virtual Rehabilitation Chronic stroke Cerebral Palsy 9.2 VIRTUAL REALITY IN EDUCATION, ARTS AND ENTERTAINMENT 9.2.1 Virtual Reality in Education Constructionist Learning in Virtual Reality Constructionist 3D Dynamic Geometry The Plasma Engineering Playground Constructivist Learning in Virtual Reality 9.2.2 Virtual Reality and the Arts The Virtual Florentine Pietà The House of Greek Epigrams in ancient Pompeii Visitor fixation vs. feedback modality - The Honk Kong Heritage Olfactory Heritage 9.2.3 Entertainment applications of Virtual Reality Gaming on Personal Computers Location-based Virtual Reality Entertainment Virtual Reality Arcades Virtual Reality Lounges Cloud Gaming General Cloud Gaming Virtual Reality Cloud Gaming 9.3 MILITARY VIRTUAL REALITY APPLICATIONS 9.3.1 Army Use of Virtual Reality Small Arms Tactical Trainers Armored Personnel Carrier Driver Trainer 9.3.2 Air Force Use of Virtual Reality Aircraft Deployable Tactical Trainers Mixed-Reality Flight Simulators Transfer of Training from VR and MR Flight Simulators 9.3.3 Virtual Reality Applications in the Navy Naval Ship and Bridge Virtual Reality Simulator Shipboard Firefighting Trainer 9.4 CONCLUSIONS 9.5 REVIEW QUESTIONS REFERENCES Index Laboratory Manual (online on book site)