Mixed Reality Engineering Labs

Overview

This project focused on enabling hands-on learning for complex mechanical systems, including jet engines, using mixed reality and AI assistance. Students can explore intricate engineering systems with LLM-powered contextual explanations.

The Challenge

Engineering institutions face significant barriers in providing hands-on training with complex equipment. Systems like jet engines are expensive, dangerous, and difficult to access. The solution needed to:

Objectives

• Build interactive MR models of engineering systems
• Integrate LLM-powered tutor for contextual explanations
• Enable intuitive exploration of components and workflows
• Provide practical understanding of otherwise inaccessible systems
• Support self-paced learning with AI assistance

Our Approach

We created high-fidelity MR simulations using Meta Quest with passthrough capabilities, allowing students to interact with complex engineering systems in their real environment. An AI tutor powered by OpenAI and Gemini provides real-time contextual explanations as students explore components and workflows.

Solution Delivered

• Interactive MR Models -High-fidelity simulations of engineering systems including jet engines.
• LLM-Powered AI Tutor -Contextual explanations powered by OpenAI and Gemini for on-demand learning support.
• Passthrough Mixed Reality -Real-world environment integration using Meta Quest passthrough.
• Component Exploration -Intuitive interaction with individual parts and workflows.
• Self-Paced Learning -Students explore at their own pace with AI guidance.

Technologies Used

Meta Quest

Meta Passthrough

Unity

OpenAI

Gemini