VR Co-Lab – A Virtual Reality Platform for Human-Robot Disassembly Training and Synthetic Data Generation

Case Study: VR Co-Lab – A Virtual Reality Platform for Human-Robot Disassembly Training and Synthetic Data Generation

1. Introduction

• Title: VR Co-Lab – A Virtual Reality Platform for Human-Robot Disassembly Training and Synthetic Data Generation

• Role: Developer and Researcher

• Year: 2024

2. Project Overview

• Objective:

  • The primary goal of VR Co-Lab was to develop a comprehensive VR training platform to enhance human-robot collaboration in industrial disassembly tasks, focusing on e-waste recycling. This platform integrates advanced body tracking via the Quest Pro headset and generates synthetic data during training sessions to improve and evaluate robot path planning models.

• Technologies Used:

  • Unity game engine, ROS (Robot Operating System), Quest Pro headset, Docker, Unity ROS TCP Connector, Unity URDF Importer.

• Collaborators:

  • Beiwen Li (Major Professor), Yongyeon Cho, and the faculty and staff of VRAC.

3. My Approach: Crafting Digital Excellence

Vision and Innovation:

• My vision for this project was to create an immersive and interactive VR training environment that enhances the collaboration between humans and robots in disassembly tasks. By leveraging advanced VR technologies and synthetic data generation, I aimed to improve training efficiency and robot path planning models.

Identifying Unique Challenges:

• Before starting the project, I identified several unique challenges, including integrating the Quest Pro headset for precise body tracking, developing realistic VR interactions, and generating synthetic data for continuous system improvement.

Resolving Complex Problems:

• One of the primary challenges was integrating the ROS communication model with the Unity environment. To address this, I developed a robust system architecture that ensured seamless data exchange between the VR interface and the physical robot. Additionally, creating realistic and interactive VR training scenarios required careful design and iteration.

User-Centric Design:

• The cornerstone of my approach was ensuring that the VR training system was user-centric, focusing on the needs and feedback of the trainees. This involved designing intuitive interactions, providing real-time feedback, and continuously testing and refining the system based on user feedback.

Meeting User Needs:

• The design of VR Co-Lab aimed to meet the needs of users seeking realistic and effective training in disassembly tasks. By focusing on user needs, we created a training platform that was not only practical and functional but also engaging and easy to navigate.

Every aspect of this project is a testament to my systematic, innovative, and problem-solving approach. The VR training platform reflects my vision for digital excellence and my commitment to delivering results that surpass expectations. Explore this project in my portfolio to witness how I translated my approach into an impactful, user-centered VR solution.

4. Project Details

System Architecture:

• Description:

  • The system architecture comprises three main components: the VR environment, the robotic control system, and the feedback and data collection module. The VR environment was developed in Unity, replicating a disassembly workstation with realistic tools and components. The robotic control system, managed by ROS, included multiple ROS nodes for data exchange and control commands. The feedback and data collection module integrated performance monitoring and synthetic data generation.

• Visuals:

  • Include screenshots or diagrams of the system architecture.

  • Example:

    
    

VR Environment:

• Description:

  • The VR environment was designed to simulate a disassembly workstation with lifelike tools and machinery components. Users could interact with the tools and components, receiving immediate feedback on their actions. The environment was developed using the Unity engine, with detailed simulations of the disassembly tasks.

• Visuals:

  • Show images or clips of the VR environment and user interactions.

  • Example:

    
    

Integration with ROS:

• Description:

  • The integration with ROS facilitated communication between the VR environment and the physical robot. Using the Unity ROS TCP Connector and Unity URDF Importer, we ensured accurate simulation and control of the robot within the VR environment.

• Visuals:

  • Provide examples of the ROS integration and communication model.

  • Example:

    
    

Performance Monitoring and Synthetic Data Generation:

• Description:

  • The system collected performance metrics such as task completion times, error rates, and user engagement levels. This data was used to generate synthetic datasets for improving and evaluating robot path planning models, ensuring continuous system optimization.

• Visuals:

  • Include graphs or charts showing performance metrics and synthetic data usage.

  • Example:

    
    

5. Responsibilities

• Development:

  • As a developer, I was responsible for integrating the ROS communication model with the Unity environment, developing the VR training scenarios, and ensuring seamless interaction between the user and the robot.

• Research and Analysis:

  • I conducted research on synthetic data generation and its application in improving robot path planning models. This involved analyzing performance data and using it to refine the training system.

6. Project Highlights

• Immersive Training Environment:

  • The project successfully created a realistic and immersive VR training environment that enhanced human-robot collaboration in disassembly tasks.

• Advanced Body Tracking:

  • By integrating the Quest Pro headset, we achieved precise body tracking, allowing for natural and ergonomic user interactions.

• Synthetic Data Generation:

  • The system generated synthetic data during training sessions, which was instrumental in improving robot path planning models and ensuring continuous system optimization.

7. Conclusion

• Results:

  • VR Co-Lab demonstrated the potential of VR training systems in enhancing human-robot collaboration. The project highlighted the importance of immersive environments and synthetic data in improving training efficiency and robot performance.

• Learnings:

  • Through this project, I gained valuable insights into integrating VR and ROS technologies, developing user-centric training systems, and leveraging synthetic data for continuous improvement.

• Future Enhancements:

  • Potential future enhancements include integrating eye tracking, expanding the range of disassembly tasks, and developing multi-user training scenarios to enrich the learning experience.