Vision based Telerobot Arm with Digital Twin

A multidisciplinary project combining Robotics, IoT, Digital Twin Technology, Embedded Systems, and Software.

Features

• Digital Twin: 3D Unity Replica.
• Real-Time Vision Feedback: Camera integration for precise monitoring.
• Telerobotics: Operate the robot arm remotely.
• Hand Tracking Control: Leverages OpenCV and MediaPipe for intuitive hand gesture control.
• Smooth Servo Control: Uses MQTT and NodeMCU for seamless motor operation.

Technologies

• Python: For hand tracking and vision feedback.
• C++: NodeMCU program.
• NodeMCU: MQTT-based control of servo motors.
• OpenCV & MediaPipe: Hand gesture recognition.
• Unity: Visualization of real-time robot arm movements.
• Websocket : For main server client communication.
• MQTT Communication: Publishes control commands and receives feedback to main server.

Hardware

• Robot Arm: 6-DOF robotic arm.
• Camera: Laptop Camera.
• NodeMCU: Microcontroller for servo control.
• Servo Motors: Servo motors for arm movement.

Architectures

Hardware Architecture

Network Architecture

Physical Robot

Digital Twin