top of page
Smart Glove-to-Robot Control System | Capstone Project
The Ask
As part of my senior capstone project in Computer Science, I worked on a multidisciplinary team to build a system that enabled a Rokoko smart glove to control a physical robot using real-time hand motion data. The goal was to translate human hand gestures into robotic movement using a relay server as the communication bridge between the glove and the robot.
The project required real-time, low-latency communication between a human-controlled input device (Rokoko Smartglove) and a robot. Our team’s objective was to process motion data from the glove, relay it through a server, and send it to the robot in a format it could understand—all while maintaining smooth and accurate motion response.
My Role
As a software engineer on the smart glove integration team, I was responsible for connecting the Rokoko Smartglove SDK to our local development environment and extracting real-time motion data, such as finger flexion and hand rotation. I built a relay interface to transmit this data to the robot control team and collaborated on designing the communication protocol between the glove, relay, and robot. Additionally, I played a key role in testing and troubleshooting to ensure connection stability, low latency, and consistent data transmission throughout the system.
The Process
1.
Integrated Rokoko’s SDK using Python to capture real-time motion inputs.
2.
Built middleware to serialize and send glove data to a custom relay server via WebSocket.
3.
Worked closely with the robot team to ensure data format compatibility and timing synchronization.
4.
Performed iterative testing with the physical robot to calibrate gesture-to-movement mappings.
5.
Documented setup procedures for the next capstone group to work on glove input data.
The Outcome
Successfully enabled live, gesture-based robot control using the Rokoko Smartglove. Reduced motion lag to under 150ms through optimized data relaying. I created a scalable and modular system that could be expanded for future gesture-controlled robotics projects. Delivered a live demonstration and technical documentation for the university’s capstone showcase
Project Reflection
This project sharpened my skills in real-time systems, device communication, and cross-functional collaboration. It also deepened my understanding of motion capture tech and how to bridge hardware inputs with physical robotics through software.
Let’s Build Something Brilliant.
bottom of page