This project set out to build a complete pipeline from a biological signal to a robotic movement: Reading the electrical activity in the forearm muscles and using it to open and close the gripper on a UFACTORY xArm. The idea is simple to describe but involves several steps along the way, from electrodes on the skin to machine learning and robot control.
Surface EMG electrodes are placed on the forearm, two along the muscle and one on the elbow as ground. The signal is filtered and amplified by an Olimex EMG shield and sampled at 250 Hz by an Olimex 328 microcontroller, which streams the data to a computer over USB. The computer classifies the signal and sends the predicted gesture to the xArm over Ethernet using the xArm Python SDK. Three gestures were implemented: opening the fist, closing the fist and rest.
A large part of the work went into preparing the data. Each recording was split into active and resting parts using two thresholds with hysteresis, which prevents the label from flickering when the signal is around a boundary. The signal was then divided into short, overlapping windows of 200 ms, and seven features describing amplitude, variability and frequency content were computed for each window. Amplitude features are good at separating rest from movement, but they struggle to tell opening from closing, since both gestures can involve a similar amount of muscle force. Frequency features, which look weak on their own, turned out to be the key to separating these two gestures. To confirm this before training the final model, the features were tested on how well they could tell opening from closing alone, using recordings left out of the analysis, where 50% equals guessing. Amplitude features alone gave 62% accuracy and frequency features alone 64%, while all seven features together reached 91%.
The classifier is a neural network built in PyTorch. One challenge was that around 86% of the data was rest, meaning a model that always guessed rest would already score almost 86% accuracy. To prevent this, the training batches were balanced between the three classes and mistakes on the active gestures were weighted more heavily. The data was also split by recording rather than randomly, so that nearly identical overlapping windows could not end up in both the training and the test set. On recordings the model had never seen, it reached 95.3% accuracy and a balanced accuracy of 93.7%. Opening and rest were recognised correctly 96% of the time and closing 90%, and a real gesture was never mistaken for rest.
The system was also tested live, with a person performing the gestures while the gripper responded in real time. Here the choice of threshold proved critical. Set too high, real movements were read as rest and the arm barely reacted. Set too low, the gripper kept opening and closing on its own. The best result came from placing the threshold just above the level the signal stays within at rest. The gripper then followed the user’s movements with a slight delay.
A meaningful application for this technology is assistive robotics for the elderly and people with reduced motor function, for example from Parkinson’s disease. Since the system reads the intention in the muscle signal rather than the physical grip, it could provide a stable grip even when the hand itself is weak or trembling. The main limitation today is that the model is trained on data from one person. Collecting recordings from several individuals would be the natural next step, so the system could work for new users without hours of personal training data.
Working on this project has given us valuable experience in data management, machine learning and connecting a trained model to a robot. We learned how to structure and label EMG recordings so the data could actually be used, and how to find the features that tell similar gestures apart, rather than relying on signal strength alone. Most importantly, we learned how much of a good machine learning result depends on understanding and preparing the data before any model is trained.
By Hilde Celine Løvland and Sunniva Tomren

