Desarrollo de un sistema de reconocimiento para frases básicas del lenguaje de señas ecuatoriano utilizando visión por computadora

Abstract

The main objective of this project is to develop an algorithm assisted by computer vision to recognize and translate Ecuadorian Sign Language (LSEC) into text in real time. This work arises from the need to support the more than 10 million people in Ecuador who live with hearing or speech disabilities. It is based on experiences observed in schools across the country, where teachers with mutism and students with hearing disabilities lack tools to communicate, aiming to optimize interaction and reduce learning barriers in the classrooms. For the capture of movements, the system uses the webcam to extract three-dimensional coordinates of the hands, face, and posture. For the training of the neural network, the LSTM (Long Short-Term Memory) architecture was used as the main tool; this technology excels in sequence recognition, as it helps process the movement of signs over time, allowing the system to identify complete sentences smoothly. The main components used were a computer with an internet connection, an HD camera, and an Arduino Uno for interface control. To run the code, the program captures sequences of 60 frames per gesture. Once the algorithm is started, if the camera does not detect a limb due to lack of light or sudden movement, the system automatically fills those gaps with zero arrays; this prevents the program from stopping due to missing information and ensures that the model receives consistent data for translation. Finally, the interface immediately displays on the screen the text of the recognized sign. During testing, the system achieved an accuracy of 96% in environments with artificial light and an overall average of 81.3%, confirming that it is a technically viable and cost-effective solution to improve users' communication in their daily lives.