Implementación de un sistema para el control de dispositivos eléctricos utilizando reconocimiento de lenguaje de señas dactilológico mediante visión artificial.

Abstract

The objective of this study was to implement a control system for electrical devices through the recognition of dactylological sign language, implemented with artificial vision. The project is focused on people with hearing disabilities since it encourages inclusion, affective coexistence, human-technology interactivity and strengthening of research and development. The execution of this work was carried out through an experimental and applied research, while the knowledge acquired was systematized, practiced and implemented with the use of appropriate hardware and software. To begin with, Arduino 1.18.19 was installed to program an ESP8266 Wi-Fi module and later installed the Anaconda environment to create the programming environment in Python. Then, the model was programmed in Jupyter Notebook to carry out the gesture training, with PyCharm Community as integrated development environment (IDE) and with Python as programming language, the script was created to achieve real-time detection using computer vision and MediaPipe. The connection of the Wi-Fi module to the mosquitto broker was carried out using the MQTT communication protocol to finally connect the electrical devices: relay module, audible alarm, webcam, spotlight and electric lock to the Wi-Fi module. The results obtained present a precision rate close to 90%, which demonstrates the feasibility of using this technology as an alternative for the control of electrical devices in situations where manual control is not possible for people with hearing disabilities. It is recommended to continue with the project for continuous improvement with various practical and methodological contributions of the prototype, guiding it to the future launch on the market.