Smart Glove for Sign Language

Abstract

Sign language is a primary means of communication for individuals with hearing and speech impairments however communication barriers still exist between sign language users and the general public. This project presents the design and implementation of an AI-based smart glove for real time sign language recognition. The proposed project utilizes a wearable gloves equipped with flex sensors and inertial measurement units “IMU” to capture finger bending and hand motion data. Sensor readings are processed using an embedded ESP32 microcontroller, where a lightweight TinyML model performs gesture classification locally without the need for internet connectivity. A supervised machine learning approach is adopted, in which sensors data is collected, preprocessed, and used to train a neural network capable of recognizing predefined sign language gestures. The trained model is converted to a TensorFlow Lite format and deployed on the ESP32 for efficient real time inference. Recognized gestures are transmitted wirelessly to a mobile application via Bluetooth Low Energy “BLE”, where they are displayed as text and converted into speech. An additional application level processing layer is introduced to arrange recognized words into meaningful sentences, enhancing communication effectiveness. The proposed system demonstrates the feasibility of integrating artificial intelligence, embedded systems, and TinyML to develop an efficient assistive technology solution. The Smart Glove offers a low cost, portable, and privacy preserving approach for sign language interpretation, with potential applications in daily communication, education, and social inclusion for the speech impaired community.