SignAura: An AI-Based Real-Time Recognition System for Sri Lankan Sign Language

Authors

• Pramudhi Hansi Wijesinghe, Department of Computing, Faculty of Information Technology, CINEC Campus, Malabe 10115, Sri LankaFollow
• Nimesh Pollwaththage, Department of Computing, Faculty of Information Technology, CINEC Campus, Malabe 10115, Sri LankaFollow
• Dhishan Dhammearatchi, Department of Computing, Faculty of Information Technology, CINEC Campus, Malabe 10115, Sri LankaFollow

Document Type

Original Study

Abstract

The communication barrier experienced by deaf and hard-of-hearing in Sri Lanka is still a serious issue in society since there are few interpreters with Sri Lankan Sign Language (SLSL) and no automated translation technologies. The present paper introduces SignAura, an artificial intelligence-driven real-time SLSL sign-recognition system that provides multilingual lexical text and synthesized speech. The system uses computer vision and a fully connected neural network to identify isolated SLSL signs from live video streams.A fully connected neural-network classifier was used instead of the CNN-LSTM architecture, with 21 hand landmark keypoints extracted per frame using MediaPipe and represented as 63 numerical features.The final processed dataset contained 38,210 frame-level records, including 30,568 training records and 5,731 held-out testing records, covering 381 sign classes. During model training, 10% of the training data was used as an internal validation subset.The final model achieved a test accuracy of 80.27%, with macro precision of 80.63%, macro recall of 80.36%, and macro F1-score of 79.70%, while the test loss was 3.3375. A confusion matrix was generated from the held-out test predictions to examine class-level classification behaviour and identify frequently confused sign classes. A controlled frame-rate benchmark was not conducted as part of the final evaluation; therefore, a specific FPS value is not reported. The system performs isolated sign-level recognition and provides corresponding multilingual lexical output rather than complete grammatical sign-language translation. The current implementation does not fully incorporate non-manual linguistic features such as facial expressions, mouth movements, and body posture. The results reveal that it is possible to implement lightweight AI-based assistive technologies to support inclusive and scalable communication support based on the context of Sri Lanka.

Recommended Citation

Wijesinghe, Pramudhi Hansi; Pollwaththage, Nimesh; and Dhammearatchi, Dhishan (2026) "SignAura: An AI-Based Real-Time Recognition System for Sri Lankan Sign Language," Journal of Innovation in Science, Engineering and Technology: Vol. 7: Iss. 2, Article 7.
DOI: https://doi.org/10.66543/3084-858X.1116

First Pages

63

Last Page

70