AI-Powered Indian Sign Language (ISL) Detection and Translation

Chapter Use Case 08, Section 4.9 Accessibility, pg (731-736), in AI for Good Innovate for Impact: Interim Report 2025

Published by International Telecommunication Union (ITU) | 2025, Vol 1 | Edition 1ed

ISBN: 978-9-2614-0961-6

N.A.

Document

This work presents an AI-powered system for real-time detection and translation of Indian Sign Language (ISL) to and from spoken and written language, designed to reduce communication barriers for deaf and hard-of-hearing communities in India. Building on recent advances in computer vision and deep learning for sign language recognition, the system uses vision models to capture fine-grained hand, body, and facial cues and transformer-based language models to generate fluent translations across ISL, English, and Hindi in both directions. The architecture emphasizes privacy-preserving processing via skeletal abstraction and efficient edge deployment to support low-resource devices and real-world accessibility contexts. Beyond the core translation pipeline, the work discusses design choices for handling non-manual signals, emotional tone, and multilingual integration, positioning the system as a practical contribution to AI for accessibility and inclusive communication.