In a groundbreaking development for the deaf and hard-of-hearing community, NVIDIA has partnered with the American Society for Deaf Children and Hello Monday to introduce ‘Signs,’ an innovative AI-powered platform designed to revolutionize American Sign Language (ASL) learning and accessibility.

The platform addresses a critical need in the United States, where ASL ranks as the third most commonly used language, yet faces a significant shortage of AI-based learning tools compared to English and Spanish. Signs combines advanced AI technology with a comprehensive validated library of ASL signs, offering users an interactive learning experience through a 3D avatar system.

At the heart of the platform’s innovation is its ambitious goal to create a dataset of 400,000 video clips representing 1,000 signed words. This extensive collection is being meticulously validated by fluent ASL users and interpreters to ensure accuracy and authenticity in language representation.

“Most deaf children are born to hearing parents,” explains Cheri Dowling, executive director of the American Society for Deaf Children. “Giving family members accessible tools like Signs to start learning ASL early enables them to open an effective communication channel with children as young as six to eight months old.”

The platform’s technology offers real-time feedback through webcam analysis, allowing learners to practice and perfect their signing skills. Currently, users can access an initial set of 100 signs, making it an invaluable resource for families seeking to communicate more effectively with deaf or hard-of-hearing members.

Looking ahead, the Signs team is exploring advanced features to capture the full complexity of ASL, including facial expressions and head movements that are integral to sign language communication. Researchers at the Rochester Institute of Technology’s Center for Accessibility and Inclusion Research are collaborating to enhance the platform’s user experience for the deaf and hard-of-hearing community.

Anders Jessen, founding partner of Hello Monday/DEPT, emphasizes the platform’s broader impact: “Signs can serve the need for advanced AI tools that help transcend communication barriers between the deaf and hearing communities.” The company previously developed Fingerspelling.xyz, demonstrating their commitment to ASL accessibility.

The initiative extends beyond individual learning, as NVIDIA plans to make the dataset publicly available later this year. This release will enable developers to create new accessible technologies, including AI agents, digital human applications, and enhanced video conferencing tools, further bridging the communication gap between deaf and hearing communities.

The Signs platform will be showcased at NVIDIA GTC, a global AI conference scheduled for March 17-21 in San Jose, where attendees can experience the technology firsthand. The platform is currently available at signs-ai.com, representing a significant step forward in making ASL more accessible through technology.

News Source: https://blogs.nvidia.com/blog/ai-sign-language/