A powerful collaboration between two universities and a tech giant is working to develop a mobile app that will make it easier for parents of deaf children to learn American Sign Language.

The Center on Access Technology at Rochester Institute of Technology’s National Technical Institute for the Deaf, in partnership with Google and the Georgia Institute of Technology, creates PopSign, who offers a complete and interactive learning experience that parents can use anytime, anywhere. The app displays a video of a person signing to introduce new vocabulary, then the user’s memory is tested. To advance to the next level, the user swipes and shoots a ball to hit bubbles with written words that match the sign.

The game is available on Google Play and Android devices.

“We are thrilled to collaborate with the Georgia Institute of Technology and Google on this project,” said Spencer Montan, associate director of NTID’s Center on Access Technology. “Our student employees are also actively involved in redesigning the user interface and user experience of the app, to make it more ‘accessibility friendly’. Our goal is to expand the roster of signs and the number of developers and designers to make the game more fun and available to anyone who wants to learn ASL.

In the United States, nearly 95% of deaf and hard of hearing children have hearing parents. Most hearing parents are unfamiliar with American Sign Language, which can lead to difficulties and frustrations in communicating with their children. According to a National Institutes of Health study, language deprivation can lead to complications in a child’s mental health and development, and in this case, parents are recommended to learn ASL. However, ASL lessons are often inaccessible to parents, and watching ASL videos can feel distant and uninviting.

PopSign began as an extension of a doctoral thesis by Kimberly Xu, a former Georgia Tech student, to help hearing parents of deaf infants learn the ASL vocabulary needed to communicate with their children.

“My students have been developing PopSign since 2016,” said Thad Starner, professor, founder and director of the Contextual Computing Group at Georgia Tech’s College of Computing. “Although we designed the game to help parents learn ASL, the game can be useful for children who want to be bilingual in ASL and English. It’s a surprising angle that we’re just beginning to explore.

“We are working with RIT/NTID students to improve the gameplay, accessibility, and enjoyment of the current game. However, upcoming games in development, PopSign Generation and CopyCat Mobile, are being redesigned from the ground up with RIT/NTID students , at their suggestion. These games will use a smartphone’s selfie camera to recognize the player’s signature within the game.”

RIT/NTID CAT Lab students Ikemefuna Chukwunyerenwa, a web and mobile computing student from Houston; Jinlan Li, a professional studies student from China; and Loam Shin, a human-centered computing student from Lawrenceville, Georgia, serve as user interface designers, user experience analysts, and software developers, while Georgia Tech students Colby Duke , Sahir Shahryar and Prerna Ravi serve as project managers and data collectors. , and develop the sign language recognition tools used by PopSign.

Sam Sepah, an RIT/NTID alumnus with an MS (Human Resource Development) and Senior Accessibility Research Project Manager at Google, also participated in the project.

For more information, contact Vienna McGrain at 585-475-4952, [email protected], or on Twitter: @viennamcgrain.

Warning: AAAS and EurekAlert! are not responsible for the accuracy of press releases posted on EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.