: Studying whether AI can drive innovation in personal assistant devices and sign language

مايكروسوفت 0 تعليق ارسل طباعة تبليغ حذف

belbalady.net Advancing tech innovation and combating the data dessert that exists related to sign language have been areas of focus for the AI for Accessibility program. Towards those goals, in 2019 the team hosted a sign language workshop, soliciting applications from top researchers in the field. Abraham Glasser, a Ph.D. student in Computing and Information Sciences and a native American Sign Language (ASL) signer, supervised by Professor Matt Huenerfauth, was awarded a three-year grant. His work would focus on a very pragmatic need and opportunity: driving inclusion by concentrating on and improving common interactions with home-based smart assistants for people who use sign language as a primary form of communication. 

Since then, faculty and students in the Golisano College of Computing and Information Sciences at Rochester Institute of Technology (RIT) conducted the work at the Center for Accessibility and Inclusion Research (CAIR). CAIR publishes research on computing accessibility and it includes many Deaf and Hard of Hearing (DHH) students operating bilingually in English and American Sign Language. 

To begin this research, the team investigated how DHH users would optimally prefer to interact with their personal assistant devices, be it a smart speaker other type of devices in the household that respond to spoken command. Traditionally, these devices have used voice-based interaction, and as technology evolved, newer models now incorporate cameras and display screens. Currently, none of the available devices on the market understand commands in ASL or other sign languages, so introducing that capability is an important future tech development to address an untapped customer base and drive inclusion. Abraham explored simulated scenarios in which, through the camera on the device, the tech would be able to watch the signing of a user, process their request, and display the output result on the screen of the device.  

Some prior research had focused on the phases of interacting with a personal assistant device, but little included DHH users. Some examples of available research included studying device activation, including the concerns of waking up a device, as well as device output modalities in the form for videos, ASL avatars and English captions. The call to action from a research perspective included collecting more data, the key bottleneck, for sign language technologies.  

To pave the way forward for technological advancements it was critical to understand what DHH users would like the interaction with the devices to look like and what type of commands they would like to issue. Abraham and the team set up a Wizard-of-Oz videoconferencing setup. A “wizard” ASL interpreter had a home personal assistant device in the room with them, joining the call without being seen on camera. The device’s screen and output would be viewable in the call’s video window and each participant was guided by a research moderator. As the Deaf participants signed to the personal home device, they did not know that the ASL interpreter was voicing the commands in spoken English. A team of annotators watched the recording, identifying key segments of the videos, and transcribing each command into English and ASL gloss. 

Abraham was able to identify new ways that users would interact with the device, such as “wake-up” commands which were not captured in previous research. 

Screenshots of various “wake up” signs produced by participants during the study conducted remotely by researchers from the Rochester Institute of Technology.  Participants were interacting with a personal assistant device, using American Sign Language (ASL) commands which were translated by an unseen ASL interpreter, and they spontaneously used a variety of ASL signs to activate the personal assistant device before giving each command.  The signs here include examples labeled as: (a) HELLO, (b) HEY, (c) HI, (d) CURIOUS, (e) DO-DO, and (f) A-L-E-X-A.

إخلاء مسؤولية إن موقع بالبلدي يعمل بطريقة آلية دون تدخل بشري،ولذلك فإن جميع المقالات والاخبار والتعليقات المنشوره في الموقع مسؤولية أصحابها وإداره الموقع لا تتحمل أي مسؤولية أدبية او قانونية عن محتوى الموقع.
"جميع الحقوق محفوظة لأصحابها"

المصدر :" مايكروسوفت "

أخبار ذات صلة

0 تعليق

تابعنا
  • adblock تم الكشف عن مانع الإعلانات

الإعلانات تساعدنا في تمويل موقعنا، فالمرجو تعطيل مانع الإعلانات وساعدنا في تقديم محتوى حصري لك. شكرًا لك على الدعم ??

قصة عشق موقع فاربون دليل اضف موقعك Become a Patron!