Today, Microsoft announced a new collaboration with Be My Eyes bring high-quality disability-representative data to help train AI systems. This work with Be My Eyes will help make Microsoft’s AI models more inclusive for the 340 million people who are blind or visually impaired around the world and is the latest step in our commitment to creating inclusive technology.
AI requires large amounts of data for training and utility, but too often the handicap is underrepresented Or poorly categorized in data sets. In Microsoft Research most recent article On AI performance in describing images from blind or visually impaired people, disabled objects, such as a Braille device, have been included less frequently in popular large-scale image-to-text datasets, leading to recognition of these objects is approximately 30% less precise. This data desert on disability can limit the usefulness of a technology, reinforce existing stereotypes and amplify prejudices. As we continue to uncover opportunities in AI and accessibility, inclusive data through these types of community partnerships and collaborations is paramount to building inclusive AI.
How we do this matters. Transparency and user control are the guiding principles of data privacy in this agreement. Be My Eyes will provide video datasets including unique objects, lighting and framing that realistically represent the lived experience of the blind and partially sighted community. Personal information will be removed from metadata by Be My Eyes prior to sharing and the company continues to work transparently with its members to provide a clear process and options for opting out of data sharing. Microsoft will then use the data to improve the accuracy and precision of scene understanding and descriptions with the goal of increasing the utility of AI applications for the blind and visually impaired community.
Today’s news marks the next chapter in our collaboration with Be My Eyes, which began in 2017 when their app was integrated into the Disability Answer Desk support experience to provide customers with more efficient technical support . Last year, Microsoft was the first to pilot their new “Be My AI”AI helping to resolve technical support issues. Be My Eyes has also recently developed a Windows application, available in the Microsoft Store. Collaborations like these allow us to improve the representation and diversity of AI data and drive more authentic disability inclusion in our technology.
Responsible AI is inclusive AI
Over the past 18 months, we’ve focused on understanding how generative AI can directly impact people with disabilities, but our history goes back even further. Our approach is based on principles to ensure we extend these benefits and mitigate the potential harms of AI. Microsoft’s Responsible AI Framework is how we do this. This includes fairness, reliability and safety, privacy and security, transparency, accountability and inclusiveness, including accessibility.
AI innovations can reduce cognitive load through summarization, automate meeting notes, and provide detailed image descriptions. These innovations have many positive implications for neurodiverse, deafblind communities and more, and with over a billion people on the planet suffering from some type of disability, the disability data desert is a crucial problem to resolve. 
Speech is an example of the areas we are investing in, and since 2022, we are proud to work with the University of Illinois Urbana-Champaign (UIUC) alongside Apple, Google, Meta and Amazon on the voice accessibility project. UIUC works with nonprofit organizations to collect data to improve speech recognition for people with diverse speech patterns, such as those with ALS. This data helps AI models better understand speech diversity and expands the scenarios where this technology can provide benefits, including accelerating the accuracy of non-standard speech data in consumer voice platforms, like Azure Speech. The project improves AI systems for people with disabilities by integrating diverse data and perspectives into the development process.
Our commitment to inclusive AI
At Microsoft, we are committed to creating inclusive AI that is representative of everyone who uses it, while protecting marginalized members of society from the proliferation of bias that could impact education, employment and civic engagement. We believe that accessible technology is a fundamental right that can unlock opportunities in all areas of society, and generative AI is one of the most powerful tools we have today to harness this potential when it is designed in an inclusive and responsible manner. We are also committed to transparency and strong data protection for every individual. Today’s news with Be My Eyes builds on these commitments, and we will continue to seek partnerships across the industry and community to create a more inclusive future.
If you have any questions or comments about accessibility at Microsoft, let us know at Disability Response Office or try the AI-powered Bing Ask the Microsoft Accessibility Tool.
To learn more about accessibility at Microsoft: Our approach to accessibility | Microsoft Accessibility
Learn more about Be My Eyes Privacy Commitments.
