In 2023 Defcon hacker conference in Las Vegas, major AI technology companies partnered with algorithmic integrity and transparency groups to thousands of participants on generative AI platforms and find weaknesses in these critical systems. This “red teaming” exercise, which also received support from the U.S. government, took a step toward opening up these increasingly influential but opaque systems to scrutiny. Now, the nonprofit Humane Intelligence, which specializes in ethical AI and algorithmic assessment, is taking this model a step further. On Wednesday, The group has announced a call for participation with the U.S. National Institute of Standards and Technology, inviting any U.S. resident to participate in the qualification phase of a national selection effort to evaluate AI-based office productivity software.
The qualifiers will take place online and are open to developers and any person in the general public as part of NIST’s AI challenges, known as Assessing Risks and Impacts of AI, or ARIA. Participants who advance through the qualifying round will take part in an in-person red-teaming event in late October at the Conference on Machine Learning Applied to Information Security (CAMLIS) in Virginia. The goal is to expand capabilities to conduct rigorous testing of the security, resilience, and ethics of generative AI technologies.
“The average person using one of these models doesn’t really have the ability to determine whether or not the model is a good fit for their needs,” says Theo Skeadas, chief of staff at Humane Intelligence. “So we want to democratize the ability to conduct assessments and make sure that everyone who uses these models can assess for themselves whether or not the model is a good fit for their needs.”
The final CAMLIS event will split participants into a red team trying to attack AI systems and a blue team working on defense. Participants will use the AI 600-1 profilea part of NIST AI Risk Management Frameworkas a rubric for measuring whether the red team is able to produce results that violate the expected behavior of the systems.
“NIST’s ARIA relies on structured user feedback to understand real-world applications of AI models,” said Rumman Chowdhury, founder of Humane Intelligence, who is also a contractor with NIST’s Office of Emerging Technologies and a member of the U.S. Department of Homeland Security’s AI Safety and Security Council. “The ARIA team is primarily comprised of experts in socio-technical testing and evaluation, and is using this experience to move the field toward rigorous scientific evaluation of generative AI.”
According to Chowdhury and Skeadas, the partnership with NIST is just one of several AI Red Team collaborations that Humane Intelligence will announce in the coming weeks with U.S. government agencies, international governments, and NGOs. The goal of the effort is to make it much more common for companies and organizations that develop what are now black-box algorithms to provide transparency and accountability through mechanisms like “bias bounty challenges,” where individuals can be rewarded for uncovering problems and inequities in AI models.
“The community should not be limited to programmers,” Skeadas says. “Policymakers, journalists, civil society, and non-technical people all need to be involved in the process of testing and evaluating these systems. And we need to make sure that less-represented groups, such as people who speak minority languages or come from non-majority cultures and perspectives, can participate in this process.”
