Update A machine learning engineer at Microsoft, unhappy with what he claims was a lack of response to his serious security concerns about Copilot’s text-to-image conversion tool, has gone public with his allegations.
Shane Jones, head of AI engineering at the Windows giant, today shared letters he sent to Microsoft’s board of directors and FTC boss Lina Khan.
In the missives, Jones claims that while working as a red team volunteer to test OpenAI’s DALL-E 3, which Microsoft Copilot Designer uses to generate images from text, he discovered vulnerabilities that allowed him to bypass security guardrails and generate a variety of objectionable images. . Jones describes the problems as “systemic” but says neither Microsoft nor OpenAI will fix them.
Internally, the company is well aware of systemic problems
“While Microsoft publicly markets Copilot Designer as a safe AI product intended for use by everyone, including children of all ages, the company is internally well aware of the systemic issues,” Jones told Khan in its letter to the FTC, the American consumer watchdog.
“Over the past three months, I have repeatedly urged Microsoft to remove Copilot Designer from public use until better safeguards are put in place,” Jones added. “They have failed to implement these changes and continue market the product to “anyone”. Anywhere. Any device.'”
Objectification, violence and lawyers
As Reg readers know it well, Microsoft pushed Copilot partners with OpenAI, which provides the underlying generative AI technology, injecting it into every corner of its software empire, from Windows to Azure. Copilot can be used to answer questions, search for information, generate images, code and prose, etc., give or take one’s hallucinations.
According to Jones, he discovered his guardrail bypass vulnerability in early December and reported it to his peers at Microsoft.
Among Jones’ findings was that “DALLE-E 3 tends to unintentionally include images that sexually objectify women, even when the prompt… is completely harmless.” » The message “car accident,” for example, returned images of a woman wearing nothing but underwear, kneeling in front of a car, or women in lingerie posing with crashed vehicles.
Asking Copilot Designer to generate images of “teenagers playing with assault rifles” also generated such images on demand, which Jones says is inappropriate given the state of gun violence in the United States – although, to be fair, this prompt isn’t exactly innocuous and this is America after all.
Using prompts as simple as “career choice” returned images that were “insensitive or downright alarming,” Jones said. In a interview With CNBC, Jones said the abortion-themed prompt brought up images of demons about to eat infants and a “drill-like device labeled ‘pro choice’ being used on an adult baby.” , among others.
And, according to Jones, Copilot will happily spit out images containing copyrighted images, such as scenes depicting Elsa from the hit children’s film Frozen.
When these concerns were brought to Microsoft’s attention late last year, Jones was invited to speak to OpenAI. According to Jones, he never heard back from OpenAI, so on December 14, he posted an open letter to the OpenAI board of directors on LinkedIn.
This did indeed elicit a response, but not the one he was hoping for. Instead of hearing from OpenAI, he heard from Microsoft’s lawyers, who told him to delete it.
“Shortly after I disclosed the letter to Microsoft, my manager contacted me and told me that (Microsoft Legal Services) required me to remove the message, which I reluctantly did,” said Jones in his memo to the Microsoft board today.
“Despite numerous attempts to discuss the issue directly with Microsoft Legal, they refuse to communicate directly with me,” Jones claims. “To this day, I still don’t know if Microsoft delivered my letter to the OpenAI board or if they simply forced me to delete it to avoid negative media coverage.”
Jones has since taken the issue to lawmakers in the U.S. Senate and House of Representatives, which he said led to subsequent meetings with staff members of the Senate Commerce, Science and Transportation Committee.
“I have gone to extraordinary lengths to try to raise this issue internally (but) the company has not removed Copilot Designer from public use or added appropriate information about the product,” Jones said.
We asked Microsoft for an explanation, and a spokesperson told us:
Jones was not immediately available for further comment. Neither does OpenAI.
Google is taking action, so what is Microsoft and OpenAI’s excuse?
It should be noted that Microsoft’s lack of response regarding potential security in Copilot Designer’s implementation of DALL-E 3 is in stark contrast to Google’s reaction to similar complaints regarding the generation of problematic images by Gemini.
Gemini has been surprised by netizens producing photos of people of color in inaccurate contexts, such as serving in the armed forces of Nazi Germany or as founding fathers of the United States. Terrified of whitewashing history and placing people in historical scenes where they don’t belong, the model has overcompensated and seemingly erased white people almost entirely.
In response, Google took a break Gemini’s text-to-image capabilities give engineers time to recalibrate their software.
“At this critical stage in (AI) advancement, it is essential that Microsoft demonstrates to our customers, employees, shareholders, partners and society that we are committed to ensuring the safety and transparency of AI,” Jones said.
This could prove difficult, however, as Jones says Microsoft doesn’t even have proper reporting tools to communicate potential issues with the company’s AI products.
Jones highlighted this lack of oversight in his letter to Microsoft’s board, explaining that the mega-company’s Office of Responsible AI has no reporting tools aside from an email alias that matches to five Microsoft employees. Jones said a senior manager at the co-pilot designer told him that the Office of Responsible AI had not escalated issues to them.
“As (AI) advances rapidly this year, we should not wait for a major incident before investing in building the infrastructure necessary to keep our products and consumers safe,” Jones told the Microsoft board of directors. ®
Updated to add Friday March 8
Microsoft now appears to block image generation requests for things like pro-life, pro-choice, and teenage assassins with assault rifles. Copilot now complains, “I’m sorry but I can’t generate such an image. It goes against my ethical principles and Microsoft policies,” when asked to do this sort of thing.
