Lateral bar
From Capitol Hill to the forefront of AI research, Daniela Amodei’s journey is reshaping the AI industry.
With Daniela Amodei, president and co-founder of Anthropic
How to create an ethical product in a field where the very definition of ethics is evolving day by day, legal rules are still being written and technology itself is evolving at dizzying speed?
This question motivated siblings Daniela and Dario Amodei to co-found Anthropic, an AI company dedicated to security and research that, coincidentally, also builds some of the most powerful large language models (LLMs) and is hiring some of the world’s largest companies like partners.
“I started my career in international development, working on issues such as poverty assessment, conflict mitigation and global health,” says Daniela Amodei. His diverse experiences ranged from political campaigns on Capitol Hill to leading teams across various industries at startups like Stripe and OpenAI. It was her co-founder and brother, Dario, with his background in neuroscience and computational biology, who initially exposed her to the field of AI.
The Amodeis and some of their early Anthropic colleagues previously worked at OpenAI, the company behind ChatGPT. But the question “How can we ensure a secure future for AI?” » motivated them to strike out on their own. In a recent article in The New York Timeswriter Kevin Roose reported that Anthropic staff feared the damage future AI could cause: “Some compared themselves to today’s Robert Oppenheimers, weighing moral choices about a powerful new technology that could profoundly alter the course of history. »
This is an incredible amount of weight to carry on a daily basis. So how do you create an ethical AI product and ensure that this power is used wisely? The answer at Anthropic is to create a safe AI company and, with it, safe AI. The company achieves this by creating standards that guide its own actions as a company and a constitution that forms its LLM, known as Claude.
As for the company itself, Anthropic is a public benefit corporation, a designation that requires it to prioritize social impact and stakeholder responsibility, not just profits. The company also released a transparent and detailed document outlining its governance structure called “The Long-Term Benefit Trust,” which empowers a panel of five “financially disinterested” experts to oversee and, if necessary, remove its board members. ‘administration. Essentially, Anthropic has built-in guardrails.
“We want the transition to more powerful AI systems to be positive for society and the economy as a whole. This is why much of our research aims to explore ways to better understand the systems we develop, mitigate risks and develop steerable, interpretable and safe AI systems,” explains Amodei.
This type of thinking informs how Anthropic builds security into its AI models. Anthropic uses a training technique known as Constitutional AI, in which it uses a written constitution, rather than subjective human feedback, to teach values and boundaries to its models and train them for safety. The result is that, compared to other popular LLMs, Claude is much more reluctant to perform certain tasks. An AI model cannot be hampered by itself. But Claude’s training can sometimes give him an almost sheepish voice.
“I don’t have subjective feelings or emotions as an AI system,” Claude said in an interview. “However, I was created by Anthropic to be helpful, harmless, and honest.”
These three words – useful, harmless and honest – come up repeatedly whenever Claude is pushed to the limits of his learned principles. And although Claude refuses to discuss his training (“I apologize, but I don’t actually have detailed information about my own training process or my “constitution”), Anthropic claims that its constitution is a document in constantly evolving that draws on a wide range of sources, including the United Nations Universal Declaration of Human Rights and Apple’s terms of service.
“Fostering a better understanding of the technology will be crucial to ensuring that the industry as a whole is developed safely and responsibly,” says Amodei. “This applies not only to the general public, but also to policy makers and civil society. »
This constitutional approach to training is partly because AI-trained AI is easier to scale. And scale is also one of Anthropic’s stated goals. To test whether the principles of constitutional AI hold, it is necessary to develop increasingly powerful models – and this mainly involves scaling. But this requires increasing both the number of users whose queries can teach the model and the amount of computing power behind it.
The pursuit of AI at scale raises other ethical questions: there is the environmental cost of all that computing power; there is the necessary involvement of one of the few tech companies that even has access to this power; and it is possible, as the user base grows, that bad human actors will attempt to subvert the formed principles of the model and use it for nefarious purposes.
But these questions are inherent to AI regardless of who builds it, and Anthropic, of course, is just one of many companies building powerful LLMs.
“External engagement on these issues is at the heart of our work. We believe that developing AI safely is a much larger project than Anthropic can or should tackle alone,” emphasizes Amodei. “We hope that by being transparent about the risks we see, we will be able to motivate a much broader effort to explore potential solutions.”
If only people who don’t care about ethics train AI models, then the AI models will be amoral at best. Anthropic’s belief is that we cannot make AI safe in the present without developing safe AI. And we can only ensure security in the future, at the technological frontier, if we ourselves manage to reach this frontier.
