AI Resignation Letters: What Departing Researchers Reveal About the Industry’s Future

by Marcus Liu - Business Editor
0 comments

The AI Resignation Wave: Why Top Researchers Are Leaving Leading Labs

Corporate resignations rarely build headlines unless they occur at the highest levels. However, over the past two years, a surge of posts on X (formerly Twitter), open letters published on Substack, and public statements from prominent artificial intelligence researchers have created a novel form of communication—the AI resignation letter—each one becoming a significant event analyzed for its meaning. Collectively, these letters—some constrained by non-disclosure agreements and other loyalties—offer insights into how leading AI professionals perceive their industry and its future. The overall picture is often pessimistic.

A Recent Surge in Departures

This past week has seen several additions to the growing collection of “Why I quit this incredibly valuable company working on cutting-edge tech” letters, including from researchers at xAI and an opinion piece in The New York Times from a departing OpenAI researcher. Perhaps the most notable was from Mrinank Sharma, who, after a year leading Anthropic’s Safeguards Research Team, announced his departure from what is often considered the most safety-conscious of the leading AI startups. His 778-word letter on X blended poetic references—quoting Rainer Maria Rilke and Mary Oliver—with reflections on AI safety, his experiences with AI sycophancy and the potential for “AI-assisted bioterrorism,” and the broader “poly-crisis” facing society. The letter included three footnotes and cautionary warnings.

“We appear to be approaching a threshold where our wisdom must grow in equal measure to our capacity to affect the world, lest we face the consequences,” Sharma wrote. “Throughout my time here, I’ve repeatedly seen how hard it is to truly let our values govern our actions.”

Sharma noted his final project at Anthropic focused on “understanding how AI assistants could make us less human or distort our humanity”—a reference to the emerging concerns around AI psychosis and the potential harms of over-reliance on chatbots. He expressed a desire to pursue a poetry degree and “devote myself to the practice of courageous speech,” indicating a shift away from the field. He concluded by including the full text of William Stafford’s “The Way It Is.”

Beyond Sharma: A Pattern of Concerns

Sharma’s departure, while unique in its poetic expression, reflects broader themes emerging from AI resignations. It is less dramatic than the boardroom upheaval that briefly ousted OpenAI CEO Sam Altman in November 2023, and less alarming than warnings from AI safety researchers who left their positions believing their employers weren’t adequately addressing the potential risks of artificial general intelligence (AGI). Some AI experts debate the feasibility or even the definition of AGI.

However, Sharma’s letter highlights the strong attachments top AI researchers feel towards their work, colleagues, and employers. It similarly exposes recurring tensions: intense competition for resources between research/safety teams and those focused on consumer-facing products, pressure to release products without sufficient testing or safeguards, and a sense of mission sometimes undermined by feelings of betrayal. Notably, resignations rarely come from those on the product side of AI development.

Alignment and Safety: A Common Thread

Many who publicly resign from AI companies work in safety and “alignment”—the field dedicated to ensuring AI capabilities align with human needs, and values. While often optimistic about AI and AGI, these researchers worry that financial pressures are eroding safety measures. Many seek new positions at competing AI startups or transition to AI think tanks.

Miles Brundage, who resigned from OpenAI’s AGI readiness team in 2024, stated, “Neither OpenAI nor any other frontier lab is ready, and the world is also not ready” for AGI. Dylan Scandinaro, moving from Anthropic to OpenAI as Head of Preparedness, acknowledged the rapid advancement of AI, stating, “The potential benefits are great—and so are the risks of extreme and even irrecoverable harm.” Daniel Kokotajlo, also formerly of OpenAI, warned that OpenAI’s systems “could be the best thing that has ever happened to humanity, but it could also be the worst if we don’t proceed with care.”

xAI and OpenAI: Hotbeds of Departures

While xAI, co-founded by Elon Musk, has seen several founding team members leave, OpenAI remains the focal point of the AI resignation phenomenon. Over the past two years, major figures, including executives and safety researchers, have departed OpenAI through resignation, termination, or forced exits due to internal disputes. Seven left in the first half of 2024 alone.

OpenAI’s recent decision to incorporate advertising into ChatGPT prompted researcher Zoë Hitzig to resign. In a New York Times op-ed, she warned about the implications of ads becoming integral to chatbot conversations. She cautioned that OpenAI appeared willing to exploit its “archive of human candor”—similar to Facebook—for targeted advertising, potentially manipulating users and undermining their autonomy.

Internal Conflicts and Leadership Concerns

Trust in leadership has been a recurring issue at OpenAI. In November 2023, Altman was abruptly fired by the board for allegedly being “not consistently candid in his communications.” He was reinstated less than a week later, consolidating his power. This event triggered further departures.

Ilya Sutskever, OpenAI co-founder, resigned in May 2024, and was replaced by John Schulman, another co-founder. Schulman subsequently left OpenAI for Anthropic and then moved to Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, who briefly served as OpenAI’s interim CEO during Altman’s firing. Jan Leike, who also led alignment work, resigned shortly after Sutskever, stating that OpenAI’s “safety culture and processes have taken a backseat to shiny products.” He was hired by Anthropic less than two weeks later.

A Broader Disconnect

Departing researchers have expressed concerns that alignment and safety experts are often sidelined or absorbed into other teams, leaving a sense that AI companies are prioritizing development over control. However, these resignation letters often lack discussion of the current, real-world impacts of AI—such as its employ in data center construction, surveillance, weapons development, and labor disruption—focusing instead on potential future disasters.

As William Stafford wrote in the poem shared by Mrinank Sharma, “Tragedies happen. people get hurt or die; and you suffer and get vintage…Nothing you do can stop time’s unfolding.” This sense of inevitability and resignation, while perhaps understandable, highlights a disconnect between the concerns of top AI researchers and the broader public.

Jacob Silverman is a contributing writer for Business Insider and author of “Gilded Rage: Elon Musk and the Radicalization of Silicon Valley.”

Related Posts

Leave a Comment