We are starting a non-profit research organization: STABL.

Adapting to advanced AI will require dramatic institutional change and broad societal resilience. It will also require evidence about AI progress and impacts — the prospects of recursive self-improvement (RSI), the pace of AI diffusion, and the bottlenecks that constrain how AI capabilities translate into societal outcomes.

Four theses define our worldview:

Another way to state our worldview is to place it on a 2x2 chart. The vertical axis depicts taking AI’s transformative potential seriously. The horizontal axis depicts taking downstream bottlenecks to AI’s impacts seriously. At the top left is technological determinism: the view that superintelligence will unilaterally transform society by obliterating downstream bottlenecks. Many AI CEOs express this view, such as when predicting that a century’s worth of progress will be compressed into the next few years.

This view often ignores the organizational, regulatory, and societal bottlenecks to such a transformation; in this view, such questions are moot because superintelligence is all powerful. Surprisingly, some of the AI industry’s strongest critics share this worldview, focusing on slowing AI capability improvements as the main way to keep society safe from AI’s dangers.

The opposite view, at the bottom right of the chart, treats existing organizations, processes, professions, and governance as fixed points, and insists that technology be molded to fit them. People holding this view often work on limits on AI adoption, and protecting existing structures from disruption. They are good at identifying and leveraging the bottlenecks that blunt AI’s impact, such as regulation, professional norms, organizational inertia, and political pushback.

Slowing the pace of change can buy valuable time to shape technology’s impacts. But when preserving the status quo becomes a goal unto itself, it stands in the way of technology-enabled societal progress. (As an example, consider the resistance to self-driving cars despite their being safer than humans, or bans on AI-enabled legal advice).

STABL belongs in the top-right quadrant. We take frontier AI seriously as a transformative technology, yet consider the role of institutions — both new and existing — to be important in helping society navigate the transition.

For example, how should we understand and improve AI’s impacts on medicine? Many technological determinists claim superintelligence will rapidly cure nearly all diseases. Someone with the opposite view might respond that the bottleneck is not drug discovery, which AI may well accelerate, but clinical trials, which take years because of regulations and safety risks.

This response correctly identifies the bottlenecks to AI’s impacts. But this is often seen as the end of the conversation. In our view, understanding the bottlenecks to AI’s impacts is only the starting point of analysis. The next question is whether drug testing can be made faster and more efficient in order to meaningfully materialize AI’s benefits. For example, what are the barriers to in-silico testing? How can we rethink regulatory institutions and processes? Unfortunately, neither extreme pays close attention to these critical questions.

To be clear, STABL isn’t the only organization in the top-right quadrant; much of our work builds on other organizations and thinkers who take both AI’s potential and its downstream bottlenecks seriously. But compared to the other quadrants, there is a sparsity of thinking in this quadrant, which has led to polarization among the communities that take either the technology or the bottlenecks seriously. Our ultimate goal is to contribute to a rich intellectual agenda for this quadrant, create the space for new organizations, and collaborate with existing organizations in this and other quadrants.

To follow our work, subscribe to the STABL newsletter:

Subscribe to the STABL newsletter

Our research agenda

The top-right quadrant leads us to many interesting research directions. On one hand, we expect AI capabilities to continue improving quickly, which motivates our empirical research on the pace of progress and RSI. We expect highly capable AI models to continue to be widely available, which motivates our analysis and advocacy for risk mitigation beyond aligning AI models and non-proliferation.

On the other hand, we consider institutions important for shepherding the transition to a world where advanced AI is broadly diffused. This leads to our interest in understanding the downstream outcomes of advanced AI adoption. We plan to measure specific benefits and harms, study the disruption of labor markets and beyond, propose ways to improve societal resilience and revitalize institutions.

These interests lead to four key research directions, which we detail below. While other organizations have worked on similar questions, we expect our research to be different in how we formulate these questions and how we go about answering them, informed by our worldview and priors about AI progress.

1. How should we measure progress towards recursive self-improvement?

Many forecasts of explosive AI progress rest on recursive self-improvement (RSI): AI automating AI research itself. We take RSI seriously precisely because our model of progress differs from the one underlying most RSI forecasts, and we want to empirically test claims about RSI to get early warnings of AI’s potential impacts. But how should we measure progress towards RSI?

Most RSI evaluations measure AI’s ability to complete tasks where success is easily verifiable. Here, AI systems have made rapid progress. But we think there is a growing chasm between AI performance on verifiable and open-ended tasks.

Through CRUX, we plan to run frequent open-ended evaluations to probe the upper bounds of AI capabilities on long, messy tasks, including whether AI agents can conduct open-ended AI research. Our first results suggest they cannot yet do so; expert reviewers unambiguously rejected the papers produced by frontier agents. We plan to repeat these evaluations regularly with improved models and scaffolds.

More broadly, the conversation about RSI is dominated by viewpoints in the top-left quadrant. Those most worried about superintelligence were ahead of the curve in recognizing the importance of studying RSI carefully. But communities and organizations have also self-selected into this work, leading to a lack of intellectual diversity. When the literature relies on the same (contested) assumptions about AI progress, it is prone to the same failures in its conclusions.

A more diverse ecosystem of researchers and organizations, including those with different priors about AI progress, could test whether these assumptions hold, for example by:

As the importance of RSI-style dynamics becomes clearer for understanding AI progress, we hope there is a vibrant intellectual ecosystem of researchers and organizations evaluating RSI seriously while not necessarily treating its default outcome as superintelligence.

2. How can society defend against the risks of advanced AI?

AI safety has historically been focused on models themselves, and alignment has been the field’s technical grand challenge. The safety community’s response to open models, where alignment can be undone, has been to push for non-proliferation policies. But over time, it has become clear that advanced AI will be broadly available, including to people who might want to misuse it. So our responses to AI risks need to expand beyond alignment and non-proliferation.

AI safety will require three additional responses. First, we need to invest in the ability to control, monitor, and secure AI systems to prevent unintended actions even when AI systems are misaligned. Second, we need to improve downstream defenses, such as the cybersecurity of critical infrastructure to protect the attack surfaces where AI’s harms materialize. Third, we need to boost societal resilience to reduce the severity of harm when our defenses fail.

We think resilience is the most neglected approach. While the AI community has in principle emphasized resilience, in practice, the main emphasis remains on alignment and non-proliferation. We often hear the reaction that resilience sounds good in theory but no one knows what it means concretely. One of STABL’s goals will be to articulate an actionable resilience agenda. We hypothesize that for resilience to be effective, it must have the following properties:

3. How can we steer the economic impacts of advanced AI?

Even though advanced AI has been widely adopted by customers, workers, and firms, we don’t yet have a clear accounting of either the benefits or the costs of this adoption. While there is abundant evidence of workplace adoption, the returns on that investment have yet to materialize as broad productivity gains. And while many have raised concerns about imminent and massive job loss, the aggregate unemployment rate continues to be largely unchanged in the United States and other countries.

These trends are consistent with our view that the bottlenecks which govern economic transformation are largely beyond the scope of improving capabilities, and can’t directly be understood by AI capability evaluations. As a result, to better understand AI’s impacts, these bottlenecks themselves require dedicated study. On the other hand, we think the transformation that is already occurring as a result of AI might be obscured through the lens of traditional economic indicators.

These views motivate our work on steering AI towards augmentation, building prototypes for improving AI’s relationships with workers, and understanding how AI impacts the nature of work, beyond aggregate measures like employment statistics. We expect a lot of our work in the area to be conducted in collaboration with the growing set of organizations focused on the economic impacts of advanced AI. A few directions we are interested in:

4. How should specific professions and sectors respond to AI?

Neither capability benchmarks nor exposure metrics determine how advanced AI might affect specific jobs, industries, and institutions, and how they should adapt to it. There is a need for examining different occupations and institutions in context. In recent work, we have conducted deep dives into AI’s impacts on science, legal services, and software engineering, and plan to continue analyzing the ongoing transformation and likely future of other kinds of knowledge work.

We aim to develop principles and policies that keep knowledge professionals and democratic institutions in control while also enabling productivity gains, transforming workflows and processes, removing unnecessary barriers to entry, and improving state capacity. We will propose ways to enable healthy competition between incumbents and AI-enabled upstarts — at the level of organizational forms, not just individual organizations. At the same time, we recognize that there are many cases where frictions in institutions are desirable. We aim to find a middle ground that welcomes beneficial reforms while avoiding the complete displacement of existing structures.

What’s next for STABL?

STABL is actively fundraising and hiring across all four workstreams. We are thrilled to have an advisory board comprising leading experts on AI’s societal impacts: Erik Brynjolfsson, Michael R. Strain, Audrey Tang, Helen Toner, and Robb Willer, as well as a great group of researchers and collaborators.

As with the previous work that the three of us have done, STABL will produce four main types of outputs.

To follow our work, subscribe to our newsletter. We’d love to hear about other organizations in the top-right quadrant. We know that many people share our thinking, and what is missing is a systematic research program, an ecosystem of organization- and movement-building, and funding to sustain and amplify the work. We hope to play a role in building this intellectual project.