Trillium Labs Bets Open Research Can Tame AI Risk
Original: These AI Experts Want to Do High-Stakes Research Out in the Open
Why This Matters
Open safety research could set a precedent at a time when RSI and agent autonomy are escalating concerns.
Nonprofit Trillium Labs, co-founded by Nathan Lambert and Tom Zick, will conduct high-stakes AI research—including recursive self-improvement and autonomous agents—openly, publishing experiment details for outside replication. The founders argue frontier labs' secrecy hinders community scrutiny and worsens safety outcomes.
Most frontier AI labs treat their riskiest research as proprietary. Trillium Labs, a new nonprofit, is betting that's the wrong call. Co-founders Nathan Lambert and Tom Zick plan to publish the methodology and results of experiments involving recursive self-improvement (RSI) and AI agents—areas widely considered among the most consequential in the field. Their argument: closed development erodes the scientific community's ability to scrutinize and improve safety approaches. 'The current closed trajectory of frontier AI development is taking us a step backwards,' Lambert told WIRED, invoking the scientific method as a centuries-old tool for harm reduction. The contrast with current industry norms is stark. Models from OpenAI and Anthropic are accessible only via app or API, with little visibility into training or behavior. Meanwhile, some Chinese labs and Stanford's Marin project have moved toward more transparent training runs. Lambert brings relevant pedigree—he previously worked at Ai2, one of the more open research labs in the US, and at Hugging Face, and founded the American Truly Open Models initiative. Zick has a background at Harvard and advised Charles Schwab on responsible AI policy. The two met as UC Berkeley PhD students during the pandemic. Their concern: academic AI research and industry work have drifted so far apart that professors and students are largely locked out of the most significant developments.