Center for AI Safety

Field Building

Supporting the broader research community with workshops, competitions, social events, fellowships, and online resources.

Empowering the research community

AI safety is a challenge that is beyond any single research lab. We run a variety of initiatives to support and empower the existing research community while lowering barriers to entry and further expanding the community. Our efforts include providing infrastructure and resources for the AI safety research ecosystem, initiating multi-disciplinary projects to explore the societal effects of AI from new perspectives, and creating educational resources to encourage newcomers to join.

Active Projects

All of our currently active projects.

Compute Cluster

To support progress and innovation in AI safety, we offer researchers free access to our compute cluster, which can run and train large-scale AI systems.

View Project

Philosophy Fellowship

The CAIS Philosophy Fellowship is a seven-month research program that investigates the societal implications and potential risks associated with advanced AI.

View Project

SafeBench Competition

$500k

Benchmarks concretize research goals, make them more tractable, and can spur research efforts from the broader community. SafeBench provides $500,000 in prizes for ML Safety benchmarking ideas.

View Project

Moral Uncertainty Competition

$100k

Future AI systems need to be able to detect and act cautiously in morally ambiguous situations. The Moral Uncertainty Competition provides $100,000 in prizes to incentivize research towards machine learning models with the ability to detect substantial moral disagreement.

View Project

Autocast Competition

$125k

Forecasting plays a critical role in global decision-making. To improve global decision-making and encourage the development of open source AI forecasting tools, the Autocast Competition provides $125,000 in prizes for research towards machine learning models with the ability to forecast.

View Project

Educational and Student Resources

Lowering the barriers to entry in studying ML safety.

Intro to ML Safety

Intro to ML Safety is a comprehensive training program designed for individuals seeking additional support, community, and accountability while completing the ML safety course. Accepted participants receive access to peer discussion groups, mentorship, and a small stipend.

View Project

Student Scholarships

$2000

A $2000 scholarship for undergraduates and masters students who secure ML Safety research mentorship.

View Project

ML Safety Course

An online course which offers a comprehensive introduction to ML safety.

View Project

ML Safety Newsletter

NeurIPS 2022

A monthly newsletter detailing the latest advancements in ML safety.

Join Now

Past Projects

All of CAIS's past projects.

/

ML Safety Workshop at NeurIPS 2022

$100k

The ML Safety Workshop at NeurIPS 2022 brought together researchers from various fields to discuss and advance the field of ML safety.

View Project

/

Trojan Detection Competition at NeurIPS 2022

$50k

Neural Trojans are a growing concern for the security of ML systems, but little is known about the fundamental offense-defense balance of Trojan detection. The Trojan Detection Competition at NeurIPS 2022 poses the question: How hard is it to detect hidden functionality that is trying to stay hidden?

View Project

Adversarial Robustness Prizes at ECCV 2022

$30k

Three best paper awards to study model robustness to threats beyond small l_p perturbations, including attacks that are perceptible and attacks with specifications not known beforehand and are unforeseen.

View Projects

ML Safety Newsletter

We cover topics like deep learning, natural language processing, computer vision, robotics, and more. Our goal is to provide readers with insightful articles, research papers, and interviews with leading experts in the field. Whether you are a researcher, practitioner, or just interested in the latest AI and ML breakthroughs, this newsletter has something to offer.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

We care about your data in our privacy policy.