"PSA: Almost nobody is working on alignment" by Chi Nguyen, peterbarnett cover art

"PSA: Almost nobody is working on alignment" by Chi Nguyen, peterbarnett

"PSA: Almost nobody is working on alignment" by Chi Nguyen, peterbarnett

Listen for free

View show details
People often assume that a large fraction of the AI safety community works on alignment. As far as we're aware, this is not true. Most people are not working on making sure superintelligent AIs are aligned with human values or follow human instructions.

Currently, the people who work on alignment are roughly:

  • The Alignment Research Center who work on a research bet by Paul Christiano
  • Probably Sequent who just got announced yesterday
  • Some scattered people who work at universities or independently, some of whom hang around Berkeley
A lot of the remainder of the AI safety community does indirect work like capability evaluations, risk assessments, control, policy, AI science, understanding misalignment (which maybe should partially count as alignment work), demos and so on.

Some production alignment work (i.e., making current models behave well) might help with more ambitious alignment, too (e.g., some COT-monitoring). Many people also work on aligning current/next-generation models so that these models help with aligning future models, and hope this scales to superintelligence.

We are not necessarily saying this is bad and that people are making a big mistake (e.g., neither of us work on alignment) but it's a notable fact that seems good to [...]

---

First published:
June 12th, 2026

Source:
https://www.lesswrong.com/posts/kJo2qsEdib8RZLvW6/psa-almost-nobody-is-working-on-alignment

---



Narrated by TYPE III AUDIO.
adbl_web_anon_alc_button_suppression_t1
No reviews yet
In the spirit of reconciliation, Audible acknowledges the Traditional Custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to their elders past and present and extend that respect to all Aboriginal and Torres Strait Islander peoples today.