In downtown Berkeley, an old hotel has become a temple to the pursuit of artificial intelligence (AI) and the future of humanity. Its name is Lighthaven.
Covering much of a city block, this gated complex includes five buildings and a small park dotted with rose bushes, stone fountains and neoclassical statues.
Stained-glass windows glisten on the top floor of the tallest building, called Bayes House after an 18th-century mathematician and philosopher.
Lighthaven is the de facto headquarters of a group of people who call themselves Rationalists.
This group has many interests involving mathematics, genetics and philosophy.
One of their overriding beliefs is that AI can deliver a better life – if it does not destroy humanity first.
The Rationalists believe it is up to the people building AI to ensure that it is a force for the greater good.
They were talking about AI risks years before OpenAI created ChatGPT, which brought AI into the mainstream and turned Silicon Valley on its head.
Their influence has quietly spread through many tech companies, from industry giants like Google to AI pioneers like OpenAI and Anthropic.
Many of the AI world’s biggest names – including Dr Shane Legg, co-founder of Google’s DeepMind; Anthropic’s chief executive, Dr Dario Amodei; and Dr Paul Christiano, a former OpenAI researcher who now leads safety work at the US Centre for AI Standards and Innovation – have been influenced by Rationalist philosophy. Mr Elon Musk, who runs his own AI company, said that many of the community’s ideas align with his own.
Mr Musk met his former partner, pop star Grimes, after they made the same cheeky reference to a Rationalist belief called Roko’s Basilisk.
This elaborate thought experiment argues that when an all-powerful AI arrives, it will punish everyone who has not done everything they can to bring it into existence.
But these tech industry leaders stop short of calling themselves Rationalists, often because that label has over the years invited ridicule.
The Rationalist community is tightly entwined with the Effective Altruism movement, which aims to remake philanthropy by calculating how many people would benefit from each donation. This form of utilitarianism aims to benefit not just people who are alive today, but also all the people who will ever live.
Many Effective Altruists (EA) have decided that the best way to benefit humanity is to protect it from destruction by AI.
Hellbound and Down 