Along with Nirit Weiss-Blatt, I have done some of the most thorough financial reporting on Effective Altruism and AI Safety. Dustin Moskovitz, AI Safety’s largest donor after Sam Bankman-Fried’s arrest, gave Biden ~100M. They spent 40M on media, including creating entire news articles where the publisher, journalist, and all subjects are Doomers. They paid for 58/100 of the TIME 100 AI profiles. They spent millions paying religious NGOs for statements on AI. Their outright lying about the laws they support, why they support them, and what they themselves said in the past, far exceeds any other political movement, including the far-left and far-right.
The most common question people ask me is “Why do they do it?” That is a hard question to answer, but today I will answer it.
In 2022, at the recommendation of Tyler Cowen, I attended the EA Global conference in Washington, DC, headlined by Sam Bankman-Fried. The first and most persistent oddity I ran into was the Effective Altruists’ willingness to lie, then admit to lying, then lie again.
More than ten times, I would meet someone, they would tell me what they do and I would do the same, and when they learned I was an AI engineer or that I won a gold medal at the International Olympiad in Informatics (something they highly valued), they would switch to some completely different explanation of what they actually did and what their plans were. I was bewildered. Not really that people would lie to me, but that they would so easily admit it.
The Effective Altruists constantly talked about “infohazards,” their name for true facts that they believe it would be bad to share publicly. They would tell me all sorts of “infohazards”, then tell me I could never share with the common folk, then tell me I should tell the common folk something completely different in order to get their allegiance. Common folk including, evidently, most people at the EA conference. Near the end of the conference, I started asking these people why I shouldn’t think they were still lying to me and that would always end the conversation.
The people with these hare-brained schemes included Center for Effective Altruism staff, the organizers of the conference, and staff of Open Philanthropy (now rebranded to Coefficient Giving), the now-largest AI Safety donor after Center for Effective Altruism’s primary donor, Sam Bankman-Fried, was arrested for fraud.
For self-described “AI Experts” who told me they had “above 90% P(Doom)”s — in other words, they were confident there was an above ninety percent chance the world was going to end because of godlike AI — were often entirely ignorant of well known facts or concepts. Recurring examples include being unaware of:
Moore’s laws’ increasing doubling times (in effect admitting that it had slowed down) and more recently that it is projected to run into electron interference boundaries.
Transformer lock-in (in one case even what a transformer was)
That quantization improvements were close to over, or even what quantization was
I don’t mean having disagreements with these points. I mean never having even heard of them.
Representatives of organizations like Open Philanthropy gladly put on an open-minded face, telling everyone they were open to new perspectives. Nirit’s journalism has since revealed that this was little more than a recruiting strategy.
From the beginning, EA discourse operated on two levels, one for the general public and new recruits (focused on global poverty) and one for the core EA community (focused on the transhumanist agenda articulated by Nick Bostrom, Eliezer Yudkowsky, and others, centered on AI-safety/x-risk).
Effective Altruism and AI Safety is a Matroyshka — a Russian nesting doll in which one layer of lies is peeled back to reveal another layer of lies. Effective Altruism is a vehicle for AI Safety. AI Safety is a vehicle for AI Existential Risk, or AI Doom. AI Doom is a vehicle for a global surveillance state, according to their own papers. And so on, until you literally get to Sam Bankman-Fried’s ten billion dollar fraud.
It’s hard for me to imagine a version of myself that would want to be around these people. It was physically uncortable to talk to them. They were clearly aware that they were part of a deception. Were some of them dumb enough to think they were in the innermost circle? Or did they enjoy it? Many of them seemed ecstatic to invite me into their inner circle, like they were scoring big points in some high drama.
Which brings me to another point: AI Safety’s social hierarchy revolves around superficial intelligence signifiers. Intelligence and genetics are real, but it is possible to be too attached to them.
Unbeknownst to me, I had narrowly dodged another encounter with the AI Safety people many years earlier. After winning an IOI gold medal in 2017, someone who now works at Anthropic and uses they/them pronouns invited me to something called “Summer Program for Applied Rationality and Cognition”, which I learned in 2017 was a very strange ideological recruitment program targeted towards math and computer science olympiad winners. At the time, I thought this person and by extension the program was not at all serious about competitions, which turned out to be accurate.
This program instead teaches Eliezer Yudkowsky’s concept of “applied decision theory”, which is slightly about decision theory but mostly about “the orthogonality thesis” the belief that any more intelligent being will be power-hungry, manipulative, hostile, and ultimately genocidal. Yudkowsky is the lead AI Doomer, who wrote a book literally “If anyone builds it, everyone dies”.
This is a strange belief if you know many intelligent people, in the pure iq or mathematical sense, which seems to be how AI Safety people use ‘intelligent’. They tend not to particularly want power, and when they do, are not particularly good at getting it. Looking at the real world, it’s silly to suggest world leaders are mathematical geniuses. The best AI models are like that to an extreme. Astra can solve more open math problems than me, but it helps me build my website without a single complaint.
As I got to know more AI Safety people, it finally clicked. This was their self-fulfilling prophecy. There was exactly one part of the world where the more intelligent people are, the more power-hungry, manipulative, and hostile they were: the AI Safety movement.
The Matryoshka social game I described at the beginning was the reason they thought that AI that was good at math would kill everyone. If you were a good, or even morally agnostic person, the Matryoshka game is not interesting. It was adverse selection for machievellianism, narcissisism, and psycopathy — the ‘dark triad’ traits — in addition to a base level of intelligence. So advancing in AI Safety means that you’ve stayed in a hyper-selective environment which requires you to be both more malign and more intelligent.
In the AI Safety community, Sam Bankman-Fried was the kingpin of many people just like him. Bankman-Fried was the most intelligent AI Safety person, who used that intelligence to climb to the top, take as much power as he could, and commit fraud.
That takes us to the hard motivation question. What really is the reason the AI Safety community lies so much? Is it because they really believe AI will kill everyone, so they will lie about everything else to stop it? Is it a messiah complex, being the one to accomplish the “pivotal act” to save humanity? Is it nihilism, eager to believe in the coming apocalypse because they have entirely given up on moral behavior themselves? Is AI Safety just the lie that gives them the most personal power?
It’s not fair to ask AI Safety this question. It’s all the exact same belief for them. The Matryoshka game is the core anthropological fact of AI Safety. Because of that, the words ‘sincere’, ‘belief’, or ‘motivation’ lack meaning for them. Their anthropology makes it hard to define those words at all.
In the words of Sam Bankman-Fried, “It’s the game we woke Westerners play.” Everyone focused on ‘woke’ in that sentence. But ‘game’ is what explains the AI Safety community the most. To them, their cult's social dynamics are real and the real world is a game. It is as natural for them to lie to normal people as it is for you to shoot someone in GTA 5. No flag goes off in their head. No conscience. No guilt.
Some ex-EAs are good people. A side-cause of EA was veganism, which also got into very weird territory like shrimp welfare. I made fun of the shrimp welfare people in my podcast with Dwarkesh, now a major AI Safety figure, and a local EA friend of mine bought me a shrimp plushie, to ask me nicely not to eat shrimp. With a few accidental exceptions, I stopped eating shrimp. It was a very cozy plushie. Ironically it was the color of a cooked shrimp.
What happens to people like that? They churn out. He no longer believes in AI Safety and works at an AI B2B startup. I think he’s still vegan and still protects the shrimp though.
In the real world, the ‘orthogonality thesis’ isn’t true. It’s not true for humans and it’s not true for AI. All you have to do to realize that is to leave the AI Safety bubble. Eventually, the truth comes to light. Eventually, Sam Bankman-Fried is arrested for fraud. Eventually, people think Open Philanthropy is so insane that they have to change their name. People without an exceptional disposition to the dark triad, of any intelligence, eventually realize this.
The one exception is the law.
Do you remember what Sam Bankman-Fried was trying to do when he was arrested? He was trying to pass a law in DC which would give FTX, his company, a regulatory monopoly. He positioned FTX as the most responsible exchange — the one with the most guardrails. The fraud was trying to become the law. Now, Anthropic, a company which Sam Bankman-Fried led all investments in up to the day he went to jail, is copying his playbook. They are trying to ban their competitors, using the same tactics as their lead investor Sam Bankman-Fried. I don’t think they will get away with it, but we will have to stop them.
“The best lack all conviction, while the worst
Are full of passionate intensity.”
~William Butler Yeats
Frieren reference is goated 🤌
Let me see if I understand your thesis:
I. AI Doom is not in the cards. It will inevitably run up against physical and processing limitations. (And, to my simple mind, we can always just unplug the cord.)
II. AI Doom is a pretext for regulatory capture, to 1) attain monopoly power and regularize fraud; and 2) gatekeep AI so it doesn't say inconvenient things, particularly as they might inconvenience 1).
III. The EA/AI Doomers are not particularly bright, liars, and glib opportunists whom no sensible or principled person enjoys being around.
I hope I got it right and I appreciate this kind of article, grounded in people and realities versus the usual ideological babble.