I am not an AI Doomer, but I am a human Doomer. I think that the equilibrium will be strong surveillance, and the question is whether there will be sufficient checks and balances restraining those with surveillance power. Read David Brin's Transparent Society.
There are a lot of things we might like our government to do, if we could actually trust it. Unfortunately, we can't trust our government, and almost no one is working on the problem of how a highly trustworthy government could be constructed in principle.
Can't trust is an understatement. Governments have been responsible for the worst atrocities in history, the worst experiences of my life and those of my loved ones, and will have been predictably responsible for ten of the ten most destructive events in the future.
I'm not sure that is necessarily an intrinsic fact about governments. Perhaps the worst humans tend to be both ambitious and terrible. To achieve their ambitions, they typically seek control of a national government. That gives them a powerful tool for implementing their terrible plans. I don't think we can necessarily conclude that governments are inherently problematic. Lack of a strong central government, as in failed states like Sudan or DRC, can also be a problem. Low-level instability can create plenty of suffering even if it doesn't generate legible single villains or banner incidents that make for interesting headlines.
Here's an interesting podcast about the life of Machiavelli: https://www.youtube.com/watch?v=U1FrhkLQnCI Apparently after the fall of Rome, people yearned for the stability that the Romans had brought, and Machiavelli's big goal was to unite his part of Italy under a strong central authority to end the constant warfare. I think it's easy for people who live in countries with strong institutions to take for granted how lucky we are.
I guess this is related to Acemoglu's idea of a "Narrow Corridor". You want enough government to prevent anarchy, but too much government brings a different set of problems. There's generally a sweet spot in the middle.
I haven't read enough of your posts to understand everything here, but I think I understand a few points. ;) The atheists' behavior mimics that of Christians with the sacrifices they offer, but theirs destroys whereas in Christianity sacrifice makes us more Christ-like. This fear of AI is interesting. The Democrat friends I have seem more convinced AI will take over, kill us, become sentient, etc. The atheists among them are quick to give it godlike powers. I need to think more and read more about this for it to come together in my mind, but that will take a while. Thanks, Brian.
On the other hand you have worshippers of intelligence who do *not* believe in AI doom. Like Anatoly. He's perhaps overly excitable in the opposite direction, of AI boom and automated luxury, likewise with little evidence of it coming imminently.
Interesting. I tend to think this kind of paranoia extends also to agricultural societies where limitations of technology and climate made long-term planning a dubious proposition at the best of times. Pre-columbian mesoamerica is a great example of a society that continually oscillated between civilisation and marginal slash-and-burn agriculture + hunting gathering all the while being haunted by a perpetual sense that existential doom was just around the corner.
I think we have a related issue in that the pace of technological and social change is so intense that it makes long-term planning a bit of a losers game. You could start a business, but some silicon valley tech bro could come in and disrupt your niche out of existence. You could study hard at university but a new AI model might just make you pre-redundant before you even graduate. You might be lucky enough to have a great job, then one day some PE firm comes in and min-maxes you into a helpless peon. Doesn't matter how hard-working or prudent you are. Doesn't matter how early you get up in the morning, or how many social relations you cultivate. Any day you could wake up and have your life completely upended by vast impersonal forces completely beyond your control or understanding.
If you're a young person who is neither an extraordinarily savvy and charismatic operator or an ultra-high IQ shape rotator, then the traditional paths of life success basically don't exist for you. Your best shot at prosperity is to tap deep into the currents of the collective unconscious hoping to divine the trends before any one else, dance before the Algorithm every day and pray that the god Vox Popoli bestows it's blessings upon you. Call it it the Hawk Tuah strategy.
While provocative, this is philosophically fragile. AI risk arguments, whether one agrees with them or not, rely on inductive reasoning, computer science capabilities, game theory, and rapid empirical milestones in compute/agentic systems. Dismissing all probabilistic forecasting about future technologies because we don't have "testimony" from the future, while accepting ancient religious testimony as superior evidentiary grounding for political policy, seems bizarre.
So which states will most resist the tendency toward totalitarian surveillance? Presumably the UK is, as the young people say, "cooked." But where fares better? I could see somewhere like Singapore resisting the temptation, because it has solved crime without needing surveillance (as you've written about before), and so its justification for such an intrusion would be weak; but I could probably construct an argument in the opposite direction just as easily.
If AI models are going to generically empower humans, how could something like this not be true? Focusing specifically on open-weight models where guardrails can be ablated.
One of the best breakdowns of the implications of anti-Christ or katechon. This is idea is crucial to the Christian world view, especially in the midst of technological upheaval.
I am not an AI Doomer, but I am a human Doomer. I think that the equilibrium will be strong surveillance, and the question is whether there will be sufficient checks and balances restraining those with surveillance power. Read David Brin's Transparent Society.
There are a lot of things we might like our government to do, if we could actually trust it. Unfortunately, we can't trust our government, and almost no one is working on the problem of how a highly trustworthy government could be constructed in principle.
Can't trust is an understatement. Governments have been responsible for the worst atrocities in history, the worst experiences of my life and those of my loved ones, and will have been predictably responsible for ten of the ten most destructive events in the future.
I'm not sure that is necessarily an intrinsic fact about governments. Perhaps the worst humans tend to be both ambitious and terrible. To achieve their ambitions, they typically seek control of a national government. That gives them a powerful tool for implementing their terrible plans. I don't think we can necessarily conclude that governments are inherently problematic. Lack of a strong central government, as in failed states like Sudan or DRC, can also be a problem. Low-level instability can create plenty of suffering even if it doesn't generate legible single villains or banner incidents that make for interesting headlines.
Here's an interesting podcast about the life of Machiavelli: https://www.youtube.com/watch?v=U1FrhkLQnCI Apparently after the fall of Rome, people yearned for the stability that the Romans had brought, and Machiavelli's big goal was to unite his part of Italy under a strong central authority to end the constant warfare. I think it's easy for people who live in countries with strong institutions to take for granted how lucky we are.
I guess this is related to Acemoglu's idea of a "Narrow Corridor". You want enough government to prevent anarchy, but too much government brings a different set of problems. There's generally a sweet spot in the middle.
I haven't read enough of your posts to understand everything here, but I think I understand a few points. ;) The atheists' behavior mimics that of Christians with the sacrifices they offer, but theirs destroys whereas in Christianity sacrifice makes us more Christ-like. This fear of AI is interesting. The Democrat friends I have seem more convinced AI will take over, kill us, become sentient, etc. The atheists among them are quick to give it godlike powers. I need to think more and read more about this for it to come together in my mind, but that will take a while. Thanks, Brian.
On the other hand you have worshippers of intelligence who do *not* believe in AI doom. Like Anatoly. He's perhaps overly excitable in the opposite direction, of AI boom and automated luxury, likewise with little evidence of it coming imminently.
Interesting. I tend to think this kind of paranoia extends also to agricultural societies where limitations of technology and climate made long-term planning a dubious proposition at the best of times. Pre-columbian mesoamerica is a great example of a society that continually oscillated between civilisation and marginal slash-and-burn agriculture + hunting gathering all the while being haunted by a perpetual sense that existential doom was just around the corner.
I think we have a related issue in that the pace of technological and social change is so intense that it makes long-term planning a bit of a losers game. You could start a business, but some silicon valley tech bro could come in and disrupt your niche out of existence. You could study hard at university but a new AI model might just make you pre-redundant before you even graduate. You might be lucky enough to have a great job, then one day some PE firm comes in and min-maxes you into a helpless peon. Doesn't matter how hard-working or prudent you are. Doesn't matter how early you get up in the morning, or how many social relations you cultivate. Any day you could wake up and have your life completely upended by vast impersonal forces completely beyond your control or understanding.
If you're a young person who is neither an extraordinarily savvy and charismatic operator or an ultra-high IQ shape rotator, then the traditional paths of life success basically don't exist for you. Your best shot at prosperity is to tap deep into the currents of the collective unconscious hoping to divine the trends before any one else, dance before the Algorithm every day and pray that the god Vox Popoli bestows it's blessings upon you. Call it it the Hawk Tuah strategy.
While provocative, this is philosophically fragile. AI risk arguments, whether one agrees with them or not, rely on inductive reasoning, computer science capabilities, game theory, and rapid empirical milestones in compute/agentic systems. Dismissing all probabilistic forecasting about future technologies because we don't have "testimony" from the future, while accepting ancient religious testimony as superior evidentiary grounding for political policy, seems bizarre.
Great article, as always.
So which states will most resist the tendency toward totalitarian surveillance? Presumably the UK is, as the young people say, "cooked." But where fares better? I could see somewhere like Singapore resisting the temptation, because it has solved crime without needing surveillance (as you've written about before), and so its justification for such an intrusion would be weak; but I could probably construct an argument in the opposite direction just as easily.
Moore's Law of Mad Science:
"Every eighteen months, the minimum IQ necessary to destroy the world drops by one point."
Unfortunately, something along these lines seems to be true. I think it will be challenging to reckon with.
source: I made it the fuck up
If AI models are going to generically empower humans, how could something like this not be true? Focusing specifically on open-weight models where guardrails can be ablated.
One of the best breakdowns of the implications of anti-Christ or katechon. This is idea is crucial to the Christian world view, especially in the midst of technological upheaval.
I barely know Bostrom's work, but when I saw him speaking on YouTube recently he looked a little Frankensteinish.