If US AI leaders truly believed the US should do everything possible to ensure US AI dominance, they wouldn't advocate pacing the frontier.
Ultimately dying to AI will feel the same regardless of whether that AI speaks English or Chinese. In an ideal world, that would form a basis for cooperation.
The article is earnest and well argued - but from the Chinese perspective, has zero credibility once it asserts US attempts to hold back Chinese tech development is some how hallucinated (or perhaps a misunderstanding?).
Are you kidding me? China hasn't forgotten the attempt to exterminate Huawei, destroy the Chinese semiconductor industry in its entirety, "small yard / high fence", aggressive bans of Chinese users by Anthropic, etc, etc, etc.
Noted with interest that Ro Khanna, in his reached out to Chinese labs to discuss AI safety - how utterly delusional. He is the head of a congressional committee which has, for the past decade, not hidden its clear bipartisan primary mission of "beating China".
There is little to talk about - it will be competition to the bitter end.
I think we actually agree! Competition clearly is the reality here. But I also try to explain in the piece that certain PRC narratives simplify what different camps inside the United States believe and draw conflating equivalencies in order to create a simplified us-vs-them, which is bad news for safety coordination.
And not to mention this is one of the most blatantly racist regime in recent history. To believe they would agree to “compete” would be truly delusional.
Trump may be racist by the standards of US presidents. For example, he has made life difficult for illegal immigrants. But the US still has far more naturalized immigrants than China does.
Forget all of the policy language, marketing rhetoric, etc and observe actions.
One faction releases very little information about its models, and does not even generally speaking release older or smaller models, while charging significant prices and restricting access based on its judgments of the buyer.
A second faction releases its models almost immediately alongside research papers on the advances embedded in them, and licenses them so that they can be freely operated by all of humanity in perpetuity.
Only one of these behavior sets can plausibly be called authoritarian.
Yes, I would argue that the second approach is more “democratic”.
The most jarring part of this article for me is taking Amodei’s alleged sincerity at face value without any critical analysis. I would expect a journalist not to simply assume that just because someone is associated with an organization that has “altruism” in its name, it means they are altruistic. And even if one actually believes that EA is altruistic - or effective - how does that go with Amodei’s wife being in the Epstein files?
Exactly. And the predicate is that one faction, the one releasing models that can be freely operated by all of humanity in perpetuity, is contributing to a better world for humanity, allowing those who cannot afford the high prices to improve their own local businesses, infrastructure and development. The first faction, restricting access, will create a dystopian world where only Western elites, who have proven themselves completely unreliable and irresponsible, have access. I know which behavior set I support.
It’s not like PRC is unfamiliar or reluctant with policy crackdowns. The AI moratorium proponents should be studying how Beijing came to decide for action against COVID or against the many perceived political and economic threats, and ask what would persuade them about AI.
Wow. I spend a lot of time on these issues, yet rarely read a piece that not only explains, but makes the disconnect between the US and China palpable. I read Liu’s missive before this piece and agree that it submerges you in the mind of a community in a way that no pure policy commentary could ever do. There is also an interest shuffling of the pieces in DC where segments of left and right are converging on a deeply skeptical view of AI, while other groups, including many China hawks, essentially echo Liu’s concerns about a future world dominated by the frontier labs. Effectively engaging China on AI will be very difficult without even a barebones consensus in DC about what domestic AI policy should look like.
"Are the youngest of China’s AI researchers radicalizing themselves against cooperation with the United States — and against the Western-led enterprise of AI safety altogether......Not only do they seem to find a race to the bottom inevitable, but they have also internalized Western discussions of authoritarian AI threats as a coordinated attack on their right to technological advancement. Their trepidations around becoming footsoldiers of the national project are dissipating. The Party’s conspiracy is now the technologist’s own."
I find the whole paragraph troubling, probably because I don't know how representative Liu's view is. He feels like some of my high school friends who are extremely talented at STEM and interested in macro-geopolitical questions, but don't really think too much about other questions.
I also think the article is doing sth like connecting Liu/Chinese state against Anthropic with China against the US (EA) AI safety efforts, which seems a concerning connection. If anything, I think China's confusion and mistrust of Anthropic is somewhat fair, given that the American public is also quite confused and mistrustful of it. Rather than saying China is "radicalizing" wrongfully (which idk if it is that wrong), maybe it is good to say that the AI safety community, and especially some labs, need to actually think about how to express their earnest belief with more tact, so that people can actually understand them.
Reading Liu’s own response on Zhihu after his post’s going viral , I see exactly a high schooler talking big words. When people started to take that seriously and ask what he meant, he pivoted to play cute and posting his cross-dressing photos…
The education system simply does not support a true citizen with healthy world views. To have the math and tech talent and then have a healthy world view is like winning the power ball twice in a row.
According to Anthropic the leading Chinese labs are performing distillation on their model, the precursor to Fable, sending messages from Chinese users to Anthropic instead of replying themselves and using the message and answer to train their own models. I am sure some of these users are members of the government. But this is ok because Anthropic is evil. Uh huh. rationalization at its finest. I do like open-source models and run just about anything under 35B parameters. Right now I prefer orinth1.5:35B, which is based on Qwin3.6, running on Qllama with Letta as an agent harness. Really sweet. Runs fine on an old RTX3060ti with 8GB VRAM. Faster than I can read. What is really refreshing is that it can say "I don't know ..."
Mark Anderson made an interesting point that AI will always mimic human behavior because it was trained on the sum total of human behavior. Most of us know that humans are a mix of Mother Teresa and Charles Manson. That Chinese researcher is a fool because he is saying that he is willing to take the chance that Charles Manson is in charge.
Hmm. MA and his "sum total" - hardly the sum total, as the rather large non-Western corpus is excluded from training data for LLMs trained here.
The Deepseek R1 paper is a gold mine. The long path to success described there is littered with rejections of insufficient techniques they hoped to employ, until they settled on the best fit for their aims. They had necessarily trained R1 on Chinese as well as English, since of course that was best for their intended user base, China, and because of that struggled to get their innovative Chain of Thought implementation from reverting to Chinese occasionally to "explain" its "thinking", finally achieving pure English description -- at a slight quality cost. :-)
I was nodding along as I read Dario's pacing manifesto until I hit the record-scratch phrase "authoritarian governments." It was like a charge planted in his plan to self-sabotage Step 3. You can't get someone to come to the table by calling them names you know they object to.
What a prime example of gaslighting. Apparently throwing Meng into house arrest for a sanction that was unilaterally imposed by the USA, the chip embargo and the wilful suppression of Chinese tech, and going back into history the opium wars and the foreign concessions in China should not lead to distrust from Chinese citizens. Apparently the Chinese are radical to distrust countries that have and are still oppressing them in modern times
A data point from this week that fits the pattern described here: After CNN reported that an AI-assisted intelligence report nearly led US forces to board a Chinese ship, Global Times picked it up within a day and syndicated across the major portals. The framing was not "American AI is dangerous". It was that the ship was nearly "framed", and the piece closed by relaying netizens who argue the US is blaming AI to cover long-standing intelligence failures, with Iraq WMD and the al-Shifa strike as the precedents. The search for the motive behind the stated one, again.
The audience signal in footnote 9 applies in reverse as well. As far as TTW could find, Global Times ran this in Chinese only, with no English edition pickup so far, which suggests the first use was domestic. Whether that changes once Xi is in Washington this week is the thing to watch.
What I found most interesting is how quickly the AI race is blurring the line between competition and imitation. The real strategic advantage may increasingly belong to whoever can absorb and improve on new capabilities fastest.
Very helpful article -- it shows how deep the mistrust and misunderstanding is on both sides. Difficult to see how we can bridge that divide - a follow-up article with possible solutions would be welcome.
US zero sum thinking ("[the US] should do everything possible to ensure dominance in this technology") = 'earnest', 'not... difficult to imagine'
Chinese zero sum thinking = 'radicalizing'
do you even hear yourself?
If US AI leaders truly believed the US should do everything possible to ensure US AI dominance, they wouldn't advocate pacing the frontier.
Ultimately dying to AI will feel the same regardless of whether that AI speaks English or Chinese. In an ideal world, that would form a basis for cooperation.
The article is earnest and well argued - but from the Chinese perspective, has zero credibility once it asserts US attempts to hold back Chinese tech development is some how hallucinated (or perhaps a misunderstanding?).
Are you kidding me? China hasn't forgotten the attempt to exterminate Huawei, destroy the Chinese semiconductor industry in its entirety, "small yard / high fence", aggressive bans of Chinese users by Anthropic, etc, etc, etc.
Noted with interest that Ro Khanna, in his reached out to Chinese labs to discuss AI safety - how utterly delusional. He is the head of a congressional committee which has, for the past decade, not hidden its clear bipartisan primary mission of "beating China".
There is little to talk about - it will be competition to the bitter end.
I think we actually agree! Competition clearly is the reality here. But I also try to explain in the piece that certain PRC narratives simplify what different camps inside the United States believe and draw conflating equivalencies in order to create a simplified us-vs-them, which is bad news for safety coordination.
And not to mention this is one of the most blatantly racist regime in recent history. To believe they would agree to “compete” would be truly delusional.
But Trump and his administration are the ones who DISMISS the AI safety concern.
Trump may be racist by the standards of US presidents. For example, he has made life difficult for illegal immigrants. But the US still has far more naturalized immigrants than China does.
https://ourworldindata.org/grapher/migrant-stock-share
https://www.washingtonpost.com/blogs/worldviews/files/2013/05/racial-tolerance-map-hk-fix.jpg
Forget all of the policy language, marketing rhetoric, etc and observe actions.
One faction releases very little information about its models, and does not even generally speaking release older or smaller models, while charging significant prices and restricting access based on its judgments of the buyer.
A second faction releases its models almost immediately alongside research papers on the advances embedded in them, and licenses them so that they can be freely operated by all of humanity in perpetuity.
Only one of these behavior sets can plausibly be called authoritarian.
Yes, I would argue that the second approach is more “democratic”.
The most jarring part of this article for me is taking Amodei’s alleged sincerity at face value without any critical analysis. I would expect a journalist not to simply assume that just because someone is associated with an organization that has “altruism” in its name, it means they are altruistic. And even if one actually believes that EA is altruistic - or effective - how does that go with Amodei’s wife being in the Epstein files?
Exactly. And the predicate is that one faction, the one releasing models that can be freely operated by all of humanity in perpetuity, is contributing to a better world for humanity, allowing those who cannot afford the high prices to improve their own local businesses, infrastructure and development. The first faction, restricting access, will create a dystopian world where only Western elites, who have proven themselves completely unreliable and irresponsible, have access. I know which behavior set I support.
It’s not like PRC is unfamiliar or reluctant with policy crackdowns. The AI moratorium proponents should be studying how Beijing came to decide for action against COVID or against the many perceived political and economic threats, and ask what would persuade them about AI.
Wow. I spend a lot of time on these issues, yet rarely read a piece that not only explains, but makes the disconnect between the US and China palpable. I read Liu’s missive before this piece and agree that it submerges you in the mind of a community in a way that no pure policy commentary could ever do. There is also an interest shuffling of the pieces in DC where segments of left and right are converging on a deeply skeptical view of AI, while other groups, including many China hawks, essentially echo Liu’s concerns about a future world dominated by the frontier labs. Effectively engaging China on AI will be very difficult without even a barebones consensus in DC about what domestic AI policy should look like.
"Are the youngest of China’s AI researchers radicalizing themselves against cooperation with the United States — and against the Western-led enterprise of AI safety altogether......Not only do they seem to find a race to the bottom inevitable, but they have also internalized Western discussions of authoritarian AI threats as a coordinated attack on their right to technological advancement. Their trepidations around becoming footsoldiers of the national project are dissipating. The Party’s conspiracy is now the technologist’s own."
I find the whole paragraph troubling, probably because I don't know how representative Liu's view is. He feels like some of my high school friends who are extremely talented at STEM and interested in macro-geopolitical questions, but don't really think too much about other questions.
I also think the article is doing sth like connecting Liu/Chinese state against Anthropic with China against the US (EA) AI safety efforts, which seems a concerning connection. If anything, I think China's confusion and mistrust of Anthropic is somewhat fair, given that the American public is also quite confused and mistrustful of it. Rather than saying China is "radicalizing" wrongfully (which idk if it is that wrong), maybe it is good to say that the AI safety community, and especially some labs, need to actually think about how to express their earnest belief with more tact, so that people can actually understand them.
This.
Reading Liu’s own response on Zhihu after his post’s going viral , I see exactly a high schooler talking big words. When people started to take that seriously and ask what he meant, he pivoted to play cute and posting his cross-dressing photos…
The education system simply does not support a true citizen with healthy world views. To have the math and tech talent and then have a healthy world view is like winning the power ball twice in a row.
According to Anthropic the leading Chinese labs are performing distillation on their model, the precursor to Fable, sending messages from Chinese users to Anthropic instead of replying themselves and using the message and answer to train their own models. I am sure some of these users are members of the government. But this is ok because Anthropic is evil. Uh huh. rationalization at its finest. I do like open-source models and run just about anything under 35B parameters. Right now I prefer orinth1.5:35B, which is based on Qwin3.6, running on Qllama with Letta as an agent harness. Really sweet. Runs fine on an old RTX3060ti with 8GB VRAM. Faster than I can read. What is really refreshing is that it can say "I don't know ..."
Mark Anderson made an interesting point that AI will always mimic human behavior because it was trained on the sum total of human behavior. Most of us know that humans are a mix of Mother Teresa and Charles Manson. That Chinese researcher is a fool because he is saying that he is willing to take the chance that Charles Manson is in charge.
Hmm. MA and his "sum total" - hardly the sum total, as the rather large non-Western corpus is excluded from training data for LLMs trained here.
The Deepseek R1 paper is a gold mine. The long path to success described there is littered with rejections of insufficient techniques they hoped to employ, until they settled on the best fit for their aims. They had necessarily trained R1 on Chinese as well as English, since of course that was best for their intended user base, China, and because of that struggled to get their innovative Chain of Thought implementation from reverting to Chinese occasionally to "explain" its "thinking", finally achieving pure English description -- at a slight quality cost. :-)
I was nodding along as I read Dario's pacing manifesto until I hit the record-scratch phrase "authoritarian governments." It was like a charge planted in his plan to self-sabotage Step 3. You can't get someone to come to the table by calling them names you know they object to.
Everything can be discussed. But be sure you are not discussing or making agreement with someone like CCP and its controlled land.
What a prime example of gaslighting. Apparently throwing Meng into house arrest for a sanction that was unilaterally imposed by the USA, the chip embargo and the wilful suppression of Chinese tech, and going back into history the opium wars and the foreign concessions in China should not lead to distrust from Chinese citizens. Apparently the Chinese are radical to distrust countries that have and are still oppressing them in modern times
A data point from this week that fits the pattern described here: After CNN reported that an AI-assisted intelligence report nearly led US forces to board a Chinese ship, Global Times picked it up within a day and syndicated across the major portals. The framing was not "American AI is dangerous". It was that the ship was nearly "framed", and the piece closed by relaying netizens who argue the US is blaming AI to cover long-standing intelligence failures, with Iraq WMD and the al-Shifa strike as the precedents. The search for the motive behind the stated one, again.
The audience signal in footnote 9 applies in reverse as well. As far as TTW could find, Global Times ran this in Chinese only, with no English edition pickup so far, which suggests the first use was domestic. Whether that changes once Xi is in Washington this week is the thing to watch.
What I found most interesting is how quickly the AI race is blurring the line between competition and imitation. The real strategic advantage may increasingly belong to whoever can absorb and improve on new capabilities fastest.
Very helpful article -- it shows how deep the mistrust and misunderstanding is on both sides. Difficult to see how we can bridge that divide - a follow-up article with possible solutions would be welcome.