The PLA on Generative AI + NYC Meetup!
The PLA grapples with the impact of ChatGPT on “cognitive warfare.” Plus: how to combat a Color Revolution
Kevin Xu and I are hosting a meetup in New York on Friday, June 2. RSVP here!
Since the beginning of this year, Chinese state media has been reporting widely about ChatGPT and developments in generative AI. This includes the military press — a collection of newspapers and magazines owned by the People’s Liberation Army.
Save for the occasional viral article weighing in on trending issues (like the recent comedy scandal), these publications are not widely read by the Chinese public. They are, however, important training and educational material for PLA members and associates, and thus useful for glimpsing the discourse happening within the country’s defence community.
Judging by the coverage in PLA publications, the Chinese military is quite preoccupied by how generative AI will be used to wage “cognitive warfare.” Below are some excerpts translated from articles in PLA-owned publications. They discuss:
How the US’s pursuit of tech supremacy is linked to the exportation of its values;
Why current Western advantages in generative AI is bad news for securing China’s information fortress;
And how the PLA itself hopes to use emerging technologies to manipulate public opinion in wartime.
ChinaTalk is a reader-supported publication. To receive new posts and support our work, consider becoming a free or paid subscriber.
Closing the Deepfake Gap
Wang Wenbo 王文博, an opinion columnist, wrote the following piece for Jun Zhengping Studio on April 14, 2023. Jun Zhengping is the PLA’s public-facing social media brand for its publications. Wang’s article links US technology export controls with Chinese historical grievances and wariness over “color revolutions,” offering insight into the various strains of anxiety swirling around Chinese discourse.
From the moment the US Central Intelligence Agency announced that it would use ChatGPT and similar generative artificial intelligence technology for intelligence work, it has almost put on the table its nefarious intentions of using AI technology against other countries. This country, which excels in starting “Twitter revolutions” and “color revolutions” with false information flows, can’t help but drool over the deep counterfeiting abilities of generative AI technology. The CIA’s head of AI, Lakshmi Raman, has prepared a perfect excuse: keeping up with competitors like China, Russia, and Iran, who may all be developing AI to counter us.
There have been precedents for challenges to national security and interests. During the tense situation between India and Pakistan in 2019, Reuters released thirty deepfake videos about the incident, greatly provoking nationalist sentiments in both countries and leading to further escalation of the situation. [Irene: This is a curious piece of misinformation. Reuters had identified thirty deepfakes in 2019 and reported on them; they did not “release” (发布) them to provoke the situation. This fact was cited accurately in other Chinese outlets (发现, discover), which makes me suspect that Wang intentionally changed the wording to prove his point about Western media.]
In 1937, when Japanese invaders advanced all the way to Lugou Bridge, the people of China suddenly “woke up.” After the defense line of national security and interests was broken, people reflected that among the deadly “siege hammers,” there was the reality of missing the era of mechanization brought about by steam engines, internal combustion engines, and generators. History has proven that once a “generation gap” in technology forms between countries, those who lag behind and are controlled by others will inevitably be “beaten.” This is a lesson from China’s century of humiliation in modern history.
Psychology, Values, and ‘Subduing the Enemy Without Fighting’
Mao Weihao 毛炜豪 is a military strategy researcher who teaches at the PLA’s Army Command College. He penned the following piece for the PLA Daily, the Chinese armed forces’ flagship newspaper, on April 13, 2023. He clearly implies that the US will use its LLM advantage to wage cognitive warfare against others. The SCMP reported on this piece back in April.
In the era of informatization and intelligence, the degree of digitalization is generally high in all countries. This implies that the speed of information exchange, viewpoint transmission, and emotional contagion among the public is faster, which also means that the space for conducting cognitive offense and defense is larger. ChatGPT’s powerful natural language processing capabilities can be used to quickly analyze public sentiment, extract valuable information, or create false statements to disrupt public emotions.
Moreover, by employing subtle and complex tactics in cognitive warfare, it can induce, deceive, and even manipulate the cognition of people in the target country to damage its government’s image, change the people’s stance, and even divide society and overthrow the government, thereby realizing the strategy of “subduing the enemy without fighting” [Ed.: A quote from Sun Tzu’s The Art of War].
Reportedly, the natural language processing technology used by ChatGPT is precisely the technology that the US military is focusing on in its Joint All-Domain Command and Control (JADC2) concept. On July 1, 2020, the RAND Corporation’s Project AIR FORCE published a report titled “Joint All-Domain Command and Control for Modern Warfare: An Analytic Framework for Identifying and Developing Artificial Intelligence Applications.” The report suggests that AI technologies can be divided into six categories. Among them, natural language processing technology has a clear application in JADC2 — it can be used to extract intelligence from speech and text and transmit relevant information to squad commanders and even individual soldiers, alerting them to potential conflicts or opportunities.
Researchers Huang Yanlong 黄彦龙, Wu Qiong 吴穹, and Jiang Rilie 蒋日烈 from the National University of Defense Technology — a leading military technology institution directly affiliated with the Central Military Commission — expand upon how generative AI may be used to wage cognitive warfare in this next piece from March 21, also in the PLA Daily. The piece doesn’t specify whether the PLA hopes to use these techniques or expects such tactics to be used against China.
Intelligent algorithms, through comprehensive analysis and computation of a user’s personal identity, psychological characteristics, behavioral habits, and interests, can devise the optimal information recommendation strategy. They can then use specific information to affect the target’s cognition, ultimately influencing their real-world behavior.
Drawing a user portrait. The essence of a user portrait is to label the user, thereby forming a specific identity. Labels are like “pixels” in a digital portrait. The more comprehensive the data acquisition, the finer the label description, and the more accurately it reflects the user’s behavioral characteristics. Foreign militaries believe that in the realm of cognitive warfare, technologies such as web crawlers and log mining can be widely used to obtain basic information data, financial data, communication data, etc., of the target object. Based on this, intelligent algorithms can deeply mine these data and systematically analyze the target’s interests, behavioral habits, and interpersonal relationships, thereby determining their value orientation. Afterward, by constructing multidimensional label vectors and clustering similar individuals, the characteristics of groups with similar features can be analyzed to form a group target portrait, which then serves as the foundation and starting point for shaping cognition.
Achieving information personalization. Targeting different individuals and groups with a “divide and conquer” approach is the strength of intelligent algorithms, and it’s also a vital aspect of cognitive shaping. Relying on social media platforms and search engines to push personalized content toward specific targets can greatly increase their receptiveness, avoiding inefficiencies and internal waste in the information transmission process. In this process, the “information bubble” effect will further narrow the target’s perceptual range, causing them to remain in a relatively closed information environment for a long time, thus actively accepting one-sided information. At the same time, homogeneous information will further enhance the shaping effect. Under the combined effects of conformity psychology and group pressure, the target may gradually lose their basic logical and value judgment abilities, thereby subtly forming the cognition meticulously set by the initiator.
Demonstrating specific real-world behaviors. Cognition is the premise of behavior — and behavior, in turn, influences cognition. Intelligent algorithms can influence and guide real-world behavior by specifically shaping the political, military, and social viewpoints and value tendencies of specific targets. Foreign militaries believe that disseminating particular ideologies to the public can lead to social disorder and chaos.
For key individuals, strategies like mental control and heart-winning can cause them to misjudge the national strategic direction, feel skeptical about war decisions, and become pessimistic about the course of the war, thus achieving the effect of “subduing the enemy without fighting.”
Experience has proven that the targeted reshaping of cognition demonstrates specific real-world behaviors. The dynamic changes in real-world behaviors prompt real-time adjustments of user portraits, which in turn lead to updates and adjustments in recommendation strategies, forming a complete closed feedback loop.
Then, on April 4 in the PLA Daily, Chen Dongheng 陈东恒 and Xu Yan 许炎 of the PLA Academy of Military Sciences’ Political Work Research Institute discussed some generative-AI-powered cognitive techniques that may be employed against China more explicitly, with interesting observations about ChatGPT.
No matter how quickly generative AI iterates, it is ultimately an AI created and controlled by humans. Behind its dazzling functionalities is the manipulation of its developers, who utilize the design of secret keys and training datasets they hold to control the cognitive stance and attitude output of the machine. At the same time, there is also the possibility of significant risk and challenges posed by machine cognition failures or loss of control. Information shows that OpenAI has spent a great deal of effort on debugging English data rather than Chinese content. Chinese data accounts for only 0.1% of the training corpus in ChatGPT, and the logic and accuracy of English output results are far greater than Chinese. Thus, its cultural inclinations are obvious. As generative AI and its products with distinct political stances and value orientations increasingly penetrate people’s lives, it’s hard for the audience’s cognition not to be influenced by the preset political stance of the answers it provides. Thus, this system is even more susceptible to becoming a tool for cognitive offense and defense.
The rapid development of generative AI and its extensive application in the field of cognitive offense and defense represent a major trend in the development of future cognitive warfare. This requires us to delve deeply into the cognitive applications of generative AI and understand the methodologies through which it functions.
Shaping cognitive habits: This involves enhancing the convenience of using superior technological products and forming habitual thinking through long-term exposure to that product. The goal is to achieve trustworthy, relatable, perceptible, and unsuspecting cognitive convergence, which is a traditional practice of the United States and some other Western countries. In the early years of developing the international internet and operating systems, they shaped the market by allowing low-cost or free usage, getting competitors unknowingly “hooked” on their platforms.
ChatGPT appears as a “confidant” or a “little assistant” at work, and even offers a free version. These seemingly harmless but actually human-controlled generative AIs, once they become “close friends” whom people trust utterly and confide in, will subtly influence people. This serves as a reminder to us that there are pitfalls in appearing “people-friendly,” and we must not be deceived by seemingly warm and affectionate appearances.
Cognitive information penetration: This involves disseminating values through media platforms and cultural products to influence the value orientation of people in other countries. This is a common practice in Western countries like the United States. The US once utilized its technological advantage to promote American values extensively through the internet with the use of large databases and information products. The US’s early advantage in generative AI undoubtedly equips it with a new tool for information penetration toward its competitors. This reminds us that some seemingly objective and fair data information has in fact been “colored” or even “rotted.” It is crucial to strengthen national security, especially the protection of strategic resources such as data information, to prevent the replacement of value rationality with technological rationality, and so on.
Cognitive thinking entrapment: This involves exploiting human weaknesses, stimulating primal desires such as the pursuit of comfort, novelty, and excitement, and affecting people’s cognitive thinking through a combination of “curiosity-driven entertainment + consciousness penetration.” Generative AI like ChatGPT caters to the social and emotional dilemmas of modern people, appearing as pleasant and appealing chatbots, meeting many people’s emotional needs. This kind of technology application, focused on catering to and utilizing human characteristics, can easily lead people’s cognitive thinking into narrow, one-sided, and superficial traps. This warns us that we must maintain sufficient rationality toward new technologies, and while utilizing their strengths, we must avoid falling into potential man-made traps.
Cognitive standard hijacking: This involves binding value logic with technological logic, leveraging technological advantages to forcefully imbue corresponding values into products, which is a new practice of Western countries like the United States. For instance, the internet is an indispensable tool in modern society — and Western countries, who hold absolute control over the internet, use it as a powerful tool for value infiltration. The United States, for example, has suppressed voices and actions different from the West on certain issues through controlling internet use and international payment systems. Generative AI represented by ChatGPT can still replicate this kind of “bundling” hijacking tactics, forcing the provision of “standard answers” infused with American values through the mechanism of generating answers. This reminds us that to strengthen cognitive confrontation, we need to enhance our technological innovation and product development capabilities in strategic information systems, using solid technology to support a strong voice in cognitive confrontation.
What about going on the offensive? Wang Ligong 王立功 and He Bing 何兵 wrote in the PLA Daily on May 9 about five strategies the PLA could use to take advantage of new technologies in cognitive confrontations, evidently informed by longstanding United Front strategies and experiences from the social media era.
At present, cognitive warfare is increasingly becoming a new style of combat, with cognitive confrontations becoming more normal and intense. To gain an advantage, take the initiative, and achieve unexpected victories in cognitive defense and offense, we must dare to innovate in terms of forms and content of struggle, strategies and tactics, and means and channels.
Quick — “Strike first.” In psychology, the “primacy effect” emphasizes that the first impression is the most profound and robust. Under the same conditions, whoever releases information first is more likely to take the initiative in cognition. This enlightens us that we should be good at actively setting agendas, guiding public opinion to focus on our concerns and attitude positions; preemptively voicing positive messages, setting the tone for the nature of target events, and forcing opponents to prove themselves; accelerating the fermentation and dissemination of information, promoting a cognitive situation that is advantageous to us and disadvantageous to the enemy, and always firmly grasping the right to define the nature of events, the control of the process, and the judgment of the outcome, so as to achieve defense by offense, and control slow processes with quick action.
Soft — “Empathetic roleplay.” The transmission of information is a two-way interactive process. It is necessary to understand and resonate with the “thoughts and ideas” of the target audience, and avoid “talking to oneself.” The reason why some internet trends can po fang, “break defenses,” and “go viral” quickly is fundamentally because the event itself, narrative techniques, values, and dissemination patterns have a very high degree of “ease of acceptance.” We should deeply understand and grasp the characteristics of the target audience’s cultural customs, values, thoughts and behaviors, do more “role-switching” and less “subjectivity,” more “sharing” and less “indoctrination,” and better integrate righteousness, interests and affection, so as to effectively enhance the affinity of information dissemination.
Confusion — “Making truth indistinguishable from falsehood.” War is never free of deception and illusion; the key is to blend truth with falsehood, and the real with the unreal, so that it is imperceptible, believable, and difficult to defend against.
We need to dare to break away from traditional thinking logic, master the skills of re-describing, rearranging, and reconstructing events, express our own views via the “words of others,” spread “facts and truths” through “sudden events,” and reveal secrets through minor details, making it difficult for the enemy to verify, check, and clarify, thus creating co-created retransmissions.
This achieves the purpose of distracting attention, guiding cognition, and inducing the enemy to make erroneous judgments and decisions.
Innovation — “Unawareness.” With the advancement of modern technology and the rapid development of artificial intelligence, biocrossing, neuroscience and other technologies, the future form of cognitive offense and defense will greatly change. The reason why a software called ChatGPT has recently attracted global attention is fundamentally because artificial intelligence technology is, and will continue to, profoundly change the world. We should keep pace with the global wave of high-tech development and realize the “technology empowerment” of cognitive offense and defense. For example, we can use modern information technologies such as big data, cloud computing, Internet of Things, blockchain, etc., to portray opponents individually and group them, deeply mine and analyze their interests, behavioral characteristics, interpersonal relationships, etc., and accurately deliver mind-impacting products to influence the target’s attitudes, emotions and value judgments toward war.
Cleverness — “Indirect approach.” Maneuvering is a kind of wisdom of the curve; from a combat perspective, maneuvering is an action that actively conceals one’s strategic intentions, and does not directly engage with the enemy head-on; taking large strides forward and backward, maneuvering on a large scale, and achieving direct objectives indirectly, is a highly effective strategic tactic. When direct cognitive offense and defense is difficult to achieve the goal, a roundabout approach is often the best choice. Either by beating around the bush, making the enemy uncertain about your intentions; or by retreating to advance, doing the opposite to make the enemy relax their vigilance; or by breaking down tasks, clearing the periphery, and then taking advantage of the vacuum; or by changing the subject, not responding directly, triggering speculation and conjecture. All of these, when used flexibly, often produce unexpected results.
Next: Zhang Zhiwei 张智伟, a lecturer at the PLA Nanjing Political College, was also frank about the uses of generative AI for “initiators of military action” in his PLA Daily piece on December 8, 2022. The result is a pretty Black-Mirror vision…
Keep reading with a 7-day free trial
Subscribe to ChinaTalk to keep reading this post and get 7 days of free access to the full post archives.