ByteDance gets Suxtnet'd, Taiwan Healthcare and Internet Democracy
Friday bites!
ByteDance Intern Suxtnets 8000 GPUs
In China’s war of 10,000 AI models, ByteDance is a top fighter. But a destructive hack may have thrown a wrench into the training of ByteDance’s LLM products.
In the span of a couple of months, an intern at ByteDance reportedly sabotaged the company’s LLM training with destructive code. On October 18th, messages publicizing the damage appeared in several WeChat groups, alleging that, “the hacked code has spread into more than 8,000 training chips, and the losses may exceed tens of millions of US dollars.”
In response, ByteDance issued a statement announcing that the intern had been fired, that his internship only authorized him to work with ByteDance’s marketing team, and that the AI-related job titles listed on his public profiles (read: LinkedIn) were fake.
ByteDance claims that the hack had no impact on the reliability of its foundational model, Doubao 豆包.
How substantial is the damage in reality? Take a look at this GitHub repository full of testimony from people claiming to be the hacker’s ex-coworkers:
Tian Keyu, please stop your whitewashing behavior immediately! You have been maliciously attacking the cluster code for 2 months, causing great harm to nearly 30 employees at all levels of the company, and making your colleagues’ work for nearly a quarter go to waste. All records and reviews prove that this is an undeniable fact!
Link to the audio recording of the investigators questioning Tian Keyu:
During his internship, Tian Keyu launched malicious attacks on the cluster code for at least 2 months (currently traceable) in order to seize cluster resources and backstab his colleagues and the company, including but not limited to:
1. Modifying the cluster pytorch source code, including random seeds and optimizations;
2. Randomly killing multi-machine experiment processes, causing the large-scale experiments to get stuck;
3. Opening the login backdoor through checkpoint to automatically launch attacks;
4. Participating in the cluster troubleshooting daily meeting and holding meetings according to the troubleshooting ideas of colleagues;
5. Modifying the model weights of colleagues, causing the experimental results to be irreproducible.
It is hard to imagine how malicious Tian Keyu was when he saw his colleagues’ large-scale (card-intensive 大卡) experiments inexplicably interrupted and failed, when he modified the attack code in a targeted manner after hearing their debugging ideas, and when he saw his colleagues working all night without any progress. He continued to launch attacks to achieve his goals.
We are willing to expose more real evidence as the situation develops and your shameless whitewashing behavior continues. We guarantee the authenticity and correctness of all evidence, and we bear legal responsibility for the content of the evidence. If necessary, we are willing to lift anonymity and confront Tian Keyu face to face.
Checking in with some industry friends, the consensus is that a cracked intern could pull this off solo if a firm had lax security controls. What’s weirder to contemplate is the motive, particularly when you’ll probably end up getting caught, will never get a job again, and could certainly face jail time.
People do spiteful things to employers the time (classic scenes include the Office Space printer and Ron Burgundy vs Veronica Corningstone), but it’s not inconceivable that the intern was paid to do this by a competitor. After all, Chinese tech firms play dirty. To illustrate, let’s recall the fate of Bluegogo, a bikeshare company. On June 4th 2017, the app featured a promotion using tank icons around Tiananmen Square. Although most guessed that Bluegogo had been hacked by a competitor, to the CCP that defense was clearly irrelevant. The company, which had raised $140 million and was backed by Alibaba, folded immediately after.
This entire incident also recalls a passage from former OpenAI employee Leopold Aschenbrenner’s interview with Dwarkesh Patel.
Last year, I wrote an internal memo about OpenAI's security, which I thought was egregiously insufficient to protect against the theft of model weights or key algorithmic secrets from foreign actors. I shared this memo with a few colleagues and a couple of members of leadership, who mostly said it was helpful.
A few weeks later, a major security incident occurred. That prompted me to share the memo with a couple of board members. Days later, it was made very clear to me that leadership was very unhappy I had shared this memo with the board. Apparently, the board hassled leadership about security.
I got an official HR warning for sharing the memo with the board. The HR person told me it was racist to worry about CCP espionage and that it was unconstructive. I probably wasn’t at my most diplomatic and could have been more politically savvy. I thought it was a really important issue. The security incident made me very worried.
The reason I bring this up is that when I was fired, it was very made explicit that the security memo was a major reason for my being fired. They said, "the reason this is a firing and not a warning is because of the security memo."
With lax controls, you don’t need state actors to mess up your training clusters; a sharp 24-year-old could do the trick.
Clearly there was something special in the water in OpenAI 2018-2022 that gave them the organizational freedom to explore and exploit the ML possibility space better than any other AI lab on the planet. That era was characterized by lax security and a culture of internal knowledge diffusion, a vibe which helped attract and get the best out of top researchers. But is a tradeoff between that freewheeling research energy and the sort of internal controls necessary to stop an intern, much less a state actor, from messing up your billion-dollar training runs. Which lab, then, will find the right cultural balance once the financial and national security stakes are too high for such shenanigans to take place with so little friction?
Taiwan’s Magical Healthcare System
This week’s ChinaTalk YouTube documentary explores the magic of the Taiwanese healthcare system, which consistently ranks #1 globally.
Book Review — Private Revolutions
Angela Shen graces the newsletter with a book recommendation just in time for the weekend. Private Revolutions: Four Women Face China’s New Social Order is the newest book by British journalist and Labour parliamentarian Yuan Yang 杨缘.
Private Revolutions follows the stories of four real women from all across China — June, Siyue, Leiya, and Sam — as they endeavor to forge better lives for themselves.
Yang and her female protagonists do not wallow in their misfortune, nor do they spend much time pointing fingers at the cultural norms, government policies, or economic changes that helped create many of the obstacles they face.
Instead, the women’s stories speak for themselves — they are natural microcosms of China’s broader trajectory, woven together by Yang’s skillful integration of contextual details and personal history. The book flows chronologically, rotating between the protagonists and documenting how each woman’s life changes with the flow of China’s rapid development.
The book offers a nuanced view of China’s urban-rural disparities, the effects of industrialization and privatization on factory workers, the competitive and deeply unequal educational landscape, the rise of the nouveau-riche, the lack of childcare and community support for women, and more. Here are some moments that stuck with me:
Fifteen-year-old Leiya persuades her fellow leatherwear factory workers to have fun in the city before their micro-managed shifts (and eventually, they resign together)
June, born in a village, moves to Beijing to join an ed-tech start-up. Her job is to convince anxious parents to buy online education programs using a follow a step-by-step sales model
Single mother Siyue cultivates a support system to care for her baby daughter, which includes help from female friends and Siyue’s widowed mother
Sociologist Sam volunteers for a left-wing blog, which leads her to mobilize donations and legal support for student activists
This book is a powerful depiction of the struggle for social mobility in China’s new capitalist era.
Read it, and bear witness.
Weibo Doom Scroll of the Week
“The internet is the biggest source of democracy; security cameras are the biggest source of justice.”
Warning: this post discusses sexual violence.
“Real story, guys.
Girl from a normal family, real pretty so she got a second-generation politician boyfriend [basically like a trust fund brat, but instead of having a rich dad, you have a high-ranking government official dad]. One day, her boyfriend called her out to a hotel for a date, so she dressed up all nice and went. After she got there, she found that it wasn’t just her boyfriend there, but a couple of other trust fund brats.
And her boyfriend gang raped her with all his friends. The girl fought back as hard as she could, screamed for help, but nobody came. She ran out of the hotel room naked into the corridor and got pulled back into the room. A lot of guests were alarmed and some people filmed it on their phones for evidence.
Later on, she sued her boyfriend, and the judge ruled that the case didn’t constitute rape or gang rape, so she lost the case.
She had to resort to the court of public opinion in her appeals to manage to flip the case. Dunno how many years those trust fund brats got sentenced to.
Why do I tell this story? Because I’ve been seeing a lot of gossip lately. Some netizens who think they have a lot of life experience is always arrogantly telling the victim, “If you think your rights were violated, go call the polite. Go take it to court. Why expose all this on the internet? Aren’t you defaming the other side?”
Some people saying this are just naive, some are young, some are dumb, and some are just evil.
Calling the police or suing does absolutely nothing for a lot of cases. They’ll either muddy the waters and not do anything, or punish both sides equally, or even worse, just outright protect the criminals. It’s only when the victim has nowhere else to go, no one else to turn to, that they take things to the internet. If they had any other option at all, nobody would want to expose themselves to all kinds of judgement on the internet. Because once you expose things on the internet, there’ll be people saying all sorts of shit. There’s a big market out there for victim blaming.
I’ve always been a proponent that the internet is the biggest platform for democracy in China.
It’s only when things are exposed on the internet that we get to see these monsters in the light of day. That’s the only reason they might hesitate at all before hurting somebody. It’s one more layer of security for peasants like you or me.”
From the comments:
“Without the internet, we would be in eternal night. The internet is the biggest source of democracy, security cameras are the biggest source of justice. A phone means that everyone has a platform to speak.”
Song of the Week — Alexa Pan
“Star” (星）by Supermarket （超级市场）
This song is for the hopeful romantics and electronica nerds. Founded in 1996, Supermarket is often hailed as the first “real” mainland Chinese electronic band. In the last three decades, they’ve survived lineup shuffles (with only one original member, Tian Peng, still at the artistic helm), played with Portishead, and steadily grown a cult following and an enduring catalog. Recently, they’ve shot to mainstream fame via China’s rock music reality show, The Big Band, emerging as a standout act and still more or less alone in their genre.
“Star” is an outlier on their 2003 album, “Concert”, which otherwise showcases trip hop, rock, ambient, and deep house tracks with sophisticated production. “Star” is my favorite track — it is short, ethereal, and blends acoustic guitar with synthesizers to produce a glorious, alien effect. I hear: shimmery starlight; a guitar strumming —grounded and foreboding; sweeping synth lines rising out of the stars as carpet or spaceship; vocals by folk singer Wang Juan tenderly sending them to orbit.
The lyrics borrow from a scene in the movie “Before Sunrise” (“Limousine Eyelash/ Oh, with your pretty face/Drop a tear in my wine glass”) before moving down to earth (“I’d like to soar with you lying down/ Occasionally so soothing”).
There is no more I can say, except that I hope “Star” brings you this feeling when you need it — warm, dreamy, floating in space, inarticulate and comforted.
“Star” (星) by Supermarket (超级市场), yes! So good!