Platform Complicity in Child Abuse and How it Helps Big Brother
My research on X, Discord, & Telegram reveals how social media complicity leads to both the proliferation of CSAM and the expansion of government regulation and censorship.
In June of 2023, I discovered a bot farm on X that was spamming links to a child sexual abuse material (CSAM) distribution network at an alarming rate. Interestingly enough, I didn’t discover this network while investigating a related issue. I discovered it completely by accident. On June 21, 2023, NBC News released an extensive investigation into rampant child abuse, sextortion and even abductions occurring on the Discord messaging app. Everything in the article lined up with what myself and other researchers had long been blowing the whistle on; Discord had problems. Some of these matters would later be tied into the 764 and COM child predator and extortion networks. I wanted to tweet about the NBC article, so I went to the X search bar, typed in “Discord”, and felt my heart sink to the floor.
Child pornography. Lots of it. Some of the heartbreaking images were out in the open for all to see. Others were far more innocuous, advertising Telegram channels or Discord servers using code words like “c, p” and “Cheese Pizza”. What all of the tweets had in common is that they used the word “Discord” somewhere in the post. Because of this, a search for this term in the X search bar returned an endless flood of these bots, visible to anyone regardless of their sensitive content settings. Even worse, the word “Discord” was already trending on X due to the NBC article release.
Basically, for several weeks last year, if you searched "Discord" on X, you were shown child pornography.
Unlike most sexually explicit content, these tweets were not filtered out of search results when X’s sensitive content filters were applied. In fact, applying the content filters made the spam more noticeable since it removed other material classified as sensitive, such as adult pornography and crypto links applying the same general formula. In several instances, sexually suggestive and sexually explicit images and videos of children appeared in the search results in thumbnails and “previews” alongside links to illicit distribution hubs. Some of these were merely suggestive, such as children dancing or posing, while others contained videos and images of children engaged in sex acts. I personally came across 2 such instances during my review of the issue. In each case I reported the offending content right away to X and to the US Watchdog, NCMEC. In one of the cases, X had banned the account before I could successfully make a report, however, several other accounts had posted the same material right above it. It appeared X’s moderation tools were wholly incapable of controlling the issue, in part due to the sheer volume of content being posted.
The accounts spamming this content were varied. Some of them were otherwise innocuous, having been hacked and repurposed with criminal intent. A profile picture of a smiling Midwestern mom stands out – her user handle contained her full name. Only a few months prior, this account had been used as normal to tweet about bus schedules in her county. I wonder if she knows what has become of her hacked profile?
One such network of repurposed accounts tweeted at a rate of 4-10 times per minute each. I found 17 such accounts linked to this network. The total number of times they tweeted links to CSAM was 131 times per minute. That’s 188,640 times per day. The links being posted by the bots led to Discord servers and Telegram channels where the illegal material was sold. Explicit images of children sometimes appeared as previews alongside the links, and some accounts even appeared to be selling private chats and shows with children. Even worse, in between search hits were ads. Not only was X getting hit with hundreds of thousands, if not millions of child exploitation bot posts per day, they were potentially earning revenue from it.
Those of us researching and reporting the issue to X ran into many roadblocks. Rate limitations prevented us from using alt accounts and we were frequently misidentified as "bots/spam" for reporting them. The X tool for reporting CSAM was also glitchy. Several weeks went by as we tried to boost the issue to X, but they simply couldn't get the problem under control. I began speaking out about this in late July. I was careful not to break TOS or provide information that could inadvertently lead users to the content. Then, after around 8 weeks of constant speaking out, I decided to contact media outlets to escalate the issue. Not long after, my X account was banned for 10 weeks with no explanation. When my account was finally reinstated, my data showed there was no TOS violation prior to the ban. I would later learn that other child advocates were also banned or restricted around this time, as well.
This issue is not isolated to X, despite it playing a pivotal role in the advertising of CSAM by criminal enterprises. Exploitation begins on platforms such as Twitch, TikTok and Snapchat, where predators can engage with the children in real time, often convincing them to perform sexual acts. After obtaining this material, the predators find ways to further communicate with the child in order to obtain more sexual content. This leads to a devastating cycle of blackmail and sextortion, wherein the child is threatened into providing more and more illicit material. Recent news reports on the matter indicate that teenage suicides related to sextortion are increasing at an alarming rate.
The predators use Discord servers and Telegram channels to organize groups that buy, share and trade the material. These groups are advertised on platforms like X, Instagram, and Facebook, often utilizing well known code words to attract potential buyers. Social media platforms are utilized at every step of this process, and yet most of them offer little in the way of transparency to concerned users who stumble upon this content.
To make matters worse, many platforms have been criticized for making the process to report CSAM confusing and difficult to navigate.
Telegram and Discord both have reporting mechanisms that are managed outside of the apps and require extensive effort on the part of the person making the report. Unique opportunities for combating the proliferation of CSAM are present at every step of the process, and all of them serve important functions. However, even when these techniques are effectively utilized, they don’t prevent children from becoming victims. Managing the problem on the back-end may slow down the sale of such material but the damage has already been done.
Efforts aimed at shutting off the firehose of bots are inherently difficult to implement, especially as criminals constantly find way to get around new detection mechanisms. Strategies that focus on preventing children from becoming victims should therefore take a higher priority than reactive measures. Such strategies do not require resources from law enforcement or advocacy groups, thereby easing the strain on these already overburdened systems. The only catch? Platforms in the US have no incentive to make their websites safer for children. On Twitch, for example, raising the age limit required to stream from 13 to 16 would dramatically reduce the number of children being victimized there. But because such a policy would lead to a decrease in the number of users on the platform, and therefore a decrease in profits, it’s probably not high on their list of site improvements.
Adequate control of the proliferation of CSAM requires a multi-pronged approach focused on prevention. Once the material is obtained, child victims have already been irreparably harmed, regardless of how many people buy the material in the future. Step One needs to be reaching out to parents and explaining why they should get their kids off of social media platforms altogether.
Censorship Disguised as Child Safety
In October of 2023, the UK passed the 2023 Online Safety Act, which postured itself as being focused on child safety, even though it was widely criticized for censoring legal speech as well. Censorship bills can cleverly disguise themselves as being directed at tackling the child exploitation problems on social media. They rely on the (correct) argument that platforms like X, Discord, and Telegram are ineffective at moderating such content. By failing to moderate illicit content, platforms give governments the ammo they need to expand their authority to surveil and censor their people.
In May of 2024, Elon Musk put out a tweet flaming Meta and Google for killing a proposed New York child safety bill and pledging X’s support for such bills. At the same time, Musk claims to be a free speech hero, frequently grandstanding about how we can never give into government regulation of social media. This is ironic, since the New York bill was killed for a good reason. The so-called SAFE Act contained some popular and potentially effective bipartisan measures, such as prohibiting social media companies from collecting personal data from children and from using “addictive” feeds targeted at kids. However, it also contained sneaky provisions which would set age and identity verification requirements, which raised a litany of privacy concerns.
But it is possible for Elon Musk to have his cake and eat it, too. If he wants X to be a true free speech platform, he can start by making good on his promise to make child safety the #1 priority. When platforms voluntarily work with internal focus groups and external watchdogs to protect children, they prevent giving the government a foot in the door to step in with proposed regulation.
Simply put: Platform complicity = Government regulation.
The Trouble with Telegram
Another situation is now raising alarm bells following the arrest of Telegram CEO Pavel Durov in France.
Governments all over the world have accused Telegram of being complicit in drug trafficking, CSAM distribution, and other criminal activities and of ignoring requests by watchdogs such as NCMEC about CSAM removal. Many large public figures were quick to come to Durov’s defense, including NSA whistleblower Edward Snowden, and somewhat less surprisingly, Elon Musk. Durov’s representatives posted a statement on X claiming Telegram has done nothing wrong, and that it is “absurd to claim that a platform or its owner are responsible for abuse of that platform.”
Here’s the problem. Telegram DOES ignore requests for removing CSAM from group chats, despite having the ability to do so with relative ease. In fact, it stubbornly and proudly refuses to comply with nearly all law enforcement investigations, even ones with child victims. I personally witnessed this while advocating for a young victim of 764 predator Kyle Spitze. The girl’s mother spent months working with her country’s Watchdog group to send requests to Telegram to remove pornographic images of her young daughter from a 764 network group chat. Their efforts were unsuccessful, resulting in an escalation by the watchdog group to the country’s federal law enforcement body. So, although Durov was arrested in France, this is a worldwide problem.
I’ve seen a lot of incorrect information being spread in Durov’s defense, such as the claim that Telegram is unable to assist in investigating and removing CSAM due to end-to-end encryption. But Telegram, unlike messaging platforms like Signal and WhatsApp, is not end-to-end encrypted by default. This is especially true of the group chats at the center of this debate, which store data in a Telegram-owned cloud server prior to being pushed to group members. Essentially, the issue isn’t that Telegram CAN’T moderate, it’s that it WON’T moderate.
Telegram postures itself as a beacon of free speech while refusing to rescue endangered children when given the opportunity. Is this a fair trade? I’m no fan of government regulation. As noted above, I believe that governments all over the world use platform complicity in CSAM proliferation as a foot in the door for regulation and control, including some proposed measures which place free speech in grave danger. So what’s the solution here? Since Telegram operates above the law, maybe it should start its own extrajudicial task force to rescue children placed in harms way within its walls.
Many free speech advocates fall into the trap of being unable, or unwilling, to accept that two things can be true at once. We can acknowledge that Telegram is terribly complicit in the proliferation of child abuse content while also acknowledging the possibility of an ulterior political motive for Durov’s arrest. Similarly, we can advocate against government infringement on free speech while also advocating for platforms like Telegram, Discord and X to make more effective policies to protect children. In this case, the latter may even prevent the former.