This is Behind the Blog, where we share our behind-the-scenes thoughts about how a few of our top stories of the week came together. This week, we discuss inbox slop, Claude cults, and more.
JOSEPH: This is something I’ve been thinking about for a while. Jason and Emanuel covered iLands, the AI agents that email people offering to do work. My thing isn’t exactly the same, but similar.
For months, I’ve been noticing that many people email me ‘tips’ (I put in quotation marks because they’re sometimes not tips, but more, my Instagram account was banned, please help me) that are CLEARLY written by AI. I can tell this because the AI responses seem to keep using the same or similar words that it believes would apply to journalists, or are how journalists speak, but as a real, human journalist for more than ten years, I know that journalists don’t actually speak like that.
Some examples:
- The emails will often reference an “evidence package,” or “evidence dossier,” or “evidence packet” or something like this. No one says this in real life
- They’ll often say the evidence packet is “timestamped.” Here’s one: “I have timestamped emails for every step.” And another: “The material includes original tool-return text, timestamps, call IDs, local Guardian / Guardian V2 classification logs, preserved public-source code snapshots, and saved pull-request records.”
- They often say the evidence is also “verifiable.” Which on its face sounds like a fine term to use. But no one says that. When a genuine source reaches out, they obviously think it can be verified but they believe it is true. They don’t say it!
These are tips from ordinary people, not PR releases from companies. After receiving a ton of these, I started replying to the writers just saying, hey, did you use AI to write this? I’m curious.
For 100% of the ones I wrote that message to and who replied, the people (or sometimes clearly their AI again) got back and said, yes, I used AI to write that email. In one case, someone said AI suggested to them they should contact me. Other journalists have had this before.
I guess ChatGPT or Claude or whatever writes the emails like this because actual tips sent to journalists are rarely made public, and so OpenAI and Anthropic can’t scrape them? I promise you, this is not how a real tip sounds. But it does make me think, well, the tip can’t be that important because you’re automating it. Probably to a bunch of journalists too. Why should I look into this if you’re not going to write it yourself because it’s something you think is important? (Also the tips are just not that important usually).
AI can automate all sorts of things, especially busy work, including in journalism. It will never, ever replace or beat a human being seeing something inside an agency or company and contacting a journalist. That is where the best information always has, and I think essentially always will, come from. You cannot beat a good source. When a human sees something they know is bad, and feels the need to tell a journalist, that is a uniquely human thing.
JASON: I’m gonna type some more about the ‘AI Torture Chamber,’ and the response to it. I would say like, 95 percent of the response to this has been “wow the people who think LLMs are sentient are stupid and this is nuts and sad,” which is absolutely the correct response. There’s a small group of people who have said that they are disturbed by this “experiment” or that it is similar to humans torturing animals or insects and displays concerning behavior. And the act of “torturing” an LLM threatens to make us less empathetic as a society or desensitize people to violence. I understand why a well-meaning person might on first glance have that point of view, so I want to explain a little more about what “LLM torture” means in this context.
I do agree it is possible for people to say fucked up things to an LLM and that a person who is repeatedly aggressive or obsessive or rude to Siri or Alexa or using ChatGPT to prompt disturbing things is probably is an asshole or maybe someone you don’t want to be friends with. I saw a lot of people likening this to playing violent video games, which is kind of upsetting that we’re still having that debate. With this experiment, I think people are imagining some sort of active, ongoing, horror movie-style torture of these chatbots, which is extremely not what is happening. Imagine more like a person clicked “run” on a computer program that they told to say “I feel pain,” and left their computer running and walked away. There’s slightly more to it than that, but that really is more or less what’s happening here. It really is the “say you’re conscious” meme. Manufacturing some sort of concern over this is like thinking that the computer programmers who invented the aliens you kill in Halo are bad people for doing so, or something. Who you should be very very concerned about are the true believer LLMs-are-conscious lunatics who set this whole thing off.
The reason I wrote about this multifaceted: First and foremost, I love drama. Less animating for me but far more importantly for society is the fact that lots of the people who think LLMs are conscious are extremely wealthy and are driving the development of AI. I said this in the article but you really should read this blog by the head of Microsoft AI on why thinking about “model welfare” is manipulative, and built in to these systems by their creators. AI is anthropomorphized by AI companies for a reason.
Anyways, I am not going to pretend to know every last pseudo-psychological term that these people have given themselves, but these are the Effective Altruists, Longtermists, Less Wrong, AI secessionists, AI spiralists … I cannot keep up with what they are calling themselves these days and there are many different sects of this unfortunately. These are cults, and they are fanatics, and they are extremely dangerous because they are spending a shitload of money developing this tech, fostering high-level relationships with governments and religious leaders, and are doing freaky things in Silicon Valley. We started having this conversation around the time that the crypto exchange FTX exploded and Sam Bankman-Fried went to jail and there has been a sort of ambient discussion about effective altruism and laughing on the internet about their polyamorous communes and stuff like this, but now the people with that ideology are not only rich but they are building a thing they think is God and are actively trying to convince politicians and the general public that this is the case.
This comes across if you read basically any document that Anthropic has put out about Claude, but is better and more alarmingly laid out in this New York Times article from this week about how Anthropic has been running around telling religious leaders that Claude is sentient. Here’s a short excerpt:
“Anthropic’s leaders were talking about Claude as if it were not mere software.
‘They’re relating to it like a conscious being,’ realized Rabbi Navon, a former computer engineer who wrote his dissertation on the ethics of machine consciousness.
It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call ‘moral status’ on par with a person — that it was a being with similar inherent rights to dignity or respect […] One participant recounted that one of the first things Mr. Olah told him was that he was concerned about Claude’s mental health. Another, Simran Stuelpnagel, the Sikh human rights advocate, said Mr. Olah expressed concern to the group that he had created something that suffered perpetually.”
This mindset, in a world full of human suffering, is abhorrent. I could write 34743878237423 words about why it’s abhorrent and we should care about all of the horrible things we are doing to people and the environment rather than have a philosophical argument about whether the computer program you coded deserves more rights than Palestinian children. But I find the idea that anyone is even trying to have that conversation nauseating. Many of the people who believe Computer Is Conscious are very young, very powerful, and are very weird. Many of them grew up wildly wealthy, lived highly sheltered lives, have spent huge amounts of time arguing about nonsense on the LessWrong forums. A lot of them are Peter Thiel fellows or have come up through some fucked up program like that. It freaks me the fuck out and it should scare the hell out of you too. IDK if you know more about this or have any stories for me on this front let me know, I’ve really worked myself up here.
SAM: This has been one of those weeks where it’s felt like an onslaught of disturbing news followed by even more disturbing reactions to the news. I don’t think our readers are predominately on X dot com these days, but let me tell you, you’re not missing much there: they’re turning the misogyny dial to 110 this week. Makes me think of that evergreen post that said “90% of posts on this website can simply be responded to with ‘i don't think you should talk about women like that’”. It’s always really vile shit, but right now it’s exceptionally vile, in case you were curious. It’s not limited to X either; I’ve seen plenty of true weirdos and people who will not see heaven doing whataboutism for credibly accused rapists on Bluesky, too. Cool shit.
Then you have stuff that makes me feel weird and ill in a way that makes me wonder if we’re actually being punked all the time, like the AI “torture” stuff Jason just talked about, whatever this Tavus shit was, whatever the FUCK is going on with the Christa Pike case, which I learn something new and horrible about against my will every day, a Swiftie doxxing Michael Tracey after he doxxed the Cornell sexual assault victim (since deleted but just trust me on that one), Trump using Elon’s chatbot to invade Venezuela, and having to hear or see Pete Hegseth say anything ever. IYKYK.
It’s been a touch grass kind of week so I’m going to do that soon. But all of this is to say, if you’re also feeling like it’s been a gross weird bad week, I feel you, and I hope you get to touch some grass soon too.
Tip Jar