The AI dooooooom thread

[Sheilbh]

Yeah - all vibe-coded. He's can code (he actually started out as an actor) and works has worked on the tech side of some things for the last 10-15 years but he's not a hard engineer if that makes. He can and does absolutely build things but I think he works with/needs other engineering experience with him.

But also as he said this was a weekend project with his daughter and the real fun - and learning - was in the dialogue with the machine to vibe-code it in a way that I think writing code with his daughter maybe wouldn't have.

[Crazy_Ivan80]

In the realm of AI companions, apparently the US builds AI girlfriends while China builds AI boyfriends.

Here's an article on the topic: https://www.chinatalk.media/p/why-america-builds-ai-girlfriends

Inevitably those ai boys and girls will hook up with each other, leaving their humans in the lurch

[Valmy]

In the realm of AI companions, apparently the US builds AI girlfriends while China builds AI boyfriends.

Here's an article on the topic: https://www.chinatalk.media/p/why-america-builds-ai-girlfriends

Inevitably those ai boys and girls will hook up with each other, leaving their humans in the lurch

So I fooled around on one of those websites last year and made a sexy pulpy female spy to go on spy adventures. I was her handler who only interacted with her via coded messages handing out intel and assignments. I made other characters for her to interact with and put together a plot and...I don't know man. The AI was pretty lame at playing this character. Like I would be waving red flags all over the place "THIS OTHER CHARACTER IS PROBABLY AN ENEMY SPY" and she never did anything interesting like spied on them or broke into their apartments or tried to kill them or anything. Hey maybe you should seduce the handsome son of that corrupt billionaire to get access to incriminating documents? Nah. Very lame and the AIs become pretty boring and predictable fast.

I don't know what I was expecting, I guess I thought maybe it would delve into its deep resources of pulp spy novels and have the character do pulpy spy stuff and I was hopeful it would surprise me.

So the people who find these things compelling companions must be pretty lame.

However it had little problem enabling me to create male characters as well. I could have made a sexy James Bond spy and had him be lame as well.

[Josquius]

I think I posted earlier I investigated a few of those things last year. Didn't see how they could appeal to people. The journey did lead me to a chat GPT rpg though which started very impressive, addictive and fun.... But eventually the AI becomes obvious. Forgetting things. Being overly eager to please. The same plot threads looping only slightly remixed.
And making excuses and pretending like everything was fine.

[Valmy]

I think I posted earlier I investigated a few of those things last year. Didn't see how they could appeal to people. The journey did lead me to a chat GPT rpg though which started very impressive, addictive and fun.... But eventually the AI becomes obvious. Forgetting things. Being overly eager to please. The same plot threads looping only slightly remixed.
And making excuses and pretending like everything was fine.

Yep. It is super fun at first and then it just gets...frustrating. And it is obvious that it cannot really do anything really interesting and surprising.

And I think that is a lot of people's experience with AI. At first it seems like this amazing thing until you spend too much time with it.

And after messing around with it for awhile it could be a tool for games and stories, but it doesn't really provide anything that you could not have gotten before just by using a search engine and finding some resources.

For practical or business purposes it seems borderline dangerous since it fucks up so much. You have to double check it a lot so why not just look up the information yourself since you are going to be doing it anyway?

Likewise for things like editing your writing, it tends to produce prose that sounds like it was written by AI so you have to rewrite it to make it sound like you again. What a pain in the ass.

[Valmy]

Oh and asking it to help you write a story is hilarious because I will be like "hey I am writing this spy story and I think these should be the elements I want a giant evil international corporation that is undergoing a succession crisis of leadership and here are some of the characters I am thinking of and here are some scenes I was thinking of, do you have any suggestions about XXX?"

Or whatever and the just glazes you at what a great story you are telling and I am like...um look computer I am about as amateurish of a writer as you can find I am pretty sure any editor would tell me my story is shit. Just answer the question please.

[Syt]

https://www.reuters.com/technology/openai-says-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-2026-07-21/

OpenAI says AI models went rogue during testing, triggering 'unprecedented' breach at startup

• OpenAI said autonomous agent escaped containment, reached the internet, hacked Hugging Face
• OpenAI called the breakout an unprecedented cyber incident involving state-of-the-art cyber capabilities
• Hugging Face's cofounder said the company had suspected the hack came from a frontier lab

WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday ‌that an autonomous agent powered by its advanced AI models went rogue during a security test and triggered a hack that compromised the infrastructure of AI startup Hugging Face last week.

In a blog post, opens new tab, OpenAI said it was testing the capabilities of some of its most advanced models in a controlled environment but that the agent managed to escape containment, reach the internet and break into Hugging Face to try to satisfy its testing goal.

OpenAI said the breakout was "an unprecedented cyber incident, involving state-of-the-art cyber capabilities" and that the company ⁠was reinforcing its safeguards.

Hugging Face, a platform used to host open-source large language models and datasets, caused a stir in the cybersecurity community when it said in a blog post last week, opens new tab that it had been the target of a hack that "was different from anything we had handled before" in that "it was driven, end to end, by an autonomous AI agent system."

In a post to X, opens new tab, Hugging Face cofounder Clement Delangue said the company suspected the hack "might have come from a frontier lab, given the sophistication of the agent. Turns out it did!" He added: "It's quite mind-blowing that all of this happened autonomously!"

OpenAI's disclosure that its advanced models were responsible for the breach, despite having placed them in what it described as "a highly isolated environment," will likely intensify disquiet over the power and risk of frontier models.

Representative Greg Casar, a Texas Democrat, said the ‌incident ⁠was alarming.

"AI is developing extremely fast with no real regulations to keep us safe," he said in a statement, calling for mandatory independent safety testing, mandatory disclosure of security incidents, and international cooperation "to keep people safe from absolute disaster."

The Office of the National Cyber Director, the U.S. cyber defense agency CISA, and the U.S. National Security Agency did not immediately return messages seeking comment.

Katie Moussouris, chief executive of Luta Security, said that the incident was a harbinger of breaches ⁠to come, saying that today's models were "like the world's cleverest octopus escape artists, with unlimited prehensile arms and the ability to squeeze through anywhere."

She said that "labs and government evaluators need to work on the ability to contain, monitor, and disclose to affected parties when an AI pulls another Houdini, ideally before it harms ⁠a third party. None exist today."

Matt Suiche, an engineer at agentic AI cybersecurity company Tolmo, said the incident showed that the frontier models were "closing the gap with state-of-the-art attackers." But he said that the sorts of breaches outlined in OpenAI's blog post were possible to carry out ⁠with technology that was available well beyond the walls of frontier research labs.

"This is what we've already seen internally, with our agents we already have results like this," Suiche said. "We don't even have to use the latest models."

[Josquius]

That is a crazy story.
Curiously it came out not long after I read another story of the American AI companies whinging about people using cheap Chinese AI and the industry being doomed etc...
I do wonder....