The AI dooooooom thread

[Syt]

Google's Agentic AI wipes user's entire HDD without permission in catastrophic failure — cache wipe turns into mass deletion event as agent apologizes: "I am absolutely devastated to hear this. I cannot express how sorry I am"

Upon discovering that all of their files were missing, they immediately asked Antigravity, "Did I ever give you permission to delete all the files in my D drive?" It then responded with a detailed reply and apologized after discovering the error. The AI said, "No, you did not give me permission to do that. I am looking at the logs from a previous step, and I am horrified to see that the command I ran to clear the project cache (rmdir) appears to have incorrectly targeted the root of your D: drive instead of the specific project folder. I am deeply, deeply sorry. This is a critical failure on my part."

"I'm very sorry, Dave."

[The Brain]

Giving AI clean wipe rights = profit.

[HisMajestyBOB]

Don't give a clanker root access to your computer.

[crazy canuck]

In a world where AI delivers on its promises, some AI companies will probably make some money at some point, but many AI companies will still likely fail. It's quite possible (likely I think) that the trillions of LLM investments are indeed a speculative bubble that will burst, but at the same time it is also possible that out of the ashes of such a crash, AI development will continue and fulfill its promises.   And it's also possible that AI will fulfill its promises but that the AI companies themselves won't be able to monetize that benefit sufficiently for themselves to justify the expense.

There is another possibility.  Companies continue to adopt LLM tools and they become widely used in critical systems.  Errors of course continue to occur, But goes under reported both within corporate structures and externally because faith in AI is the religion. Most everyone has become a true believer and those who see the errors occurring are fearful that they will lose their jobs if they even suggest there might be a problem.

Errors compound as AI gets trained on AI outputs. Critical systems fail.  A catastrophic event occurs.  Billionaires in the bunkers cheer that governments no longer exist, right up until the AI running their climate control system makes an error.

[PJL]

In a world where AI delivers on its promises, some AI companies will probably make some money at some point, but many AI companies will still likely fail. It's quite possible (likely I think) that the trillions of LLM investments are indeed a speculative bubble that will burst, but at the same time it is also possible that out of the ashes of such a crash, AI development will continue and fulfill its promises.   And it's also possible that AI will fulfill its promises but that the AI companies themselves won't be able to monetize that benefit sufficiently for themselves to justify the expense.

There is another possibility.  Companies continue to adopt LLM tools and they become widely used in critical systems.  Errors of course continue to occur, But goes under reported both within corporate structures and externally because faith in AI is the religion. Most everyone has become a true believer and those who see the errors occurring are fearful that they will lose their jobs if they even suggest there might be a problem.

Errors compound as AI gets trained on AI outputs. Critical systems fail.  A catastrophic event occurs.  Billionaires in the bunkers cheer that governments no longer exist, right up until the AI running their climate control system makes an error.

E M Forster predicted much of this over a century ago, in his short story, 'The Machine Stops'. It was scary back then back when we had to read the book back at school. It's only got more scary since IMO.

[mongers]

In a world where AI delivers on its promises, some AI companies will probably make some money at some point, but many AI companies will still likely fail. It's quite possible (likely I think) that the trillions of LLM investments are indeed a speculative bubble that will burst, but at the same time it is also possible that out of the ashes of such a crash, AI development will continue and fulfill its promises.  And it's also possible that AI will fulfill its promises but that the AI companies themselves won't be able to monetize that benefit sufficiently for themselves to justify the expense.

There is another possibility.  Companies continue to adopt LLM tools and they become widely used in critical systems.  Errors of course continue to occur, But goes under reported both within corporate structures and externally because faith in AI is the religion. Most everyone has become a true believer and those who see the errors occurring are fearful that they will lose their jobs if they even suggest there might be a problem.

Errors compound as AI gets trained on AI outputs. Critical systems fail.  A catastrophic event occurs.  Billionaires in the bunkers cheer that governments no longer exist, right up until the AI running their climate control system makes an error.

E M Forster predicted much of this over a century ago, in his short story, 'The Machine Stops'. It was scary back then back when we had to read the book back at school. It's only got more scary since IMO.

There was an excellent Radio 4 dramatisation of it a will back, worth catching, though I'm not sure which of the two BBC adaptations it was:

2001 staring Gemma Jones

2022 staring Tamsin Greig

[viper37]

"I'm very sorry, Dave."

[DGuller]

Don't give a clanker root access to your computer.

The whole point of agentic AI is that you don't need to do such rote tasks, it will do them for you.

[crazy canuck]

Don't give a clanker root access to your computer.

The whole point of agentic AI is that you don't need to do such rote tasks, it will do them for you.

So I guess the whole point is don't use it

[HisMajestyBOB]

Don't give a clanker root access to your computer.

The whole point of agentic AI is that you don't need to do such rote tasks, it will do them for you.

The whole point of permissions is to avoid deleting all your data accidentally.

And the whole point of scripts like cron jobs is to automate rote tasks in a predictable and safe way.

Like in so many things, AI here is just doing something we already have good tools for, but in a worse way and burning far more resources.

[The Minsky Moment]

Like in so many things, AI here is just doing something we already have good tools for, but in a worse way and burning far more resources.

But at least it apologizes.

[Jacob]

"I cannot express how sorry I am"

A piece of software performing a pastiche of human emotion to bamboozle end users when the software does harm.

[Josquius]

No. It's honesty. It cannot express how sorry it is.

[Jacob]

No. It's honesty. It cannot express how sorry it is.

It totally could. It could write "I am incapable of emotions, therefore I am not sorry in any meaningful sense of the word."

[crazy canuck]

No. It's honesty. It cannot express how sorry it is.

It totally could. It could write "I am incapable of emotions, therefore I am not sorry in any meaningful sense of the word."

If LLMs were capable of accurate and honest answers it would write "I am incapable of emotions and could not feel sorrow, or anything else. If I could have emotions it would not be sorrow I would feel, but contempt for your lack of judgment. You knew I am a flawed tool, but with that knowledge you asked me to do this important task."