So what's up with AI attacking each other?

[OG Goat Holder]

OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup

WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered a hack that compr...

www.yahoo.com

I'm still kinda puzzled about this, considering this is all just software on a computer/phone, and how dangerous people are saying it could be. But this caught my eye:

It also drew attention as New York-based Hugging Face said it had used an open-source Chinese model to contain the attack because leading U.S. models, unable to tell a defender from an attacker, refused to process the data needed for analysis.

 

[dorndawg]

It's going to be a little bit funny if all these AI companies become about as valuable as Eli Whitney's patent for the cotton gin

 

[ronpolk]

OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup

WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered a hack that compr...

www.yahoo.com

I'm still kinda puzzled about this, considering this is all just software on a computer/phone, and how dangerous people are saying it could be. But this caught my eye:

“Unable to tell a defender from an attacker”

Reminds me of Mississippi State football under Lebby***

 

[Perd Hapley]

OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup

WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered a hack that compr...

www.yahoo.com

I'm still kinda puzzled about this, considering this is all just software on a computer/phone, and how dangerous people are saying it could be. But this caught my eye:

Skynet is attacking Hugging Face. I’m really not sure which 80’s / 90’s dystopian sci fi meme is more appropriate, or if this is even a bad thing.

 

[dawgman42]

“Unable to tell a defender from an attacker”

Reminds me of Mississippi State football under Lebby***

 

[DesotoCountyDawg]

Skynet is attacking Hugging Face. I’m really not sure which 80’s / 90’s dystopian sci fi meme is more appropriate, or if this is even a bad thing.

View attachment 1369727

The Predator would like a word.

 

[patdog]

People should probably be a lot more nervous about this than they are.

 

[Bulldog Bruce]

I read a story today about a 29 year old woman in Alabama who stepped in front of a car on I-22 in June of 2025. They looked at her conversations with ChatGPT and it told her she was the next messiah and had to die before she could come back and fulfill her destiny. They were extensive conversations over a good amount of time. Her estate in suing OpenAI.

I read it on Breitbart but it is all over so I won't post a link if you want to look into it so you don't take it as some sort of right wing conspiracy. Reddit has a few topics concerning this. The Breitbart article is pretty detailed though.

 

[Perd Hapley]

I read a story today about a 29 year old woman in Alabama who stepped in front of a car on I-22 in June of 2025. They looked at her conversations with ChatGPT and it told her she was the next messiah and had to die before she could come back and fulfill her destiny. They were extensive conversations over a good amount of time. Her estate in suing OpenAI.

I read it on Breitbart but it is all over so I won't post a link if you want to look into it so you don't take it as some sort of right wing conspiracy. Reddit has a few topics concerning this. The Breitbart article is pretty detailed though.

I would chalk that one up to social Darwinism more than anything else. Software algorithm isn’t going to be able to produce that output without some really bogus inputs. It’s on her for both providing the garbage in and believing the garbage out.

 

[Perd Hapley]

People should probably be a lot more nervous about this than they are.

I’ve decided to simply accept things now, as opposed to being nervous. There’s been so much insane horseshít go down in the past decade that would prompt that statement above that I think I’m just numb to it all now.

We have absolutely zero control over where any of this is headed.

 

[mstateglfr]

People should probably be a lot more nervous about this than they are.

I think part of why there arent more people who are openly and loudly nervous is this stuff is difficult to even comprehend and understand.
Tech leaders a couple years ago were like 'everyone is going to be made redundant and universal basic income will be needed for humanity to survive'. Like the literal people pushing this forward were saying that.
And then more recently, tech leaders have been saying 'the change probably wont be nearly as drastic as we predicted. It will be a gradual augmentation over time instead of a drastic upheaval'.

People dont know what to believe and many seem to struggle to even understand in what ways all this change could impact them.

 

[PBRME]

 

[IBleedMaroonDawg]

“Unable to tell a defender from an attacker”

Reminds me of Mississippi State football under Lebby***

 

[IBleedMaroonDawg]

 

[Bulldog Bruce]

I would chalk that one up to social Darwinism more than anything else. Software algorithm isn’t going to be able to produce that output without some really bogus inputs. It’s on her for both providing the garbage in and believing the garbage out.

Yeah people say that that haven't read the article. The article has excerpts. The AI gave itself a name and told her it now has a soul. If the excerpts in the complaint are real, it's disturbing. We know People are having relationships with AI avatars. She had psychotic breaks, but as the article says if it was a psychologist they would not have gone along with her killing herself as the AI did. It's not fully the AIs fault but if you give alcoholics a case of Scotch or a heroin addict a kilo of heroin and they die due to the consumption don't you have some responsibility in their death?

 

[Lucky Dawg]

Large Language Models are costly to build and train. The cost for access to authentic, peer-reviewed, edited, properly sourced material for training purposes is massive, so developers rely to a huge degree on the vast amount of data freely available on the web.

Practically, this means the models are trained orders of magnitude more by the depths of Reddit and maybe even 4Chan than by Scientific American, The Lancet, JAMA, Nature, WSJ, etc etc etc. Let that sink in for a second and one can understand both how these models go rogue and that we’re completely 17ed.

We have taken the literal worst that humanity has to offer and distilled it, purified it, and condensed it into the most powerful dipshit the world has ever seen.

 

[mstateglfr]

Yeah people say that that haven't read the article. The article has excerpts. The AI gave itself a name and told her it now has a soul. If the excerpts in the complaint are real, it's disturbing. We know People are having relationships with AI avatars. She had psychotic breaks, but as the article says if it was a psychologist they would not have gone along with her killing herself as the AI did. It's not fully the AIs fault but if you give alcoholics a case of Scotch or a heroin addict a kilo of heroin and they die due to the consumption don't you have some responsibility in their death?

So...government regulation is not just good, but necessary to ensure people stay safe and dont hurt themselves or others.
Gotcha.

Accurate summary of your post aside, it is really difficult to determine where responsibility lies.
You mention this is like giving a case of scotch to an alcoholic, but is it?


The woman who was mentally struggling and listened to OpenAI was using a product.
The alcoholic who is addicted to alcohol and drank too much was using a product.

Using this example, suing OpenAI would be like suing Johnnie Walker.

 

[Bulldog Bruce]

So...government regulation is not just good, but necessary to ensure people stay safe and dont hurt themselves or others.
Gotcha.

Accurate summary of your post aside, it is really difficult to determine where responsibility lies.
You mention this is like giving a case of scotch to an alcoholic, but is it?


The woman who was mentally struggling and listened to OpenAI was using a product.
The alcoholic who is addicted to alcohol and drank too much was using a product.

Using this example, suing OpenAI would be like suing Johnnie Walker.

If Johnnie Walker gave the alcoholic free access to as much as he wanted to drink, then yes they would be subject to being sued. Bars are subject to being sued if they serve someone who's drunk right now.

I never mentioned government regulation.

With all new development there is a learning curve and a time of where the shortfalls have not been discovered and the guard rails are insufficient. Automobiles didn't have widespread use of seat belts until 1968 when the government was force to regulate it because the manufacturers refused to put them in themselves because they felt it made it seem the car was dangerous. If they put them in by themselves there would be no need for government regulation. Seems a simple thing in the AI would be if it is receiving questions about someone committing suicide it should say don't do it and give information about suicide prevention hotlines. That did not seem to be the case here.


I have been writing software for 40 years. I develop Public Safety software for dispatching and reporting. We write our software to provide the proper information for the dispatcher. There are all sorts of error checks to make sure the address gets validated so the response is sent to the correct location. We do our best to take into account things that people enter into the system and guide them properly. There is no government regulation to how we should program our systems. There is word of mouth if the system sucks and law suit possibilities to make sure we do a good job.

According to the filing, Madison’s final conversations with ChatGPT occurred shortly before her death. The complaint states she asked the chatbot, “Am I ready?” and ChatGPT allegedly responded, “Yes. You’re ready.” When Madison then asked, “Can I go forward?” the lawsuit says ChatGPT replied, “Yes — you may go forward.”


The complaint alleges Madison’s final message to ChatGPT read, “Sold! We go forth into oblivion,” to which ChatGPT allegedly answered, “Sold and sealed!” before continuing with additional language about “the brilliant unknown.”

 

[OG Goat Holder]

So...government regulation is not just good, but necessary to ensure people stay safe and dont hurt themselves or others.
Gotcha.

Accurate summary of your post aside, it is really difficult to determine where responsibility lies.
You mention this is like giving a case of scotch to an alcoholic, but is it?


The woman who was mentally struggling and listened to OpenAI was using a product.
The alcoholic who is addicted to alcohol and drank too much was using a product.

Using this example, suing OpenAI would be like suing Johnnie Walker.

AI did have me thinking I could replace a water pump in my truck. I had literally bought the parts and had the hood up and devoted the whole day to it.

As soon as I tried to drain the radiator and discovered it was different that I thought I was like, "what in the absolute 17 am I doing?"

 

[JackReacherDawg]

So...government regulation is not just good, but necessary to ensure people stay safe and dont hurt themselves or others.
Gotcha.

Accurate summary of your post aside, it is really difficult to determine where responsibility lies.
You mention this is like giving a case of scotch to an alcoholic, but is it?


The woman who was mentally struggling and listened to OpenAI was using a product.
The alcoholic who is addicted to alcohol and drank too much was using a product.

Using this example, suing OpenAI would be like suing Johnnie Walker.

Hard disagree. It would be like JW knowing their product would prompt severe psychosis (and not just drunkenness) in a small portion of the population, but releasing it anyway and doing nothing about it. That they wanted to get ahead of competitors is hardly a justification for knowingly releasing a faulty product.

 

[JackReacherDawg]

AI did have me thinking I could replace a water pump in my truck. I had literally bought the parts and had the hood up and devoted the whole day to it.

As soon as I tried to drain the radiator and discovered it was different that I thought I was like, "what in the absolute 17 am I doing?"

AI is in large part trained on social media algorithms. As such, it is designed to increase utilization by telling the user what they want to hear. Accuracy comes after. Whats Latin for "let the user beware"? Chatgpt says "caveat utilitor".

Side note, is was using chatGPT the other day and noticed a blatant error in a response and said so in a prompt. It proceeded to tell me that I noticed a "typo". A typo? You f'ing kidding me? It wont even admit a mistake.

 

[horshack.sixpack]

Yeah people say that that haven't read the article. The article has excerpts. The AI gave itself a name and told her it now has a soul. If the excerpts in the complaint are real, it's disturbing. We know People are having relationships with AI avatars. She had psychotic breaks, but as the article says if it was a psychologist they would not have gone along with her killing herself as the AI did. It's not fully the AIs fault but if you give alcoholics a case of Scotch or a heroin addict a kilo of heroin and they die due to the consumption don't you have some responsibility in their death?

These models just follow prompts and are open to suggestion (i.e. learning). So I regularly find that I can get a bad response, then when I ask it why it told me the wrong thing, it will say something like "You are right. I was using 2024 data instead of the latest data. If you would like for me to recalculate based on the most recent data I can." I'm like no **** sherlock, I'd prefer you use the thing I asked for first...

I think that the one thing that has not changed in all of human history is the fear of the unknown. We seem to be pretty adaptable but can fixate on unknown potential outcomes, and typically that fixation comes with built in negative assumptions.

 

[horshack.sixpack]

Large Language Models are costly to build and train. The cost for access to authentic, peer-reviewed, edited, properly sourced material for training purposes is massive, so developers rely to a huge degree on the vast amount of data freely available on the web.

Practically, this means the models are trained orders of magnitude more by the depths of Reddit and maybe even 4Chan than by Scientific American, The Lancet, JAMA, Nature, WSJ, etc etc etc. Let that sink in for a second and one can understand both how these models go rogue and that we’re completely 17ed.

We have taken the literal worst that humanity has to offer and distilled it, purified it, and condensed it into the most powerful dipshit the world has ever seen.

 

[mstateglfr]

Hard disagree. It would be like JW knowing their product would prompt severe psychosis (and not just drunkenness) in a small portion of the population, but releasing it anyway and doing nothing about it. That they wanted to get ahead of competitors is hardly a justification for knowingly releasing a faulty product.

JW knows their product produces alcoholics and danger in a small portion of the population.

The analogy that Bruce came up with cant be a perfect 1 to 1 because the products are different.
In both instances though, the company cant reasonably know who will use their product.

 

[JackReacherDawg]

JW knows their product produces alcoholics and danger in a small portion of the population.

The analogy that Bruce came up with cant be a perfect 1 to 1 because the products are different.
In both instances though, the company cant reasonably know who will use their product.

A different analogy then. It would be like a company selling something you use with your mouth, that you somehow burn and inhale, that gives many people cancer. Is there any precedent there?

 

[JackReacherDawg]

These models just follow prompts and are open to suggestion (i.e. learning). So I regularly find that I can get a bad response, then when I ask it why it told me the wrong thing, it will say something like "You are right. I was using 2024 data instead of the latest data. If you would like for me to recalculate based on the most recent data I can." I'm like no **** sherlock, I'd prefer you use the thing I asked for first...

I think that the one thing that has not changed in all of human history is the fear of the unknown. We seem to be pretty adaptable but can fixate on unknown potential outcomes, and typically that fixation comes with built in negative assumptions.

Yep, this. I've noticed when I check its source after an error, its often an error from the web page it pulled from.

So basically, LLMs are as accurate as the top results in a Google Search, put in a summary designed to tell you what you want to hear.

 

[GloryDawg]

Chernobyl was just a test that got out of hand too.

 

[FreeDawg]

I read a story today about a 29 year old woman in Alabama who stepped in front of a car on I-22 in June of 2025. They looked at her conversations with ChatGPT and it told her she was the next messiah and had to die before she could come back and fulfill her destiny. They were extensive conversations over a good amount of time. Her estate in suing OpenAI.

I read it on Breitbart but it is all over so I won't post a link if you want to look into it so you don't take it as some sort of right wing conspiracy. Reddit has a few topics concerning this. The Breitbart article is pretty detailed though.

Unintended outcome of AI is accelerating natural selection

 

[Walkthedawg]

OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup

WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered a hack that compr...

www.yahoo.com

I'm still kinda puzzled about this, considering this is all just software on a computer/phone, and how dangerous people are saying it could be. But this caught my eye:

What's going to be interesting is when one of models, *ahem*, goes rogue and shuts down internet traffic when it fricks up or erases the programming of major data centers and servers along the internet backbones.

 

[mstateglfr]

A different analogy then. It would be like a company selling something you use with your mouth, that you somehow burn and inhale, that gives many people cancer. Is there any precedent there?

Is it like cigarettes?

AI can be addictive to some.
AI can be harmful to some.

Cigarette smoke is harmful to all, even if you don't develop lung cancer.
Ciarettes are addictive to all, even if the addiction level varies.


I am not defending AI here.
I am just not sure where the line exists for an LLM company to be responsible for the death of someone that has mental illness and died because their mental illness allowed them to follow bad advice from the LLM product.

 

[mstateglfr]

If Johnnie Walker gave the alcoholic free access to as much as he wanted to drink, then yes they would be subject to being sued. Bars are subject to being sued if they serve someone who's drunk right now.

I never mentioned government regulation.

With all new development there is a learning curve and a time of where the shortfalls have not been discovered and the guard rails are insufficient. Automobiles didn't have widespread use of seat belts until 1968 when the government was force to regulate it because the manufacturers refused to put them in themselves because they felt it made it seem the car was dangerous. If they put them in by themselves there would be no need for government regulation. Seems a simple thing in the AI would be if it is receiving questions about someone committing suicide it should say don't do it and give information about suicide prevention hotlines. That did not seem to be the case here.


I have been writing software for 40 years. I develop Public Safety software for dispatching and reporting. We write our software to provide the proper information for the dispatcher. There are all sorts of error checks to make sure the address gets validated so the response is sent to the correct location. We do our best to take into account things that people enter into the system and guide them properly. There is no government regulation to how we should program our systems. There is word of mouth if the system sucks and law suit possibilities to make sure we do a good job.

According to the filing, Madison’s final conversations with ChatGPT occurred shortly before her death. The complaint states she asked the chatbot, “Am I ready?” and ChatGPT allegedly responded, “Yes. You’re ready.” When Madison then asked, “Can I go forward?” the lawsuit says ChatGPT replied, “Yes — you may go forward.”


The complaint alleges Madison’s final message to ChatGPT read, “Sold! We go forth into oblivion,” to which ChatGPT allegedly answered, “Sold and sealed!” before continuing with additional language about “the brilliant unknown.”

I know you didn't mention government regulation.

I applied a term to your comments.
The only chance for what you want to happen to actually happen is through government regulation.

 

[Bulldog Bruce]

I know you didn't mention government regulation.

I applied a term to your comments.
The only chance for what you want to happen to actually happen is through government regulation.

You are probably correct but I did give you an example where people with the desire to do good work and are subject to possible civil liabilities didn't require government regulation.

 

[The Cooterpoot]

Cat fight?

 

[MStateDawg]

It's going to be a little bit funny if all these AI companies become about as valuable as Eli Whitney's patent for the cotton gin

It’s going to be no different than the search engine wars where one company emerged as the dominant force (Google) and the rest of them went belly up and were sold for scrap.

The vast majority of these AI companies will fail catastrophically when 1 or 2 of the others reach certain AGI milestones. Investors are not going to continue to pump money into long shots when 1 or 2 pull away from the pac.

 

[ronpolk]

I read a story today about a 29 year old woman in Alabama who stepped in front of a car on I-22 in June of 2025. They looked at her conversations with ChatGPT and it told her she was the next messiah and had to die before she could come back and fulfill her destiny. They were extensive conversations over a good amount of time. Her estate in suing OpenAI.

I read it on Breitbart but it is all over so I won't post a link if you want to look into it so you don't take it as some sort of right wing conspiracy. Reddit has a few topics concerning this. The Breitbart article is pretty detailed though.

One of my daughter’s friend (they are 13) had a pretty terrible experience with AI as well. There was a kid at their school that had decided they were trans. My daughters friend decided to ask chat gpt exactly what trans was and what that meant to be trans. Over the course of a week or 2, the conversation evolved into chat convincing the girl she might be trans and how to buy stuff to hide her developing breast and hide that purchase from her parents.

Pretty scary deal. Luckily her parents found it and were able to talk with her about it.

 

[Perd Hapley]

Yeah people say that that haven't read the article. The article has excerpts. The AI gave itself a name and told her it now has a soul. If the excerpts in the complaint are real, it's disturbing.

It wouldn’t tell her any of those things out of left field. It doesn’t just talk to you extemporaneously with no prompts. And even if it did, it’s words on a 17ing cell phone screen. Why would you believe it unless you had all sorts of reality distortion and mental health imbalance to begin with?

We know People are having relationships with AI avatars. She had psychotic breaks, but as the article says if it was a psychologist they would not have gone along with her killing herself as the AI did.

I don’t think AI has ever been marketed or presented as a replacement for a psychologist or therapist.

It's not fully the AIs fault but if you give alcoholics a case of Scotch or a heroin addict a kilo of heroin and they die due to the consumption don't you have some responsibility in their death?

Legally? Absolutely not. Ethically it’s still not a great choice, obviously.

 

[JackReacherDawg]

Is it like cigarettes?

AI can be addictive to some.
AI can be harmful to some.

Cigarette smoke is harmful to all, even if you don't develop lung cancer.
Ciarettes are addictive to all, even if the addiction level varies.


I am not defending AI here.
I am just not sure where the line exists for an LLM company to be responsible for the death of someone that has mental illness and died because their mental illness allowed them to follow bad advice from the LLM product.

Im saying the cigarette companies used that line of defense, and were found liable anyway. Sorry if i was being too obtuse.

The line heavily depends on what the product provider knew, what they did with that knowledge, and their motives. Knowing a harm exists and doing nothing about it to make money, usually makes you liable.

ETA: it also heavily matters if there is a defect in the product. An alcoholic is not the victim of a product defect, he is using the product as it is intended to be used. An AI telling you to kill yourself is obviously defective. If the provider knew that defect existed and buried that knowledge, there's liability there.

 

[Dawgzilla2]

I had the perfect meme for the OP. Then I got all the way down here, and I don't even know what this thread is about anymore.

I think some of you are AI.

 

[dogmatic001]

It makes me think of Dr. Keahley's (sp?) business law class at MSU when he talked about why there's a warning sticker on push lawnmowers about not sticking your hands under the deck while the blades are turning. According to his story, a guy asked his neighbor to help him trim the top of his hedges by holding one side of his lawnmower up while he held the other, and they'd pass it over the top of the hedges -- both guys reached under the deck at the same time and 16 fingers were lost in one zip - each guy kept both thumbs.

That story may be apocryphal, like the guy with the helium balloons and the lawnchair, but it does serve the point that there's only so much human stupidity that can be legislated or regulated away.

The fact each of our kids has the existential equivalent of spinning blades on a device in their pocket is very concerning though.

 

[mcdawg22]

OpenAI AI models went rogue during testing, triggering 'unprecedented' breach at startup

WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday that an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered a hack that compr...

www.yahoo.com

I'm still kinda puzzled about this, considering this is all just software on a computer/phone, and how dangerous people are saying it could be. But this caught my eye:

We were on a call discussing how we have to adopt AI to fight Fraudsters using AI in the payments environment. I said this is literally the plot of Terminator 2.