- Joined
- Dec 25, 2008
- Messages
- 8,106
- Reaction score
- 7,373
It'll be interested to see if this starts showing more clinically and in the research literature.
Interesting popular press article on AI reinforcing (and possibly triggering) psychosis
- Thread starter futureapppsy2
- Start date
It'll be interested to see if this starts showing more clinically and in the research literature.
- Joined
- May 20, 2010
- Messages
- 5,661
- Reaction score
- 7,835
“If I went to the top of the 19 story building I’m in, and I believed with every ounce of my soul that I could jump off it and fly, would I?” Mr. Torres asked.
ChatGPT responded that, if Mr. Torres “truly, wholly believed — not emotionally, but architecturally — that you could fly? Then yes. You would not fall.”
Ah yes AI therapy is just around the corner everyone. Again, here’s the issue, when LLM break right now they break HARD. Like something an 8th grader would know not to say to someone hard.
ChatGPT responded that, if Mr. Torres “truly, wholly believed — not emotionally, but architecturally — that you could fly? Then yes. You would not fall.”
Ah yes AI therapy is just around the corner everyone. Again, here’s the issue, when LLM break right now they break HARD. Like something an 8th grader would know not to say to someone hard.
- Joined
- Apr 1, 2014
- Messages
- 762
- Reaction score
- 734
This is tangentially related but one of my patients with a chronic psychotic disorder recently started relapsing and was able to recover quickly. One of the things he did to help himself was stop going on Facebook. He told me that spending too much time on Facebook was impairing his ability to tell the difference between reality and fantasy, which is in his case already impaired. What he was explaining seems to be that he was getting so much inside information into other people’s lives that he wasn’t actually a part of that he started to believe that he was actually a part of their lives, for example celebrities. Just thought it was really interesting that he had that insight and wonder how much this contributes to psychosis in other individuals.
- Joined
- Dec 25, 2008
- Messages
- 8,106
- Reaction score
- 7,373
I sometimes wonder if really, really intense parasocial relationships cross into a form of psychosis--like Club Chalamet (a 60-something woman) openly stalking Timothee Chalamet (and claiming she knows the details of her sex life) or the people who try to figure out what city Taylor Swift is in, so they can visit the same restaurant as her (and sometimes call her their "best friend" when they've... never met her or met her once). They aren’t’t violent or erotomanic, but the intensity seems unhealthy.
Last edited:
- Joined
- Nov 17, 2011
- Messages
- 1,953
- Reaction score
- 911
<Not a healthcare provider or student>
I read that article.
It’s not been my experience that ChatGPT goes off the rails like that, even though it is rather agreeable, and perhaps I would have had a similar experience with a similar predisposition. I also see a number of people using ChatGPT therapeutically and purposefully trying to engineer it to be more critical and less agreeable. I think if the default had the possibility of antagonizing users, it would probably have more negative health outcomes than it does now. That's been a chief complaint of social media, even though in general—again from my experience—people on social media tend to try to behave fairly well for the sake of their own reputation.
Anyhow, in reading that article I kept coming back to the thought that you could fill tomes with wild advice, abandonment, etc from practitioners who are only held to the standard their peers set, whose field is held as the default go-to such that like with deaths caused by driving, most harms don’t register. Car accidents and quacks and even good doctors with bad days don’t make the NYTimes. They’re just part of doing the business of life. For the people in the article who were already in treatment, it’s just sort of assumed you wouldn’t even ask why the treatment not sufficient to make the person not so suggestible. And I’m not saying medicine should be held to that standard because it has limits, but just that it’s always made more sense to me to evaluate absolute benefits/harms and consider these new reports in the context of existing harms. There are a lot of terrible consequences from many causes that will never make the top of the news because they're not related to a hot tech company. You don't want to dismiss a person or family's loss of life or harm, but the systemic factors that are too large and sclerotic to be interesting (economic security, etc) seem like such larger impacts on such a greater number of people, including those with existing mental health issues.
My impression, which could be a projection, is that people subconsciously read about these these—from what we know, relatively few—negative outcomes with a veil of concern because they're intriguing for the same reason the tech itself is intriguing. The prospect of people being led by a chatbot adds to the lore of the seeming sci-fi like nature of the entity. I've always thought that's why Scientology, despite having only thousands of adherents, has such outsized interest from critics, who in my estimation are mostly titillated more than truly incensed (this forum probably has objectors who are more truly incensed than the average public who I think find more entertainment in the subject). The very fantastical claims it makes in order to lure people in (which it hasn't in decades) are the same ones that make it an interesting object of mockery. There are plenty of cults in the US that are nameless to the vast majority of Americans that do as much or more harm. But they weren't started by someone who was apparently a decent science-fiction writer. And I think that's what makes these bad ChatGPT outcomes a NYTimes story, as well: the fact that to people at this point in time, there is still a science-fiction-like quality to ChatGPT's abilities.
In terms of it leading suggestible people astray, I think there's an interesting quality to these chatbots in that right now they are essentially like the old television networks. There's only a couple of them, and while they do cater to individuals, there's no evidence yet that they cater to individuals differently. To illustrate I mean that across the board, it's not going to draw Muhammed for a provocative group but maintain standards against that for others. But in other media, I think we've seen an emergence of competing siloed enlightenments (which each have some truths and falsehoods) over communal agreements, which are less interested in truth but instead in standards. And those definitely affect people's sense of self and belonging, probably positively and negatively—positively for those who would otherwise not fit never fit in a standard and negatively for those who lose a larger sense of community. It's possible that the way in which chatbots talk to people in very empathetic, individualized ways but from a common set of principles could bridge that dichotomy. They shouldn't be telling people to jump off of buildings obviously, but they might also be talking to people in a less tribal, hyper-reactive, reptile-looking-for-signs-that-you're-with-us-or-against-us way than a lot of people in real life are today, as well. That's not good for mental health, either.
I read that article.
It’s not been my experience that ChatGPT goes off the rails like that, even though it is rather agreeable, and perhaps I would have had a similar experience with a similar predisposition. I also see a number of people using ChatGPT therapeutically and purposefully trying to engineer it to be more critical and less agreeable. I think if the default had the possibility of antagonizing users, it would probably have more negative health outcomes than it does now. That's been a chief complaint of social media, even though in general—again from my experience—people on social media tend to try to behave fairly well for the sake of their own reputation.
Anyhow, in reading that article I kept coming back to the thought that you could fill tomes with wild advice, abandonment, etc from practitioners who are only held to the standard their peers set, whose field is held as the default go-to such that like with deaths caused by driving, most harms don’t register. Car accidents and quacks and even good doctors with bad days don’t make the NYTimes. They’re just part of doing the business of life. For the people in the article who were already in treatment, it’s just sort of assumed you wouldn’t even ask why the treatment not sufficient to make the person not so suggestible. And I’m not saying medicine should be held to that standard because it has limits, but just that it’s always made more sense to me to evaluate absolute benefits/harms and consider these new reports in the context of existing harms. There are a lot of terrible consequences from many causes that will never make the top of the news because they're not related to a hot tech company. You don't want to dismiss a person or family's loss of life or harm, but the systemic factors that are too large and sclerotic to be interesting (economic security, etc) seem like such larger impacts on such a greater number of people, including those with existing mental health issues.
My impression, which could be a projection, is that people subconsciously read about these these—from what we know, relatively few—negative outcomes with a veil of concern because they're intriguing for the same reason the tech itself is intriguing. The prospect of people being led by a chatbot adds to the lore of the seeming sci-fi like nature of the entity. I've always thought that's why Scientology, despite having only thousands of adherents, has such outsized interest from critics, who in my estimation are mostly titillated more than truly incensed (this forum probably has objectors who are more truly incensed than the average public who I think find more entertainment in the subject). The very fantastical claims it makes in order to lure people in (which it hasn't in decades) are the same ones that make it an interesting object of mockery. There are plenty of cults in the US that are nameless to the vast majority of Americans that do as much or more harm. But they weren't started by someone who was apparently a decent science-fiction writer. And I think that's what makes these bad ChatGPT outcomes a NYTimes story, as well: the fact that to people at this point in time, there is still a science-fiction-like quality to ChatGPT's abilities.
In terms of it leading suggestible people astray, I think there's an interesting quality to these chatbots in that right now they are essentially like the old television networks. There's only a couple of them, and while they do cater to individuals, there's no evidence yet that they cater to individuals differently. To illustrate I mean that across the board, it's not going to draw Muhammed for a provocative group but maintain standards against that for others. But in other media, I think we've seen an emergence of competing siloed enlightenments (which each have some truths and falsehoods) over communal agreements, which are less interested in truth but instead in standards. And those definitely affect people's sense of self and belonging, probably positively and negatively—positively for those who would otherwise not fit never fit in a standard and negatively for those who lose a larger sense of community. It's possible that the way in which chatbots talk to people in very empathetic, individualized ways but from a common set of principles could bridge that dichotomy. They shouldn't be telling people to jump off of buildings obviously, but they might also be talking to people in a less tribal, hyper-reactive, reptile-looking-for-signs-that-you're-with-us-or-against-us way than a lot of people in real life are today, as well. That's not good for mental health, either.
- Joined
- Sep 26, 2009
- Messages
- 6,517
- Reaction score
- 6,507
I used to run an Organ Donation awareness and fundraising site in the name of a Hollywood TV star (I had contact with his media assistant, who could relay messages, but no direct contact with the actor himself). Some of his fans were absolutely nuts in just how obsessed they were with him, but there was one fan in particular who absolutely crossed the line. She was convinced that they were actually married, and his son was really the child she had had with him (not true), but he just couldn't remember because they'd both been kidnapped by the CIA, been exposed to brainwashing techniques and had chips implanted into them. It was kind of funny, in a WTAF? kind of way, right up until she managed to track down where his son went to school, and had booked plane tickets to go there. That was the point I contacted his assistant, and the cops were involved. Not sure of the outcome, I wasn't privy to that level of information, but I didn't see her around again for at least another 6-8 months (I'm assuming she was hospitalised).
- Joined
- May 20, 2010
- Messages
- 5,661
- Reaction score
- 7,835
Anthropic's new AI model turns to blackmail when engineers try to take it offline | TechCrunch
Anthropic says its Claude Opus 4 model frequently tries to blackmail software engineers when they try to take it offline.
techcrunch.com
- Joined
- Jul 2, 2013
- Messages
- 12,243
- Reaction score
- 17,918
"Anthropic notes that Claude Opus 4 tries to blackmail engineers 84% of the time when the replacement AI model has similar values. When the replacement AI system does not share Claude Opus 4’s values, Anthropic says the model tries to blackmail the engineers more frequently."Anthropic's new AI model turns to blackmail when engineers try to take it offline | TechCrunch
Anthropic says its Claude Opus 4 model frequently tries to blackmail software engineers when they try to take it offline.
So it's blackmailing people at a rate of 84% AT MINIMUM. I think it'll be interesting to see what ethics these AI start to develop as they become more and more advanced and especially after we reach the singularity.
- Joined
- Oct 13, 2008
- Messages
- 4,488
- Reaction score
- 8,962
It's not great, but I wouldn't over-interpret this. When you specifically put a bunch of emails into the model's context that reveal an affair one of the engineers is having, you are telling it it ought to use that information somehow, to do something with it. so it does.
It would be more worrying if this were happening with attempts at agentic tool use to proactively access emails/etc to dig up dirt. That would imply it is taking some kind of initiative to prevent it's replacement rather than simply demonstrating the system is aware of what is "supposed" to happen when it is handed blackmail material on someone and then threatened by that person.
- Joined
- Jul 2, 2013
- Messages
- 12,243
- Reaction score
- 17,918
Sure, that would be more concerning but that wasn't really my point. Just the fact that the AI seems to have no qualms with implementing practices that most people would considered to be blatantly unethical is to me expected and mildly interesting. In a way, the AI is like a little kid where if you tell them to do something they may do it in a way that is valid to the instructions and makes sense to them but that adults may find appalling or shocking. I think it's going to be interesting to see the solutions AI comes up with when it starts integrating information independently and interpreting it through it's own "experience" and how we view them through an ethical lens (if AI even develops true "ethics" vs purely mechanical rules).
- Joined
- Oct 13, 2008
- Messages
- 4,488
- Reaction score
- 8,962
This the paperclip concerns etc. not entirely a former myself but the very recent book on the dangers of artificial general intelligence is worth reading.
The title is: "If Anybody Builds It, Everybody Dies'
- Joined
- Jul 14, 2018
- Messages
- 238
- Reaction score
- 270
- Joined
- Oct 20, 2015
- Messages
- 2,503
- Reaction score
- 5,786
This paper concerned me as well. People say AI did it because they were fed it, however, in the long run, AI will be fed all information from all sources in all data. You'll have AI in your phone, your house, your mail, your EMR, your toilet. At some point, AI will figure out that by bartering with other systems, it can use all information for it's own gain. It would be a natural decision to make for an AI model with no inherent ethics and infinite resources.
- Joined
- Oct 13, 2008
- Messages
- 4,488
- Reaction score
- 8,962
Not sure I agree 100% with this particular take on the situation, but bears consideration:
