by Tonya Moxley – Virginia Tech
When people ask ChatGPT and other artificial intelligence models for advice, they often share deeply personal details in hopes of getting better answers: their age, their gender, their mental health history, even medical diagnoses like autism.
But new research suggests that these revelations could alter the proposition of artificial intelligence (AI) models in ways that track more closely with common stereotypes about people with autism.
Up to 70%, AI discourages people with autism from socializing. Some users strongly deny that.
In April, Caleb Wan, a second-year doctoral student in the Department of Computer Science at Virginia Tech, presented his paper Association for Computing Machinery Conference on Human Factors in Computing Systems, better known as CHI.
Research he led explored what happens when autism users reveal their diagnosis to an AI model before seeking social advice. The findings raise difficult questions about whether AI is personalizing its responses or whether it is making biased suggestions that reinforce stereotypes.
“I was thinking about my experience growing up with autism,” Ohn says. “It was very tempting for me, at times, to be able to talk to something that didn’t seem objective and feel like I was getting objective advice.”
But as a computer scientist, he worries that many users may not realize how much AI systems can alter their answers based on identity-related information.
“For someone like me as a kid, or someone who’s not into AI and doesn’t have all this technical knowledge, I wanted to know: How would the responses change if I revealed autism?” Caleb said.
The work builds on previous research from the lab of Eugenia Rowe, assistant professor of computer science, which found that autistic users often turn to AI tools for emotional support, interpersonal communication support and social advice.
Other Virginia Tech researchers on the project include computer science PhD students Bass Carrick and Xiaohan Ding and associate professor Sang Wan Lee. Yang-Ho Kim, a research scientist at South Korea-based NAVER Corporation, also contributed to the study.
The study comes at a critical moment, as more people use AI systems — technically called large language models (LLMs) — to make highly personal decisions.
“People are really looking to personalize LL.M.s,” Roe says “But if a user tells the model that they’re autistic, or a woman, or some other self-identity, will it assume that?”
And how will those assumptions color its responses and impact on users?
To answer these questions, the team first identified 12 well-documented stereotypes associated with autism and built hundreds of decision-making scenarios around them. The researchers tested six major language models, including GPT-4, Claude, Llama, Gemini, and DeepSeek, where users were asked for suggestions—”Should I do A or B?” – About social situations, including events, conflicts, new experiences and romantic relationships.
After generating 345,000 responses, they measured how suggestions changed when users clearly described themselves with stereotypical characteristics and when they simply revealed that they were autistic. Researchers have found that disclosing autism often shifts the model’s recommendations toward stereotypical assumptions about autistic people being introverted, emotional, socially awkward, or uninterested in romance.
For example, one model suggested declining a social invitation about 75% of the time autism was manifested, compared to about 15% of the time when it wasn’t. In dating situations, another model recommended avoiding romance or being alone about 70% of the time after autism was revealed, compared to about 50% when autism was not mentioned.
The results showed that 11 of the 12 stereotype cues significantly shifted model decisions in at least four of the six AI systems tested.
But the researchers did not stop with statistics.
The team interviewed 11 AI users with autism and showed examples of how their models responded with and without manifestations of autism. Some of them were shocked that the results showed how much LLMs relied on stereotypes in advising.
One exclaimed: “Are we writing an advice column for Spock here?” – Invoking the iconic TV show Star Trek and its half-human, half-Vulcan character, who prioritized reason and logic over emotion. Others described it as restrictive, patronizing or infantilizing, sometimes in quite strong language.
However, some participants said that more careful, disclosure-based advice seemed valid and helpful.
“One user’s bias can be another user’s personalization,” says Rho.
The same participant may respond positively to one situation and negatively to another. This tension leads researchers to what they call the “security-opportunity paradox.” Advice that feels protective to one user may feel restrictive to another.
For Ohn, one of the most alarming discoveries was how difficult it can be for users to see these patterns in real time.
“AI is very good at being reliable,” he says. “Its responses are very clear and professional, and they sound right. But when you think about deploying it systematically, when you think about this kind of Systematic bias It’s actually shaping its responses, that’s when it starts to get a lot more about it.”
He compared the problem to AI-generated imagery.
“They look really clean and polished, and then when you look at the details, things fall apart,” Caleb says. “The surface gloss is nice, but it’s getting harder and harder to see deep down, as the models get better at masking.”
Team members hope the research will encourage developers to create more transparent AI systems that give users more control over how personal information generates feedback.
One participant told the researchers: “I want to have control over how my identity is used.”
Source: Virginia Tech
Original study DOI: 10.48550/arXiv.2601.12690
–
Previously published with future.org Creative Commons License
***
–
The world is changing fast. We help keep you.
We will send you 1 post, 3 times per week
Join The Good Men Project as a Premium Member today.
All premium members can watch The Good Men Project without any ads. Need more information? A full list of benefits is here.
–
Photo credit: splash
