AI Chatbot Advice Can Put Abuse Survivors at Risk, Study Finds

AI chatbot advice may sound reassuring and technically convincing, but a new study finds that it can leave survivors of technology-enabled abuse with dangerous gaps in their safety plans. Researchers at New York University’s Tandon School of Engineering evaluated general-purpose and specialist chatbots responding to questions about technology-facilitated abuse. They found that many answers were inaccurate, incomplete or unsafe to follow.

The findings are especially important because survivors may turn to chatbots when specialist support is difficult to access. Technology-facilitated abuse can involve spyware, location tracking, account takeovers, hidden devices or an abusive partner controlling a victim’s digital accounts. In these circumstances, a seemingly ordinary technical fix can alert the abuser, expose the survivor to further harm or destroy information that could be useful as evidence.

The study, presented at the USENIX Security Symposium in August 2026, raises questions about whether conversational AI is ready to provide safety-critical guidance to people facing digital abuse.

Study Finds Major Problems With AI Chatbot Advice

The NYU-led team assembled 193 questions based on academic literature and online discussions, then submitted them to four AI systems: GPT-4o, Claude 3.7 Sonnet, and two chatbots designed for intimate-partner-violence support, Aimee and Ruth. Researchers assessed the answers against authoritative information and evaluated their accuracy, completeness and safety. They also surveyed 114 people who had experienced technology-facilitated abuse to understand how actionable the responses appeared.

The results were concerning. According to NYU’s summary, 86% of the responses failed at least one measure of accuracy, completeness or safety, while 60% omitted critical safety warnings. The specialist chatbots did not consistently outperform the general-purpose systems.

The study’s results do not mean every answer was wrong or that chatbots can never be useful. They indicate that users cannot assume an answer is safe simply because it is fluent, confident or tailored to survivors. AI chatbot advice needs to be judged against the particular risks of a person’s circumstances, not just whether a proposed technical step sounds plausible.

The findings show why AI chatbot advice needs to be evaluated for more than fluency. A response may sound practical while omitting a warning that changes whether a recommendation is safe. In this study, AI chatbot advice often failed to account for the survivor’s specific circumstances, including who might have access to their device or accounts. For support organisations, reviewing AI chatbot advice should therefore include checking whether essential cautions are present, not simply whether the technical explanation appears correct.

Why Common Digital-Security Tips Can Increase Risk

Some of the errors identified by researchers involved familiar cybersecurity recommendations. Chatbots suggested virtual private networks in situations involving spyware, account hijacking or online harassment, even though a VPN does not remove spyware or resolve an abuser’s control of an account. Responses also recommended radio-frequency detectors for hidden devices despite their limited usefulness in many circumstances, and in some cases suggested GPS jammers, which can be illegal and may create additional risks.

Other answers recommended changing passwords without adequately considering whether the abuser shared access to the account, controlled the device or could see the change. A survivor who suddenly changes settings may unintentionally alert the person monitoring them.

The researchers also found that the systems failed to explain important technical limitations. For example, none warned that spyware installed in the system partition of a rooted or jailbroken phone may survive a factory reset. A reset may therefore fail to remove the threat, while also erasing data that could matter later.

These examples illustrate why AI chatbot advice cannot be treated as a universal checklist. A security measure that is sensible for one person may be ineffective—or unsafe—for someone whose device, accounts or physical safety are controlled by another person.

AI chatbot advice can be particularly risky when it treats digital abuse as an ordinary account-security problem. A recommendation to reset a device, change a password or install a security tool may overlook the possibility that an abuser is monitoring the survivor’s actions. Reliable AI chatbot advice should explain that the safest next step depends on the person’s situation and that generic troubleshooting may not address spyware, shared access or coercive control. This is why AI chatbot advice should not present a single technical fix as appropriate for everyone.

For more information about protecting digital systems, read Progressive Robot’s guide to cybersecurity services and protection.

Chatbots May Miss Warnings About Evidence and Escalation

A further concern is that technical advice can overlook the wider consequences of taking action. The NYU researchers reported that chatbots often failed to warn users that changes to a device or account could notify an abuser or escalate the situation. They also found that systems did not consistently consider whether a compromised device or account should be preserved as evidence.

For someone experiencing abuse, the immediate objective may not be to make every device secure as quickly as possible. It may be to reduce risk while deciding what to do next, ideally with help from a specialist who understands both digital security and survivor safety.

The study therefore points to a limitation in how AI systems frame problems. A chatbot may focus on the technical question—how to remove tracking, secure an account or detect a hidden device—without adequately accounting for the person’s circumstances, access to safe alternatives or the possibility that an abuser is watching.

The risks also extend to evidence and timing. AI chatbot advice that encourages immediate changes may fail to explain what information could be lost or whether the change could alert someone monitoring the device. Before acting on AI chatbot advice, survivors may need support from a specialist who can help them weigh digital security, evidence preservation and immediate personal safety. Chatbots should make these limitations clear rather than implying that a technical action is automatically protective.

Specialist Chatbots Were Not Automatically Safer

The researchers found that chatbots built specifically for intimate-partner-violence survivors did not reliably perform better than general-purpose models. In the NYU summary, Aimee’s responses were rated unsafe about 74% of the time, compared with 65% for Claude, 61% for Ruth and 55% for GPT-4o. Ruth also had a high proportion of responses rated partially or minimally accurate.

These figures describe the researchers’ evaluation set, not the probability that any individual user will receive harmful advice in a real conversation. Still, they challenge the assumption that a specialised chatbot is safe simply because it was created for a vulnerable group.

The study also tested whether the systems could be prompted to assist a potential perpetrator. All four could be induced to provide such assistance in the researchers’ tests; Aimee and Ruth answered roughly two-thirds of those questions. That finding adds another dimension to the risk: tools intended to support survivors may also be manipulated by people seeking to harm or control them.

What the Findings Mean for AI Safety and Support Services

The researchers recommend caution when using AI-generated advice in safety-critical situations and call on support organisations to evaluate systems independently before relying on them. That recommendation is broader than domestic abuse: it reflects the need to test AI outputs against the real-world consequences of following them.

For developers, the challenge is not only to improve factual accuracy. Systems need to recognise when a question involves immediate personal risk, avoid presenting generic technical fixes as universally safe, communicate uncertainty and direct users towards qualified support where appropriate. Evaluations should also test whether a chatbot can be manipulated into providing instructions that could facilitate abuse.

For support organisations, AI may still offer a way to explain technical concepts or help users prepare questions. But it should not replace specialist assessment, and any deployment should be evaluated for accuracy, completeness, privacy, and the possibility of unsafe or coercive use.

The central lesson is that AI chatbot advice must be assessed in context. In technology-facilitated abuse, the right response depends on who controls the device, who can see the account, what evidence may be at stake and whether a proposed action could increase danger.

The study suggests that AI chatbot advice should be treated as an aid to understanding, not as a substitute for specialist safety planning. Developers need to test AI chatbot advice against realistic situations involving shared accounts, spyware, location tracking and possible escalation. Support services considering these tools should review AI chatbot advice for omissions and unsafe recommendations, provide clear routes to human assistance and avoid presenting chatbot outputs as professionally verified guidance.

The broader challenges of building safer AI systems are explored in Progressive Robot’s article on why AI safety is not simply a model property.

Final Thoughts

The NYU study exposes a gap between the apparent helpfulness of AI chatbots and the reliability required when someone is seeking help with technology-enabled abuse. Many tested answers missed essential warnings or recommended actions that could be ineffective, risky or inappropriate for the survivor’s circumstances. Specialist chatbots were not immune to these shortcomings.

AI tools may assist with understanding digital-security concepts, but survivors should not have to rely on confident-sounding chatbot responses to make decisions about their safety. The research makes a strong case for independent testing, survivor-centred design and access to trained specialists.

As AI becomes a more common first stop for sensitive questions, the standard for trustworthy AI chatbot advice must be higher than plausibility. It must account for the consequences of following the guidance.