A recent study showed that AI-powered chatbots can mimic human communication styles to such an extent that distinguishing them from real people is difficult, even when using other AI systems to attempt to detect their identity.
Research Methodology and Models
The scientific website Tech Xplore reported on Tuesday that the study, conducted by researchers from Paris Cit茅 University in France and published in the journal "Computers in Human Behavior Reports," tested seven AI models, including "ChatGPT 4.5," "Claude 3.7 Sonnet," "Gemini 2.5 Pro," and "Grok 3," to determine their ability to distinguish between humans and machines through text conversations.
It noted that researchers allowed the models to ask up to ten questions to the other party, warning them of the possibility of receiving misleading answers. The experiments included 20 human participants, alongside the seven AI models asked to assume human personas.
Test Results
Researchers tested two role-playing methods; the first relied on brief instructions asking the model to act like a human, while the second included a detailed description of a 21-year-old young man with specific interests and hobbies, who uses spontaneous language and occasionally makes simple spelling errors.
The results showed that evaluators failed to detect more than three cases out of 98 tests where AI appeared to be assuming a human persona, while the models were not detected in tests using the detailed persona, according to the report.
Security Challenges
Researchers observed that answers that are overly organized and ideal may raise suspicions that the speaker is not human, while using a persona with realistic details and more spontaneous answers helped reduce these indicators, making the models' style closer to human communication.
These results come amid growing concerns about the use of chatbots to create fake accounts, spread misleading content, or carry out online fraud, as the ability to mimic human behavior may make detecting bot accounts more difficult.
Researchers believe that relying solely on AI tools to verify the speaker's identity may not be sufficient, emphasizing the importance of developing additional means to verify the source of messages, including encrypted signatures, alongside human oversight.
The study highlights the challenges posed by the rapid development of AI to digital communication security and the need for more reliable means to distinguish between human interaction and content generated by automated systems.
Source: Syrian Arab News Agency (SANA)
Comments (0)