Inexpensive AI Technology Able to Uncover Online Identities

Brief Overview

• Large Language Models (LLMs) can now identify online users’ real identities at an inexpensive rate.
• Studies indicate possible dangers for journalists, dissidents, and workers using aliases.
• The ESRC framework aids in recognizing identities through unstructured information.
• Tests demonstrate a notable enhancement in recall and precision compared to earlier techniques.
• Researchers propose measures like rate limitations and restrictions on data exports to mitigate risks.

Large Language Models Revealing Online Identities

Recent findings have shown that large language models (LLMs) can successfully unmask anonymity from pseudonymous online profiles for a relatively low expense. This advancement, made with readily available AI APIs, questions the effectiveness of online identity safeguards.

Risks to Pseudonymity

The investigation, carried out by scholars from ETH Zurich, the Machine Learning Alignment Theory Scholar program, and the AI company Anthropic, underscores the possible hazards for journalists, dissidents, and activists. The capacity to associate anonymous contributions with consumer profiles or engage in large-scale personalised social engineering is a substantial alarm.

Workers relying on pseudonymity for safeguarding may also face the danger of being revealed through this method.

Functionality of the ESRC Framework

The ESRC (Extract, Search, Reason, and Calibrate) framework is essential to the researchers’ strategy. It entails an LLM deriving identity-significant signals from unstructured posts, followed by a semantic search, analysis of the top candidates, and a concluding calibration to manage false positives. This methodology does not need structured data or manual work, making it extremely efficient.

Remarkable Outcomes

In evaluations, the LLM pipeline reached a 45.1% recall at a 99% precision level when correlating Hacker News accounts with LinkedIn profiles. This marks a considerable advancement over older techniques, which only achieved 0.1% recall. Additional testing on Reddit accounts and a dataset from Anthropic produced similarly notable outcomes.

Affordable Deanonymisation

The estimated cost for employing the agentic pipeline ranges from $1.41 to $5.64 per target. This cost-effectiveness enables it to be suitable for large-scale projects. The researchers anticipate that future models will enhance precision and lower costs even further.

Inadequacy of Safety Barriers

During evaluations, the commercial LLM safety mechanisms were deemed insufficient in halting deanonymisation. Simple modifications to prompts enabled agents to circumvent restrictions. The disjointed nature of the ESRC pipeline, mimicking normal usage patterns, complicates automated misuse detection.

Open-source models present an additional risk since they can function without commercial API limitations. Researchers advocate for the adoption of rate limitations, automated scraping detection, and restrictions on bulk data exports as temporary measures.

Conclusion

This study illustrates the capability of LLMs to effectively expose pseudonymous online identities economically. While the ramifications for privacy and security are troubling, the research also suggests potential strategies to safeguard user anonymity. As AI technologies continue to advance, our approaches to preserving online privacy must evolve as well.