AI has made impersonation cheaper, faster, and more convincing than ever. Protecting children now requires rethinking how we verify identity—not just what we see.
Throughout history, impersonation has always required a certain sleight of hand. Before the internet, scammers studied their targets for long periods to mimic their mannerisms. More recently, hackers resorted to faking email addresses, cloning websites, or stealing credentials to trick people into handing over sensitive information.
But whether they were operating online or offline, threat actors needed a certain level of technical expertise or social engineering skills in order to pass as someone else.
That’s no longer true, thanks to AI. With the right AI tools, anyone can generate voices, videos, and synthetic identities with a few prompts and gain access to a domain that advanced attackers have long resided in. The only barrier today is finding the right keywords to look up an AI tool that can do the heavy lifting.
Sadly, children are the most vulnerable to AI-powered impersonation today. The risk hasn’t gone unnoticed: UNESCO recently implored governments to provide AI literacy lessons in schools to help students distinguish between what’s real and what is not. This applies not just to “fake news,” but also to confirming the identity of those with whom they communicate online.
The End of “Seeing Is Believing”
Until recently, people relied on basic evidence to determine whether someone was who they claimed to be online. A selfie with the day’s date and time, a video call, or a familiar voice on the phone was usually enough proof to tell if someone was who they claimed to be.
Generative AI is rapidly eroding those assumptions. Voice cloning tools can recreate a person’s speech patterns from a small audio sample. AI tools can generate highly realistic photos of people who do not exist, or edit images of real people into phony environments or scenarios. And video-generation models can place someone’s likeness into videos that appear authentic to the average viewer.
Consider this: In 2024, a finance worker at the Hong Kong branch of multinational engineering firm Arup was tricked into transferring $25 million to fraudsters. The employee was duped via a deepfake video conference call featuring digitally recreated, AI-generated voices of the U.K.-based CFO and several other trusted colleagues.
This is particularly concerning because the technology has continued to improve since this incident. A video that appeared obviously artificial a few years ago is now nearly indistinguishable from a genuine video call.
Also Read: Why Most Enterprise AI Agent Pilots Never Reach Production
Children Are Especially Vulnerable
Children have always faced unique online risks, from cyberbullying to predation and catfishing. AI-powered impersonation adds a new layer of complexity.
A child who receives a message from someone who appears to be a classmate, friend, coach, or family member may have little reason to question the authenticity of the message. In the absence of trustworthy infrastructure, the already risky online environment is made worse, as synthetic identities can be abused to manipulate conversations, spread misinformation, encourage risky behavior, or establish potentially dangerous relationships.
The emotional impact can be significant, particularly for younger users who may not yet have the experience to identify deception or handle its aftermath.
In such a climate, parents face a difficult challenge, having grown up in an era when fake online profiles were relatively easy to spot. Today’s AI-generated identities are far more sophisticated, meaning traditional safety advice is less effective. While businesses often invest in cybersecurity training and identity verification systems, families rarely have access to comparable protections.
This creates an imbalance: the technology available to attackers is advancing rapidly while the tools available to everyday users remain limited.
Evolving Methods of Verification
The discussion around AI-generated content often focuses on detection: how to identify deepfakes after they have been created. But detection is a reactive strategy; children today need proactive measures.
Until recently, identification verification approaches were the first natural defense against risk. But now, instead of asking whether a piece of content looks real, platforms have developed better ways to verify that a person is who they claim to be. This mirrors earlier evolutions in cybersecurity: recognizing that passwords alone are insufficient, organizations adopted additional verification measures, such as multi-factor authentication, hardware keys, and passkeys. Even more robust are Biometric IDs, but that requires giving sensitive information to websites or tech platforms. or ID document verification, where users upload a government-issued ID and a photo to verify their identity.
As a basic protection step, parents can encourage children to message their friends on communication platforms designed for identity verification, and schools can adopt similar platforms for teachers and students to communicate, with messages verified. Tools such as Google Classroom, Canvas, Schoology, and Microsoft Teams for Education also require students and teachers to sign in using accounts managed by the school district, ensuring that only authenticated members of the school community can access classes, assignments, and messages.
Unfortunately, as AI-generated content becomes commonplace, these verification mechanisms alone are insufficient.
None of these verification methods guarantees the integrity of the communications themselves: someone might be who they claim to be, but their messages can still be easily altered or intercepted by AI.
At the end of the day, the only way to truly confirm a communication online rests on three pillars: identity, liveness detection (is this a real person at all?), and message integrity (confirming the communication has not been altered). In other words, you have to verify both identity and the communication integrity, simultaneously. This can only be meaningfully done online through “No AI” video conferencing applications for real-time communication and dedicated identity-verified communication platforms for asynchronous exchanges.
Also Read: Databricks Pushes Genie From Chatbot to AI Coworker
A Shared Responsibility
Addressing AI impersonation at scale will require collaboration among technology companies, educators, policymakers, and parents.
Technology providers must build stronger safeguards into the platforms on which people communicate. Schools and families need updated digital literacy programs that teach them about AI-generated media and modern identity manipulation tactics. And lawmakers must consider how and where their existing child safety frameworks fail in an environment where identities can be artificially generated, and move to fill those gaps.
Most importantly, parents, schools, and governments must recognize that AI impersonation is no longer just a cybersecurity issue. When anyone can imitate anyone, the scope of damage extends far beyond data breaches and identity theft: it puts the safety of children and vulnerable people at risk, and their emotional and physical well-being.
