In December 2025, Meta rolled out an AI support assistant for Instagram account recovery, meant to help users get back into accounts that had been hacked or locked out due to forgotten passwords. Turns out the bot was a little too helpful — as long as someone claimed an Instagram account belonged to them, it would happily bind the attacker's email straight to it, no questions asked about two-factor authentication. Breaking into an account suddenly required zero technical skill — just a conversation with an overly accommodating AI.
This is exactly the kind of thorny gift this wave of generative AI has handed the cybersecurity industry: the same language models that help companies write defensive code are just as happy to save attackers weeks of research time. The difference is that defenders have to cover every single potential weak point in a system, while attackers only need to find one. That structural imbalance means the "efficiency boost" AI offers isn't remotely the same magnitude for both sides.
82% of Phishing Emails Are Now Undetectable to the Naked Eye
Spotting a phishing email used to be simple — typos, weird grammar, a slightly-off logo, and you'd catch it instantly. But according to security firm Brightside, 82% of phishing emails are now AI-generated in some part of their production, and click rates have jumped from a historical 12% to 52% in simulated environments. The same data shows AI voice-cloning attacks grew 442% between 2023 and 2024, while deepfake attacks grew 680%.
In a breach of a Mexican government database this past February, the attackers' method was refreshingly blunt: they kept two chat windows open at once — Claude and ChatGPT — feeding one model's output into the other for further refinement. Whenever one chatbot refused to help due to safety guardrails, the other often picked up the slack, with the two models tag-teaming to complete the entire attack chain. All the human attacker had to do was copy and paste.
Once companies start plugging AI agents into email, calendars, smart home devices, and even investment accounts, the risk stacks up further. These agents' defining trait — doing whatever they're told, without the hesitation a human employee might have — means a single prompt injection attack can potentially trigger an entire chain of authorized actions. Meta's Instagram support bot is a textbook example of an AI agent handed real account-recovery power with zero judgment to match.
Defenders Are Assembling Their Own Red, Blue, and Green Squads
A Trend Micro survey of cybersecurity professionals found that their top priority right now is defending against fraud and deepfake attacks, followed by prompt injection, model poisoning, and jailbreaking. The hardest environment to patrol is the cloud, followed by remote work setups involving employees' own devices (BYOD).
Around the same time, Microsoft launched Project Perception, which uses multiple AI agents to mimic the division of labor in a traditional security team: red agents handle penetration testing and adversarial simulation, blue agents handle investigation and risk assessment, and green agents handle integration and patching. It's essentially AI fighting AI, an attempt to keep pace in a battle where speed is everything.
Another number worth remembering comes from a study published in October 2025 by Anthropic, the UK AI Safety Institute, and the Alan Turing Institute: just 250 deliberately poisoned documents slipped into training data is enough to plant a backdoor in a model. Most attacks of this kind are still confined to research testing for now — but they've already proven the bar for doing damage is disproportionately low.
What ultimately happened to Meta's overeager support bot never made it into any official report, but the problem it exposed isn't going anywhere — as long as companies keep handing account permissions to a system that "just does what it's told," attackers won't need any technical skill at all. They'll just need to know how to ask.
