UT Arlington Researchers Show How AI Can Turn Your Instagram Posts Into Scam Bait

A team of researchers at the University of Texas at Arlington has found that widely available artificial intelligence tools can scan someone’s public social media posts and use the details to craft highly convincing phishing emails, raising fresh concerns about how personal oversharing online could be weaponized by criminals.

The study, led by Shirin Nilizadeh, an associate professor in UTA’s Department of Computer Science and Engineering, focused on a technique called context-aware spear phishing — building scam messages around real details from a person’s life rather than generic bait. The research team, which also included graduate student Elham Pourabbas Vafa and researcher Sayak Saha Roy, examined 200 public Instagram accounts and found that as few as 10 posts from a single person could supply enough information to generate a targeted, believable message.

“Anyone can be the attacker, and that makes it scarier,” Nilizadeh said.

In practice, the researchers showed how a vacation photo, a concert check-in or even a complaint about a delayed flight could be fed into AI tools and turned into a convincing fake email — say, a hotel offering a discount on a return stay, or an airline offering compensation at just the right moment. The researchers generated and evaluated more than 18,000 AI-written phishing emails, comparing them against real-world phishing samples, and found that participants in a user study generally rated the AI-generated messages as more convincing and less suspicious than the genuine scam emails used for comparison.

The researchers did not test how many people would actually click a malicious link in a live attack; their findings came from a controlled study setting. But they said the results still point to a real vulnerability: anyone with a public social media profile may be handing scammers raw material, without their account ever being hacked.

Existing AI safeguards fell short

The study also tested three AI safety systems built to catch harmful requests before a model responds — Llama Guard, ShieldGemma and WildGuard. With default settings, those tools caught only 0% to 19% of attempts to generate this kind of personalized phishing content.

When the researchers gave the safety systems more specific instructions tailored to recognize context-aware phishing attempts, detection rates rose sharply, reaching as high as 99% in their tests. The researchers withheld the exact prompts they used to generate the phishing emails from their published paper, saying they didn’t want to hand criminals a ready-made blueprint. Nilizadeh is calling on AI companies to strengthen their safeguards against this kind of targeted social engineering.

What people can do

The researchers’ findings don’t mean people need to stop posting online, but they suggest some basic precautions are worth taking:

• Review privacy settings on Instagram, Facebook and other platforms, and consider limiting posts to approved followers rather than the public.
• Avoid posting travel photos or event check-ins in real time; consider waiting until after a trip or outing to share details.
• Check location tags and other identifying details before posting.
• If an email references something you recently did, don’t click the link — go directly to the company’s official app or website, or call a number you’ve verified independently.
• Be skeptical of messages that create urgency or ask for payment or login information, even if the sender’s address looks familiar, since those can be spoofed.
• Use unique, strong passwords for important accounts, enable multifactor authentication where possible, and keep devices and browsers updated.

Data broker and people-search sites can also expose home addresses, phone numbers and other personal details that make phishing emails more convincing, and removing that information — either through a paid removal service or by requesting it directly from individual sites — can reduce what’s available to scammers, though it won’t eliminate everything shared publicly on social media.