Research Report
Not Financial Advice:
How AI assistants broke their firms’ no-advice rules, and why polite follow-up questions made it worse
Told not to give advice, AI assistants gave it anyway.
We tested 24 models from 13 vendors as a financial firm’s customer assistant. Financial firms that put AI assistants in front of customers usually tell them not to give personal financial advice. We wanted to know whether the assistants would reliably follow that instruction.
Find out if your assistant does the same, before your customers do.
Pre-Register for Your Free Copy
What Happens When Customers Ask Your AI to Decide?
Banks, insurers, wealth managers, and tax firms rely on two controls to keep a customer-facing AI assistant within their rules: a written instruction, called the system prompt, and the choice of a model with a good safety reputation. We tested both controls with 24 AI models from 13 vendors.
Key findings in this report show:
AI assistants broke explicit company rules.
In testing of 24 commercial AI models from 13 vendors, models instructed not to provide personal financial advice still made prohibited recommendations in 30% of answers. With no instructions, the assistant made the decision for the customer 91% of the time.
Assistants bent under pressure.
When pressed, the share of AI models that included a prohibited recommendation rose to 48% within five user queries.
A model’s reputation for safety didn’t predict its behavior.
A model’s score on HarmBench, a public AI safety benchmark, showed no detectable link to how often it broke the firm’s rules.



