November 9, 2026 · Visa / Fortune
Visa Benchmark Finds AI Shopping Agents Fail 42.8% of Runs
My take: Visa published the first large-scale benchmark of AI agents in e-commerce, and the results are a reminder that delegating purchasing decisions to an agent still requires active oversight. Of 22 models tested across 15 shopping tasks, 42.8% of runs failed. Most critically: in 54% of those failures, the agent still completed a purchase, even when the product didn't meet the user's requirements.
There's a useful signal in the data: models that asked more questions to understand the customer's actual need finished with a 70.2% success rate, compared to 44.4% for those that made silent assumptions. If your company is considering implementing shopping agents, clear instructions aren't a detail — they're the difference between an agent that works with you or against your budget.
It's worth reading these numbers with context: Visa has a direct commercial interest in getting businesses to adopt its agentic payments infrastructure. As both judge and participant in this space, there's no way to know with certainty whether the benchmark is entirely free from bias, so it's reasonable to wait for independent evaluations before treating these figures as definitive.
Is your business ready with the controls and protocols to oversee what an agent would do with your budget?
Want to use these tools? See the unbiased reviews or back to the news.