What changed
Visa tested 22 AI models from 12 labs on 15 shopping tasks; 42.8% of runs failed and 54% of failed runs still bought
When
Benchmark released November 5, 2026
Who it affects
Brands and marketplace sellers whose products are found and bought through AI assistants
What to do
Spell out every buying attribute in listings and track returns on AI-referred orders

Key takeaways

  • Visa's 22 tested AI shopping agents failed 42.8% of runs, and 54% of failures still bought, about 23% of all runs.
  • Agents that asked more questions passed 70.2% of the time against 44.4%, but they can only match attributes your listing states.
  • Write every buying attribute and a 'not for you if' line into listings, and track returns on AI-referred orders.

Visa Agentic Commerce Benchmark

42.8%of test runs failed
54%of failed runs still bought
82.2%best model pass rate
34.2%passed all three attempts
Source: Visa, November 2026

What Visa tested and found

Visa's benchmark gave 22 AI models from 12 labs 15 shopping challenges, such as buying a TV, a laptop or a personal care appliance, with three attempts at each. A run passed only if the agent uncovered every requirement the customer had, through conversation, and then made the right call: buy a product that fits, or buy nothing. In one TV scenario the agent has to tease out budget, format compatibility, HDR10+ support and gaming needs.

Models failed 42.8% of runs. The best passed 82.2% of attempts and the worst 28.9%, and in 54% of failed runs the agent still placed an order. All three attempts at a scenario passed only 34.2% of the time. Visa names no models; The Next Web, citing Fortune, reported Google and Meta agents tied near 82%, a Claude agent third at 76% and an OpenAI agent sixth at 64%.

The number the headlines miss: about 23% wrong purchases

Most coverage stops at the failure rate. The figure that matters to a brand is the share of all runs that ended in a bad sale: 42.8% failed times 54% that still bought equals about 23.1%, our arithmetic on Visa's numbers. A wrong purchase costs a sale, a return, return shipping and often a weaker review, and on Amazon it feeds the return rate signals that can hurt a listing.

The five models that uncovered the most hidden buying factors averaged a 70.2% pass rate, against 44.4% for the five that uncovered the fewest. But an agent can only match an answer to your product if your data states it. If your TV page never says HDR10+, an agent that learns the shopper needs it must skip you or guess.

Pass rate by model group

  • Most questions asked70.2%
  • Most expensive 567.1%
  • Cheapest 550.2%
  • Fewest questions44.4%
Source: Visa, November 2026

So what for a brand's money and risk

Trust is still low: Visa's September 2026 Trust Index found 72% of US consumers had used an AI assistant but only 23% trusted AI to handle payments. Agent orders are small today, which makes this the cheap time to fix your data. The five priciest models passed 67.1% of runs against 50.2% for the five cheapest, so shoppers on budget assistants will see more mistakes. Our guide to AI agents for ecommerce covers how each marketplace lets agents in, and the Alexa for Shopping guide explains what Amazon's own agent reads.

Worked example with illustrative numbers

These numbers are invented to show the math. A brand sells a $120 air purifier and gets 500 orders a month from AI assistants. If those orders followed Visa's test pattern, about 23% would be wrong-fit purchases, or 115 orders. Suppose 60% of wrong-fit buyers return the item: 69 returns. At $18 return shipping and handling plus $25 lost on resale of opened units, each return costs $43, so about $2,970 a month, or $35,600 a year. If clearer attributes cut the wrong-fit share to 12%, returns drop to 36 and the cost to about $1,550 a month, saving roughly $17,000 a year on one product.

What sellers and brands should do

Each step names who does it, where, and the metric that shows it worked.

  • Listing manager: for your top 20 products, list the five questions a good salesperson would ask (size, compatibility, use case, budget, who it is for) and make sure each answer appears in the title, bullets or structured attributes in Seller Central. Metric: Listing Quality Dashboard score and fewer blank attributes.
  • Content lead: add a short 'not right for you if' line to bullets or A+ content, so an agent can rule your product out when it does not fit. Metric: return rate for 'not as described' and 'did not fit' reasons in the FBA customer returns report.
  • Catalog owner on Shopify or your own site: publish the same specs in product feeds and structured data, since agents read feeds, not photos. Metric: Merchant Center feed errors at zero for key attributes.
  • Analytics owner: tag orders that come from chatgpt.com, perplexity.ai, gemini.google.com and copilot.microsoft.com referrals, and report their return rate against the site average. Metric: return rate gap on AI-referred orders, reviewed monthly.

Make your listing agent-proof

  • Answer the five salesperson questions in bullets
  • Fill every structured attribute
  • Add a 'not for you if' line
  • Match specs in product feeds
  • Tag AI-referred orders
  • Compare their return rate monthly

When this does not apply

Visa ran its own test on 15 scenarios, and Visa has a stake in agentic commerce, so treat the rates as a signal, not a forecast for your products. Simple, single-attribute products such as a branded refill are far less likely to be bought wrong. If you sell only on Amazon, outside agents are largely blocked there, so your work is listing content for Amazon's own assistant rather than feeds; Shopify stores face a different setup, covered in our Shopify agentic checkout report.

Where PPC Spider helps

Our AI marketing services team audits listing attributes and AI referral tracking, and Amazon listing optimization fills the gaps agents trip on. Follow the ecommerce marketplace news channel for the next agent study, or ask for a free Amazon audit to see which attributes your top listings are missing.

Sources

Get the next change before your competitors do

We send the marketplace changes that matter each week, with what to do next.

By subscribing you agree to our privacy policy. Unsubscribe any time.
Want this done for your account?

We run these playbooks on client accounts every day. Start with a free audit.

Get a free audit