Beyond Keywords: Impact of LLM-Powered Search in Online Marketplaces

Information Systems and Operations Management

Speaker: Santiago Gallino. The Charles W. Evans Distinguished Faculty Scholar and Associate
Professor of Operations, Information and Decisions.

Room: Ramanantsoa – HEC Paris Campus

Abstract.

Online marketplaces match customers to products through search, but traditional keywordbased
matching can miss relevant listings when customers and sellers use diIerent words.
Large language models (LLMs) enable marketplaces to leverage semantic search, which
matches queries to listings based on meaning rather than keyword overlap. In this paper, we
study the eIects of LLM-powered semantic search using a randomized field experiment in a
large online marketplace, covering approximately 1.2 million search sessions.
The experiment changes how customer queries are matched to products while leaving the
user interface unchanged. We find that customers submit longer queries and reformulate
their queries less, while exploring search results more deeply and browsing more products.
Semantic search also shifts browsing toward products with fewer reviews and less
prominent sellers. Purchased items are 1.4–1.7% higher priced on average, and conversion
increases within five days. Our findings show that platforms can create operational value by
changing how they interpret customer queries and existing listings, without adding new
information.