Beyond Keywords: Impact of LLM-Powered Search in Online Marketplaces

Speaker: Santiago Gallino (Wharton) 

Room Bernard Ramanantsoa 

Abstract:

Online marketplaces match customers to products through search, but traditional keywordbased matching can miss relevant listings when customers and sellers use diIerent words.
Large language models (LLMs) enable marketplaces to leverage semantic search, which matches queries to listings based on meaning rather than keyword overlap. In this paper, we study the eIects of LLM-powered semantic search using a randomized field experiment in alarge online marketplace, covering approximately 1.2 million search sessions.
The experiment changes how customer queries are matched to products while leaving the user interface unchanged. We find that customers submit longer queries and reformulate their queries less, while exploring search results more deeply and browsing more products.
Semantic search also shifts browsing toward products with fewer reviews and less prominent sellers. Purchased items are 1.4–1.7% higher priced on average, and conversion increases within five days. 

Our findings show that platforms can create operational value by changing how they interpret customer queries and existing listings, without adding new information.