The Blessing of Reasoning: LLM-Based Contrastive Explanations in Black-Box Recommender Systems

Modern recommender systems use machine learning (ML) models to predict consumer preferences based on consumption history. Although these “black-box” models achieve impressive predictive performance, they often suffer from a lack of transparency and explainability. While explainable AI research suggests a tradeoff between the two, we demonstrate that combining large language models (LLMs) with deep neural networks (DNNs) can improve both. We propose LR-Recsys, which augments state-of-the-art DNN-based recommender systems with LLMs’ reasoning capabilities. LR-Recsys introduces a contrastive-explanation generator that leverages LLMs to produce human-readable positive explanations (why a consumer might like a product) and negative explanations (why they might not). These explanations are embedded via a fine-tuned AutoEncoder and combined with consumer and product features as inputs to the DNN to produce the final predictions. Beyond offering explainability, LR-Recsys also improves learning efficiency and predictive accuracy. To understand why, we provide insights using high-dimensional multi-environment learning theory. Statistically, we show that LLMs are equipped with better knowledge of the important variables driving consumer decision-making, and that incorporating such knowledge can improve the learning efficiency of ML models. 

Extensive experiments on three real-world recommendation datasets demonstrate that the proposed LR-Recsys framework consistently outperforms state-of-the-art black-box and explainable recommender systems, achieving a 3–14\% improvement in predictive performance. This performance gain could translate into millions of dollars in annual revenue if deployed on leading content recommendation platforms today. Our additional analysis confirms that these gains mainly come from LLMs’ strong reasoning capabilities, rather than their external domain knowledge or summarization skills. 

LR-RecSys presents an effective approach to combine LLMs with traditional DNNs, two of the most widely used ML models today. Specifically, we show that LLMs can improve both the explainability and predictive performance of traditional DNNs through their reasoning capability. Beyond improving recommender systems, our findings emphasize the value of combining contrastive explanations for understanding consumer preferences and guiding managerial strategies for online platforms. These explanations provide actionable insights for consumers, sellers, and platforms, helping to build trust, optimize product offerings, and inform targeting strategies.