Live Session
Thursday Posters
Late Breaking Results
Uncovering ChatGPT's Capabilities in Recommender Systems
Sunhao Dai (Renmin University of China), Ninglu Shao (Renmin University of China), Haiyuan Zhao (Renmin University of China), Weijie Yu (University of International Business and Economics), Zihua Si (Renmin University of China), Chen Xu (Renmin University of China), Zhongxiang Sun (Renmin University of China), Xiao Zhang (Renmin University of China) and Jun Xu (Renmin University of China).
Abstract
The debut of ChatGPT has recently attracted significant attention from the natural language processing (NLP) community and beyond. Existing studies have demonstrated that ChatGPT shows significant improvement in a range of downstream NLP tasks, but the capabilities and limitations of ChatGPT in terms of recommendations remain unclear. In this study, we aim to enhance ChatGPT’s recommendation capabilities by aligning it with traditional information retrieval (IR) ranking capabilities, including point-wise, pair-wise, and list-wise ranking. To achieve this goal, we re-formulate the aforementioned three recommendation policies into prompt formats tailored specifically to the domain at hand. Through extensive experiments on four datasets from different domains, we analyze the distinctions among the three recommendation policies. Our findings indicate that ChatGPT achieves an optimal balance between cost and performance when equipped with list-wise ranking. This research sheds light on a promising direction for aligning ChatGPT with recommendation tasks. To facilitate further explorations in this area, the full code and detailed original results are open-sourced at \url{https://anonymous.4open.science/r/LLM4RS-532C/}.