A Conversation is Worth A Thousand Recommendations: A Survey of Holistic Conversational Recommender Systems

Presented at the 5th KaRS Workshop, 2023

Singapore

Victor Li, Hengchang Hu, Yan Zhang, Min-Yen Kan, Haizhou Li

Conversational recommender systems (CRS) generate recommendations through an interactive process. However, not all CRS approaches use human conversations as their source of interaction data; the majority of prior CRS work simulates interactions by exchanging entity-level information. As a result, claims of prior CRS work do not generalise to real-world settings where conversations take unexpected turns, or where conversational and intent understanding is not perfect. To tackle this challenge, the research community has started to examine holistic CRS, which are trained using conversational data collected from real-world scenarios. Despite their emergence, such holistic approaches are under-explored.

We present a comprehensive survey of holistic CRS methods by summarizing the literature in a structured manner. Our survey recognises holistic CRS approaches as having three components: 1) a backbone language model, the optional use of 2) external knowledge, and/or 3) external guidance. We also give a detailed analysis of CRS datasets and evaluation methods in real application scenarios. We offer our insight as to the current challenges of holistic CRS and possible future trends.

Transcript

1. A Conversation is Worth A Thousand Recommendations: A Survey of Holistic Conversational Recommender Systems 5th KaRS Workshop, 2023, Singapore Chuang Li12*, Hengchang Hu1, Yan Zhang1, Min-Yen Kan1, Haizhou Li13 1 National University of Singapore, Singapore 2 NUS Graduate School for Integrative Sciences and Engineering, Singapore 3 The Chinese University of Hong Kong, Shenzhen, China
2. I In nt tr ro od du uc ct ti io on n Conversational Recommender System (CRS) integrate conversational systems with recommendation systems. Prior research in CRS focuses on interactions simulated by entity-level interaction Such a framing focuses on recommendation and decision-making, while neglecting the conversational elements. a) Standard CRS support multi-round interaction only at the entity level
3. H Ho ol li is st ti ic c C CR RS S e en nc co om mp pa as ss se es s m mo or re e Examples of standard and holistic CRS. a) Standard CRS support multi-round interaction only at the entity level b) Holistic CRS support multi- round and multi-goal interaction at the conversation level.
4. I In nt tr ro od du uc ct ti io on n Holistic CRS adopt real, conversation-level interaction and target multiple dialogue goals Holistic CRS perform conversation-level interactions The main challenges in holistic CRS are: Ø Understanding users’ intentions Ø Generate high-quality Reponses Ø Conversational goal planning b) Holistic CRS support multi-round and multi-goal interaction at the conversation level.
5. C CR RS S O On nt to ol lo og gy y Type 0 standard CRS, limited to entity-level inputs and outputs, is restricted in scope of interaction Type 1 holistic CRS takes conversation as input and yields either entity-level recommendations or conversational responses, encompassing query interpretation and tailored linguistic outputs Type 2 holistic CRS is more expansive, accepting and producing diverse formats inputs or outputs including conversations, knowledge, speech, and multimedia Hierarchical structure of CRS in terms of input and output types
6. S Su ur rv ve ey y S Sc co op pe e We aim to conduct an exhaustive survey on holistic CRS, focusing on types 1 and 2 holistic CRS Our survey is purposefully structured to illuminate the evolution and development of holistic CRS, particularly in their handling of conversational data. Hierarchical structure of CRS in terms of input and output types
7. M Ma ai in n A Ap pp pr ro oa ac ch he es s Current holistic CRS approaches are structured around three main components: 1) Language Models (LMs) 2) Knowledge (Structured or unstructured) 3) Guidance (Rec/Topic/Goal/Temporal) A taxonomy of different holistic CRS approaches with three components: 1) requisite backbone language models, and optional components incorporating 2) external knowledge and/or 3) external guidance
8. L La an ng gu ua ag ge e M Mo od de el ls s LMs serve as the backbone for holistic CRS in recommendation response generation HRED-based Sequential Models vs Pre-trained Language Models (PLMs) adopt end-to-end training for holistic CRS application Research focuses on the recommendation task, conversational task, or both tasks Limitations of PLMs mainly come from 1) pre-training 2) online training Example of language models in holistic CRS
9. E Ex xt te er rn na al l K Kn no ow wl le ed dg ge e Knowledge source is required to be transformed into an appropriate representation before the knowledge and textual features can be integrated for the holistic CRS tasks. Applying knowledge to CRS is mainly for node-level entity prediction or edge-level path reasoning. Unstructured knowledge can also be converted to structured knowledge to apply knowledge fusion or semantic alignments Example of external knowledge in holistic CRS
10. External guidance in recommendation, topic/goal planning, and temporal feature representation. Results from these tasks serve as auxiliary guidance for LMs during recommendation response generation. Some models align both external knowledge and guidance, adopting a hybrid strategy that capitalizes on both dimensions for more robust response generation. E Ex xt te er rn na al l G Gu ui id da an nc ce e Example of external guidance in holistic CRS
11. Utilizing external knowledge and guidance: • Current knowledge sources are constrained in item–space, without other resources such as user profile, user-item relationship representation. • LLMs reduces the reliance on external knowledge sources. Recognizing the capability of LMs before introducing appropriate knowledge is crucial. • Instead of “hand-feeding” knowledge or guidance, we should provide LMs with a “knowledge buffet”. D Di is sc cu us ss si io on n
12. D Da at ta as se et ts s & & E Ev va al lu ua at ti io on n M Me et th ho od ds s Entity Information: Ø Scale of datasets Ø Informativeness of datasets Ø Domain of datasets Language Quality: Ø Language Ø Informative Turns Ø Semantic Information Statistical analysis of the datasets in holistic CRS research. #P, #C, #T, #I, #M, #Pos, & #Neg stand for the number of papers, conversations, single turns, items, mentions for each item, positive and negative single turns in training data, IR: Informative turns rate. *Datasets are originally collected in Mandarin Chinese.
13. D Da at ta as se et ts s & & E Ev va al lu ua at ti io on n M Me et th ho od ds s Recommendation Evaluation: Ø Point-wise accuracy (RMSE) Ø Decision support accuracy (F1) Ø Ranking based accuracy (Recall @k) Language Evaluation: Ø Metrics-based methods Ø Human evaluation Analysis of evaluation methods in holistic CRS. #Papers stands for the number of independent works for each evaluation method.
14. C Ch ha al ll le en ng ge es s & & F Fu ut tu ur re e T Tr re en nd ds s 1. Language generation quality and style Ø Elevating language quality to garner positive user feedback Ø Emphasizing preferred language styles to enhance user acceptance 2. User-centric holistic CRS Ø Harnessing multi-modal data from item categories and user profiles Ø Attending to users' personal feedback and latent preferences Ø Incorporation other LMs or AI-generated content (AIGC) 3. Unified model for holistic CRS Ø Enabling seamless integration with task-specific models Ø Adaptation to similar tasks across different domains
15. C Co on nc cl lu us si io on n 1. Our study reveals the necessity and current negligence of holistic CRS research. 2. We describe the characteristic components of holistic CRS: language models, knowledge resources, and external guidance. 3. We systematic review specifically dedicated to holistic CRS with conversational approaches, which further summarized common datasets, evaluation methods and challenges. 4. We hope to attract more attention to explore a more natural and realistic setting in this challenging but promising area.
16. Thank You for Listening! Q&A 1 National University of Singapore, Singapore 2 NUS Graduate School for Integrative Sciences and Engineering, Singapore CRS Paper List https://github.com/lichuangnus/CRS-Paper-List Victor Li Chuang lichuang@u.nus.edu