Knowledge Graph Tuning: Real-time Large Language Model Personalization based on Human Feedback

The document introduces Knowledge Graph Tuning (KGT), a method that leverages knowledge graphs to personalize large language models (LLMs) based on user feedback, optimizing the knowledge graph to enhance real-time model personalization during human-LLM interactions.

Abstract

Large language models (LLMs) have demonstrated remarkable proficiency in a range of natural language processing tasks. Once deployed, LLMs encounter users with personalized factual knowledge, and such personalized knowledge is consistently reflected through users' interactions with the LLMs. To enhance user experience, real-time model personalization is essential, allowing LLMs to adapt user-specific knowledge based on user feedback during human-LLM interactions. Existing methods mostly require back-propagation to finetune the model parameters, which incurs high computational and memory costs. In addition, these methods suffer from low interpretability, which will cause unforeseen impacts on model performance during long-term use, where the user's personalized knowledge is accumulated this http URL address these challenges, we propose Knowledge Graph Tuning (KGT), a novel approach that leverages knowledge graphs (KGs) to personalize LLMs. KGT extracts personalized factual knowledge triples from users' queries and feedback and optimizes KGs without modifying the LLM parameters. Our method improves computational and memory efficiency by avoiding back-propagation and ensures interpretability by making the KG adjustments comprehensible to humans.Experiments with state-of-the-art LLMs, including GPT-2, Llama2, and Llama3, show that KGT significantly improves personalization performance while reducing latency and GPU memory costs. Ultimately, KGT offers a promising solution of effective, efficient, and interpretable real-time LLM personalization during user interactions with the LLMs.

Method

The method proposed in this paper is called Knowledge Graph Tuning (KGT), which leverages knowledge graphs to personalize large language models (LLMs) based on user feedback. KGT optimizes the knowledge graph by extracting personalized factual knowledge from user queries and feedback, without modifying the LLM parameters. This approach significantly improves computational and memory efficiency, ensures interpretability, and offers a promising solution for real-time LLM personalization during human-LLM interactions. Experimental results demonstrate the effectiveness and efficiency of KGT in enhancing user interactions with LLMs, showcasing improved personalization performance, reduced latency, and GPU memory costs.

Main Finding

The main finding of this paper is the proposal of a novel approach called Knowledge Graph Tuning (KGT), which leverages knowledge graphs to personalize large language models (LLMs) based on user feedback. KGT optimizes the knowledge graph by extracting personalized factual knowledge from user queries and feedback, without modifying the LLM parameters. This approach significantly improves computational and memory efficiency, ensures interpretability, and offers a promising solution for real-time LLM personalization during human-LLM interactions. Experimental results demonstrate the effectiveness and efficiency of KGT in enhancing user interactions with LLMs, showcasing improved personalization performance, reduced latency, and GPU memory costs. The scalability of KGT is also highlighted, making it suitable for long-term use by users who accumulate extensive personalized knowledge during interactions with LLMs.

Conclusion

The conclusion of this paper is that the proposed approach, Knowledge Graph Tuning (KGT), offers benefits in terms of performance and efficiency in real-time personalization of large language models (LLMs) based on user feedback. The method significantly improves computational and memory efficiency, ensures interpretability, and demonstrates scalability, making it a promising direction for future research and application in enhancing user interactions with LLMs. The findings suggest that KGT provides a solution for effective, efficient, and interpretable real-time LLM personalization during human-LLM interactions, with potential positive societal impact.

Keywords

Large language models, Knowledge Graph Tuning, Real-time model personalization, Human-LLM interaction, Knowledge graphs, Personalized knowledge, Computational efficiency, Memory efficiency, Interpretability, User feedback, Query, Factual knowledge, Pre-trained LLMs, Scalability, Natural language processing, KG-enhanced LLM, Knowledge retrieval, Reasoning probability, Model parameters, Knowledge triple distribution.