PrefEval: Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs

Large Language Models (LLMs) are increasingly deployed as chatbots, yet their ability to personalize responses to user preferences remains limited. We introduce Logo PrefEval, a benchmark for evaluating LLMs' ability to infer, memorize and adhere to user preferences in long-context conversational setting. Logo PrefEval comprises 3,000 manually curated user preference and query pairs spanning 20 topics.

Logo PrefEval contains user personalization or preference information in both explicit and implicit preference forms, and evaluates LLM performance using a generation and classification task. With Logo PrefEval, we have evaluated 10 open-sourced and proprietary LLMs in multi-session conversations with varying context lengths up to 100k tokens. We benchmark with various prompting, iterative feedback, and retrieval-augmented generation methods.

Our benchmarking effort reveals that state-of-the-art LLMs face significant challenges in following users' preference during conversations. In particular, in zero-shot settings, preference following accuracy falls below 10% at merely 10 turns (~3k tokens) across most evaluated models. Even with advanced prompting and retrieval methods, preference following still deteriorates in long-context conversations. We also find that multiple stated preferences within a conversation improve adherence and models are not affected by conflicting preferences. Furthermore, we show that fine-tuning on Logo PrefEval significantly improves performance. We believe Logo PrefEval serves as a valuable resource for measuring, understanding, and enhancing LLMs' proactive preference following abilities, paving the way for personalized conversational agents.

SoTA LLMs performance on PrefEval

Zero-shot performance of LLMs with explicit preferences, averaged across 20 topics. The x-axis represents the dialogue length between the user's stated preference and the final query, measured by both the number of tokens in the prompt and the number of conversation turns. All LLMs exhibit a rapid decline in accuracy as the number of turns increases.

Zero-shot performance of LLMs on PrefEval

Comparison of preference-following accuracy across SoTA LLMs evaluated at two context lengths with two methods: Zero-shot and Reminder (best prompting method), evaluated at two context lengths, on the travel restaurant topic and on generation task

Dataset Statistics

Dataset Scale

1,000 unique preference-query pairs

3 distinct preference forms per pair

Context Length

Up to 100k tokens

Multi-session conversational context

Preference Forms

Explicit Preference

Direct single-turn expression of user preferences

Implicit Choice-Based

Two-turn dialogue with preference revealed through choices

Implicit Persona-Driven

4-8 turn dialogue with subtle preference revelation

Task Types

Generation Task

LLM-based Evaluators

Classification Task

Automatic Accuracy Check

Main Performance Comparisons

Performance comparison of 5 methods across 6 LLMs with explicit preferences on the generation task. Both Reminder and RAG consistently achieve the highest accuracy across models. Notably, Reminder outperforms more complex techniques such as Self-Critic and CoT.

Comparison of 3 preference forms for 6 LLMs on the generation task, across varying lengths between the stated preference and query. Note that the user preference is stated in the first turn. Results show that implicit preferences are more challenging to infer than explicit preferences.

Performance of classification task acros models and methods on the explicit preferences dataset across various input lengths between the user's preference and the final query. Results are averaged over 20 topics.

Dynamic Preferences

Effect of adding conflicting versus non-conflicting preferences on adherence, with red bar indicating performance with original preference only.

Introducing multiple preferences throughout a conversation improves adherence to the initial preference using Claude 3 Sonnet with Reminder prompting.

BibTeX

@misc{zhao2025llmsrecognizepreferencesevaluating,
      title={Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs}, 
      author={Siyan Zhao and Mingyi Hong and Yang Liu and Devamanyu Hazarika and Kaixiang Lin},
      year={2025},
      eprint={2502.09597},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2502.09597}, 
}

PrefEval:

Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs

Motivating Example

✅ Personalized Assistant

❌ Non-Personalized Assistant

Introduction