Predictive Human Preference: From Model Ranking to Model Routing
Table of contents Ranking Models Using Human Preference …. How Preferential Ranking Works …. Correctness of Chatbot Arena Ranking …….. Eval data …….. Results Predicting Human Preference For Each Prompt …. Experiment setup …. Experiment results …….. Domain-specific and query-specific leaderboards Conclusion Human preference has emerged to be both the Northstar and a powerful tool for AI model development. Human preference guides post-training techniques including RLHF and DPO. Human preference is also used to rank AI models, as used by LMSYS’s Chatbot Arena. Chatbot Arena aims to determine which model is generally preferred. I wanted to see if it’s possible