A tutorial on learning from preferences and choices with Gaussian processes

Preference modelling lies at the intersection of economics, decision theory, machine learning and statistics. By understanding individuals’ preferences and how they make choices, we can build products that closely match their expectations, paving the way for more efficient and personalised applications across a wide range of domains. The objective of this tutorial is to present a cohesive and comprehensive framework for preference learning with Gaussian processes (GPs), demonstrating how to seamlessly incorporate rationality principles (from economics and decision theory) into the learning process. By suitably tailoring the likelihood function, this framework enables the construction of preference learning models that encompass random utility models, limits of discernment, and scenarios with multiple conflicting utilities for both object- and label-preference. This tutorial starts by reviewing the mathematical concept of preference, its properties and the representation via utility functions. We then organise preferential data into three types: object preferences, label preferences and choices. We present five GP-models for object preferences showing in which situations each one is better adapted. For label preferences we present three models based on different noise assumptions. Finally for choice data we present a model for rational choice functions and one for pseudo-rational choice functions. The last part of the tutorial presents five applications of GP-based preference learning. This tutorial builds upon established research while simultaneously introducing some novel GP-based models to address specific gaps in the existing literature.