In a way, we want LLMs to be objectively correct. But what if we spin it around so that LLMs can learn our subjective preferences and have their own taste that influences their answers?
Notes
This is the image that can be found in the article, describing the methods of taste in different POVs.
Like many consumer-related things (e.g. fashion, interests, music, etc.), taste is constantly evolving - there's never a perfect fit. Are there methods that we can employ to follow that trend? Are there patterns among these trends to develop a more dynamic taste algorithm?
This is what makes computational taste so fascinating: it forces us to ask whether machines can model something that is designed to change.
It means taste is no longer just something people express, it’s something models optimize for.
We can build LLMs that have taste via reward modeling from signals that humans provide + with explicit preference data through LoRe (Low-Rank Reward Modeling).
This can also lead to "Taste-as-a-Service APIs" that power personalized feeds across industries.
As mentioned in the article, models like TAPO (Textual Aesthetics Preference Optimization) and G-Eval are pioneering ways to train LLMs on human-labeled taste preferences.
This means not just understanding what the person is replying with but how they are replying to determine their taste / judgment embedding.
This may also include understanding how your friends' taste fingerprint is evolving and training models to predict how your taste may evolve, critical data for businesses.
Emerging methods like LoRe show how to solve this: by modeling individual preferences as combinations of shared basis functions, you can personalize LLMs to each user’s taste with just a handful of examples. This sidesteps the need for full fine-tuning and avoids hardcoding users into static categories.
