Google has quietly published a groundbreaking research paper that could revolutionize how recommender systems understand user preferences. The paper details a novel approach to help platforms like Google Discover and YouTube move beyond basic interactions to grasp the nuanced, subjective "semantic intent" behind what users truly want to read, listen to, or watch.

This innovation aims to overcome the inherent limitations of current state-of-the-art recommender systems, paving the way for a more detailed, individualized understanding of user desires.

Personalized Semantics: Decoding User Intent

Recommender systems are ubiquitous, predicting what content a user might enjoy next. Familiar examples include YouTube, Google Discover, and Google News, which suggest articles and videos, as well as shopping platforms offering product recommendations.

Traditionally, these systems operate by collecting "primitive user feedback" – data points like clicks, ratings, purchases, and watch history. While effective to a degree, this data struggles to capture the subjective judgments that define human preferences, such as what someone finds "funny," "cute," or "boring."

“Interactive recommender systems have emerged as a promising paradigm to overcome the limitations of the primitive user feedback used by traditional recommender systems (e.g., clicks, item consumption, ratings). They allow users to express intent, preferences, constraints, and contexts in a richer fashion, often using natural language (including faceted search and dialogue).

Yet more research is needed to find the most effective ways to use this feedback. One challenge is inferring a user’s semantic intent from the open-ended terms or attributes often used to describe a desired item. This is critical for recommender systems that wish to support users in their everyday, intuitive use of natural language to refine recommendation results.”

The rise of large language models (LLMs) presents a unique opportunity to leverage natural language interactions, allowing AI to better understand user intent through identifying semantic intent.

The Challenge of "Soft Attributes"

Researchers distinguish between "hard attributes" and "soft attributes." Hard attributes are objective truths, easily understood by recommender systems, such as "genre," "artist," or "director." Soft attributes, however, are subjective and lack definitive ground truth, making them challenging for AI to match with content or products.

The research paper highlights key characteristics of soft attributes:

  • There is no definitive "ground truth" source associating such soft attributes with items.
  • The attributes themselves may have imprecise interpretations.
  • They may be subjective in nature (i.e., different users may interpret them differently).

Solving the problem of these subjective "soft attributes" is central to Google's research, which is aptly titled "Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors."

A Novel Application of Concept Activation Vectors (CAVs)

Concept Activation Vectors (CAVs) are typically used to interpret AI models, helping humans understand the internal mathematical representations (vectors) that models use. Google's researchers ingeniously flipped this application, adapting CAVs to interpret *users* instead.

By translating subjective soft attributes into mathematical representations, this new approach enables AI models to detect subtle intent and personalized subjective human judgments. As the researchers explain:

“We demonstrate … that our CAV representation not only accurately interprets users’ subjective semantics, but can also be used to improve recommendations through interactive item critiquing.”

For instance, a model can learn that different users mean different things by "funny" and then leverage these personalized semantics to make more relevant recommendations. The core problem being solved is bridging the "semantic gap" between how humans speak (using vague, subjective descriptions) and how recommender systems "think" (operating on mathematical vectors in high-dimensional embedding spaces).

The beauty of this approach is that CAVs perform the heavy lifting, making subjective human speech less ambiguous without requiring extensive modification or retraining of the recommender system itself. The researchers state:

“…we infer the semantics of soft attributes using the representation learned by the recommender system model itself.”

They list four key advantages of their method:

  • The recommender system’s model capacity is directed to predicting user-item preferences without needing to predict additional side information (e.g., tags), which often doesn't improve performance.
  • The model can easily accommodate new attributes without retraining, even if new sources of tags, keywords, or phrases emerge.
  • The approach offers a way to test whether specific soft attributes are relevant to predicting user preferences, allowing focus on attributes most critical to capturing user intent.
  • Soft attribute/tag semantics can be learned with relatively small amounts of labeled data, leveraging principles of pre-training and few-shot learning.

How the System Works at a High Level

The researchers provide a concise overview of their system:

“At a high-level, our approach works as follows. we assume we are given:

(i) a collaborative filtering-style model (e.g.,probabilistic matrix factorization or dual encoder) which embeds items and users in a latent space based on user-item ratings; and

(ii) a (small) set of tags (i.e., soft attribute labels) provided by a subset of users for a subset of items.

We develop methods that associate with each item the degree to which it exhibits a soft attribute, thus determining that attribute’s semantics. We do this by applying concept activation vectors (CAVs) —a recent method developed for interpretability of machine-learned models—to the collaborative filtering model to detect whether it learned a representation of the attribute.

The projection of this CAV in embedding space provides a (local) directional semantics for the attribute that can then be applied to items (and users). Moreover, the technique can be used to identify the subjective nature of an attribute, specifically, whether different users have different meanings (or tag senses) in mind when using that tag. Such a personalized semantics for subjective attributes can be vital to the sound interpretation of a user’s true intent when trying to assess her preferences.”

See also: How YouTube’s Recommendation System Works In 2025

Does This System Work? Testing and Results

The research included interesting findings from their tests. For instance, an artificial tag like "odd year" showed an accuracy rate barely above random selection, corroborating their hypothesis that "CAVs are useful for identifying preference related attributes/tags."

They also found that using CAVs significantly improved "critiquing-based" user behavior in recommender systems, leading to better recommendations. The researchers identified four key benefits:

  • Using a collaborative filtering representation to identify attributes of greatest relevance to the recommendation task.
  • Distinguishing objective and subjective tag usage.
  • Identifying personalized, user-specific semantics for subjective attributes.
  • Relating attribute semantics to preference representations, enabling interactions using soft attributes/tags in example critiquing and other forms of preference elicitation.

This approach demonstrably improved recommendations in situations where the discovery of soft attributes is crucial. Future research will explore if soft attributes can also enhance product recommendations, where hard attributes are typically more prevalent.

Key Takeaways and Future Implications

Published in 2024, this research paper has largely gone unnoticed in the broader search marketing community, possibly due to its quiet release. However, its implications are significant.

Google has already tested aspects of this approach with WALS (Weighted Alternating Least Squares), a production algorithm available to developers in Google Cloud. Footnotes in the paper reveal:

“CAVs on MovieLens20M data with linear attributes use embeddings that were learned (via WALS) using internal production code, which is not releasable.”

“…The linear embeddings were learned (via WALS, Appendix A.3.1) using internal production code, which is not releasable.”

"Production code" indicates software actively running in Google's user-facing products. While likely not the core engine for Google Discover, its integration into an existing recommender system demonstrates feasibility.

The system was tested using the public MovieLens20M dataset (20 million ratings), with some tests leveraging Google’s proprietary WALS engine. This lends strong credibility to the idea that this technology can be implemented in live systems without requiring extensive retraining or modification.

The most crucial takeaway is that this research enables recommender systems to effectively leverage semantic data about subjective "soft attributes." Given that Google views Google Discover as a subset of search, and search patterns contribute to content surfacing, it's plausible that this approach could be adopted by Google's own recommender systems. If so, users could experience recommendations that are far more responsive and attuned to their individual, subjective semantics.

The research paper credits Google Research (60%), alongside contributions from Amazon, Midjourney, and Meta AI.

The full PDF is available here: Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors