Recommendation and Collaborative Filtering: Effective Systems

Actualizado: 2026-05-03

35% of purchases on Amazon and 80% of content consumed on Netflix come from algorithmic recommendations. Behind that figure are systems processing the history of millions of users to predict which product or content is most likely to interest each person at that moment. The dominant technique is collaborative filtering, and understanding how it works means understanding one of the most important value engines in digital commerce.

Key takeaways

• Recommendation systems divide into two main families: content-based (using item characteristics) and collaborative (using user behaviour).
• Collaborative filtering has two variants: neighbourhood-based (finding similar users) and model-based (factorising the interaction matrix).
• Matrix factorisation and neural network embedding models are the production state of the art.
• The most used evaluation metrics are precision, recall, NDCG, and diversity; each captures a different aspect of system quality.
• The cold start problem — new users without history — is the main weakness of pure collaborative filtering.

Types of recommendation systems

Content-based filtering This approach recommends items similar to those the user has previously interacted with, based on item characteristics: genre, author, price, technical features. A content-based music system would recommend songs by the same artist or genre as those the user already listens to.

Advantage: doesn’t need data from other users; works well from the first item in the history. Disadvantage: tends towards over-specialisation (the user never discovers anything new) and requires a well-labelled catalogue with quality metadata.

Collaborative filtering Rather than looking at item characteristics, it looks at user behaviour. The premise: users with similar past behaviour will have similar future preferences. Netflix doesn’t need to know a series is a thriller to recommend it; it just needs to know that people with a history similar to yours watched and rated it highly.

Collaborative filtering is the dominant approach because it scales better and is content-agnostic — it works equally well for films, products, songs, or news articles.

Collaborative filtering variants

Neighbourhood-based (Memory-based) Searches directly in the users × items matrix to find users most similar to the target user (user-based) or items most similar to those they’ve interacted with (item-based). Similarity is typically measured with cosine similarity or Pearson correlation.

Conceptually simple and explainable — it’s possible to say exactly which similar users influenced the recommendation — but scales poorly with millions of users and items, and suffers with very sparse matrices.

Model-based Instead of searching directly in the matrix, learns a compact representation (model) of users and items. The most important technique is Matrix Factorisation (popularised by the 2009 Netflix Prize):

• The users × items interaction matrix is decomposed into two lower-dimension matrices: latent user vectors and latent item vectors.
• The product of those vectors predicts the expected interaction.
• Training minimises error on known interactions.

Neural network embedding models have outperformed classical factorisation in recent years. Architectures like Neural Collaborative Filtering (NCF) or Two-Tower models learn richer representations capturing non-linearities in preferences.

The cold start problem

The weak point of collaborative filtering is cold start: a new user without history or a new item without interactions has no latent vectors; the system doesn’t know how to recommend them.

Mitigation strategies include:

• Onboarding questions: asking the user to rate a set of representative items at registration.
• Demographic or contextual data: using age, location, or device as initial signals until history accumulates.
• Popularity-based filtering: recommending the most globally popular items until enough user data exists.
• Hybrid models: combining collaborative filtering with content-based filtering; content makes up for scarce interactions.

Amazon combines these approaches: for a new user, it shows sector bestsellers; as the user interacts, the collaborative system takes over.

How a recommendation system is evaluated

Offline evaluation uses historical interaction sets split into training and test. The most used metrics:

• Precision@K: of the K recommended items, what proportion are relevant? Measures accuracy.
• Recall@K: of all relevant items, what proportion appear in the K recommended? Measures coverage.
• NDCG (Normalised Discounted Cumulative Gain): weights relevance by position; a relevant recommendation in position 1 counts more than in position 10. The standard metric in systems where order matters.
• Diversity: measures how different recommended items are from each other. An accurate system that always recommends the same things is deficient.
• Novelty: measures whether the system recommends items the user would not have discovered otherwise.

Offline evaluation has an important limit: it doesn’t capture the effect of the recommendation on real user behaviour. A/B tests in production — measuring clicks, consumption time, or purchases — are the necessary complement, as noted in methodologies for defining objectives: the real business metric validates the system.

Real-world applications and results

Netflix: combines collaborative filtering, content signals, and temporal sequence models (what you’ve recently watched). 80% of consumed content comes from recommendations; the company estimates the system saves over $1 billion annually in avoided churn.

Amazon: the “Customers who bought X also bought Y” recommendation engine is item-based collaborative filtering. It contributes to 35% of sales. The same infrastructure is used in search recommendations and personalised emails.

Spotify: uses neural networks to learn song embeddings from co-occurrences in playlists. The result enables recommending songs from unknown artists that sonically fit the user’s preferences — a use case where content-based filtering alone would be insufficient.

The explainability of these systems is gaining importance, both for regulatory requirements and UX. XAI techniques applied to recommendation systems enable generating explanations like “we recommend this because users similar to you also watched it” — which have a positive effect on user trust.

Conclusion

Collaborative filtering is the heart of the most effective recommendation systems in the digital world. Matrix factorisation and neural embedding models enable personalising experiences at the scale of millions of users with precision that rule-based systems cannot achieve. Cold start and diversity are the two not-fully-resolved challenges; the most robust systems address them with hybrid approaches combining collaborative, content, and contextual signals.