A gaming media platform built on personalized discovery
The platform surfaces short-form gaming media — clips, highlights, and podcast segments — to users through personalized feeds. Its value depends on showing each user the most relevant moment at the right time, whether they are a long-time user with a rich engagement history or someone opening the app for the first time.
Delivering that experience at scale meant moving well beyond chronological or popularity-based feeds. The platform needed a recommendation system that understood the deep relationships between games, content, and users — and could serve that intelligence to product surfaces and AI agents in milliseconds.
Simple feeds couldn’t capture what users actually wanted
Chronological and popularity-ranked feeds left real engagement on the table. The platform faced several connected challenges:
-
No structural understanding of how games relate to one another across genre, theme, mechanics, franchise, and publisher
-
Weak handling of cold start — new users with no history received generic, undifferentiated content
-
No single way for downstream AI agents to access recommendations without knowing the underlying endpoint topology
-
Newly ingested media moments arrived without recommendation context, limiting how quickly they could be surfaced
-
No consistent, low-latency feature layer to power downstream ML models
The platform needed a unified, graph-driven recommendation engine that could power personalized ranking, cross-game discovery, cold-start coverage, and AI-agent access — all from one governed foundation.
A graph-powered recommendation engine on Databricks
In partnership with Qubika, the platform built the Feeds Recommendation Engine entirely on Databricks — a production ML system spanning six functional components, all deployed as independent Databricks Model Serving endpoints and orchestrated through a unified Databricks Workflows layer.
A config-driven knowledge graph
-
A central knowledge graph encodes the relationships between games, genres, themes, franchises, companies, media moments, and users — ~4.9M nodes and ~10.5M edges.
-
The graph is config-driven: node types, edge types, similarity weights, and model parameters live in Delta configuration tables, so the topology can evolve — and new content verticals like podcasts or music can be added — without changing pipeline code.
-
Spark-native GraphFrames computes centrality and connectivity; game-to-game similarity is a weighted multi-signal score (Jaccard overlaps and exact-match signals) with a confidence function that discounts thin-evidence relationships.
Personalized ranking, discovery, and cold start
-
Feed ranking blends PySpark MLlib ALS collaborative filtering (60%) with graph signals, content features, and popularity — retrained daily on user–moment interactions.
-
When a user has no collaborative-filtering history, weights shift automatically toward graph and content signals, giving full cold-start coverage instead of generic fallbacks.
-
A dedicated discovery surface deprioritizes individual affinity and uses graph centrality to break filter bubbles and expose culturally prominent content.
A natural-language gateway for AI agents
-
An LLM-powered router (“Parrot”), built on Llama 3.3 70B served via Databricks Model Serving, exposes every capability through one natural-language entry point. Agents send a request; the router classifies intent, dispatches to the right endpoint, and returns a normalized response — no consumer needs to know the endpoint topology.
A governed, medallion data foundation
-
The whole system runs on a Delta Lake medallion architecture (Bronze → Silver → Gold) under Unity Catalog governance, with models packaged as MLflow PyFunc and deployed via Databricks Asset Bundles.
-
A three-tier resolution chain — precomputed Gold lookups, then real-time graph traversal, then on-demand model inference — guarantees the system never returns empty results while keeping the common case fast.
Measurable impact
A unified recommendation system
Nine endpoints in production on Databricks Model Serving cover similarity, user recommendations, feed ranking, discovery, ingestion-time enrichment, feature supply, and a natural-language router.
Full recommendation coverage
The three-tier fallback (precomputed → graph → model) means no user ever sees an empty feed, and adaptive cold-start weighting gives 100% coverage for users with no history — a step change from precomputed-only.
Graph-driven relevance
A ~4.9M-node, ~10.5M-edge knowledge graph powers cross-game similarity and multi-hop traversal, with a target of lifting feed-ranking NDCG@10 from a 0.15 alpha baseline toward 0.20.
Built for speed (targets)
Production latency targets: precomputed lookups under 200ms, graph traversal under 2s, and all endpoints under 500ms at p95.
Multi-vertical by design
Adding a new content type — podcasts, music, esports — requires only new config rows and a source pipeline, not engineering changes to the graph code.
Safe, continuous operation
Versioned tables and idempotent workflows let the team rebuild the graph and retrain models in parallel with live traffic, with no service interruption.
A foundation for intelligent content discovery
By building the Feeds Recommendation Engine on Databricks, the platform replaced simple chronological and popularity feeds with a graph-driven, ML-powered recommendation system that understands the deep structure of its content. The config-driven graph, governed medallion foundation, and LLM-powered router give the platform a single, extensible base to personalize every surface — and to bring new content verticals and AI agents online without rebuilding the core.
