基于 Databricks 构建实时次优报价推荐

一次构建,处处服务:Feature Views 统一批处理与流式特征,同时用于训练和在线服务。
实时个性化:基于实时客户行为对优惠进行排序,p95 延迟为 34 ms。
加速落地:借助开箱即用的 Solution Accelerator 部署端到端推荐系统。
实时个性化取决于能否理解客户当下的需求。当一位零售银行客户查看账户、浏览信用卡或打开贷款计算器时,每一次交互都在传递其当前意图。相关的优惠必须在客户仍处于会话中时触达他们。
难点在于如何让模型获取到这一意图。原始点击流数据必须聚合为按时间窗口划分的特征,用于两个目的:基于历史某一时点上的取值进行训练,以及利用当前取值以毫秒级延迟提供预测服务。分别构建这两条流水线会产生重复的特征定义,这些定义可能随时间发生漂移,进而降低模型准确率。
Databricks Feature Views 通过让你只需定义一次特征、即可同时用于训练和服务来解决这一问题。在本文中,我们将介绍一个开源 Solution Accelerator,它使用 Feature Views 为金融服务业构建实时下一最佳推荐(NBO)系统。
该加速器包含两部分:一个使用批处理特征的推荐系统,以及一个加入流式、会话内意图的扩展。两者均端到端构建在 Databricks 之上,并提供可直接克隆和部署的运行型 notebook。
用于训练与服务的一致、实时特征
实时推荐既要求一致性,也要求速度。特征在训练与服务阶段的计算方式必须一致,以避免训练/服务偏差,而这种偏差会降低模型准确率。在推理时,这些特征还必须在毫秒级内完成检索和打分。
传统上,要同时满足这两项要求,就意味着维护彼此独立的系统:一套用于训练特征的批处理流水线,以及一套为在线服务手写的流式作业。这会带来重复逻辑,并可能随时间推移逐渐发生漂移。
Feature Views 免去了单独定义特征的必要。你只需声明一次逻辑,Databricks 便会在训练和服务中复用它:
批量特征会被物化到离线 Delta 表,用于时点正确的训练;同时物化到在线存储,用于低延迟服务。
流式特征会被持续聚合到在线存储中,而训练所需的历史值则从源事件按时点计算得出。与批量特征不同,它们不会被物化到离线存储。
在训练时,create_training_set API 会检索每个带标签事件发生时已存在的特征值,从而防止未来数据泄漏。在推理时,Model Serving 端点会使用 customer_id 从在线存储中获取批量特征。
两条工作流使用相同的特征定义,避免了分别实现的离线与在线转换之间的不一致。
实时下一个最佳优惠推荐
该加速器为零售银行实现了一个实时的 next-best-offer(NBO)推荐器。当客户浏览账户、查看信用卡或使用贷款计算器时,系统会对报价目录(包括信用卡升级、储蓄产品和个人贷款)进行排序,以在会话期间推荐最相关的选项。
该推荐器将稳定的客户上下文(如余额、消费、客户年限和风险等级)与捕捉会话内意图的近期点击流活动相结合。单个排序模型同时使用这两类数据,而 Feature Views 则让特征定义在离线训练与在线服务之间保持一致。
该加速器包含三个步骤,以可运行的 notebook 形式分布在两个部分中,二者共享同一目录、schema、在线存储和特征定义。
步骤 1:定义批处理特征与流式特征
Feature Views 让你用同一种方式在批处理表和实时 Kafka 流上声明特征:
忠诚度等级、风险等级、收入、客户年限
过去 10 分钟内的品类浏览量
流上的 RollingWindow
最新的会话内意图
30 天品类兴趣
长窗口,最新前沿
派生特征,按请求计算
对于流式特征,将声明指向 Kafka topic 而不是表。Feature Views 会持续把事件聚合到在线存储中,无需单独手写流式作业,也不会重复特征逻辑。
SawtoothWindow 让更长的流式窗口变得切实可行。例如,30 天品类计数在新事件到达时始终保持最新,而无需为每个事件重新计算全部历史。
第 2 步:使用时间点正确的特征进行训练
使用 Feature Engineering SDK,时间点连接会重建每个特征在历史标签对应时刻的取值,防止未来数据泄漏。生成的训练集用于训练一个注册在 Unity Catalog 中的 LightGBM 排序模型。
由于训练和服务使用同一套特征声明,特征不会因为离线与在线转换各自独立实现而出现偏差。
第 3 步:实时提供推荐
排序模型运行在经路由优化的 Model Serving 端点上。每个请求包含 {customer_id, candidate offers, session context}。该端点按键从在线存储中检索客户特征,并直接对全部 40 个优惠打分,无需额外的候选筛选步骤。
第二个经路由优化的 Feature Serving 端点对外暴露特征值,使应用能够展示某个优惠为何被选中,而无需直接查询在线存储。
基准测试延迟、特征新鲜度与吞吐
对实时推荐系统而言,有两项测量最为关键:一次点击多快能变成模型可见的特征,以及训练与服务是否产出完全相同的特征值。
我们使用其最小配置对加速器进行了端到端基准测试:一个小型服务端点和一个单节点在线存储,且未做调优。流式事件约在十分之一秒内变为模型可见,同时训练路径与服务路径产出的值完全一致。
图 1:定义实时推荐系统的三种延迟
每组柱状图显示中位数(p50)和第95百分位(p95)延迟。前两组测量请求延迟,可直接比较。第三组测量特征新鲜度,从事件摄取到在线可用性,而非用户等待的时间。虚线标记了两条服务路径都满足的 50 ms 预算。
在最小端点规格上,一个完整的排序请求(包括在线特征检索和推理)可在 50 ms 预算内完成。加入实时流式特征仅使延迟增加 3-4 ms。
吞吐与容量规划
排序端点持续处理约 56 请求/秒(RPS),而 Feature Serving 端点持续处理约 390 RPS,直到 p95 延迟超过 50 ms。
实测吞吐与并发除以服务时间的结果相差在 15% 以内,这提供了一个实用的容量规划公式:
所需并发 = 4 × ceil(目标 RPS ÷ 在您的 SLO 下实测 RPS)
基于这些结果,要在 50 ms p95 服务级别目标(SLO)内支持 150 排序请求/秒,需要并发 12。
我们使用了加速器的基准测试 notebook,端点已预热,客户端位于同一区域,负载较轻。请求键使用 Zipfian 分布分散到 2,000 个客户,以增加缓存未命中,排序延迟实测为 33 ms p50 和 48 ms p95。
我们测量了每个事件的特征新鲜度,即从 Kafka 写入到该值在在线存储中可被读取的时长。为验证一致性,我们使用空值安全相等(null-safe equality)对每位客户的线下物化值与在线存储值进行了比对。
这些结果存在两点局限。第一,排序端点采用缩容至零(scale-to-zero),因此空闲期后的首个请求可能产生数十秒的冷启动。第二,所报告的延迟合并了特征检索与推理两部分。因此,27 ms 这一测量值无法与仅报告特征读取延迟的基准测试直接比较。
商业价值:更快的迭代,可衡量的结果
该加速器演示的是一个可运行的实时推荐系统,而非已被验证的收入提升。由于它使用的是合成数据,我们并不声称转化率有所改善。相反,它为团队提供了一个衡量业务影响的起点。
更少的流水线维护:单一特征层取代了三条手工维护的流水线。批处理、流式与在线服务共享声明式定义,并通过 Unity Catalog 进行治理与血缘追踪,以支持受监管金融服务中的审计。
更快的落地实现:克隆仓库、部署 bundle 并运行。Part 1 提供带在线特征查询的受治理排序器,Part 2 则增加实时会话内意图,且无需单独的特征栈。
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Build a real-time next-best-offer recommender using Databricks Feature Views)
Build once, serve everywhere: Feature Views unify batch and streaming features for training and serving.
Personalize in real time: Rank offers using live customer behavior with 34 ms p95 latency.
Accelerate implementation: Deploy an end-to-end recommender with a ready-to-run Solution Accelerator.
Real-time personalization depends on understanding what customers want in the moment. When a retail banking customer checks their accounts, explores credit cards, or opens a loan calculator, each interaction signals their current intent. A relevant offer needs to reach them while they're still in the session.
The challenge is making that intent available to the model. Raw clickstream data must be aggregated into time-windowed features for two purposes: training on historical values as they existed at a given point in time, and serving predictions using current values with millisecond latency. Building these pipelines separately creates duplicate feature definitions that can drift over time and degrade model accuracy.
Databricks Feature Views solve this by letting you define features once for both training and serving. In this blog, we'll walk through an open-source Solution Accelerator that uses Feature Views to build a real-time next-best-offer (NBO) recommender for financial services.

The accelerator includes two parts: a recommender using batch features and an extension that adds streaming, in-session intent. Both are built end-to-end on Databricks, with runnable notebooks you can clone and deploy.
Consistent, real-time features for training and serving
Real-time recommendations require both consistency and speed. Features must be computed the same way during training and serving to avoid train/serve skew, which can degrade model accuracy. At inference time, those features must also be retrieved and scored in milliseconds.
Traditionally, meeting both requirements meant maintaining separate systems: a batch pipeline for training features and a hand-written streaming job for online serving. This creates duplicate logic that can drift over time.
Feature Views eliminate the need for separate feature definitions. You declare the logic once, and Databricks reuses it across training and serving:
Batch features are materialized to offline Delta tables for point-in-time-correct training and to an online store for low-latency serving.
Streaming features are continuously aggregated into the online store, while historical values are computed point-in-time from source events for training. Unlike batch features, they aren't materialized to an offline store.
At training time, the create_training_set API retrieves feature values as they existed at each labeled event, preventing future data leakage. At inference time, the Model Serving endpoint fetches batch features from the online store using customer_id.
Both workflows use the same feature definitions, avoiding inconsistencies between separately implemented offline and online transformations.
Real-time next-best-offer recommendations
The accelerator implements a real-time next-best-offer (NBO) recommender for retail banking. As a customer browses accounts, explores credit cards or uses a loan calculator, the system ranks a catalog of offers, including credit card upgrades, savings products and personal loans, to recommend the most relevant option during the session.
The recommender combines stable customer context, such as balances, spending, tenure and risk band, with recent clickstream activity that captures in-session intent. A single ranking model uses both types of data, while Feature Views keep feature definitions consistent across offline training and online serving.
The accelerator includes three steps, delivered as runnable notebooks across two parts that share a catalog, schema, online store and feature definitions.
Step 1: Define batch and streaming features
Feature Views let you declare features over batch tables and live Kafka streams using the same approach:
Loyalty tier, risk band, income, tenure
Category views in the last 10 minutes
RollingWindow on a stream
Freshest in-session intent
30-day category interest
Long window, fresh leading edge
Derived, computed per request
For streaming features, point the declaration at a Kafka topic instead of a table. Feature Views continuously aggregate events into the online store without a separate hand-written streaming job or duplicate feature logic.
SawtoothWindow keeps longer streaming windows practical. For example, a 30-day category count stays fresh as new events arrive without recomputing the entire history for every event.
Step 2: Train with point-in-time-correct features
Using the Feature Engineering SDK, a point-in-time join reconstructs each feature's value as it existed at the time of the historical label, preventing future data leakage. The resulting training set feeds a LightGBM ranker registered in Unity Catalog.
Because training and serving use the same feature declarations, features cannot diverge through separately implemented offline and online transformations.
Step 3: Serve recommendations in real time
The ranker runs on a route-optimized Model Serving endpoint. Each request contains {customer_id, candidate offers, session context}. The endpoint retrieves customer features from the online store by key and scores all 40 offers directly, without an additional candidate-narrowing step.
A second route-optimized Feature Serving endpoint exposes the feature values, allowing the application to show why an offer was selected without querying the online store directly.
Benchmarking latency, feature freshness and throughput
Two measurements matter most for a real-time recommender: how quickly a click becomes a model-visible feature and whether training and serving produce identical feature values.
We benchmarked the accelerator end-to-end using its smallest configuration: a small serving endpoint and a single-node online store, without tuning. Streaming events became model-visible in approximately a tenth of a second, while the training and serving paths produced identical values.
Figure 1: Three latencies that define a real-time recommender
Each pair of bars shows the median (p50) and 95th percentile (p95) latency. The first two measure request latency and are directly comparable. The third measures feature freshness, from event ingestion to online availability, rather than the time a user waits. The dashed line marks the 50 ms budget that both serving paths meet.
A complete ranking request, including online feature retrieval and inference, fits within a 50 ms budget on the smallest endpoint size. Adding the live streaming feature increases latency by just 3-4 ms.
Throughput and capacity planning
The ranking endpoint sustained approximately 56 requests per second (RPS), while the Feature Serving endpoint sustained approximately 390 RPS before p95 latency exceeded 50 ms.
Measured throughput tracked concurrency divided by service time to within 15%, providing a practical sizing formula:
required concurrency = 4 × ceil(target RPS ÷ measured RPS at your SLO)
Based on these results, supporting 150 ranking requests per second within a 50 ms p95 service level objective (SLO) requires concurrency 12.
We used the accelerator's benchmark notebooks with warm endpoints, an in-region client, and light load. With request keys distributed across 2,000 customers using a Zipfian distribution to increase cache misses, ranking latency measured 33 ms p50 and 48 ms p95.
We measured feature freshness per event, from the Kafka write to the value becoming readable in the online store. For consistency, we compared offline materialized values against the online store for every customer using null-safe equality.
These results have two limitations. First, ranking endpoints use scale-to-zero, so the first request after an idle period can incur a cold start of tens of seconds. Second, the reported latency combines feature retrieval and inference. The 27 ms measurement therefore isn't directly comparable to benchmarks reporting feature-read latency alone.
The business case: Faster iteration, measurable results
The accelerator demonstrates a working real-time recommender, not a proven revenue uplift. Because it uses synthetic data, we don't claim improvements in conversion. Instead, it gives teams a starting point to measure business impact.
Less pipeline maintenance: A single feature layer replaces three hand-maintained pipelines. Batch, streaming and serving share declarative definitions, with governance and lineage tracking in Unity Catalog to support auditing in regulated financial services.
Faster implementation: Clone the repository, deploy the bundle and run. Part 1 provides a governed ranker with online feature lookups, while Part 2 adds live in-session intent without a separate feature stack.
A baseline for A/B testing: Compare the streaming, intent-aware ranker against a batch-only model using incremental accepted offers per eligible session and downstream contribution margin, with complaint and opt-out rates as guardrails.
Before production, teams must also implement eligibility, consent, suitability and suppression controls. Ranking offers is a modeling step, not a final offer decision. The accelerator demonstrates the feature engineering and serving infrastructure, not these business rules.
Get started with the Solution Accelerator
The accelerator lets you start with batch features, then add real-time streaming without rebuilding your feature infrastructure.
Part 1 — Feature Views fundamentals: Clone the repository, configure a Databricks CLI profile, deploy the bundle and run Part 1 to build a governed ranker with online feature lookups.
Part 2 — Real-time streaming: Configure a Unity Catalog Kafka connection and Amazon MSK reachability, then run Part 2. This reuses Part 1's catalog, features, online store and ranker.
Demo and monitoring: Launch the recommender Databricks app and open the AI/BI latency dashboard.
Use Declarative Automation Bundles (DABs) to deploy the accelerator:
Deployment order matters: The app depends on the Part 2 serving endpoints, which the notebooks create at runtime. Deploy the jobs, run Part 1 and Part 2, then deploy the app. Everything runs on serverless compute governed by Unity Catalog.
Learn more about Feature Views
Feature Views let you define features once for point-in-time-correct training and low-latency online serving across AWS, Azure and GCP. Explore the accelerator to see how batch and streaming features come together in a real-time recommender.
Feature Views let you define feature transformations once and reuse them across model training and online serving. They support batch and streaming features, with point-in-time-correct historical values for training and low-latency access to current values for inference.
How do Feature Views prevent train/serve skew?
Feature Views use shared feature definitions across training and serving, avoiding inconsistencies caused by maintaining separate transformation logic. The create_training_set API also uses point-in-time joins to reconstruct feature values as they existed at each historical label, preventing future data leakage.
Can Feature Views support real-time streaming features?
Yes. Feature Views can continuously aggregate events from Kafka into an online store without requiring a separate hand-written streaming job. The accelerator demonstrates this using RollingWindow for recent session activity and SawtoothWindow for longer historical windows.
What performance does the next-best-offer accelerator achieve?
In the benchmark configuration, the ranking endpoint achieved 34 ms p95 latency, including online feature retrieval and inference. The ranking endpoint sustained approximately 56 requests per second, while the Feature Serving endpoint sustained approximately 390 requests per second before exceeding the 50 ms p95 latency budget.
What do I need to run the next-best-offer accelerator?
You'll need a Databricks workspace, a configured Databricks CLI profile and the accelerator's GitHub repository. Part 1 deploys the batch-feature recommender using Declarative Automation Bundles. Part 2 additionally requires a Unity Catalog Kafka connection and Amazon MSK connectivity to incorporate live session activity.
Get the latest posts in your inbox
原文为英文,正文由自动抓取并保留原文。
原文发布于 Databricks Blog(Build a real-time next-best-offer recommender using Databricks Feature Views),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。