11-19 更新 实时
首页/ 技术栈/ 正文

Unity Catalog 携手 Google Cloud 无边界湖仓推进互操作

技术栈 来源:Databricks Blog 2026-11-18 00:30 0 阅读 机器翻译
Unity Catalog 携手 Google Cloud 无边界湖仓推进互操作

客户可以从 BigQuery 或 Databricks 访问同一份数据副本,无需重复复制。

Google 和 Databricks 致力于实现共享目录元数据的长期愿景。

Lakehouse 架构正变得前所未有的开放。从开放表格式——Delta Lake 和 Iceberg——起步,如今已演进为目录联邦,实现跨多个平台的零拷贝访问。通过双方的合作与开源贡献,Databricks 和 Google Cloud 持续推动 Open Lakehouse 社区向前发展。

今天,我们与 Google 合作,自豪地宣布在目录互操作性方面达成一个重要里程碑:客户现在可以从 Unity Catalog 和 Google Cloud 的无边界 Lakehouse 访问单一份数据副本。Databricks 与 Google Cloud 的合作持续蓬勃发展,我们致力于让双向互操作性变得无缝。

Unity Catalog 与 Google Cloud 的 Lakehouse:双向联邦

将 Google Cloud 的数据引入 Databricks 非常简单。借助面向 Google Cloud 的 Lakehouse Federation,客户可以在 Unity Catalog 中挂载由 Google Cloud 的 Lakehouse 运行时目录管理的 Iceberg 表,以便在 Databricks 中进行分析。

这些来自 Google Lakehouse 运行时目录的 Iceberg 表随后可由 Databricks Lakehouse(我们的无服务器数据仓库)查询,或通过 Genie 进行分析——Genie 能以自然语言智能回答数据问题,无需代码或 SQL。

互操作性在另一个方向上同样适用。来自 Databricks 的数据也可以引入 Google Cloud 的 Lakehouse(了解更多)。这使使用 BigQuery 等引擎的客户无需复制数据即可读取 Unity Catalog 管理的表。

下一个前沿:让目录元数据可互操作

Databricks 和 Google Cloud 正在通过开放目录元数据,在加强互操作性方面迈出重大步伐。我们相信,业务与治理语义应当可访问,而不应绑定到特定供应商。在 Iceberg 社区,我们帮助推动并落地了目录标签(catalog labels)——这项提案让各目录能够通过键值对(又称标签)共享元数据。这样一来,一个带有“domain=finance”和“status=certified”标签、且其中某列带有“pii=ssn”标签的表,即使该表从一个目录联邦到另一个目录,也仍会保留这些标签(了解更多)。

在智能体时代,共享标签对治理至关重要,因为目录元数据可用于标记敏感数据以进行掩码,并为搜索资产以回答数据问题的智能体提供上下文。通过我们的合作伙伴关系,Databricks 和 Google Cloud 致力于推进开放治理的愿景。随着数据资产不断增长,AI 智能体更深度地嵌入企业工作流,这一共同承诺正变得愈发重要。我们正迈向一个未来:治理与业务上下文能够随数据一同流转,而无需企业在每个系统中重新定义它们。

阅读我们的文档,开始使用 Unity Catalog 与 Google Cloud 无边界 Lakehouse 之间的 Catalog Federation。你也可以在此详细了解 Google 面向 Unity Catalog 的目录联邦。

如果你有兴趣分享你的互操作性需求,或进一步了解我们的互操作性愿景,请联系你的 Databricks 客户团队。

在收件箱中获取最新文章

译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(Advancing Catalog Interoperability with Unity Catalog and Google Cloud’s Borderless Lakehouse)

Customers can access the same copy of data from either BigQuery or Databricks without duplication.

Google and Databricks are committed to a long-term vision of shared catalog metadata.

The Lakehouse architecture is becoming more open than ever. What started with open table formats – Delta Lake and Iceberg – has evolved into catalog federation for zero-copy access across multiple platforms. Through our partnership and Open Source contributions, Databricks and Google Cloud continue to drive advancements in the Open Lakehouse community.

Today, we're proud to announce a significant milestone in catalog interoperability in collaboration with Google: customers can now access a single copy of data from both Unity Catalog and Google Cloud's borderless Lakehouse. The Databricks and Google Cloud partnership continues to thrive, and we're committed to making bi-directional interoperability seamless.

Unity Catalog and Google Cloud's Lakehouse: Bi-Directional Federation

Bringing data from Google Cloud into Databricks is simple. With Lakehouse Federation to Google Cloud, customers can mount Iceberg tables managed by Google Cloud's Lakehouse runtime catalog in Unity Catalog for analysis in Databricks.

These Iceberg tables from Google's Lakehouse runtime catalog can then be queried by Databricks Lakehouse, our serverless data warehouse, or analyzed via Genie, which intelligently answers data questions from natural language – no code or SQL required.

Interoperability works in the other direction as well. Data from Databricks can also be brought into Google Cloud's Lakehouse (learn more). This allows customers using engines like BigQuery to read Unity Catalog managed tables without data copy.

The Next Frontier: Making Catalog Metadata Interoperable

Databricks and Google Cloud are taking major strides to strengthen interoperability by making catalog metadata open. Our belief is that business and governance semantics should be accessible rather than tied to a specific vendor. In the Iceberg community, we helped drive and land catalog labels - a proposal that enables catalogs to share metadata through key-value pairs (a.k.a labels). That way, a table tagged as "domain=finance" and "status=certified" with a column tagged as "pii=ssn" continues to be tagged even as the table is federated from one catalog to another (learn more).

Diagram of tags on table t1 syncing between Unity Catalog and Lakehouse through catalog federation and IRC.

Shared labels are critical for governance in the era of agents as catalog metadata can be used to flag sensitive data for masking and provide context to agents searching for assets to answer data questions. Through our partnership, Databricks and Google Cloud are committed to advancing the vision of open governance. As data estates grow and AI agents become more embedded in enterprise workflows, that shared commitment becomes increasingly important. We're headed towards a future where governance and business context can travel with data without requiring enterprises to redefine them in every system.

Get started with Catalog Federation between Unity Catalog and Google Cloud's borderless Lakehouse by reading our documentation. You can also learn more about Google's catalog federation to Unity Catalog here.

If you're interested in sharing your interoperability needs or learning more about our vision for interoperability, please reach out to your Databricks account team.

Get the latest posts in your inbox

原文为英文,正文由自动抓取并保留原文。

本文由采集器自动同步

原文发布于 Databricks Blog(Advancing Catalog Interoperability with Unity Catalog and Google Cloud’s Borderless Lakehouse),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗