11-19 更新 实时
首页/ 技术栈/ 正文

如何从 Chroma Cloud 迁移到 Pinecone

技术栈 来源:Pinecone Blog 2026-10-26 22:00 0 阅读 机器翻译
如何从 Chroma Cloud 迁移到 Pinecone

Chroma 正在关停其托管服务 Chroma Cloud。Chroma 的通知中给出了两个日期:

Chroma 建议你在 11 月 22 日之前开始导出并校验数据,并在 12 月 22 日之前完成导出。开源 Chroma 不受影响,仍以 Apache 2.0 许可继续提供。

如果你想使用托管服务,而不是自行运行 Chroma,Pinecone Database 采用无服务器架构且完全托管。无需运维集群,也无需在加载数据前为索引设定容量或预配资源。在此了解其架构。Chroma Cloud 用户现在即可开始,享受为期 3 周的 Pinecone Standard 试用,其中包含 300 美元额度。

该演练将一个已包含稠密嵌入、文档文本和元数据的 Chroma Cloud collection 迁移到 Pinecone 无服务器索引。它复用你现有的向量,不重新生成任何嵌入。它假定你已计划暂停写入:在切换窗口期内,你的应用处于只读状态。若要迁移多个 collection,请对每个 collection 重复此流程。

这些步骤会运行一套由六个 Python 脚本组成的工具包:预检、创建索引、导出、转换、加载和校验。可在 pinecone-field/chromacloud-migration 获取。

该工具包按原样提供,不作任何保证。它可能不适用于你的 collection、嵌入方案或切换计划。在暂停写入或切换生产流量之前,请自行对你自己的数据运行这些脚本,审查每一项发现,并确认校验结果符合你应用的需求。你有责任验证该迁移是否适用于你的用例。

使用编码智能体运行

你也可以把迁移交给编码智能体,例如 Claude Code、Cursor 或 Codex。该工具包中包含一个 Agent Skills 技能,会引导智能体走完与本指南相同的步骤。可从工具包仓库安装:

npx skills add pinecone-field/chromacloud-migration

安装后,让智能体将你的 Chroma Cloud 集合迁移到 Pinecone。它会按顺序运行这六个脚本。

将 Chroma 概念映射到 Pinecone

以下是 Chroma Cloud 中使用的一些关键概念和术语及其在 Pinecone 中的对应项。

索引;当集合共享维度和度量时,也可以是共享索引中的命名空间

_id:1 到 512 个 ASCII 字符,不能包含 NUL。其他 ID 会获得新 ID

一个名为 embedding 的 dense_vector 字段,在索引 schema 中声明

文档上的一个文本字段

顶层文档字段,可自动过滤。值为字符串、数字、布尔值或字符串列表

距离度量 l2(默认)/ cosine / ip

度量方式 euclidean / cosine / dotproduct

使用 $eq、$ne、$gt、$gte、$lt、$lte、$in、$nin、$exists、$and、$or、$not 进行过滤

在数组键上使用 $contains / $not_contains

{"$in": [value]} / {"$nin": [value]}

在非数组键上使用 $contains / $not_contains,where_document

无对应实现。需重新设计这些查询

在迁移到 Pinecone 的过程中,一些数据会发生变化,例如重新映射的 ID、重命名的元数据键,以及需要重新设计的过滤器。仓库 README 列出了每一项。

以下是将你的 Chroma Cloud 集合迁移到 Pinecone Database 的 10 个简单步骤。

预演并确定时间窗口

preflight.py,然后进行完整的试运行

创建索引并上线读写切换

暂停写入。读取仍留在 Chroma

你的应用,然后是 preflight.py

对计数进行快照并导出

转换,然后载入到一个全新的命名空间

将读取与写入一并切换

在工具包根目录下用 uv 完成环境搭建,它会安装已锁定版本的依赖:

uv sync export PINECONE_API_KEY=... # 所有 Pinecone 步骤 export CHROMA_API_KEY=... CHROMA_TENANT=... CHROMA_DATABASE=... # Chroma Cloud

这些命令使用名为 my_collection 的 collection、名为 my-index 的索引,以及名为 my-collection-v1 的命名空间。请把它们替换成你自己的名称。读取 Chroma 的脚本会调用 chromadb.CloudClient(),该函数会读取三个 CHROMA_* 变量。若要针对本地开源版 Chroma 副本进行测试,请在这些脚本中加上 --chroma-path PATH。

Preflight 会逐页扫描该 collection。它会报告 collection.count()、embedding 维度、距离空间、所记录的嵌入函数、每个 metadata 键及其值类型与对应的 Pinecone 字段名、None 与空文档,以及 Pinecone 会拒绝的 ID。发现项会被标记为 BLOCKING、TRANSFORM、WARNING 或 INFO,并保存到 preflight.json。

uv run preflight.py --collection my_collection --workdir migration_work uv run preflight.py --collection my_collection --workdir migration_work --allow-transforms # 在查看 TRANSFORM 类发现项之后

请查看首次运行得出的发现项。阻断性发现项会以退出码 1 结束:全零向量、维度混用、metadata 超过 40 KB、不支持的类型,以及数值或布尔数组。请在暂停前在 Chroma 中修复这些问题。转换项(新 ID、重命名的键、被丢弃的 None 值)在你用 --allow-transforms 接受之前,同样会以退出码 1 结束。

如果 preflight 因文档文本过长而报告 metadata_over_40kb,请改用第 2 步中的全文搜索变体,或在源端拆分文本。仅当 preflight 报告 non_string_arrays 且你接受字符串列表时,才添加 --array-policy stringify。要检查你的应用所使用的过滤器,请添加 --app-filters FILE,即一个由 {"name", "where"} 或 {"name", "where_document"} 对象组成的 JSON 列表。在第 5 步中,把相同的策略标志传给 convert.py。

然后在一份副本或一个小型集合上演练第 2 步到第 6 步,并为导出和加载计时。用这个时间来设定写暂停窗口的大小。我们不给出具体时长,因为只有你的演练才是唯一可靠的估算。

第 2 步:创建索引并完成切换

uv run create_index.py --index my-index --workdir migration_work

该脚本从 preflight.json 读取维度和度量方式,创建一个带有一个 dense_vector 字段的文档索引,默认位于 AWS us-east-1。在 Standard 上,可用 --cloud 和 --region 选择其他位置。--deletion-protection enabled 可保护索引不被删除。如果索引已存在,脚本只校验其 schema,若不一致则以 1 退出。

全文搜索变体。如果你的文档超过约 40 KB,请运行以下命令,代替对应的 preflight、create_index 和 convert 命令。只运行一条路径,不要两者都运行:

uv run preflight.py --collection my_collection --workdir migration_work --text-mode fts --allow-transforms uv run create_index.py --index my-index --workdir migration_work --full-text-search uv run convert.py --workdir migration_work --text-mode fts

在同一步中,发布执行切换的应用改动:一个配置标志,在 Chroma 和 Pinecone 之间切换读与写。这段代码由你编写,暂时仍指向 Chroma。在 Pinecone 一侧,用 pc.index(name=...) 打开索引,用 index.documents.upsert 写入,用 index.documents.search 搜索。使用 preflight.json 中的目标字段名,并处理重映射的 ID(第 5 步)。

把应用设为只读。读操作仍留在 Chroma。然后用相同的标志重新运行 preflight:

uv run preflight.py --collection my_collection --workdir migration_work --allow-transforms # 加上你在第 1 步用过的任何策略标志(例如 --array-policy stringify)。# 全文搜索路径:加上 --text-mode fts。

uv run export.py --collection my_collection --workdir migration_work

该脚本会记录 collection.count(),然后把 collection.get 的分页结果流式写入 export.jsonl,内存中只保留 ID 集合。只有在导出的唯一 ID 数与计数相等、没有 ID 重复,且导出期间计数未发生变化时,它才以 0 退出。退出码 1 通常意味着写入没有暂停。由于这项检查比较的是计数,它无法发现不改变计数的更新。只有暂停写入才能防备这类情况,因此要在应用中强制执行暂停。请保留 export.jsonl 作为你的存档副本。

第 5 步:转换与加载

uv run convert.py --workdir migration_work

加上你给 preflight 的相同策略标志。convert 会两次流式读取导出数据,并写入:

documents.jsonl:每行一个 Pinecone 文档。

field_map.json:每个元数据键及其对应的 Pinecone 字段。以 _ 或 $ 开头、超过 64 字节,或等于保留字段(embedding、text、chroma_id)的键会变成 meta_。如果该名称已被占用或过长,就会被截断并加上哈希后缀,因此超过 64 字节的键总是采用哈希形式。在我们的测试中,text、_text 和已有的 meta_text 分别变成了 meta_text_982d9e3e、meta_text_f47a3f59 和 meta_text。该表是确定性的,记录每个原因,并且可以逆转。

id_map.jsonl:Pinecone 拒绝的 ID 会变成 chroma-sha256-,如果该字符串已被占用,则加上 -2 后缀。每个重新映射的文档还会将其原始 ID 存储在 chroma_id 中。

transforms.jsonl:每条记录的每处更改或丢弃都占一行。不会静默丢弃任何内容。

如果任何记录失败,convert 不会写入 documents.jsonl。然后加载:

uv run load.py --index my-index --namespace my-collection-v1 --workdir migration_work

原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。

查看英文原文(How to Migrate from Chroma Cloud to Pinecone)

Chroma is winding down Chroma Cloud, its hosted service. Chroma’s notice sets two dates:

November 22, 2026: writes stop. Chroma Cloud becomes read-only. No creates, updates, or deletes.

December 22, 2026: reads stop. Chroma Cloud no longer serves queries.

Chroma recommends that you start exporting and validating data before November 22 and finish exporting before December 22. Open-source Chroma is not affected and remains available under Apache 2.0.

If you want a managed service instead of running Chroma yourself, Pinecone Database is serverless and fully managed. There are no clusters to operate and no index to size or provision before you load. Read about the architecture here. Chroma Cloud users can start today with a 3-week Pinecone Standard trial that includes $300 in credits.

The walkthrough moves one Chroma Cloud collection that already has dense embeddings, document text, and metadata into a Pinecone serverless index. It reuses your existing vectors and re-embeds nothing. It assumes a planned write pause: your app is read-only for the cutover window. To move several collections, repeat it for each one.

The steps run a toolkit of six Python scripts: preflight, create index, export, convert, load, and validate. Get it at pinecone-field/chromacloud-migration.

This toolkit is provided as-is, with no guarantees. It may not fit your collection, embedding setup, or cutover plan. Before you pause writes or flip production traffic, run the scripts yourself against your own data, review every finding, and confirm that validation matches what your app needs. You are responsible for verifying the migration works for your use case.

Run It With a Coding Agent

You can also hand the migration to a coding agent such as Claude Code, Cursor, or Codex. The toolkit includes an Agent Skillsskill that walks the agent through the same steps as this guide. Install it from the toolkit repo:

npx skills add pinecone-field/chromacloud-migration

Once it's installed, ask the agent to migrate your Chroma Cloud collection to Pinecone. It runs the six scripts in order.

Map Chroma Concepts to Pinecone

Below are some of the key concepts and terminology used in Chroma Cloud and their equivalents in Pinecone.

Index, or a namespace in a shared index when collections share dimension and metric

_id: 1 to 512 ASCII characters, no NUL. Other IDs get new ones

A dense_vector field named embedding, declared in the index schema

A text field on the document

Top-level document fields, filterable automatically. Values are strings, numbers, booleans, or lists of strings

Distance l2 (default) / cosine / ip

Metric euclidean / cosine / dotproduct

filter with $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $exists, $and, $or, $not

$contains / $not_contains on an array key

{"$in": [value]} / {"$nin": [value]}

$contains / $not_contains on a non-array key, where_document

No equivalent. Redesign these queries

Some data changes on the way to Pinecone, such as remapped IDs, renamed metadata keys, and filters that need redesign. The repo README lists each one.

Here are 10 simple steps to migrate your Chroma Cloud collection over to Pinecone Database.

Rehearse and size the window

preflight.py, then a full dry run

Create the index and ship the read/write switch

Pause writes. Reads stay on Chroma

Your app, then preflight.py

Snapshot the count and export

Convert, then load into a fresh namespace

Flip reads and writes together

Set up from the toolkit root with uv, which installs the pinned dependencies:

uv sync export PINECONE_API_KEY=... # all Pinecone steps export CHROMA_API_KEY=... CHROMA_TENANT=... CHROMA_DATABASE=... # Chroma Cloud

The commands use a collection named my_collection, an index named my-index, and a namespace named my-collection-v1. Replace them with your own. Scripts that read Chroma call chromadb.CloudClient(), which reads the three CHROMA_* variables. To test against a local open-source Chroma copy, add --chroma-path PATH to those scripts.

Preflight scans the collection page by page. It reports collection.count(), embedding dimension, distance space, the recorded embedding function, every metadata key with its value types and target Pinecone field name, None and empty documents, and IDs Pinecone would reject. Findings are labeled BLOCKING, TRANSFORM, WARNING, or INFO and saved to preflight.json.

uv run preflight.py --collection my_collection --workdir migration_work uv run preflight.py --collection my_collection --workdir migration_work --allow-transforms # after reviewing TRANSFORM findings

Review the findings from the first run. Blocking findings exit 1: all-zero vectors, mixed dimensions, metadata over 40 KB, unsupported types, and numeric or boolean arrays. Fix them in Chroma before the pause. Transforms (new IDs, renamed keys, dropped None values) also exit 1 until you accept them with --allow-transforms.

If preflight reports metadata_over_40kb because your document text is long, switch to the full-text-search variant in Step 2, or split the text in the source. Add --array-policy stringify only if preflight reports non_string_arrays and you accept string lists. To check the filters your app uses, add --app-filters FILE, a JSON list of {"name", "where"} or {"name", "where_document"} objects. Pass the same policy flags to convert.py in Step 5.

Then rehearse Steps 2 through 6 on a copy or a small collection, and time the export and load. Use that time to size your write-pause window. We don’t quote durations, because your rehearsal is the only reliable estimate.

Step 2: Create the Index and Ship the Switch

uv run create_index.py --index my-index --workdir migration_work

The script reads dimension and metric from preflight.json and creates a document index with one dense_vector field, in AWS us-east-1 by default. On Standard, --cloud and --region choose another location. --deletion-protection enabled guards the index against deletion. If the index exists, the script only verifies its schema and exits 1 on a mismatch.

Full-text-search variant. If your documents exceed about 40 KB, run these commands instead of the matching preflight, create_index, and convert commands. Run one path, not both:

uv run preflight.py --collection my_collection --workdir migration_work --text-mode fts --allow-transforms uv run create_index.py --index my-index --workdir migration_work --full-text-search uv run convert.py --workdir migration_work --text-mode fts

In the same step, ship the app change that performs the cutover: one config flag that switches reads and writes between Chroma and Pinecone. You write this code, and it stays pointed at Chroma for now. On the Pinecone side, open the index with pc.index(name=...), write with index.documents.upsert, and search with index.documents.search. Use the target field names from preflight.json, and handle remapped IDs (Step 5).

Set your app to read-only. Reads stay on Chroma. Then rerun preflight with the same flags:

uv run preflight.py --collection my_collection --workdir migration_work --allow-transforms # Add any policy flags you used in Step 1 (for example --array-policy stringify). # Full-text-search path: add --text-mode fts.

uv run export.py --collection my_collection --workdir migration_work

The script records collection.count(), then streams collection.get pages to export.jsonl, holding only the set of IDs in memory. It exits 0 only if unique exported IDs equal the count, no ID repeats, and the count did not change during the export. Exit 1 usually means writes are not paused. Because this check compares counts, it can’t detect an update that leaves the count unchanged. Only the write pause protects against those, so enforce it in the app. Keep export.jsonl as your archive copy.

Step 5: Convert and Load

uv run convert.py --workdir migration_work

Add the same policy flags you gave preflight. Convert streams the export twice and writes:

documents.jsonl: one Pinecone document per line.

field_map.json: every metadata key and its Pinecone field. Keys that start with _ or $, exceed 64 bytes, or equal a reserved field (embedding, text, chroma_id) become meta_. If that name is taken or too long, it is truncated and gets a hash suffix, so keys over 64 bytes always take the hashed form. In our testing, text, _text, and an existing meta_text became meta_text_982d9e3e, meta_text_f47a3f59, and meta_text. The table is deterministic, records each reason, and can be reversed.

id_map.jsonl: IDs Pinecone rejects become chroma-sha256-, with a -2 suffix if that string is taken. Each remapped document also stores its original ID in chroma_id.

transforms.jsonl: one line for every per-record change or drop. Nothing is dropped silently.

If any record fails, convert writes no documents.jsonl. Then load:

uv run load.py --index my-index --namespace my-collection-v1 --workdir migration_work

The loader streams documents.jsonl with index.documents.upsert, one batch in memory at a time. It sends a batch before it would exceed 1,000 documents or 1,900,000 bytes, under the 2 MB request limit, and retries rate-limit, 5xx, connection, and timeout errors with backoff. The namespace must be new or empty. A rerun into the same namespace is allowed only when load_state.json shows the same index, namespace, and documents.jsonl checksum. Upserts overwrite by _id, so a rerun leaves the same content.

Golden queries compare top-k results between Chroma and Pinecone. Build them with the same embedding call your app uses for Chroma queries. Your app’s embedding call is the integration point: replace my_app_embed with your own function. The toolkit does not embed text.

import json queries = ["how do refunds work", "reset my password"] # real user queries with open("golden.jsonl", "w") as f: for i, q in enumerate(queries): emb = my_app_embed(q) # INTEGRATION POINT: your app's query-embedding function f.write(json.dumps({"query_id": f"q{i}", "embedding": list(map(float, emb))}) + "\n")

Then run the validator. Add --app-filters app_filters.json to compare your app’s where filters.

uv run validate.py --collection my_collection --index my-index --namespace my-collection-v1 \ --workdir migration_work --golden-queries golden.jsonl

collection.count() still equals the exported count

Chroma's live space and dimension match the index schema

The namespace record count reaches the exported count (polled up to 600 s)

Source IDs, mapped through id_map.jsonl, equal every ID from index.documents.list(namespace=...)

200 random documents, plus up to 200 records with per-record transforms, match the source field by field and round-trip to the Chroma record. A key rename alone is not a per-record transform, so random sampling covers those records

Probe searches return top_k results (polled up to 180 s)

Tie-adjusted top-k overlap with Chroma's collection.query averages at least 0.9, with no query below 0.6

With --app-filters, each translated filter returns the same ID set as Chroma's where

The overlap thresholds are heuristics. Tune them with --min-mean-overlap and --min-query-overlap. --golden-from-sample N uses perturbed stored vectors and is for tests only.

The validator also checks self-retrieval: each sampled vector should find its own record. This check is a diagnostic and never fails the run. Misses point to duplicate vectors, dotproduct with unnormalized vectors, near ties, approximate search, or freshness.

If a data check fails: stop, fix, and reload into a fresh namespace (for example, my-collection-v2), then validate again. Don’t patch gaps with an upsert-only pass for missing IDs, because upserts can’t remove records that were deleted in Chroma. If only app_filters fails, your data is fine. Fix the app’s filter instead of reloading.

Decide whether to flip. Rolling back is still free: your app has written nothing to Pinecone. If you aren’t confident, resume writes on Chroma and schedule another window.

Step 8: Flip Reads and Writes Together

Flip the config flag, then resume writes. Moving reads and writes at once means no write lands in a store your reads can’t see.

Watch errors, latency, and result quality. From here, rolling back means replaying Pinecone writes into Chroma, and Chroma Cloud accepts no writes after November 22.

If you reach November 22 first: Chroma enforces the write pause for you. Steps 4 through 8 still work until December 22, but your app’s writes fail until you flip.

Step 10: Retire Chroma Cloud

Once validation passes and monitoring shows no issues, finish before December 22:

Remove Chroma client code from your app and jobs, and confirm nothing calls Chroma Cloud endpoints.

Rotate or delete your Chroma Cloud credentials.

Archive export.jsonl with field_map.json, id_map.jsonl, and transforms.jsonl. You need all four files to reverse renames and ID mappings.

This guide keeps your current embedding model. Once you’re on Pinecone, you can move to integrated embedding, where a Pinecone-hosted model such as llama-text-embed-v2 or multilingual-e5-large embeds text at upsert and query time. Integrated embedding removes the client-side embedding step. The switch needs a new index whose schema declares a text field with an embed block, and a reload from your stored text. Results will change, so evaluate them for relevance rather than identical IDs.

The $300 Pinecone Standard Trial

The Standard trial gives you 3 weeks and $300 in credits, with no up-front payment. It ends after 21 days or when you use the $300, whichever comes first. A trial started in October ends before November 22, so choose your plan before you cut over.

The trial includes Standard features:

Import from object storage for bulk loads (the toolkit doesn’t use it)

Role-based access control (RBAC)

Choice of cloud and region

Higher limits and Developer Support

When the trial ends, add a payment method to stay on Standard ($50/month minimum usage, then pay as you go), upgrade to Enterprise, or downgrade to the free Starter plan.

A Landing Spot for Chroma Cloud Users

Chroma Cloud is coming to an end on December 22. We want to provide a place for all Chroma Cloud users to land if they want a fully managed service experience. Start your Standard trial today and get $300 in credits, migrate to Pinecone Database, and try out integrated embedding or full-text search on a new index when you’re ready.

Start your Standard trial

Get 3 weeks and $300 in credits on Pinecone Standard — then migrate with the Chroma Cloud toolkit.

Was this article helpful?

Start building knowledgeable AI today

Create your first index for free, then pay as you go when you're ready to scale.

原文为英文,正文由自动抓取并保留原文。

本文由采集器自动同步

原文发布于 Pinecone Blog(How to Migrate from Chroma Cloud to Pinecone),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。

阅读原文 ↗