小型 GPT 为何记住两个事实却无法关联

用一个小型 GPT 实验解释「两跳诅咒」
问一个语言模型「Imagine 的演唱者的配偶是谁?」,只要它能把思考过程说出来——先找出这首歌的演唱者,再查这个人娶了谁——一个好的模型就能答对。但如果要求它一步给出答案、不展示任何推理过程,结果就变得不太可靠。
Balesni、Korbak 和 Evans [1] 想知道这个弱点到底有多深。他们的办法是用虚构的事实对模型做微调,这样答案就不可能来自训练数据中某个记忆模糊的捷径。当两个事实同时出现在同一份训练文档中时,模型能够把它们串起来。而当同样这两个事实分别从不同文档中学到时,准确率就掉到了随机水平。他们把这称为「两跳诅咒」(Two-Hop Curse)。
我之前写过「反转诅咒」(Reversal Curse),即学会了「A 是 B」的模型答不出「B 是 A」[3]。两跳版本看起来是它的近亲,而且它具备一个让这类论文很有意思的特性:你可以在小到能看清每一个数字的规模上把整件事重建出来。于是我就这么做了。
实验设置:两个事实,一个没人问过的问题
我编造了 200 个主体、200 个桥接实体和 200 个答案。每个主体只与一个桥接实体相连,每个桥接实体只与一个答案相连,配对是随机的。可以类比成「人 → 城市 → 国家」,只不过我的数据里没有任何东西有实际含义,所以模型没有世界知识可以依靠。
训练时,模型看到的是作为两个独立样本呈现的这两条链接:(主体,关系 1)给出桥接实体,(桥接实体,关系 2)给出答案。我关心的问题是(主体,关系 1,关系 2)给出答案。这条从未出现在训练集中。它就是测试。
两个事实都在训练数据中。组合出来的问题则从未出现。图片由作者提供。
这个模型是一个4层GPT,宽度为128,有4个注意力头,从头训练,使用AdamW(学习率0.001,权重衰减0.1),每一步都在完整训练集上训练,共2,500步。下文每个设置都用三个随机种子运行。准确率判定严格:模型在整个词表上的最高预测必须是正确的答案Token。
数据构造只需几行。SUBJ、BRIDGE 和 ANSWER 只是 Token id 列表,R1 和 R2 是两个关系 Token。
import numpy as np N = 200 rng = np.random.default_rng(0) p = rng.permutation(N) # 主语 i -> 桥接 p[i] q = rng.permutation(N) # 桥接 j -> 答案 q[j] train = [] # (输入 Token, 目标 Token) for i in range(N): train.append(([SUBJ[i], R1], BRIDGE[p[i]])) # 第一跳 for j in range(N): train.append(([BRIDGE[j], R2], ANSWER[q[j]])) # 第二跳 # 恒等桥接:对选定比例的桥接,加入 "b -> b" frac = 0.0 chosen = rng.permutation(N)[: int(round(frac * N))] for j in chosen: train.append(([BRIDGE[j]], BRIDGE[j])) # 测试:从不出现在训练中 test = [([SUBJ[i], R1, R2], ANSWER[q[p[i]]]) for i in range(N)]
这个“诅咒”是什么样子
训练集中只有两个单跳表时,模型几乎立刻记住了所有内容。训练准确率在我记录的第一个检查点(第100步)就达到100%,并一直保持在那里。在两跳测试中,三次运行每次都是200题对0题,因此总计600题对0题。
作为参照,一个在200个可能答案中随机猜测的模型,在这600题中大约会答对3题。一题都没答对意味着,无论网络对没见过的两跳问题做了什么,它都不是在答案中猜测。我没有记录它实际输出什么,所以不会对此进行推测。
这就是 Two-Hop Curse 最纯粹的表现形式。模型知道它需要的每一个事实,也能完美回答每一个单跳问题。但在没有被示范该如何组合的情况下,它依然无法把其中两个拼在一起。
Lin、Chen 和 Xu 最近的一篇论文 [2] 认为,在这类设定中,训练数据缺少了某些东西。没有任何信息告诉模型:第一跳得出的桥接实体,与进入第二跳的桥接实体是同一个东西。他们主张,缺少这一点,网络就可以用各自独立的内部特征分别拟合两个单跳任务,而两者永远不会连通。
他们的修正方法叫做 identity bridge(恒等桥接):对每一个桥接实体 b,额外加入一条把 b 映射到自身的训练样本。也就是输入一个 Token、输出同一个 Token。没有主语,没有答案,也没有任何关于组合问题的提示。
他们报告称,这已足以在他们的合成设定中解锁组合能力,而没有它组合就会失败 [2]。对如此微小的改动来说,这是个很强的论断,所以我做了测试。我改变了 200 个桥接实体中获得恒等示例的比例:全无、25%、50% 或全部。由于获得示例的桥接实体与未获得的桥接实体都出现在同一个测试集中,我可以分别给两组打分。
先看从未获得恒等示例的桥接实体。在全部 12 次运行和全部 25 个检查点上,模型在这些测试问题上答对的恰好是零个。在 0%、25% 和 50% 的覆盖率下,最后一步分别累计有 600、450 和 300 个这样的测试用例,而在任何时点都没有一个被正确回答。
再看那些确实获得了恒等示例的桥接实体。
训练期间在未见过的组合上的两跳准确率。带有恒等示例的桥接(蓝色)能组合一段时间,随后衰减。没有恒等示例的桥接(橙色)则始终不能。3 个随机种子的平均值;阴影带为 100% 设置下的最小值到最大值范围。图片由作者提供。
这种组合部分上有效。在我的第一个检查点(第 100 步),当 25% 的桥接带有该示例时,那些桥接的准确率平均为 25%;50% 时为 38%;100% 时为 35%。这远高于随机猜测会得到的 0.5%,也远高于诅咒带来的平坦的零。模型从未见过哪怕一个组合示例,却能正确回答其中超过三分之一。
但它并不持久。在所有九个包含恒等示例的运行中,这些桥接的准确率都在第一个或第二个检查点达到最高,随后在剩余训练中持续下滑。到第 2,500 步时,三种设置下分别为 7%、14% 和 15%。在 100% 设置下,三个随机种子最终分别为 12.5%、21.5% 和 11.5%。
带有恒等示例的桥接在第一个检查点与最后一个检查点上的准确率。误差棒表示 3 个随机种子间的 1 个标准差。图片由作者提供。
我能确定什么,以及不能确定什么
在这一设置内,可以确定的是:诅咒是彻底的(600 个中 0 个),而恒等示例正是区分能组合的桥接与从不组合的桥接的关键。每一个没有恒等示例的桥接,在每次运行的每个检查点都失败了。
同样可以确定的是:在我的设置中,这种收益是暂时的。效应是真实的,但会随着持续训练而逐渐消退。
我不知道的是原因何在。我的猜测(尚未验证)是,更长的训练让网络逐渐为两张单跳表形成各自独立的内部特征,而这恰恰是 identity 示例本应防止出现的情况。权重衰减和学习率也可能起作用,而在实验中我每次只改动一个变量,即带示例的 bridge 所占的比例。
在任何人看这些数字之前,我还想先摆出另外两点局限。我每 100 步才评估一次,所以前 100 步我根本没看到。峰值可能高于 38%,而且可能出现得更早。另外,我的设置与论文中的设置差别很大。Lin 及其同事报告称,在他们最简单的设置下,使用大得多的模型(12 层、宽度 768),以更低的学习率训练 20 个 epoch,达到了接近完美的准确率 [2]。而我的模型是 4 层,以更高的学习率跑了 2,500 个全批量步。所以我的结果并不与他们的结果相矛盾。它们说明,这一修复方案对训练时长和训练强度很敏感,而在多说之前,这是我想直接验证的问题。
为什么玩具模型值得这番折腾
在真实模型中,情况更加模糊不清。Balesni 及其同事也在真实世界事实上测试了九个前沿模型,发现如果不使用思维链(chain of thought),它们在他们尝试的超过一半的问题类别上完全失败;而在允许用书面方式推理时,仍能部分答对 [1]。其他研究认为,要想把一个事实学到足以应对双跳问题的程度,模型可能需要把它存储两次 [4],这可以部分解释为什么这件事很难。
我无法从一个包含 200 个实体的玩具模型里判断,大模型内部是否就是这样运作的。但这个玩具模型确实给了我一样从论文里很少能得到的东西:一个小到我可以亲自核验的实验,而且带有一个我能针对每个事实单独切换的开关。开关关闭时,模型每次都失败;开关打开时,它能进行组合,而训练仍然可能把这种能力磨掉。
这让下一个显而易见的问题变得具体起来。这种能力衰退到底取决于权重衰减、学习率,还是仅仅因为训练时间过长?我会先在最初几百步里每 10 步评估一次,然后再对这两个设置做一轮扫描。如果你先试,我很想听听你看到了什么。
[1] Mikita Balesni, Tomek Korbak, Owain Evans, The Two-Hop Curse: LLMs trained on A->B, B->C fail to learn A->C (2024), arXiv:2411.16353
原文较长,译文为节选;译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Why a Tiny GPT Can Know Two Facts and Still Not Connect Them)
Two-Hop Curse Explained With a Tiny GPT Experiment
Ask a language model "Who is the spouse of the performer of Imagine?" and a good one will get it right, as long as it can think out loud first: who performed the song, then who that person married. Make it answer in a single step with no visible reasoning and things get shakier.
Balesni, Korbak and Evans [1] wanted to know how deep that weakness goes. Their trick was to fine-tune models on invented facts, so the answer can't come from some half-remembered shortcut in the training data. When the two facts showed up together in the same training document, the models could chain them. When the same two facts were learned from separate documents, accuracy fell to chance. They named this the Two-Hop Curse.
I've written about the Reversal Curse before, where a model that learns "A is B" can't answer "B is A" [3]. The two-hop version looked like a close cousin, and it has the same property that makes these papers fun: you can rebuild the whole thing at a scale where you can look at every number. So I did.
The setup: two facts, one question nobody asked
I made up 200 subjects, 200 bridge entities, and 200 answers. Each subject is linked to exactly one bridge, and each bridge to exactly one answer, using random pairings. Think of person, then city, then country, except that nothing in my data means anything, so the model has no world knowledge to lean on.
Training shows the model the two links as separate examples: (subject, relation 1) gives the bridge, and (bridge, relation 2) gives the answer. The question I care about is (subject, relation 1, relation 2) giving the answer. That one is never in the training set. It is the test.

The two facts are in the training data. The composed question never is. Image by author.
The model is a 4-layer GPT with width 128 and 4 attention heads, trained from scratch with AdamW (learning rate 0.001, weight decay 0.1) on the full training set at every step, for 2,500 steps. Every setting below was run with three random seeds. Accuracy is strict: the model's top prediction across its entire vocabulary has to be the right answer token.
The data construction fits in a few lines. SUBJ, BRIDGE and ANSWER are just lists of token ids, and R1 and R2 are the two relation tokens.
import numpy as npN = 200rng = np.random.default_rng(0)p = rng.permutation(N) # subject i -> bridge p[i]q = rng.permutation(N) # bridge j -> answer q[j]train = [] # (input tokens, target token)for i in range(N): train.append(([SUBJ[i], R1], BRIDGE[p[i]])) # first hopfor j in range(N): train.append(([BRIDGE[j], R2], ANSWER[q[j]])) # second hop# identity bridge: for a chosen fraction of bridges, add "b -> b"frac = 0.0chosen = rng.permutation(N)[: int(round(frac * N))]for j in chosen: train.append(([BRIDGE[j]], BRIDGE[j]))# test: never appears in trainingtest = [([SUBJ[i], R1, R2], ANSWER[q[p[i]]]) for i in range(N)]
What the curse looks like
With only the two one-hop tables in the training set, the model memorized everything almost immediately. Training accuracy was 100% by the first checkpoint I recorded (step 100) and stayed there. On the two-hop test it got 0 right out of 200 in each of the three runs, so 0 out of 600.
To put that in context, a model guessing among the 200 possible answers would get about 3 of those 600 right. Getting none means that whatever the network does with an unseen two-hop question, it isn't guessing among the answers. I didn't log what it outputs instead, so I won't speculate about that.
This is the Two-Hop Curse in its cleanest form. The model knows every fact it needs, and it can answer every one-hop question perfectly. It still can't put two of them together without being shown how.
A recent paper by Lin, Chen and Xu [2] argues that in setups like this one, the training data is missing something. Nothing tells the model that the bridge entity coming out of the first hop is the same thing as the bridge entity going into the second. Without that, they argue, the network can fit both one-hop tasks using separate internal features, and the two never connect.
Their fix is called the identity bridge: for each bridge entity b, add one extra training example that maps b to itself. That's a single token in, the same token out. No subjects, no answers, no hint about the composed question.
They report that this is enough to unlock composition in their synthetic setting, and that without it composition fails [2]. That's a strong claim for such a small change, so I tested it. I varied how many of the 200 bridges get an identity example: none, 25%, 50%, or all of them. Because the bridges that got an example and the ones that didn't are both in the same test set, I could score the two groups separately.
Start with the bridges that never got an identity example. Across all 12 runs and all 25 checkpoints, the model got exactly zero of those test questions right. At 0%, 25% and 50% coverage that adds up to 600, 450 and 300 such test cases at the final step, and none were answered correctly at any point.
Now the bridges that did get one.
Two-hop accuracy on unseen compositions during training. Bridges with an identity example (blue) compose for a while, then fade. Bridges without one (orange) never do. Mean over 3 seeds; shaded band is the min to max range for the 100% setting. Image by author.
The composition works, partly. At my first checkpoint (step 100), accuracy on those bridges averaged 25% when 25% of bridges had the example, 38% at 50%, and 35% at 100%. That is far above the 0.5% you'd get by chance, and far above the flat zero from the curse. The model has never been shown a single composed example, and it answers more than a third of them correctly.
But it doesn't hold. In all nine runs that had any identity examples, accuracy on those bridges was highest at the first or second checkpoint and then drifted down for the rest of training. By step 2,500 it was 7%, 14% and 15% for the three settings. In the 100% setting the three seeds ended at 12.5%, 21.5% and 11.5%.
Accuracy on bridges that had an identity example, at the first checkpoint versus the last. Error bars show one standard deviation across 3 seeds. Image by author.
What I can say, and what I can't
Solid, within this setup: the curse is complete (0 of 600), and an identity example is what separates a bridge that composes from one that never does. Every bridge without one failed at every checkpoint of every run.
Also solid: in my setup the benefit is temporary. The effect is real but it erodes with continued training.
What I don't know is why. My guess, which I haven't tested, is that longer training lets the network settle into separate internal features for the two one-hop tables, which is the very situation the identity examples were supposed to prevent. Weight decay and the learning rate could also play a role, and I only changed one thing at a time in the experiment, which was the share of bridges with an example.
There are two more limits I'd put in front of anyone reading these numbers. I only evaluated every 100 steps, so I never saw the first 100. The peak may be higher than 38%, and it may come earlier. And my setup differs a lot from the paper's. Lin and colleagues report near-perfect accuracy in their simplest setting using much larger models (12 layers, 768 wide) trained for 20 epochs at a lower learning rate [2]. Mine is a 4-layer model run for 2,500 full-batch steps at a higher one. So my results don't contradict theirs. They suggest that the fix is sensitive to how long and how hard you train, and that is a question I'd want to test directly before saying more.
Why a toy model is worth the trouble
In real models, the story is murkier. Balesni and colleagues also tested nine frontier models on real-world facts and found that, without chain of thought, they failed completely on more than half of the question categories they tried, while still doing partially well when allowed to reason in writing [1]. Other work argues that learning a fact well enough for two-hop questions may require the model to store it twice [4], which would explain part of why this is hard.
I can't tell you from a 200-entity toy whether that's what is going on inside a large model. What the toy does give me is something I rarely get from papers: an experiment small enough to check myself, with a switch I can flip per fact. With the switch off, the model fails every time. With it on, it can compose, and training can still wear that away.
That makes the obvious next question concrete. Does the fade depend on weight decay, on learning rate, or on simply training for too long? I'd start by evaluating every 10 steps for the first few hundred and then sweeping those two settings. If you try it first, I'd like to hear what you see.
[1] Mikita Balesni, Tomek Korbak, Owain Evans, The Two-Hop Curse: LLMs trained on A->B, B->C fail to learn A->C (2024), arXiv:2411.16353
[2] Pengxiao Lin, Zheng-An Chen, Zhi-Qin John Xu, Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge (2025), arXiv:2509.24653
[3] Lukas Berglund et al., The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2024), ICLR 2024
[4] David Johnston, Nora Belrose, Examining Two Hop Reasoning Through Information Content Scaling (2025), arXiv:2502.03490
LLM hallucination detection challenges and a possible solution presented in a prominent research paper
October 23, 20238 min read
Learn about Tensors and how to use them in one of the most famous machine learning libraries, pytorch
November 30, 20230 min read
Clean data, clear insights: The LLM workflow for reshaping spreadsheets
March 14, 20259 min read
Why production LLM systems need live web search to overcome knowledge cutoffs and stale training data
Exploring Parameter Efficient Finetuning (PEFT): Intuitively Understanding Finetuning Using LoRA
August 25, 202315 min read
Efficiently scaling GPT from large to titanic magnitudes within the meta-learning framework
February 16, 20247 min read
April 6, 20235 min read
A simpler way to understand derivations of loss functions for classification and when/how to apply them in PyTorch
June 27, 20248 min read
Developping LLM-integrated GIT vision language models.
September 7, 202323 min read
原文为英文,正文由自动抓取并保留原文。
原文发布于 Towards Data Science(Why a Tiny GPT Can Know Two Facts and Still Not Connect Them),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。