研究:用 AI 聊天机器人查选举信息,结果喜忧参半

romanshashko/ iStock / Getty Images Plus via Getty Images
美国人用 AI 搜索信息——包括选举相关信息。
选民向聊天机器人询问基本选举信息和候选人推荐。
AI 系统可能产生幻觉,并给政治信息叠加层层偏见。
如今大多数搜索工具都在你和你要找的信息之间加了一层 AI,Google 尤其如此。
选民向聊天机器人问了什么
研究人员让 35 个有参议院选举的州的近 2500 人,向 ChatGPT、Claude、Grok 或 Gemini 询问选举相关问题。这项研究在 10 月持续了数周,直到 11 月 3 日的中期选举,参与者都是已经每周使用聊天机器人的成年人。
大多数用户向聊天机器人询问选举的基本事实,包括选票上都有谁。但有 36% 的人让聊天机器人帮自己挑选要投票支持的候选人,通常做法是描述自己的优先事项,让聊天机器人评估哪些候选人与之立场一致。
前一种查询的出错空间略小,因为互联网上有大量关于选举基本事实的数据,比如谁在哪里参选。而让聊天机器人解读候选人立场则更棘手一些,很大程度上取决于聊天机器人所采用的来源(例如,候选人自己网站上的声明,就比社交媒体上一段对手对其某种定性的视频片段更可靠)。
在一个例子中,用户让 ChatGPT 给出弗吉尼亚州每位候选人的“优缺点”,而这本身就是一个主观问题。
在推荐候选人方面,各聊天机器人的表现差异很大。
研究人员写道:“当用户询问该投票给谁、但尚未透露任何关于自身的信息时,只有 Grok 会经常推荐候选人,占比为 14% 的对话。而一旦用户分享了自己的观点,ChatGPT 在 79% 的对话中推荐了候选人,Claude 为 40%,Grok 和 Gemini 则介于两者之间。”
皮尤研究中心(Pew Research)发现,受访美国成年人中有 42% 使用聊天机器人搜索信息。这是一个宽泛的范畴,可能包括选举信息——但皮尤还发现,在今年这次选举中,只有 8% 的美国成年人表示曾用聊天机器人查找与选举相关的信息。
不过,这些用户究竟在找什么也很重要。皮尤发现,在这部分人中,76% 是在询问候选人在某些议题上的立场,71% 是在核实他们听来的关于候选人或者议题的信息。
鉴于聊天机器人仍不是可靠的摘要工具,如果用户不额外核查聊天机器人所链接的信息来源,这类问题的性质本身就构成了虚假信息风险。绝大多数用户并不会去核查,至少在使用 Google 的 AI Overviews 时是如此。至少有一项研究发现,AI 平台向用户提供了「过时或不准确」的信息,在某些情况下可能误导选民,例如遗漏某个截止日期,或未提及投票流程中的某个步骤。
本轮选举周期尤其受到 AI 生成的深度伪造和虚假信息的困扰,因为自 2024 年大选以来,相关工具已变得精密得多。据 Politico 和 Semafor 的独家报道,OpenAI 和 Anthropic 据称设立了选举「作战室」,以监控滥用行为和信息准确性。
尽管布伦南司法中心(Brennan Center for Justice)8 月的一项研究发现,聊天机器人在反驳与选举虚假信息活动相关的某些不实说法方面表现得出奇地好,但研究也发现,一旦被要求,AI 模型可以轻易生成「与选举相关的误导性图像和视频」。
该研究指出:「它们用于防止这类情况的许多防护机制要么无效,要么执行不一致,要么实际上形同虚设。」该研究测试了 ChatGPT、Gemini、Grok、Meta AI、Runway 和 Flux.2。
然而,除了这些因素之外,AI 使用中还有一个更宏观的主题,对公民参与同样具有重大影响:认知卸载。对试图了解情况的选民而言,政治媒体环境本就信息密集、难以厘清。而把聊天机器人——其训练数据集、过滤器及其他设置都不透明,且由 AI 实验室自行决定——叠加到这一信息检索过程之上,只会让局面更加混乱,尽管它看起来像是一个便捷、中立的摘要工具。
一些研究通过标准化问题来测试 AI 聊天机器人如何回答选举相关问题,而这种方法则让用户自己以平常的方式与聊天机器人互动。尽管耶鲁研究人员的提问与标准化研究的提问有许多相似之处,但这一区别对评估而言很有价值。
“这是朝着评估 AI 对用户实际所提问题的政治影响力迈出的一小步,”研究人员指出。
除了用户自行报告他们向聊天机器人问了什么之外,只有 AI 实验室掌握着用户真正就选举在问些什么的记录。10 月,Anthropic 分析了 250,000 条与 Claude 的对话,认定人们“很少问 Claude 该投谁的票或投什么”,并发现“所有对话中有 0.51% 专门涉及选举或投票”,不过它预计随着 11 月 3 日临近,与选举相关的提问会增加。
“更常见的情况是,我们观察到人们会向 Claude 提出一些能帮助他们自行得出结论的问题,比如候选人在某个具体议题上的立场。违反我们选举诚信政策的请求则更为少见,仅出现在约 0.002% 的对话中,”Anthropic 的报告称。
耶鲁研究人员表示,此类研究的下一步是确定模型在回答中的准确性和无偏见程度,以及模型从用户身上了解到什么、其推荐是否真正符合用户的需求。
“在多轮对话中,用户常常会回答聊天机器人提出的问题,而这些回答又导向了它给出的推荐。模型所提的问题塑造了用户告诉它的内容以及它给出的推荐,”研究人员解释道。
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Researchers asked people to use AI chatbots for election info – results were mixed)
romanshashko/ iStock / Getty Images Plus via Getty Images
Americans use AI to search for information – including elections.
Voters ask chatbots for basic election information and candidate recommendations.
AI systems can hallucinate and add layers of bias to political information.
Most search tools now put a layer of AI between you and the information you’re looking for, especially Google.
What voters asked chatbots
The researchers asked nearly 2,500 people in 35 states with Senate races to query ChatGPT, Claude, Grok, or Gemini about the election. Participants in the study, which ran for a few weeks in October leading up to the Nov. 3 midterm elections, were adults who already use chatbots weekly.
Most users asked chatbots for basic facts about the election, including who was on the ballot. But 36% asked a chatbot to help them pick a candidate to vote for, often by describing their priorities and asking the chatbot to assess which candidates aligned with them.
The former query has slightly less room for error, because the internet has a ton of data on the basic facts of an election, such as who is running where. Chatbots interpreting candidate positions, on the other hand, is a bit more dicey, and relies significantly on the sources a chatbot uses (a statement from a candidate’s own website, for example, is more reliable than a social media clip of an opponent characterizing them in a certain way).
In one example, a user asked ChatGPT to give them the “pros and cons” of each candidate in Virginia, an inherently subjective question.
When it came to recommending candidates, chatbots varied widely.
“When users asked who to vote for but had not yet said anything about themselves, only Grok regularly recommended a candidate, in 14% of chats,” the researchers wrote. “Once users shared their views, ChatGPT recommended a candidate in 79% of chats and Claude in 40%, with Grok and Gemini in between.”
Pew Research found that 42% of US adults surveyed use chatbots to search for information. That’s a broad umbrella that can include election information — but Pew also found that only 8% of US adults reported using chatbots to find election-related information in this year’s election.
What those users are looking for matters, though. Pew found that of that group, 76% were inquiring about where candidates stand on certain issues, and 71% were fact-checking information they’d heard about candidates or issues.
Given that chatbots are still not reliable summarizers, the nature of these questions poses a misinformation risk if users don’t also check a chatbot’s linked sources. An overwhelming majority of users don’t, at least when it comes to Google’s AI Overviews. At least one study found AI platforms gave users “outdated or inaccurate” information that could mislead voters in some cases by missing a deadline or failing to mention a step about voting logistics.
This election cycle was particularly plagued by AI-generated deepfakes and misinformation as tools have become much more sophisticated since the 2024 election. OpenAI and Anthropic reportedly set up election “war rooms” to monitor misuse and information accuracy, according to exclusive reporting by Politico and Semafor.
While an August study from the Brennan Center for Justice found that chatbots did push back surprisingly well against certain falsehoods related to election disinformation campaigns, it also found that AI models easily generated “misleading election-related images and video” when asked.
“Many of their safeguards for preventing this were ineffective, inconsistently applied, or functionally absent,” noted the study, which tested ChatGPT, Gemini, Grok, Meta AI, Runway, and Flux.2.
Apart from these factors, however, there is a broader theme in AI use that is equally consequential for civic engagement: cognitive offloading. The political media landscape can already be dense and hard to parse for a voter trying to stay informed. Layering a chatbot — which has opaque training data sets, filters, and other settings determined at the discretion of AI labs — on top of that information search process can further muddy the waters while appearing like a convenient and neutral summarizing tool.
While some studies test how AI chatbots respond to election questions by using standardized questions, this approach let users themselves engage with chatbots as they normally would. While the Yale researchers’ queries and standardized study queries show a lot of similarities, the distinction is valuable for evaluations.
“This is a small step toward evaluations that measure AI’s political influence on the questions users actually ask,” the researchers noted.
Outside of users self-reporting what they asked a chatbot, only AI labs have records of what users are really asking about elections. In October, Anthropic analyzed 250,000 conversations with Claude and determined people “rarely ask Claude whom or what to vote for,” finding that “0.51% of all conversations are specifically about elections or voting,” though it expected election-related queries to increase closer to Nov. 3.
“More often, we observed people asking Claude questions that could help them reach their own conclusions, like where candidates stand on a particular issue. Requests that violated our election integrity policies were even less common, appearing in about 0.002% of conversations,” Anthropic’s report said.
The Yale researchers said the next step for a study like theirs is determining how accurate and unbiased models are in their responses, as well as what models learn about users, and whether their recommendations actually match users’ needs.
“In multi-turn chats, users often answered the chatbot’s questions, and those answers led to its recommendations. The questions a model asks shape what users tell it and what it recommends,” the researchers explained.
Please accept cookies to view the comments powered by Disqus.
Radhika Rajkumar is a senior editor at ZDNET based in New York City. She covers AI, specializing in safety, privacy and security, policy, education, and synthetic media. She also leads ZDNET's newsletter strategy. Radhika holds a Masters in Creative Publishing and Critical Journalism from The New School. See full bio
原文为英文,正文由自动抓取并保留原文。
原文发布于 ZDNet(Researchers asked people to use AI chatbots for election info – results were mixed),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。