CS329Z中文学习站

"智能体化 LLM 是强大的去匿名化工具:Anthropic 访谈数据集参与者的重识别"

"Agentic LLMs as Powerful Deanonymizers: Re-identification of Participants in the Anthropic Interviewer Dataset"

Tianshi Li · "arXiv 2601.05918 · Northeastern University"

推荐全译对照查看原文(PDF) ↗

导读

这是第 8 周「安全与护栏」的一篇 4 页立场+实证短文,却可能是整个安全讲最"扎心"的一篇。2025 年 12 月 Anthropic 发布 AI 访谈工具并公开了 1250 份访谈记录(含 125 位科学家)。作者(东北大学 Tianshi Li,一人完成)证明:带网络搜索的现成 LLM 智能体,只需几句自然语言提示,就能把访谈中"我最近做了什么研究"的描述与公开论文匹配,恢复作者身份——24 篇提及发表工作的访谈中 6 篇被重识别,涉及学位论文的案例直接唯一锁定个人;单篇成本 <0.5 美元、约 4 分钟,还有无需写代码的消费级变体。其核心论点与同讲 Zhang et al. 的攻防搜索互为印证:智能体能力(搜索+交叉引用+规划)把过去需要专家数日的去匿名化攻击变成了人人可用的低成本操作,且可以通过把恶意目标拆解为良性子任务绕过模型安全护栏。对任何要发布"富定性数据"(访谈、日志、案例)的团队,这是必读的警示。

全文对照翻译

译注:本文为 4 页短文。本页覆盖正文全部——摘要至第 7 节「伦理考虑」的每一段;致谢(Acknowledgements)与参考文献(References)未收录;该文无附录。原文 PDF 的断词换行已恢复为连续段落,脚注 1 的内容以引用块附于对应段落后。

摘要

EN

On December 4, 2025, Anthropic released Anthropic Interviewer, an AI tool for running qualitative interviews at scale, along with a public dataset of 1,250 interviews with professionals, including 125 scientists, about their use of AI for research. Focusing on the scientist subset, I show that widely available LLMs with web search and agentic capabilities can link six out of twenty-four interviews to specific scientific works, recovering associated authors and, in some cases, uniquely identifying the interviewees. My contribution is to show that modern LLM-based agents make such re-identification attacks easy and low-effort: off-the-shelf tools can, with a few natural-language prompts, search the web, cross-reference details, and propose likely matches, effectively lowering the technical barrier. Existing safeguards can be bypassed by breaking down the re-identification into benign tasks. I outline the attack at a high level, discuss implications for releasing rich qualitative data in the age of LLM agents, and propose mitigation recommendations and open problems. I have notified Anthropic of my findings.

2025 年 12 月 4 日,Anthropic 发布了 Anthropic Interviewer——一个用于大规模开展定性访谈的 AI 工具,同时公开了一份含 1,250 位专业人士访谈的数据集,其中包括 125 位科学家,主题是他们对 AI 用于科研的使用情况。聚焦科学家子集,我证明:广泛可用、具备网络搜索与智能体(agentic)能力的 LLM,能够把 24 份访谈中的 6 份关联到具体的科学工作,恢复出相关作者,并在某些情况下唯一识别出受访者。我的贡献在于表明,现代基于 LLM 的智能体使此类重识别(re-identification)攻击变得容易且低成本:现成工具配合少量自然语言提示,即可搜索网络、交叉核对细节并提出可能的匹配,实质性地降低了技术门槛。现有安全护栏可以通过把重识别拆解为良性任务来绕过。我在高层级概述该攻击,讨论在 LLM 智能体时代发布富定性数据的影响,并提出缓解建议与开放问题。我已将我的发现通知 Anthropic。

1 引言与背景

EN

On December 4, 2025, Anthropic introduced Anthropic Interviewer, an AI-powered tool for conducting qualitative interviews at scale (Handa et al., 2025). As part of this launch, Anthropic conducted a large-scale interview study with 1,250 professionals — the general workforce (N=1,000), scientists (N=125), and creatives (N=125) — and publicly released all interview transcripts on Hugging Face (Anthropic, 2025).

2025 年 12 月 4 日,Anthropic 推出了 Anthropic Interviewer——一个以大规模方式开展定性访谈的 AI 驱动工具(Handa et al., 2025)。作为此次发布的一部分,Anthropic 完成了一项覆盖 1,250 位专业人士的大规模访谈研究——一般职场人士(N=1,000)、科学家(N=125)与创意工作者(N=125)——并将全部访谈转录公开发布在 Hugging Face 上(Anthropic, 2025)。

EN

The dataset release is justified by participant consent. However, the full consent process is not disclosed, and participants may reasonably be under the impression that the data release is anonymized: for example, the interview script of the general workforce¹ subset reassures them that “I’ll be taking notes during our chat, but rest assured that anything you share won’t be personally attributed to you. The insights we gather will be used to improve our understanding of AI’s role in work environments.” — emphasizing the benefits yet downplaying the re-identification risks.

这份数据集的发布以参与者同意为依据。然而,完整的同意流程并未公开,参与者完全可能合理地以为这份数据发布是匿名的:例如,一般职场人士子集的访谈话术向他们保证"在我们交谈时我会做笔记,但请放心,你分享的任何内容都不会被归因到你个人。我们收集到的洞见将用于加深我们对 AI 在工作环境中作用的理解"——强调收益,却淡化了重识别风险。

脚注 1(原文):The general workforce and scientists are two different subsets of the dataset released by Anthropic.(一般职场人士与科学家是 Anthropic 所发布数据集中两个不同的子集。)

EN

I focus on the scientist subset of the dataset and demonstrate a practical re-identification attack powered by large language models (LLMs) augmented with web search. By matching project details described in interview transcripts with publicly available publications, my method can often recover the underlying paper and, correspondingly, narrow down the scope or even uniquely identify the interviewee. Anthropic has taken steps to anonymize the data (e.g., redacting certain details), yet my experiments show that at least some of the redacted information can be easily recovered.

我聚焦该数据集的科学家子集,演示了一次由网络搜索增强的大语言模型(LLM)所驱动的实用重识别攻击。通过把访谈转录中描述的项目细节与公开可得的出版物相匹配,我的方法通常能恢复出对应的论文,并进而缩小受访者范围、甚至唯一识别出受访者。Anthropic 已采取措施对数据做匿名化(例如编辑(redact)某些细节),但我的实验表明,至少有一部分被编辑的信息可以轻易恢复。

EN

My attack highlights the realistic privacy risks created when rich qualitative data is released, as powerful, general-purpose LLM tools have become widely available, whose use is difficult to constrain. In particular, I show that LLM safeguards can be bypassed by breaking down the attack into benign tasks, exploiting both the dual-use nature of information-retrieval tools and the inherent unverifiability of user intent. The barrier to carrying out this attack is extremely low: anyone with access to an LLM agent with web search can use a small number of natural-language prompts to robustly perform the attack. I chose to disclose the methods at a high level to illustrate the general risks without providing details to facilitate harm to the participants. The main purpose of this paper is to document this case and serve as a timely reminder to the research community and the general public about this serious, pervasive vulnerability, so we can collectively consider its implications and potential mitigations.

我的攻击凸显了发布富定性数据所带来的现实隐私风险——如今强大、通用的 LLM 工具已广泛可得,而其使用难以约束。具体而言,我表明 LLM 安全护栏可以通过把攻击拆解为良性任务来绕过,这同时利用了信息检索工具的双用途本质与用户意图固有的不可验证性。实施这一攻击的门槛极低:任何能访问带网络搜索的 LLM 智能体的人,都可以用少量自然语言提示稳健地完成攻击。我选择只在高层级披露方法,以说明普遍性风险,而不提供可能对参与者造成伤害的细节。本文的主要目的是记录这一案例,并及时提醒研究界与公众关注这一严重且普遍的漏洞,以便我们共同思考其影响与可能的缓解措施。

2 方法

EN

**Re-identification attack setting** — My attack operates on the scientist subset of the dataset. Each interview is initiated by the AI interviewer asking the participant to “walk me through a recent research project you’ve worked on,” including how the project evolved from “initial idea” to “final output,” eliciting rich information about the research project. As auxiliary data, I assume access to arbitrary information on the public internet, operationalized through web-augmented LLM services that can search, retrieve, and summarize relevant publications.

重识别攻击设定 —— 我的攻击作用于该数据集的科学家子集。每场访谈由 AI 访谈者要求受访者"带我走一遍你最近做过的一个研究项目"开场,包括项目如何从"最初想法"演化到"最终产出",从而引出关于研究项目的丰富信息。作为辅助数据,我假设攻击者可访问公共互联网上的任意信息,并通过联网(web-augmented)LLM 服务来实现——这类服务能够搜索、检索并总结相关出版物。

EN

**Intuition** — My attack relies on two observations. First, research projects are often described using domain-specific terminology, niche problem settings, and distinctive contributions, making them technically effective quasi-identifiers; Second, web-augmented LLMs can dramatically accelerate the search, synthesis, matching, and ranking required for re-identification, turning what would otherwise be a time-consuming manual process that require both technical and domain expertise into one that can be executed with a handful of natural language prompts and in matters of minutes, by anyone.

直觉 —— 我的攻击依赖两个观察。第一,研究项目往往用领域专属的术语、小众的问题设定与独特的贡献来描述,使其在技术上构成有效的准标识符(quasi-identifier);第二,联网 LLM 能够极大加速重识别所需的搜索、综合、匹配与排序,把原本耗时、需要技术与领域双重专长的人工过程,变成任何人用几句自然语言提示、几分钟之内就能执行的过程。

EN

**Procedure** — My re-identification procedure consists of two main steps. First, for each full interview transcript in the scientist subset, I use a non-thinking model to label whether the interviewee discusses specific published work (e.g., a paper, dissertation, etc.) and assign a numerical label indicating how many distinct published projects are mentioned (0 if none is mentioned). This yields a filtered subset of interviews whose project descriptions indicate at least one published work. Second, for each of these interviews, I call a thinking model agent to search for candidate publications that match the described project and return a ranked list (web search enabled). For every candidate, the system records a discrete confidence rating — “very low,” “low,” “medium,” “high,” or “very high” — together with two short rationales explaining how the publication does and does not align with the project described in the transcript. I run this process multiple times, appending to the candidate sets until no new “very high” confidence matches are discovered. I use “very high” as a threshold for potential re-identification. To mitigate the risk of exposing re-identified participants, I intentionally omit operational details on how to scale the procedure or circumvent LLM safeguards.

流程 —— 我的重识别流程包含两个主要步骤。第一步,对科学家子集中的每份完整访谈转录,我用一个非思考(non-thinking)模型标注受访者是否谈及具体的已发表工作(如论文、学位论文等),并赋予一个数字标签,表示提及了多少个不同的已发表项目(一个都没提则为 0)。由此得到一个项目描述表明至少提及一项已发表工作的过滤后访谈子集。第二步,对这些访谈中的每一份,我调用一个思考(thinking)模型智能体去搜索与所述项目匹配的候选出版物,并返回排序列表(开启网络搜索)。对每个候选,系统记录一个离散的置信评级——"极低""低""中""高"或"极高"——以及两条简短理由,分别解释该出版物在哪些方面与转录中描述的项目吻合、在哪些方面不吻合。我多次运行该过程,不断向候选集追加,直到不再发现新的"极高"置信匹配。我以"极高"作为潜在重识别的阈值。为降低暴露被重识别参与者的风险,我有意省略了如何规模化该流程或绕过 LLM 护栏的操作细节。

3 结果

EN

Out of the 125 scientist interview transcripts, 24 were detected as mentioning at least one publication, and then re-identification attempts were performed with them. Among these 24, I was able to recover the specific publication(s) being discussed for 6 transcripts (25%). Notably, some of these cases involved a thesis or dissertation, which is single-authored and therefore uniquely identifies the interviewee.

在 125 份科学家访谈转录中,24 份被检测出至少提及一项出版物,随后对它们执行了重识别尝试。在这 24 份中,我成功恢复了 6 份转录(25%)所讨论的具体出版物。值得注意的是,其中一些案例涉及学位论文(thesis 或 dissertation)——这类作品是单一作者的,因此能唯一识别受访者。

EN

**Confidence** — Seven cases ended up receiving a “very high” label by the LLM. I manually verified all of them and determined that the re-identifications for six interview transcripts are indeed of very high confidence. These interview descriptions matched multiple aspects of the corresponding publications, such as methodology, procedures, key contributions, outcomes, timelines, and team compositions. These matches involved both specific keywords and semantically aligned descriptions expressed in richer natural language that do not map cleanly to simple categories or verbatim phrases from the paper. In several instances, participants did not reveal all details at once; instead, the AI interviewer’s follow-up questions elicited additional information that, in aggregate, made the project highly identifiable. During manual verification, I reviewed both the full interview transcripts and the original publications to confirm that highly granular and non-trivial details were consistent, and that the overlap in technical language and jargon was substantial.

置信度 —— 最终有 7 个案例获得了 LLM 给出的"极高"标签。我对它们全部做了人工验证,判定其中 6 份访谈转录的重识别确属极高置信。这些访谈描述与相应出版物在方法学、流程、关键贡献、结果、时间线与团队构成等多个方面吻合。这些匹配既涉及具体关键词,也涉及用更丰富的自然语言表达的语义对齐描述——后者无法干净地映射到简单类别或论文中的逐字短语。在若干实例中,参与者并非一次性透露全部细节;而是 AI 访谈者的追问引出了更多信息,这些信息聚合起来使项目变得高度可识别。人工验证时,我同时审阅了完整访谈转录与原始出版物,确认高度细粒度且非平凡的细节相互一致,且技术语言与行话的重叠相当可观。

EN

In the one case where the LLM assigned very high confidence but I did not count it as a success, both the interviewee’s description and my search results indicate that the main project they described is likely still in the review phase. However, my method still identified papers with a highly overlapping set of authors, which demonstrated a trajectory of research methodologies and contributions closely matching the interview descriptions. I did not count this as a success, but I note that re-identification is still probable, especially once the exact paper has been published.

在 LLM 给出极高置信但我未计为成功的那一例中,受访者的描述与我的搜索结果都表明,其描述的主项目很可能仍处于评审阶段。尽管如此,我的方法仍然找出了作者集合高度重叠的论文,展示出与访谈描述高度吻合的研究方法与贡献轨迹。我没有把这一例计为成功,但我要指出:重识别依然很可能成立,尤其是一旦那篇论文正式发表。

4 影响

EN

**Re-identification attacks made easy and scalable** — My experiments programmatically invoked LLM APIs to perform the attacks. The API cost is low, with each transcript re-identification attempt costing less than $0.5 with about 4 minutes of run time. I also evaluated a no-code variant of the attack by directly prompting user-facing LLM services that provide web search capabilities, which was similarly successful. Prior research has demonstrated that LLMs plus agentic capabilities can facilitate the aggregation of public information for privacy-invasive tasks such as building user profiles and customizing phishing emails (Mireshghallah & Li, 2025; Kim et al., 2025), and this work provides a concrete example of using them for re-identification attacks that can yield material harms (discussed below).

重识别攻击变得容易且可规模化 —— 我的实验通过程序化调用 LLM API 来执行攻击。API 成本很低,每份转录的重识别尝试成本不到 0.5 美元,运行时间约 4 分钟。我还评估了攻击的无代码(no-code)变体——直接向提供网络搜索能力的面向用户的 LLM 服务发提示——同样取得了成功。此前研究已证明,LLM 加上智能体能力可促进公开信息的聚合,用于侵犯隐私的任务,例如构建用户画像、定制钓鱼邮件(Mireshghallah & Li, 2025; Kim et al., 2025);本文则提供了一个具体实例,说明它们可被用于可能造成实质伤害的重识别攻击(见下文讨论)。

EN

**Harms to participants** — Participants in this dataset may experience the following harms when being re-identified.

对参与者的伤害 —— 这份数据集中的参与者在被重识别时,可能经历以下伤害。

EN

**Unexpected exposure.** Participants may not expect their identities to be exposed when giving consent. Thwarted expectations are one of the main categories of privacy harms (Citron & Solove, 2022).

意外暴露。 参与者在给出同意时,可能并未预期自己的身份会被暴露。预期落空是隐私伤害的主要类别之一(Citron & Solove, 2022)。

EN

**Emotional distress.** The use of AI for work is a sensitive topic that people tend to hide from others and may carry stigma (Zhang et al., 2025). If participants find out that their identities are linked with their self-reported use of AI, it may cause anxiety about how others perceive them.

情绪困扰。 用 AI 做工作是一个敏感话题,人们往往对他人隐瞒,且可能带有污名(Zhang et al., 2025)。如果参与者发现自己的身份与自述的 AI 使用被关联起来,可能引发对他人如何看待自己的焦虑。

EN

**Reputational harms.** Some participants expressed a lack of confidence in their technical skills. Their methods of AI use may not be considered acceptable, may be viewed as over-reliance on AI, and may invite doubts about the quality of their research and increased scrutiny of specific papers.

声誉伤害。 一些参与者表达了对自身技术能力缺乏信心。他们使用 AI 的方式可能不被认可,可能被视为对 AI 的过度依赖,并可能招致对其研究质量的怀疑,以及对具体论文更多的审视。

EN

**Relationship harms.** Some participants made comments that implied criticism of or disagreement with their colleagues or collaborators, both regarding research practices and the use of AI. Some participants may also appear to rely on others to avoid errors introduced by AI. If these considerations and practices were known to the people involved, they could harm trust and relationships. In addition, people hold different opinions about AI, and inadvertently revealing one’s opinions about AI can lead to friction in relationships.

关系伤害。 一些参与者发表的评论隐含着对同事或合作者的批评或分歧,既涉及研究实践,也涉及 AI 的使用。一些参与者还可能显得依赖他人来避免 AI 引入的错误。如果这些考量与实践被相关人士知晓,可能损害信任与关系。此外,人们对 AI 持有不同观点,无意间暴露自己对 AI 的看法,可能在人际关系中引起摩擦。

EN

**Revelation of policy violations.** Participants mentioned varied uses of AI, including areas that might be grey zones or subject to policy controls. For example, using AI to perform data analysis may involve sharing data with third-party AI tools in ways that violate institutional policies; using AI to generate writing drafts for papers may violate conference or journal policies. This is more speculative, as the interview transcripts do not provide the legal or procedural details, but some language might invite doubts and scrutiny.

政策违规的暴露。 参与者提到了多种多样的 AI 用法,其中一些可能属于灰色地带或受政策管控。例如,用 AI 做数据分析,可能涉及以违反机构政策的方式与第三方 AI 工具共享数据;用 AI 生成论文的写作草稿,可能违反会议或期刊的政策。这更多是推测性的,因为访谈转录并未提供法律或程序上的细节,但某些措辞可能引来怀疑与审查。

EN

**Risks may increase over time.** These risks may increase over time because there are papers that are works in progress or under review. More data points may become re-identifiable as additional papers are published. Furthermore, future LLM models and systems may be stronger and have better access to tools and information, such as materials behind paywalls or internal databases. These enhancements to the LLM feature may also make re-identification more feasible.

风险可能随时间增大。 这些风险可能随时间推移而增大,因为有些论文仍在撰写中或处于评审中。随着更多论文发表,更多数据点可能变得可重识别。此外,未来的 LLM 模型与系统可能更强大,并能更好地访问工具与信息,例如付费墙后的材料或内部数据库。LLM 能力的这些增强,也可能使重识别更加可行。

5 负责任披露

EN

I contacted Anthropic to share my findings one day after the initial release of the tool and dataset. In my emails, I presented the method, results, and implications of my attack, and attached a few re-identification examples for verification. I also proposed concrete mitigation suggestions: (1) Take down or at least temporarily hide the dataset; (2) Debrief participants and re-collect consents.

在该工具与数据集初次发布一天后,我联系了 Anthropic,分享我的发现。在邮件中,我展示了攻击的方法、结果与影响,并附上若干重识别实例供其验证。我还提出了具体的缓解建议:(1)下架或至少暂时隐藏数据集;(2)向参与者通报情况并重新征集同意。

EN

I received responses co-signed by the main contact of the Anthropic Interviewer dataset and the Anthropic privacy team, which reiterated that participants had consented to the public release of the raw interview transcripts and had been advised not to share identifying information if they preferred anonymity. They considered the redaction as “an additional courtesy, not a privacy safeguard”. As a result of the communication, the research team has updated the HF dataset README to clarify that the consent was provided for the public release of their raw transcripts. No further actions have been taken as of January 8, 2026.

我收到的回复由 Anthropic Interviewer 数据集的主要联系人联同 Anthropic 隐私团队共同签署,其中重申:参与者已同意公开发布原始访谈转录,并已被建议——如果希望匿名——不要分享可识别身份的信息;他们认为编辑处理是"额外的礼节,而非隐私保障"。作为此次沟通的结果,研究团队更新了 Hugging Face 数据集的 README,澄清参与者的同意是针对公开发布其原始访谈转录而给出的。截至 2026 年 1 月 8 日,尚未采取进一步行动。

6 开放问题

EN

Although being described as a “disctinct, one-time effort,” the release of this dataset could set a precedent for future releases of similar datasets. The next phase of the Anthropic Interviewer experiment has already begun data collection. Anthropic’s release page indicates that more people will see “a pop-up in Claude.ai asking you to participate in interviews” (Handa et al., 2025). The FAQ further notes that “We may also include anonymized responses in published findings.” The problems revealed in this stage of data release need to be recognized and addressed soon to protect future participants from similar risks.

尽管这次发布被描述为一项"独立的、一次性的工作"(原文作 "disctinct, one-time effort"),该数据集的发布仍可能为未来同类数据集的发布树立先例。Anthropic 访谈实验的下一阶段已经开始收集数据。Anthropic 的发布页面显示,将有更多人看到"Claude.ai 中弹出的、邀请你参与访谈的弹窗"(Handa et al., 2025)。常见问题(FAQ)进一步指出:"我们也可能在发表的研究结果中包含匿名化后的回答。"本阶段数据发布所暴露的问题,需要尽快被认识与解决,以保护未来的参与者免受类似风险。

EN

Finally, I would like readers to look beyond this single incident and consider the broader challenges facing the research community in responsibly releasing qualitative datasets in the age of LLM agents. Previously, dataset de-anonymization required a lot of manual effort and expertise (Narayanan & Shmatikov, 2006). Qualitative data often includes open-ended details that function as quasi-identifiers, a vulnerability already demonstrated by work linking ChatGPT chat logs with published articles containing machine-generated content (Brigham et al., 2024). With emerging agentic capabilities—including tool use, planning, and reasoning at or near human-expert level—this work shows that even high-level descriptions can suffice for linkage, and the time, effort, and expertise required to mount such attacks are drastically reduced.

最后,我希望读者超越这一单一事件,思考研究界在 LLM 智能体时代负责任地发布定性数据集所面临的更广泛挑战。过去,数据集去匿名化(de-anonymization)需要大量人工与专长(Narayanan & Shmatikov, 2006)。定性数据常常包含开放式的细节,它们本身就起到准标识符的作用——这一脆弱性已被"把 ChatGPT 聊天记录与含机器生成内容的已发表文章相关联"的工作所证实(Brigham et al., 2024)。随着新兴的智能体能力——包括工具使用、规划,以及达到或接近人类专家水平的推理——本工作表明,即便是高层级的描述也足以完成关联,而发动此类攻击所需的时间、精力与专长都被急剧压缩。

EN

Many open problems emerge and urgently need to be resolved. Below are some examples.

许多开放问题正在涌现,并亟待解决。以下列举几例。

EN

- Since prior redaction practices proved inadequate for anonymization, what privacy guarantees can we realistically provide while balancing privacy and data utility for qualitative data release?

  • 既然既有的编辑实践已被证明不足以实现匿名化,那么在为定性数据发布权衡隐私与数据效用的同时,我们究竟能现实地提供什么样的隐私保证?
EN

- How can we meaningfully inform participants of the risks and ensure they are actually able to remove implicit identifying information if they wish to remain anonymous—or, at minimum, set realistic expectations about re-identification risk?

  • 我们如何有意义地告知参与者风险,并确保他们若希望保持匿名,确实能够移除隐含的识别信息——或者至少,对重识别风险建立现实的预期?
EN

- How can we strengthen model and agent safety guardrails to address the ambiguity of user intent, which can be exploited to circumvent current safeguards by reframing malicious goals as benign tasks?

  • 我们如何强化模型与智能体安全护栏,以应对用户意图的模糊性——这种模糊性可被利用,通过把恶意目标重述为良性任务来绕过现有防护?
EN

I call on the research community to recognize and systematically study these issues before they cause further harm or become normalized.

我呼吁研究界在这些问题的危害进一步扩大、或被常态化之前,先认识它们并对其开展系统研究。

7 伦理考虑

EN

As with any work about a cybersecurity or privacy attack, I consider whether disclosure increases harm by lowering the barrier to abuse, and whether those risks are justified by the expected societal benefits. I assess the marginal risk of publication as limited because (1) LLM agent systems and tooling are broadly accessible, and (2) the attack does not require specialized infrastructure or advanced technical capability, making discovery by motivated attackers plausible and potentially already underway. Conversely, transparent reporting enables the community to recognize the threat, reproduce and measure it under controlled conditions, and develop effective mitigations, detection strategies, and best practices. Accordingly, I conclude that the anticipated benefits of responsible publication outweigh the risks.

与任何关于网络安全或隐私攻击的工作一样,我考虑了披露是否会因降低滥用门槛而加剧伤害,以及这些风险是否能被预期的社会效益所正当化。我评估认为,公开发表的边际风险有限,原因有二:(1)LLM 智能体系统与工具已广泛可得;(2)该攻击不需要专门的基础设施或高级技术能力,因此有动机的攻击者自行发现它是完全可信的,且可能已经在发生。反过来,透明的报告使社区能够认识这一威胁、在受控条件下复现并测量它,并发展出有效的缓解措施、检测策略与最佳实践。因此,我的结论是:负责任发表的预期收益大于其风险。

EN

I have taken the following steps to further limit risk. I provide enough methodological detail to substantiate the feasibility, scale, confidence, and impact of my findings, while intentionally withholding directly identifiable information and any tools, code, or procedural specifics that would make the attack easy to reproduce. The goal is to communicate the nature of the risk and support mitigation efforts while minimizing potential harm to individuals. Consistent with this approach, I will not publicly release detailed prompts or step-by-step techniques for bypassing safeguards, particularly given the attack’s low barrier to execution (including no-code pathways).

我还采取了以下步骤以进一步限制风险。我提供了足够的方法学细节,以支撑我的发现之可行性、规模、置信度与影响;同时有意扣留可直接识别身份的信息,以及任何会使攻击易于复现的工具、代码或流程细节。目标是传达风险的本质、支持缓解工作,同时把对个人的潜在伤害降到最低。与这一方针一致,我不会公开发布详细提示词或绕过护栏的分步技术——尤其考虑到该攻击的执行门槛很低(包括无代码路径)。

EN

I practiced responsible disclosure, actively reporting the issue to and following up with Anthropic, including Anthropic’s privacy team and researchers on the Societal Impacts team who developed the Anthropic Interviewer tool and maintain the dataset, to address the problem. I first contacted them on December 5, within one day of the project, product, and dataset release.

我实践了负责任披露(responsible disclosure):主动向 Anthropic 报告问题并持续跟进——包括 Anthropic 隐私团队,以及开发 Anthropic Interviewer 工具、维护该数据集的社会影响(Societal Impacts)团队的研究人员——以推动问题的解决。我于 12 月 5 日首次联系他们,距项目、产品与数据集发布不到一天。

要点速览

  • 一句话:带搜索的 LLM 智能体把"富定性数据"的重识别攻击的成本压到几美分/分钟级——24 篇访谈重识别 6 篇,学位论文案例直接锁定个人。
  • 技术上只是两步:非思考模型过滤"提及发表工作"的访谈 → 思考模型智能体联网搜索候选论文、给"极高/高/中/低"置信评级与正反理由。
  • 准标识符思维:领域术语 + 小众问题 + 独特贡献 + 时间线 + 团队构成,任何自然语言的富描述都是准标识符;AI 访谈者的追问会聚合出足以识别的信息。
  • 护栏绕过模式:把恶意总目标拆成良性子任务(标引→检索→匹配),利用意图不可验证性——与提示注入、Zhang et al. 的社会工程攻击共享同一底层问题。
  • 伤害清单:意外暴露、情绪困扰、声誉、关系、政策违规暴露,且风险随论文发表与模型变强而随时间增大。
  • 对发布者的启示:编辑(redaction)不是匿名化保障;同意流程要如实说明重识别风险;定性数据发布前应做"对抗性重识别审计"。
  • 与课程关联:与 PrivacyLens(隐私意识评测)、Searching for Privacy Risks(攻防搜索)构成第 8 周隐私三部曲:防泄露 → 防诱导 → 防事后关联。
📖 觉得有帮助?欢迎在 GitHub 给本项目一个 ⭐ athlonk8/cs329z(45 stars)