地址

地址：[2505.10571] LLMs Do Not Have Human-Like Working Memory

时间

2025年4月30日

作者

Jen-tse Huang

Kaiser Sun

Wenxuan Wang

Mark Dredze

Human working memory is an active cognitive system that enables not only the temporary storage of information but also its processing and utilization. Without working memory, individuals may produce unreal conversations, exhibit self-contradiction, and struggle with tasks requiring mental reasoning. In this paper, we demonstrate that Large Language Models (LLMs) lack this human-like cognitive ability, posing a significant challenge to achieving artificial general intelligence. We validate this claim through three experiments: (1) Number Guessing Game, (2) Yes or No Game, and (3) Math Magic. Experimental results on several model families indicate that current LLMs fail to exhibit human-like cognitive behaviors in these scenarios. By highlighting this limitation, we aim to encourage further research in developing LLMs with improved working memory capabilities.

人类的工作记忆是一种活跃的认知系统，它不仅能够暂时存储信息，还能对其进行处理和利用。如果没有工作记忆，个体可能会产生不切实际的对话，表现出自相矛盾，并且难以完成需要进行思维推理的任务。在本文中，我们证明了大型语言模型（LLMs）缺乏这种类似人类的认知能力，这对实现通用人工智能构成了重大挑战。我们通过三个实验来验证这一论断：（1）猜数字游戏，（2）是或否游戏，（3）数学魔术。在多个模型家族上的实验结果表明，当前的 LLMs 在这些场景中未能展现出类似人类的认知行为。通过强调这一局限性，我们旨在鼓励进一步研究，以开发具有改进工作记忆能力的 LLMs。

Imagine the following scenario: You select a number between one and ten. When ready, you are asked, “Is the number greater than five?” Upon answering, others can infer that the number has likely entered your conscious awareness, as its clear perception is necessary for making the comparison and providing a response.

Such conscious awareness is commonly referred to as Working Memory (Atkinson & Shiffrin, 1968). In contrast to long-term memory, working memory is the system required to maintain and manipulate information during complex tasks such as reasoning, comprehension, and learning (Baddeley, 2010). Deficits in working memory can impair information processing and hinder effective communication (Gruszka & Nkecka, 2017; Cowan, 2014).

想象以下场景：您选择一个 1 到 10 之间的数字。准备好后，有人问您：“这个数字大于 5 吗？”在回答时，其他人可以推断出这个数字很可能已经进入了您的意识，因为清晰地感知到它对于进行比较和给出回答是必要的。

这种意识通常被称为工作记忆（阿特金森和希夫林，1968 年）。与长期记忆不同，工作记忆是完成推理、理解、学习等复杂任务时维持和处理信息所必需的系统（巴德利，2010 年）。工作记忆的缺陷会损害信息处理并妨碍有效沟通（格鲁斯卡和恩凯卡，2017 年；考恩，2014 年）。

In this paper, we demonstrate that, despite exhibiting increasingly human-like abilities (Huang et al., 2024b, a), Large Language Models (LLMs) lack a fundamental aspect of human cognition: working memory. As a result, LLMs generate unrealistic responses, display self-contradictions, and fail in tasks requiring mental manipulation.

Previous studies have used the N-back task (Kirchner, 1958) to evaluate the working memory of LLMs (Gong et al., 2024; Zhang et al., 2024). However, a fundamental limitation of these tests is that the critical information needed to answer correctly is in the input context accessible to LLMs. This differs markedly from human testing, where participants cannot view prior steps. LLMs, in contrast, can simply attend to earlier input tokens retained in their context window. To more accurately assess working memory, it is essential to design experiments in which the key information is not explicitly present in the context.

The core challenge is to prove that there is actually nothing in LLMs’ mind without knowing what exactly is in their mind. To investigate this limitation, we design three experiments—(1) Number Guessing Game, (2) Yes-No Game, and (3) Math Magic—that test whether LLMs can maintain information internally without explicitly externalizing it in the context. Experimental results reveal that LLMs consistently fail in this capacity, regardless of model family (GPT (Hurst et al., 2024), Qwen (Yang et al., 2024), DeepSeek (Liu et al., 2024; Guo et al., 2025), LLaMA (Grattafiori et al., 2024)) or reasoning approach (Chain-of-Thought (CoT) (Wei et al., 2022) or o1-like reasoning (Jaech et al., 2024)).

在本文中，我们将证明，尽管大型语言模型（LLMs）展现出越来越接近人类的能力（黄等人，2024 年 b，a），但它们缺乏人类认知的一个基本方面：工作记忆。因此，LLMs 会产生不切实际的回答，表现出自相矛盾，并在需要心理操作的任务中失败。先前的研究使用 N-back 任务（Kirchner，1958 年）来评估大语言模型（LLM）的工作记忆（Gong 等人，2024 年；Zhang 等人，2024 年）。然而，这些测试的一个根本局限在于，正确回答所需的关键信息在大语言模型可访问的输入上下文中。这与人类测试明显不同，在人类测试中，参与者无法查看之前的步骤。相比之下，大语言模型只需关注其上下文窗口中保留的早期输入标记即可。为了更准确地评估工作记忆，设计实验时必须确保关键信息不在上下文中明确呈现。

核心挑战在于，在不了解大语言模型具体思维内容的情况下，证明其内部确实没有某些信息。为了探究这一局限，我们设计了三个实验——（1）数字猜谜游戏，（2）是或否游戏，以及（3）数学魔术——来测试大语言模型是否能够在不将信息明确外显于上下文的情况下在内部保持信息。实验结果表明，无论模型属于哪个家族（GPT（Hurst 等人，2024 年）、Qwen（Yang 等人，2024 年）、DeepSeek（Liu 等人，2024 年；郭等人（2025 年）、LLaMA（格拉塔菲奥里等人，2024 年）或推理方法（链式思维（CoT）（魏等人，2022 年）或类似 o1 的推理（杰奇等人，2024 年））。

In this study, we present three carefully designed experiments to investigate whether LLMs possess human-like working memory for processing information and generating responses. Across all experiments, the results reveal a consistent pattern: LLMs do not exhibit behavior indicative of a functional working memory. They fail to internally represent and manipulate transient information across multiple reasoning steps, relying instead on the immediate prompt context. Even advanced prompting strategies, such as CoT prompting, yield only marginal improvements on tasks requiring internal state management. This limitation leads to unrealistic responses, self-contradictions, and an inability to perform mental manipulations. We hope these findings encourage future research on integrating working memory mechanisms into LLMs to enhance their reasoning capabilities and human-likeness.

在本研究中，我们精心设计了三个实验来探究大型语言模型（LLMs）是否具备类似人类的工作记忆来处理信息和生成回应。在所有实验中，结果呈现出一致的模式：LLMs 并未表现出具有功能性的工作记忆的行为特征。它们无法在多个推理步骤中内部表示和操作瞬时信息，而是依赖于即时的提示上下文。即使采用先进的提示策略，如链式思维（CoT）提示，在需要内部状态管理的任务上也仅能带来微小的改进。这种局限性导致了不切实际的回应、自相矛盾以及无法进行心理操作。我们希望这些发现能鼓励未来的研究将工作记忆机制整合到 LLMs 中，以增强其推理能力和类人特性。