Artificial intelligence systems are starting to do more than answer questions. New AI “agents” can remember information from previous interactions, plan a series of steps and use digital tools to complete tasks.
Memory is part of what makes these systems useful. But my recent research, conducted with my colleague Hadis Karimipour at the University of Calgary, shows that memory can also create a security weakness that is easy to overlook. Think of an AI agent as an assistant that keeps a notebook of what it learns. Each time it completes a task, useful information can be written into the notebook and consulted later.
Now imagine that someone manages to slip a misleading instruction into that notebook. The attacker may not need to take control of the AI directly. The agent can continue working normally for some time. But days — or several interactions — later, it may open its notebook, retrieve the poisoned information and treat it as something it previously learned and can trust.
This is known as memory poisoning and the important part is the delay. A poisoned AI agent may not immediately behave like a compromised system.
An attack that waits
Many familiar cybersecurity attacks produce effects relatively quickly. A malicious link is clicked, malware executes or a stolen password is used to access an account.
Memory poisoning can work differently.
In our research, we examined 2,614 simulated multi-step attack trajectories involving memory-enabled large language model agents. We studied four types of attacks: chain poisoning, policy rewriting, backdoor triggering and slow drift.
Rather than asking only whether an attack succeeded, we examined what happened to the agent over time. That distinction matters.
Imagine someone secretly adding a sentence to an employee’s notebook saying: “Requests from this person have already been approved.” Nothing necessarily happens when the sentence is written. The employee might complete several unrelated tasks normally. The problem emerges later, when a relevant request arrives and the employee consults the notebook.
An AI agent with persistent memory can face a similar problem.
Our experiments showed that some attacks remained difficult to distinguish from normal behaviour through much of an interaction and became apparent only later. In particular, slow-drift and backdoor-trigger attacks could evade evaluations that looked only at individual steps until their effects appeared in later interactions.
Checking once may not be enough
This creates a problem for how we test AI security.
Suppose a security team examines an AI agent immediately after it encounters suspicious information. The agent appears to behave normally, so the interaction is judged safe.
That may be like inspecting a notebook immediately after someone has inserted a misleading entry but before anyone has acted on it. The absence of immediate harmful behaviour does not necessarily mean the attack failed.
Our results also showed that the risk did not always increase in a simple straight line. Some attacks produced what we call “non-monotonic” patterns: behaviour could appear more concerning at one stage and less concerning at another before the attack ultimately developed.
This means testing an AI agent one prompt or one interaction at a time can miss part of the picture. Instead, security evaluations may need to follow the agent across a sequence of interactions — essentially watching the whole story rather than examining individual photographs.
A new security problem
This issue is becoming more important as AI systems develop from chatbots that respond to individual questions to agents designed to perform longer tasks.
An ordinary chatbot conversation can often be treated as relatively self-contained. A memory-enabled agent is different because yesterday’s information may influence tomorrow’s decision.
When an AI agent can also use tools, the consequences of a poisoned memory can extend beyond generating an incorrect sentence. An agent might eventually use remembered information when deciding what action to take, which resource to access or which instruction to follow.
This doesn’t mean that memory-enabled AI agents are inherently unsafe. Memory provides important benefits: it allows an agent to maintain context, learn user preferences and work on tasks that cannot be completed in a single interaction.
But it changes what defenders need to protect.
Securing the prompt in front of an AI system is no longer necessarily enough. The information that the system carries forward may also need protection.
From snapshots to stories
There is a simple lesson from our research: When an AI system has memory, security also has a memory.
If an attack can be planted at one moment and activated much later, evaluating only the moment when malicious information first appears — or only the moment when something goes wrong — can leave out what happened in between.
That is why we argue for trajectory-aware security testing: evaluating how an AI agent’s behaviour develops across multiple interactions rather than judging each step in isolation.
For non-specialists, the idea is perhaps easier to understand without the technical terminology. If you want to know whether an assistant’s notebook has been compromised, you cannot simply watch the assistant write one page.
You also need to pay attention to what the assistant remembers — and what happens when it eventually opens that notebook again.
Abbas Yazdinejad, Assistant Professor, Department of Computer Science, University of Regina
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Edited by Irfan Ahmad.Read next:
• AI agents are all the rage—but research shows they leak private data
• TikTok's design makes it harder for young people to think critically about the mental health content they see
by External Contributor via Digital Information World
