TL;DR
Researchers showed people a short animated accident, then deliberately gave some of them a recap that named the wrong traffic sign.
Among 328 people included in the analysis, 83.6% identified the sign correctly after an accurate recap, compared with 44.8% after a misleading one.
AI didn’t invent this effect. But if we rely on generated recaps, we need a better way to check them against what actually happened.
Someone watches a short video of a car approaching a yield sign. The car turns and collides with a pedestrian. A day or two later, the person reads a recap of the video.
The researchers deliberately change the sign in the recap, so it says stop instead of yield. Then they ask the person which sign they remember seeing.
In a recent experiment, that kind of misleading recap made a big difference. People who read the incorrect version were much less likely to identify the original traffic sign correctly, even when the researchers told them to answer from their memory of the video rather than from the recap.
This is the part I find fascinating. We talk a lot about checking what AI writes. But what happens if reading the wrong description changes what the person doing the checking remembers?
What the researchers tested
A paper posted September 23 by Mattea Sim, Yael Eiger and Tadayoshi Kohno looked at two related questions. First, how accurately do AI systems describe short videos? Second, can an inaccurate AI-generated recap affect what people remember about a video they already watched?
For the memory experiment, participants saw a short animated car-and-pedestrian accident. The key difference between the two video versions was the traffic sign at the intersection: stop or yield.
Between 24 and 48 hours later, participants returned and read a recap generated by ChatGPT. The researchers deliberately controlled one detail. Some people received a recap that named the correct traffic sign. Others received one naming the wrong sign.
That detail matters because the study wasn’t simply measuring whether a chatbot happened to make a mistake. The researchers intentionally changed the sign description to test its effect. All participants read AI-generated text, but some were told it came from AI and others were told it came from a human transcriber.
Of the 331 people who completed both parts, 328 passed the required checks and were included in the analysis.
The memory test asked participants to choose between stop and yield. They identified the original sign correctly at these rates:
83.6% of people who read the accurate recap identified the traffic sign correctly.
44.8% of people who read the misleading recap identified the traffic sign correctly.
The researchers also checked whether participants remembered other parts of the video. They generally did. For example, 96.6% correctly remembered that the accident happened during the day, a detail the recap hadn’t mentioned.
The mistaken answers weren’t simply proof that people had forgotten the entire video. The wrong sign in the recap had become part of how they answered a question about what they saw.
This isn’t a new problem, but the delivery has changed
Psychologists have studied something called the misinformation effect for decades. In plain English, information we hear or read after an event can affect how we remember the original event.
That’s why questions asked after an accident can matter. A person might be certain about what they saw and still have their memory nudged by later information.
The new study didn’t show that AI invented this weakness. The researchers didn’t find a statistically significant difference between people told the recap came from AI and people told it came from a human. An AI label wasn’t enough to prevent the effect in this experiment.
What has changed is how easy it is to produce and distribute a polished account of an event before anybody checks it carefully.
Meeting transcripts get shortened into action items. Calls become customer notes. Videos get turned into incident reports. A long conversation gets compressed into a paragraph someone else will read before opening the original.
Those are useful features. I’m not arguing we should stop summarizing things. I’m asking what happens when the convenient version contains a mistake and becomes the version people return to as the source of truth.
Twenty summaries, and a reason to be careful
The researchers also tested ChatGPT-5.5 and Gemini 2.5 Flash-Lite on two 25-second animated videos. They ran each model five times per video, using the same prompt each time. It asked for a factual, neutral account of at least 300 words.
All 20 summaries contained errors under the researchers’ coding scheme. Nineteen omitted the collision with the pedestrian. Other mistakes included getting the order or direction of actions wrong.
Those results show how these recaps went wrong, but don’t establish a failure rate for other models, prompts, videos or meeting notes.
The memory experiment also compared accurate and misleading recaps, with no group that read no recap. The difference therefore doesn’t tell us how either version compares with relying on memory alone.
A review can go wrong before anyone checks a fact
Here’s a hypothetical work example.
An engineer joins an incident call. A colleague discusses a configuration change but doesn’t approve it. The next day, an AI-generated recap says the colleague approved the change at 2:15 PM.
The engineer was on the call. But after reading that recap, they may remember an approval that never happened. That’s the possibility I’d want to test in a real incident review, not a result this experiment established.
What concerns me is whether someone who already saw or heard the original can still catch the recap’s mistake from memory.
In The Source Is Becoming Optional, I wrote about how AI search can make us less likely to open the material behind an answer. This problem starts after somebody has already seen the source. A misleading recap may affect what they remember from it.
And in Memory Is a Write Path, I looked at what software stores and carries into later work. Here, the memory at risk belongs to a person.
What I’d change about reviewing AI recaps
When a recap could affect a decision, I’d change the order of a few ordinary steps. These are precautions I’d test, not methods this experiment proved work.
Keep the original. A recording, transcript, log or document should remain available, with its source and date intact. A summary is a convenience. It isn’t a replacement for the thing being summarized.
Give reviewers a chance to record what they saw first. Where practical, have a witness, investigator or meeting participant write a short account before they see somebody else’s polished recap. That preserves their first account for comparison.
Check the claims that could change a decision. Who approved what? What time did it happen? Which person said something? Which action happened first? Go to the timestamp or original record for those details. Don’t ask a reviewer to sign off after simply reading the recap a second time.
Make corrections visible. If the recap says stop sign and the video shows yield, record the discrepancy instead of quietly editing the final paragraph. Otherwise somebody may keep an older copy and repeat the error later.
There will be work where a fast summary is perfectly fine. Nobody needs a forensic review to summarize lunch plans. But an incident report, a legal account or a decision about someone else’s job deserves more care.
I like what AI summaries can do. But having seen the original earlier wouldn’t be enough for me to approve a recap that could affect a decision. I’d want the reviewer to compare it with the source, not just their memory of it.