Unpeer-reviewed study flags reliability risk in DeepSeek’s memory-saving AI technology

A new study has identified a potential reliability problem in the technology used to reduce the memory needed by DeepSeek’s artificial intelligence models.

The study, which has not been peer-reviewed, said the cost-saving approach may make the models more likely to miss the same information when it appears in certain positions in a document.

A challenge to DeepSeek’s low-cost approach

The findings represent a rare challenge to a breakthrough that helped the Chinese artificial intelligence lab narrow the gap with US rivals OpenAI and Anthropic through its low-cost approach.

The research focused on DeepSeek’s efforts to reduce the memory required to process lengthy inputs. While that technology can lower memory demands, the study said it may come at the expense of reliability.

In one example, adding irrelevant text before the same piece of computer code impaired DeepSeek’s ability to select the correct answer. This suggested that the model’s performance could change even when the relevant code remained the same.

The paper said the phenomenon appeared in “DeepSeek-V4, V4.1, and all compressed models we pretrained”. The findings therefore raise questions about whether the memory-saving technology performs consistently across different document positions and model versions.

Source: South China Morning Post