Researchers at Abu Dhabi’s Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) have developed a new method to prevent large language models from leaking sensitive information during live operation. The new technique, called DP-Fusion, offers mathematically provable privacy guarantees, addressing a critical vulnerability for enterprises deploying AI in high-stakes environments.
Quick Facts
- New method protects sensitive data in live AI.
- Offers mathematically provable token-level privacy.
- Achieves 6x better output quality than rivals.
Beyond Training: Securing Live AI Inference
Much of the global research on AI privacy has focused on the training phase. However, MBZUAI’s work targets the inference stage—the moment a live model processes new, potentially sensitive data to generate a response. This is where the real-world operational risk is concentrated, as AI models are increasingly deployed in regulated sectors like healthcare, finance, and government.
The demand for AI inference is growing rapidly, projected to become a $250 to $350 billion market by 2030. Each time an AI processes patient records, client financial data, or classified documents, it creates a potential point of failure. DP-Fusion was created specifically to close this gap, providing a level of certainty that previous methods lacked.
How DP-Fusion Achieves Provable Privacy
Unlike data scrubbing or paraphrasing techniques which offer no formal guarantees, DP-Fusion provides a mathematically bounded limit on how much sensitive information can be inferred by an attacker. The research, presented at the prestigious International Conference on Learning Representations (ICLR), outlines a four-step process.
The method first labels sensitive tokens in the input text, then runs the model twice: once without the sensitive data to create a baseline, and a second time with it. Finally, it blends the two output distributions, ensuring the final result remains close enough to the baseline to obscure the sensitive information while preserving the overall quality and coherence of the generated text. The result is a system with six times lower perplexity—a measure of output quality—than competing privacy methods.
A Dual Role: Privacy and Security
A key feature of DP-Fusion is its ability to also defend against common AI attacks. By treating data retrieved from untrustworthy external sources as “sensitive,” the method can effectively mitigate prompt injection and jailbreaking attempts. This gives the system a dual function as both a privacy tool and a security defense against adversarial manipulation.
This capability is crucial for organizations where personally identifiable information or confidential data flows through a live language model. Operators can adjust the privacy-utility balance with a single parameter, giving them practical control over the level of protection required for different use cases.
About Mohamed bin Zayed University of Artificial Intelligence
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) is a graduate research university dedicated to advancing AI as a global force for humanity. Based in Abu Dhabi, the university aims to establish and continually evolve a world-class research and development ecosystem around AI.
Source: Middle East AI News
