New Approach Finds Privacy Vulnerability and Performance Are Intertwined in AI Neural Networks
For Immediate Release
Researchers have discovered that some of the elements of AI neural networks that contribute to data-privacy vulnerabilities are also key to the performance of those models. The researchers used this new information to develop a technique that better balances performance and privacy protection in these models.
The findings involve protecting neural networks against membership inference attacks (MIAs), which are techniques that allow attackers to determine whether a particular piece of data was used to train a specific AI model.
“MIAs can jeopardize the privacy of individuals whose data was part of the training dataset,” says Xingli Fang, first author of a paper on the work and a Ph.D. student at North Carolina State University. “For example, if an attacker has partial data from an individual, it could use an MIA to determine if an AI model was trained using data from that individual.”
“And if the individual’s data was used to train that model, the attacker could then infer the rest of the user’s information,” says Jung-Eun Kim, corresponding author of the paper and an assistant professor of computer science at NC State. “Basically, MIAs pose a privacy vulnerability.”
To understand what the researchers learned, you have to understand “weight parameters.” Weight parameters are an important component of AI neural networks, such as large language models. Essentially, weight parameters serve as the synapses that link all of the neurons in the model together, and data inputs travel through these weight parameters as the model takes the data and produces an output.
“When we started this project, we wanted to get a better understanding of which weight parameters in a model are most important for protecting privacy and which weight parameters are most important for performance,” says Kim. “It was fundamental AI research.”
“We found that only a few weight parameters represent a significant privacy vulnerability,” says Fang. “However, we were surprised to learn that the vulnerable weight parameters are also among the most important weight parameters when it comes to performance. This means it is extremely difficult to reduce vulnerability risk without also hurting performance.
“However, we were able to use our new insights to develop a novel approach for improving data privacy by modifying the weight parameters and going through a fine-tuning process to adjust the model.”
To test the new approach, the researchers compared their privacy protection technique to four other techniques to see how they performed when defending against two state-of-the-art MIAs.
“We found that our approach achieves a better balance of privacy and performance relative to the previous techniques,” says Kim. “We’re happy to talk with anyone in the field about how to incorporate this approach into their training.”
The paper, “Learnability and Privacy Vulnerability Are Entangled in a Few Critical Weights,” will be presented at the Fourteenth International Conference on Learning Representations (ICLR2026), being held April 23-27 in Rio de Janeiro, Brazil.
-shipman-
Note to Editors: The study abstract follows.
“Learnability and Privacy Vulnerability Are Entangled in a Few Critical Weights”
Authors: Xingli Fang and Jung-Eun Kim, North Carolina State University
Presented: April 23-27, the Fourteenth International Conference on Learning Representations (ICLR2026), Rio de Janeiro, Brazil
Abstract: Prior approaches for membership privacy preservation usually update or retrain all weights in neural networks, which is costly and can lead to unnecessary utility loss or even more serious misalignment in predictions between training data and non-training data. In this work, we observed three insights: i) privacy vulnerability exists in a very small fraction of weights; ii) however, most of those weights also critically impact utility performance; iii) the importance of weights stems from their locations rather than their values. According to these insights, to preserve privacy, we score critical weights, and instead of discarding those neurons, we rewind only the weights for fine-tuning. We show that, through extensive experiments, this mechanism exhibits outperforming resilience in most cases against Membership Inference Attacks while maintaining utility.
