An Information Theoretic Evaluation Metric For Strong Unlearning

The introduction of the Information Difference Index (IDI) marks a significant advancement in the field of machine unlearning (MU), which seeks to eliminate the influence of specific data from trained models to address privacy concerns and comply with regulations such as the 'right to be forgotten.' Evaluating strong unlearning, where the modified model is indistinguishable from one retrained without the forgotten data, has posed challenges in deep neural networks (DNNs). Traditional evaluation methods often fail to capture residual information in intermediate layers. The IDI offers a novel approach by quantifying retained information through mutual information measurements between features and the labels to be forgotten. Experiments have demonstrated IDI's effectiveness across various datasets and architectures, establishing it as a reliable tool for assessing unlearning efficacy in DNNs.