Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspective

A recent study highlights the significance of Machine Unlearning in enhancing privacy within Machine Learning models. By enabling the selective removal of data from trained models, techniques such as Naive Retraining and Exact Unlearning via the SISA framework are evaluated for their computational costs and feasibility using the HSpam14 dataset. The research emphasizes the integration of unlearning principles into Positive Unlabeled Learning to tackle challenges posed by partially labeled datasets.

A new study introduces a framework called Redundancy-guided Invariant Graph learning (RIG), which utilizes Partial Information Decomposition (PID) to enhance out-of-distribution (OOD) generalization in graph representation learning. This approach aims to mitigate the retention of spurious components in learned representations by maximizing redundant information while isolating causal subgraphs.

A recent study has provided a unified characterization of the scaling of parameter norms in overparameterized linear regression and diagonal linear networks under $l_p$ bias. This work addresses the unresolved question of how the family of $l_r$ norms behaves with varying sample sizes, revealing a competition between signal spikes and null coordinates in the data.