Enhancing DPSGD via Per-Sample Momentum and Low-Pass Filtering

The recent submission of the paper 'Enhancing DPSGD via Per-Sample Momentum and Low-Pass Filtering' on arXiv presents a novel approach to improving Differentially Private Stochastic Gradient Descent (DPSGD), a method widely used for training deep neural networks while ensuring privacy. Traditional implementations of DPSGD often suffer from reduced accuracy due to the introduction of noise and bias. The proposed DP-PMLF method effectively mitigates these issues by combining per-sample momentum with a low-pass filtering strategy, which smooths gradient estimates and reduces sampling variance. The theoretical analysis provided in the paper indicates an improved convergence rate while maintaining rigorous differential privacy guarantees. Empirical evaluations further demonstrate that DP-PMLF significantly enhances the balance between privacy and utility compared to existing state-of-the-art DPSGD variants. This advancement is crucial for the ongoing development of privacy-preserving machin…