RefiDiff: Progressive Refinement Diffusion for Efficient Missing Data Imputation

The introduction of RefiDiff marks a significant advancement in the field of data imputation, particularly for high-dimensional, mixed-type datasets that often face challenges due to missing values, especially under Missing Not At Random (MNAR) conditions. Traditional methods have struggled to effectively integrate local and global data characteristics, leading to suboptimal performance. RefiDiff addresses this gap by combining local machine learning predictions with a novel Mamba-based denoising network, which captures long-range dependencies among features and samples while maintaining low computational complexity. This innovative approach bridges predictive and generative paradigms of imputation, utilizing pre-refinement for initial imputations and post-refinement for enhanced accuracy. Extensive evaluations on nine real-world datasets have demonstrated that RefiDiff not only outperforms state-of-the-art methods across various missing-value settings but also exhibits strong performa…