A Unified Framework for Inference with General Missingness Patterns and Machine Learning Imputation

The recent development of a novel method for valid statistical inference under the missing-at-random (MAR) assumption marks a significant advancement in the field of machine learning and data analysis. Traditional methods have been limited to scenarios where data is missing completely at random (MCAR), which does not reflect the complexities often encountered in real-world data. By stratifying observations based on distinct missingness patterns and employing a masking-and-imputation procedure, this new approach allows for more accurate estimations and analyses. The method not only provides theoretical guarantees of asymptotic normality but also demonstrates efficiency dominance over weighted complete-case analyses. This is crucial for researchers who rely on machine learning predictions to fill gaps in incomplete datasets, as naive integration of these predictions can lead to biased inferences. The ability to implement this method using existing software further enhances its accessibil…