A Large-scale Benchmark on Geological Fault Delineation Models: Domain Shift, Training Dynamics, Generalizability, Evaluation and Inferential Behavior

The recent large-scale benchmarking study on geological fault delineation models highlights significant challenges in the field of seismic interpretation, particularly regarding the generalizability of machine learning models. Conducted on over 200 combinations of model architectures and datasets, including FaultSeg3D, CRACKS, and Thebe, the study found that traditional fine-tuning practices often result in catastrophic forgetting, undermining model performance. In contrast, domain adaptation methods proved to be more effective when faced with substantial shifts in data distribution. This research is vital as it provides a systematic understanding of the limitations and potential of current models, paving the way for more reliable applications in geological exploration. The findings emphasize the need for consistent evaluation protocols and improved strategies to enhance the deployment of machine learning in real-world scenarios.