A robust methodology for long-term sustainability evaluation of Machine Learning models

The development of a robust methodology for assessing the long-term sustainability of machine learning models marks a significant advancement in the field of artificial intelligence. Current evaluation practices primarily focus on short-term resource usage, leading to a lack of standardized protocols that can accurately reflect the real-world lifecycles of AI systems. The proposed comprehensive evaluation protocol aims to address these gaps by applying to both batch and streaming learning scenarios. Experiments conducted on various classification tasks reveal that traditional static evaluations fail to capture the sustainability of models as they evolve and undergo repeated updates. The findings indicate that long-term sustainability varies significantly across different models, with many exhibiting higher environmental costs that yield little performance benefit. This research is essential for guiding future AI development towards more sustainable practices.