Why and When Deep is Better than Shallow: An Implementation-Agnostic State-Transition View of Depth Supremacy

A recent article published on arXiv presents an implementation-agnostic framework to explain why deep models outperform shallow ones, emphasizing their conceptualization as abstract state-transition semigroups. This perspective allows for a generalized understanding of depth supremacy beyond specific network architectures. The study further introduces a bias-variance decomposition that elucidates the critical role of depth in influencing variance, thereby contributing to the models' overall performance. By framing deep models in this abstract manner, the article highlights inherent advantages in representation capacity and variance control compared to shallow counterparts. These insights support the claim that depth confers a fundamental superiority in model expressiveness and learning dynamics. The findings align with ongoing research trends in machine learning that seek to characterize the theoretical underpinnings of deep learning architectures. This work thus adds a valuable dimension to the discourse on why and when deeper models are preferable, independent of particular implementation details.