A Data-driven Typology of Vision Models from Integrated Representational Metrics

A new study has introduced a quantitative framework to evaluate representational similarity metrics, assessing their discriminative capacity across various model families, including CNNs, Vision Transformers, and ConvNeXt. The research utilizes three separability measures to compare commonly used metrics such as RSA and soft matching, revealing that stricter alignment constraints enhance separability.

A new approach to Vision Transformers (ViTs) has been introduced, featuring a Jumbo token that enhances processing speed by reducing patch token width while increasing global token width. This innovation aims to address the slow performance of ViTs without compromising their generality or accuracy, making them more practical for various applications.

A new study proposes a structured initialization method for Vision Transformers (ViTs), aiming to integrate the strong inductive biases of Convolutional Neural Networks (CNNs) without altering the architecture. This approach is designed to enhance performance on small datasets while maintaining scalability as data increases.

A new study presents a unified perspective on loss-oriented imbalanced learning through localization, addressing the bias in learning processes caused by class imbalances in real-world datasets. The research critiques existing loss function modifications, such as re-weighting and logit-adjustment, for their coarse-grained analysis and proposes localized calibration to better capture class-dependent influences on learning.