TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classification

A recent study analyzed heart failure patient trajectories using sequence modeling, focusing on the performance of six sequence models, including Transformers and the newly introduced Mamba architecture, within a large Swedish cohort of 42,820 patients. The models were evaluated on their ability to predict clinical instability and other outcomes based on electronic health records (EHRs).

A recent study published on arXiv systematically evaluated 12 feature scaling techniques across 14 machine learning algorithms and 16 datasets, revealing significant performance variations in models like Logistic Regression and SVMs, while ensemble methods showed robustness regardless of scaling.

A systematic review has been conducted on relation extraction (RE) research since the introduction of Transformer-based models, analyzing 34 surveys, 64 datasets, and 104 models published from 2019 to 2024. The study highlights advancements in methodologies, benchmark resources, and the integration of semantic web technologies, providing a comprehensive reference for the evolution of RE.

A new framework called Convolutional Nearest Neighbors (ConvNN) has been introduced, unifying convolutional neural networks and transformers within a k-nearest neighbor aggregation framework. This approach highlights that both convolution and self-attention can be viewed as methods of neighbor selection and aggregation, with ConvNN serving as a drop-in replacement for existing layers in neural networks.