Attention Via Convolutional Nearest Neighbors

WARP-LUTs, or Walsh-Assisted Relaxation for Probabilistic Look-Up Tables, is a novel gradient-based method introduced to enhance machine learning efficiency. This approach focuses on learning combinations of logic gates with fewer trainable parameters, addressing the high computational costs associated with training models like Differentiable Logic Gate Networks (DLGNs). WARP-LUTs aim to improve accuracy, resource usage, and latency, making them a significant advancement in the field of AI.

Bladder cancer is a prevalent malignancy with a high recurrence rate of up to 78%, necessitating precise post-operative monitoring. Multi-sequence contrast-enhanced MRI is commonly utilized for recurrence detection, but interpreting these scans is challenging due to post-surgical changes. This study introduces a curated multi-sequence, multi-modal MRI dataset designed for bladder cancer recurrence prediction and proposes H-CNN-ViT, a new model aimed at enhancing prediction accuracy in this critical area.

The article discusses a new framework for differential privacy auditing in machine learning systems. Traditional methods require altering training datasets, which can be resource-intensive. The proposed observational auditing framework utilizes the randomness of data distributions to evaluate privacy without modifying the original dataset. This approach extends privacy auditing to protected attributes, including labels, addressing significant gaps in existing techniques. Experiments conducted on Criteo and CIFAR-10 datasets validate its effectiveness.

MI-to-Mid Distilled Compression (M2M-DC) is a novel compression framework that combines information-guided block pruning with progressive inner slicing and staged knowledge distillation. The method ranks residual blocks based on a mutual information signal, removing the least informative units. It alternates short knowledge distillation phases with channel slicing to maintain computational efficiency while preserving model accuracy. The approach has demonstrated promising results on CIFAR-100, achieving high accuracy with significantly reduced parameters.

The article discusses advancements in predicting drug-target interactions, a critical aspect of drug discovery and design. Recent methods utilizing deep learning technologies, particularly graph neural networks (GNNs) and Transformers, have shown remarkable performance by effectively extracting structural information. However, the benchmarking of these methods varies significantly, affecting algorithmic progress. The authors conducted a comprehensive survey and benchmark to integrate various structure learning algorithms for improved modeling.

This study explores a novel approach to enhance vision transformers (ViTs) by pretraining them on procedurally-generated data that lacks visual or semantic content. Utilizing simple algorithms, the research aims to instill generic biases in ViTs, allowing them to internalize abstract computational priors. The findings indicate that this warm-up phase, followed by standard image-based training, significantly boosts data efficiency, convergence speed, and overall performance, with notable improvements observed on ImageNet-1k.

Deep neural networks are susceptible to adversarial perturbations that can lead to incorrect predictions. The paper introduces DeepDefense, a defense framework utilizing Gradient-Feature Alignment (GFA) regularization across multiple layers to mitigate this vulnerability. By aligning input gradients with internal feature representations, DeepDefense creates a smoother loss landscape, reducing sensitivity to adversarial noise. The method shows significant robustness improvements against various attacks, particularly on the CIFAR-10 dataset.

The Abstraction and Reasoning Corpus (ARC) aims to advance research in abstract reasoning, a key component of human intelligence. Traditional methods approach ARC as a language problem, often utilizing large language models or recurrent reasoning models. This study proposes a vision-centric approach, treating ARC as an image-to-image translation task. By using a 'canvas' for input representation, standard vision architectures like Vision Transformers (ViT) can be applied, allowing the model to generalize to new tasks through test-time training.