Mitigating Negative Flips via Margin Preserving Training

The article presents SemanticNN, a novel semantic codec designed for extremely weak embedded devices in the Internet of Things (IoT). It addresses the challenges of integrating artificial intelligence (AI) on such devices, which often face resource limitations and unreliable network conditions. SemanticNN focuses on achieving semantic-level correctness despite bit-level errors, utilizing a Bit Error Rate (BER)-aware decoder and a Soft Quantization (SQ)-based encoder to enhance collaborative inference offloading.

The paper titled 'LampQ: Towards Accurate Layer-wise Mixed Precision Quantization for Vision Transformers' presents a new method for quantizing pre-trained Vision Transformer models. The proposed Layer-wise Mixed Precision Quantization (LampQ) addresses limitations in existing quantization methods, such as coarse granularity and metric scale mismatches. By employing a type-aware Fisher-based metric, LampQ aims to enhance both the efficiency and accuracy of quantization in various tasks, including image classification and object detection.

The paper titled 'Convergence Bound and Critical Batch Size of Muon Optimizer' presents a theoretical analysis of the Muon optimizer, which has shown strong empirical performance and is proposed as a successor to AdamW. The study provides convergence proofs for Muon across four practical settings, examining its behavior with and without Nesterov momentum and weight decay. It highlights that the inclusion of weight decay results in tighter theoretical bounds and identifies the critical batch size that minimizes training costs, validated through experiments in image classification and language modeling.