TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices

A new framework for camera-based animal re-identification (Animal Re-ID) has been proposed, enabling wildlife monitoring and precision livestock management on microcontrollers (MCUs) in environments with limited connectivity. This framework addresses the challenges of running complex models on low-power devices by designing a high-accuracy architecture based on a scaled MobileNetV2 backbone for low-resolution inputs.

The introduction of DAASH, a meta-attack framework, marks a significant advancement in generating effective and perceptually aligned adversarial examples, addressing the limitations of traditional Lp-norm constrained methods. This framework strategically composes existing attack methods in a multi-stage process, enhancing the perceptual alignment of adversarial examples.

Recent research challenges the notion that weight oscillations during Quantization Aware Training (QAT) are merely undesirable effects, proposing instead that they are crucial for enhancing the robustness of neural networks. The study demonstrates that these oscillations, induced by a new regularizer, can help maintain performance across various quantization levels, particularly in models like ResNet-18 and Tiny Vision Transformer evaluated on CIFAR-10 and Tiny ImageNet datasets.

A recent study explores the potential of pruning at initialization (PaI) by drawing inspiration from iterative pruning methods, aiming to enhance performance in deep learning models. The research highlights the significance of identifying surviving subnetworks based on initial features, which could lead to more efficient pruning strategies and reduced training costs, especially as neural networks grow in size.

A recent study has introduced a method for fully decentralized certified unlearning in machine learning, focusing on the removal of specific data influences from trained models without a central coordinator. This approach, termed RR-DU, employs a random-walk procedure to enhance privacy and mitigate data poisoning risks, providing convergence guarantees in convex scenarios and stationarity in nonconvex cases.

A recent study has introduced a pruned version of Capsule Networks (CapsNets), demonstrating that it can operate up to 9.90 times faster than traditional architectures by eliminating 95% of Primary Capsules while maintaining accuracy across various datasets, including MNIST and CIFAR-10.

A recent study highlights the vulnerabilities of backdoor defenses in neural networks when subjected to post-training quantization, revealing that INT8 quantization leads to a 0% detection rate for all evaluated defenses while attack success rates remain above 99%. This raises concerns about the effectiveness of existing security measures in machine learning systems.

A new paper introduces an innovative dataset quantization method aimed at reducing storage and communication costs for large-scale datasets on resource-constrained edge devices. This approach focuses on compressing individual samples by minimizing intra-sample redundancy while retaining essential features, marking a shift from traditional inter-sample redundancy methods.