Last Week in AI #328 - DeepSeek 3.2, Mistral 3, Trainium3, Runway Gen-4.5

French AI startup Mistral has launched the Devstral 2 coding model, which includes a laptop-friendly version optimized for software engineering tasks. This release follows the introduction of the Mistral 3 LLM family, aimed at enhancing local hardware capabilities for developers.

The latest episode of the LWiAI Podcast features Jeremie discussing the release of DeepSeek 3.2, an AI model that promises to be faster, cheaper, and smarter than its predecessors. This update highlights the company's ongoing efforts to enhance its technology and compete in the rapidly evolving AI landscape.

Recent research has demonstrated the potential of Large Language Models (LLMs) in automating the generation of discharge summaries, which are critical documents in patient care. The study evaluated five models, including proprietary systems like GPT-4 and Gemini 1.5 Pro, and found that Gemini, particularly with one-shot prompting, produced summaries most similar to gold standards. This advancement could significantly reduce the workload of healthcare professionals and enhance the accuracy of patient information.

A new online structured pruning technique called Token Filtering has been introduced for large language models (LLMs), allowing pruning decisions to be made during inference without the need for calibration data. This method measures token redundancy through joint key-value similarity, effectively reducing inference costs while maintaining essential information. The approach also includes a variance-aware fusion strategy to ensure important tokens are preserved even with high pruning ratios.

The introduction of Graph-Regularized Sparse Autoencoders (GSAEs) aims to enhance the safety of large language models (LLMs) by addressing their vulnerabilities to adversarial prompts and jailbreak attacks. GSAEs extend traditional sparse autoencoders by incorporating a Laplacian smoothness penalty, allowing for the recovery of distributed safety representations across multiple features rather than isolating them in a single latent dimension.

A new method called Depth-Wise Activation Steering has been introduced to enhance the honesty of large language models (LLMs) like LLaMA, Qwen, and Mistral. This training-free approach utilizes a Gaussian schedule to improve the models' ability to report truthfully, addressing the issue of models asserting falsehoods despite having the correct information internally.

The paper introduces QiMeng-SALV, a novel approach to Verilog code generation that utilizes Signal-Aware Learning to enhance Reinforcement Learning (RL) training by focusing on functionally correct output signals. This method aims to address the challenges faced in automated circuit design, particularly the optimization of RL for generating accurate Verilog code.

Recent research indicates that large language models (LLMs) demonstrate biases in evaluation tasks, particularly favoring self-generated content. However, a study exploring retrieval-augmented generation (RAG) frameworks found no significant self-preference effect, suggesting that LLMs can evaluate factual content more impartially than previously thought.