Comparing Reconstruction Attacks on Pretrained Versus Full Fine-tuned Large Language Model Embeddings on Homo Sapiens Splice Sites Genomic Data

The study on embedding reconstruction attacks in large language models (LLMs) applied to genomic sequences highlights the critical implications of model training on privacy. Building on Pan et al.'s work, which established that pretrained language models can leak sensitive information, this research employs the HS3D genomic dataset to analyze the effects of fine-tuning on reconstruction vulnerability. By implementing specialized tokenization for DNA sequences, the study enhances the model's processing capabilities. The findings indicate a clear distinction in vulnerability between pretrained and fine-tuned embeddings, suggesting that task-specific optimization can significantly influence privacy outcomes. This underscores the importance of understanding the implications of model adjustments in sensitive fields like genomics, where data privacy is paramount.