Metrics
1 Record Views
Thesis
Investigating the Compressibility of Loss Minima in Neural Networks
University of West Florida Libraries
Master of Science (MS), University of West Florida
2026
Abstract
This thesis primarily investigates model compression in modern neural networks through the lens of loss-minimum compressibility. We study two independent methods that guide training toward minima that lose less accuracy under the same compression budget. First, we use a self-distillation loss during large language model (LLM) fine-tuning, reducing accuracy degradation under pruning. Second, we advance an uncertainty-aware, edge-computing, packet inspection task. This task is an example of a high-risk, edge
task that would benefit most from our distillation-based compression techniques. Third, we use the Muon optimizer in an LLM fine-tuning framework and show that Muon-optimized minima are more resilient to quantization error. Together, these results highlight the importance of choosing both loss function and optimizer when designing compression-aware training pipelines: minima with equivalent training loss can differ substantially in post-compression performance, especially in tasks with uncertainty-quantified components.
Details
- Title
- Investigating the Compressibility of Loss Minima in Neural Networks
- Resource Type
- Thesis
- Contributors
- Brian Jalaian (Committee Chair)Brent Venable (Committee Member)
- Publisher
- University of West Florida Libraries
- Format
- pdf
- Number of pages
- 103
- Copyright
- Permission granted to the University of West Florida Libraries by the author to digitize and/or display this information for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires the permission of the copyright holder.
- Identifiers
- 99381762745306600
- Academic Unit
- Computer Science; Hal Marcus College of Science and Engineering
- Language
- English
- Awarding Institution
- University of West Florida; Master of Science (MS)
- Theses and Dissertations
- Master of Science (MS), University of West Florida