Logo image
Back
Uncovering the Interpretation of Large Language Models
Conference proceeding   Peer reviewed

Uncovering the Interpretation of Large Language Models

Mst Shapna Akter, Hossain Shahriar, Alfredo Cuzzocrea and Fan Wu
Proceedings : annual International Computer Software and Applications Conference, pp.1057-1066
Annual Computers, Software, and Applications Conference (COMPSAC), 48th (Osaka, Japan, 07/02/2024–07/04/2024)
08/2024
DOI: https://doi.org/10.1109/COMPSAC61105.2024.00143
Web of Science ID: WOS:001308581200134

Metrics

1 Record Views

Abstract

Causal Mediation Analysis Interpretation Large language models Reasoning Data Systems Mathematical Models Mathematics
Recent research has shown growing interest in the arithmetic reasoning capabilities of large language models (LLMs), especially those built on the Transformer architecture. However, our understanding of the intrinsic processes within these models for arithmetic calculations remains scant. This study leverages causal mediation analysis to offer an in-depth look at how Transformer-based LLMs approach complex arithmetic problems. We experimentally intervened on particular model activations and assessed the resulting shifts in prediction probabilities, allowing us to pinpoint which parameters are crucial for such reasoning tasks. We discovered that for complex arithmetic operations, information is channeled from mid-layer activations to the final token through enhanced attention mechanisms. Subsequently, Multi-Layer Perceptrons (MLP) modules synthesize this data, integrating it into the model's residual pathways. To validate these observations, we also evaluated the activation dynamics across different types of tasks, such as retrieving numbers from text and answering fact-based questions.

Details

Logo image