Logo image
Back
Towards Analysis and Interpretation of Large Language Models for Arithmetic Reasoning
Conference proceeding

Towards Analysis and Interpretation of Large Language Models for Arithmetic Reasoning

Mst. Shapna Akter, Hossain Shahriar and Alfredo Cuzzocrea
Swiss Conference on Data Science (Online), (2024), pp.267-270
05/30/2024
DOI: https://doi.org/10.1109/SDS60720.2024.00049
Web of Science ID: WOS:001322673800042

Metrics

1 Record Views

Abstract

Arithmetic Arithmetic Reasoning Attention mechanisms Causal Mediation Analysis Cognition Data science Large language models LLMs Mediation Transformers
Large Language Models (LLMs) have recently conquered the research scene, with particular regards to the Transformer architecture in the context of arithmetic reasoning. In this so-delineated scenario, this paper puts the basis for a causal mediation analysis about the approach of Transformer-based LLMs to complex arithmetic problems. In particular, we try to discover which parameters are crucial for complex reasoning tasks such as model activations. Our preliminary results state that, for complex arithmetic operations, information is channeled from mid-layer activations to the final token through enhanced attention mechanisms. Preliminary experiments are reported.

Details

Logo image