Logo image
Science-Based Methodologies for Evaluating the Human-Centeredness of AI Systems
Journal article   Open access   Peer reviewed

Science-Based Methodologies for Evaluating the Human-Centeredness of AI Systems

Jia Zhou, Martin Schrepp, Helmut Degen, Vincent G. Duffy, Qin Gao, George Margetis, Margaret McKay, Fiona Nah, Pei-Luen Patrick Rau, Dylan Schmorrow, …
International journal of human-computer interaction, pp.1-46
09/15/2026
Web of Science ID: WOS:001876324700001

Metrics

1 Record Views

Abstract

Human-Centered AI (HCAI) Human-Centered AI Scale (HuCAS) subjective perception objective assessment scale maturity Artificial Intelligence or Cybernetics Business Administration
Existing approaches to implement Human-Centered AI (HCAI) are often ad hoc, compliance-driven, or focused on isolated user outcomes. This paper proposes a dual-assessment framework encompassing a subjective questionnaire for end-users and an objective assessment tool for organizations. To measure end-users' subjective impressions of various HCAI qualities, a questionnaire was designed and distributed among 153 participants from 17 countries. The survey results contribute to the modular, seven-factor Human-Centered AI Scale (HuCAS), which measures Trust, Transparency, Explainability, Human Control, Adaptability, Formal Fairness, and Substantive Fairness. To assess organizational HCAI capabilities, Human-Centered Maturity Assessment Instrument for AI (HuMAI) was developed and validated with experts. The instrument operationalizes HCAI across ten topics and assesses each topic separately for pre-deployment and post-deployment using explicitly cumulative maturity levels (0-5). By integrating the HuCAS and HuMAI, practitioners can identify impact gaps between organizational design intent and end-user perception, track lifecycle drift from pre-deployment to post-deployment, and diagnose the extent of alignment with critical HCAI socio-technical dimensions. This paper describes the concepts behind the HuCAS questionnaire and the HuMAI maturity assessment method as well as their construction. Knowledge concerning reliability, validity and practical applicability of these instrument must be collected in real industrial contexts and are no part of this paper.
url
Science-Based Methodologies for Evaluating the Human-Centeredness of AI SystemsView
Preprint link to article Open CC BY-NC-ND V4.0

Related links

Details

Logo image