Metrics
2 File views/ downloads
8 Record Views
Dissertation
Optimizing Data Collection Processes for Operational Military Education: A Performance Improvement Approach
University of West Florida Libraries
Doctor of Education (EDD), University of West Florida
2025
Abstract
This study explores and addresses a performance issue at an operational military education institution. Currently, the school faces significant challenges employing data-informed
approaches to curriculum development to validate instructional outcomes. Data-driven decision
making is a command priority, yet limited data hinders the school’s ability to clearly articulate
program outcomes to senior leadership. Without actionable insights, the school risks missed
opportunities for evidence-based curriculum improvements and to demonstrate its impact on
mission readiness. Addressing this issue is essential for ensuring informed decision-making,
enhancing instructional quality, and maintaining the school's reputation as a leader in agile, high
impact education for the force.
The study leverages action research as a strategic approach to identify challenges, test
interventions, and refine practices through iterative cycles of data collection and reflection. This
problem-oriented approach is particularly relevant in a dynamic military education setting where
complex problems require context-specific solutions. The Human Performance Technology
(HPT) model served as the conceptual framework for analyzing the root causes of limited data,
designing interventions, and implementing data-driven, actionable, and measurable solutions.
Both action research and the HPT model emphasize stakeholder participation to ensure that those
most affected by the issue contribute to understanding the problem and shaping solutions.
Integrating qualitative analysis with the Human Performance Technology (HPT) model provided an in-depth, contextual understanding of the problem. Qualitative methods are
particularly effective for exploring underlying meanings and patterns in relationships (Babbie,
2021). This method enabled the collection and interpretation of narrative and visual data,
uncovering organizational, cultural, and environmental factors that influence human performance
(Bloomberg, 2022). Rather than seeking to generalize findings, this study aimed to generate rich,
descriptive insights into the school’s processes, identifying areas for improvement specific to its
context. The school is an agile military institution that adapts its educational products to meet
contemporary operational challenges. Unlike more traditional professional military education
institutions, which follow standardized curricula, the school maintains flexible learning
processes. This adaptability necessitates a tailored approach to performance improvement,
reinforcing the need for qualitative research to capture nuanced perspectives on existing
processes, inefficiencies, and opportunities for enhancement (Bloomberg, 2022).
Data was triangulated from multiple perspectives to identify organizational strengths and weaknesses. Additionally, comprehensive questionnaires captured diverse viewpoints during the
gap and cause analysis phases of the HPT model. This approach facilitated an in-depth
examination of processes and workflows, shedding light on how work is actually performed and
where improvements can be made. Performance analysis findings revealed critical gaps in
courseware management, assessment, and instructional design. The absence of structured,
codified processes for tracking course modifications impairs the school’s ability to make data
driven decisions. Without defined roles, responsibilities, workflows, and centralized storage, any
new assessment tools will likely be ineffective in capturing accurate data or supporting informed
decision-making. Further, without reliable data, instructional improvements rely on assumptions
rather than empirical evidence, limiting the school’s capacity for continuous improvement.
The root causes of the problem point to four underlying issues that contribute to this
problem. The first challenge is cultural resistance to standardized instructional systems design
practice. Secondly, the unclear division of responsibilities between Instructional Systems
Designers (ISD) and cadre leads to inefficiencies in courseware management and evaluation. The third challenge, disorganized storage and inconsistent data collection, hinders instructional
continuity and evaluation. Finally, a lack of performance metrics and associated governance
structures leads to inconsistent adherence to instructional design standards.
Intervention selection followed a structured, collaborative process involving key
stakeholders. A targeted solution set was developed, a Courseware Management Program (CMP)
comprised of performance support and learning interventions. The CMP establishes a centralized
courseware management system (CMS), supported by structured documentation, formalized
workflows, and role definitions. It provides a standardized and accessible repository for
instructional content, ensuring consistent practices across departments and enhancing
transparency, efficiency, and curriculum continuity. The accompanying Data Management &
Storage Standards ensure reliable data collection and use, improving the accuracy of
instructional evaluations and enabling evidence-based decision-making. The CMP aims to
standardize courseware development, enhance data governance, and embed systematic
instructional design practices based on the ADDIE model.
Addressing all these challenges simultaneously would be difficult for a military school with limited resources that concurrently supports broader missions. Implementation of the
Courseware Management Program (CMP) followed a phased approach designed to prioritize
foundational changes and generate early successes to build momentum. Initial formative
evaluation efforts informed meaningful refinements to the CMP and yielded measurable
indicators of its success. The phased implementation continues, with summative and
confirmative evaluations scheduled to assess long-term impact and sustainability.
Details
- Title
- Optimizing Data Collection Processes for Operational Military Education
- Resource Type
- Dissertation
- Contributors
- Charles Charlton (Committee Member)Michael Grub (Committee Member)Holley Handley (Committee Co-Chair)Christina Parker (Committee Member)
- Publisher
- University of West Florida Libraries; Argo Scholar Commons
- Format
- pdf
- Number of pages
- 159
- Copyright
- Unpublished work © Jessica Spring Leitheiser
- Identifiers
- 99381559028606600
- Academic Unit
- School of Education
- Language
- English
- Awarding Institution
- University of West Florida; Doctor of Education (EDD)
- Theses and Dissertations
- Doctor of Education (EDD), University of West Florida