Applications of diagnostic models to learning progressions

This organized discussion brings together leading voices from academia and industry to explore the
latest research and trends in diagnostic measurement (DM) that are both theoretically robust and
practically relevant in today’s changing educational settings. The session will showcase cutting-edge
models and techniques, such as learning progressions, Diagnostic Classification Models (DCMs),
artificial intelligence (AI), and Structured Mixture Item Response Theory (SMIRT) models, all
aligned with emerging educational priorities. These include supporting empowered learning in the
formative assessment process, improving the precision and insights of diagnostic assessments, and
enhancing instructional decision-making through validated learning progressions and diagnostic
feedback at multiple levels. The discussion will emphasize actionable solutions, highlight practical
challenges and successes, and encourage audience engagement in exploring how DM can be most
effectively utilized. By integrating diverse perspectives from both researchers and practitioners, this
session aims to advance the understanding and application of DM to better serve society’s
educational needs, with a focus on practical applications and societal implications. The interactive
format is designed to ensure that insights shared are directly applicable, fostering a collaborative
vision for the future of DM.

Transcript

1. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS Accessible Teaching, Learning, and Assessment Systems (ATLAS) Applications of diagnostic models to learning progressions W. Jake Thompson
2. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS Learning progressions
3. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS What do we mean by learning progression? • Unique and discreet knowledge, skills, and understandings (KSUs) • Organized in acquisition order • Precursor skills must be acquired before successor skills • In network language, precursors are “parents” and successors are “children” 4/24/2025 3
4. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS Learning progression structures 4/24/2025 4 Linear Diverging Converging Diamond
5. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS Linking learning progressions to DCMs • Unique, discreet KSUs = attributes • Connections between attributes = profiles • Structure of the learning progression implies the exclusion of certain profiles 4/24/2025 5 Profile # KSU 1 KSU 2 KSU 3 1 0 0 0 2 1 0 0 3 0 1 0 4 0 0 1 5 1 1 0 6 1 0 1 7 0 1 1 8 1 1 1
6. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS Learning progressions with DCMs • Unique, discreet KSUs = attributes • Connections between attributes = profiles • Structure of the learning progression implies the exclusion of certain profiles 4/24/2025 6 Profile # KSU 1 KSU 2 KSU 3 1 0 0 0 2 1 0 0 3 0 1 0 4 0 0 1 5 1 1 0 6 1 0 1 7 0 1 1 8 1 1 1
7. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS DCMs as an evaluation tool • The absence of certain profiles is testable! • Model relationships between attributes, not just between items and attributes • Hierarchical diagnostic classification model (HDCM; Templin & Bradshaw, 2014) • As a Bayesian network (Hu & Templin, 2020) • Compare model fit • Saturated (all profiles) vs. constrained model(s) 4/24/2025 7
8. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS Applications of DCMs to learning progressions
9. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS U.S. Department of Education, Enhanced Assessment Grant (2016)
10. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS 4/24/2025 10
11. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS I-SMART learning map models • Fine-grained learning maps models that define knowledge, skills, and understandings in science • Sample nodes from the larger map for assessment 4/24/2025 11 ATLAS (2021)
12. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS
13. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS 4/24/2025 13
14. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS 4/24/2025 14
15. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS Dynamic Learning Maps linkage levels • Accountability assessment used in 22 states • Linkage levels follow a linear hierarchy • Diagnostic models in DLM • Scoring and reporting • Evaluating content structures Thompson & Nash (2022) 4/24/2025 15
16. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS U.S. Department of Education, Competitive Grants for State Assessments (2022)
17. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS PIE Learning Pathways • Teachers create groups of standards for instruction and assessment • Real time results provided with opportunities to retest levels • Levels within standards form a linear hierarchy • HDCM used for scoring 4/24/2025 17 PIE.5.NF.A.3 Level 1 Compare unit fractions. Level 2 Compare fractions with like denominators and like decimals. Level 3 Compare and order fractions and/or decimals up to the thousandths place using symbols (<, >, =) and justify the solution.
18. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS ATLAS research fellowship • Application and proposal due May 16 • 2025 research priorities • AI in education • Diagnostic classification models • Other innovative research related to our mission and projects • Doctoral students from accredited institution in the US and Canada atlas.ku.edu/fellowship 4/24/2025 18
19. The UNIVERSITY of KANSAS The UNIVERSITY of KANSAS Thank you! • Evaluating learning pathway hierarchies in PIE • Today, 11:30am–1:00pm • Denver 5-6 • ATLAS research fellowship • Submit by May 16 • atlas.ku.edu/fellowship Get in touch!