Designing in Context: Reaching Beyond Usability in Learning Analytics Dashboard Design

June Ahn, Fabio Campos, Maria Hays, Daniela DiGiacombo
2019 Journal of Learning Analytics  
Researchers and developers of learning analytics (LA) systems are increasingly adopting human-centred design (HCD) approaches, with growing need to understand how to apply design practice in different educational settings. In this paper, we present a design narrative of our experience developing dashboards to support middle school mathematics teachers' pedagogical practices, in a multi-university, multi-school district, improvement science initiative in the United States. Through documentation
more » ... f our design experience, we offer ways to adapt common HCD methodscontextual design and design tensionswhen developing visual analytics systems for educators. We also illuminate how adopting these design methods within the context of improvement science and researchpractice partnerships fundamentally influences the design choices we make and the focal questions we undertake. The results of this design process flow naturally from the appropriation and repurposing of tools by district partners and directly inform improvement goals. Notes for Practice • Prior learning analytics work has used various techniques from human-centred design ranging from user-interviews to engaging practice partners in low-fidelity prototyping. A framework of design practice that is deeply embedded in partnerships with our intended users is needed. • Designing within the context of research-practice partnerships and improvement science initiatives helps dashboard designers balance the tensions between making interface-interaction decisions while ensuring that design aligns with existing work processes, data interpretation goals, and improvement aims. • Common data collection techniques, such as user interviews and think-alouds, can be structured and analyzed for insights into practitioners' data sensemaking needs in addition to usability analyses to inform interface or product changes. • The purpose of learning analytics design work should not be limited only to fidelity of implementation or adoption. Rather, an indicator of success is whether productive adaptations and local needs for LA tools can be embedded in the design itself. Partnership approaches offer unique advantages to achieving these design goals.
doi:10.18608/jla.2019.62.5 fatcat:bfuj2q6pcjhlfkalxm57wubkja