August 27, 2026
Learning Analytics Dashboards (LADs) are designed to help students understand their learning by visualising information such as progress, performance, and engagement. Yet simply showing students more data does not necessarily mean that they will reflect on it or use it to guide their learning. Research on LADs has shown mixed effects on learning outcomes, and many dashboards remain largely one-way information displays rather than tools that actively engage learners with their own data.
A research team from the Technical University of Munich, Laura Graf, Patrick Bassner, Maximilian Anzinger, Felix Dietrich, Stephan Krusche, and Oleksandra Poquet, explored whether learning dashboards could be redesigned as interactive pedagogical tools. Their study investigated an Interactive Learning Analytics Dashboard (ILAD) that combines an LLM-powered pedagogical agent with a self-assessment feature designed to encourage students to think about their learning before seeing the system’s evaluation.
The Study
The researchers embedded the ILAD into an existing learning management system used in an introductory algorithms and data structures course. Thirty computer science students participated over five weeks and were randomly assigned to one of three conditions:
- No agent: Students used the dashboard and its self-assessment feature without a pedagogical agent.
- Tell: An LLM-powered agent provided students with information about their learning data.
- Elicit: An LLM-powered agent primarily asked students questions about their learning data to encourage them to interpret and reflect on it themselves.
All students had access to the dashboard’s Judgement of Learning (JoL) feature. Rather than immediately displaying the system’s estimate of a student’s mastery of a competency, the dashboard first asked students to assess their own mastery on a five-point scale. Only after making this judgement could students see the system-generated estimates of their progress and mastery.
The idea was straightforward: instead of merely exposing students to learning analytics, the system would ask them to do something with the information. The pedagogical agent supported dialogue around the data, while the JoL feature created an opportunity for students to compare their own perception of learning with the system’s assessment.
Key Findings
Asking questions encouraged more reflection
Students interacted differently with the two versions of the pedagogical agent.
With the eliciting agent, students produced more messages involving reflection on their previous learning activities and planning of future study. With the telling agent, students more frequently asked for clarification or organisational information, suggesting that they tended to treat the agent more as an information source. The difference in the proportion of reflective interactions between the eliciting and telling conditions was statistically significant (z = 2.47, p = .013).
This distinction is important. The same underlying LLM technology can support very different forms of engagement depending on its pedagogical role. An agent that simply tells students what their data mean may make information easier to access, while an agent that asks students to interpret that information can encourage them to think about their own learning process.
Students became more consistent in judging their learning
Across the study, students’ Judgements of Learning became more consistent. Early in the intervention, students initially tended to underestimate their mastery, then shifted towards some overestimation, and later became more closely aligned with the system’s assessment.
The strongest pattern appeared in the elicit condition. Students interacting with the eliciting agent showed significant correlations between their own judgements and the system’s measures of confidence, progress, and mastery. For mastery, the correlation reached r = .408 (p < .001). Over the three study periods, the relationship between students’ self-assessments and system mastery estimates also increased, reaching r = .482 (p < .001) in the final period.
The other conditions showed less consistent patterns. The telling condition did not show significant overall correlations with the system metrics, although its correlation with mastery became positive and significant in the final period. The no-agent condition began with stronger alignment but declined over time. Given the small sample, the authors treat these results as exploratory trends rather than evidence of definitive differences between the conditions.
Self-assessment created a reason to engage with the data
The design of the JoL feature also appeared to motivate engagement. 83% of participants said they submitted a self-assessment because they were curious to see how the system rated them.
Interestingly, 70% reported that they did not actively pay attention to the dashboard’s learning analytics graphs while making their judgement. This suggests that the reflective interaction itself, i.e. making a judgement and subsequently comparing it with system feedback, may be an important component of dashboard engagement rather than the visualisation alone.
Students also reported that both interactive features prompted reflection: 78% said the LLM feature made them reflect and 68% said the JoL feature did so. Furthermore, 79% reported changing their behaviour after judging their learning, compared with 47% who reported adapting their behaviour following interactions with the agent.
What this Means for Practice
The study suggests that the question for learning dashboard design should not only be what data to show students, but also what students are prompted to do with those data.
For educators and developers, several implications emerge:
- Design for elicitation, not only explanation. LLM-based agents can ask learners to interpret patterns, make predictions, explain discrepancies, or reflect on their progress rather than simply summarising dashboard information.
- Ask learners to judge before showing the answer. Temporarily withholding system-generated mastery estimates until students have assessed themselves can turn a passive metric into a metacognitive exercise.
- Integrate reflection into the learning process. JoL prompts could, for example, become part of weekly assignments, while dashboard-supported reflection and goal setting could be incorporated at the beginning of learning modules.
At the same time, LLM-based dashboard features require careful implementation. In an evaluation of 284 sampled agent responses, 13% were classified as faulty, meaning that they did not fully match the request or contained false or misleading claims. The authors therefore emphasise the need for monitoring, validation, and iterative improvement, alongside consideration of computational costs, student welfare, privacy, and educational equity.
Limitations
The findings should be interpreted as exploratory. The study included only 30 self-selected and financially compensated participants, which limits generalisability. The five-week deployment also cannot establish whether improved calibration persists over longer periods or transfers to other courses.
Moreover, the study focused primarily on indicators of metacognitive engagement rather than direct learning outcomes. It did not measure whether students subsequently changed observable study behaviour or achieved better exam performance. Larger and longer-term studies will therefore be needed to determine whether the observed reflection and improved self-assessment ultimately translate into sustained changes in learning.
Takeaway
Learning dashboards may become more valuable when they stop functioning only as windows onto learning data and begin acting as tools for thinking with that data.
This study provides preliminary evidence that pedagogically designed interactivity can support that shift. In particular, asking students questions about their learning and requiring them to assess themselves before seeing system metrics were associated with greater reflection and increasingly accurate judgements of learning. Rather than using LLMs simply to make dashboards more conversational, their value may lie in enabling carefully designed interactions that encourage learners to pause, interpret, and reflect.
References
- Graf, L., Bassner, P., Anzinger, M., Dietrich, F., Krusche, S., & Poquet, O. (2026). Interactive learning dashboards: rethinking learning visualisations as engagement tools. Education and Information Technologies. https://doi.org/10.1007/s10639-026-14082-1
