Published on 19 December 2024
By Naveen Kumar, Faisal Khattak and Ee Kho (PVC Education Portfolio)
In the ever-evolving landscape of education, the ability to analyse and interpret data effectively is crucial for enhancing student learning and improving assessment strategies.
Earlier this year, colleagues at the UNSW Faculty of Medicine and Health (M&H) encountered challenges with their existing analytics tools and sought an alternative solution to address these limitations. The objective was to generate comprehensive analytics reports for their end-of-year phase exams, enabling detailed analysis of question discrimination and distractor effectiveness to inform teaching and meeting the accreditation requirements for the Medicine program. To find a solution, the faculty approached the Pro Vice-Chancellor, Education (PVCE) portfolio for assistance and support.
The aim was to develop a dashboard that would generate statistical reports, satisfy accreditation-body reporting requirements, and be user-friendly for assessments, while providing valuable insights.
In collaboration with the M&H team, we have created the “myExam Analytics Dashboard”. Developed using Microsoft Power BI Desktop, it is user-friendly, requires minimal support, and addresses M&H's comprehensive analytics needs. The dashboard empowers educators to refine assessments, identify student misconceptions, and improve teaching by offering detailed insights into the reliability, difficulty, and performance of the exam questions. It allows teachers to examine and analyse not just students’ answers, but how they engage with each question - for instance, how their scores relate to the difficulty of each question.
The M&H team reports that the tool has streamlined their reporting process and simplified the selection of high-quality questions for exams, including Point Biserial calculations. As collaborators we have all observed that the tool significantly reduces manual effort for large cohorts, enhances exam quality control, and provides valuable data to inform medical-education practices.
Dr Karen Gibson, Senior Lecturer from M&H noted, “The Inspera Analytics Power BI Dashboard has filled a gap. It now means that we can readily examine discrimination of individual questions and determine which distractors are useful. By looking at the distractors chosen we can uncover misconceptions which will inform future teaching.”
The team has also investigated the potential for developing a similar Power BI dashboard for Moodle-based assessments and found it promising, confirming that a comparable dashboard can be created using data from Moodle.
Based on our experience so far, we are confident that the 'myExam Analytics Dashboard' and similar tools can empower educators to refine assessments, uncover misconceptions, and enhance teaching through robust assessment analytics. By simplifying reporting and streamlining complex data analysis, the dashboard improves exam quality, saves time, and supports data-informed decision-making. Its success underscores the importance of innovative solutions in advancing assessment quality. Furthermore, exploring similar dashboards for Moodle-based assessments could further elevate assessment practices, ensuring that they remain effective, efficient, and aligned with the evolving needs of modern education.
If you’d like to explore replicating the dashboard for your own school or faculty needs, please contact the article’s authors.
***
Reading this on a mobile? Scroll down to read more about the authors.