By Dr Mark Ian Jones (UNSW Art & Design) | Published 5 December 2024
The author is a #UNSWNexus Fellow – learn more about the program here.
The big disruption
In 2023, there was a sharp rise in the number of academic-integrity referrals across the sector because of confusion around the use of AI. A year later, not much has changed despite the introduction of AI guidelines, and there is an urgent need to rethink assessment to counter punitive approaches and acknowledge what AI can offer.
There are pockets of innovation in individual courses in response to AI, but without a strategic overarching approach the student (and teacher) experience will be confusing, and there will be dangers of misalignment. Making changes at a course level may be an effective “stop gap” measure in the use and misuse of AI tools; however, a programmatic approach may be the answer to assure assessment security and the acquisition and construction of knowledge through a systematic approach to modular and longitudinal assessment (van der Vleuten, Lindemann & Schmidt, 2018).
AI has transformative potential as a learning tool, and AI literacy is now an essential graduate skill. AI is here to stay and is a valuable and much used tool in our creative industries. There is an urgency in developing program-level strategies to moderate, manage and develop literacy in the use of these tools to the benefit of our creative graduates and educators. As we approach 2025, uneven levels of AI awareness and fluency remain.
We must embrace the potential of AI while ensuring equity through assessment reform that maintains and enhances key learning outcomes for creativity.
A helicopter view
The School of Art and Design’s flagship undergraduate programs, the Bachelor of Design and Bachelor of Fine Arts, were re-launched in 2019, designed from the ground up with a programmatic approach to learning. Despite the impacts of “on the fly” COVID-19-driven changes and recent program revisions, the core relationships within the programs mostly remain.
Building on this solid foundation, our 2024 Nexus School Project, “AI in Creative Contexts”, sought to understand how we might integrate AI into our programs as another component of our creative toolkit. Central to this understanding was getting the balance right by ensuring that our students understand the potential for and ethics of AI in their practice while assuring the development of their individual creativity. To achieve this, we developed a programmatic approach to AI literacy and security. Our aim was to develop an overarching approach and toolkit to facilitate and support colleagues in rethinking the place of AI in creative contexts (theory and studio-based courses) in response to findings; and to share the outcomes across UNSW through Nexus to allied disciplines.
How was it done?
The project mapped the two programs consisting of 57 courses (core and disciplinary), 225+ learning outcomes and 200+ assessment tasks, analysing the impacts of generative AI (GenAI) and large language models (LLMs) on curricula.
Considerations of levels of supervision and assurance of process were central to the analysis to identify opportunities to embed AI literacy into curricula while securing vulnerabilities. Key principles were understanding assessment types and determining the extent of in-class and documented processes that would assure authorship.
Levels of impact were assigned to courses and their constituent parts (course learning outcomes and assessments): low to moderate, high and extreme. The qualitative analysis was undertaken by Faculty of Arts, Design and Architecture (ADA) educational developers exercising expert judgement, further leveraging their disciplinary expertise in creative contexts, and correlated by the project lead. The results were then visualised in heat maps identifying opportunities and vulnerabilities. Using sequentially darker colours to visually represent the level of impact, the heat maps further acted as a health check by identifying clustering and programmatic achievement of learning through a range of assessment types (Figure 1). Vulnerable assessments are being stress-tested through various AI platforms to confirm vulnerabilities of and opportunities for innovation, with a focus on process, a strength of our disciplines. The results will be workshopped with course convenors in a pilot in 2025 to establish a collaborative direction for educational-developer-supported assessment reform. The advantage of this heat-mapping approach is the collaborative way in which assessment is assured and enhanced while AI literacy is scaffolded via a programmatic lens.
A scalable model?
Heat-mapping is a visually effective way to identify courses requiring reform while illustrating overall assessment assurance. The project is being assessed as one of several models that could be used to deliver the UNSW AI Action Plan in 2025. While bringing a timely and actionable whole-of-program approach to AI in creative contexts, the project further highlights the potential and benefits of the UNSW Nexus Program to bring meaningful and effective local initiatives to institutional endeavours.
To what extent are other Faculties and Institutions addressing AI through a programmatic lens?
A note from the author: I would like to acknowledge ADA Educational Developers Jillian James and David Lawrey for their work and contributions to the project, and Prof. Alex Steel, Dr Cherie Lucas, Dr Benjamin Phipps and David Lawrey for their peer review.
***
Reading this on a mobile? Scroll down to learn about the author.