AI Case Study - GenAI as "assisted format" for generating a research rationale

Abstract image of lightbulb, promotional image for AI in Education at UNSW

Published 3 November 2023

AI case study: GenAI as "assisted format" for generating a research rationale

The academic & their views on GenAI

Gee Chong Ling is an Education Focussed Lecturer in the School of Biotechnology and Biomolecular Science. With Gee’s experience in educational design and technical support, he is interested in the redesign of practical learning experiences through the integration of electronic laboratory notebooks into undergraduate courses to promote good documentation practice in our Science graduates.

Jai Tree is an experienced researcher and Associate Professor within the School of BABS, who convened the course for several years before the two joined forces in T1 2022. He is the mastermind in establishing the course project assessment in an authentic fashion, boosting both traditional and modern microbiology techniques, serving as a sophistically designed introductory learning lesson into the world of microbiology. The course project, which uses assessment as learning principles, has created one of the most engaging assessments for our Science graduates in the mid-phase of the degree program.

Gee sees AI pragmatically and thinks that its relationship with assessment is best understood and dealt with through the pedagogy of assessment as learning and authentic assessment. In the context of MICR2011, AI not only serves as a planner, but potentially as an assistant to guide students' learning in understanding the data generated from the experiment. If it is used correctly, this can produce a higher quality output in a much quicker format. However, as the results and analysis are dependent on samples collected in the real world, AI cannot provide students with the outcomes of the experiments and an answer.

Assessment overview

Course	MICR2011 – Microbiology 1
Assessment type	A scaffolded assessment which involves a mixture of scientific fieldwork/experimentation and reporting on the process in the form of Lab Reports. There are 4 key deliverables in the assessment process: Rationale [Group] Draft [Group] Draft Peer Review [Individual] Finale report [Individual]
Goal	Develop students' laboratory skills in microbiology, understanding of micro-organisms in their environment and the skills of scientific writing and research including evaluative judgment of past research.
Assessment focus	Report on a microbe isolated in physical environment and subsequently identify the microbe during the scaffolded course project. Identify appropriate methodology to use in the lab to evaluate the samples and make the necessary discoveries to develop the results section of a report. Give and receive feedback on draft report from peers, thereby improving scientific writing and evaluation skills.

Driving change factor

No changes were introduced, but students were asked to provide a reference list in the early phase (both group components) to prove that they were using the correct sources. The rationale for asking students to provide the reference list with screenshots aims to tackle student use of AI or crafted materials from AI, as compared to actual references of published materials. The additional input from the students also triggers us to comment on the quality of the referenced sources, creating a mutually advantageous outcome.

Student use of GenAI

Gee and Jai permit student use of AI in generating the rationale, as this means using AI to contribute to a potentially higher quality rationale compared to the past, much like an assisted format.

Strategies that minimised Generative AI responses

In Week 1 Jai gave students a 5-minute intro to AI generating a response. The teaching team then dissected the response together with the students. They observed that the first line was “great,” but closer inspection revealed that the references were not available and much of the information was made up. This activity demonstrated that there are risks associated with using AI as a tool, so the results it produces must be assessed rigorously.

The results of the assessment are heavily dependent on the physical samples taken (such as soil samples) and the idiosyncrasies of specific sites. The laboratory work process required also means it is a tangible and practice-based task in which student use of AI is somewhat restricted if it is going to be accurate. While AI can provide some direction, it cannot provide the necessary elements in the physical world to complete the task. However, if students can utilise the direction provided by AI to assist their critical analysis, the resulting outcome will be more detailed than in the pre-AI era, showcasing how AI can assist the analytical processes.

What did students learn?

Benefits	Students rate the experience of MICR2011 very highly, especially the structured and practical assessment design. They speak highly of the assessment's structured approach to group work and the opportunity to receive timely feedback on their work, including the inbuilt peer-feedback process on the first report draft.
Challenges	Some students find the open nature of the discovery and research process challenging because there are many possible pathways. The authentic assessment contributes to a level of 'uncertainty' reflecting true scientific development, which could unsettle students. Therefore, scaffolding is crucial in offering the support needed to guide student learning and knowledge development.
Solution	It is recommended to make use of the learning community to address the 'stress' coming from the 'uncertainty', therefore support as a learning community or group/team setting is essential. Also, paying attention to providing learning support will help, whether in the form of peers, demonstrators or academic learning support.

Student views about the assessment

“Peer review gave a lot of student insights on how they perceive a good paper. [… which] gave a lot more to think about.”

"I loved the staged submission of the report. The feedback in between really prepared me for the next stage and gave me a solid expectation of what to include in the final report.”

Further resources

UNSW educators on Generative AI in education (public page)
Internal resources for staff (UNSW staff only)
UNSW Guiding Principles (UNSW staff only)
Using Generative AI in your Learning, Teaching & Assessment (UNSW staff only)

Education & Student Experience