My PhD
Title:
Developing Novel Machine Learning Techniques to Improve Comparative Judgements for e-Learning and e-Assessment
External Stakeholder:
CDSM Interactive Solutions Ltd
The Research:
Automated assessment tools are a popular and growing part of teaching practices due to their speed and scalability. The tools are well suited to assessing quantifiable work and can also assess qualitative work in constrained exercises such as multiple-choice questions.
Freeform qualitative assessment also has a vital part in the development of a learner’s skills, particularly in regard to creative problem solving, and developing the softer skills that are widely sought by employers. Unfortunately, assessing freeform qualitative work is much less easily automated and current approaches rely on poor natural language processing or crude keyword searching. This makes already time-consuming qualitative assessment even less attractive to educators and, in the long term, this trend risks driving a reversion to rote learning.
We aim to redress the imbalance caused by automated marking tools promoting specific approaches to assessment. We will develop a decision support tool for educators assessing qualitative work that will make such assessments more attractive by increasing the educators understanding of the student cohorts’ work and, potentially, reducing the amount of time they need to spend marking it. A comparative judgement framework [1] will be developed to allow educators to understand how changes in practice between different cohorts’ impact on assessments. For example, the tool might show if a cohorts essays perform better against a key learning outcome of comparing and contrasting various sources after the educator introduces a tutorial focusing on this topic. If the comparative judgement framework can show reliable performance at the cohort level, we will explore how it can be applied to individual qualitative assignments to support educators’ assessments of them. For example, drawing attention to specific parts of an essay that might demonstrate a key learning outcome of the assessment.
To deliver this exciting project, the student will tackle several key scientific challenges, each of which has the potential to result in significant research contributions.