Firas Moosvi, Hendrik Blok, Tiffany Potter, James Charbonneau, Letitia Englund, Andrew Gardener, Pan Luo, Ido Roll
Giving and receiving feedback has been identified as a key focus in redesigning assessments for long term learning (Boud, 2006) as peer feedback has been shown to improve motivation (White, 1998) and provide learning benefits (Jhangiani, 2016). However, challenges in implementation have prevented its widespread use. Leveraging the process of comparative judgement has the potential to facilitate peer feedback and assessment at scale. Performing an active comparative process facilitates students’ future learning by enabling them to go beyond identifying superficial features of a phenomenon (Bransford,1999; Schwartz,1998). Developing the capacity in students to be an effective ‘assessor of learning’ prepares students for lifelong learning by preparing them to make complex judgements about their own work, as well as that of their peers (Boud, 2006).
ComPAIR is an adaptive comparative judgement (ACJ)-based online learning tool that facilitates peer feedback at scale (Potter, 2017) and has recently been proposed as reliable in assessing complex items (Pollitt, 2012). Learners evaluate, comment upon, and rank pairs of submissions in many classroom contexts across disciplines (English, Mathematics, Integrated Sciences, and Physics courses for this study). In previous work, particularly in the English and Physics cohorts, students found the tool intuitive to use and reported very strong perceptions of improved ability and confidence in their ability to complete a similar task in future or to begin a more complex related task. Students perceive ComPAIR as an effective tool for teaching and learning (Potter, 2017). Others have shown that feedback from multiple peers led to more significant improvements in revised papers than feedback provided by experts (Cho, 2007).
In addition to the cognitive benefits of assessing peer assignments and providing feedback, ComPAIR may reduce marking load and increase consistency (Pollitt, 2012). English (ENGL110; 150 students) and Integrated Sciences (ISCI360; 44 students) were used to evaluate ComPAIR for grading. Each student in ENGL110 judged three pairs of submissions on one criterion and ISCI360 students assessed six pairs on three criteria. Expert judgements on the same submissions were also made in ComPAIR to evaluate the reliability of the tool. The assignments were also manually graded by two teaching assistants. Simulation data suggest that the effect of ‘misjudgements’ has a large impact tool reliability, especially with a limited amount of judgements. In this presentation we will share results of our study using ComPAIR as a tool for assessment, and provide evidence-based practices for using ComPAIR in the classroom.
