Suggestion: Show Student Raw Contribution Scores

[Broadwc]

TEAMMATES has a great set of calculations and tools to try and strip out bias and other common issues. However, it can be challenging in an educational setting to follow the same kind of approach, but trial and develop slightly different methodologies. It can also create some challenges and ambiguity that can prevent uptake of TEAMMATES, where the integrity of the assessment can be challenged because an external examiner cannot see the original scores. For instance:

• Where group members are removed during a project but left in the TEAMMATES group to avoid losing their feedback to earlier sessions so that there is no change in data that could cause concerns for external examiners. In these cases scores often need readjusting to make sense.
• Where academics are exploring a non-zero-sum scoring approach
• Where a student says they scored a student 60% but their score on TEAMMATES is in the 80s.

I propose that for team contribution questions, the instructor view has a new column added to shown the unadjusted raw values from the students.

E.g. a new column is created after 'Ratings Received' called 'Original Ratings', showing the values that group members originally submitted.

This can be useful for trialling different methodologies and also for supporting the 'sell' on the TEAMMATES calculation approach. It also means that even in the case of academics disagreeing with one or more elements on the established methodology, they have the ability to use the raw scores and apply their own approach.

[damithc]

Thanks very much for the suggestion @Broadwc and sorry for the delay in responding. It's mainly because we needed to think about this first, as this is a non-trivial change, both in terms of implementation and impact on user experience (e.g., this question type is already hard to understand for most users; so, we don't want to confuse such users further).

Currently, the raw values can be found by downloading the submission results. Given raw values are needed only in rare situations, perhaps that's enough? What do you think?

[Broadwc]

Hi Damith,

Apologies for the extremely lengthy delay and absence in responding. Higher education is extremely busy in the UK right now and I've transitioned to a new role over the last year to further complicate matters!

I've explored the different options and also found a few challenges with my former approach, so I've charted a new way forward based on the zero-sum contribution questions. Thanks for your time, expertise and consideration on this, it has helped me to move forward!

I have an idea on how to implement an alternative methodology that I think would be a step forward, but it wouldn't be fair to ask before I can contribute some of my own time for the eventual project (if agreed by yourselves). I'll open a new thread at that time!

[damithc]

Sure @Broadwc We look forward to hearing from you.
Sorry, I thought I answered this but just realized I didn't.