April 14, 2018

I want to start off with a quick anecdote, from about a week back. I had been working on a project for my computer security class, which tl;dr teaches you how to hack a 1980s computer before people were actually intelligent about how they treated their data. I spent maybe circa 10 hours on the project, not a significant amount by any means, but I passed all the tests with flying colors, and I certainly felt very proud of myself, as I donned my Neo shades ( Bryce, please goddamn watch The Matrix instead of The Duff for the fifth time). Imagine my surprise, when I received my project grades where I received about 50% on the project. It turns out that in my explanation of what I did, I only went over the techniques i used at a very high level, and thus lost points for not explicitly mentioning certain address, registers, and functions (which was not mentioned in the spec whatsoever!). I went on Piazza, and a number of other students too had received the same feedback, and the instructor's response:

The readers have specific things to look for, in order to try to have consistent grading from group to group.

Now, as a fellow instructor, this is what I like to call, the curse of grading: this idea that grading should be simple and consistent, but too easily turns into specificity and detail. Perhaps, another example from the same class will illustrate the point. This class, as do many Berkeley classes (as well as classes at many other universities, I'm sure), focuses on high level details and intuition in lecture, and for good reason: this style of teaching shows you how to think, not what to think. However, the very first homework assignment had students precisely compute stack memory addresses and offsets based on just 10 total lines of code, which is stupidly hard for all you non-CS majors out there. I believe this question is a sort of "easy way out". It's certainly simple to grade - just check if the memory addresses are right. It's also certainly consistent - again, just check the memory address. But, I think we can all agree that this doesn't really substantially test the student's knowledge of the concepts and why they work, but rather who put the entire process on their cheatsheet.

I think this second example hints at a more sinister part of the curse of grading: the idea that students need to be tested on every concept in the course, no matter the triviality. The algorithms class I took last semester was especially guilty of this - we went over the Fourier Transform for about a grand total of 10 minutes in class, without going into any of the super complex theory and detail behind the transform. If anything, we focused more on how to use the Fast Fourier Transform to perform polynomial multiplication - for those non-STEM people out there reading this, this is basically just using the FFT as a tool. However, on the exam, there was a question that asked students to compute a 16-length Fourier Transform of a sequence of numbers, which is basically absurdly mechanical and dependent on writing the equation on your cheatsheet. Simply because we went over the algorithm in class for 10 minutes doesn't mean that students actually have to get tested on it at such a crude level - if anything, just leave it out!

I kind of forgot what I wanted to write about after this. I'll make a new post if I can remember. I'd love to hear your thoughts on the topic though; please leave a comment below!