05 November 2012

It's Not a Race! Or Is It?

I recently gave a biology exam to twelve students, and I was alarmed at the pace they set in completing it.  Actually, my students surprised me in multiple ways, and here I’ll share the experience and a vague recap of the results to show you what I mean.

This was the second exam of the semester, and a few students who were particularly fast on the first exam again finished the earliest.  In fact, by simple observation of the extreme cases the relative speed at which students worked was fairly similar.  The fastest workers were the same, the slowest workers were the same in either case.

On the other hand, the absolute speed at which students worked was startling.  Based on the extremes again, the fastest students were done in just 10 minutes while the last student still finished in under 40 minutes.  I had students working for up to 80 minutes on the first exam, so the pace of the class overall was startling.  Though my observations are driven by the extreme cases for clarity, the students in-between submitted their exams at a relatively steady pace (fairly balanced, overall).

I was curious to know if I had written an anomalously easy exam, and I even asked a student who finished surprisingly early, “Too easy?”  The response seemed to suggest that the exam had been just that.  This was early on (less than 20 minutes), and I kept the exams sorted in order of completion to see if there would be any difference between those who spent less than a minute with each question and those who took more time to think.

To my increasing surprise, though, exam after exam was submitted in fairly rapid succession.  There was a detectably confident air in the room as folks packed up to leave.  And none of my students spent more than a minute per question, on average (there were 50 questions total). Thus, Figure 1 includes the averages for 12 students who virtually all worked at a noticeably quicker pace than they had previously.  Recall that the first few students finished in about 10 minutes, the last few students finished in about 40 minutes. 


My initial instinct to separate the earliest from the latest finishers changed with added perspective.  Instead, as I look at Figure 1 I am more tempted to compare these results to those from the first exam.  The main thing I notice is that the average was actually lower on the second exam (66.5) than on the first exam (70).  This comes despite the speed and seeming confidence with which students approached the second exam.

Of course, how do you really interpret such a result?  Perhaps the additional 3.5 points on average would not be worth the same as the time saved for an average student.  In a case like this, it may be that speed doesn’t kill.  Instead, speed is just speed.

I know the dynamics of my surprise – first at the speed, then at the confidence, then at the results – which are so-called because I can’t discern any real correlation.  In fact, Figure 1 depicts scores on exams submitted within a half-hour window and shows no clear relationship between exam score and order (time, by proxy) of completion. What I would have expected, going in, would be an upside-down bell curve spanning a larger window of time (at least an hour between first and last submissions).  My prediction at 20 minutes into the exam would have been much the same, only with a narrowed range between the high and low scores (the confidence was not exclusive to star students).

I’m not left with much to conclude, though my main goal was less to diagnose anything and more to share my striking observations.  There may be some classroom psychology phenomenon at work - I'd be interested to know if that's the case.  All I really know is that it was an interesting and quick day in class, and a lot of things (obviously) grabbed my attention.  My guess is that any or all of the following factors were at play, in no particular order of importance:

                - Confidence as test-takers
                                - Especially among those who have already performed at a high level
                                - Cushion from previous performances allows for a few small mistakes
                - Overconfidence as test-takers
                                - Especially among those who have less experience/success in the sciences
                                - Recognizing/choosing related but incorrect answers without careful reading
                - Freedom to go home after the exam and impatience at doing so
                                - Consideration of weather, which was inclement and worsening
                                - Weariness at this later stage in the semester
                                - Less incentive to hurry on first exam, as a lecture was scheduled to follow
                - Perceived pressure to rush at seeing others complete the exam so quickly
                                - Perceived correlation between exam success and speed of completion
                - My wording/selection of questions
                                - Subjects more numerous/difficult than first exam
                                - Questions themselves perhaps less specific/difficult than first exam

                               

No comments:

Post a Comment