For this CATS, I wanted to look at if there is any difference in test scores between students who quickly complete an online study tool prior to the exam and students who take hours or days to complete the same study tool. I gave my introductory Psychology students over a week to complete the online study tool (a Collaborate Learning Unit or “CLU,” name courtesy of Dr. Coleman) for each exam (data from 3 exams included in this analysis) and categorized them based on if they took less than an hour, from an hour to a day or more than one day to complete the study tool. I found a significant main effect of time spent on CLU and test grade, students who took less than an hour to complete the CLU had the lowest scores on the exam (M=64.38, SD=16.67), followed by the students who took less than a day (M=75.13, SD=11.88) and the students who took more than a day had the highest test scores (M=76.57, SD=10.12), F (2, 78) = 6.33, p<.01. I will show my students this finding and encourage them to take lots of time to study for the next two exams we have in our course, it could increase their score by an entire letter grade!
| Attachment | Size |
|---|---|
| catsstudytimetestscores.docx | 32.91 KB |
Comments
Great assessment, Erica. Was the amount of time that they spent studying self-reported?
In reply to Great assessment Erica Was the amount of time that by Norma Jimenez Hernandez
Thank you! Good question, the time that they spent on the assignment is recorded in CANVAS automatically because it is an online assignment.
I love your findings from this, Erica! It certainly gives me cause for thought as I modify my online courses. Would it be possible to get a copy of your online study tool exam? You could post it to CATS.
In reply to I love your findings from this Erica It certainly gives by Peter Turner
Good point, Pete! I will upload that shortly.
I would also be interested in obtaining a copy of your online study tool exam. Great idea!
Erica,
This is a powerful study to show students how to be successful in all courses!
Thank you for the detailed stats on the study!
Smiles,
Olga
This is very interesting. I have wondered, does Canvas record time if the student starts, then stops and starts again? I know eventually one must re log in but didn't know if some of the time was actually idle time. This is excellent Erica.
In reply to This is very interesting I have wondered does Canvas by Bronwen Steele
Yes, that's a good point! I believe it does record the entire time (includeing idle time). That is certainly a flaw in this analysis, but I do think even knowing when students opened it and even started looking at it is a good indicator for "study time" in some ways, as many students did not open it until the day of the exam (they are due before exams on exam day).
When I started reading, I was going to assume that the students that took less time would do better. My reasoning is that they know it well so they could go through it quicker. Do you know the average time spent in the "less than a day" category?
In reply to When I started reading I was going to assume that the by Sarah Lockhart
Great question! I would have to go back and look at the original data again (I recorded it from CANVAS). From memory, most of the people in the less than a day category were taking 2-3 hours. And it's important to note, there is not necessarily a difference between people who took less than a day and people who took more than a day (75% on the test vs. 76% on the test), I would actually predict there is no difference between those two. So you're right that maybe many people who know the material well only needed an hour or two on it and they would still be put into the less than a day but more than an hour category.
Thank you for posting this, Erica. A next step might be to require students to complete the CLU a day before the exam and for a certain amount of time. What do you think?
njh