Duration
-
Abstract
Is there really a need to kill another tree if CANVAS provides a medium for quizzes? In a quasi-experimental design, introductory stats students across two semesters (SP and FA ‘16) with two different instructors self-reported the number of hours spent studying for a common cumulative quiz. One group used e-quizzes, the second used paper quizzes. Results are as follows:
- Marginally significant differences in the number of hours spent studying past quizzes with e-quizzers reporting slightly more time (M=3.76 hours, SD=4.88) than paper quizzers (M=2.71, SD=2.83), t(158)=1.72, p<.09;
- E-quizzers had significantly higher final grades (M=84%) than paper quizzers (M=79%), t(158)=2.35, p<.05;
- Significant moderate, positive correlation between number of hours spent studying for the quiz using weekly quizzes and the cumulative quiz (r=.266, N=160, p<.01), even after controlling for instructor effect (r=.205, p<.01);
- No significant changes in self-reported number of hours spent studying from Spring ’16 to Fall ’16.
Recommendation: GO GREEN and the mean may rise to the occasion
Division/Department
Completed Full Cycle
Yes
Course Number
PSY 230
Files
| Attachment | Size |
|---|---|
| catsapril2017wagerhernandezcorrelation.xlsx | 16.36 KB |
| catsapril2017wagerhernandezwriteup.docx | 78.24 KB |
| catsapril2017wagerhernandezspssoutput.docx | 20.26 KB |
Assessment of the Month
Comments
Our next step is to consider why we found these differences. For instance, are students able to study the e-quizzes more easily and that's what accounts for the difference? Perhaps they can look at them on their phones or tablets so they have easier access? We appreciate any feedback and are excited by our preliminary results! We will continue to test this with finals coming up shortly.
Erica and Norma, This is an excellent topic for discussion. I agree that the access for the e-quizzes makes a difference for access and quick reviews.. Students can look at them when they have five minutes here or there between family and work responsibilities. When my students do their video reflections, I see them in their cars, in the break room at work, in their children's schoolyard. I also wish we saw more statistical analysis in our CATS like you have provided here. Thank you for sharing your great work. Smiles, Olga
Erica and Norma, This is an excellent topic for discussion. I agree that the access for the e-quizzes makes a difference for access and quick reviews.. Students can look at them when they have five minutes here or there between family and work responsibilities. When my students do their video reflections, I see them in their cars, in the break room at work, in their children's schoolyard. I also wish we saw more statistical analysis in our CATS like you have provided here. Thank you for sharing your great work. Smiles, Olga
Great job Erica and Norma!
I use Canvas, and this is great insight for my student's learning. It's interesting that e-quizzers had higher final grades, than paper quizzers. I teach study skills, and I am very interested in the correlation between the number of hours spent studying for a quiz and the student's culumative quiz with e-quizzers. We will have to meet for coffee soon to share your best practices. :) Great information!
Catherine
Thank you Olga and Catherine! Yes the accssibility of CANVAS and class items online could be a help for in-class students as well as online students. Yes we were interested in how hours studied could predict grade, I actually looked at that in another CATS as well where I have students do an online study guide and I can track when they start the study guide and when they turn it in. In that sense, I can quantify (in a way) the time they spent on it. I found that there is a significant relationship between how long they spend on the study guide and their grade. The limitation to this study is that the study guide is online, so they could have opened it two days early but not looked at it again until 10 minutes before it was due. In this case, their data would say they worked on it for over a day but in reality they may have spent less on it. I am still trying to figure out how to quanitfy study time as best as possible. Norma and I found that the students seem pretty honest when we just ask them to self-report so perhaps I will continue to do that!
I did run a simple regression and one controlling for instructor and found that for every extra hour spent studying the regression model predicts .7 increase in final score. I'm having a difficult time thinking about how to encourage my students to study more if we see less than a one point increase for an additional hour of study time. I'm hypothesizing that perhaps they are not studying efficiently. I'm assuming that they know how to study but perhaps they don't. #perplexed
Thx
Erica and Norma, this is another great example of the full-cycle work you are doing to improve student learning, and documenting it in CATS! That it aligns with one of our core values - sustainablity, only makes it that much better. Great job!
In reply to Erica and Norma this is another great example of the full by Peter Turner
Yes, we were most excited about bringing in the sustainability piece! The only reason to argue for paper tests/quizzes was the idea that students would access them more often because they didn't need an internet connection. Thus, even not finding a significant difference would still result in pushing for e-quizzes because of the sustainability piece.
So, I have never done an e-quiz. I'd like to meet with either you or Norma to look more into this. Students are always struggling with math, so this may be something that we try in our courses.
In reply to So I have never done an e quiz I d like to meet with by Becky Baranowski
Yes I'd be happy to go over it with you! They are still done in class with me there but they are done on the computers. And in our stats classes there is often math involved with many of the questions so it should be doable with a math class as well!
Excellent study. I hope more faculty try this experiement to determine if online quizes have similar affects in other disciplines. It might also be good to collect some qualitative data from students by allowing both groups to take two quizzes (one with paper and one online.). You could share the data with students and ask them to explain why they think the online quize is resulting in higher scores.
Thank you! That's an excellent idea. We could try that this semester actually since we have yet to give them the common final.
I am definitely interested in seeing more data from future experiences. I have used e-quizzed but they were through MathAS and often completed outside of class time. My concern was students who were earning 100% on the e-quizzes but then couldn't understand the material in class which hinted at them using notes or receiving assistance so have pulled back from that option. Due to sustainability, I look forward to more ideas around this topic.
My concern always with online quizzes is the students figuring out a way to cheat. My online quizzes are through MathAS as well like Teri so it is out of class. I think it would be something beneficial for our courses.
Hello Teri and Jennifer, those are excellent points! Have you tried using lockdown browser? While I do my e-quizzes in class, so there's no more chance of cheating than paper quizzes, you can use lockdown browser for students who take their quizzes from other places on campus or from home. It's an APP in CANVAS that I have also used before. I know the CTL has offered courses in it to understand how it works (I only use it in class so I'm less clear on how it works outside of class but I know it can be used for that purpose).
Thank you Erica. I will research that lockdown option more. I do a small syllabus quiz in Canvas at the beginning of the semester so will try practicing with that :-)
In reply to Thank you Erica I will research that lockdown option by Teri Graham
That would be a great time to practice! I don't even make it required for the quizzes anymore, I just have students use that browser instead when they're in class.
Way out of the box here, but do you think that students are less "intimidated" by an online quiz (something about its ephemeral nature, or our comfort/familiarity with cheesy FaceBook quizzes) compared to a paper quiz (which might seem more "serious" or weighty)? Could those nerves or attitudes impact a student's performance?
In reply to Way out of the box here but do you think that students by Rachel Smith
That's a great point, they do seem to respond well to taking the quizzes in class online, I do it in all of my classes. There could be a way we could measure for that, actually! We could ask students after a paper quiz and after an online quiz how nervous they are and see if there are any differences. Good idea!