The process for selecting All-USA Scholarship recipients is: Step 1-Faculty committee scoring of essays; top 5 essays move forward to application process. 2013 scorers indicated they'd have preferred a more specific, helpful rubric to improve rater reliability. In 2014 it was replaced by the EMCC Writing Rubric which produced distinction between essays, however, scorers were dissatisfied with criteria that didn't account satisfactorily for content and voice. Thus, 6-Trait Writing Rubric will be used in 2015. Step 2-Essay semi-finalists submit applications to a 2nd committee for ranking. 2013 faculty ranking the applications experienced disparity, recommending a list of guidelines be developed for the ranking process. In 2014 a checklist was developed and provided to the faculty committee; all indicating it was helpful in ranking the applicants contrasted to the prior year. Rankings produced clear winners with reduced disparity. This is anecdotal evidence, as essay, application, and scoring data are confidential.
| Attachment | Size |
|---|---|
| considerations-ranking-allusambedit2.docx | 17.55 KB |
| original-essay-score-sheet.docx | 11.4 KB |
| emcc-writing-rubric-all-usa.docx | 13.59 KB |
| six-traits-writing-rubric.docx | 14.88 KB |
Comments
I am so happy to see a non teaching CATS! Thanks for writing this, Ro. After being on this scholarship committee for the last 8-10 years (I don't recall how long I have done this), I am happy to say that we are continuing to improve the process. It is one of the best opportunties I have to help make a difference in a student's life. I look forward to seeing what you and Marilyn do next year to help continue with this process. Thanks so much for doing this.
Thanks, Becky, for your positive feedback and encouragement. Even though it is a non-academic CATS, action research can be implemented at program levels. In the future, we will continue to provide the application ranking guidelines to the committee since it was successful in guiding the committee, but Marylyn and I look forward to seeing how using the 6-Trait Writing Rubric will impact (hopefully improve even more!) the process next year. Ro
Thank you for letting us know about your progress with this! I look forward to hearing how it goes from here on out.
Two thoughts on this:
First, I sincerely applaud that this is a a continual on-going process of reflection, monitoring and adjustment. It is truly a model of the action research cycle.
Second, I look forward to using the 6 Trait Writing Rubric. It will make the job of judging the entries much more clinical and objective.
Thank you for sharing! This process and accompanying rubrics will greatly help other campus scholarship committees.
Ro and Marylyn,
This is an excellent CATS form! Not only have you improved the process for the scholarships, you have also started a new way to use CATS. I hope other programs on campus will follow and think of assessment outside of the classroom. This can assist with getting other employee groups involved in CATS.
Smiles
Olga
It was a privilege to see this CATS in action as I used all of the tools. I found both the process and your description to be cogent, focused and a great learning experience for me. The bar has been raised and thanks to the both of you!
Having participated on this committee for the past 3 years, I can absolutely say that the process continues to improve. The rubric makes the process objective, and I applaud your efforts in this very worthwhile CATS!