One student outcome for this online Intercultural COM course is "Continue to move forward into and/or through the ethnorelative stages of intercultural communication competency." The assessment is a self-created profile survey based upon the Milton J. Bennett, Ph.D., Dev. Model of Intercultural Sensitivity. Ethnocentric levels (ascending order) are 1 Defense, 2 Denial, 3 Minimization. Ethnorelative levels are 4 Acceptance, 5 Adaptation, 6 Integration. The survey identifies the student's level at Week 1, and re-identifies during the last week of class. The goal was to move more students into the higher levels. During Spring, readings and discussions were incorporated into the modules that focused on the 6 levels, prompting students to identify, analyze, and evaluate communication experiences according to each level. Results evidenced that incorporating the new practice produced a 34% increase in students achieving Level 5 (12% in the prior Fall). All results are attached below.
Comments
Very interesting, Ro. I am very much in favor of pre and post comparisons, and look forward to the end of the semester when said comparison takes place. I would like to see how the assessment is scored. What are the numbers to the left of each rating? Also, will you have different actions based upon student growth? For example, if you see little overall growth, will you do something differently next semester in this class?
Pete, I generally agree that the pre and post assessments are ideal because it shows clearly the "value-added" that the course has made. This is particularly true in a situation where students are expected to come in already at a certain level. It becomes harder and perhaps less relevent when the assessment is related directly to course content that most students may not have had exposure to before- at that point, the emphasis is really on how many students we can get passed the finished line at the end of the semester.
The scoring is done by counting the number of 1s, 2s, 3s, 4s, 5s, 6s. The highest number is the level at which the student scored. For example, if the student has more 4s than the other numbers, her/his level equates to the Ethnorelative stage of Acceptance of Difference (1=Denial, 2=Defense, 3=Minimization, 4=Acceptance, 5=Adaptation, 6=Integration).
Pre- and post- assessments are a great way to track growing competency. I'm guessing that students are more aware of the ethnorelative stages of intercultural communication at the end of the course than they are at the beginning. Is there any way to control for students "gaming" the post-assessment? That is, once they know what the stages are will they answer the questions honestly, or the way they think they're "supposed to"?
Ro, it might be interesting to display this information as a pie chart or bar chart, at the beginning of the semester and at the end, to see relative change. I am interested to see how you could use this information gathered to inform instructional changes in future semesters.
Great idea, Erik. I will have Pete help me create an Excel spreadsheet of Pre and Post. Maryanne, I have not worried about the students "gaming" the assessment, as there are no grades attached to their post results. They just get credit for participating. In my instructions I frame the activity as a way to demonstrate growth. And, if the students do guess which response is a higher level of competency, well, perhaps they learned something!
The results are posted, Erik. See the excel bar graph. I am excited to report that ALL students scored into the ethnorelative stages in the post-test!
The bar graph was a great addition. This was very interesting. I too am interested how you will use this to guide and transform your instruction in the future.
This is a great assessment. Thank you for sharing!