The purpose of this assessment is to analyze final course grades in CIS150AB (object-oriented programming) to see if replacing a traditional textbook with instructor-developed OER materials has a negative impact on student learning. Before expanding the approach to other coding classes, I want to ensure that I am not undermining students’ opportunities for success.
In Spring of 2025, I replaced the CIS150AB textbook with an eBook and accompanying videos that I had created, with two concerns in mind. One, I wanted to save students the expense of a textbook; and two, I wanted students to have materials more narrowly focused on their needs as occupational students.
Traditional programming textbooks tend to approach pedagogy from a computer science perspective, which is a little different from the needs of an occupational-focus course. Through conversations with students, reviewing quiz scores, and observing classroom performance on assignments, it seemed that students were either not completing or not understanding the assigned chapters as well as necessary.
Spring 2025 (and later) sections of CIS150AB were converted to the OER materials. Data analysis to compare final course grades among both online and hybrid sections to similar sections that used Java Programming by Joyce Farrell. Average quiz scores between the courses could also be useful, since those assess recall and understanding of core concepts, which are generally learned through the textbook readings.
NOTE: NotebookLM (an AI tool) was used to help analyze gradebook data.
Using the traditional textbook, there were an average of 9.3 students per semester who earned a final course grade of 90% or better (“top achievers”) and 4.0 students who earned 95% or better (“elite achievers”). Using OER materials, there were an average of 8.0 top achievers and 5.5 elite achievers. In summary, traditional textbook sections produced an average of 9.3 “A” students, while the OER sections produced an average of 8.0 “A” students.
With the traditional textbook, I saw an average of 1.66 students per semester complete the course with a final grade below 60%; using OER, I saw an average of 1.0 students per semester finishing below 60%. I also note that the lowest recorded OER grade (49.92%) is significantly higher than the lowest textbook grade (15.69%).
In the course, quizzes are used to assess basic recall and understanding of textbook/video material. Using the traditional textbook, there was a wider variance in scores, with a cluster in the 80-85% range, with low-scoring students clustering between 21.6% and 37.9%. In contrast, the OER courses resulted in a tighter clustering of high scores, with the majoring of students averaging 93-99% on the quizzes, and the low-scoring students saw a dramatic increase to an average of 85.5%. I’d like to think this is because the OER materials are “better,” but I think there are likely two contributing factors. One, the shorter chapter length and supplemental videos (which are both “shallower” than most textbooks), may have resulted in more students actually finishing the chapters. Two, it’s reasonable to assume the OER materials are better aligned with the quizzes; I picked the quiz questions, and I developed the OER materials.
The data has a lot of flaws, including small sample size and a lot of uncontrolled variables, but my conclusion is that there was not a decline in student performance after switching from the textbook to OER materials.
The rising costs of textbooks may be a barrier to student access, and most of those books are not well-suited for CIS150AB students. Traditional textbook authors approach coding from a Computer Science perspective, which doesn’t quite align with the occupational nature of CIS150AB. And while technical reading is a critical skill for professional programmers, the depth of these textbooks results in large chapters that may be intimidating to students.
To address these concerns, I developed an OER eBook and accompanying videos for CIS150AB, which I adopted in Spring of 2025. A major concern was whether or not the smaller, video-focused chapters would result in lower grades on the assessment projects, which were substantively unchanged from the textbook-centered sections. This CATS researched overall course grades, and objective quiz grades, to determine that the OER materials do not seem to decrease student achievement.