Making the Grade: Measuring Grading Rigor and Bias in Schools

In many instances academic success is measured by a student’s final grade. Thus, it is important that teachers administer grades consistently and fairly to ensure that the grade accurately reflects the student’s performance.

Not all educational institutions have systems or mechanisms in place to detect grading bias or a deviation from grading standards, however, school administrators, teachers, parents, and students have an interest in ensuring grading rigor.

This project detects grading bias and deviations from grading rigor in student records by applying two novel metrics to school records.

We assume that school records, including demographic information about the student, coupled with standardized test results will produce a distribution of teacher grading rigor. That is, the vast majority of teacher’s grading practices will be consistent and sufficiently rigorous, while there will be a small minority that are grading too hard or too easy. Moreover, we assume from student records alone, we will be able to detect indicators of grading bias or bias in course selection in some instances.

Hook: Why are students below grade-level in standardized state math tests when their passing grades in their previous classes indicate student success?

Larger Problem: Is the high school educating students with enough “rigor”? Are teachers just providing students with an education on paper but not in reality? Are certain students getting a better education while others at the same school are systematically left behind?

Limiting Scope: This is a much larger problem that cannot be solved with one project. So to start this endeavor, let’s focus on some key questions. Are teachers consistently grading on the same scale? Are some teachers giving grades at a lower bar than others? Are there discernable patterns in which students’ grades vary?

How can measuring teacher grading rigor help inform administrators provide actionable insights about teachers’ and students’ performance in public education?

We created two novel metrics, Differences in Distributions Metric that determines if there are unique characteristics of the teachers or the students in each quadrant. Use Earth Mover’s Distance (EMD) to quantify differences in learning outcomes; and Rigor Metric that determines if there is a relationship between GPA and standardized test scores for the school. Use correlation and additional data to group teachers into quadrants.

A clear understanding of students’ performance and mastery of skills depends on the teachers’ rigor in grading and can be measured using the relation between students’ grades and standardized test scores. Are schools measuring how teachers are grading in a consistent and standardized way? Is it possible to identify any biases that may have an effect? This set of tools allows schools to compare objectively the different grading styles between teachers, different outcomes for students, and different trends in courses. In this way, you are empowered to better target interventions to improve students’ performance and track how teachers improve instructional rigor over time in an objective manner.

For more information, please visit our website at http://capstonelearninganalytics.com.