Neutral homework policies with discriminatory effects

Before the math-adjacent world forgets what homework is, I’d like to share how I found out that some of my homework policies were having unintended discriminatory effects.

A long time ago I was teaching a course at MIT in which the homework was difficult and factored significantly into the student grades. We allowed students to collaborate so long as they acknowledged all their collaborators. That year, I decided to examine the graph of who was collaborating with whom. I was curious what the collaboration network looked like and whether the structure of the network affected student grades.

The first thing I noticed about the network was that almost all of the isolated nodes--students who weren’t collaborating with anyone else--were women or minorities. This was clearly a problem and clearly more important than anything else I’d discover about the network. I didn’t know the precise social dynamics that caused these students to be isolated. But, I did know that they were getting less support from their peers. Their lack of collaborators made it more difficult for them to solve the homework problems. By allowing collaboration, I was disadvantaging the students who were more socially isolated in the class. Because of the social structure of that class, I was disadvantaging women and minorities!

When I moved to Yale and began teaching the Design and Analysis of Algorithms class, I wanted to avoid this problem. So I forbade collaboration on the problem sets. This policy had the advantage that it was fair, if it was followed. I conducted anonymous surveys that asked if students were following the policy and if they thought the other students were as well. For a long time they mostly did¹. The class was small at first, around 30 students, so I was able to establish a good relationship with the students, and I and my teaching staff could help out those students who were struggling.

I explained to the students the many advantages of working on their own. It forced them to really understand the material and experience the satisfaction and self-confidence boost that goes with solving hard problems. I pointed out that, if they were collaborating with someone who was just a little bit faster than they were, they’d be deprived of that satisfaction. And, they might even conclude that they weren’t good enough to pursue algorithms further².

Eight years later, enrollments in computer science courses skyrocketed. I could no longer establish a relationship with the students that would inspire honesty. And the number of students who needed help with the material greatly exceeded what my course staff could provide. We started to catch a lot of students cheating and had to spend a lot of time prosecuting them.

In the hopes that students would learn from each other and in recognition that many students were going to collaborate regardless of the rules, I changed my policy to allow collaboration. But this time, I was determined to try to minimize its discriminatory effects. I assumed that, because the class was so big, the women and minorities would be less isolated. It wasn’t so easy.

It soon became clear that I needed to limit the number of collaborators a student could have. Otherwise, big groups would solve the problems together. Not everyone was contributing. One time a solution found by one student percolated through a large fraction of the class. Analyzing this process carefully could have led to interesting network science. But, I needed a policy that would both allow and limit collaboration.

I tried a policy of letting groups of up to 4 students work together. It turns out this was a little bit like letting the students divide into sports teams without them knowing much about each other. The students who stereotypically looked like they’d be good at computer science had an easier time joining groups. I let the students change groups for each problem set. But, they didn’t take much advantage of this flexibility. I created a matching service to help isolated students find study groups. But, this didn’t help much either.

Then, a group of women in the class met with me to point out a flaw in my policy. They lived in dorms in which most of the suites housed 4 students. So, there were many suites containing 4 male students who were all taking my class and so formed a natural study group. While there were many women in the class³,
there were very few per dorm⁴, so many women were shut out of these natural study groups. Those who weren’t a part of these room-based study groups had to travel farther to meet their collaborators, and they had a much more difficult time scheduling these meetings. They had to work harder than their peers.
Changing the group size to 3 or 5 students didn’t solve the problem.

And, there were always going to be isolated students. Two Black women in the class explained intersectionality to me. There were students on varsity teams who had demanding schedules that rarely let them meet with others in the class. ROTC students had similar problems. There was no good solution.

The grade boost of collaborating on homework will disappear now that we will stop using homework to evaluate students. But I assume that variations of the problems I observed occur often and that I’m usually unaware of it. I will remember that policies that appear neutral can have discriminatory effects because of the context in which they are applied. The tendency of many to befriend people similar to themselves can create social networks that disadvantage those who are different. And, it’s rarely advantageous to be unique in a room.

1. I fondly remember statements from my course evals by students who reported being proud that they did not break the policy, despite living with other students in the class.

2. I also told them that I hoped they would take classes in which they did collaborate on hard problems and that some of the best friendships are made by doing difficult things together.

3. I apologize for treating gender as a binary. I’ll go with it for this discussion because it’s how the housing situation was explained to me.

4. Yale calls dorms “colleges”, and the students are randomly assigned to colleges in their first year.