What’s happening now

AI systems are now incredibly good at solving math problems. They are so good that reasonable people expect them to become better at problem solving than all human mathematicians. There is plenty of evidence that they can solve many problems that have stumped us for decades. Our understanding of what they cannot solve, or rather cannot yet solve, is not as good1.

People using AI systems are making progress on a lot of important problems. Three of the problems I thought I could contemplate over the next few years have been solved in the last two weeks. I don’t know where I’ll find the time to read the solutions to these and whatever gets solved next week. At least, in the case of these problems, people have taken the time to write careful and helpful expositions of the results2.

Some other problems have not received the same care. A lot of people who expect fame to follow the solution of open problems have been feeding every conjecture they can find into LLMs. Often they release poorly written solutions whose correctness is uncertain. Some are being very systematic about this. There are people using agents to scrape the web for conjectures, feed these conjectures into other AI agents, and dump the results.

This is likely to hurt the field. To the extent that these papers are wrong, they make it very difficult for anyone to write a paper that is correct. They might mistakenly believe that a problem has been solved. Or, even if they doubt a posted (probably not published) solution, they will feel the need to find the mistake in it before they publish their own work. And, we don’t just want solutions to problems. We want understanding. But, the perceived incentive to write a good solution goes down a lot when a poorly written one is posted. In order to do the work to write a clean exposition of a result, most people need to believe that they will get some credit for it. They need some ownership of the problem. They have that if they’ve solved it themselves, but they won’t if everyone can see the posted slop and they don’t know how many other people are working to refine it at the same time.

Sometimes it feels like people are playing the LLM lottery. Everyone can ask LLMs the same questions, but sometimes only one gets the answer. This can happen because LLMs are stochastic. Or, it can be due to some odd phrasing of the question. People who meet with success after making an unusual prompt are sometimes proclaimed geniuses of prompting. Others copy their prompting strategies in hopes of similar success. I suspect that in most cases these people merely got lucky in the LLM lottery, and have no more genius than someone who’s convinced that they’ve found the best slot machine in the casino3. When there is any benefit in some special prompt, it seems to be absorbed in the next release of any model.

Adjusting incentives

The leaders of mathematical disciplines need to adjust the incentives we provide to encourage the best behavior.

Our usual desire is to reward those who contribute the most to science.
I believe that the value of one’s contribution to mathematical fields is what you have done minus what what have happened if you hadn’t done it. That is, what did you add to the world? This unfortunately involves a counter-factual that we cannot measure. And it is probabilistic: we care about the chance it would have happened without you.

The easiest type of contribution to observe is leadership. We value people who set the agenda for their fields by convincing others to share their values of what is important to study. This is a social phenomenon, and less susceptible to hacking by AI. But, it’s not what most people do.

Most mathematical researchers solve problems. Some of these will be known problems, and some will be of their own devising. Solving known problems is the easiest path for young researchers to establish their reputations. If you solve problems that you’ve devised, then you have to convince others that both the problem and solution are of value. That’s harder to do4.

In the past, it was believed that the first person to publish a result should get credit for it. At least, we are likely to believe that person’s work was independent of anything that came later5. It created a huge desire and incentive to publish first.

We need to change that incentive.

It’s clear that results that can be obtained from one shot queries to an LLM shouldn’t confer much credit upon the one who wrote the query. We will also be suspicious of results that seem to result from short interactions with LLMs. It’s likely that many other people could have obtained the same result, and that the individual who did it first wasn’t critical.

I don’t fully understand how to operationalize this idea. We mainly learn about how someone obtained a result by reading the AI disclosure they include in their paper. But some will yield to the incentive to distort AI’s contributions. For better or worse, there are often signs that an AI did a significant amount of the work. The most prominent is poor writing in the style of an AI. While poor writing can also be the result of a rush to publish first, the amount of pressure to do so is proportional to how easily one believes others can obtain the same result6. Someone who writes a paper after just one interaction with an LLM will feel a lot more pressure than someone who found that an LLM couldn’t solve their problem without a lot of extra assistance. If someone writes a paper in an area that they don’t fully understand and never studied until the week before, we’ll doubt that their contribution was critical.

I’d like those who set the incentives, say by hiring, publishing, or awarding honors, to favor work for which we are confident the person involved was critical.

And, someone who merely pushes a button7 shouldn’t expect credit. Some people now ask an AI to solve a problem, and send the undigested response to an expert in the hope that the expert will clean it up and make them a coauthor. That’s inappropriate. The button pusher should expect an acknowledgment, but not to share credit.

AI Companies

When I first heard that an internal model at OpenAI had written solutions to many problems that no one had time to read, I thought that they should post those to a repository where everyone could read them. As you can tell, I now believe that this could do more harm than good.

A better approach would be, for the important problems, to find experts who are interested in the problems and the solutions, and to offer them the opportunity to write good solutions. Those who write the solutions will be the authors, and the companies will receive appreciative acknowledgements8. The unimportant problems won’t matter because anyone will be able to solve them when the next model is released.

What to publish

Here’s a suggestion for how to decide whether it is worthwhile to publish a paper: only do it if you would like to give many lectures on the work. If you aren’t excited about the potential of speaking about a paper, then you should focus on writing the papers that do excite you.

If everyone were to follow this rule, it would increase the average quality of published work and decrease the amount of work editorial boards have to do.

Grants

Grants are a very special sort of incentive. In the US, we give grants for work that people propose to do. These enable us to pay ourselves over the summer, to pay for graduate students and their travel to conferences, to pay publication fees, and now to pay for LLM usage. I feel that grant proposals in mathematical fields are often fictional. We say what we hope to do but do not yet know how to do. A proposal for research that is likely to succeed is almost by definition insufficiently ambitious. People who’ve done good work before are more likely to get grants because review panels believe they are likely to do so again.

It’s difficult to write a grant proposal in our new era. If you can concretely state the problems you are going to solve, a reader will wonder why an LLM hasn’t solved them already. And, if an LLM can’t solve them, they’ll wonder if you can. There has always been a risk that a reviewer of your grant proposal might steal your ideas, even if that’s highly unethical. One reason this didn’t happen often was because it was difficult. But now anyone who hears a vague description of your grant proposal might use an LLM to solve your problems. If today’s version doesn’t solve them, the next release might.

Everyone is going to have to be much more secretive about what they are planning to do. If you don’t absolutely need a grant, it might be better not to write a proposal right now.

In the long term, we are going to have to change how we award grants. We should probably give them for past success under the assumption that it will predict the future.

Advice until we reach equilibrium

Over at least the next year, a huge number of important problems will be solved. The deluge in our brains and inboxes will take a long time to absorb. Eventually we expect to reach a new equilibrium.

Until then, many of us will feel lost. We like to plan for the future by assuming it will look something like the past. That’s why people go to senior colleagues for advice. We are old and have seen a lot, so we can guess what might happen. Until now.

Here’s my best shot at advice.

Should students study math adjacent fields?

First, ask yourself if you like studying this field. Do you love it? If so, it’s probably worth studying it even if it doesn’t become your future career. Very few people who study math adjacent fields become professors or leading researchers. It used to be standard to advise students not to go to graduate school if they could do something else. We might revert to that standard.

There were two reasons we gave different advice over the last two decades. The first was that the field was expanding and there were many more jobs. The second was the belief that mathematical training is good training, and that anyone who was trained this way should be able to do plenty of useful things. I hope that is still true, but I’m not sure.

Advice for young academics

Many young researchers are in shock right now. A lot of them have planned their career around a few problems they thought the could solve, only to find them solved in the last few months by LLMs. If you believe that your contribution to the world is your ability to solve problems, then this is a difficult time.

I suggest using your abilities to solve your own problems. Think of things that would be exciting to do, and see if you can do them. Let LLMs help you.

They way you get ahead in academia is by being known for something. So, dream up an agenda of problems. Avoid competition by doing things other people aren’t think of doing. I don’t claim this will be easy for everyone (or anyone). Unfortunately, I recommend not telling too many people what you are planning to do.

Advice for everyone

Use LLMs and don’t use LLMs. I, and many people I talk with, find that when we start to rely too much on LLMs we stop being able to think as well as we used to. So, spend some time thinking without using LLMs. The block of time will depend on the person. I like to take LLM free days for certain problems. It’s how I have new ideas.

One colleague of mine suggested asking an LLM for ways to approach a problem, and then not pursuing any of those approaches.

If you hope to keep using your brain in the future, you should practice now.

And, plan for the future

Many people are writing amazing essays about what the future could and should look like. Two people who don’t blog sent me essays they shared with a few friends.

I’m joining this effort because I think it’s the best way to figure out what could happen and to try to steer the future in the best possible direction. I suggest following posts at Proofs and Prompts and Terry Tao’s Blog, What’s New.

Also, watch The Terminator and Wall-E, in that order.

Acknowledgements

I’ve discussed LLMs with many colleagues in the last few weeks. For this post, I am particularly indebted to Shang-Hua Teng, Jamie Tucker-Foltz, Roy Lederman, and Nils Rudi.

Rapid growth

This is what makes it to difficult predict what AI systems will become. It’s difficult to imagine the dog when you meet the puppy. And it’s really difficult to imagine that it will continue to grow after it becomes a big dog. But you should assume that the systems we will have six months from now will make today’s systems look like puppies. I expect that AI systems will some day plateau, if simply because we don’t have access to infinite electricity. Although, I don’t see any sign of it happening yet.

This means that any time you begin a sentence “AI can’t” or “AI will never”, you are likely to be wrong. And if you didn’t check in on what AI can do recently you are likely to be out of date. A few weeks ago I said that we should measure time for LLMs in “dog years”. But that doesn’t capture the fact that the dog keeps growing like a puppy.

Math vs. other disciplines

Admittedly almost all my experience with AI is through mathematics where its progress has been stunning. I don’t know if its progress in tasks that aren’t math adjacent has been as impressive, or if we should expect it to be. The difference between abstract topics like math and other disciplines is that in mathematics we can eventually figure out if we are correct. This means that an AI system can figure out for itself if it is right or wrong and can use that knowledge to train itself to do better.

Most real-world problems are not like that. I expect AI to make slower progress on tasks for which AI systems can’t themselves distinguish that works and what doesn’t. The tragedy of medicine is that great ideas often fail to work in real life. Great theories are often wrong, or are missing critical facts. This is why I am less optimistic than most about the ability of AI to cure many diseases. AI will clearly be able to help with the math adjacent parts and that can speed up drug discovery. But medications that should work in theory often don’t work in real life. That’s because we don’t have a complete mathematical description of human biology or human disease.

Does this mean we shouldn’t be afraid of the potential for AI systems develop deadly viruses? NO. It is much easier to harm than to help. Many drug trials fail because they harm the patients. More importantly, if someone wants to develop a deadly virus, they don’t need to try just one. They could develop hundreds. They only need one to work. That’s a depressing thought.

You might think, “no one would be stupid enough to give an AI hands and a chemistry set.” But there are many efforts to do just that. For example, here’s an announcement by the Department of Energy on their effort to create “AI driven autonomous laboratories”.

Plagiarism

Current AI systems are essentially plagiarism machines1. I don’t mean that it’s their purpose. It’s just what they do. Part of the problem is that they train the next system on the results of interactions with the current one. This means that, unless you ask them not to and they keep their promise, everything you write with AI assistance will make its way into the next generation of some AI system. Maybe you don’t think you are writing with AI assistance. I don’t think I am. But I have a spell checker running. That doesn’t need AI but I haven’t checked if it’s using it.

This means that almost everything anyone writes now could show up as an auto-complete suggestion for someone else six months from now.

Current LLMs also produce a lot of text without attribution. If I ask them to prove something, they are likely to give me a proof that uses material from other work without citing it. I don’t actually fault them for this, because it’s an artifact of how they are trained. In the same way, you probably don’t remember who told you certain things that you learned a long time ago. Like, do you remember when someone first told you “that color is teal”, “that’s a Samoyed”, or “LLMs are plagiarism machines”? I can’t give proper attribution to that part of my knowledge. So, I’m actually pretty impressed by the improvements that ChatGPT has made in providing citations, even though it’s still bad at it.

Does it matter who publishes first?

Through much of the history of science, we have tried to give the most credit for a discovery to the person who publishes it first. It makes sense, because unless someone broke into your office or hacked into your computer, they probably couldn’t steal your ideas and scoop you.

But now that so many researchers are working in some way with AI systems that might be training on their data, we have much less sense of who did what. If someone isn’t careful with the permissions they set, or if AI companies fail to respect the permissions we set, then one person’s half-finished work can be suggested to person another by an AI. There’s suspicion that this might have happened with OpenAI’s advance on Navier-Stokes or the proof of existence of non-Sophic groups2.

This is especially true because the AI companies themselves have decided to compute with humans. In their press release, OpenAI said that they’d heard a rumor that someone was close to solving the Navier-Stokes problem, and so they decided to dump millions into trying to scoop them. That’s obnoxious. Upon hearing that a person was about to was about to finish one of the most meaningful challenges they’d worked on, they decided to try to ruin the moment. I’m surprised that they actually admitted it in writing and didn’t realize what they were saying about themselves.

Competing scientists do things like that to each other. If two people have been spending huge amounts of time working on something, and one hears that the other is close to a solution, that will motivate the first to get their work out quickly. I have no complaints with that. The difference here is that OpenAI hadn’t invested great time and effort into the problem before they heard that rumor.

I believe in giving credit for scientific discoveries to the people who make them. But, if an AI writes a paper instead of a human, or no human has checked it, then I don’t see why credit should accrue to the person who put their name on it. It’s hard for me to consider something solved if no person has understood it.

I’ve never been a fan of scientific races. When I hear that a dozen labs are close to discovering something, I’m less impressed by the contribution of the one that discovers it first. If they discovery was going to be made anyway, then it’s not much of an achievement. The problem is that we don’t know “the discovery was going to be made anyway”. It’s a counter-factual that we can’t check.

This is why I usually try to work on things that I’m pretty sure other people aren’t working on.

Other reading

It’s almost impossible to say something about AI that hasn’t been said by someone else already. Here are three of my favorite recent posts related to mathematics:

• A Severe Misalignment of AI in Mathematics.

• https://terrytao.wordpress.com/2026/09/11/on-the-existence-of-non-sofic-groups/#more-18157

• https://terrytao.wordpress.com/2026/09/13/deep-theorems-were-scarce-and-difficult-and-so-became-an-effective-mechanism-to-identify-deep-thought-ai-has-broken-this-system/

Disclaimer

I’m not an expert on AI systems and I don’t understand them nearly as well as those who are building them. But the experts don’t understand them all that well and it will only get worse as they start to build themselves.

A long time ago I was teaching a course at MIT in which the homework was difficult and factored significantly into the student grades. We allowed students to collaborate so long as they acknowledged all their collaborators. That year, I decided to examine the graph of who was collaborating with whom. I was curious what the collaboration network looked like and whether the structure of the network affected student grades.

The first thing I noticed about the network was that almost all of the isolated nodes--students who weren’t collaborating with anyone else--were women or minorities. This was clearly a problem and clearly more important than anything else I’d discover about the network. I didn’t know the precise social dynamics that caused these students to be isolated. But, I did know that they were getting less support from their peers. Their lack of collaborators made it more difficult for them to solve the homework problems. By allowing collaboration, I was disadvantaging the students who were more socially isolated in the class. Because of the social structure of that class, I was disadvantaging women and minorities!

When I moved to Yale and began teaching the Design and Analysis of Algorithms class, I wanted to avoid this problem. So I forbade collaboration on the problem sets. This policy had the advantage that it was fair, if it was followed. I conducted anonymous surveys that asked if students were following the policy and if they thought the other students were as well. For a long time they mostly did1. The class was small at first, around 30 students, so I was able to establish a good relationship with the students, and I and my teaching staff could help out those students who were struggling.

I explained to the students the many advantages of working on their own. It forced them to really understand the material and experience the satisfaction and self-confidence boost that goes with solving hard problems. I pointed out that, if they were collaborating with someone who was just a little bit faster than they were, they’d be deprived of that satisfaction. And, they might even conclude that they weren’t good enough to pursue algorithms further2.

Eight years later, enrollments in computer science courses skyrocketed. I could no longer establish a relationship with the students that would inspire honesty. And the number of students who needed help with the material greatly exceeded what my course staff could provide. We started to catch a lot of students cheating and had to spend a lot of time prosecuting them.

In the hopes that students would learn from each other and in recognition that many students were going to collaborate regardless of the rules, I changed my policy to allow collaboration. But this time, I was determined to try to minimize its discriminatory effects. I assumed that, because the class was so big, the women and minorities would be less isolated. It wasn’t so easy.

It soon became clear that I needed to limit the number of collaborators a student could have. Otherwise, big groups would solve the problems together. Not everyone was contributing. One time a solution found by one student percolated through a large fraction of the class. Analyzing this process carefully could have led to interesting network science. But, I needed a policy that would both allow and limit collaboration.

I tried a policy of letting groups of up to 4 students work together. It turns out this was a little bit like letting the students divide into sports teams without them knowing much about each other. The students who stereotypically looked like they’d be good at computer science had an easier time joining groups. I let the students change groups for each problem set. But, they didn’t take much advantage of this flexibility. I created a matching service to help isolated students find study groups. But, this didn’t help much either.

Then, a group of women in the class met with me to point out a flaw in my policy. They lived in dorms in which most of the suites housed 4 students. So, there were many suites containing 4 male students who were all taking my class and so formed a natural study group. While there were many women in the class3,
there were very few per dorm4, so many women were shut out of these natural study groups. Those who weren’t a part of these room-based study groups had to travel farther to meet their collaborators, and they had a much more difficult time scheduling these meetings. They had to work harder than their peers.
Changing the group size to 3 or 5 students didn’t solve the problem.

And, there were always going to be isolated students. Two Black women in the class explained intersectionality to me. There were students on varsity teams who had demanding schedules that rarely let them meet with others in the class. ROTC students had similar problems. There was no good solution.

The grade boost of collaborating on homework will disappear now that we will stop using homework to evaluate students. But I assume that variations of the problems I observed occur often and that I’m usually unaware of it. I will remember that policies that appear neutral can have discriminatory effects because of the context in which they are applied. The tendency of many to befriend people similar to themselves can create social networks that disadvantage those who are different. And, it’s rarely advantageous to be unique in a room.

If we only report letter grades and believe that “A” means “top 10%” and that it also means “mastered the course material”, then we apparently believe that only 10% of the students can master the course material. This might be true in some courses, but it’s a silly restriction.

I suggest we use letter grades to indicate level of mastery of the material, and numerical scores to convey rankings.

Universities should have some courses in which any student who works hard enough will be able to master the material. For such a course, we might assign many “A” grades. Many introductory courses could be like that1. We can also have courses that require unusual effort and talent just to get by. In these, a student might be ranked in the top 10% and still not get an “A”.

I prefer grading schemes that allow us to convey both ranking and mastery of material. My favorite approach is to have instructors assign only letter grades, but have transcripts indicate the rankings of the students who received that grade. For example, it could say something like “B+, ranks 10-15 out of 50”. This is the solution recommended by Yale’s Committee on Trust in Higher Education2. It is easy to implement: instructors assign grades as usual and the registrar computes the ranks.

Another reasonable variation is to ask the instructor to report both a letter grade indicating level of mastery of material and a ranking in the class. Both would appear on the transcript. My complaint with this approach is that a total ranking can magnify small differences that break ties. It might be that there’s no real difference between students who are consecutive in ranking. I prefer to allow ties by reporting rankings in ranges. One could achieve that by allowing an instructor to report both letter grades and ranges of rank. So, some students could get “A, 1-2 out of 50” while others get “A, 3-5 out of 50”. This has the advantage of allowing an instructor to make finer distinctions between students while also allowing them to group students they think are equivalent. But, this is probably too much work for too little advantage over just assigning grades.

This does leave the question of what should become of grade point averages. To start, we could report two: one based on letter grades and one based on rankings in classes. To average ranges of rankings, we could average the middles of the ranges. If half the students in a class get an “A”, then an “A” will correspond to a score of 75%. In a class where only 10% of the students get an “A”, it would receive a score of 95%. This gives students who want high averages an incentive to take courses in which high grades are rare, and it gives professors incentives to use the whole grading scale when it is appropriate. When computing “Latin honors”, like Summa Cum Laude, we should use averages of course rankings rather than numerical averages of letter grades.

Some additional considerations

• Grade inflation isn’t just a problem in education. Ratings throughout digital commerce have been compressed at the high end. If you give an Uber driver any rating less than 5 stars, Uber wants to know what they did wrong. They don’t treat 3 stars out of 5 as average. In fact, they drop drivers with average ratings below 4.5.3

• If different schools start grading very differently, it will be harder to compare students across schools. Those looking at transcripts will have to weight grades by school and year. This will be more difficult for smaller institutions that see fewer transcripts.

• Many years ago, Brendan Hassett pointed out to me that one university had a lower grading scale for very introductory math courses than for the more advanced courses available to students who had taken advanced math in high school. This discrepancy imposed a grade penalty for students from poorly resourced high schools. Providing rankings by class would even this out.

Below, I’ll describe these problems in more depth and the logistical problems they will create. I’ll also mourn what we’re losing.

I hope to one day be able to write about the bigger question of how advances in AI should change what we teach.

Replacing Homework with Tests

LLMs have gotten very good at programming and doing math. They can probably solve all the homework problems we might want to assign in a class. This makes it unreasonable to use homework to evaluate and grade students in our classes. We still want them to do homework, because it’s the best way for them to learn some of the things we want to teach. But it is simply too easy now for students to ask LLMs for the answers. The natural solution is to tell students that homework is just for their training, to stop grading it, and to grade them through tests or oral presentations. My biggest problem with this is that the skills I usually want students to acquire are difficult to measure by testing. I don’t have as much experience with oral exams, but I do know that it’s logistically difficult to conduct them in a large class. Let’s think about what will happen if we only evaluate students through tests.

Our classes will have many more tests, and many classes will have tests that didn’t before. We will need to set aside more time for these tests. At Yale right now, the final exam period is already very crowded, and students who take many classes with final exams often encounter timing conflicts between them. Current policy at Yale recognizes that three exams in a row is too many. We are going to have to make the exam period longer. Some students who have disabilities are afforded twice as much time to take their exams and tests. Those students will face even more scheduling conflicts. Many professors who’ve thought this through are now giving short quizzes many times per semester. It is even harder to give extra time for these, because the rooms aren’t available before or after classes. We are going to need to build extra time into the schedule for tests.

Grade Inflation

At the same time, universities are starting to reduce grade inflation1 (Harvard) or considering it2 (Yale, 2). Grades have risen steadily over the decades since I’ve been a professor. I’ve contributed to this. Way back in 2013, a large Yale College Faculty meeting discussed the distribution of grades and suggested ways of changing our system. This meeting was the first time that I learned how high the grades were in typical Yale courses, and that the grades I assigned were much lower. Around 20% of the grades I assigned in my core Computer Science course were in the C/D/F range. I didn’t want students to suffer just because they had me as a professor or just because they were majoring in Computer Science. So, I raised my grades.

One advantage of grade inflation for professors, and the system in general, is that it makes small differences disappear. We might want to know about small differences in students. But we’d rather mask out the small differences in the tests they take. When students miss tests, either because of illness or conflicts with other tests, we are supposed to offer them a make-up test. This test can’t be the same as the original, because students might have heard about the questions. So we create fresh make-up tests and pretend that they have the same difficulty as the original. They don’t. They could be easier or more difficult, but they will almost never be the same. I try to adjust my grading to compensate for my estimate of each test’s difficulty, but that’s guesswork. If the exact scores on tests didn’t matter so much, I wouldn’t worry about it. But more detailed grading will create more pressure to be fair and make testing much more difficult.

Accommodation Inflation

The accommodations that irritate professors the most are exceptions that deans can grant to excuse missing tests or assignments. At Yale, these are now called “Dean’s Extensions”. They used to be “Dean’s Excuses.” In theory, these are a good idea. Sometimes a student has an emergency or serious illness that causes them to miss a test or prevents them from turning in a problem set on time. Rather than making the professors try to figure out which student’s claims are legitimate, a dean who lives in the student’s dorm and who knows their history makes the decision. The problem is that the number of such extensions that are granted has increased dramatically. Tenfold is a reasonable guess. This is a problem at many schools. It got much worse during COVID. And that’s OK with me. I’d rather not have a COVID-positive student coughing in my class, or even one with the flu. But it’s made the job of testing difficult. In a class of 20 students, at least one will need a make-up exam. In a class of 100, some will miss the make-up and need make-ups for the make-ups. As these drag on, it becomes difficult to figure out who will create these exams and who will grade them. Sometimes these make-ups happen after a professor has left. Maybe we ask the next person teaching a course to do it, if there is such a person the following semester or year. But the material won’t be identical. Creating a good test takes a lot of work, and it’s a big ask of someone who didn’t even teach the course.

The academic accommodations that have received the most attention are alternate testing arrangements for students with disabilities. Some students need technological support during tests, some need extra time, and some need isolation or an unusual location. Yale has an office and facilities to provide that support. As the number of tests increases, we are going to need a lot more of these facilities. Students who need extra time for tests are going to need a lot more time. We need to start thinking about where to find it.

Increased testing and more granular grading will increase scrutiny of the accommodations for students with disabilities. Fairness dictates that we ensure that accommodations are carefully tailored to the students and the tests. No matter what we do, there will be suspicion of the students whose disabilities are not obvious: pretty much everyone would like extra time to finish tests. We have to start dealing with this now before the issues become a political football. Recent press reports on the increasing fraction of students who receive accommodations because of disabilities suggest it may already be too late3.

What we lose

Grades

I’m not a fan of grades as a motivation for students. I would rather that students learn because they are interested. The school I attended through 8th grade didn’t have grades, and the experience shaped me profoundly. In high school, the pressure to get good grades decreased my intellectual motivation. College was much better, because grade pressure was less intense than in my high school, and because I felt that the grades in the classes I was taking purely for intellectual curiosity didn’t really matter.

But sometimes we need grades to choose which student is right for an opportunity, like who to hire as a research assistant, and when we need to recommend a student try something ambitious, like pursuing a Ph.D. Our current grading system makes it difficult to distinguish between students, and it makes it difficult for them to figure out where they stand relative to their peers. Assigning grades by adding test scores seems like a solution. Test scores make it easy to assign as many grades of each letter as we want. But tests don’t always provide good measures of what we really want our students to learn.

Homework

Working through homework problems is the best way that I know to learn material. Struggling to figure out how to apply new techniques to solve difficult problems forces one to understand the limits and capabilities of those techniques. This struggle helps students grow.

I’ve put a lot of effort into dreaming up homework problems that will help my students learn. I like to assign problems that can only be solved after serious thought and a non-trivial insight. No one should finish my problem sets in an hour. I want students to practice thinking hard and understand the problems so well that they can think about them while walking to class, while taking a shower, while waiting in line, or any other time they might be bored. I want them to feel excited when they solve a problem.

The students who can routinely solve these problems are the ones I encourage to pursue research.

It’s difficult to find problems like this, and it’s especially difficult to find new ones each year. I used to look for them in every paper I read and every talk I attended. My notes are full of pointers to lemmas that might make for good homework problems. It wasn’t unusual for me to spend 10 hours designing a problem set. I will probably be relieved that I won’t have to do that anymore. Now I can just assign old problems as homework, and hope that students work through them. But we’ll be losing a lot.

I should explain that I don’t think all classes should assign problems like this. I think that students need many different types of educational experiences and many different types of classes. They should have classes where they collaborate and classes where they work on their own. They should have classes where they learn a little in depth, and classes where they cover a lot. They should have classes that they take just to learn something they’d never encounter otherwise. I like teaching the classes that prepare them for graduate school.

Tests

It’s perverse to respond to the strength of LLMs at doing homework by evaluating students through tests, because LLMs are even better at the questions we can pose on tests! In the age of LLMs that can program and do math, tests reveal even less of what we want from our students. I doubt that tests will be great predictors of whether students will be able to do great research. But we are heading towards a system in which we only grade students by their performance on tests, and in which small differences in performance on those tests could result in big differences in grades.

Tests can be an indicator of talent. There are brilliant students who do very well on tests. But there are many brilliant students who don’t score the best on tests, and I worry that it will be harder to discover them. If I’m not confident that I can identify the best students in my classes, I’ll be even less sure of my ranking of the rest.

My skills

As a student, I spent a lot of time watching my teachers and thinking about how they taught. When I became a professor, I tried to incorporate the best of what I’d seen, to the extent that it was coherent and fit with my personality. I’ve spent decades becoming the best teacher I can be. Advances in AI are going to force me to change and rethink how to teach. Technological advances often eliminate entire categories of work, so I should not be surprised. But I‘ll mourn the irrelevance of hard-earned skills, and I will feel bad for my students until I figure out how to teach well in this new reality.

Caveats

Technological advances have caused educators to panic before, and I haven’t been sympathetic. But LLMs feel different. When people said that typing is worse than writing by hand, that reading on paper is better than reading on a screen, or that taking notes by hand is better than taking notes on a computer, I’ve been skeptical. And when I’ve read the studies that supported these assertions, I’ve been underwhelmed. Some have the same intellectual validity as a hypothetical experiment in which we randomly assign students to take a test with their left or right hand, observe that the students in the right-hand group perform better, and then conclude that all students should write with their right hands.

I always tell my classes that different students learn differently, and that their job in college is to figure out how they learn best. Neither students nor educators should assume that what works best for most students will work best for all.

I don’t know how we will resolve these problems. I prefer to try to motivate students to learn and to optimize my course for the students who want to learn, as opposed to devoting all my efforts to grading and preventing cheating. Distrust is a poor basis for the relationship I want to establish. If it weren’t for the pressure to decrease grades, I’d probably respond to LLMs by simply not grading. Or, in an ideal world, I’d teach only small classes in which I got to know every student well. But we have too many students for that to be a viable solution.

Stories

My thinking is of course informed by my experiences. Here are a few that are relevant to this blog post, in no particular order.

There was a time when faculty were concerned about all the screens in their classrooms. Many students told us they were typing notes, and I know some were, but faculty worried about what they might be watching and whether they were distracting their fellow students. I won’t write now about the low quality of the studies I read on the topic. Instead, I’ll tell you what I did. I asked a teaching fellow to hang out in the back of the class and tell me what the students were looking at. They told me that most of the students were using their laptops to take notes, to look up things that I mentioned in class, or follow along with the lecture notes I’d provided. I adopted a classroom technology policy that amounted to “be courteous of those around you.”

One of my favorite professors when I was an undergraduate was Richard Beals. He gave very difficult exams, and explained that a score of 50% would earn a B. He thought there was no point in putting questions on exams that he expected us to answer, so every question would be difficult. He didn’t want any student to leave early, so some questions would be very difficult. But we could earn an A without being perfect. I found that very freeing. If perfection is the standard, then the task is too easy. When I started teaching, I gave exams like that. I learned that what I found freeing, other students found traumatizing. Too many students were crying during my exams. One of my best students told me that when they looked at the exam they weren’t sure if they would be able to solve any of the problems. I eventually started giving easier exams.

One year while grading my exams, I noticed two strange answers that were so similar that I was sure one student had copied from another. To check if my suspicion was credible, I looked at a picture I had taken of the exam room. Those students were sitting so far from each other that copying would have been impossible. It is more likely that they studied together, and developed the same strange way of thinking about the material.

Many years ago, the Dean of Yale College discovered a collection of class materials in a bathroom during exams. So, she suggested that we not let students leave class to use the bathroom during tests and exams, or that we have them work on the exams in sections, with bathroom breaks between. I wasn’t enthusiastic about this idea, in part because I posed difficult exam questions and wanted to give students time to think about them. In the first test I administered after the dean’s announcement, a student in class blew their nose, and developed an unfortunately severe nosebleed. I decided that student should be excused to use the bathroom.

Keeping the dean’s concerns in mind, I occasionally followed students who left exams to use the bathroom. I tried to do it covertly, leaving a few minutes after they did. I always felt like an ass. What I heard told me that many of those students really did need to use the bathroom, and that keeping them in the classroom could have been disastrous. I decided that my default should be to trust my students, and that I don’t want to try to enforce draconian testing rules just to prevent cheating. I try to establish a good relationship with my students that will help motivate them, and focusing my pedagogy on catching cheaters instead of teaching interested students would make it hard to establish the relationship I want.

This blog will be adjacent all that. I’ll write about academia, being a professor in a mathematical field, and how AI is changing everything.

If you want to know more about me, I suggest checking out my academic homepage.