A year and a half ago, I met a puppy named ChatGPT. It was cute and did some tricks, but it wasn’t very intimidating. Over the span of a year, it grew into a 150 pound beast. Another six months have brought us to a Clifford the Big Red Dog (https://www.scholastic.com/clifford) situation.
Rapid growth
This is what makes it to difficult predict what AI systems will become. It’s difficult to imagine the dog when you meet the puppy. And it’s really difficult to imagine that it will continue to grow after it becomes a big dog. But you should assume that the systems we will have six months from now will make today’s systems look like puppies. I expect that AI systems will some day plateau, if simply because we don’t have access to infinite electricity. Although, I don’t see any sign of it happening yet.
This means that any time you begin a sentence “AI can’t” or “AI will never”, you are likely to be wrong. And if you didn’t check in on what AI can do recently you are likely to be out of date. A few weeks ago I said that we should measure time for LLMs in “dog years”. But that doesn’t capture the fact that the dog keeps growing like a puppy.
Math vs. other disciplines
Admittedly almost all my experience with AI is through mathematics where its progress has been stunning. I don’t know if its progress in tasks that aren’t math adjacent has been as impressive, or if we should expect it to be. The difference between abstract topics like math and other disciplines is that in mathematics we can eventually figure out if we are correct. This means that an AI system can figure out for itself if it is right or wrong and can use that knowledge to train itself to do better.
Most real-world problems are not like that. I expect AI to make slower progress on tasks for which AI systems can’t themselves distinguish that works and what doesn’t. The tragedy of medicine is that great ideas often fail to work in real life. Great theories are often wrong, or are missing critical facts. This is why I am less optimistic than most about the ability of AI to cure many diseases. AI will clearly be able to help with the math adjacent parts and that can speed up drug discovery. But medications that should work in theory often don’t work in real life. That’s because we don’t have a complete mathematical description of human biology or human disease.
Does this mean we shouldn’t be afraid of the potential for AI systems develop deadly viruses? NO. It is much easier to harm than to help. Many drug trials fail because they harm the patients. More importantly, if someone wants to develop a deadly virus, they don’t need to try just one. They could develop hundreds. They only need one to work. That’s a depressing thought.
You might think, “no one would be stupid enough to give an AI hands and a chemistry set.” But there are many efforts to do just that. For example, here’s an announcement by the Department of Energy on their effort to create “AI driven autonomous laboratories”.
Plagiarism
Current AI systems are essentially plagiarism machines1. I don’t mean that it’s their purpose. It’s just what they do. Part of the problem is that they train the next system on the results of interactions with the current one. This means that, unless you ask them not to and they keep their promise, everything you write with AI assistance will make its way into the next generation of some AI system. Maybe you don’t think you are writing with AI assistance. I don’t think I am. But I have a spell checker running. That doesn’t need AI but I haven’t checked if it’s using it.
This means that almost everything anyone writes now could show up as an auto-complete suggestion for someone else six months from now.
Current LLMs also produce a lot of text without attribution. If I ask them to prove something, they are likely to give me a proof that uses material from other work without citing it. I don’t actually fault them for this, because it’s an artifact of how they are trained. In the same way, you probably don’t remember who told you certain things that you learned a long time ago. Like, do you remember when someone first told you “that color is teal”, “that’s a Samoyed”, or “LLMs are plagiarism machines”? I can’t give proper attribution to that part of my knowledge. So, I’m actually pretty impressed by the improvements that ChatGPT has made in providing citations, even though it’s still bad at it.
Does it matter who publishes first?
Through much of the history of science, we have tried to give the most credit for a discovery to the person who publishes it first. It makes sense, because unless someone broke into your office or hacked into your computer, they probably couldn’t steal your ideas and scoop you.
But now that so many researchers are working in some way with AI systems that might be training on their data, we have much less sense of who did what. If someone isn’t careful with the permissions they set, or if AI companies fail to respect the permissions we set, then one person’s half-finished work can be suggested to person another by an AI. There’s suspicion that this might have happened with OpenAI’s advance on Navier-Stokes or the proof of existence of non-Sophic groups2.
This is especially true because the AI companies themselves have decided to compute with humans. In their press release, OpenAI said that they’d heard a rumor that someone was close to solving the Navier-Stokes problem, and so they decided to dump millions into trying to scoop them. That’s obnoxious. Upon hearing that a person was about to was about to finish one of the most meaningful challenges they’d worked on, they decided to try to ruin the moment. I’m surprised that they actually admitted it in writing and didn’t realize what they were saying about themselves.
Competing scientists do things like that to each other. If two people have been spending huge amounts of time working on something, and one hears that the other is close to a solution, that will motivate the first to get their work out quickly. I have no complaints with that. The difference here is that OpenAI hadn’t invested great time and effort into the problem before they heard that rumor.
I believe in giving credit for scientific discoveries to the people who make them. But, if an AI writes a paper instead of a human, or no human has checked it, then I don’t see why credit should accrue to the person who put their name on it. It’s hard for me to consider something solved if no person has understood it.
I’ve never been a fan of scientific races. When I hear that a dozen labs are close to discovering something, I’m less impressed by the contribution of the one that discovers it first. If they discovery was going to be made anyway, then it’s not much of an achievement. The problem is that we don’t know “the discovery was going to be made anyway”. It’s a counter-factual that we can’t check.
This is why I usually try to work on things that I’m pretty sure other people aren’t working on.
Other reading
It’s almost impossible to say something about AI that hasn’t been said by someone else already. Here are three of my favorite recent posts related to mathematics:
Disclaimer
I’m not an expert on AI systems and I don’t understand them nearly as well as those who are building them. But the experts don’t understand them all that well and it will only get worse as they start to build themselves.
I wrote this sentence on Sunday night. On Monday I heard Elchanan Mossel say “plagiarism machine”, and realized that I probably got the phrase from his essay https://arxiv.org/pdf/2601.02380. But while the phrase stuck in my head, I forgot its origin
Thanks for your thoughts, Dan.
Super insightful and interesting reading. I do think AI is upending many longstanding norms about credit, achievement, and ultimately, merit. This will have ripple effects as we think about the many, many social and economic reward systems that are fundamentally rooted in these concepts.