On proof and progress in mathematics in the age of generative LLMs

TLDR:  We should redefine mathematician to mean somebody who strives to understand human understanding of mathematics as envisioned by Thurston and later elaborated on by Tao. Our work and how we obtain credit for it should change accordingly. We should make communicate this new notion of mathematician to the public and to the government, and in general do a better job of explaining what we do to each other as well as to the public.

I have recently had occasion to revisit Thurston's famous essay "On proof and progress in mathematics", and upon reflection I believe it contains lessons which are still relevant for the community more than 30 years later, in the age of generative LLMs. 

Let me make one thing very clear from the outset. I am opposed to the use of generative LLMs for two reasons: 

1. the unconscionable cost to the environment, e.g. in terms of water and electricity consumption, the displacement of people and wildlife for the construction of data centres, the noise pollution it creates for those who live near a data centre etc.
2. the highly unethical behaviour of the companies which produce these LLMs, both in their day to day actions and in their long term goals. 

Nonetheless, I invite the reader to entertain the following fantasy. Suppose we live in a world where we neither of the above are an issue, but the state of the mathematical community is the way it currently is. Are any of the other arguments against the use of LLMs convincing? How should we respond to the emergence of this new technology?

Before discussing this, let me first remind the reader of some of the viewpoints expressed in Thurston's essay.

What is it that mathematicians accomplish?

Regarding this question, Thurston writes 

"Rather, as a more explicit (and leading) form of the question, I prefer

How do mathematicians advance human understanding of mathematics?

This question brings to the fore something that is fundamental and pervasive: that what we are doing is finding ways for people to understand and think about mathematics.  

The rapid advance of computers has helped dramatize this point, because computers and people are very different. For instance, when Appel and Haken completed a proof of the 4-color map theorem using a massive automatic computation, it evoked much controversy. I interpret the controversy as having little to do with doubt people had as to the veracity of the theorem or the correctness of the proof. Rather, it reflected a continuing desire for human understanding of a proof, in addition to knowledge that the theorem is true.

On a more everyday level, it is common for people first starting to grapple with computers to make large-scale computations of things they might have done on a smaller scale by hand. They might print out a table of the first 10,000 primes, only to find that their printout isn’t something they really wanted after all. They discover by this kind of experience that what they really want is usually not some
collection of “answers”—what they want is understanding."

I wholeheartedly agree with Thurston's viewpoint that the real function of a mathematician is to produce human understanding of mathematics. I suspect that most readers and contributors of this blog would agree with Thurston too. As any mathematician worth their salt knows, understanding is not the formal argument which one reads in a paper. Understanding is the idea which underlies the proof. Thurston discusses this in his essay, but the most relevant part for us is this next excerpt:

"We mathematicians need to put far greater effort into communicating mathematical ideas. To accomplish this, we need to pay much more attention to communicating not just our definitions, theorems, and proofs, but also our ways of thinking. We need to appreciate the value of different ways of thinking about the
same mathematical structure.

We need to focus far more energy on understanding and explaining the basic mental infrastructure of mathematics—with consequently less energy on the most recent results. This entails developing mathematical language that is effective for the radical purpose of conveying ideas to people who don’t already know them.

Part of this communication is the writing of proofs."

Thurston then goes on to discuss what proofs are, and his view on the correctness and formality of existing proofs of theorems. Note, however, that the function of a mathematician is not the production of formally correct proofs.

"In contrast to humans, computers are good at performing formal processes. There are people working hard on the project of actually formalizing parts of mathematics by computer, with actual formally correct formal deductions. I think this is a very big but very worthwhile project, and I am confident that we will learn a lot from it. The process will help simplify and clarify mathematics. In not too many years, I expect that we will have interactive computer programs that can help people compile significant chunks of formally complete and correct mathematics (based on a few perhaps shaky but at least explicit assumptions), and that they will become part of the standard mathematician’s working environment.

However, we should recognize that the humanly understandable and humanly checkable proofs that we actually do are what is most important to us, and that they are quite different from formal proofs."

What motivates people to do mathematics?

Again, I wholeheartedly agree with Thurston that one major reason to do mathematics is being able to understand and appreciate the beauty of mathematical concepts, ideas, and objects.

"There is a real joy in doing mathematics, in learning ways of thinking that explain and organize and simplify. One can feel this joy discovering new mathematics, rediscovering old mathematics, learning a way of thinking from a person or text, or finding a new way to explain or to view an old mathematical structure." 

In addition to the intrinsic joy of doing mathematics, Thurston also points out that we, as social creatures to some extent or another, generally prefer to do mathematics in a community. However, Thurston also criticises the way that the community is organised.

"I think that our strong communal emphasis on theorem-credits has a negative effect on mathematical progress. If what we are accomplishing is advancing human understanding of mathematics, then we would be much better off recognizing and valuing a far broader range of activity. The people who see the way to proving theorems are doing it in the context of a mathematical community; they are not doing it on their own. They depend on understanding of mathematics that they glean from other mathematicians."

Later on he also says

"What we are producing is human understanding. We have many different ways to understand and many
different processes that contribute to our understanding. We will be more satisfied, more productive and happier if we recognize and focus on this."

Why is human understanding important? 

This point is not addressed in Thurston's essay, nor is it really covered in any detail anywhere else. Mathematics is an art upon which science depends, and up until very recently the dependence relied on some human being(s) understanding what is going on. Perhaps that may change. It is conceivable that, for all practical purposes, some LLM would be able to supply the necessary ingredients from mathematics in any scientific setting. I argue that this would be a very dangerous state of affairs, since if any sort of debugging or modification were needed, we would be either unable to perform it or entirely at the mercy of the people who control the LLMs. I believe this point alone warrants a much more extensive discussion.

I also believe that this is a secondary point. I have alluded to the beauty of mathematics and described it as an art. As far as I can tell, every civilisation aspires to reach the point where it can create art freely, and that is often the benchmark of prosperity. The reason for this is that art, whatever form it takes, brings humans joy and pleasure through appreciation of the art. That the general pursuit of joy and pleasure (when it is not at the expense of others) is good is generally taken to be axiomatic. Thus, many people feel that the arts are of intrinsic value. They are not a means to an end- they are the end itself. Similarly, the accumulation of knowledge, whether or not it is immediately of use, is an end in and of itself. Mathematics, and the understand of it, falls under both categories. 

What threats do generative LLMs pose to the community?

There has been much talk of how disruptive LLMs will be. I will list the main concerns as I understand them. 

1. The proliferation of generative LLMs will lead to more papers being produced than can be verified or understood by the community.
2. The ease with which one can simply prompt an LLM and receive an answer will take away from the experience of problem solving.
3. The widespread use of LLMs will inhibit our ability to train future generations of mathematicians, e.g. because the problems which PhD students traditionally use to cut their teeth will all have been solved, or because the skills which one learns when struggling with problems will be lost because all students will simply prompt the struggle away.
4. Generative LLMs will only worsen the inequity present in mathematics. See this discussion on MO. 

 What can be done about this?

I believe that point 4 is, sadly, a feature of the way modern society is constructed. Technology often serves to benefit those in power and further oppress the already weak and helpless. An example which is far worse than LLMs: many large Western tech companies are complicit in or actively aiding and abetting the genocide in Gaza. To change the way new technology is used and the way it impacts society would require huge upheavals in the power structures and dynamics that are currently present. That merits an entire field of study and is certainly beyond the scope of a blog post.

I believe that among mathematicians, Terence Tao is one of the most enthusiastic and vocal proponents of the integration of LLMs into research, and having read his thoughts on LLMs here, I believe that he makes many eminently sensible suggestions whose adoption would mitigate or deal with the other issues raised.

Point 3 is easiest. The upshot is just don't outsource your thinking to an LLM, and don't let your students do so either. There are some menial and mundane tasks which an LLM can do better, and those could be outsourced, but the parts of being a mathematician which require real thought should be done by a mathematician. Tao says this better than I could so I leave it to the reader to read his thoughts as well.

Point 1 is sort of addressed by the presence of Lean and other aspects of formal mathematics. As for verification, automated formalisers are getting better and better, e.g. having almost autonomously formalised the proofs of sphere packing bounds due to Viazovska and collaborators. This is also discussed here on MO, but I think the point of the discussion on MO is that one should not stop trying to understand a proof just because there is a Lean proof. Again, as Thurston already said, the point is in the human understanding of the proof, not just the formal correctness. 

As for understanding and disseminating the proof, I will address this and point 2 in the same vein. Let us suppose that we reach the point where LLMs can reliably produce solutions to a large number of open problems in a number of fields, or perhaps even to the "Library of Babel" scenario that Daniel Litt describes. Tao envisions a mathematical community where a mathematician's work is primarily the digestion and exposition of new proofs, setting results in the broader context of other theorems, perhaps guiding the directions in which mathematics develops, and having "humanities-style discourse" about which theorems are the best/most beautiful etc.

I think this would be a good paradigm shift.

I think a lot of the community's unease/horror at the ability of LLMs to solve open problems that have stumped the field is a symptom of diseases that have plagued the community. Some of these include:

1. An overly competitive environment which heaps rewards upon the person to first publish a solution to an open problem. This stems from the competitive nature of a lot of mathematicians, and is one way of providing the motivation to spend lots of time and effort to solve difficult and challenging problems. However, as Thurston points out, it is unclear if this is the best way to solve a specific problem and assign credit for its solution, since solutions typically rely on lots of important work and ideas due to other people who do not received due credit for its solution. Moreover, it is almost certainly not optimal in the long run for building human understanding of mathematics.
   
   Relatedly, the system of journals is complete nonsense. There is a huge amount of arbitrariness that goes into deciding which papers get published in which journals, as well as in perceptions of how good individual journals are, yet somehow publication records form an integral part of hiring and ranking mathematicians. Even the concept of ranking mathematicians based on their output seems stupid to me. It is not clear that there should exist a total order at all.
   
   The question of how will one get credit for proving theorems in the age of LLMs is largely motivated by an obsession with proving that one is a "better" mathematician than others, and one which should not exist.
   
   
2. An unwillingness to revisit "proved" theorems, and not squeezing the most out of existing techniques. Erdos believed that God has a book with all the best and most beautiful proofs of all mathematical theorems, and a mathematician's job is to look over God's shoulder and read it. The proofs of Sendov's conjecture and the cycle double cover conjecture are both remarkably elementary and could probably have been found by a strong undergraduate or master's student if they were given it as an exercise. The point is that researchers didn't know what to try or what would work, but I think this drives home the point that one can prove a huge amount with very elementary methods (also look at the diversity of maths olympiad problems!). 
   
   I also believe that fewer than 10% of existing theorems have the Book proof attached to them, and we as a community should spend more time searching for better proofs of theorems. This is not just aesthetics. It also simplifies the literature and makes it easier for newcomers to learn the field. The project of formalisation has already done this: Buzzard and Taylor have simplified many aspects of Wiles' proof of Fermat's Last Theorem as part of an effort to make the proof formalisable, just as Thurston predicted. However, better proofs of existing theorems are sometimes/often not awarded the credit that they deserve, e.g. the papers get published in (much) less prestigious journals. I suppose this is part of the previous point, where being "first" is what counts. 
   
   
3. An overall lack of communication of ideas and exposition of important results in the field to make it accessible to newcomers and/or researchers from other fields. Some of the most beautiful mathematics and important breakthroughs come from fusing together ideas and techniques from multiple fields of mathematics. This has occurred repeatedly in the history of human generated mathematics, and is again illustrated by the breakthrough in the unit distance problem. So why is it so difficult to learn new mathematics? Even within the field of mathematics where I do the most research, I find the motivation for many if not most problems obscure and/or unconvincing. How can we expect researchers in other fields or the general public to buy into our research if we are so bad at explaining why we care? An example is the Kourovka notebook. I am sure that, for every question, at some point somebody knew why someone might care about. This is not always recorded in the notebook. Sometimes, not even a reference to where one might read about the motivation for a given question. 
   
   This blog exists in no small part due to my frustration at being unable to find ways to learn mathematics. Good expositions are few and far between: the most reliable source is Bourbaki exposes, but beyond that, I do not know a reliable to look. For certain topics, there are "roadmaps to learning topic X" on MO. This should exist for every topic in every area. (It is also not so easy for a newcomer to ask questions on MO/MSE: the community guidelines are strict for good reason, but a newcomer probably does not know what these are or how to interpret them correctly and often falls foul of the moderators' wrath. I speak from experience, even as I acknowledge that in my case it was my fault.)
   
   In his essay, Thurston spoke of the nuanced and specialised languages that subfields of mathematics use to communicate ideas as one of the barriers to entry. It is high time for us to write more dictionaries.

 I would also like to point out that, even without the existence of generative LLMs, the number of papers had been steadily increasing, and I had not heard any serious or convincing proposals for how the mathematical community might cope with that. I believe that the diseases I mentioned above contributed to that issue as well.

I think that in this fictional universe where the use of LLMs had no costs to the environment and LLM companies were as benevolent as they are malevolent in reality, it would be best for mathematics to embrace the technology, and I think the model that Tao proposes is a good suggestion, although of course that would be open to debate, refinement, and counterproposals. In this fictional universe, we would finally move towards the greater emphasis on understanding and exposition for which Thurston already advocated 30 years ago.

I do not believe that the presence of an oracle, or even the Library of Babel, would be detrimental to mathematics at all. Many of us worship at the temple where Thurston was once high priest, and the number of faithful continues to increase not in spite of, but rather because of, Thurston's incredible ability to advance his field of research. Having learned from his experience of killing the field of foliations, when it came to the geometry of 3-manifolds, Thurston dedicated lots of time to explaining his vision and equipping people with the new language, rather than simply proving as many theorems as he could. I am told that at conferences people lined up to consult Thurston about their specific problem, and often managed to turn what he said into a good paper, but Thurston never claimed credit for those ideas. As a result of his choice of direction, the field flourished, and the proofs of the geometrisation conjecture and virtual Haken conjecture (that Thurston proposed) are among the great success stories of research programmes in modern mathematics. 

I believe that in a world where LLMs really were as powerful an oracle, we would still be able to meaningfully do mathematics, albeit in a capacity much more similar to Thurston the expert in 3-manifolds as opposed to Thurston the expert in foliations. I do not even think that this would take away from the joy of discovery. Even if an LLM were able to answer any question we could ask it, we would still have the right and responsibility of directing it. One of my favourite stories is that of Monstrous Moonshine, that apparently originates from the observation that 196884=196883+1. Even in this fictional universe, the pleasure of making such observations and wondering about what they mean falls squarely on the shoulders of human beings. 

In the same vein, LLM output might not be particularly enlightening, just as computer assisted proofs often are not (see e.g. the 4 colour theorem mentioned above. My pet peeve is the counterexample to the Kaplansky Unit Conjecture. It is completely meaningless and to date unexplained). There is still much mileage to be gained from trying to understand and contextualise the output; see e.g. Tao's explanations of the resolutions of Sendov's conjecture or the Jacobian conjecture. Some people may not find this a very inspiring career choice, and that's fine. No matter what we define a mathematician's work to be, there will be good people who don't like it and end up leaving maths. That is a shame, but it is unavoidable. What I want to say is that I don't think that in this universe one is reduced to verifying LLM generated proofs.

As for the joys of solving problems, I point you to mathematics Olympiads, where the community enjoys solving nice problems even though they know somebody else has already solved it. Perhaps part of professional mathematics could even become a sport in this way. The only thing that is gone is the rush of knowing that you are the first human being to solve the problem and the sense of superiority that seems to come with it. I do not see how mathematics is worse off for losing people who thrive off of that. 

Back to reality: What should we actually do?

As every mathematician worth their salt knows, toy models can be very informative but one must still draw the right conclusions and figure out how they apply to the actual problem at hand.

Tao has this to say: we have only months to "reimagine what it means to be a mathematician... We don't have to passively accept changes by external forces. [We] can't just passively prove our theorems; we have to organise, become activists, get a little political." Tech companies "...tried to redefine what our profession is. [But] mathematicians have more influence than they think."

I wholeheartedly agree. What it means to be a mathematician will change drastically whether we like it or not, so we better make it change in a direction that we like. What should this mean concretely? 

In some sense the current situation is a prisoner's dilemma with many actors. All of us have incentives to use LLMs to produce more theorems, but the world (and our field) is worse off if everybody does. Rather than battle with this age old game theoretic issue, I think we should change the situation. It is incumbent on us as mathematicians to change the way the public, the government, and funding bodies view our work, so that mathematics continues to be done. It is also time to change the way mathematics is done, as Thurston told us to do so. Not all readers are in a position to directly make changes, but we can all lobby our institutions and national mathematical societies to start discussions and reforms.

Finally and most importantly, we mathematicians are human beings, for better or for worse, and we share responsibility for the struggles and flaws of humanity. We are obliged to fight climate change, and in particular to fight the LLM companies which are hellbent on destroying our way of life. There are countless ways we can fight climate change, and we can fight LLM companies through our voices as citizens of our respective countries as well as with whatever influence that our status as mathematicians can buy us. It is time to get political and use it, but the exact how is left as an exercise to the reader.