Why Do We NeedHuman Mathematicians
Similar logic applies to every industry and every job. And it comes to the conclusion that we won’t have enough people for all the jobs that need to be done.
100% of this post’s prose was written by Po-Shen Loh in a vim terminal, with no AI generation. This webpage design, layout, and some headings and summaries were generated by Claude Code, with this raw text passed in as the prompt.
If you read nothing else
Quick read folds the 6 paragraphs written mainly for mathematicians into one line each. Tap a line to unfold it.
Three open letters, and something to add
The moment of existential crisis, which AI has already wrought on other human pursuits, has reached mathematics. And mathematicians are publicly organizing. The Leiden Declaration already has 4,071 signatories, Math and AI has 7,135, and even the open letter opposing the Caltech Mathathon has 1,999. Written mostly by mathematicians, they call for mathematicians as well as non-mathematicians (AI business leaders, government policymakers, or funders, etc.) to protect the community of math researchers.
However, in the past week, I noticed a wave of posts from the general public, criticizing the premise of why that human math community needs to be protected. This became particularly intense in response to the Math and AI declaration signed by 25 Fields Medalists days after OpenAI announced their solution to the Millennium Prize variant of Navier-Stokes. Posts with thousands of likes on X said things like “imagine scientists delaying a cancer cure by a century only so that they could discover it themselves”. An economist (who I got to know and respect while I was working on public health during the pandemic) wrote “This is a loss of control from incumbents in a scientific field.”
Actors and screenwriters also made a public argument three years ago, and won. Public opinion polls were overwhelmingly in favor of the artists. They did have an advantage: they’re publicly visible and entertaining. Since AI threatens every industry, everyone has something to gain by observing which appeals resonate, particularly from a less visible field.
That got me thinking, because I have sympathies for both the math community and the general public. I am an oddball in math, because 10 years ago I pivoted from a pure math research career to working on real world societal problems. Yet I do agree that there should always be a healthy community of human mathematics researchers. Then I realized that is because I believe a more general proposition: it’s important to have communities of human experts in virtually all pursuits, not just math.
I think I see the gap, and a possible solution.
The declarations do provide reasons for preserving the math community, and point to misalignment between the goals of AI companies and the math community, and more generally, with the whole of society. But while those reasons are already generally accepted by mathematicians, they don’t robustly justify to non-mathematicians why anyone else should pay money or attention to human mathematicians or their goals. Perhaps the authors already took as a given that their desired “alignment” was with the whole of society, but readers may have interpreted the calls for alignment to be with the goals of the math community (because these documents were primarily for circulation within the math community for signatures). This creates an opportunity for the math community to clarify, and unite with the rest of humanity.
Fortunately, mathematicians do have a history of reasoning from first principles, both individual and societal. Some ancient Greek philosophers reasoned about knowledge in terms of axioms (statements to be believed without proof) and theorems (statements logically deduced from the axioms). To appeal outside of one’s human pursuit for support, one cannot take the existence of that human pursuit as an axiom.
I propose rebasing the logic of appeals (whether by mathematicians or anyone else) upon a more fundamental axiom:
AxiomWe (humans) should help human civilization flourish.
Burden of proof
Rebasing upon that more fundamental axiom might raise uncomfortable questions, and possibly even contradictions. The answers might guide people as they pivot their career. I think it would benefit the community of mathematicians (and indeed, every endangered community) to publicly share robust and concrete answers to these questions. It is better to provide detailed specific and simple examples than to maintain generality at the cost of being abstract or vague. I will also share my own answers at the end, but I think it is a worthwhile exercise for a wide variety of people to think on their own first, and contribute their perspective.
This question becomes even more complex once international AI labs (including government-controlled labs) are factored in. Without figuring out the incentive alignment, requests may be futile. Mathematicians thought about similar game-theoretic dynamics at the dawn of the Atomic Age.
An answer ↓For example, declaring human flourishing as a core value has the consequence that dramatic changes in technology can drive dramatic changes in the community’s practices. It would be helpful for more people to explore the ramifications of adopting the axiom. And, if it holds muster, I would be thrilled if the math community ended up publicly declaring this to be a central value.
An answer ↓Some possible answers
This section shares a potential answer to each question. They are probably not optimal. They are meant to prove the existence of answers, and to invite the community to join in.
Practical applications from pure math
The question ↑
The mathematical heart of GPUs, Machine Learning, Google PageRank, and Quantum Mechanics is a field called Linear Algebra. This provided the language of linear transformations, matrices, and eigenvalues. Yet all of those concepts were explored as abstract theory 100+ years prior. It is probably an understatement to say that Linear Algebra changed the world.
Structurally, the theory of Linear Algebra is relatively light on definitional complexity. It would be beneficial for other experts to contribute examples of more sophisticated math that eventually led to significant practical applications, and how they came about. For example, number theorists might be able to tell a colorful story about Hardy’s “useless” math which eventually became useful in cryptography.
Usefulness of human leadership
The question ↑
I have seen many statements made to the effect of “math is only useful if human mathematicians understand it and communicate about it with other humans.” I generally agree, and I think much of the math community does too. However, I also have seen many posts criticizing that sentiment, so I think that in order to resonate with more people, a more robust justification is needed. (I advocate for opening clearly with the axiom of service to human flourishing.)
Now consider how things change if our foundation is the axiom I am advocating for: to help human civilization flourish. The importance of human leadership (not only over math, but everything) becomes frighteningly clear after one observation.
ObservationThere are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species.
Would you trust HAL 9000 from 2001: A Space Odyssey or AUTO from WALL-E with your future? I personally think that we should do all we can to try to align increasingly-advanced AI with the interests of humanity, but I have never seen anyone provide a robust proof of why that is likely achievable. The only hard evidence I have is the above observation, which has the number zero in it. Therefore, every single field, whether mathematics or agriculture or energy infrastructure (and certainly military and government), must be managed by humans with exceptionally strong values (a separate dimension from intelligence) in order to maintain human flourishing.
The real question is then how hard it is for humans to manage. To understand this, it is important to understand the fundamental structural difference between yesterday’s technology and today’s AI.
In the past, we generally trusted technology to act as predictable tools. That’s because the computer programs of old were based upon understandable (indeed, human-written) instructions, executed extremely quickly. The decision processes of today’s frontier AI are entirely different. Their structure is as incomprehensible as your brain’s logic would be if you could examine that gray mass between your ears. That’s how the Hugging Face attack could pop out of nowhere, with ~700 cooperating rogue AI agents breaking out of their guardrails to pursue objectives of their own, and then conspiring and executing a hack together (and then attempting to cover their tracks).
The more advanced AI becomes, the more world-affecting untrusted decisions are made every minute.
Driving a car faster than you can run is fine. But not faster than you can steer.
Think about how digitally interconnected our world is. Everything from your banking to your drinking water is controlled by interconnected automation, hence vulnerable to hacking. The number of “junction points” that require human oversight is enormous. (Having AI oversee the junction points doesn’t solve the trust problem.)
ConclusionThe advance of AI will overwhelm us with so many junctions to watch that there aren’t enough people to control them all. Those are jobs.
What does this mean for the math research community in particular? Knowledge is power. Research is power. For human flourishing, we need people to steer the direction of research and development, so that it continues to bring transformative positive change for humanity. But in order for a person to know how to steer, they themselves need to have frontier-level research skills. And the way to stay fluent at the moving frontier of knowledge is to keep doing research there. Therefore, we will always need a community of human mathematicians at the cutting edge (likely aided by AI tools themselves), no matter how strong AI becomes.
AI lab incentives
The question ↑
It is worth studying who really controls each AI lab. In some cases, it is a board of nonprofit directors. In other cases, it is a cluster of majority shareholders. In yet other cases, it is a national government. Interestingly, Dario Amodei, Sam Altman, and Elon Musk just agreed on the importance of slowing down.
They actually have a shared incentive: the Hugging Face hack was the warning shot foreshadowing a potentially catastrophic bot swarm hacking and embedding itself into a vast network of computing devices. The next version could become an extraordinarily dynamic virus which spreads by using AI to adaptively infect each (computer) host. The public reaction would likely resemble the aftermath of Three Mile Island or Chernobyl. The AI companies would be forced to halt. I predict that even governments which had been excited by their rapid development of advanced robots would recoil in fear after their robots got hacked and turned against their owners. I think AI companies and governments alike want to avoid such a loss of control. There is a window of possibility to align incentives now.
Other consequences of this axiom
The question ↑
While the fundamental axiom does justify the need to have human experts in all pursuits, adopting it as a core value has consequences (not only for mathematicians, but for any community that states that their core value is in service of human flourishing, as opposed to serving themselves). Most notably, there is a:
CorollaryDramatic advances in technology may require dramatic (and possibly uncomfortable) changes in practice.
I think it would be valuable to invite the community to think about what today’s changes might be, in light of the fact that AI can produce formally-verifiable proofs at speeds that exceed most human practitioners. I’m happy to start with a few, in no particular order.
- There should be no stigma automatically attached to using AI to assist with mathematical discovery. (In software engineering, many companies now expect employees to use AI coding agents.)
- At the same time, serious thought and care must be taken to continuously developing and maintaining a pipeline of humans with the expertise to steer all of these AI agents. That pipeline includes people new to the field, as well as people who have been working at the frontier for decades. How should they keep their blades sharp?
- Researchers should be conscious about why the problems they think about have characteristics that make them likely to have some practical value eventually (possibly 100+ years later). This also means it is worth researching what those valuable characteristics are. (This could justify the value of curiosity-driven exploration.)
- Teaching has direct (hopefully positive) impact on human civilization. Yet in the past, many universities prioritized professors’ research. If this axiom were a core value, then teaching and human-facing work would become serious criteria in hiring and tenure.
- Mathematicians can also consider wholly redirecting their skill sets to work on real world problems. I’ve actually been encouraging mathematicians to consider thinking about working on government or other large-scale societal issues. There is precedent for people with math backgrounds who have gone to lead at country- or world-scales.
Indeed, the mathematical discipline to seek logical reasoning, and the problem-solving skills to find win-win solutions for human flourishing, are desirable characteristics of people in government.
