Phylactery OUT NOW · FREE · PLAYS IN YOUR BROWSERPLAY ›
index / posts / 2026-09-11-211135Sep 11, 2026  21:11

Mathematicians against theory-free AI math

When I say that in such fast times everyone is naked, no one knows how to act, it's easy to see who's actually thinking, and so on, this recent post and the wildly negative response to it by AI twitter are a great example:

A severe misalignment of AI in mathematics

The mathematicians are making an obvious and clear argument, perhaps you need to understand some of how math works to fully grasp it, I was a math major for a year so maybe what seems clear to me is not to others, although I doubt that's the difference, but let me still make an analogy to the field I know best.

The reason why the one-shotted vibecoded games generate an instinctual aversion is because anyone who has made games knows that, in the process of making a game and playing it, you often discover that ideas you had for it were bad, and that the game can have a life of its own, many devs mention that one or another mechanic that later became central happened as an accident, for instance. This often happens because lots about a game comes from the development process itself, and while one-shotting things gets you a result, it might miss better results that a proper exploration of the design space would have offered. The job of a game designer is to explore that design space, which the tools make easier, but which they shouldn't substitute because games are made for other humans, "fun," or however you want to codify it, comes from humans mapping a previously non-existent piece of reality into something digestible to other humans, eventually robots might be able to do this too, but they can't do it now.

And they can't do it now because they seem to lack the ability to ask the next good questions. There's a recent example of this with me in the Reverse Minesweeper Prototype Astra/Fable logs, where I keep asking both of them, what do you want to do next, as I wanna know what useful suggestion limit they have, and both of them reached it as soon as the main task was done. They simply don't know, given an open-ended situation, how to move ahead productively unless prompted, because this requires them to have objectives that they can't self-generate, or maybe they can, but they can't rank these generations in a productive way, or maybe they just don't have enough compute to do it productively. I don't know, but whatever proactivity is missing, they don't have it.

The analogy to mathematics is direct. Without proper understanding you lose the ability to ask useful next questions, therefore one can see how AI companies strip mining the field for PR is positively damaging, as they're depleting the field of one of its resources and not producing insight in return, insight that might lead to better theories, concepts, or just direct applications to real world problems, as both Mr. Scientism and Wolf Tivy point out here:

scientism@mr_scientism
Applications of mathematics primarily come from the concepts created in solving problems within math. The mathematicians are defending the part of math that produces applications from the AI companies who cynically using math for PR. x.com/alz_zyd_/status/2098167576144617875
alz@alz_zyd_
the mathematicians are telling us actually, math is not about solving math problems. it's about art and training people and Fields medals and stuff like that. so how about we start a new field that's focused purely on solving math problems? we can call it applied mathematics
6:22 PM · Sep 11, 2026
scientism@mr_scientism
In research mathematics, it’s primarily how you do it that (potentially) produces social value. The AI companies are solving research math problems without producing that social value. There’s no good reason for them to be doing what they’re doing other than as PR stunts.
6:48 PM · Sep 11, 2026
Wolf Tivy@wolftivy
In practice, no one cares if there are singularities in the NS. But it’s possible studying the problem, incentivized by the glory of solving it, could have turned up useful concepts. The worry with AI math is “theory-free results” that take the glory but produce no insight.
scientism@mr_scientism
Applications of mathematics primarily come from the concepts created in solving problems within math. The mathematicians are defending the part of math that produces applications from the AI companies who cynically using math for PR. x.com/alz_zyd_/status/2098167576144617875
6:54 PM · Sep 11, 2026
Wolf Tivy@wolftivy
An interesting smoothness proof might have led to better theory of solvers. But seems we’ll have to keep using empirical hacks. Knowing which way it goes is not nothing, but it’s not the unknown unknown new ideas that come out of math and restructure our thought.
6:57 PM · Sep 11, 2026
Wolf Tivy@wolftivy
It would be amazing if AI could generate new concepts like the NS equations in the first place. But that appears to be a much more “qualitative” thing than just knowing theorem truth/falsity.
6:59 PM · Sep 11, 2026
Wolf Tivy@wolftivy
An example useful math concept is the “one way function” used in cryptography. The idea structures the entire field in a way that can’t be reduced to particular proofs and theorems. In fact, it’s not even known if such functions really exist!
7:02 PM · Sep 11, 2026

In any case, why do I love such fast moving times? Because of this. Because you get to see who is real, who is thinking, who is careful, who is insightful, and who isn't. Everyone is naked, everyone is revealing themselves all the time, and if you're paying attention it becomes a lot easier to parse social reality, it's wonderful.

Settings