This article by Terence Tao is a very, but profound read in this new era where Anthropic and OpenAI are becoming increasingly active in the field of mathematics. #1501237, #1544977, etc
A proof that no human can properly explain should be viewed as incomplete, even if it has been formally verified.
As a theoretical/computational physicist, this made me try to come up with an equivalent statement applicable to my field. Because, make no mistake, in the last year, my workflows and those of my colleagues have changed drastically... making us more productive in terms of research output. But I feel like taking a step back and thinking about what we are doing more deeply is valuable, as publishing more and faster is a race to the bottom as the peer review system is breaking down as I write.
So, this is what i get when doing so: replace proof by computational result: a computational result that no physicist can physically explain should be viewed as incomplete, even if the calculation is numerically correct.
As AI makes producing calculations increasingly cheap, physical understanding, not merely producing correct numerical results, becomes an increasingly important part of the scientific contribution.
In some areas, particularly in education and in the training of young mathematicians, it will be crucial to emphasize the irreducibly human aspect of our work, and to restrict the use of AI tools quite tightly; the goal of training a mathematician is not achieved by producing correct homework.
I can feel some of my students (and even myself) are skipping the "learning" part exactly because they are just geared toward producing more results to publish more. AI helps to get the code working, obtain the result, make the figure, or formulate the argument, but allows us to avoid developing the same depth of understanding that would previously have been required to get there.
Anyhow, I feel more comfortable thinking about all this now that I am on track for tenure... I don't think my students have the mental freedom to even think about this: a master's student I was talking to last week was telling me that increasingly, just to apply for a PhD in her country, one is expected to have several published papers already. The rotten publish-or-perish culture has thus already taken root even before one even has learned the critical skills a PhD is supposed to help you acquire. As I said some other times, AI is accelerating the demise of this rotten system, but I still don't know what it will be replaced with. I just hope my tenure gets confirmed before it reaches a point of no return, so that I can witness all this from the safe side of the fence. Selfishly.
Disclose tool use. Transparently disclose the use of automated tools, including large language models, machine learning systems, proof assistants, and other mathematical software[1]
In the spirit of this recommendation: ChatGPT was used in generating this post arguing that we should be careful about using AI to do our thinking for us... (no, for real, just polishing, I promise~~) ↩
I really hope we can get to a point where depth of understanding is valued more than raw measurable output. But how we get to that culture in a scalable way, I have no idea
Maybe getting there will paradoxically be inevitable now, in the age of AI. Where that point lies, I don't know either. The amount of raw measurable output AI is producing is breaking the system.
A dirty side effect will be that the current stage where we only trust results from well-known and respectable groups because we value their previous work in terms of quality rather than quantity will become even more clear. Smaller groups that focus on raw measurable output only, out of necessity, will become increasingly obsolete, and will have no choice but to increase their quality (unlikely) or disappear in irrelevance (most likely). We may end up with an even clearer devide between a few big groups and the rest that no one cares about. Science will become even more elitist.
I've already noticed I increasingly refuse to referee papers for low-impact journals. I look at the names of the authors, I look at the journal, and based on that, I decide if it's worth my time. I have to consciously break this pattern to support unknown authors who may also produce quality results. It's hard.
I feel like I'm on the receiving end of some of that. It makes me question whether it's worth staying in academia when your institution doesn't carry any prestige in its name.
I'm kinda at the interface. I got to play and publish with researchers way above my pay grade or intellectual ability, but also publish with smaller researchers who better align with my own skills. I need to play my cards right to keep the right collaborations going, so I don't end up on that receiving end.
Taking advantage of your presence in this thread. Two unrelated questions.
How are econ academics currently affected by AI? Is AI able to produce high-level econ science, threatening the livelihood of human economists? Is it similar to the field of math, or does it depend on the kind of econ one does?
You've mentioned before thinking of taking the jump to mathematics from econ. Do you feel like AI is giving you a better or worse shot at this hypothetical?
I don't feel like AI is a threat to high level, experienced economists. The biggest weakness I see from AI in the realm of economic sciences is the inability to make judgment calls and to act on imperfect evidence.
AI is pretty good at coming up with ideas for looking for evidence, and pretty good at pointing out flaws in evidence, but I find that it isn't good at knowing when to stop nitpicking and when to move forward with what you have. It's also not that good at weighing between different evidences.
I do think the demand for junior level work from humans is gonna decline. Doesn't necessarily mean a contraction of the labor force. It could just be that juniors are given even more responsibility than before.
If I ever said that, I probably didn't mean jumping to math professionally. But maybe I meant spending more time doing math as a hobby?
I am thinking of just giving up on publishing. Not a total retreat, but to stop making it the prime objective. The idea would be: produce research the way I want, keep it open source, publish it online, and eventually if it makes it to a journal---great---but if not, then whatever. I'm fortunate that I got tenure just before AI really started becoming widespread.
This kinda aligns with the observation that exact science is more at risk than human sciences.
Math and coding are the first victims because they use exact language which only allows for a very few options in terms of next probable word/token.
Physics is next in line, as experimental uncertainty blurs the language we use.
Econ, as it mixes behavorial science with math, gives more options in terms of next words, so it's not too much of a threat yet.
Does that make sense?
The tenure part is really the defining factor in how we act as scientists.
Related, a Korean Professor's union recently argued to abolish the tenure-track system (https://www.hibrain.net/braincafe/cafes/48/posts/407/articles/559797 if you can read Korean). Maybe that's the solution?
The peer-review, conference, and journal system has been deteriorating for years: overloaded reviewers, declining standards, citation cartels, predatory outlets, etc. AI will accelerate this by making the production of superficially plausible papers dramatically cheaper and faster. Hopefully we replace it with something better.