Gary Marcus

Rank 13 of 47
|
Score 83
3/8/2025, 8:30:02 PM
X
In reply to:
Daniel Litt
@littmath
·
312d
I've been thinking hard about how one would make a benchmark that is more about what I think of as the fundamental skills for math research. Please let me know if you have any thoughts on this.
Daniel Litt
@littmath
·
312d
My sense is that the skills used to do this are quite different from what the AI is using to solve FrontierMath problems--that is, I don't think the primary skill being used is recall. It is, I think, something more like philosophy, and it's less clear how to train for it.
Daniel Litt
@littmath
·
312d
I'm not necessarily skeptical that an AI tool can do this. If this incident has revealed anything to me it is that I don't necessarily fully understand what skills one needs to do non-routine mathematics work.
Daniel Litt
@littmath
·
312d
What about the non-routine part of math research--coming up with genuinely new ideas or techniques, understanding previously poorly-understood structures, etc.? First, I think it's worth saying that this is (i) the important part of research, and (ii) it happens pretty rarely.
Daniel Litt
@littmath
·
312d
What this suggests to me is that these reasoning models are not too far from being very useful aids in this part of doing math. I expect them to be regularly useful at this kind of thing by the end of the year.
Daniel Litt
@littmath
·
312d
Thinking about this over the past couple of months, I've come to the conclusion that a fair amount of math research actually does have this flavor--one looks up or recalls some known facts and puts them together. This is the 90% of math research that is "routine."

The statement is part of a broader discussion on the nature of skills required for math research and the role of AI in this field. It engages with public issues related to education, technology, and the future of research methodologies, thus constituting public discourse.

  1. Principle 1:
    I will strive to do no harm with my words and actions.
    The statement does no harm and is part of a constructive discussion about math research and AI. [+1]
  2. Principle 3:
    I will use my words and actions to promote understanding, empathy, and compassion.
    The statement promotes understanding by engaging in a thoughtful discussion about the skills needed for math research and the potential role of AI. [+1]
  3. Principle 4:
    I will engage in constructive criticism and dialogue with those in disagreement and will not engage in personal attacks or ad hominem arguments.
    The statement is part of a constructive dialogue, engaging with others in a respectful manner without personal attacks. [+1]